Analysing a grammar

Checking for issues

ts-bnf-tool check runs all static checks on a grammar file and exits with a non-zero status if any issue is found. This makes it easy to wire into a CI pipeline:

ts-bnf-tool check json.bnf
echo $?   # 0 if clean, 1 if warnings only, 2 if any errors

Checks performed:

Check	Severity	Example diagnostic
Undefined rule references	error	error: undefined rule reference 'foo'
Undefined %axiom rule	error	error: %axiom references undefined rule 'foo' (line 1)
Duplicate %axiom	error	error: %axiom declared more than once (line 2)
Undefined %conflicts rules	error	error: %conflicts references undefined rule 'foo'
Undefined %inline rules	error	error: %inline references undefined rule 'foo'
Undefined %supertypes rules	error	error: %supertypes references undefined rule 'foo'
Undefined %extras rules	error	error: %extras references undefined rule 'foo'
Unreferenced rule	warning	warning: rule 'foo' is never referenced (line 4)
Non-productive rule	error	error: rule 'foo' can never derive a terminal string (line 4)
Unused %precedences level	warning	warning: %precedences level 'unary' is declared but never used by a %prec annotation (line 1)
Unused %reserved set	warning	warning: %reserved set 'typeNames' is declared but never used by a %reserved annotation (line 2)

A duplicate rule, %axiom, or %word produces a second diagnostic alongside the one shown above, pointing at the earlier declaration: warning: previous %axiom declaration is here (line 1). This is what lets you find both sides of the conflict — especially useful when the two declarations come from different files via %include.

Pass --json to get diagnostics as a JSON object on stdout instead of plain text on stderr. Exit codes are not affected:

ts-bnf-tool check --json json.bnf

{"diagnostics":[{"severity":"warning","message":"rule 'unused' is never referenced","line":3}]}

file/line/column are structured location fields, present whenever the diagnostic has a concrete source position to point at (omitted otherwise, as for the undefined rule reference 'foo' example above). The plain-text form still renders the same location as a (file:line)/(line N) suffix on the message.

Syntax errors

If the file does not parse at all, check reports each syntax error with its file, line, column and a snippet of the offending source, then exits 2:

root => 'a' ;
value -> 'b'

error: syntax error near '=> 'a' ;' (broken.bnf:1:6)
error: syntax error: missing ';' (broken.bnf:2:13)

At most 10 syntax errors are listed; any excess is summarised in a final … and N more syntax errors line. With --json, syntax errors appear as regular entries in the "diagnostics" array. Every other subcommand (convert, format, graph, …) aborts with the same located messages on stderr and exits 1.

Left-recursion

Left-recursive rules are not flagged by check. Tree-sitter is a GLR parser generator: left recursion is fully supported and is the idiomatic style for binary and postfix expression rules.

# OK — directly left-recursive, idiomatic for binary operators
expr -> expr '+' term | term ;

Left recursion is still a grammar property worth knowing about — for instance, a left-recursive rule may need a %prec annotation or a %conflicts entry to resolve ambiguity. The check --summary block reports how many rules are directly or mutually left-recursive (see Summarising grammar shape below).

What actually makes tree-sitter generate fail is unresolved ambiguity — for example expr -> expr '+' expr | 'n' with no precedence annotation. See Shift-reduce conflicts and operator precedence for how to resolve these with %prec annotations. Ahead-of-time detection of such conflicts is planned separately (#31).

Non-productive rules

Left recursion is fine, but a rule that can never reach a terminal — no alternative in its body ever bottoms out at a literal, a pattern, or an %externals token — is a different, genuine error: tree-sitter generate rejects it outright with an opaque Unresolved conflict for symbol sequence error. check catches this ahead of time instead:

a -> a ;

error: rule 'a' can never derive a terminal string (line 1)

The same applies to a mutual cycle with no terminal escape (a -> b ; b -> a ; flags both a and b). This is unrelated to reachability from the root — unreferenced rules above — and unrelated to left recursion itself: expr -> expr '+' term | term ; is left-recursive and productive (via term), so it is not flagged.

Unreferenced rules

A rule that is defined but not reachable from the root — directly or transitively through other rules’ bodies — is reported as a warning. The root is either the rule named by %axiom, or — when %axiom is absent — the first-declared rule:

root   -> item+ ;
item   -> /[a-z]+/ ;
unused -> 'x' ;   # never referenced

warning: rule 'unused' is never referenced (line 3)

Reachability is transitive, so a rule that only references itself, or a group of rules that only reference each other, still counts as unreachable if none of them connect back to the root:

root -> item+ ;
item -> /[a-z]+/ ;
a    -> a ;        # references only itself
b    -> c ;        # b and c reference each other, but neither
c    -> b ;         # connects back to the root

warning: rule 'a' is never referenced (line 3)
warning: rule 'b' is never referenced (line 4)
warning: rule 'c' is never referenced (line 5)

An island where one unreachable rule simply calls another is different: only the rule that actually starts the island is reported, not everything it pulls in with it. This matters in practice — if you write a new rule and forget to reference it from anywhere reachable, but it calls other rules you already wrote, you get a single pointed warning at the rule you forgot to wire in, instead of being flooded with one warning per rule underneath it:

root -> item+ ;
item -> /[a-z]+/ ;
b    -> c ;    # never referenced — the island's entry point
c    -> 'x' ;  # referenced by b, so not reported separately

warning: rule 'b' is never referenced (line 3)

An island with no entry point at all — like the mutual cycle above, where b and c only reference each other — has no single rule to blame, so every rule in it is still reported.

Unused %precedences levels and %reserved sets

check also flags the reverse direction from the reference checks above: a %precedences string-literal level or %reserved set that’s declared but never actually used anywhere else in the grammar. Both are typically leftovers from refactoring, or a typo on the use side that happens to match nothing — tree-sitter accepts them silently, so nothing else catches it either.

%precedences ['unary']
%reserved kw: [a], typeNames: [a]

a -> 'x' ;

warning: %precedences level 'unary' is declared but never used by a %prec annotation (line 1)
warning: %reserved set 'typeNames' is declared but never used by a %reserved annotation (line 2)

A %precedences group’s rule-name items are never flagged this way — unlike a string level, a rule name orders rules relative to each other regardless of whether any %prec annotation references it, so it’s meaningful on its own (#243). Likewise, the first declared %reserved set is exempt: it’s the implicit global reserved-word set, meaningful without any rule-level %reserved annotation naming it.

Summarising grammar shape

check --summary appends a compact metrics block to stdout after the run. Diagnostics still go to stderr, so the two streams can be captured independently in shell pipelines.

ts-bnf-tool check --summary json.bnf

Rules            6  (leaf: 2, unreachable: 0)
Terminals       12  (literals: 10, patterns: 2, unique values)
Undefined refs   0
Left-recursive   0  (direct: 0, mutual: 0)
FIRST sets      min 1  max 7  avg 2

Each row measures a different aspect of the grammar:

Row	What it tells you
Rules	Total named productions. leaf = rules whose body contains no rule references (only terminals). unreachable = rules never reached from the root, which check also flags as warnings.
Terminals	Unique terminal values across all rule bodies, split into string literals and regex patterns. See the note on uniqueness below.
Undefined refs	Rule names used in bodies but never defined — check flags these as errors too.
Left-recursive	Rules involved in left-recursion, split into direct (a → a …) and mutual (a → b …, b → a …). Informational only — left recursion is idiomatic tree-sitter style, not a defect.
FIRST sets	Size statistics (min / max / average) of the FIRST set of each rule — the set of terminals that can open a derivation. A large max or high average suggests the grammar may have ambiguous alternatives.

Terminal uniqueness is measured by raw source text, not by what the lexer matches. 'x' and "x" are counted as two distinct literals even though they match the same character. The count reflects how many distinct token patterns the grammar author wrote, which is a useful proxy for lexer complexity.

Using --summary with --json

Combining --json and --summary adds a "summary" key to the JSON output alongside "diagnostics", making both machine-readable in a single pass:

ts-bnf-tool check --json --summary json.bnf | jq .summary.rules

The full "summary" object shape:

{
  "rules": 6,
  "leaf_rules": 2,
  "unreachable_rules": 0,
  "unique_literals": 8,
  "unique_patterns": 6,
  "undefined_refs": 0,
  "left_recursive_direct": 0,
  "left_recursive_mutual": 0,
  "first_sets": { "min": 1, "max": 7, "avg": 3.3 }
}

first_sets is null when the grammar has no productions.

check options

  --json      Emit output as a JSON object instead of plain text
  --summary   Append a grammar metrics block after diagnostics

Inspecting FIRST sets

ts-bnf-tool firsts prints the FIRST set of each rule — the set of terminals that can appear as the very first token of any string the rule can derive. This is useful for understanding LL(1) feasibility: if two alternatives in a choice(…) share a terminal, a single token of look-ahead cannot tell them apart.

ts-bnf-tool firsts json.bnf

array: '['
number: /\-?[0-9]+(\.[0-9]+)?([eE][+-]?[0-9]+)?/
object: '{'
pair: '"'
string: '"'
value: '"', '[', 'false', 'null', 'true', '{', /\-?[0-9]+(\.[0-9]+)?([eE][+-]?[0-9]+)?/

Pass --json to get a JSON object instead, suitable for editor plugins or other tooling that consumes structured output:

ts-bnf-tool firsts --json json.bnf

{
  "array":  ["'['"],
  "number": ["/\\-?[0-9]+(\\.[0-9]+)?([eE][+-]?[0-9]+)?/"],
  "object": ["'{'"],
  "pair":   ["'\"'"],
  "string": ["'\"'"],
  "value":  ["'\"'", "'['", "'false'", "'null'", "'true'", "'{'", "/\\-?[0-9]+(\\.[0-9]+)?([eE][+-]?[0-9]+)?/"]
}

firsts options

  -n, --no-check   Skip static checks and suppress all warnings
  --json           Emit output as JSON instead of plain text

---

Previous: Worked example · Next: Formatting and refactoring