Support '_' type placeholder and type inference for NonTerminals

[ehwan]

Summary

Introduces support for the _ type placeholder when defining NonTerminal rule types. If _ is specified, the parser generator (rusty_lr_parser) automatically infers the concrete type by tracing Identity reduce actions. If the parser detects a circular dependency or is unable to resolve the type, it returns a clear compilation error (ParseError::TypeInferenceFailed).

Detailed Changes

1. rusty_lr_parser Type Inference Engine

• Unique Placeholder Assignment: Nonterminals declared with type _ are temporarily assigned a unique identifier token (__rustylr_placeholder_<Name>) during early setup. This prevents conflicts and allows robust type mapping.
• Type Resolution Loop: Iterates through all rules of placeholder nonterminals and attempts to resolve their type using ReduceAction::Identity(idx). It propagates types recursively, supporting nested structures (e.g. Vec<T>).
• Circular Dependency Detection: If the resolution loop makes no progress but unresolved placeholders remain (e.g. A(_): B { $1 }; B(_): A { $1 };), it raises ParseError::TypeInferenceFailed pointing to the first failed nonterminal.
• Substitution: Substitutes all successfully resolved types into every nonterminal type (including auto-generated helper nonterminals) before proceeding to parser table generation.

2. Error Reporting

• Added ParseError::TypeInferenceFailed(Location) variant to error.rs to report inference failures and circular dependency loops with their origin span.

3. Documentation

• SYNTAX.md: Documented _ type inference usage, rules, and circular dependency behavior.
• README.md: Added a brief feature overview section.

4. Examples Updated

• Modified existing calculator examples to demonstrate the feature and verify it works end-to-end:
  • calculator/src/parser.rs: Changed nonterminals A, M, and E to use _.
  • calculator_u8/src/parser.rs: Changed Digit, E, and Op to use _.

[gemini-code-assist]

Code Review

This pull request introduces automatic type inference for non-terminals using the _ placeholder in RustyLR, updating documentation, examples, and error handling accordingly. Feedback on the implementation highlights a bug in substitute_placeholders that incorrectly discards empty groups, order-dependency issues in the type inference logic when handling multiple identity rules, and the corresponding updates needed at the caller site.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

[gemini-code-assist]

The current implementation of substitute_placeholders has a critical bug where any empty group (such as (), [], or {}) in a type signature is completely omitted from the output. This happens because substitute_placeholders returns None when the substituted stream is empty, and the caller completely skips extending the token stream if the result is None.

We should change substitute_placeholders to return TokenStream directly. If a group's stream is empty, it will correctly preserve the empty group (e.g., keeping () as ()).

fn substitute_placeholders(
    ts: TokenStream,
    resolved: &std::collections::HashMap<String, Option<TokenStream>>,
) -> TokenStream {
    let mut new_ts = TokenStream::new();
    for token in ts {
        match token {
            proc_macro2::TokenTree::Ident(ident) => {
                let s = ident.to_string();
                if s.starts_with("__rustylr_placeholder_") {
                    if let Some(replacement_opt) = resolved.get(&s) {
                        if let Some(replacement) = replacement_opt {
                            new_ts.extend(replacement.clone());
                        }
                    } else {
                        new_ts.extend([proc_macro2::TokenTree::Ident(ident)]);
                    }
                } else {
                    new_ts.extend([proc_macro2::TokenTree::Ident(ident)]);
                }
            }
            proc_macro2::TokenTree::Group(group) => {
                let sub = substitute_placeholders(group.stream(), resolved);
                let new_group = proc_macro2::Group::new(group.delimiter(), sub);
                new_ts.extend([proc_macro2::TokenTree::Group(new_group)]);
            }
            other => {
                new_ts.extend([other]);
            }
        }
    }
    new_ts
}

[gemini-code-assist]

The current type inference logic is order-dependent and can lead to incorrect type resolution or silent mismatches. If a non-terminal has multiple identity rules, the loop immediately breaks on the first rule that resolves (even if it resolves to None or a conflicting type), ignoring subsequent rules.

We should collect all inferred types from all identity rules, prefer concrete types over None, and ensure that all inferred concrete types are consistent (reporting a conflict error if they differ).

                        let mut inferred_types = Vec::new();

                        for rule in &grammar.nonterminals[i].rules {
                            if let Some(ReduceAction::Identity(idx)) = &rule.reduce_action {
                                if *idx < rule.tokens.len() {
                                    let token = rule.tokens[*idx].token;
                                    match token {
                                        Token::Term(_) => {
                                            inferred_types.push(Some(grammar.token_typename.clone()));
                                        }
                                        Token::NonTerm(to_nonterm_idx) => {
                                            let target_type = &grammar.nonterminals[to_nonterm_idx].ruletype;
                                            let substituted = match target_type {
                                                Some(ts) => {
                                                    let sub = substitute_placeholders(ts.clone(), &resolved);
                                                    if sub.is_empty() {
                                                        None
                                                    } else {
                                                        Some(sub)
                                                    }
                                                }
                                                None => None,
                                            };
                                            if get_placeholder_name(&substituted).is_none() {
                                                inferred_types.push(substituted);
                                            }
                                        }
                                    }
                                }
                            }
                        }

                        if !inferred_types.is_empty() {
                            let concrete_types: Vec<TokenStream> = inferred_types.iter().filter_map(|t| t.clone()).collect();
                            let ty = if !concrete_types.is_empty() {
                                let first_ty = &concrete_types[0];
                                let first_ty_str = first_ty.to_string();
                                for ty in &concrete_types[1..] {
                                    if ty.to_string() != first_ty_str {
                                        return Err(ParseError::TypeInferenceFailed(grammar.nonterminals[i].name.location()));
                                    }
                                }
                                Some(first_ty.clone())
                            } else {
                                None
                            };
                            resolved_this_round.push((p_name.clone(), ty));
                        }

[gemini-code-assist]

Update the placeholder substitution caller site to handle the new TokenStream return type from substitute_placeholders.

                nonterm.ruletype = match &nonterm.ruletype {
                    Some(ts) => {
                        let sub = substitute_placeholders(ts.clone(), &resolved);
                        if sub.is_empty() {
                            None
                        } else {
                            Some(sub)
                        }
                    }
                    None => None,
                };