Skip to content

optimize generated parser code via ZST, static slices, and direct vec! initialization#47

Merged
ehwan merged 3 commits into
mainfrom
parser_static
Jun 18, 2026
Merged

optimize generated parser code via ZST, static slices, and direct vec! initialization#47
ehwan merged 3 commits into
mainfrom
parser_static

Conversation

@ehwan

@ehwan ehwan commented Jun 18, 2026

Copy link
Copy Markdown
Owner

Summary

This PR optimizes the code generated by the parser generator (rusty_lr_parser::emit::Grammar) to reduce runtime allocation overhead, remove atomic lock checks from the hot path during parsing, and dramatically improve compilation times for generated files.

Key Changes

  1. ZST Parser Struct & OnceLock Constructor Initialization
    • The generated parser struct (e.g. GrammarParser) is now a Zero-Sized Type (ZST) holding 'static references to rules and states slices (rules: &'static [GrammarRule], states: &'static [GrammarState]).
    • The OnceLock lazy initialization has been moved into the parser's new() constructor. Static tables are built and cached only once upon the first parser instantiation.
    • Parser::get_rules() and Parser::get_states() now return fields directly without checking the OnceLock atomic state, providing lock-free and atomic-free hot-path lookups.
    • Implemented Clone, Copy, and explicit unsafe impl Send and unsafe impl Sync for the parser struct to guarantee thread safety regardless of the user's custom token type thread safety constraints.
  2. Simplified Vector Generation for ruleset and reduce_map
    • Replaced complex iterator chains (zip().map().collect()) for states' ruleset with direct vec![ShiftedRuleRef { ... }] initialization.
    • Replaced temporary static terminal sets, local rules, and runtime BTreeMap insertion and conversion (into_iter().collect()) for states' reduce_map with clean, direct vec![(TermClass, Vec<RuleIndex>)] initialization.
    • Completely removed unused static terminal set initializers (__RUSTYLR_TSETx), significantly cleaning up the generated code and reducing compilation burden on rustc.

Benefits

  • Zero-Allocation Parser Instantiation: Creating a parser instance is now completely free.
  • Lock-Free Hot Path: Eliminated OnceLock atomic load operations on every token feed/reduction step.
  • Improved Compilation Speed: Simplified generated source code enables rustc to type check and compile generated files much faster with less memory.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request refactors the parser generation to make the parser struct lightweight by referencing static parser tables instead of holding owned vectors. The parser tables are now lazily initialized once using std::sync::OnceLock and returned as 'static slices. Two critical compilation issues were identified in the regenerated files (calculator.rs and calculator_u8.rs) where OnceLock::get_or_init returns &'static Vec<T> but the struct fields expect &'static [T]. Since Rust does not perform implicit deref coercion in struct initializers, calling .as_slice() explicitly is required to resolve the type mismatch.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment thread scripts/diff/calculator.rs Outdated
];
states.into_iter().map(|state| state.into()).collect()
});
Self { rules, states }

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

critical

Rust does not perform implicit deref coercion in struct initializers. Since rules and states are of type &'static Vec<T> (returned by OnceLock::get_or_init), but the struct fields expect &'static [T], this will result in a compilation error. Calling .as_slice() explicitly resolves this type mismatch.

        Self {
            rules: rules.as_slice(),
            states: states.as_slice(),
        }

Comment thread scripts/diff/calculator_u8.rs Outdated
];
states.into_iter().map(|state| state.into()).collect()
});
Self { rules, states }

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

critical

Rust does not perform implicit deref coercion in struct initializers. Since rules and states are of type &'static Vec<T> (returned by OnceLock::get_or_init), but the struct fields expect &'static [T], this will result in a compilation error. Calling .as_slice() explicitly resolves this type mismatch.

        Self {
            rules: rules.as_slice(),
            states: states.as_slice(),
        }

@ehwan
ehwan merged commit b8993cd into main Jun 18, 2026
1 check passed
@ehwan
ehwan deleted the parser_static branch June 18, 2026 00:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant