Skip to content

add IntermediateState for lightweight state generation#22

Merged
ehwan merged 5 commits into
mainfrom
builder_state
Aug 9, 2025
Merged

add IntermediateState for lightweight state generation#22
ehwan merged 5 commits into
mainfrom
builder_state

Conversation

@ehwan

@ehwan ehwan commented Aug 9, 2025

Copy link
Copy Markdown
Owner

this could reduce binary size by 10%

we were using builder::State for state generation which was using BTreeMap.
add new IntermediateState struct for common type convertion into any parser state type, and it is using Vec.

@ehwan
ehwan requested a review from Copilot August 9, 2025 16:07

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR introduces a new IntermediateState struct to serve as a lightweight intermediate representation for state generation in the LR parser. The change refactors the code generation and state conversion pipeline to use Vec instead of BTreeMap/BTreeSet collections for better performance during code generation.

Key changes:

  • Introduces IntermediateState struct with Vec-based fields for shift/goto maps and reduce rules
  • Replaces BTreeMap/BTreeSet usage with Vec in code generation
  • Updates conversion logic to handle sorted vector extraction instead of map key removal

Reviewed Changes

Copilot reviewed 3 out of 4 changed files in this pull request and generated 4 comments.

File Description
rusty_lr_parser/src/emit.rs Updates code generation to use IntermediateState and Vec collections instead of BTreeMap/BTreeSet
rusty_lr_core/src/parser/state.rs Adds IntermediateState struct and updates conversion logic to handle vector-based state representations
rusty_lr_core/src/builder/state.rs Adds conversion from builder State to IntermediateState
Comments suppressed due to low confidence (1)

Comment thread rusty_lr_core/src/parser/state.rs Outdated
/// this intermediate_state is common structure to convert from generated code, grammar builder
/// into various types of parser states ( SparseState, DenseState, ... )
pub struct IntermediateState<Term, NonTerm, StateIndex, RuleIndex> {
pub shift_goto_map_term: Vec<(Term, StateIndex)>, // must be sorted

Copilot AI Aug 9, 2025

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The sorting requirement is critical for correctness but only mentioned in comments. Consider adding runtime assertions or using a type that enforces this invariant.

Copilot uses AI. Check for mistakes.
Comment on lines +333 to +336
debug_assert!(
TerminalSymbol::Term(0) < TerminalSymbol::Error
&& TerminalSymbol::<i32>::Error < TerminalSymbol::Eof
);

Copilot AI Aug 9, 2025

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The hardcoded TerminalSymbol::Term(0) in the assertion makes assumptions about the type parameter. Consider using a more generic approach or documenting this constraint.

Suggested change
debug_assert!(
TerminalSymbol::Term(0) < TerminalSymbol::Error
&& TerminalSymbol::<i32>::Error < TerminalSymbol::Eof
);
// Removed hardcoded assertion about TerminalSymbol ordering to avoid assumptions about type parameter.

Copilot uses AI. Check for mistakes.
Comment on lines +459 to +475
debug_assert!(
TerminalSymbol::Term(0) < TerminalSymbol::Error
&& TerminalSymbol::<i32>::Error < TerminalSymbol::Eof
);
let keys = builder_state
.shift_goto_map_term
.iter()
.map(|(term, _)| term)
.collect::<Vec<_>>();
debug_assert!(keys.is_sorted());

let keys = builder_state
.reduce_map
.iter()
.map(|(term, _)| term)
.collect::<Vec<_>>();
debug_assert!(keys.is_sorted());

Copilot AI Aug 9, 2025

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Duplicate assertion logic exists in both SparseState and DenseState conversion methods. Consider extracting this into a shared validation function.

Copilot uses AI. Check for mistakes.
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
@ehwan
ehwan merged commit 65cf533 into main Aug 9, 2025
1 check passed
@ehwan
ehwan deleted the builder_state branch August 9, 2025 16:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants