fix: add missing space between comma-separated digit groups#342
Merged
Conversation
#291) Numbers like 0,013,225 were normalized to "oh zero one threetwo two five" with "three" and "two" concatenated. The comma-separated digit groups in single_digits_graph_with_commas lacked a space separator between groups.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #291 (the threetwo bug)
Summary
Fix missing space between digit groups in
single_digits_graph_with_commas. Numbers like0,013,225were normalized tooh zero one threetwo two five— the last digit of one group and the first digit of the next were concatenated.Before:
0,013,225→oh zero one threetwo two fiveAfter:
0,013,225→oh zero one three two two fiveRegarding the speed question in the issue: English TN is slower than Chinese TN (~17ms vs ~3ms) because the English FST has significantly more states (larger rule set, non-deterministic mode). The recent refactoring (#333) already improved build time by 4x. Runtime compose+shortestpath cost is proportional to FST size and hard to optimize further without simplifying the grammar.
Test plan