Fix handling of templates in preProcessFile

[lenkan]

Previously, the preProcessFile function would erroneously identify import statements inside template literals if they occurred after a template literal that contained a placeholder.

`${1}`
`import module from "m1"`

Would report m1 as being an imported file.

Inversely, it would fail to find valid imports in the similar situation:

`${1}`
import module from "m1"

This is because the code is failing to recognize the `}`` as the end of the first template literal.

This PR fixes that by keeping track of if we are inside a template, if so, we run the "rescanTemplateToken" when we encounter a close brace.

Fixes #30878

[typescript-bot]

Thanks for the PR! It looks like you've changed 'preProcess.ts' in some way. Please ensure that any changes here don't break consumers with unique project systems such as Visual Studio. Pinging @sheetalkamat, @amcasey, and @minestarks so they are aware of the changes.

[amcasey]

CI failure looks unrelated. Closing and re-opening to retrigger.

[weswigham]

This technically needs to track more than one scanning context. For example,

`${(<div>This is JSX text so these: `` ${} mean nothing, and in fact, the {} are
(empty) interpolation areas! So ${import("mod")} and {import("mod")} are dynamic imports! Also,
you could have ${/*A comment*/} or ${/*import("ignored")*/} a commented module </div>)}`;

The textmate grammar chokes on that one, too, since it involves multiple nested scanning contexts, but it is parsed correctly.

[lenkan]

I am a bit unfamiliar with how this works, but I will add these test cases and see if I can figure it out. Thanks!

[sandersn]

Seems like a superset of #33688, or maybe that PR should be folded into this PR.

[weswigham]

@lenkan Do you need any help? Do you know if you'll get back to this?

[lenkan]

@weswigham - I constructed some test cases based on the examples you gave in #36143 (comment). But I couldn't patch the current solution to support those. I have struggled to find the time to dig deeper into it.

If you can give me any pointers towards what I need to consider in the implementation that would be appreciated. I feel that the solution direction I took is not very robust, but I lack enough knowledge of this project to come up with something different at this stage.

Meanwhile, I've rebased this branch and resolved the conflicts in the test file.

[weswigham]

Rather than a single templateNesting variable counting how many templates nest, I'd probably try keeping a stack of nestable scanning contexts (statement vs template vs jsxtext vs comment) that you can push/pop contexts from to control scanner behavior.

[weswigham]

Requesting the changes mentioned above~

[sandersn]

@lenkan Do you want to keep this open? Let us know if we can help with more, detailed explanation.

[lenkan]

@sandersn I will close it because I won't have time to figure out how to change this behaviour without breaking something else in preProcessFile.ts. I have been working around this issue by not using string templates.