[Experiment] Cache Dts Emit result / transform

[sheetalkamat]

Fixes #43995

[sheetalkamat]

@typescript-bot perf test this

[typescript-bot]

Heya @sheetalkamat, I've started to run the perf test suite on this PR at 5bba57e. You can monitor the build here.

Update: The results are in!

[sheetalkamat]

Hmm Actually perf test is not useful since we dont do d.ts signatures any more as part of first run but only on changes.

[sheetalkamat]

@typescript-bot pack this

[typescript-bot]

Heya @sheetalkamat, I've started to run the tarball bundle task on this PR at 5bba57e. You can monitor the build here.

[typescript-bot]

Hey @sheetalkamat, I've packed this into an installable tgz. You can install it for testing by referencing it in your package.json like so:

{
    "devDependencies": {
        "typescript": "https://typescript.visualstudio.com/cf7ac146-d525-443c-b23c-0d58337efebc/_apis/build/builds/126477/artifacts?artifactName=tgz&fileId=AB67468A4BDAFD4425DD4B3A623B9C658DA4D3C4BEE05A9788FA683B0835507802&fileName=/typescript-4.8.0-insiders.20220518.tgz"
    }
}

and then running npm install.

---

There is also a playground for this build and an npm module you can use via "typescript": "npm:@typescript-deploys/pr-build@4.8.0-pr-49156-5".;

[typescript-bot]

@sheetalkamat
The results of the perf run you requested are in!

Here they are:

Comparison Report - main..49156

Metric	main	49156	Delta	Best	Worst
Angular - node (v10.16.3, x64)
Memory used	357,705k (± 0.03%)	357,705k (± 0.02%)	+0k (+ 0.00%)	357,540k	357,897k
Parse Time	2.07s (± 0.32%)	2.07s (± 0.29%)	-0.00s (- 0.05%)	2.05s	2.08s
Bind Time	0.86s (± 0.77%)	0.86s (± 0.78%)	-0.01s (- 0.70%)	0.85s	0.88s
Check Time	5.83s (± 0.53%)	5.82s (± 0.64%)	-0.01s (- 0.21%)	5.74s	5.89s
Emit Time	6.00s (± 0.21%)	6.00s (± 0.56%)	-0.01s (- 0.08%)	5.91s	6.06s
Total Time	14.77s (± 0.27%)	14.74s (± 0.41%)	-0.02s (- 0.16%)	14.65s	14.91s
Compiler-Unions - node (v10.16.3, x64)
Memory used	206,010k (± 0.02%)	206,008k (± 0.03%)	-2k (- 0.00%)	205,883k	206,148k
Parse Time	0.84s (± 0.62%)	0.84s (± 0.74%)	-0.00s (- 0.24%)	0.82s	0.85s
Bind Time	0.51s (± 1.02%)	0.51s (± 0.58%)	+0.00s (+ 0.20%)	0.51s	0.52s
Check Time	7.92s (± 0.17%)	7.94s (± 0.66%)	+0.02s (+ 0.19%)	7.84s	8.10s
Emit Time	2.50s (± 0.94%)	2.53s (± 0.80%)	+0.03s (+ 1.04%)	2.48s	2.58s
Total Time	11.78s (± 0.26%)	11.82s (± 0.47%)	+0.04s (+ 0.34%)	11.70s	12.00s
Monaco - node (v10.16.3, x64)
Memory used	343,841k (± 0.02%)	343,772k (± 0.02%)	-69k (- 0.02%)	343,631k	343,904k
Parse Time	1.59s (± 0.58%)	1.59s (± 0.66%)	-0.01s (- 0.31%)	1.56s	1.61s
Bind Time	0.75s (± 0.48%)	0.76s (± 0.44%)	+0.01s (+ 0.80%)	0.75s	0.77s
Check Time	5.81s (± 0.42%)	5.81s (± 0.46%)	-0.00s (- 0.09%)	5.76s	5.87s
Emit Time	3.25s (± 0.89%)	3.23s (± 0.99%)	-0.02s (- 0.62%)	3.19s	3.34s
Total Time	11.41s (± 0.46%)	11.39s (± 0.34%)	-0.03s (- 0.24%)	11.30s	11.48s
TFS - node (v10.16.3, x64)
Memory used	305,496k (± 0.02%)	305,543k (± 0.01%)	+47k (+ 0.02%)	305,454k	305,613k
Parse Time	1.29s (± 0.63%)	1.29s (± 0.46%)	-0.01s (- 0.54%)	1.28s	1.30s
Bind Time	0.72s (± 0.51%)	0.71s (± 0.67%)	-0.00s (- 0.42%)	0.70s	0.72s
Check Time	5.24s (± 0.46%)	5.26s (± 0.48%)	+0.01s (+ 0.27%)	5.21s	5.30s
Emit Time	3.42s (± 1.11%)	3.44s (± 1.34%)	+0.02s (+ 0.56%)	3.34s	3.53s
Total Time	10.67s (± 0.55%)	10.69s (± 0.46%)	+0.02s (+ 0.21%)	10.58s	10.81s
material-ui - node (v10.16.3, x64)
Memory used	468,492k (± 0.01%)	468,562k (± 0.01%)	+70k (+ 0.01%)	468,467k	468,675k
Parse Time	1.83s (± 0.60%)	1.83s (± 0.41%)	-0.01s (- 0.27%)	1.81s	1.84s
Bind Time	0.67s (± 1.49%)	0.68s (± 0.50%)	+0.00s (+ 0.45%)	0.67s	0.68s
Check Time	14.29s (± 0.37%)	14.33s (± 0.58%)	+0.04s (+ 0.28%)	14.14s	14.52s
Emit Time	0.00s (± 0.00%)	0.00s (± 0.00%)	0.00s ( NaN%)	0.00s	0.00s
Total Time	16.80s (± 0.32%)	16.84s (± 0.51%)	+0.04s (+ 0.22%)	16.64s	17.03s
xstate - node (v10.16.3, x64)
Memory used	575,305k (± 1.31%)	571,914k (± 0.01%)	-3,392k (- 0.59%)	571,725k	572,098k
Parse Time	2.58s (± 0.14%)	2.59s (± 0.27%)	+0.00s (+ 0.19%)	2.58s	2.61s
Bind Time	1.01s (± 0.49%)	1.00s (± 0.37%)	-0.00s (- 0.10%)	1.00s	1.01s
Check Time	1.53s (± 0.87%)	1.53s (± 0.52%)	+0.00s (+ 0.07%)	1.51s	1.55s
Emit Time	0.07s (± 0.00%)	0.07s (± 3.14%)	+0.00s (+ 1.43%)	0.07s	0.08s
Total Time	5.19s (± 0.36%)	5.19s (± 0.25%)	+0.00s (+ 0.04%)	5.16s	5.21s
Angular - node (v12.1.0, x64)
Memory used	335,407k (± 0.02%)	335,362k (± 0.02%)	-46k (- 0.01%)	335,264k	335,509k
Parse Time	2.07s (± 0.60%)	2.07s (± 0.42%)	+0.00s (+ 0.00%)	2.05s	2.09s
Bind Time	0.82s (± 0.68%)	0.83s (± 0.88%)	+0.01s (+ 0.73%)	0.82s	0.85s
Check Time	5.63s (± 0.37%)	5.64s (± 0.61%)	+0.01s (+ 0.21%)	5.58s	5.72s
Emit Time	6.25s (± 0.74%)	6.23s (± 0.74%)	-0.02s (- 0.37%)	6.14s	6.35s
Total Time	14.77s (± 0.35%)	14.76s (± 0.49%)	-0.01s (- 0.05%)	14.59s	14.92s
Compiler-Unions - node (v12.1.0, x64)
Memory used	193,424k (± 0.15%)	193,558k (± 0.10%)	+134k (+ 0.07%)	193,048k	193,817k
Parse Time	0.83s (± 1.09%)	0.83s (± 0.80%)	+0.00s (+ 0.48%)	0.82s	0.85s
Bind Time	0.53s (± 0.94%)	0.53s (± 1.10%)	+0.00s (+ 0.19%)	0.51s	0.54s
Check Time	7.45s (± 0.76%)	7.38s (± 0.35%)	-0.07s (- 0.89%)	7.33s	7.44s
Emit Time	2.54s (± 0.78%)	2.53s (± 0.93%)	-0.00s (- 0.08%)	2.49s	2.59s
Total Time	11.34s (± 0.65%)	11.28s (± 0.41%)	-0.06s (- 0.55%)	11.19s	11.38s
Monaco - node (v12.1.0, x64)
Memory used	326,855k (± 0.02%)	326,735k (± 0.07%)	-120k (- 0.04%)	325,831k	326,957k
Parse Time	1.56s (± 0.65%)	1.54s (± 0.72%)	-0.02s (- 1.02%)	1.52s	1.57s
Bind Time	0.74s (± 0.65%)	0.74s (± 0.70%)	+0.00s (+ 0.27%)	0.73s	0.75s
Check Time	5.63s (± 0.42%)	5.65s (± 0.59%)	+0.02s (+ 0.30%)	5.60s	5.75s
Emit Time	3.27s (± 0.28%)	3.25s (± 0.71%)	-0.02s (- 0.58%)	3.20s	3.32s
Total Time	11.20s (± 0.25%)	11.19s (± 0.43%)	-0.01s (- 0.12%)	11.07s	11.32s
TFS - node (v12.1.0, x64)
Memory used	290,167k (± 0.02%)	290,194k (± 0.02%)	+27k (+ 0.01%)	290,081k	290,344k
Parse Time	1.29s (± 0.69%)	1.29s (± 0.81%)	-0.00s (- 0.15%)	1.28s	1.33s
Bind Time	0.72s (± 0.82%)	0.73s (± 1.19%)	+0.00s (+ 0.55%)	0.71s	0.75s
Check Time	5.22s (± 0.49%)	5.20s (± 0.39%)	-0.02s (- 0.44%)	5.14s	5.23s
Emit Time	3.50s (± 0.56%)	3.47s (± 0.69%)	-0.02s (- 0.66%)	3.40s	3.51s
Total Time	10.74s (± 0.41%)	10.70s (± 0.26%)	-0.04s (- 0.41%)	10.64s	10.77s
material-ui - node (v12.1.0, x64)
Memory used	447,550k (± 0.02%)	447,572k (± 0.01%)	+22k (+ 0.00%)	447,423k	447,730k
Parse Time	1.83s (± 0.41%)	1.82s (± 0.42%)	-0.00s (- 0.05%)	1.81s	1.84s
Bind Time	0.65s (± 0.80%)	0.65s (± 0.46%)	+0.00s (+ 0.15%)	0.65s	0.66s
Check Time	12.88s (± 0.32%)	12.90s (± 0.36%)	+0.02s (+ 0.16%)	12.82s	13.01s
Emit Time	0.00s (± 0.00%)	0.00s (± 0.00%)	0.00s ( NaN%)	0.00s	0.00s
Total Time	15.36s (± 0.28%)	15.38s (± 0.31%)	+0.02s (+ 0.16%)	15.30s	15.50s
xstate - node (v12.1.0, x64)
Memory used	537,510k (± 0.02%)	537,557k (± 0.02%)	+47k (+ 0.01%)	537,318k	537,798k
Parse Time	2.53s (± 0.36%)	2.54s (± 0.55%)	+0.01s (+ 0.40%)	2.52s	2.58s
Bind Time	1.02s (± 1.04%)	1.03s (± 0.56%)	+0.00s (+ 0.20%)	1.02s	1.04s
Check Time	1.47s (± 0.53%)	1.47s (± 0.85%)	+0.00s (+ 0.14%)	1.46s	1.51s
Emit Time	0.07s (± 0.00%)	0.07s (± 0.00%)	0.00s ( 0.00%)	0.07s	0.07s
Total Time	5.10s (± 0.36%)	5.11s (± 0.35%)	+0.01s (+ 0.29%)	5.09s	5.17s
Angular - node (v14.15.1, x64)
Memory used	333,635k (± 0.01%)	333,634k (± 0.01%)	-2k (- 0.00%)	333,593k	333,689k
Parse Time	2.04s (± 0.39%)	2.04s (± 0.54%)	+0.00s (+ 0.00%)	2.02s	2.07s
Bind Time	0.87s (± 1.01%)	0.87s (± 0.38%)	-0.00s (- 0.34%)	0.86s	0.88s
Check Time	5.63s (± 0.57%)	5.62s (± 0.15%)	-0.01s (- 0.14%)	5.61s	5.65s
Emit Time	6.30s (± 0.55%)	6.30s (± 0.64%)	-0.00s (- 0.02%)	6.24s	6.40s
Total Time	14.84s (± 0.35%)	14.83s (± 0.25%)	-0.01s (- 0.10%)	14.74s	14.91s
Compiler-Unions - node (v14.15.1, x64)
Memory used	192,268k (± 0.02%)	192,272k (± 0.02%)	+4k (+ 0.00%)	192,181k	192,333k
Parse Time	0.85s (± 0.56%)	0.84s (± 0.43%)	-0.00s (- 0.35%)	0.84s	0.85s
Bind Time	0.56s (± 0.59%)	0.56s (± 0.80%)	-0.00s (- 0.36%)	0.55s	0.57s
Check Time	7.52s (± 0.39%)	7.51s (± 0.60%)	-0.00s (- 0.04%)	7.44s	7.65s
Emit Time	2.51s (± 0.47%)	2.50s (± 1.20%)	-0.01s (- 0.36%)	2.46s	2.60s
Total Time	11.43s (± 0.26%)	11.42s (± 0.51%)	-0.01s (- 0.12%)	11.32s	11.59s
Monaco - node (v14.15.1, x64)
Memory used	325,634k (± 0.01%)	325,620k (± 0.00%)	-14k (- 0.00%)	325,607k	325,650k
Parse Time	1.57s (± 0.84%)	1.57s (± 0.62%)	+0.00s (+ 0.06%)	1.55s	1.59s
Bind Time	0.78s (± 1.34%)	0.77s (± 1.06%)	-0.01s (- 0.64%)	0.76s	0.80s
Check Time	5.53s (± 0.29%)	5.53s (± 0.28%)	+0.00s (+ 0.09%)	5.50s	5.57s
Emit Time	3.33s (± 0.71%)	3.32s (± 0.94%)	-0.02s (- 0.54%)	3.27s	3.43s
Total Time	11.20s (± 0.35%)	11.19s (± 0.43%)	-0.01s (- 0.12%)	11.11s	11.36s
TFS - node (v14.15.1, x64)
Memory used	289,138k (± 0.01%)	289,124k (± 0.01%)	-14k (- 0.00%)	289,096k	289,158k
Parse Time	1.36s (± 1.73%)	1.36s (± 1.04%)	-0.00s (- 0.07%)	1.34s	1.40s
Bind Time	0.72s (± 1.05%)	0.72s (± 1.03%)	-0.00s (- 0.14%)	0.71s	0.75s
Check Time	5.22s (± 0.47%)	5.20s (± 0.48%)	-0.02s (- 0.33%)	5.16s	5.27s
Emit Time	3.59s (± 1.20%)	3.51s (± 2.02%)	-0.08s (- 2.23%)	3.37s	3.63s
Total Time	10.89s (± 0.46%)	10.79s (± 0.67%)	-0.10s (- 0.91%)	10.66s	10.95s
material-ui - node (v14.15.1, x64)
Memory used	445,756k (± 0.01%)	445,617k (± 0.06%)	-139k (- 0.03%)	444,479k	445,770k
Parse Time	1.86s (± 0.41%)	1.87s (± 0.41%)	+0.01s (+ 0.32%)	1.86s	1.89s
Bind Time	0.70s (± 0.53%)	0.70s (± 0.82%)	+0.00s (+ 0.72%)	0.69s	0.71s
Check Time	13.03s (± 0.71%)	13.02s (± 0.69%)	-0.01s (- 0.06%)	12.86s	13.32s
Emit Time	0.00s (± 0.00%)	0.00s (± 0.00%)	0.00s ( NaN%)	0.00s	0.00s
Total Time	15.59s (± 0.60%)	15.59s (± 0.58%)	-0.00s (- 0.01%)	15.42s	15.88s
xstate - node (v14.15.1, x64)
Memory used	535,264k (± 0.00%)	535,292k (± 0.01%)	+28k (+ 0.01%)	535,208k	535,373k
Parse Time	2.58s (± 0.51%)	2.58s (± 0.56%)	+0.00s (+ 0.12%)	2.56s	2.62s
Bind Time	1.15s (± 0.87%)	1.14s (± 0.39%)	-0.01s (- 0.78%)	1.13s	1.15s
Check Time	1.52s (± 0.55%)	1.52s (± 0.22%)	+0.00s (+ 0.07%)	1.51s	1.52s
Emit Time	0.07s (± 0.00%)	0.07s (± 0.00%)	0.00s ( 0.00%)	0.07s	0.07s
Total Time	5.32s (± 0.21%)	5.31s (± 0.24%)	-0.01s (- 0.21%)	5.29s	5.35s

System

Machine Name	ts-ci-ubuntu
Platform	linux 4.4.0-210-generic
Architecture	x64
Available Memory	16 GB
Available Memory	1 GB
CPUs	4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz

Hosts

• node (v10.16.3, x64)
• node (v12.1.0, x64)
• node (v14.15.1, x64)

Scenarios

• Angular - node (v10.16.3, x64)
• Angular - node (v12.1.0, x64)
• Angular - node (v14.15.1, x64)
• Compiler-Unions - node (v10.16.3, x64)
• Compiler-Unions - node (v12.1.0, x64)
• Compiler-Unions - node (v14.15.1, x64)
• Monaco - node (v10.16.3, x64)
• Monaco - node (v12.1.0, x64)
• Monaco - node (v14.15.1, x64)
• TFS - node (v10.16.3, x64)
• TFS - node (v12.1.0, x64)
• TFS - node (v14.15.1, x64)
• material-ui - node (v10.16.3, x64)
• material-ui - node (v12.1.0, x64)
• material-ui - node (v14.15.1, x64)
• xstate - node (v10.16.3, x64)
• xstate - node (v12.1.0, x64)
• xstate - node (v14.15.1, x64)

Benchmark	Name	Iterations
Current	49156	10
Baseline	main	10

Developer Information:

Download Benchmark

[sheetalkamat]

@typescript-bot pack this

[typescript-bot]

Heya @sheetalkamat, I've started to run the tarball bundle task on this PR at 1be59b2. You can monitor the build here.

[typescript-bot]

Hey @sheetalkamat, I've packed this into an installable tgz. You can install it for testing by referencing it in your package.json like so:

{
    "devDependencies": {
        "typescript": "https://typescript.visualstudio.com/cf7ac146-d525-443c-b23c-0d58337efebc/_apis/build/builds/127785/artifacts?artifactName=tgz&fileId=46642C0DDD3E668571B6CCF24513AC8BB9E2EE86E668F55AFAC63EF1F15D9EDE02&fileName=/typescript-4.8.0-insiders.20220609.tgz"
    }
}

and then running npm install.

---

There is also a playground for this build and an npm module you can use via "typescript": "npm:@typescript-deploys/pr-build@4.8.0-pr-49156-9".;

[sheetalkamat]

@DanielRosenwasser I am having hard time reasoning about this change.
There are three places currently doing declaration emit/transforms:

1. Builder: To get signature
2. DeclarationDiagnostics: Does only transform and gets error
3. Actual d.ts emit

As part of this i am caching either transform or text (preferring text over transform) and declaration diagnostics so we can reuse it.
I am unable to see lot of benefit of this though. Mainly because:

1. By default when we everything is clean and not emitted, the first run: Builder is not going to do signature emit any more (Do not calculate signatures if old state is not used #43314)
2. Change to store signature computed during emit helps with not having big working set for the next iteration: (During emit, if shape signature for the file is same as version, then update it with emitted d.ts file #48616) so this makes it as if in first build we did d.ts emit so number of files on which d.ts emit would be done to calculate their signature by the builder should be smaller. So perf win is possible only in next edits but that is not huge since the file list is smaller.
3. noEmitOnError is only when we really get declaration diagnostics separately so thats the only place we would save d.ts transform cost.

I havent been able to find repo where this benefits significantly. Numbers remain more or less same.

[sheetalkamat]

cc @amcasey