`SemanticTokensFeatureShape.on` handler does not allow returning `null`

[DetachHead]

accoring to the spec, all of the textDocument/semanticTokens/* requests support returning null:

Response:

• result: SemanticTokens | null where SemanticTokens is defined as follows:

export interface SemanticTokens {
	/**
	 * An optional result id. If provided and clients support delta updating
	 * the client will include the result id in the next semantic token request.
	 * A server can then instead of computing all semantic tokens again simply
	 * send a delta.
	 */
	resultId?: string;

	/**
	 * The actual tokens.
	 */
	data: uinteger[];
}

however the on, onDelta and onRange functions do not allow handlers that return null:

vscode-languageserver-node/server/src/common/semanticTokens.ts

Lines 22 to 24 in f5e23e2

	on(handler: ServerRequestHandler<SemanticTokensParams, SemanticTokens, SemanticTokensPartialResult, void>): Disposable;
	onDelta(handler: ServerRequestHandler<SemanticTokensDeltaParams, SemanticTokensDelta | SemanticTokens, SemanticTokensDeltaPartialResult | SemanticTokensPartialResult, void>): Disposable;
	onRange(handler: ServerRequestHandler<SemanticTokensRangeParams, SemanticTokens, SemanticTokensPartialResult, void>): Disposable;

this caused a bug in my language server, because we incorrectly assumed that this meant the only way to return an empty response is by sending an empty array in SemanticTokens.data:

this.connection.languages.semanticTokens.on(async (params, token) => {
    const uri = this.convertLspUriStringToUri(params.textDocument.uri);
    const workspace = await this.getWorkspaceForFile(uri);
    if (workspace.disableLanguageServices) {
          return {
              resultId: undefined,
              data: [],
          };
    }
    return workspace.service.run((program) => {
        return new SemanticTokensProvider(program, uri, token).onSemanticTokens();
    }, token);
});

this causes the language server to override semantic tokens provided by another language server, which isn't what we went. see facebook/pyrefly#3602 (comment)