Implement Header encoding selectors on SocketsHttpHandler

[MihaZupan]

Contributes to #38711

Implemented:

namespace System.Net.Http
{
    public delegate Encoding? HeaderEncodingSelector<TContext>(string headerName, TContext context);

    public sealed class SocketsHttpHandler
    {
        public HeaderEncodingSelector<HttpRequestMessage>? RequestHeaderEncodingSelector { get; set; }
        public HeaderEncodingSelector<HttpRequestMessage>? ResponseHeaderEncodingSelector { get; set; }
    }
}

Part of the change is exposing overloads taking an Encoding in HPack and QPack encoders. HPack code is shared with aspnetcore, how are these changes propagated?

Loopback server changes:

• Exposed the raw bytes for the received header value in LoopbackServer
• Added support for Loopback servers to send header values with the specified encoding

[ghost]

Tagging subscribers to this area: @dotnet/ncl
Notify danmosemsft if you want to be subscribed.

[Dotnet-GitSync-Bot]

Note regarding the new-api-needs-documentation label:

This serves as a reminder for when your PR is modifying a ref *.cs file and adding/modifying public APIs, to please make sure the API implementation in the src *.cs file is documented with triple slash comments, so the PR reviewers can sign off that change.

[Tratcher]

Part of the change is exposing overloads taking an Encoding in HPack and QPack encoders. HPack code is shared with aspnetcore, how are these changes propagated?

See https://github.com/dotnet/runtime/blob/43b8caf5d455d7a2e32de13f813f4caf66d53340/src/libraries/Common/src/System/Net/Http/aspnetcore/ReadMe.SharedCode.md

[halter73]

@Tratcher What do you think about switching Kestrel's RequestHeaderEncodingSelector to be a delegate System.Text.Encoding? HeaderEncodingSelector(string headerName) delegate?

It would be a new delegate type most likely defined in the Microsoft.AspNetCore.Server.Kestrel namespace since we don't have an HttpRequestMessage or really anything comparable in the early stages of request parsing. Having the same name as the System.Net.Http delegate might be too confusing though.

I originally went with Func<string, Encoding?> (that's at least what it will be once we support nullable reference types in that assembly) because we don't use delegates very much in configuration callbacks in ASP.NET Core. Thinking about it more though, that's because normally the argument is a specific options or context type. Since the argument is just a string, naming it feels more worthwhile. Increased consistency with System.Net.Http is an added bonus.

[Tratcher]

It would be a small usability improvement.

I'd avoid using the same name though, that gets awkward when you import both namespaces.

[scalablecory]

It looks pretty good but I'm up late and sleep deprived; I will do a 2nd pass when I have a clearer mind.

[scalablecory]

looks good, thanks. only trivial questions/suggestions.

[scalablecory]

Do we have evidence that Latin1 will be used frequently or in perf-sensitive scenarios?

I would have expected people to largely use UTF-8.

[MihaZupan]

Latin1 is the scenario required/requested to achieve compatibility with .Net Framework behavior (not throwing on non-ascii chars).

I expect the 99% use case for this to be UTF8 and Latin1, so I believe it is worth special-casing both, as keeping Latin1 optimizations shouldn't hurt UTF8 perf.

[scalablecory]

UTF-8 uses a 4-byte max encoding, so this isn't immediately obviously correct. Can you use a comment similar to this one to make it clear?

runtime/src/libraries/System.Private.CoreLib/src/System/Text/UTF8Encoding.cs

Lines 49 to 56 in 6072e4d

	/// <summary>
	/// Transcoding to UTF-8 bytes from UTF-16 input chars will result in a maximum 3:1 expansion.
	/// </summary>
	/// <remarks>
	/// Supplementary code points are expanded to UTF-8 from UTF-16 at a 4:2 ratio,
	/// so 3:1 is still the correct value for maximum expansion.
	/// </remarks>
	private const int MaxUtf8BytesPerChar = 3;

@GrabYourPitchforks do we have some UTF-8 support code in /Common that we could place this constant in?

[MihaZupan]

Simplified to just GetMaxByteCount, as it is a very fast method anyway.

[MihaZupan]

/azp run runtime-libraries outerloop

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).