Advancing Metrics in .NET

[noahfalk]

In .NET 6 we added the Meter API, a metrics instrumentation API designed in coordination with OpenTelemetry. The goal was to make it easy to have great metrics monitoring experience for .NET apps using common tools. Although this is now possible to do it isn't yet as easy, widespread, or powerful as we would like. These are my thoughts on next steps. Feedback is welcome if you think the goals should be adjusted, there are issues I've missed, or anything else. Thanks!

Goals

1. It is easy to set up an ASP.NET Core web app and monitor both platform metrics and custom metrics using (OpenTelemetry or dotnet-monitor) + Grafana. A default Grafana dashboard should be available as a simple starting point.
2. Using the Meter API in .NET libraries and apps is easy with idiomatic API patterns and documented best practice.
3. Basic platform level metrics desired by most apps are already instrumented out-of-the-box.

Issues to Address

1. We have general usage guidance but we need to create usage guidance specific to ASP.NET Core. ASP.NET Core has distinct differences in what code patterns are considered idiomatic. We will may find that it is hard to make the existing API appear idiomatic in ASP.NET Core in which case some judicious use of new APIs may be necessary. (Related: [API Proposal]: Introduce DI friendly IMeter<T> for modern services runtime#77514)
2. We need to identify any critical missing metrics and add them, for example a request latency histogram.
3. Existing instrumentation in .NET has been implemented as EventCounters. We need to define the path forward both for new instrumentation and pre-existing instrumentation. (Modify EventCounter usage to support the new metric APIs as well aspnetcore#33387 and Implement/Convert existing EventCounters to Metrics runtime#79371)
4. We either need to identify an existing Grafana dashboard we can re-use at https://grafana.com/grafana/dashboards, or create a new one. We should also update our tutorial docs to show how to set it up.

cc @samsp-msft @davidfowl @tarekgh @reyang

Tagging subscribers to this area: @dotnet/area-meta
See info in area-owners.md if you want to be subscribed.

Issue Details

---

In .NET 6 we added the Meter API, a metrics instrumentation API designed in coordination with OpenTelemetry. The goal was to make it easy to have great metrics monitoring experience for .NET apps using common tools. Although this is now possible to do it isn't yet as easy, widespread, or powerful as we would like. These are my thoughts on next steps. Feedback is welcome if you think the goals should be adjusted, there are issues I've missed, or anything else. Thanks!

Goals

1. It is easy to set up an ASP.NET Core web app and monitor both platform metrics and custom metrics using (OpenTelemetry or dotnet-monitor) + Grafana. A default Grafana dashboard should be available as a simple starting point.
2. Using the Meter API in .NET libraries and apps is easy with idiomatic API patterns and documented best practice.
3. Basic platform level metrics desired by most apps are already instrumented out-of-the-box.

Issues to Address

1. We have general usage guidance but we need to create usage guidance specific to ASP.NET Core. ASP.NET Core has distinct differences in what code patterns are considered idiomatic. We will may find that it is hard to make the existing API appear idiomatic in ASP.NET Core in which case some judicious use of new APIs may be necessary. (Related: [API Proposal]: Introduce DI friendly IMeter<T> for modern services runtime#77514)
2. We need to identify any critical missing metrics and add them, for example a request latency histogram.
3. Existing instrumentation in .NET has been implemented as EventCounters. We need to define the path forward both for new instrumentation and pre-existing instrumentation. (Modify EventCounter usage to support the new metric APIs as well aspnetcore#33387 and Implement/Convert existing EventCounters to Metrics runtime#79371)
4. We either need to identify an existing Grafana dashboard we can re-use at https://grafana.com/grafana/dashboards, or create a new one. We should also update our tutorial docs to show how to set it up.

cc @samsp-msft @davidfowl @tarekgh @reyang

Author:	noahfalk
Assignees:	-
Labels:	area-Meta, untriaged
Milestone:	-

[xakep139]

Regarding request latency histogram (issue #2): dotnet/aspnetcore#47536
Final naming is in dotnet/aspnetcore#48536

[tommcdon]

All remaining .NET 8 work is tracked on issues. The remaining item below can be done out of band from the .NET release, so moving to the dotnet/diagnostics repo:

4. We either need to identify an existing Grafana dashboard we can re-use at https://grafana.com/grafana/dashboards, or create a new one. We should also update our tutorial docs to show how to set it up.

[noahfalk]

I think everything described above has been done.

-DI friendly Meter: https://learn.microsoft.com/en-us/dotnet/core/diagnostics/metrics-instrumentation#get-a-meter-via-dependency-injection
-ASP.NET built-in metrics: https://learn.microsoft.com/en-us/aspnet/core/log-mon/metrics/built-in?view=aspnetcore-9.0
-Useful EventCounters have been re-implemented as metrics: https://learn.microsoft.com/en-us/dotnet/core/diagnostics/built-in-metrics
-ASP.NET Core dashboard: https://devblogs.microsoft.com/dotnet/introducing-aspnetcore-metrics-and-grafana-dashboards-in-dotnet-8/