[Feature] About support multi threads and auto finalize for skywalking-rust.

[jmjoy]

Search before asking

• I had searched in the issues and found no similar feature requirement.

Description

Now the skywalking-rust only support one thread tracing, but usually there are some scenes that require multi threads/coroutines tracing.

Use case

Rust application requires multi threads and coroutines tracing.

Related issues

apache/skywalking-rust#20 (comment)
apache/skywalking-rust#26
apache/skywalking-rust#28
apache/skywalking-rust#29

Are you willing to submit a PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[jmjoy]

I think the new api design is (fake code):

use skywalking_rust::context::trace_context::TracingContext;
use skywalking_rust::reporter::grpc::Reporter;
use tokio;

static TRACING_MANAGER: Lazy<TracingManager> = Lazy::new(|| TracingManager::default("svc", "ins"));

async fn handle_request(reporter: ContextReporter) {
    let mut ctx = TRACING_MANAGER.create_context();

    {
        // Generate an Entry Span when a request
        // is received. An Entry Span is generated only once per context.
        let span = ctx.create_entry_span("operation1").unwrap();

        // Something...

        {
            // Generates an Exit Span when executing an RPC.
            let span2 = ctx.create_exit_span("operation2").unwrap();
            
            // Something...

            // Drop span2 and finalize to ctx.
        }

        // Drop span and finalize to ctx.
    }

    // Create coroutine to handle async task.
    spawn(async move {
        let mut ctx2 = TRACING_MANAGER.create_context();

        {
            let span3 = ctx2.create_entry_span("operation3").unwrap();

            // Something...

            // Drop span2 and finalize to ctx2.
        }

        // Drop ctx2 and send to TRACING_MANAGER.
    });

    // Drop ctx and send to TRACING_MANAGER.
}

#[tokio::main]
async fn main() {
    let tx = Reporter::start("http://0.0.0.0:11800").await;

    // Start server...

    // Block to report.
    loop {
        let context = TRACING_MANAGER.receive_context();
        reporter.send(context).await;
    }
}

Add the TracingManager which should be singleton in the application, to manage the contexts, and use notify mechanism for reporting.

[wu-sheng]

Do you want to implement ctx#capture and ctx2#continous in these APIs, which could link ctx and ctx in a whole trace although they are created in two threads originally?

[jmjoy]

Do you want to implement ctx#capture and ctx2#continous in these APIs, which could link ctx and ctx in a whole trace although they are created in two threads originally?

OK, let me think about the design of API.

[jmjoy]

What is the purpose of ctx#capture and ctx2#continous? Is there other language can be referenced? I just think it can add a parameter like TRACING_MANAGER.create_context(parent_context).

[lujiajing1126]

What is the purpose of ctx#capture and ctx2#continous? Is there other language can be referenced? I just think it can add a parameter like TRACING_MANAGER.create_context(parent_context).

Every Segment in SkyWalking exists in a single thread while a transaction/trace may consist of multiple operations across threads. To rebuild the whole trace in the backend (i.e. OAP), we have to link segments together with the help of contextual information, e.g. traceID, segmentID, spanID and user-defined correlation context.

Context.capture and Context. continous are key methods to capture the "context" in a thread and restore it in another thread.

You may refer to java impl, https://github.com/apache/skywalking-java/blob/main/apm-sniffer/apm-agent-core/src/main/java/org/apache/skywalking/apm/agent/core/context/ContextManager.java#L157

[wu-sheng]

What is the purpose of ctx#capture and ctx2#continous? Is there other language can be referenced? I just think it can add a parameter like TRACING_MANAGER.create_context(parent_context).

The parent segment is not a very accurate concept, as there are many spans in current and parent context, you can't know which span of ctx is referring another span in ctx2.

This doc may help you understanding this concept, https://skywalking.apache.org/docs/skywalking-java/latest/en/setup/service-agent/java-agent/java-plugin-development-guide/#contextsnapshot.

In the protocol, the span's reference doesn't only represent the peer of an RPC/MQ, it also could represent a cross-thread reference.

https://github.com/apache/skywalking-data-collect-protocol/blob/906a834dc0c3fb66719b8313f8f9298be3a27349/language-agent/Tracing.proto#L125-L128

https://github.com/apache/skywalking-data-collect-protocol/blob/906a834dc0c3fb66719b8313f8f9298be3a27349/language-agent/Tracing.proto#L79-L82

[jmjoy]

I'm studying capture and continous through Java Kafka. I've opened foo and bar consumers, and produced messages through the request of /mq2.

I've determined through breakpoint debugging that the application has called to generate LocalSpan of Kafka/Producer/Callback.

But I'm confused that LocalSpan of Kafka/Producer/Callback is not displayed on the interface.

[wu-sheng]

Maybe some visualization bug? I am not sure about this.
Because this span has two references, one is RPC peer, the other is from the local span's snapshot.

If you purely want to check capture and continuous, all async HTTP call or callback should include this case.

[jmjoy]

Maybe some visualization bug? I am not sure about this. Because this span has two references, one is RPC peer, the other is from the local span's snapshot.

If you purely want to check capture and continuous, all async HTTP call or callback should include this case.

Yes, I switch to using the old 8.x UI, and the Kafka/Producer/Callback is shown.

[wu-sheng]

FYI @Fine0830 We may miss mutiple references in the span page in 9.x