Discussion: Integrate our fork

[jonashaag]

Intro and motivation

As already noted here vmprof/vmprof-python#90 (comment), we have implemented our own vmprof server, for the following reasons:

• At that time, vmprof-server was very slow on large profiles (multiple hours of runtime) as it stored the profiles in the SQL database. (I'm not sure how the current implementation compares to ours.)
• We wanted to have a good memory profile viewer in the server
• We don't really need user accounts etc.

Features of our implementation

I can't share the source code of our server just yet, for bureaucratic reasons, but I can share some information and a few screenshots here.

Properties and differences to vmprof-server:

• About 1000 LOC
• Uses vmprof-server CPU viewer (no jitlog integration yet)
• Much improved memory viewer based on Plotly:
  • Shows time and date on X axis
  • Data points aren't simply sampled from the complete data, but binned (mean), so that you don't miss spikes due to too-low sampling interval
  • Show memory usage mean, max or both
  • Nice interaction with the graph due to Plotly
  • Shows absolute or relative runtime on X axis
• Allows to search for projects and functions/callables
• Stores profiles as gzipped msgpack files, no decoding done in the server whatsoever: Data is encoded to .msgpack.gz in the client once, and delivered to the browser UI as-is. The only exception to this is for memory profile resampling.

We have also implemented a new client:

• About 100 LOC

• Interface isn't a script runner like python -m vmprof yourscript.py but a decorator that is applied to to-be-profiled callables, like

  @profile
  def somefunc():
      ...

• Allows to tag your submissions with a project name

• Automatically tags your submissions with the top-level function/callable name (somefunc)

• Client can upload normal vmprof profile files

• Client protocol not compatible to the vmprof-python protocol (but very similar)

Screenshots

Landing page with project names and top-level function names (in red). Search filters may be shared using the arrow on the right

Integration of vmprof-server CPU viewer

The memory viewer, showing max memory usage for each bin (no hidden spikes!). On the right: Upper stacktrace shows the largest common denominator of all stack traces of the bin. Lower stacktrace extends the upper one by the most common stack trace of the bin (28% of the stack traces in the bin were equal to the "concatenation" of the two stack trace parts).

Memory viewer showing mean + max of each bin

Future of our server, integration in vmprof-server

I think our server has some nice properties, mainly the memory viewer and the storage system (although I'm not sure how it compares to the current vmprof-server JSON/Gzip storage system in terms of performance). We'd love the contribute most of it back to vmprof-server proper.

Possibility A: Integrate vmprof-server into our server

• Integrate jitlog into our server
• Maybe integrate user accounts into our server
• Make our server the new official server

Possibility B: Integrate our memory viewer into vmprof-server

• Add memory view to vmprof-server
• Change protocol accordingly

What do you guys think?

[jonashaag]

cc @StephanErb

[planrich]

Hi, looks like a solid enhancement to vmprof-server. As I see things now: I would stand up for possibilty B. We already maintain and run the service and this will continue that way.

How did you solve the SQL storage issue? Store the gzipped profiles on the file system? If you ask me we should not continue to store json in the relational database. For jitviewer the gzipped file is stored on the file system which is good enough as far as I can tell.

[planrich]

In any case, is the source available on the web? I would to look at it for a bit to estimate how much work it would be to merge it. Or would you just open a pull request then?

[jonashaag]

How did you solve the SQL storage issue? Store the gzipped profiles on the file system?

Yes

In any case, is the source available on the web? I would to look at it for a bit to estimate how much work it would be to merge it. Or would you just open a pull request then?

As I said I can't make it available at the moment but I'll send you a private, confidential copy in a few seconds.

[planrich]

Any news? I'm planning to release a bug fix version today, so we could aim for the next major release

[jonashaag]

Have you had a look at the code? I haven't put in the effort of making our server open source yet; I expected your "complexity of integration assessment" first. But we can also make it open source first.

[planrich]

Yes I did, nothing that takes tremendous amount of re-engineering. If you can make it open source, we will integrate it into the public service. After all this is a feature many people want to have!

[jonashaag]

Cool, we are through with the internal open source process, I'll release the source by the end of the week.

[jonashaag]

Here we go!

https://github.com/blue-yonder/vmprof-viewer-client
https://github.com/blue-yonder/vmprof-viewer-server

[planrich]

Great! I assume that the licence of both is compatible to MIT? I'll probably find some time soon to pull in your fork!

[jonashaag]

Awesome! I'm happy to assist. License is MIT, yes. The features that are most important for us are memory profiles with good stack traces and higher-performance profile storage (not in the SQL database)

[jonashaag]

What's the current state of this? Is there anything I can do to help get the merge done?

[planrich]

There are a view commits I have made, the migrations are ready but I need to apply some changes here and there (model names have changed, ...). We also need to copy and modify the client in the vmprof package. That would be a small project on it's own.

[jonashaag]

OK, let me know if when should help out.