Cutting room floor: plotting memusagestat by stagnation · Pull Request #11 · meroton/docs

stagnation · 2023-11-30T14:31:44Z

No description provided.

moroten

Generally good. Just some minor stuff.

moroten · 2023-12-05T13:19:21Z

blog/2024-01-01-dumping-memusagestat-data-points.md

@@ -0,0 +1,294 @@
+# Plotting memusagestat's data
+
+This came up during my preparations for the Bazelcon talk [Dude, where is my RAM?].


What "This"?

moroten · 2023-12-05T13:20:06Z

blog/2024-01-01-dumping-memusagestat-data-points.md

+# Plotting memusagestat's data
+
+This came up during my preparations for the Bazelcon talk [Dude, where is my RAM?].
+But the investigation and the graphs produced by `memusagestat` did not make the cut for the talk.


Replace ". But" with "but".

moroten · 2023-12-05T13:29:04Z

blog/2024-01-01-dumping-memusagestat-data-points.md

+The plot is often nice but there are few options for how to present the data.
+Either by sequential allocation or by linear time,
+and the labels are very small.


I understand this as the existence of the options is a good thing and enough, so the only bad thing is that the labels are small. Is that correct?

moroten · 2023-12-05T13:30:15Z

blog/2024-01-01-dumping-memusagestat-data-points.md

+So we want to generate new figures for the same data.
+
+We use `memusage` to profile a `bazel` invocation that runs out of memory


Suggested change

So we want to generate new figures for the same data.

We use `memusage` to profile a `bazel` invocation that runs out of memory

So to generate new figures for the same data,

we use `memusage` to profile a `bazel` invocation that runs out of memory

moroten · 2023-12-05T13:30:56Z

blog/2024-01-01-dumping-memusagestat-data-points.md

+    //...
+```
+
+And render a plot


Suggested change

And render a plot

and render a plot

moroten · 2023-12-05T13:39:08Z

blog/2024-01-01-dumping-memusagestat-data-points.md

+Note, the x-axis is the number of allocations,
+when plotted against time the graph instead looks like a noisy square wave.
+Which is interesting in its own right,


Suggested change

Note, the x-axis is the number of allocations,

when plotted against time the graph instead looks like a noisy square wave.

Which is interesting in its own right,

Note that the x-axis is the number of allocations.

When plotted against time the graph instead looks like a noisy square wave,

which is interesting in its own right.

moroten · 2023-12-05T13:39:38Z

blog/2024-01-01-dumping-memusagestat-data-points.md

+
+### memusage writes a binary datafile
+
+Glancing at [the code] it is quite straight forward,


Suggested change

Glancing at [the code] it is quite straight forward,

Glancing at [the code] it is quite straight forward

moroten · 2023-12-05T13:52:31Z

blog/2024-01-01-dumping-memusagestat-data-points.md

+Some file is missing, this error is often raised when the program itself is missing,
+which it is not.
+But some other file is.


Suggested change

Some file is missing, this error is often raised when the program itself is missing,

which it is not.

But some other file is.

Some file is missing. This error is often raised when the program itself is missing,

which it is not,

but some other file is.

moroten · 2023-12-05T13:55:05Z

blog/2024-01-01-dumping-memusagestat-data-points.md

+
+![single scale stack usage](/img/memusagestat-python-single-scale.png)
+
+[available on github]: github.com/meroton/parse-memusagestat/


Bad link. Both relative and non-existing repo.

moroten · 2023-12-05T13:55:43Z

blog/2024-01-01-dumping-memusagestat-data-points.md

+Our `memusagestat` parser and plotter is [available on github].
+And can draw prettier images:


Suggested change

Our `memusagestat` parser and plotter is [available on github].

And can draw prettier images:

Our `memusagestat` parser and plotter is [available on github]

and can draw prettier images:

meroton-benjamin

General notes, you tend to use so, this and but a lot. Try varying this, you can almost always remove So from the beggining of a sentence.

meroton-benjamin · 2023-12-12T09:42:54Z

docs/captains-log/plotting-memusagestat-with-python.md

+for GNU/Linux 3.2.0, with debug_info, not stripped
+```
+
+Aha! This is an interpreted file that wants `ld` to start


missing period

meroton-benjamin · 2023-12-12T09:43:23Z

docs/captains-log/plotting-memusagestat-with-python.md

+### Compile memusagestat
+
+For good measure we can patch the parsing code in `memusagestat`
+to have oracle data for our parser.


When introducing an esoteric concept in a text, such as an oracle, define it or reference it, otherwise the reference will be lost to a significant chunk of the audience

Alternatively use a more common concept (reference data)

meroton-benjamin · 2023-12-12T09:44:04Z

docs/captains-log/plotting-memusagestat-with-python.md

+
+This shows each entry in the binary file,
+heap, stack, lower 32 bits of the time and the higher,
+as well as the joined time within parentheses


missing period

meroton-benjamin · 2023-12-12T09:55:28Z

docs/captains-log/plotting-memusagestat-with-python.md

+For good measure we can patch the parsing code in `memusagestat`
+to have oracle data for our parser.
+`memusagestat` comes with `glibc` and can be built with its `make` build system.
+We need to do some preparation, to have all the dynamic libraries work well.


In order to have all the dynamic libraries work well we need to do some preparation

meroton-benjamin · 2023-12-12T10:03:53Z

docs/captains-log/plotting-memusagestat-with-python.md

+
+`memusage` is a great tool to track all allocations for a program,
+this comes to many Linux distributions with `glibc` itself.
+So if you need a quick-and-easy allocation tracker `memusage` is the tool for you!


Remove initial So

meroton-benjamin · 2023-12-12T10:41:57Z

docs/captains-log/index.md

+# Captain's Log
+
+Follow the Captain's Odyssey to the best and fastest builds.
+This salt-stained log is where he documents the fair sights


This salt-stained log documents the fair sights...

meroton-benjamin · 2023-12-12T10:42:44Z

docs/captains-log/index.md

+This salt-stained log is where he documents the fair sights
+and obstacles encountered.
+
+This contains information about many auxiliary tools and technologies,


The log contains....

meroton-benjamin · 2023-12-12T10:45:31Z

docs/captains-log/index.md

+and obstacles encountered.
+
+This contains information about many auxiliary tools and technologies,
+and notes on all the moving pieces in the remote execution ecosystem.


as well as

(avoid using and as a conjunction between sentances after using it as a list separator right before)

meroton-benjamin · 2023-12-12T10:46:08Z

blog/2023-12-08-plotting-memusagestat-data-points.md

+tags: [linux, glibc, python]
+---
+
+`memusage` is a great tool to track all allocations for a program,


see comments on first heading in article

meroton-benjamin · 2023-12-12T10:53:35Z

blog/2023-12-08-plotting-memusagestat-data-points.md

+You can read the full document here: [Plotting memusagestat's data].
+There are two take-home messages:
+
+  * One, is a case-study in building and modifying `glibc` tools for your computer:


Either use full sentences (A case study in... and our code that you can leverage to plot...) or an ordered list

A case study in building and modifying

Our code to plot allocations

stagnation force-pushed the feature/plotting-memusagestat-in-python branch 2 times, most recently from ff9b4d6 to 44a55e2 Compare December 5, 2023 13:14

moroten requested changes Dec 5, 2023

View reviewed changes

stagnation force-pushed the feature/plotting-memusagestat-in-python branch 2 times, most recently from b87719e to beed912 Compare December 8, 2023 09:37

meroton-benjamin requested changes Dec 12, 2023

View reviewed changes

stagnation added the editorial label Dec 19, 2023

stagnation force-pushed the feature/plotting-memusagestat-in-python branch 2 times, most recently from 6379c27 to 9cbdc67 Compare September 18, 2024 11:44

stagnation force-pushed the feature/plotting-memusagestat-in-python branch 3 times, most recently from 35032a9 to 361ae6d Compare October 1, 2024 07:51

Plotting memusagestat in python

2d4c40f

stagnation force-pushed the feature/plotting-memusagestat-in-python branch from 361ae6d to 2d4c40f Compare October 7, 2024 08:55

		@@ -0,0 +1,294 @@
		# Plotting memusagestat's data

		This came up during my preparations for the Bazelcon talk [Dude, where is my RAM?].

		So we want to generate new figures for the same data.

		We use `memusage` to profile a `bazel` invocation that runs out of memory


		### memusage writes a binary datafile

		Glancing at [the code] it is quite straight forward,


		![single scale stack usage](/img/memusagestat-python-single-scale.png)

		[available on github]: github.com/meroton/parse-memusagestat/

		Our `memusagestat` parser and plotter is [available on github].
		And can draw prettier images:

Conversation

stagnation commented Nov 30, 2023

Uh oh!

moroten left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

meroton-benjamin left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants