Bug 391635 - heaptrack_print: output to easily parsable data format
Summary: heaptrack_print: output to easily parsable data format
Status: RESOLVED WORKSFORME
Alias: None
Product: Heaptrack
Classification: Applications
Component: general (show other bugs)
Version: unspecified
Platform: Other Linux
: NOR normal
Target Milestone: ---
Assignee: Milian Wolff
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2018-03-09 22:02 UTC by Andrew Somerville
Modified: 2022-12-18 05:15 UTC (History)
0 users

See Also:
Latest Commit:
Version Fixed In:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Andrew Somerville 2018-03-09 22:02:34 UTC
Using the output of heaptrack would be easiest by using AccumulatedTraceData (Rather than trying to reparse the text output format). To do that, however, more headers would need to be exposed.

It would be great if all the headers were installed to make this possible.
Comment 1 Milian Wolff 2018-03-10 21:30:44 UTC
Sorry, but I don't have any indication of providing a stable API to parse the heaptrack data files at this point. Or would you be OK even with something that can change it's ABI and API? Where do you want to integrate heaptrack? Why is the existing _print/_gui consumers not enough?
Comment 2 Andrew Somerville 2018-03-13 19:01:40 UTC
I really love heaptrack. Thank you so much for writing it/ and putting in so much time!

The particular use case on my end is that we're using heaptrack for continuous integration to compare successive versions of our software to detect heap behavior changes.

I don't know about others, but I'd be ok with API and ABI changing often as long the version number incremented such that incompatibilities can be known to dependees.

Using the current text output requires that we develop/test a parser that we are confident is correct and then create data-structures to parse into to re-consume the data. That seems wasteful/undesirable considering before it's printed it's already in well structured objects.

Another possible alternative would be a printer which outputs to a standard output format that already has easily available parsers, like yaml, or json. Such a change would add a dependency on a yaml or json parser library unless it was formatted by hand. 

From a practical standpoint for myself, we can produce a custom version of heaptrack which exports the interfaces we care about rather than depending on the official version, but I can imagine other might also benefit.

I can offer to make a patch for such an output if it would be acceptable/desirable.
Comment 3 Milian Wolff 2018-03-13 19:06:20 UTC
Creating yaml/json/xml or whathever output is much more preferrable to me. And yes, patches for that would be very much welcome! I suggest you start looking into heaptrack_print's source code and add a "format" switch there. It should be relatively easy to change the output from the current plain text to something more structured. And no need to pull in separate libs for that purpose, all of the formats are so trivial to write that we can just do it by hand.

One more note: heaptrack_print can already output data in flamegraph format. This is actually also a potential format you could leverage in your CI. What's missing is selecting the cost to print - right now afaik it will always print number of allocations, but you are probably more interested in leaks. Or maybe even in multiple cost sources.
Comment 4 Milian Wolff 2018-03-21 17:19:49 UTC
Note that flamegraph cost type can now be selected, so potentially this is all you need?
Comment 5 Andrew Somerville 2018-03-21 17:38:24 UTC
I'm not familiar with the flamegraph output format in enough detail, but at first glance I think it looks like it still requires pre-processing to get the relevant data back into an easily queryable data structure.

I'll look at this in more depth this weekend.

I also spent some time this last weekend seeing if adding json would be easy and didn't get very far. Going to try again this weekend as long as other tasks don't get in the way.

In any case, I'll try to keep this ticket up to date with any progress I make, or if if we settle on using one of the current output formats.
Comment 6 Justin Zobel 2022-11-18 04:30:23 UTC
Thank you for reporting this issue in KDE software. As it has been a while since this issue was reported, can we please ask you to see if you can reproduce the issue with a recent software version?

If you can reproduce the issue, please change the status to "REPORTED" when replying. Thank you!
Comment 7 Bug Janitor Service 2022-12-03 05:17:37 UTC
Dear Bug Submitter,

This bug has been in NEEDSINFO status with no change for at least
15 days. Please provide the requested information as soon as
possible and set the bug status as REPORTED. Due to regular bug
tracker maintenance, if the bug is still in NEEDSINFO status with
no change in 30 days the bug will be closed as RESOLVED > WORKSFORME
due to lack of needed information.

For more information about our bug triaging procedures please read the
wiki located here:
https://community.kde.org/Guidelines_and_HOWTOs/Bug_triaging

If you have already provided the requested information, please
mark the bug as REPORTED so that the KDE team knows that the bug is
ready to be confirmed.

Thank you for helping us make KDE software even better for everyone!
Comment 8 Bug Janitor Service 2022-12-18 05:15:41 UTC
This bug has been in NEEDSINFO status with no change for at least
30 days. The bug is now closed as RESOLVED > WORKSFORME
due to lack of needed information.

For more information about our bug triaging procedures please read the
wiki located here:
https://community.kde.org/Guidelines_and_HOWTOs/Bug_triaging

Thank you for helping us make KDE software even better for everyone!