PerfFlowAspect Reader#101
Closed
r-yin wants to merge 17 commits into
Closed
Conversation
slabasan
added
area-readers
- reorganize data files - restore ts and dur into frame
Member
spencer-gre
added a commit
to spencer-gre/hatchet
that referenced
this pull request
Jul 25, 2024
This commit follows remarks that llnl#144 will supersede llnl#101, and specifically prepares for the integration of both an array reader for PerfFlowAspect Array Trace Files (already present) and PerfFlowAspect Object Trace Files (to be added). Additionally removes .json files in the pytest data directory as no longer needed.
slabasan
added a commit
that referenced
this pull request
Jul 25, 2026
* perf_reader initial commit * graph updated * add front-level reader API, add PFA reader example - add node object to dataframe * graph updated * added pfw testfile with one iteration of laghos output * pfw file processor added * pfw to literal converter * sample json * pfw folder processor * finalized pfw to caliper reader * cleaning up code * update example for pfa reader * add perfflowaspect unit test - reorganize data files - restore ts and dur into frame * add AMS MPI pfa unit test (concat pfw files per rank), flake, black * restore kwargs in from_hdf * update comment * read cpu/mem usage * updates to pfa reader, prepare tests * forgot this test file * tests for cpu-mem * fix tests * Read All PFW Files The PerfFlowAspect Reader used to require that .pfw files (PerfFlowAspect Traces) conform to JSON standard. However, PerfFlowAspect does not generate files that are consistent with JSON. This commit modifies a PerfFlowAspect trace file that does not conform to JSON standard - by adding the closing brace and removing the trailing comma. * Cleanup/Prep for Object Reader This commit follows remarks that #144 will supersede #101, and specifically prepares for the integration of both an array reader for PerfFlowAspect Array Trace Files (already present) and PerfFlowAspect Object Trace Files (to be added). Additionally removes .json files in the pytest data directory as no longer needed. * Removing PFW JSON Files Missed two in the prior commit. Refer there for full explanation. * perf_reader initial commit * graph updated * graph updated * added pfw testfile with one iteration of laghos output * pfw file processor added * pfw to literal converter * sample json * pfw folder processor * finalized pfw to caliper reader * cleaning up code * add perfflowaspect unit test - reorganize data files - restore ts and dur into frame * add AMS MPI pfa unit test (concat pfw files per rank), flake, black * Add perfflowaspect object reader and update related tests and scripts * updated the test file. add extra fields for the time unit and matedata. Renamed and moved pfw test file to the correct folder * added us to s converter. fixed object reader tests * Cleanup after merge This branch cleans up conflicts/redundancies introduced after object reader added. Specifically, JSON files were reintroduced, and references to the original perfflowaspect_reader was being used in the graphframe (and not perfflowaspect_array_reader) * code format * pushed * nearly working single pfa reader * wrapping up pfa reader * remove changes to install * clean up and patch hatchet reader * update tests * restore hpctoolkit and timemory readers * fix license issues * flake and black format --------- Co-authored-by: r-yin <ryin2@bu.edu> Co-authored-by: Spencer Greene <ssgreene@wpi.edu> Co-authored-by: loudsun1997 <zhiwei.yang1997@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
adding PerfFlowAspect Reader with some example files
https://github.com/flux-framework/PerfFlowAspect