WIP: Rust score calculation

[nicoburns]

• Work towards fixing Rewrite score calculation in Rust #25

Actual score calculation is implemented in the wptreport crate:

• https://crates.io/crates/wptreport
• https://github.com/nicoburns/wpt-rs

This PR adds a CI job to run the CLI from the crate on Github CI here with the purpose of seeing how fast the code runs on GIthub Actions.

Todo:

• Implement "ingesting a new report"
  • Merging WPT report fragments
  • Converting WPT report to "Servo Test Scores" format (stored in this repo)
  • Add a CLI command
  • xz compression
  • Add to CI job
• Make focus areas configurable
• Make output location configurable
• Output scores-last-run.json (or remove)
• Pass "run_info" through unchanged (unsorted)?
  (current implementation uses a Rust struct to store "run_info" which leads to the key sort order changing - we should probably use an IndexMap?)
• Use cargo-binstall to eliminate wpt cli installation time
• Use in nightly CI job

[nicoburns]

Ok: this PR is doing score recalculation in 2m 49s vs. 27m 8s on main. That's a pretty nice improvement!

[mukilan]

The numbers look really promising! I've given the crate only a cursory read through - it seems like if the goal is to use this for Servo's WPT dashboard, the pending items are

• A way to recalculate past runs by scoring them against a given run.
• Support regex based areas to support /css/CSS2/tables & /css/css-tables. Doesn't have to be regex, I guess, we just need a way to combine arbitrary folders.
• Few unit tests for confidence. Maybe just generating scores.json against a fixed dataset using the crate and diffing it against the current one would also work.

[nicoburns]

The numbers look really promising! I've given the crate only a cursory read through - it seems like if the goal is to use this for Servo's WPT dashboard, the pending items are

Some up to date numbers: this PR is now calculating in 4m 18s vs 35m 47s for the existing JS version.

• A way to recalculate past runs by scoring them against a given run.

This is now implemented. Unfortunately it is a little slower, but it's not too bad.

• Support regex based areas to support /css/CSS2/tables & /css/css-tables. Doesn't have to be regex, I guess, we just need a way to combine arbitrary folders.

This is implemented as an array of areas to support. Any exact prefix of segment (separated by a /) is supported (/css/, /css/CSS2 and /css/CSS2/tables will all work, but /css/CSS2/tab would not). Currently this is hardcoded into the CLI, but I will probably make it load from the JSON file.

• Few unit tests for confidence. Maybe just generating scores.json against a fixed dataset using the crate and diffing it against the current one would also work.

I have generating identical output to #67 modulo float precision (in the last 2 decimal places of numbers with ~8 decimal places - so I expect this is fine?). This PR is generating a scores.json file and uploading it as an artefact if anyone wishes to compare

[nicoburns]

Ok: this PR is doing score recalculation in 2m 49s vs. 27m 8s on main. That's a pretty nice improvement!

This is getting more important over time. The timings are now 5m (this PR) vs 50m (main).
(presumably this is just because there is more data to process now)

Will try to finish this up soon.