@dkorchevgithub adding lines suggested by @EtoDemerzel0427 to fix "g… - #13
Open
dkorchevgithub wants to merge 796 commits into
Open
@dkorchevgithub adding lines suggested by @EtoDemerzel0427 to fix "g…#13dkorchevgithub wants to merge 796 commits into
dkorchevgithub wants to merge 796 commits into
Conversation
* Update submission_checker.py DLRM dataset size is 330067, not 204800 (a setting for perf measurement)
* Fix Typo in Interactive Latencies * Update submission_checker.py --------- Co-authored-by: Arjun Suresh <arjun@gateoverflow.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
* Add model path to reference * Update readme instructions * Add LibriSpeech attribution
Co-authored-by: Miro <mirhodak@amd.com>
* Update compliance table * Add config file for whisper TEST01
* Update verify_performance.py Fixing mismatch in function header: result_log -> parse_result_log * Update verify_performance.py * Update run_verification.py * Update verify_performance.py * Update submission_checker.py DLRM dataset size is 330067, not 204800 (a setting for perf measurement) * Update mlperf.conf Adding sample_concatinate_permutation for Whisper model * Update submission_checker.py --------- Co-authored-by: Arjun Suresh <arjun@gateoverflow.com>
* [Automated Commit] Format Codebase * Update publish.yaml * Update mkdocs.yml * Update index.md --------- Co-authored-by: github-actions <github-actions@github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Add FAQ issue from inference_policies repo. Co-authored-by: hanyunfan <frank.han@dell.com>
…2272) * Add interactive in the TEST06 * Bump version
Resolving error due to change in vllm main branch.
* Update MLCFlow commands for v5.1 (#2237) * updating for 5.1-dev (inference doc)
…nt logging (#2278) Co-authored-by: Arjun Suresh <arjun@gateoverflow.com>
* Add v6.1 Interactive scenario support and fix report generator - constants.py: add Interactive→result_completed_samples_per_sec to RESULT_FIELD_NEW["v6.1"] so performance_check.py doesn't KeyError on Loadgen++ (endpoints) Interactive submissions - generate_final_report.py: split the shared v6.0/v6.1 filter_scenarios into separate per-version blocks; add e2e/e2e_vectorDB to the v6.1 datacenter block (Offline only) and to columns_order; remove v6.0-only models (mixtral-8x7b, llama3.1-405b, pointpainting) from the v6.1 block; add explicit edge exclusions for all v6.1 datacenter-only models to prevent datacenter-edge system results leaking into the edge worksheet Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: normalize endpoints online scenario to server in loader For endpoints submissions, the directory is named "online" but the effective scenario is "server". Centralizing the online→server transformation in loader.py ensures all downstream consumers (accuracy_check, power_check, results exporter) see the correct scenario without needing per-class transformations. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * [Automated Commit] Format Codebase --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> Co-authored-by: Pablo Gonzalez <pgmpablo97@gmail.com> Co-authored-by: mlc-automations <3246381+mlc-automations@users.noreply.github.com>
Replace the 5 random sample entries in samples_filename_ids.txt with IDs from tools/random_ids.txt (123, 43, 22, 238, 32). Fixed IDs from tools/fixed_idx.txt (130, 106, 84, 59, 12) remain unchanged.
…ration dataset reference (#2635) Signed-off-by: Palanivelg <12385994+Palanivelg@users.noreply.github.com> Co-authored-by: Palanivelg <12385994+Palanivelg@users.noreply.github.com>
* Update folder structure for endpoints submissions * Remove unused code * Revert "Remove unused code" This reverts commit 7d8ae1a. * fix: skip truncation for endpoints submissions instead of targeting stale results.json (#2631) The endpoints accuracy artifact was renamed from results.json (with a responses field) to accuracy/accuracy_results.json in mlcommons/endpoints#400, which also dropped the responses field entirely (per-sample text now lives in events.jsonl, not part of the submission). truncate_accuracy_log.py still looked for the old results.json/responses shape and would only log "missing" for every endpoints submission. Detect the endpoints layout via the scenario-root config.yaml and skip truncation, and remove the now-dead PERFORMANCE_ENDPOINTS_DIR/ACCURACY_ENDPOINTS_DIR constants left over from the loader.py refactor to ENDPOINTS_SCENARIO_DIR. Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com> * Add compliance checking for endpoints submission (#2633) * Quick fixes for endpoints parser * Add qwen3.6-27b benchmark to submission checker (#2634) * Additional quick fixes * Parse endpoint and performance accuracy separately * Fix endpoints submission checker latency parsing (#2636) * [Automated Commit] Format Codebase * Fix submission directory structure for endpoints (#2637) * Fix submission directory structure for endpoints * Fix filename in endpoints submission structure Corrected the filename from 'results_summary.json' to 'result_summary.json' in the endpoints submission structure documentation. --------- Co-authored-by: Pablo Gonzalez <pgmpablo97@gmail.com> --------- Co-authored-by: ANANDHU S <71482562+anandhu-eng@users.noreply.github.com> Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com> Co-authored-by: zihaok <161090975+zihaok@users.noreply.github.com> Co-authored-by: mlc-automations <3246381+mlc-automations@users.noreply.github.com>
) The README pointed at deepseek-r1-0528.uri, which resolves to DeepSeek-R1-0528 rather than the intended deepseek-ai/DeepSeek-R1 (revision 56d4cbb) referenced above it. The R2 bucket now hosts the correct checkpoint under deepseek-r1.uri; verified via full download and md5sum comparison against the reference HF checkpoint (173/173 files match, zero mismatches). Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com> Co-authored-by: hanyunfan <frank.han@dell.com>
* e2e-rag: Improve JSON parsing robustness and URL extraction
This commit improves the reliability of the multi-shot retrieval system:
1. **Robust JSON Parsing** (multi_shot_retrieval.py, accuracy_eval.py):
- Use regex-based extraction to handle JSON in markdown code blocks
- Extract JSON objects from mixed text/markdown LLM responses
- Add fallback behavior when JSON parsing fails instead of crashing
- Applied to: evaluate_document_relevance(), check_sufficiency(),
generate_search_queries(), query_rewriter(), call_judge()
2. **URL Extraction Fixes** (multi_shot_retrieval.py):
- Support both 'original_url' and 'source' metadata fields
- Convert Wikipedia filenames (en.wikipedia.org_wiki_*.html) to URLs
- Debug logging for URL extraction issues
3. **Metrics Compatibility** (multi_shot_retrieval.py):
- Add backward-compatible aliases (precision, recall, f1)
- SUT expects different key names than evaluation module produces
- Added 'retrieved_urls' to metrics output
4. **Config Updates** (user.conf):
- Added e2e-datasetup.Offline configuration
These changes make the system more resilient to LLM output variations
without changing the underlying retrieval logic or model assignments.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* Create download_additional_urls.py
Download additional 470 URLs files.
* model name update to match the path and folder names that come with mlc storage download
* fixed the judge model back to meta-llama/Llama-3.1-8B-Instruct
* Fix reranker output silently ignored in multi-shot
* Update the default judge to Llama3.1-8B
* Update judge prompt for robustness
* Correct judge service port across the repo
* Rename e2e-rag workload components to e2e-rag-qna / e2e-rag-db
Align naming end-to-end across the workload and submission tooling:
- checker constants.py: e2e -> e2e-rag-qna, e2e_vectorDB -> e2e-rag-db
- user.conf loadgen keys and FromConfig namespaces
- reference-script output dirs (nested accuracy/performance to avoid
the run_output/output collision between QnA accuracy and perf runs)
- datasetup scripts: patch e2e-rag-db.Offline.* (was dead e2e-datasetup.*)
- compliance/TEST09/e2e-rag -> e2e-rag-qna (+ README, run_compliance paths)
- fix stale "E2E DocGrader" header comments
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Add DB manifest verification to RAG-DB accuracy test
The RAG-DB accuracy test only did local self-consistency checks and
never verified the built vector DB against the reference manifest, so a
divergent corpus/embedding could still pass.
- db_manifest.py: extract cmd_verify body into reusable verify_manifest()
that returns {passed, failures, metrics} instead of sys.exit; add a
--retriever_model override (manifest's stored path is a system-specific
absolute path that breaks verify on other systems) with fallback to the
manifest value.
- datasetup_accuracy_eval.py: run verify_manifest() when --manifest is
given; record results in accuracy_results["manifest"], gate overall pass
on it, and write a manifest section into accuracy.txt. Skipped cleanly
when no manifest is provided.
- reference_mlperf_datasetup_accuracy.sh: pass the bundled reference
manifest (scripts/db_manifest_intel_xpu.json.gz) by default.
- scripts/verify_db_manifest.sh: pass local retriever model to verify.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* RAG-QnA: write LLM judge score to accuracy.txt
The RAG-QnA accuracy run produced only accuracy_results.json; the
submission checker needs an accuracy.txt with an "Accuracy:" line.
- accuracy_eval.py: write accuracy.txt into the loadgen log dir with the
LLM judge answer accuracy as a percentage (answer_accuracy * 100),
matching the checker's E2E_ACCURACY pattern. hash= line and truncation
are added later by truncate_accuracy_log.py during submission prep.
- reference_mlperf_{accuracy,perf}.sh: fix stale user.conf sed patch that
still targeted the old e2e.Offline key (now e2e-rag-qna.Offline).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Enable TEST09 compliance check for e2e-rag-qna
The submission checker gates compliance validation on the models_TESTxx
lists; e2e-rag-qna was in none, so TEST09 results were silently unchecked
despite the workload shipping run_compliance_test09.sh and producing
verify_output_len.txt.
Add e2e-rag-qna to models_TEST09 (v6.1) so the checker requires
TEST09/verify_output_len.txt and validates it contains "TEST PASS".
e2e-rag-db has no compliance test and is left out.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* e2e-rag: robustness fixes for submission-checker compatibility
Workload-side fixes validated end-to-end via a 10-doc/10-query smoke run
through OpenRouter in a container:
- run_compliance_test09.sh: pass --audit_conf to reference_mlperf.py so
loadgen actually applies the TEST09 audit.config (previously the default
audit.conf was used, leaving accuracy_log_sampling_target=0 and an empty
mlperf_log_accuracy.json -> verification found no entries). Make PERF_COUNT
overridable and sync the audit.config min_query_count to it. Source the
TEST09 config from the main repo's compliance tree (copied in / overridable
via COMPLIANCE_DIR).
- reference_mlperf_{perf,accuracy}.sh: capture and propagate the python exit
code (was masked by a trailing echo, so failures looked like success).
- datasetup_accuracy_eval.py: write a checker-compliant "Accuracy:" line
(indexing success rate) and always emit the DB Manifest Verification
section (PASS/FAIL/SKIPPED) in accuracy.txt.
- reference_mlperf_datasetup_accuracy.sh: honor an explicit empty/"none"
MANIFEST to skip the manifest check (use ${VAR-default} so empty is
respected).
- user.conf: e2e-rag-db counts 2503->2515; drop dead e2e-datasetup.* keys.
- .gitignore: ignore run_output_*/output/submission smoke artifacts.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Remove submission_checker/constants.py change from workload branch
The e2e-rag rename originally also edited the submission checker's
constants.py, but all checker changes now live in the dedicated
e2e-rag-submission-checker-fixes branch/PR. Revert constants.py to
upstream here so the workload PR touches no checker files and does not
conflict with the checker PR.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
Co-authored-by: rpoornac <rajesh.poornachandran@amd.com>
Co-authored-by: Saehanseul <saehanseul.yi@intel.com>
Add first-class submission-checker support for the two e2e-rag workload components (RAG-QnA and the vector-DB setup): - constants.py: rename e2e -> e2e-rag-qna and e2e_vectorDB -> e2e-rag-db across the v6.1 config; set real accuracy targets (E2E_ACCURACY 33.95 for qna, 98 for db) replacing the empty-string placeholder that raised a TypeError in accuracy_result_check; set Offline min-queries to 1 (loadgen reports effective_min_query_count=1 for Offline; the real count is enforced via performance-sample-count and dataset-size); size db by its 2515-document corpus (performance-sample-count, dataset-size, OFFLINE_MIN_SPQ_SINCE_V4) while qna stays 824; add e2e-rag-qna to models_TEST09 so TEST09 compliance is validated. - performance_check.py: exempt e2e-rag-qna/e2e-rag-db from the 600s minimum-duration rule (fixed single pass over the dataset, not a time-bounded loop). - generate_final_report.py: use the new model names in the report's model list and v6.1 datacenter/edge scenario maps. Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> Co-authored-by: Pablo Gonzalez <pgmpablo97@gmail.com>
* Add pre-commit setup documentation * [Automated Commit] Format Codebase * [Automated Commit] Format Codebase --------- Co-authored-by: hanyunfan <frank.han@dell.com> Co-authored-by: mlc-automations <3246381+mlc-automations@users.noreply.github.com> Co-authored-by: Arav Agarwal <43820556+arav-agarwal2@users.noreply.github.com>
* [Automated Commit] Format Codebase * scanoss.json implementation * Update SCANOSS workflow to trigger only on pull requests Removed push event trigger for all branches. * Remove only joint package. * Update scanoss.yml * Update * [Automated Commit] Format Codebase * Trying to pin scanoss version to osskb * Update SCANOSS License & BOM Check: approval-gated, changed-files scan - Trigger on PR approval (pull_request_review) instead of every push - Scan only changed files (--files-from), whole-tree on manual dispatch - CycloneDX BOM via convert (no second scan) - scanoss.json: BOM include/exclude + skip patterns (tests/vendor/generated) - Pin scans to api.osskb.org * Add changes to make reports more useful and scanning on master / main changes * Moved files for cleanliness --------- Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: mlc-automations <3246381+mlc-automations@users.noreply.github.com> Co-authored-by: tanvi-mlcommons <tanvigour@mlcommons.org>
* Initial test structure * TEST ONLY codecov impl * [Automated Commit] Format Codebase * Remove bad file * test * Add __init__.py to files to ensure full coverage knowledge * Add snapshot testing * [Automated Commit] Format Codebase * Add codecov * Remove extranneous files for this version of coverage --------- Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: mlc-automations <3246381+mlc-automations@users.noreply.github.com> Co-authored-by: Pablo Gonzalez <pgmpablo97@gmail.com>
… the repo. (#2643) * Add autopep8 format check workflow This workflow checks that modified Python files in pull requests are formatted according to autopep8 standards. It is set as a required status check to prevent unformatted code from being merged. * Delete .github/workflows/format.yml --------- Co-authored-by: hanyunfan <frank.han@dell.com>
* e2e-rag: gate DB manifest on retrieval reproducibility; emit real token count Manifest verification (Item 1): different implementations chunk and index the corpus at different times, so passage counts, corpus sha256, and per-index sample-embedding cosine legitimately differ across correct submissions. Gate verify_manifest() solely on mean probe-query top-K document-URL overlap (retrieval_accuracy >= 0.999), with URLs normalized to a corpus-stable key and compared as sets. Passage count, sha256, and cosine are now informational metrics that never fail; only embedding_dim mismatch (wrong retriever model) still fails. Widen the probe set to 100 queries so the 99.9% threshold is statistically meaningful. Thread --retrieval_threshold through datasetup_accuracy_eval.py. TEST09 token count (Item 3): the SUT logged the answer's UTF-8 byte length as n_tokens, but run_verification.py decodes response data as int32 token IDs and counts them. Emit an int32 blob whose length equals the real answer-generation token count (LLM usage.completion_tokens, exposed via LLMLogger.get_component_output_tokens), captured before end_query() clears the per-query state; pass that count as n_tokens. The human-readable answer stays in results.json for auditing. Update the TEST09 threshold to the real reference mean OSL of 273.81 (n=810 from logs_result.json) -> [246.43, 301.19]. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * e2e-rag: mandate frozen corpus; drop URL normalization from manifest gate Root cause of cross-system manifest failures is corpus sourcing, not string formatting. Verified against the official download: docs.tar.gz ships a FROZEN Wikipedia snapshot (2515 files) with a fixed url_mapping.json, but download_docs.py re-scrapes live Wikipedia and fetches a newer revision (e.g. 100_metres: 1360491832 vs 1364600010) -- identical filenames, different bytes, different passage counts, and genuinely different retrieval results. Any submission that ingests the official corpus stores the identical original_url per document, so retrieved URLs match verbatim. - download_dataset_and_models.sh: extract docs.tar.gz into doc_html/ and copy url_mapping.json; drop the download_docs.py re-scrape from the next-steps and warn against it. - README: document that the corpus is frozen and download_docs.py is for snapshot regeneration only, not submissions. - db_manifest.py: compare probe-query top-K URLs verbatim as sets (remove _normalize_url and the now-unused re import). With a shared frozen corpus, normalization is unnecessary and its #anchor stripping would mask real section-level retrieval divergence the gate should catch. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * e2e-rag: drop corpus sha256 comparison from manifest verify The docstore sha256 depends on passage ordering, which varies with ingestion thread timing across systems, so it can never match even for identical source documents. It was already informational (never failed the check); remove the comparison entirely so it no longer appears as a confusing MISMATCH diagnostic. Retrieval overlap remains the sole gate. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * e2e-rag: record original_url in datasetup passage metadata The DB manifest gate compares canonical Wikipedia URLs across systems, but datasetup passages only carried the filename-form source. Load the frozen corpus url_mapping.json and stamp each passage's metadata with its original_url so cross-system manifest verification can match documents. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * new vector db manifest reference created with 2515 files * update db manifest top-k overlap threshold to 0.95 * Correct the DB verification default values in datasetup_accuracy_eval --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> Co-authored-by: Saehanseul <saehanseul.yi@intel.com> Co-authored-by: hanyunfan <frank.han@dell.com>
The text_to_image tooling downloaded COCO 2014 annotations over plain HTTP with no integrity check, making benchmark calibration vulnerable to a tampered or truncated archive. Upgrade the annotations URL to the S3 origin (images.cocodataset.org's CDN has a broken HTTPS cert - the S3 path-style URL serves the same bytes under a valid certificate) and pin the expected SHA-256. Mismatch raises before unzip so a bad download cannot poison downstream benchmark state. Also rewrites the per-image URLs in the annotations JSON so urllib.request.urlretrieve downloads are TLS-authenticated. Addresses the HTTPS + checksum bullet of #2502. Signed-off-by: Kayra Arai Ozturk <kayraaraio@gmail.com> Co-authored-by: hanyunfan <frank.han@dell.com>
Co-authored-by: hanyunfan <frank.han@dell.com>
Co-authored-by: hanyunfan <frank.han@dell.com>
…rom endpoints (#2652) Co-authored-by: Pablo Gonzalez <pgmpablo97@gmail.com>
Remove llama2-70b-99 from the v6.1 config in submission_checker_old.py. #2651 removed llama2-70b-99 from the v6.1 block of the modular checker's constants.py but left the legacy submission_checker_old.py untouched, so the two configs disagreed about whether the model exists in v6.1. preprocess_submission.py imports submission_checker_old, and its infer_low_accuracy_results pass copies a -99.9 result to -99 whenever the low accuracy variant appears in config.required. Against the legacy v6.1 config that condition still held, so preprocessing recreated results/<system>/llama2-70b-99 from llama2-70b-99.9. The modular checker then ran over that directory, got None back from Config.get_required(), and died in lower_list() with "TypeError: 'NoneType' object is not iterable" - failing every round 6.1 closed submission generation CI job. This mirrors the #2651 deletions site for site; v6.0 and v5.1 keep the model. Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> Co-authored-by: hanyunfan <frank.han@dell.com>
* [Automated Commit] Format Codebase * Update publish.yaml * Update mkdocs.yml * Update index.md * [Automated Commit] Format Codebase * [Automated Commit] Format Codebase * [Automated Commit] Format Codebase * [Automated Commit] Format Codebase * [Automated Commit] Format Codebase * [Automated Commit] Format Codebase * [Automated Commit] Format Codebase * [Automated Commit] Format Codebase * [Automated Commit] Format Codebase * [Automated Commit] Format Codebase * [Automated Commit] Format Codebase * [Automated Commit] Format Codebase * [Automated Commit] Format Codebase * [Automated Commit] Format Codebase * [Automated Commit] Format Codebase * [Automated Commit] Format Codebase * [Automated Commit] Format Codebase * [Automated Commit] Format Codebase * [Automated Commit] Format Codebase * [Automated Commit] Format Codebase * [Automated Commit] Format Codebase * [Automated Commit] Format Codebase * [Automated Commit] Format Codebase * [Automated Commit] Format Codebase * [Automated Commit] Format Codebase * [Automated Commit] Format Codebase * [Automated Commit] Format Codebase * [Automated Commit] Format Codebase * [Automated Commit] Format Codebase * [Automated Commit] Format Codebase * [Automated Commit] Format Codebase * [Automated Commit] Format Codebase * [Automated Commit] Format Codebase * [Automated Commit] Format Codebase * [Automated Commit] Format Codebase * [Automated Commit] Format Codebase * [Automated Commit] Format Codebase * Add system info collection documentation for MLPerf Inference Adds docs/system-info/index.md covering the get-mlperf-multi-node-system-info script for MLPerf Inference submissions: installation, quick start, config file usage, serving framework detection, sample output, and a verified field reference table aligned with SYSTEM_DESC_REQUIRED_FIELDS. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * docs: add v6.1 scope note to system-info page Clarifies that power and network mode fields are scoped for future rounds and must be filled in manually before submission. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * update benchmark run docs for v6.1 * Update README.md * Update main.py * Update main.py * fix directory structure info * Document optional nameplate power YAML in system-info guide Adds a section covering the new _redfish and _inference_optional_nameplate variations of get-mlperf-multi-node-system-info: the skeleton template written when used alone, and real PSU data populated from a live Redfish BMC when stacked with _redfish, including the Redfish-to-nameplate field mapping and required CLI parameters. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * Update main.py * Update README.md * Resolve docs/master conflicts in e2e-rag Take master's versions of the three files that conflict when merging docs into master: - e2e-rag/db_manifest.py - e2e-rag/datasetup_accuracy_eval.py - e2e-rag/reference_SUT_datasetup.py The conflicts came from master's functional rewrites (#2645, #2625) landing on top of the automated "Format Codebase" reflow of the older code on docs. Docs never made a manual edit to these files, so master's content is authoritative and nothing is lost. The files are left byte-identical to master rather than reformatted: autopep8 on the docs side rewrites the same line ranges master did, which reintroduces the conflicts. Formatting can follow once both branches agree on content. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * [Automated Commit] Format Codebase * Delete .github/workflows/format.yml * Revert "[Automated Commit] Format Codebase" This reverts commit 7c17261 for e2e-rag/db_manifest.py. The format.yml workflow reformatted db_manifest.py right after 765c476 made it byte-identical to master, which reintroduced the merge conflict against master: autopep8 on the docs side rewrites the same line ranges master's functional changes did. Now that format.yml is removed from this branch (297a81d), the file can stay byte-identical to master and docs merges into master cleanly. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * Drop autopep8 churn in e2e-rag: mirror master exactly The docs branch carried autopep8 reformatting of e2e-rag Python files that master never had, which showed up as pure formatting noise in the docs -> master PR diff. Reset these files to master's exact bytes. Verified at the token level (ignoring whitespace and comments) that none of them contains any docs-authored code: every difference was either autopep8 churn from the now-removed format.yml workflow, or docs lagging master. The autopep8 artifacts included reflowed call sites, trailing whitespace stripped inside docstrings, reordered imports, and one long comment split in a way that injected a stray '#'. main.py is deliberately left alone: it carries docs-only commits. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * Sync e2e-rag with master Copy the entire e2e-rag directory from master so the docs branch carries no divergence there at all. The remaining files were docs lagging master: shell scripts, user.conf, .gitignore, README.md and the scripts/ helpers. e2e-rag on docs is now byte-identical to master. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * Update main.py * Update main.py --------- Co-authored-by: github-actions <github-actions@github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: Arjun Suresh <arjun@gateoverflow.com> Co-authored-by: mlc-automations <3246381+mlc-automations@users.noreply.github.com> Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
20 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
…ood" count (mlcommons#965)
patch for the latest dlrm
updated Docker CPU as suggested in
Issues:
mlcommons#917
mlcommons#604
updated for issues DLRM inference README out of date mlcommons/inference#917 and DLRM: save downloaded and generated files outside of Docker containers mlcommons/inference#604
Update README.md
updating readme
fixed typo noticed by @psyhtest
adding lines suggested by @EtoDemerzel0427 to fix "good" count
Co-authored-by: Anton Lokhmotov psyhtest@users.noreply.github.com
Co-authored-by: rameshchukka rnaidu02@yahoo.com