Skip to content

tests: add unit tests for GSM8K dataset - #2613

Open
moonturbo wants to merge 1 commit into
open-compass:mainfrom
moonturbo:tests/add-gsm8k-unit-tests
Open

tests: add unit tests for GSM8K dataset#2613
moonturbo wants to merge 1 commit into
open-compass:mainfrom
moonturbo:tests/add-gsm8k-unit-tests

Conversation

@moonturbo

Copy link
Copy Markdown

Motivation

GSM8K is one of the most widely used math benchmarks, but it currently has no dedicated unit tests for its postprocessors or evaluator, unlike other common datasets (e.g. humaneval, beyondaime). This PR adds coverage to lock in the text-extraction and scoring behavior.

Modification

  • Add tests/datasets/test_gsm8k.py covering:
    • gsm8k_postprocess: number extraction, negative numbers, last-number preference, NULL fallback
    • gsm8k_dataset_postprocess: answer separator parsing and thousands-separator removal
    • Gsm8kEvaluator.score: exact match, float/int tolerance, partial accuracy, length-mismatch error
    • Gsm8kEvaluator.is_equal: non-numeric / None handling

Verification

  • All 17 tests pass locally against the current gsm8k.py implementation.
  • The module is loaded via importlib (same pattern as tests/datasets/test_beyondaime.py) to avoid pulling in heavy dependencies.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant