Skip to content

Add METEOR score to the text package - #3477

Open
ChrisJr404 wants to merge 1 commit into
Lightning-AI:masterfrom
ChrisJr404:add-meteor-score
Open

Add METEOR score to the text package#3477
ChrisJr404 wants to merge 1 commit into
Lightning-AI:masterfrom
ChrisJr404:add-meteor-score

Conversation

@ChrisJr404

Copy link
Copy Markdown

What does this PR do?

Fixes #595

Adds a METEOR metric to the text package, both the functional meteor_score and the modular METEORScore. It follows the sequential exact, stemmed and WordNet synonym alignment from the Lavie and Agarwal paper, scores each hypothesis against the best matching reference, and averages over the corpus. The nltk stemmer and WordNet corpus are reused (nltk is already a text dependency), so the results line up with nltk.translate.meteor_score, which is what the tests compare against.

alpha, beta and gamma are exposed with the usual defaults (0.9, 3.0, 0.5), and there's a return_sentence_level_score option like chrF and TER have.

Before submitting
  • Was this discussed/agreed via a Github issue? (no need for typos and docs improvements)
  • Did you read the contributor guideline, Pull Request section?
  • Did you make sure to update the docs?
  • Did you write any new necessary tests?

Did you have fun?

Always do 🙂

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add METEOR metric

1 participant