Skip to content

feat: upgrade to transformers v5 - #659

Merged
dushyantbehl merged 21 commits into
foundation-model-stack:mainfrom
yash4242:yash-tfv5
Mar 9, 2026
Merged

feat: upgrade to transformers v5#659
dushyantbehl merged 21 commits into
foundation-model-stack:mainfrom
yash4242:yash-tfv5

Conversation

@yash4242

@yash4242 yash4242 commented Feb 24, 2026 •
edited by dushyantbehl
Loading

Copy link
Copy Markdown
Contributor

Description of the change

Making fms-hf-tuning transformers v5 compatible

Related issue number

NA

How to verify the PR

run tests. As of now, some tests fail, most of which are due to a tensor size mismatch. This error comes inside the trainer.train() code

Was the PR tested

Yes.

  • I have added >=1 unit test(s) for every new method I have added.
  • I have ensured all unit tests pass
    Tests status as above. I tested out with 9 pytests as follows:
pytest tests/acceleration
pytest tests/build
pytest tests/data
pytest tests/trackers
pytest tests/trainercontroller
pytest tests/utils
pytest tests/test_sft_trainer.py

@github-actions

github-actions Bot commented Feb 24, 2026

Copy link
Copy Markdown

Thanks for making a pull request! 😃
One of the maintainers will review and advise on the next steps.

@dushyantbehl dushyantbehl changed the title Yash tfv5 feat: upgrade to transformers v5 Feb 24, 2026
@github-actions github-actions Bot added the feat label Feb 24, 2026
@yash4242
accommodate renaming of AutoModelForVision2Seq to AutoModelForImageTe…
ee02546 
…xtToText

Signed-off-by: Yash Mehan <yashmehan@gmail.com>
@yash4242
accomodate renaming of include_tokens_per_second to include_num_input…
6b797d2 
…_tokens_seen

Signed-off-by: Yash Mehan <yashmehan@gmail.com>
@yash4242
accommodate mllama model.language_model issue and gptneoxtokenizer an…
b749727 
…d llama tokenizer

Signed-off-by: Yash Mehan <yashmehan@gmail.com>
@yash4242
replace addition_special_tokens with extra_special_tokens
a3bc638 
Signed-off-by: Yash Mehan <yashmehan@gmail.com>
dushyantbehl
dushyantbehl reviewed Feb 24, 2026
View reviewed changes
Comment thread tests/build/test_launch_script.py Outdated
dushyantbehl
dushyantbehl reviewed Feb 24, 2026
View reviewed changes
Comment thread tests/utils/test_embedding_resize.py
dushyantbehl and others added 7 commits February 24, 2026 22:07
@dushyantbehl
Merge branch 'main' into yash-tfv5
f823261
@yash4242
changed library versions in pyproject.toml
28605d4 
Signed-off-by: Yash Mehan <yashmehan@gmail.com>
@yash4242
fix pyproject toml after the rebase
ba6ffb0 
Signed-off-by: Yash Mehan <yashmehan@gmail.com>
@yash4242
add more handing for the filenotfound exception handling in multi ran…
6c28f16 
…k case

Signed-off-by: Yash Mehan <yashmehan@gmail.com>
@yash4242
merge existing extra_special_tokens with new ones to prevent replacement
800b288 
Signed-off-by: Yash Mehan <yashmehan@gmail.com>
@yash4242
remove the old commented lines, only letting the new renamed lines st…
c7c9d89 
…ay, adding justification for what was renamed

Signed-off-by: Yash Mehan <yashmehan@gmail.com>
@yash4242
remove the old commented lines
d5e1862 
Signed-off-by: Yash Mehan <yashmehan@gmail.com>
@dushyantbehl
dushyantbehl marked this pull request as ready for review February 26, 2026 11:29
@yash4242
remove the old commented lines
407a43d 
Signed-off-by: Yash Mehan <yashmehan@gmail.com>
@yash4242
Merge branch 'main' into yash-tfv5
35033ec 
Signed-off-by: Yash Mehan <71321431+yash4242@users.noreply.github.com>
dushyantbehl
dushyantbehl reviewed Feb 26, 2026
View reviewed changes
Comment thread tests/utils/test_embedding_resize.py Outdated
dushyantbehl
dushyantbehl reviewed Feb 26, 2026
View reviewed changes
Comment thread tests/utils/test_embedding_resize.py Outdated
dushyantbehl
dushyantbehl reviewed Feb 26, 2026
View reviewed changes
Comment thread tests/test_sft_trainer.py Outdated
dushyantbehl
dushyantbehl reviewed Feb 26, 2026
View reviewed changes
Comment thread tests/test_sft_trainer.py Outdated
dushyantbehl
dushyantbehl reviewed Feb 26, 2026
View reviewed changes
Comment thread tuning/data/tokenizer_utils.py Outdated
dushyantbehl
dushyantbehl reviewed Feb 26, 2026
View reviewed changes
Comment thread pyproject.toml Outdated
dushyantbehl
dushyantbehl reviewed Feb 26, 2026
View reviewed changes
Comment thread pyproject.toml Outdated
dushyantbehl
dushyantbehl reviewed Feb 26, 2026
View reviewed changes
Comment thread pyproject.toml Outdated
dushyantbehl
dushyantbehl reviewed Feb 26, 2026
View reviewed changes
Comment thread pyproject.toml Outdated
dushyantbehl
dushyantbehl suggested changes Feb 26, 2026
View reviewed changes

@dushyantbehl dushyantbehl left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Delete commented code...retain comments only in code files and not in test files.
fix the pyproject file as per comments.

yash4242 and others added 4 commits February 27, 2026 05:15
@yash4242
complete housekeeping and removing old comments and repetitive comments
68ca44c 
Signed-off-by: Yash Mehan <yashmehan@gmail.com>
@yash4242
put upperbounds on dependencies in pyproject
3168e54 
Signed-off-by: Yash Mehan <yashmehan@gmail.com>
@yash4242
fix input_ids
c507131 
Signed-off-by: Yash Mehan <yashmehan@gmail.com>
@yash4242
Merge branch 'main' into yash-tfv5
eb7b243
dushyantbehl
dushyantbehl reviewed Feb 27, 2026
View reviewed changes
Comment thread pyproject.toml Outdated
dushyantbehl
dushyantbehl reviewed Feb 27, 2026
View reviewed changes
Comment thread pyproject.toml Outdated
dushyantbehl
dushyantbehl reviewed Feb 27, 2026
View reviewed changes
Comment thread pyproject.toml Outdated
dushyantbehl
dushyantbehl reviewed Feb 27, 2026
View reviewed changes
Comment thread pyproject.toml
dushyantbehl
dushyantbehl reviewed Feb 27, 2026
View reviewed changes

@dushyantbehl dushyantbehl left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good progress @yash4242
minor changes requested along with updates to the dockerfile to match the incoming changes.

@yash4242
linting and formatting
4f482f1 
Signed-off-by: Yash Mehan <yashmehan@gmail.com>
@dushyantbehl

dushyantbehl commented Mar 5, 2026

Copy link
Copy Markdown
Contributor

/build

@github-actions

github-actions Bot commented Mar 5, 2026

Copy link
Copy Markdown

Build failed for 4f482f1 (NVCR image)

View run

yash4242 added 3 commits March 6, 2026 06:51
@yash4242
remove multiple excepts
e14d2ce 
Signed-off-by: Yash Mehan <yashmehan@gmail.com>
@yash4242
add messages control to remove in pylintrc
fd3b977 
Signed-off-by: Yash Mehan <yashmehan@gmail.com>
@yash4242
upgrade support to trl 0.27+
c854702 
Signed-off-by: Yash Mehan <yashmehan@gmail.com>
@yash4242

yash4242 commented Mar 6, 2026

Copy link
Copy Markdown
Contributor Author

Please have a look at the PR. trl >=0.27 is supported, transformers >= 5.2 is supported. It should pass the lint check as well as the other checks.

@dushyantbehl

dushyantbehl commented Mar 9, 2026

Copy link
Copy Markdown
Contributor

/build

dushyantbehl
dushyantbehl approved these changes Mar 9, 2026
View reviewed changes

@dushyantbehl dushyantbehl left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me. Good work @yash4242 !

@github-actions

github-actions Bot commented Mar 9, 2026

Copy link
Copy Markdown

Build succeeded for c854702 (NVCR image)

View run

@github-actions

github-actions Bot commented Mar 9, 2026

Copy link
Copy Markdown

Build succeeded for c854702 (NVCR image)

View run

@dushyantbehl
dushyantbehl merged commit 09810e3 into foundation-model-stack:main Mar 9, 2026
9 checks passed
dushyantbehl added a commit that referenced this pull request Mar 26, 2026
@dushyantbehl
Revert "feat: upgrade to transformers v5 (#659)"
f80bdb8 
This reverts commit 09810e3.
@dushyantbehl dushyantbehl mentioned this pull request Mar 26, 2026
Merged
YashasviChaurasia added a commit to YashasviChaurasia/fms-hf-tuning that referenced this pull request Mar 27, 2026
@YashasviChaurasia
Revert "feat: upgrade to transformers v5 (foundation-model-stack#659)"
426f05f 
This reverts commit 09810e3.
dushyantbehl added a commit that referenced this pull request Mar 27, 2026
@dushyantbehl
Revert "feat: upgrade to transformers v5 (#659)" (#674)
72dc048 
This reverts commit 09810e3.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@romitjain romitjain romitjain left review comments
@anhuong anhuong Awaiting requested review from anhuong
@aluu317 aluu317 Awaiting requested review from aluu317 aluu317 is a code owner
@fabianlim fabianlim Awaiting requested review from fabianlim fabianlim is a code owner
@kmehant kmehant Awaiting requested review from kmehant kmehant is a code owner
+1 more reviewer
@dushyantbehl dushyantbehl dushyantbehl approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

feat

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@yash4242 @dushyantbehl @romitjain