Skip to content

Migration to fairseq2>=0.5#79

Merged
avidale merged 6 commits intomainfrom
migrate-to-fairseq2-v05
Oct 10, 2025
Merged

Migration to fairseq2>=0.5#79
avidale merged 6 commits intomainfrom
migrate-to-fairseq2-v05

Conversation

@avidale
Copy link
Copy Markdown
Contributor

@avidale avidale commented Oct 10, 2025

Why ?

SONAR depends on https://github.com/facebookresearch/fairseq2 which introduced a ton of backward-incompatible changes in its recent 0.5 version. We update our code in order to keep up (also addressing #78).

How ?

Mostly by syncing the changes from the internal repo, including:

  • Refactoring the model cards by moving tokenizer-related fields one level down
  • Dealing with a lot of renamed interfaces (such as Tokenizer=>TextTokenizer)
  • Using BatchLayout to describe batches
  • Changing the way the model families are created (moving code from handler.py to checkpoint.py and config.py)
  • Start using DependencyContainer for registering models and DependencyResolver for retrieving them

Additional changes, not directly required by fairseq2 migration:

  • Getting rid of requirements.txt and requirements-dev.txt to use only pyproject.toml as a single reference
  • A few extra tips in README.md

The sonar-space package version gets upgraded from 0.4.0 to 0.5.0.

Test plan

The Github CI + local integration tests (python -m pytest tests/integration_tests/; you may need to remove ~/.cache/fairseq2 if there are any problems with fairseq2 assets)

@meta-cla meta-cla bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Oct 10, 2025
@avidale avidale marked this pull request as ready for review October 10, 2025 10:12
@avidale avidale requested a review from artemru October 10, 2025 10:12
# Format
black==22.3.0
black==25.1.0
isort>=5.10.1
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why not updating isort as well?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not! Which version do you like? :)

Copy link
Copy Markdown
Contributor

@artemru artemru left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(i would go with fs2>0.5.2 to be safe)

@avidale avidale merged commit 3a95f40 into main Oct 10, 2025
5 checks passed
@avidale avidale deleted the migrate-to-fairseq2-v05 branch October 10, 2025 11:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants