Repository navigation

Update before release - #9

Merged

matteospanio merged 40 commits into

mainfrom

claude/switch-to-roberta-base-5fq0i

Mar 28, 2026

Commits on Mar 21, 2026
Commits on Mar 22, 2026
- Fix OOM: release futures after processing and stream shard metadata
  claude
  committed
  fa9ff71
- Update SLURM scripts to reflect CodeBERT and streaming sharding
  claude
  committed
  bd829f6
- Fix tokenizer loading from local path in trainer
  claude
  committed
  93b7ce8
- Fix SLURM tokenizer path to match cluster layout
  claude
  committed
  5211117
- 🐛 Change max_position_embeddings (2048 > 514)
  matteospanio
  committed
  e045a45
Commits on Mar 23, 2026
- 🔧 Set max seq length to 512
  matteospanio
  committed
  9ec779c
Commits on Mar 24, 2026
- ⚡ Add bf16, gradient accumulation, and training perf params
  
  matteospanio
  and
  claude
  committed
  b4e1bb9
- 🔄 Switch from CodeBERT to RoBERTa-base for MLM pretraining
  
  matteospanio
  and
  claude
  committed
  91788ff
- 🔨 Update training params in slurm script
  matteospanio
  committed
  b782ae8
- 🐛 Remove old flag
  matteospanio
  committed
  35136d1
- ✨ Set default to ddp find unused param false
  matteospanio
  committed
  5d51812
- 🐛 Pass configuration to train CLI
  matteospanio
  committed
  a49b71f
- 🐛 Set save steps as a multiple of eval steps
  matteospanio
  committed
  d8e6526
- 🔧 Update batch size conf
  matteospanio
  committed
  0bb3c30
- 🔧 Update config
  matteospanio
  committed
  04570c4
- 🔨 Add cpu-bind=none to slurm
  matteospanio
  committed
  f928143
- 🔨 Refactor to use srun
  matteospanio
  committed
  38e828c
- 🔨 Add rsync to move data
  matteospanio
  committed
  d41b2ac
Commits on Mar 25, 2026
- ⚡️ Load dataset into RAM
  matteospanio
  committed
  92abddf
- 🧑‍💻 Add progress bar loading sharded dataset
  matteospanio
  committed
  626c1f6
- 🔨 Reduce shards size from int64 to int32
  matteospanio
  committed
  55724a1
- ♻️ Go to codebert base default
  matteospanio
  committed
  3c3e0bc
- 🔧 Set early stopping threshold to 0
  matteospanio
  committed
  818f347
- 🔊 Add early stopping callback log
  matteospanio
  committed
  b5f7a15
Commits on Mar 27, 2026
- ✨ Add trainer config restore from checkpoint
  matteospanio
  committed
  b4ff54d
- 🐛 Update pipeline API and increase train memory
  matteospanio
  committed
  fc1355b
- ♻️ Refactor emb extraction
  matteospanio
  committed
  ee89f77
- 🐛 Update model api
  matteospanio
  committed
  312a954
Commits on Mar 28, 2026
- ✨ Add support to raw codebert
  matteospanio
  committed
  f699dc2
- 🔖 Cleanup before release
  matteospanio
  committed
  d6ac81e
- ✅ Pass smoke test
  matteospanio
  committed
  fb79c15