project_updates.rst

Project updates

This page archives Transformer Engine releases, technical articles, research, and community news. The README highlights only the most recent updates.

2026

• [09/2026] Transformer Engine v2.19 adds Rubin support, hybrid quantization, MXFP8 EP communication, and expanded FP8 attention support.
• [09/2026] Accelerating Dropless MoE Training in JAX with NVIDIA Transformer Engine.
• [08/2026] Transformer Engine v2.18.
• [07/2026] Transformer Engine v2.17.
• [06/2026] Boosting MoE Training Throughput with Advanced Fusion Kernels.
• [06/2026] Nemotron 3 Ultra: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoning.
• [06/2026] Train Models Faster with JAX and MaxText Using NVFP4 on NVIDIA Blackwell.
• [04/2026] Run High-Throughput Reinforcement Learning Training with End-to-End FP8 Precision.
• [02/2026] Using NVFP4 Low-Precision Model Training for Higher Throughput Without Losing Accuracy.

2025

• [12/2025] NVIDIA Nemotron 3: Efficient and Open Intelligence — trained with NVFP4 on Transformer Engine.
• [11/2025] NVIDIA Blackwell Architecture Sweeps MLPerf Training v5.1 Benchmarks.
• [11/2025] Scale Biology Transformer Models with PyTorch and NVIDIA BioNeMo Recipes.
• [11/2025] FP8 Training of Large-Scale RL Models.
• [09/2025] Pretraining Large Language Models with NVFP4.
• [09/2025] Native FP8 Mixed Precision Training for Ling 2.0, Open Sourced!.
• [09/2025] Faster Training Throughput in FP8 Precision with NVIDIA NeMo.
• [08/2025] How We Built DeepL's Next-Generation LLMs with FP8 for Training and Inference.
• [08/2025] NVFP4 Trains with Precision of 16-Bit and Speed and Efficiency of 4-Bit.
• [06/2025] Floating Point 8: An Introduction to Efficient, Lower-Precision AI Training.
• [05/2025] Advanced Optimization Strategies for LLM Training on NVIDIA Grace Hopper.
• [03/2025] Stable and Scalable FP8 Deep Learning Training on Blackwell.
• [03/2025] Measure and Improve AI Workload Performance with NVIDIA DGX Cloud Benchmarking.
• [02/2025] Understanding the Language of Life's Biomolecules Across Evolution at a New Scale with Evo 2.
• [02/2025] NVIDIA DGX Cloud Introduces Ready-To-Use Templates to Benchmark AI Platform Performance.
• [01/2025] Continued Pretraining of State-of-the-Art LLMs for Sovereign AI and Regulated Industries with iGenius and NVIDIA DGX Cloud.

2024 and earlier

• [11/2024] Developing a 172B LLM with Strong Japanese Capabilities Using NVIDIA Megatron-LM.
• [11/2024] How FP8 Boosts LLM Training by 18% on Amazon SageMaker P5 Instances.
• [11/2024] Efficiently Train Models with Large Sequence Lengths Using Amazon SageMaker Model Parallel.
• [09/2024] Reducing AI Large Model Training Costs by 30% Requires Just a Single Line of Code from FP8 Mixed Precision Training Upgrades.
• [05/2024] Accelerating Transformers with NVIDIA cuDNN 9.
• [03/2024] Turbocharged Training: Optimizing the Databricks Mosaic AI Stack with FP8.
• [03/2024] FP8 Training Support in SageMaker Model Parallelism Library.
• [12/2023] New NVIDIA NeMo Framework Features and NVIDIA H200.
• [11/2023] Inflection-2: The Next Step Up.
• [11/2023] Unleashing the Power of Transformers with NVIDIA Transformer Engine.
• [11/2023] Accelerating PyTorch Training Workloads with FP8.
• [09/2023] Transformer Engine Added to the AWS Deep Learning Container for PyTorch Training.
• [06/2023] Breaking MLPerf Training Records with NVIDIA H100 GPUs.
• [04/2023] Benchmarking Large Language Models on NVIDIA H100 GPUs with CoreWeave.