DeepSeek-V4 Flash RL Training Now Accelerated on AMD Instinct MI355X GPUs with Miles

The LMSYS Org, in collaboration with AMD and the Miles team, has announced the successful enablement of DeepSeek-V4 Flash Reinforcement Learning (RL) training on AMD Instinct MI355X GPUs, utilizing the ROCm software stack and the Miles framework. This achievement involves a complex technical integration, ensuring that the 284-billion-parameter Mixture-of-Experts (MoE) model, DeepSeek-V4 Flash, can be effectively trained in an end-to-end RL workflow on AMD's latest hardware. Key challenges addressed included aligning model behavior between SGLang rollout and Megatron training, preserving quantized state during online weight updates, and establishing stable multi-node parallel strategies for the large-scale MoE architecture. This development is highly significant for practitioners in cloud, DevOps, and AI. It directly impacts the strategic decisions around hardware procurement and infrastructure design for AI workloads. The availability of a robust, high-performance training solution for a cutting-edge LLM like DeepSeek-V4 Flash on AMD GPUs provides a viable alternative to the prevalent NVIDIA ecosystem. This diversification can lead to increased competition, potentially driving down hardware costs and improving the accessibility of advanced AI capabilities. For organizations heavily invested in open-source AI models, this expands their options for scalable and efficient training environments, reducing reliance on a single vendor's hardware and software stack. This move fits into a broader, well-established trend within the AI and cloud computing landscape: the push for hardware diversity and open-source software stacks to democratize access to high-performance AI. Companies like Google, AWS, and Microsoft Azure are continually investing in their own custom AI accelerators (TPUs, Inferentia, Gaudi) and supporting open standards to offer alternatives to NVIDIA's CUDA. The collaboration between LMSYS (known for its open-source contributions like SGLang and FastChat) and AMD with its ROCm platform exemplifies this trend. It underscores the industry's collective effort to optimize performance, reduce costs, and foster innovation by moving beyond proprietary ecosystems. The challenges overcome, such as managing complex MoE routing and hybrid compressed attention on new hardware, highlight the ongoing engineering effort required to make these powerful models practical for widespread use. In practice, cloud architects and MLOps engineers should closely monitor the performance benchmarks and cost-efficiency of DeepSeek-V4 Flash RL on AMD MI355X GPUs. This could inform future infrastructure investments, especially for those building or operating large-scale reinforcement learning systems or fine-tuning large language models. Evaluating the ease of integration with existing MLOps pipelines and the maturity of the ROCm ecosystem will be critical. Furthermore, the focus on resolving model-alignment and online-update issues points to the increasing complexity of managing stateful, dynamic AI training processes. Practitioners should prepare for a future where multi-vendor hardware environments are the norm, requiring adaptable software stacks and robust validation processes to ensure model integrity and performance across diverse compute platforms.