LatentMoE: Novel Expert Design

Super and Ultra utilize Latent MoE, a novel hardware-aware expert design for improved accuracy.

Key Benefits

• Experts designed in latent space for better parameter efficiency
• Hardware-aware optimization for Blackwell architecture
• Improved accuracy through specialized expert routing
• Reduced memory footprint compared to conventional MoE