Anthropic rewrites its service API in Rust with Claude's help, cutting CPU usage by 40–50%
Anthropic's Messages API was originally a single-GCP-region Python monolith; after CPU became a bottleneck, the team rewrote it in Rust. Engineers handled system design and verification, while Claude wrote the code, and the migration was validated by comparing outputs from the old and new systems. After migration, CPU usage dropped by about 40% to 50%, and first-token response time fell by about 10% to 20%.
Anthropic's Messages API initially ran as a Python monolith in a single GCP region. As traffic grew, CPU (rather than GPU) became the bottleneck. The team decided to migrate the entire system to Rust, even though at the outset "there wasn't a single Rust developer in the room."
Engineers handled system design and validation, while Claude handled code implementation. The team validated the migration by comparing the outputs of the old and new systems.
The team says that after the migration, CPU usage dropped by about 40% to 50%, and time to first token was reduced by about 10% to 20%.
Source: Latent Space