Etched, a four-year-old AI chip startup, emerged from stealth on June 30 with a set of numbers that demanded attention: $800 million raised, over $1 billion in signed customer contracts, a working transformer ASIC called Sohu entering production, and an investor list that reads like a who's who of AI royalty. The company claims its purpose-built silicon can deliver an order of magnitude more inference throughput than NVIDIA's best GPUs — at lower cost and power.
Sohu is not another GPU. It is an application-specific integrated circuit (ASIC) that hard-codes the transformer attention mechanism directly into silicon on TSMC's N4P process node. Every major large language model — GPT, Llama, Claude, Gemini — runs transformer attention at the core of every inference request. Etched's bet is that general-purpose GPUs, which must remain flexible for graphics, scientific computing, and training workloads, carry structural inefficiency for the one workload that now consumes the majority of AI compute spending: transformer inference.
Architecture: Low Voltage and Cluster Scale Memory
Etched's architecture targets the FLOP utilization gap through two hardware innovations. The first, Low Voltage Inference, runs the chip's compute blocks at under half the voltage of conventional AI accelerators. This dramatically increases FLOP density without triggering the thermal throttling that forces GPUs to downregulate clock speeds under sustained load. The result: Sohu clusters can run trillion-parameter sparse models — including mixture-of-experts architectures — at over 80 percent of peak FLOP throughput. By contrast, general-purpose GPUs typically achieve 30 to 40 percent of their theoretical FLOP capacity on transformer inference. The rest sits idle.
The second innovation, Cluster Scale Memory, creates a shared, low-latency memory pool across chips in the scale-up domain using a proprietary high-bandwidth interconnect. Standard HBM struggles with the decode phase of inference — where each new token requires reading the full key-value cache — because memory bandwidth is shared between weight reads and cache reads. Etched separates those paths and adds an ultra-low-latency fabric across chips, addressing both the throughput bottleneck and the per-token latency that degrades user experience at high batch sizes.
Throughput Claims vs. Reality
The raw numbers are arresting. Etched says a single eight-chip Sohu server can process more than 500,000 tokens per second running Llama 70B. An equivalent eight-GPU H100 server manages roughly 23,000 tokens per second. NVIDIA's own B200 cluster hits approximately 43,000 to 45,000 tokens per second on the same workload. If the Sohu figures hold under third-party validation, the performance gap represents a generational leap in inference economics.
The chip pairs its compute die with 144 gigabytes of HBM3E memory. An eight-chip server can hold a 400- to 600-billion parameter model with tensor parallelism, placing frontier-scale open-weight models within reach of a single rack unit. First production racks ship this summer.
The caveat is that no independent third party — not MLPerf, not a cloud provider, not a standards body — has yet published verifiable throughput measurements from physical Sohu hardware under production conditions. The numbers come from Etched's own published materials and internal testing. The startup's credibility rests on a $5 billion valuation, blue-chip investors, and signed contracts, but the proof will arrive when customers run real workloads at scale.
The Investor Signal
Etched's cap table is unusual even by AI startup standards. Institutional investors include VentureTech Alliance, Jane Street, Hudson River Trading, Two Sigma, Ribbit Capital, and Stripes — names more associated with quantitative finance and systematic trading than silicon. Angel investors include Andrej Karpathy, Geoffrey Hinton, Fei-Fei Li, Arthur Mensch (Mistral CEO), and Scott Wu (Cognition CEO). Billionaires Stanley Druckenmiller and Peter Thiel also hold positions.
Founders Gavin Uberti (CEO) and Robert Wachen (President) — both Harvard dropouts and Thiel Fellows — spent 2023 struggling to raise capital, operating month-to-month and close to running out of cash. A 30-page memo arguing that AI would eventually need specialized chips, not general-purpose GPUs, was passed on by every major investor. The shift in sentiment over three years reflects how dramatically the AI infrastructure market has matured since the early ChatGPT era.
Etched's emergence comes at a moment when the entire AI hardware landscape is fracturing. Cerebras had the first breakout AI IPO of 2026. Groq just raised $650 million. Hyperscalers Amazon, Google, and Microsoft all build custom in-house AI chips. OpenAI announced its first custom chip, built by Broadcom. NVIDIA's GPU dominance — built on CUDA's ecosystem lock-in and generational manufacturing leads — is being attacked from every direction.
Whether Sohu's performance claims survive contact with production reality will determine if Etched becomes the company that redefined inference hardware — or a cautionary tale about the gap between benchmark numbers and customer outcomes. For now, the industry is watching.
Comments