In a groundbreaking development, Cerebras Systems has unveiled the world’s fastest artificial intelligence (AI) chip, the Wafer Scale Engine 3 (WSE-3). This remarkable 5nm chip packs an astonishing 4 trillion transistors, setting a new record in the industry.
The WSE-3 is the powerhouse behind Cerebras’ latest CS-3 AI supercomputer, which delivers a staggering 125 petaflops of peak AI performance. With 900,000 AI-optimized compute cores and 44GB of on-chip memory, the CS-3 is designed to train the next generation of massive AI models, up to 24 trillion parameters in size – a feat previously unimaginable.
“When we started this journey eight years ago, everyone thought wafer-scale processors were a pipe dream,” said Andrew Feldman, CEO and co-founder of Cerebras. “We’re proud to introduce the third generation of our groundbreaking wafer-scale AI chip, the fastest in the world, purpose-built for cutting-edge AI work.”
The CS-3’s immense memory system, capable of up to 1.2 petabytes, allows for training models 10 times larger than GPT-4 and Gemini without partitioning or refactoring the data, significantly simplifying the training workflow and accelerating developer productivity.
Cerebras claims that training a one-trillion parameter model on the CS-3 is as straightforward as training a one-billion parameter model on traditional GPUs. Remarkably, the compact four-system configuration can fine-tune 70 billion parameter models in a single day, while the full-scale 2,048-system setup can train the Llama 70B model from scratch in just one day – an unprecedented achievement in generative AI.
The latest Cerebras Software Framework provides native support for PyTorch 2.0, multi-modal models, vision transformers, mixture of experts, and diffusion, while offering native hardware acceleration for dynamic and unstructured sparsity, speeding up training by up to 8 times.
Cerebras’ innovations extend beyond raw performance to power efficiency and software simplicity. While GPU power consumption doubles with each new generation, the CS-3 doubles performance within the same power envelope. Additionally, the CS-3 requires 97% less code than GPUs for large language models (LLMs), with a GPT-3 sized model implementation requiring just 565 lines of code – an industry record.
The CS-3 has already garnered significant interest from enterprises, governments, and international clouds, with Cerebras reporting a sizable backlog of orders. Prominent partners, including Argonne National Laboratory and the Mayo Clinic, have expressed excitement about the potential of the CS-3 for exploring the limits of frontier AI and improving patient outcomes.
Moreover, Cerebras’ strategic partnership with G42 has yielded impressive results, with the companies announcing the construction of Condor Galaxy 3, a 64 CS-3 system that will produce 8 exaFLOPs of AI compute, making it one of the largest AI supercomputers in the world.
As the AI revolution continues to accelerate, Cerebras Systems’ WSE-3 and CS-3 AI supercomputer promise to push the boundaries of what’s possible, enabling researchers, scientists, and developers to tackle unprecedented challenges and unlock new frontiers in artificial intelligence.
Source: Cerebras
Like this article? Keep up to date with AI news, apps, tools and get tips and tricks on how to improve with AI. Sign up to our Free AI Newsletter
Also, come check out our free AI training portal and community of business owners, entrepreneurs, executives and creators. Level up your business with AI ! New courses added weekly.
You can also follow us on X