G42 and Cerebras Systems announced their strategic partnership in July 2023, along with a roadmap that the two companies said will significantly reduce the time it takes to train artificial intelligence (AI) models. Work is underway to build Condor Galaxy, a cluster of nine supercomputers with a planned capacity of 36 exaFLOPs in total.
This, according to Andrew Feldman, CEO and co-founder of Cerebras Systems, “will add meaningfully to the worldwide inventory of AI computing power”.
Condor Galaxy is owned by G42, based in Abu Dhabi. Cerebras Systems, based in Sunnyvale, California, manages and operates Condor Galaxy through the Cerebras Cloud. The first of the nine AI supercomputers, Condor Galaxy 1 (CG-1), has already been deployed. With 54 million cores, it performs at four exaFLOPs, which makes it one of the largest AI supercomputers in the world.
Lending credence to the partners’ claim that their technologies will speed up training time, CG-1 has already been used to train Jais, which uses 13 billion parameters, making it the world’s most advanced large language model for the Arabic language. Much of the work on Jais had taken place before G42 and Cerebras formed a partnership. What was missing was time on a supercomputer to train the model – and that’s exactly where G42 and Cerebras Systems made the difference.
“G42 and Cerebras had already been discussing potential collaboration since 2021,” said Feldman. “Then earlier this year, we saw the rise of large language models. Our technology had matured, and there was interest in the UAE to do big things in AI.”
“It was the right time and the right people,” he said. “So, we began a big proof of concept and things went really well. We began flying out to visit them and got to know them – not just in a business context, but also in a personal context. And their team flew out here and got to know us a little bit better. By May, we agreed to build enough compute to change the worldwide inventory. Not only did we change the inventory of computing power, but we also added new large language models to the mix.”
It will open the Arab-speaking world to the benefits of AI. “One obvious use case for the new language model will be in Arabic chatbots,” according to Andrew Jackson, CEO of Inception, a subsidiary of G42, and the entity behind the launch of Jais. “As the model learns, it can be used in government to answer a whole set of questions without requiring you to wait for a person to respond. It will also be used for scientific projects and in other sectors, including healthcare and financial services. And we expect a range of other applications – including many that we haven’t even thought of.”
Speeding up training through new technologies
Cerebras developed two technologies to speed up the data-intensive training process for large AI models. The first was a new architecture, called Wafer-Scale Cluster, that connects up to 192 CS-2 systems and allows them to operate as a single logical accelerator. Cerebras CS-2 systems, which make up the Condor Galaxy AI supercomputer, are powered by Cerebras’ Wafer-Scale Engine, a processor developed specifically to accelerate deep learning.
According to the company website, it is the largest and most powerful AI processor in the world. The design takes machine learning to a new level by decoupling memory from compute, which increases the memory available to AI models from gigabytes (with graphics processing units) to terabytes.
The second technology was weight streaming, which exploits the compute and memory features of the hardware to distribute work by streaming a model one layer at a time in a purely data-parallel fashion.
“The partnership with G42 enables us to combine expertise,” said Feldman. “Their team had rich expertise in managing and deploying clouds. At Inception they are model-builders and have world-class capabilities in the construction of models. We bring our expertise to that, and then we brought some of our technology that allows us to distribute these models extremely quickly across giant compute clusters and use the performance of these machines to reduce the amount of time it takes to train models.”
Boosting the UAE to global leadership in AI
The G42-Cerebras Systems partnership is part of the UAE’s broader national AI initiative, which aims not only to increase the use of AI in the country, but also to develop the skills and technology to boost the country to a position of leadership. The UAE has already invested heavily in AI and is the first country in the world to appoint a minister for AI in their federal government. Moreover, one of the country’s universities, Mohamed bin Zayed University of Artificial Intelligence (MBZUAI), is the first post-graduate university in the world focused entirely on AI.
The partners plan to deploy the next two supercomputers, CG-2 and CG-3, in the US by early 2024. Then, G42 and Cerebras intend to bring online the other six Condor Galaxy supercomputers later in 2024, with the ambition to benefit not only customers in the Middle East, but also anywhere else in the world.
