Meta talked about its own AI Research SuperCluster (RSC) supercomputer. It is designed to solve artificial intelligence problems and is already being used by the company in natural language processing and machine vision research. The development took about two years.
Now the supercomputer consists of 760 NVIDIA DGX A100 systems, which in total have 6080 graphics accelerators. This is comparable to the Perlmutter supercomputer, which also includes more than 6,000 of the same GPUs from Nvidia and is the fifth most powerful in the world, Dev notes. Meta is set to complete its build by mid-year, increasing the number of GPUs by 2.5 times to 16,000, making it the world’s fastest AI supercomputer. Where it will be located and how much it will cost the company, it is not disclosed.
Meta wants to train models with a trillion or more parameters on exabyte datasets (1 billion gigabytes), which is equivalent to 36,000 years of high-quality video. RSC is already 20 times faster than its 2017 predecessor at image recognition and three times faster at speech recognition. In the future, it will allow creating AI models that will be able to understand the context and nuances of the world around us much better than now, and “think” like a person, notes NIX Solutions.
The company will be able to use an AI supercomputer to train multi-modal models that process various types of inputs (such as audio, images, and video), which will improve moderation and search for inappropriate content on Meta social networks. It will also come in handy when developing the metaverse, for example, to train models that will know hundreds of languages or be able to create AR applications.