OpenAI Research Laboratory from San Francisco (USA), which specializes in the development of artificial intelligence, introduced a new successful project, says TechCult. It is called “Jukebox” and is intended for meticulous study and subsequent reproduction of voices and manner of performance of real singers. The training began with the works of Sinatra and Elvis and Jukebox showed some amazing results.
Jukebox works as a synthetic duet of the composer and singer-songwriter, which distinguishes it from live musicians. On the one hand, it combines notes and sounds unmistakably, on the other – it still makes ridiculous mistakes, it is “mistaken” in how to reproduce individual fragments correctly. As AI itself writes music from scratch and also sings the words, as the initial data, they give it only the text that still needs to be analyzed and understood how to sing it, specifies NIX Solutions.
Texts are created by another artificial intelligence, a special language model, edited by living OpenAI participants. They roughly correspond to the work of the selected artists – Jukebox was trained in 1.2 million individual songs. The peculiarity of this AI is that when you are listening to a song, it splits it into three levels. At the first one, this is an extremely compressed bitrate, just a melody and arrangement of words. The next two levels add detail to specific musicians and singers.
When Jukebox starts modeling a song, it goes the same way. First, it builds a “frame”, and then saturates it with details to the desired level. Only the AI decides how exactly this happens, which explains the still high probability of errors. But when Jukebox sings successfully, even an experienced listener is unlikely to suspect a catch, unless they know the work of the same Sinatra by heart and firmly believe that he never sang such a song. However, this does not interfere with experiments from the series “What if an opera singer of the beginning of the last century would be instructed to sing a fresh pop hit?” Examples of artificial intelligence can be heard on the project website.