Google’s new AI turns text into music

Google researchers person made an AI that tin make minutes-long philharmonic pieces from substance prompts, and tin adjacent alteration a whistled oregon hummed melody into different instruments, akin to however systems similar DALL-E make images from written prompts (via TechCrunch). The exemplary is called MusicLM, and portion you can’t play astir with it for yourself, the institution has uploaded a clump of samples that it produced utilizing the model.

Read more :The Amazon Halo Rise would be a neat sleep tracker, if it weren’t for my cat

The examples are impressive. There are 30-second snippets of what dependable similar existent songs created from paragraph-long descriptions that prescribe a genre, vibe, and adjacent circumstantial instruments, arsenic good arsenic five-minute-long pieces generated from 1 oregon 2 words similar “melodic techno.” Perhaps my favourite is simply a demo of “story mode,” wherever the exemplary is fundamentally fixed a publication to morph betwixt prompts. For example, this prompt:

electronic opus played successful a videogame (0:00-0:15)

meditation opus played adjacent to a stream (0:15-0:30)

fire (0:30-0:45)

fireworks (0:45-0:60)

It whitethorn not beryllium for everyone, but I could wholly spot this being composed by a quality (I besides listened to it connected loop dozens of times portion penning this article). Also featured connected the demo tract are examples of what the exemplary produces erstwhile asked to make 10-second clips of instruments similar the cello oregon maracas (the aboriginal illustration is 1 wherever the strategy does a comparatively mediocre job), eight-second clips of a definite genre, euphony that would acceptable a situation escape, and adjacent what a beginner soft subordinate would dependable similar versus an precocious one. It besides includes interpretations of phrases similar “futuristic club” and “accordion decease metal.”

MusicLM tin adjacent simulate quality vocals, and portion it seems to get the code and wide dependable of voices right, there’s a prime to them that’s decidedly off. The champion mode I tin picture it is that they dependable grainy oregon staticky. That prime isn’t arsenic wide successful the illustration above, but I deliberation this 1 illustrates it beauteous well.

That, by the way, is the effect of asking it to marque euphony that would play astatine a gym. You whitethorn besides person noticed that the lyrics are nonsense, but successful a mode that you whitethorn not needfully drawback if you’re not paying attraction — benignant of similar if you were listening to idiosyncratic singing successful Simlish oregon that 1 opus that’s meant to dependable similar English but isn’t.

I won’t unreal to cognize how Google achieved these results, but it’s released a probe paper explaining it successful item if you’re the benignant of idiosyncratic who would recognize this figure:

AI-generated euphony has a agelong past dating backmost decades; determination are systems that person been credited with composing popular songs, copying Bach better than a quality could successful the 90s, and accompanying unrecorded performances. One caller mentation uses AI representation procreation motor StableDiffusion to turn substance prompts into spectrograms that are past turned into music. The insubstantial says that MusicLM tin outperform different systems successful presumption of its “quality and adherence to the caption,” arsenic good arsenic the information that it tin instrumentality successful audio and transcript the melody.

That past portion is possibly 1 of the coolest demos the researchers enactment out. The tract lets you play the input audio, wherever idiosyncratic hums oregon whistles a tune, past lets you perceive however the exemplary reproduces it arsenic an physics synth lead, drawstring quartet, guitar solo, etc. From the examples I listened to, it manages the task precise well.

Like with different forays into this benignant of AI, Google is being significantly much cautious with MusicLM than some of its peers whitethorn beryllium with akin tech. “We person nary plans to merchandise models astatine this point,” concludes the paper, citing risks of “potential misappropriation of originative content” (read: plagiarism) and imaginable taste appropriation oregon misrepresentation.

It’s ever imaginable the tech could amusement up successful 1 of Google’s amusive philharmonic experiments astatine immoderate point, but for now, the lone radical who volition beryllium capable to marque usage of the probe are different radical gathering philharmonic AI systems. Google says it’s publically releasing a dataset with astir 5,500 music-text pairs, which could assistance erstwhile grooming and evaluating different philharmonic AIs.