GOOGLE’S MAGENTA, CREATING NEW SOUNDS WITH NEURAL NETWORKS
Google Brain is a team of Artificial Intelligence researchers at Google, focused on deep learning, a branch of machine learning loosely inspired in how the human brain works. Google Brain’s stated mission is to improve people’s lives by making machines smarter.
One recent breakthrough of Google Brain has been image enhancement, in which neural networks improve the resolution of a low-resolution picture by adding pixels based on knowledge of what the picture should be, after analysing higher resolution images. Another exciting breakthrough is multilingual neural translating, in which a machine can translate speech in one language directly into text in another language, without first transcribing it to text in the first language. This was possible thanks to neural networks. Researchers exposed the system to many hours of Spanish audio and the corresponding English text. The neural networks were able to match the corresponding parts and manipulate the audio waveform until it was transformed into English. Pretty impressive.
Magenta is a project within Google Brain, specifically focused on teaching machines to produce new and interesting art and music using neural networks. Their latest project is called NSynth, which will provide musicians with an entirely new range of tools for making music. NSynth takes sounds from hundreds of instruments in a data base and feeds them into a neural network that analyses the notes creating a mathematical “vector” for each instrument. This vectors can be used to replicate the sound of an instrument, but they can also do something more interesting. They can be used to combine the two sounds -for example a trumpet and a clavicord- to create a completely new “vector”, that is, a new sound. Not a yuxtaposition or layering of sounds, but an entirely new frequency, using the mathematical characteristics of the two notes. You can also explore, with an interface, the audible space between several instruments at once. The team is also using a second neural network to combine those sounds in innovative ways.
Because another goal of Magenta is to build a community of artists, coders and machine learning researchers, they are building an infrastructure around TensorFlow -an open-source library for machine intelligence- and have made public the NSynth algorithms, so that anyone can download their sound database or create new tools for other artists to use.
More in this article.
