AI Can Sing: A look at generative music models

These days, it is impossible to scroll through most social media platforms without running into AI generated media. The recent advancements in generative AI which conveniently coincides with the boom in popularity of many AI tools has inspired everyday users to create “art” using some of the popular generative models like DALL-E, Synthesia, and of course ChatGPT. More recently, this new wave has inspired a new wave of automatic music generation using deep learning neural network frameworks. While some of these AI-generated music is completely made up by the model, some are created using the AI versions of the voices of popular musicians. 

Heart on My Sleeve is one of the latter and was created using the voices of Drake and The Weeknd, two famous musicians. When the song was released earlier this year, it instantly became a viral hit before it was eventually removed from multiple platforms. Now, the existence and success of Heart on My Sleeve raises a lot of questions about AI music generation and the ethical and legal implications.

Like any other generative model, the process of training a music generation model requires large amounts of data, as well as a pattern learning and recognition algorithm. With music generation specifically, the major obstacle is translating music into an understandable format for the model. This is usually done by representing music with a series of numeric tokens which can include a set of chords, beats, rhythm, and other useful data points.This can then be processed by the algorithm.  The model can also be trained using MIDI files, a type of file that contains information about the notation, pitch and velocity of different parameters in music. This makes it easier to transmit musical information and data, and can also be used as a natural language representation for model training.

Apart from the occasional viral full-length son generation, AI is mostly used as a tool to aid in music creation rather than the creation itself. This can look like helping to generate melodies or baselines, AI assistants to help in producing music, and generating music to server as inspiration. This method of integrating AI and music is usually more accessible than making AI generated music, and not as controversial. It also helps that many artists and music producers already use AI in one form or the other when making music.

Before he died in March 1827, Beethoven completed 40 sketches for his unfinished 10th symphony. In 2021, a group of researchers and music experts at Rutgers started a project to use AI to complete his 10th symphony. They began by creating an algorithm trained on Beethoven’s complete works to teach the model Bethoven’s creative process. The generated work was then sent to a music expert who selected the bits that most closely matched Beethoven’s previous work. Beethoven’s 10th symphony would go on to premier with the Beethoven Orchestra Bonn almost 200 years after his death.

The use of AI in creative industries is relatively new but its rapid popularity has made it both a cause for concern and an efficient assistant. For the music industry specifically, AI music creation has given music lovers and enthusiasts the opportunity to create music featuring their favorite artists across different genres at an unprecedented speed. As such, social media platforms like Tiktok are full of the AI created songs and covers like Micheal Jackson singing in the style of Drake, and Rihanna singing a cover of a Beyoncé song. Tech companies are also taking advantage of this trend with  Youtube recently announcing their AI tool that clones famous singers allowing users to experiment with them. Google Deepmind has also announced their new AI music generation model, Lyria, in partnership with Youtube.

On the other hand, music executives and labels, as well as some artists have not been as welcoming or open to AI generated music.This comes alongside a larger skepticism of AI content in Hollywood. Earlier this year, the Screen Actors Guild-American Federation of Television and Radio Artists (SAG-AFTRA), and the Writers Guild of America (WGA) both went on strike and cited AI and its use as one of their important issues. Universal Music Group, the music label of both Drake and The Weeknd has also released a statement asking “which side of history all stakeholders in the music ecosystem want to be on: the side of artists, fans and human creative expression, or on the side of deep fakes, fraud and denying artists their due compensation.”

The ethics of AI-generated music is a little tricky to navigate because there has to be a balance between respecting and compensating artists for their work, and ensuring that the music industry is not left behind. To do this, there has to be a high degree of transparency on the part of AI developers. This includes asking for permission to use music or the voices of artists for projects as well as making sure that AI-generated music is clearly labeled as such. In short, this technology has to be used responsibly and ethically, and hand in hand with actual musicians.

As with all AI-generated art, there is a huge copyright issue when it comes to AI created art. Since there are so many parties involved (the generative model, the system user, the owner of the data the model is trained on etc.) it is often difficult to determine who owns the copyright to the generated output. This also extends to if artists should be compensated for their data used to train AI models and the acceptable amount for this service. Since there are no laws regulating this issue yet, the burden falls on developers to make sure that they are respecting the moral right of artists and musicians.

Generative AI is expanding the scope of music and changing people’s perceptions of what music can look like. This can be a good thing because it allows for more experimentation and helps with the creative process of making music. It also allows for music lovers to try out and share their own projects. At the same time, actual artists and musicians have to be at the forefront of this technology since it relies heavily on their work. This means that AI-generated music must be created ethically and with an appropriate amount of respect for the art.

Note: If you like this content and would like to learn more, click here! If you want to see a completely comprehensive AI Glossary, click here.