MAKING MUSIC WITH AI
Leveraging AI tools to create music
Tools
ChatGPT, Vocaloid6, Dall-E, RunwayML, Final Cut Pro X
Collaborators
Audra (research, lyrics, and synthesizing), Jon (research and synthesizing), Jiashi (visuals and music video)
To kickstart the project, we leveraged ChatGPT, an advanced language model. By feeding it prompts about the theme of "making music with AI," we engaged in a fascinating dialogue with the model, refining the lyrics and chord progressions.


To breathe life into the lyrics, we researched different AI tools that could let us synthesize realistic singing voices. We then employed Vocaloid6, by tuning the parameters and infusing them with emotion, the synthesized vocals transformed the lyrics into an enchanting melody.
Complementing the music, we used Bing Image Creator (powered by Dall-E), an AI model specializing in image generation, to produce captivating visuals that resonated with the lyrics and the narrative that we wanted, which is about a human that met a humanoid AI, who made music together, and eventually the person became an AI themself.
And finally, we utilized RunwayML's Frame Interpolation (which uses GAN technology) to animated the images and Final Cut Pro X to put together all the assets that we generated.