What are the best machine learning models that have been used to compose music? Are there some good research papers (or books) on this topic out there?

I would say, if I use a neural network, I would opt for a recurrent one, because it needs to have a concept of timing, chord progressions, and so on.

I am also wondering how the loss function would look like, and how I could give the AI so much feedback as they usually need.


There are a few of them. The most recent I've found is from DeepMind: The challenge of realistic music generation: modelling raw audio at scale. This video is a great analysis of it.

  • 1
    $\begingroup$ I like this answer, but maybe you could link to some other ones. EvoMusArt puts out a few papers every year, for example: evostar.org/2018/cfp_evomusart.php#abstracts. $\endgroup$ Aug 30 '18 at 11:36
  • $\begingroup$ @JohnDoucette I don't know much on the theme, and I didn't know that article/team. You should compile your own answer, so the OP gets notified $\endgroup$
    – BlueMoon93
    Aug 30 '18 at 12:14

I am also new to the neural network architecture game but from what I have learned so far I think you have a few good options to choose from.

A recurrent neural network (RNN) would be a standard approach but if you're looking for something more robust you could look into a Long Short Term Memory network (LSTM). The neurons have a memory of past events and can recall that later on. It is a subset of RNN.

Perhaps you could go a little further and use a Convolutional Neural Network (CNN). So far these type of networks have been highly successful for image recognition. You could abstract a song piece as an image. Each pixel could be a progression in time and the value of the pixel could be the actual note.

Also take a look at this article for a good overview of several different neural network types.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.