What are the best machine learning models that have been used to compose music? Are there some good research papers (or books) on this topic out there?

I would say, if I use a neural network, I would opt for a recurrent one, because it needs to have a concept of timing, chord progressions, and so on.

I am also wondering how the loss function would look like, and how I could give the AI so much feedback as they usually need.


2 Answers 2


There are a few of them. The most recent I've found is from DeepMind: The challenge of realistic music generation: modelling raw audio at scale. This video is a great analysis of it.

  • 1
    $\begingroup$ I like this answer, but maybe you could link to some other ones. EvoMusArt puts out a few papers every year, for example: evostar.org/2018/cfp_evomusart.php#abstracts. $\endgroup$ Aug 30, 2018 at 11:36
  • $\begingroup$ @JohnDoucette I don't know much on the theme, and I didn't know that article/team. You should compile your own answer, so the OP gets notified $\endgroup$
    – BlueMoon93
    Aug 30, 2018 at 12:14

I am also new to the neural network architecture game but from what I have learned so far I think you have a few good options to choose from.

A recurrent neural network (RNN) would be a standard approach but if you're looking for something more robust you could look into a Long Short Term Memory network (LSTM). The neurons have a memory of past events and can recall that later on. It is a subset of RNN.

Perhaps you could go a little further and use a Convolutional Neural Network (CNN). So far these type of networks have been highly successful for image recognition. You could abstract a song piece as an image. Each pixel could be a progression in time and the value of the pixel could be the actual note.

Also take a look at this article for a good overview of several different neural network types.


You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .