0
$\begingroup$

First time submitter here. I'm a composer of (for want of a better term) contemporary classical music - and I would like to use AI to improve my productivity. The specific issue I'm dealing with is that I have plenty of raw ideas (inspirations) but converting an inspiration into a finished composition is (to a certain extent) more mechanical and tedious.

Here's what I'm thinking: I would first train the AI in my musical style using my existing compositions (including version histories). The training input would be somewhere between 2-5K files in MusicXML format - my composition software (Dorico) does this. The AI would learn my style of composing. I would then feed a new idea or a partly finished composition (again in MusicXML format) to the AI and ask where it should go next based my style. The output should be in MusicXML format. I would take this and feed it back into my composition software and then go from there - I could pick & choose what I like - and then feed that back into the AI for further ideas. I.e. - I'm envisioning some sort of iterative process.

Any thoughts / suggestions? Thanks!

$\endgroup$

1 Answer 1

2
$\begingroup$

Generative AI isn't a great starting point if you haven't worked with AI before. I would recommend starting with something simpler and working your way up to that. There is good stuff online though for learning it, and if you want to dive into the deep end that is where I'd start. Build a few generative AI models to create text or images and then start thinking about how to apply them to your musical application. Here's a good place to get started that isn't hidden behind a pay wall:

https://lore.com/generative-ai

If you want to mess around with AI applications in general, but still want something useful for inspiration while your work your way up to your generative AI masterpiece, a fun project might just be to build something that will play things you or other people have already composed that could provide inspiration related to what you are working on now. You know how Spotify gives you song suggestions based on what you have listened to before? Imagine something like that, but for composers. You play a little something you have been working on, and then the program finds stuff in its catalog with a similar feel to that an plays them back. This could provide inspiration in the moment by showing you a bunch of stuff sort of like what you just played to give you ideas where to spin off next. Even better though, this will give you data and tools to use later to create your generative AI, as well as giving you a deeper understanding of what you are doing when you get to that point.

Spotify's song recommendations are driven by a nearest neighbors algorithm called Annoy. Another I particularly like is Facebook's Faiss. Use some signal processing methods to extract feature vectors from various compositions, index those feature vectors using some nearest neighbors library like those mentioned above, and then you will have something that will spit out songs sort of like the one it just heard when you play it something new. It's not creating a new composition, but it's a good start toward learning what AI models are doing when they try to generate something "like" something else.

$\endgroup$
10
  • $\begingroup$ Thanks for the response, but this does not help me (at least not yet!). The key thing to make my concept work is the ability for the AI to take file input in MusicXML format. Let me dig in a bit deeper. There are various software packages for composers these days - Finale, Sibelius, Dorico, etc. These packages are not compatible with one another as they each have their own internal storage mechanism, but you can exchange compositions using MusicXML which is the lingua franca for composition software. $\endgroup$
    – E Heilner
    Commented Dec 8, 2023 at 2:42
  • $\begingroup$ So - as a composer you enter the music using standard musical notation. E.g., here is Mary Had a Little Lamb: i.sstatic.net/z7sOp.png And then you export the composition using MusicXML. If you're curious, here's the full MusicXML file. $\endgroup$
    – E Heilner
    Commented Dec 8, 2023 at 2:42
  • $\begingroup$ I mostly work with images and am not super well versed on NLP but this feels like an NLP-ish problem. You have a sort of "language" you want to extract information from to generate predictive "text". It's kind of a complicated one though, with multiple properties that need to be encoded for each token. LSTMs are definitely something you should look into, as well as NLP libraries like Gensim, CoreNLP, or NLTK. I'm not familiar enough with this area to really give a step by step breakdown but I think you can basically treat each measure like a word or sentence and play around from there. $\endgroup$
    – Jeremiah
    Commented Dec 8, 2023 at 14:42
  • $\begingroup$ Getting it into XML format is the easy part. That's just normal programming. Take the values your model returns and format them as XML. You aren't really trying to train a thing to generate an XML document. When you actually get to the generative AI phase what you really want to generate is a time series of notes, with properties like pitch and duration, and then you can format them however you'd like. $\endgroup$
    – Jeremiah
    Commented Dec 8, 2023 at 14:55
  • $\begingroup$ Jeremiah - Thanks again for taking the time to respond. So far I am not seeing anything that meets my needs. Just to give a few more details for anyone else reading, music is far more than a time series of notes - there are literally hundreds of properties that have to be accounted for: time signature, key signature, staff, accents, embellishments, playing techniques, slurs, phrasing, formatting, etc, etc. For this to be practical both the input & output have to be in MusicXML format to work with any of the standard composition software packages. MusicXML $\endgroup$
    – E Heilner
    Commented Dec 11, 2023 at 19:52

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .