A class I'm taking this semester on computational linguistics, which is a continuation of the one I took last semester, is covering statistical analysis of linguistics. The subject covers everything from calculating how often specific words occur by simply counting them and dividing by the total number of words, to automatically grouping words by grammatical class and determining simple rules of grammar, to determining the author of unknown works by comparing them to samples of works by known authors. It's not what I thought the field would be when I started taking the course, but it's extremely interesting (to me, at least.) I've been wondering what else you could do with this, and then it hit me.
What if, instead of words, letters, or sounds, you used musical notes. You could create, more or less, the same type of analysis for music as you can for language. Some applications could be interesting, like determining the composer of unknown works, but the same tools that allow you to describe language, or in this case music, allow you to go the other way: you can create random variations based on what you've seen and analyzed. I tried this a while ago with my blog, and while a lot of the text it produced is garbage (you need a ton of sample text to get consistently good English, and my blog isn't that large), there were a few gems. Like "Tim can confiscate your property if he wants to." I'm looking into what's necessary to do this with music. I have a ton of Scott Joplin sheet music that I could run the analysis on, create a model, then generate novel music from. Chances are a lot of it will be crap, but with a basic understanding of music theory I could add some assumptions on how music is structured that could improve the results.
The only thing holding me back is that, while I really want to do this, I would have to translate all the sheet music into something a program could understand, which will involve a ton of typing. I could also do it on waveforms taken from MP3s on my computer, but that'll add an extra layer to the problem: figuring out the notes from the sound, which seems trivial but is actually fairly hard, especially with all the distortion used in a lot of my music. Maybe if I stick around another 4 years after I get my linguistics degree and get a music degree it'll all come together.
If I pull my act together and do this, I'll post some MP3s. Just don't hold your breath.
Tuesday, February 20, 2007
Subscribe to:
Post Comments (Atom)
9 comments:
I can probably automate the whole waveform-to-score problem by using the same tools that solve every other language problem... I can just assume that there are (say) 100 notes, from (say) 20 various instruments, and let some algorithm decide what's what. I wouldn't think this sort of thing would be possible, but then I had a homework assignment where we had to have a program determine part-of-speech tags from a large text file... the part-of-speech tags didn't tell you what the part-of-speech meant, but it would group them into classes like G1:{the, a, some, every, any, ...} G2:{dog, cat, fly, man, ...} G3:{run, chase, fly, sleep, ...} etc. Not much work to then say, ok, G1 are determiners, G2 are nouns, G3 are verbs, etc, so it probably also won't be much work to say, ok, G1 is a guitar, G2 is a piano, G3 is drums, etc. Time to break out some signal analysis textbooks.
Or, rather than having the system work on a score (like a language whose alphabet is pitches), I could just have the alphabet be some simplified segmentation of the waveform, like every 10ms or something.
I'll just write my own... with blackjack... and hookers...
you could also do this with art as well. as what is art but visual language? line art images could be converted to vectors which could then be analyzed for patterns and compared. Other art could be converted to a combination of vectors and groups of properties--colors, media used, techniques, age, etc.--that could then be compared. i'm guessing that this would be ridiculously hard though.
and i would assume that this general idea can be applied to any man-made item in order to find its original creator or to generate new items in the same vein or an amalgam of styles.
Approaches like this have been used with brush-stroke based paintings. A large collection of paintings by multiple artists are analyzed for consistent patterns specific artists use, and then new paintings by unknown artists are compared against the models created for each painter. Whichever model does the best job predicting the brush strokes of the unknown painting informs you of which painter is most likely to have produced the painting. This is also done for author identification for anonymous works. When I get a chance I could build a model of each of our uses of language based on our blogs, and then show who it predicts new blog entries were written by.
yeah... one of the leading author sleuths in the country was a professor at vassar. didn't know they were doing it with paintings though.
...but then the exciting thing anyway is having the computer then generate new works of art based on these various influences. ...though, if a computer is the creator, is it art?
Computers as creators of creative works can be a fairly heated issue. The book Principia Mathematica (1910s), by Alfred North Whitehead and Bertrand Russell, contains proofs of mathematical and logical statements. Allen Newell and Herbert Simon designed a computer program in the 1950s that attempted to generate proofs of the theorems in the Whitehead and Russell book. One of the proofs the program generated wound up being shorter than the proof in the book, and they tried to have the results published in the Journal of Symbolic Logic with the computer program as the author. The journal declined to publish the proof, and in general philosophers freaked out at the suggestion that machines could be thought of in such a way.
I would think that there would be less resistance to such a claim nowadays, especially in light of AI programs like Deep Blue (the chess program developed in the 1990s by IBM) and general advancements in artificial intelligence. So, if the computer is the artist, do I still get paid?
if you own the computer and someone buys the artwork. maybe. i wouldn't pay for it though.
I'm turning blue over here from holding my breath waiting on those MP3s. I know you told me not to, but dammit Fuzzy, you're not the boss of me.
Post a Comment