Monday, August 22, 2016

music21 v 3 Released (3.1.0)

Version 3 of music21 is here!  This is the first major release in 11 months, with nearly 600 commits since the last version.  As a new major version, there are both huge new features as well as significant (often backwards incompatible) changes.

Upgrades for most users should work automatically by typing:

   pip3 install --upgrade music21

or (Python 2)

   pip install --upgrade music21

Or download from GitHub

A summary of the most important changes since v.2.2 are below:

Fun and easy to understand changes:

Lots more MusicXML support -- better support for time signatures, for grace notes, for spanners, for metadata, for...you name it!  And the system is refactored in such a way as to make contributing missing features quite easy.

MIDI files play back in Jupyter/IPython contexts.  Lots of improvements there for people who use MuseScore.

The User's Guide has become much more awesome, and you can play with all the good features there.

Many new ways to search scores -- LyricSearcher is fully polished and documented in the User's Guide.  Carl Lian's search.serial is upgraded from alpha to a full release that will be easy to expand in the future -- want to know where certain motives are used or transformed? This will make it easy to do. And search.segment -- the old standby -- is even better than ever (see below)

Try splitting notes and recombining them -- lots of intelligence going into this.

There's a big difference between taking a passage up a major third and taking it up 4 semitones.  In Chromatic contexts, music21 will now spell things how a musician would like to see them.  If you're working with MIDI data, without explicit enharmonics specified, you'll appreciate this.

Go ahead and use '~/../dir' and things like that in your file parsing -- music21 groks all that.

Look at you, fancy programmer, with your 4-CPU laptop! Why not give common.runParallel(tasks, function) a try and get music21 working 2-3x faster than before?  (just make sure that "tasks" isn't a list of Streams).  Oh, and if you're using search.segment to search for a particular passage inside a large collection of files, don't worry about using runParallel -- we'll do that for you automatically.

Docs are pretty and much better.  The User's Guide is the place to start.

Too many other changes to mention, but some shoutouts to Shimpe for new Lilypond code (triplet chords, etc.), Frank Zalkow for enharmonic spelling , dynamics, tempos and other things,  Chris Antilla for continued dedication to MEI processing, Emily Zhang for hashing functions and speeding up MIDI quantization,  Sonovice for articulation handling,  Dr. Schmidt for Chord symbol translations, Bagratte for IO cross Py2/3 fixes, Bo-Cheng Jhan for great braille contributions.


Big under the hood changes.

Big, backwards incompatible change: Many calls such as .parts, .notes, .getElementsByClass(), .getElementsByOffset(), etc. no longer return Streams.  They now are iterators (returning something called a StreamIterator).  For most uses, this is not going to change anything.  You can still use: for n in myStream.notes: and it'll work great. It makes many parts of music21 much, much faster.  For small scores, the differences will be small.  For large scores, the differences will be tremendous, especially when filters are chained, such as: myScore.recurse().notes.getElementsByClass('Chord').getElementsByOffset(0.0).  You're going to find that writing iterator chains is an amazing way to only get at items you want, especially with custom filters.  To get the old behavior, just add .stream() to the end of your iteration.

Because none of these filters change the activeSite of an element, you'll find that this is much more stable than before.

If you want to know what the note after a given note is in a musical context, call n.next() or n.previous().  If it's the last note of a measure, it'll move on the first note of the next. And once you've called .next() on one note of a stream, the remaining calls will be super super fast. I still haven't wrapped my mind completely around this paradigm, but it sure beats all the fooling around I used to do to figure out if one note was the same pitch as the next one, etc.

If you've been using music21 for some time, but have never looked at the docs for base.contextSites(), do it -- this is a very fast and extremely powerful way of figuring out how two objects relate to each other.  Together with the .next(), .previous(), and better support for .derivation, many extremely powerful systems can be written in music21 easily that could only be written with huge difficulty before.

Nearly all functions marked deprecated in v.2. have been removed.  Lots of super obscure functions in .base.Music21Object or .sites.Sites are gone.  This is a positive step since it'll make the documentation for these objects simple enough to understand.

Sorting works.  I mean, it just works.  With grace notes, with oddly positioned elements, with what have you. And it's pretty darn fast. This might seem like something small, but it's enormous for us.

Corpus managing is much simplified -- if you ever thought in the past, "Hey, I'd like to use a custom corpus" and then thought, "uhh...no thanks..." give a look at what is needed to set one up now.  You'll be glad for it!

Musescore and not Lilypond is used in Jupyter/IPython notebooks.

Complex durations are a lot less complex -- and faster.  

PyLint on all code -- I estimate that at least 200 undetected bugs vanished through this major effort. 


The Future

As noted in messages to the music21 mailing list (music21list at Google Groups), v.3.1 is the first non-beta release in the v.3 lineup.  Version 3 happens to share a version number with Python 3, but that is merely coincidence.  Music21 version 3 continues to work with Python 2.7 as well as Python 3.4.  Version 3 adds explicit support for Python 3.5 and drops support for Python 3.3. Music21 will continue to develop into a Version 4 to be released next summer (4.0.x will be alpha and beta releases and 4.1.0 will be the public release).  Version 4 will likely be the last version to support Python 2.

This release represents the end of a year's sabbatical where I got to work on low-low-level music21 functions that I didn't think anyone else would want to.  Due to teaching and other obligations, I'll be taking off work on the heart of music21 until the holidays (I'll still be taking bug fixes, etc.) and working more on documentation, examples, and applications.  The changes put in place for music21 v.3 has made working with it a lot more fun for me, so you'll probably see more a lot more applications get added, first to the alpha directory and then into the main set.  I've also put up a version 4 roadmap (trees are almost done, they should make it in. Style objects will be introduced so that beautiful musical scores can be created or at least imported and exported properly without major speed loses) so if anyone wants to take the lead on a project you can do so.  I'm working on a project called STAMR, Small Tools for Agile Music Research, which should create standalone tools using music21 and music21j for musicologists to get their work done faster.  Given that I'm teaching music fundamentals online again, you should see music21 and music21j integration working far better than ever before.

Thanks to MIT for supporting my work, and the Seaver Institute and the NEH for initial funding to make music21 a success.  And thanks to this great community for all your contributions in the past and contributions to come.



Wednesday, August 3, 2016

music21 v. 3 beta-3.0.6 released

A release candidate of version 3 of music21 (3.0.6) is available now as a package on GitHub.  As a beta release it's still not 100% ready for general usage, so "pip install music21" will still install version 2.

Get it at:
https://github.com/cuthbertLab/music21/releases 

This is likely to be the Release Candidate also for music21 v.3.  I've indicated several times in the past that music21 v.3 is the last release that I guarantee will continue to support Python 2.  I now suspect that there will be a Python 2.7-compatible v.4, since Python 2.7 is still the shipping version of Python w/ macOS Sierra.

Notes about the substantial changes in Version 3 have been posted on this list several times.  The biggest changes are that .getElementsByClass() and .notes, etc. all return a new class called a “StreamIterator” which makes working with stream filtering much much faster than before, especially for very large streams. 

The new tree-based storage system is still too flaky to turn on by default except in a few cases, and will be deferred to Music21 v.4.

Laundry list of items changed since 3.03-alpha:

min Python 3 version is now 3.4 -- 3.3 should still work but is untested.Notes parsed from MIDI or transposed by a number of semitones now get a .spellingIsInferred attribute which indicates that they can change their spelling ("G#" vs "A-") as needed for the situation. This is incompatible behavior, but much improved!
More User's Guide chapters...
Lots of improvements to Braille output, and refactoring to make more improvements in the future possible. (Thanks Bo-Cheng Jhan! and esp. to Jose Cabal-Ugaz in the first place).
Better documentation for Searching Lyrics (User's Guide, Chapter 28)
MIDI will now handle part names, etc. that use unicode.
Long files (>10 pages) in Musescore automatic PDF generation format now work (thanks Emily Zhang)
TimeSignatures such as 2/4+3/8 display much better now.
TimeSignatures import and export symbols ('common', 'cut', etc.) in MusicXML properly.
Fixes for musicxml parsing where both voices and chords interact.
configure.py now finds common MusicXML readers (Finale, Sibelius, MuseScore) on Windows!
InsertIntoNoteOrChord works in more cases
Streams are faster to calculate their own durations
Major speedups to large makeMeasure() calls.
A failure in makeTies now gives a warning instead of an exception
The old "musicxmlOld" format is removed. 
metronomeMarkBoundaries works better (thanks Frank Zalkow!)
improved ability to specify how to quantize MIDI (thanks EZ!)
improvements to automatic instrument detection.
lots more MEI articulations work (thanks sonovice)
Fingerings import and export to MusicXML better.
noteheadParentheses work well now.
Roman.fromChordAndKey gets quite a few more chords now.
incompatible change: normalOrder instead of normalForm for chords -- gives the untransposed ordering of a chord -- this is the correct behavior; prior behavior is wrong.
incompatible change: Stream.offsetMap is now a method
incompatible change: common.isStr() removed -- use isinstance(s, str) or (s, (str, unicode))
incompatible change: key.KeySignature() no longer supports a .mode attribute.  Use key.Key() instead.  key.KeySignature() objects get an .asKey('major') etc. attribute.
incompatible change: pitches lose ".getMidiPreCentShift()" method.  Just call .midi instead!

in case you missed what is new in v.3.0.3:

Python 3.5 compatible.
Splitting notes via .splitXXXX() is now much safer and more efficiently
Many, many parts of music21 can now use multiprocessing efficiently (see common.misc.runParallel())
BUG: Haydn Opus 74no2 was not actually ever in the corpus and has been removed.
Several MEI bugs fixed (thanks Chris Antilla)
Better automatic generation of chords from pitch classes (thanks Frank Z.)
Triplet chords work in lilypond
midi.realtime works with PyGame on Python3.
Music21Object.next() and .previous() work well and FAST 99.99% of the time.  (see docs for the tiny case when it might be a problem)
Many routines that used to return tuples now return NamedTuples
Many deprecated routines removed — code is simpler to use
Working with multiple corpora now has “only one way to do it” — simplify.
.show(‘midi’) works in IPython.

Much more documentation