Music21 v.4 released (4.1.0)

The first release of music21 version 4 was released today (August 6, 2017).  This is the first major release of music21 in a year, and it brings with it a wealth of new tools for analyzing music with a computer and performing digital musicology, music theory, and composition.

Download from https://github.com/cuthbertLab/music21/releases or from the terminal, type:

    pip3 install --upgrade music21

(or without the "3" if you are using Python 2)

Version 4 is the last version of music21 that will support Python 2.7.  If you run Version 4 on Python 2.7, you will see a warning that it's time to move up to the brilliance that is Python 3.6.

As with all new "X" release names, v.4 has backward incompatible behaviors that I think are worth it for the great new features. 

Among the 272 commits since v.3.1:

Major new features:

• Graphing rewrite!

.plot() and Graphing has always been one of the most powerful parts of music21 since long before v.1.0 (mad props, Christopher Ariza!) but it's also been one of the most daunting aspects of using music21. It shouldn't be any more. The code has gone through a major rewrite to improve the simplicity of doing easy things and the power when doing difficult things. The easy things are documented in Chapter 22 and the hard things in Chapter 44.

• Local Corpora are great!

There's been a major rewrite of the corpus.corpora.LocalCorpus() function that makes it a fantastic way to work quickly with files you are working on from your own hard drive.  Set up your own local corpus, add paths to it, set the cacheFilePath to somewhere to store the metadata and you'll be able to search for pieces with particular features and metadata without needing to parse the score.

• Style!

the all new style module and style.Style object handles aspects of a note or other object's visual display that are not (usually) semantic. This class has allowed a major increase in the ability to properly preserve MusicXML visual formatting on input and export.

Style objects are created only when needed, so el.hasStyleInformation() allows for checking for the presence of a .style object without creating one.

(To be documented more soon)

• Major rewrite of TinyNotation allows for easy extensibility.

Documented in the User's Guide! Check it out!

• Always improving docs

The User's Guide goes up to chapter 24 now, with major new examples in Chapter 20 along with rewritten chapters on keys, time signatures, sorting, and so on and so on. Plus all examples are now Retina quality for viewing fine details of scores.

Documentation has been moved out of the music21 directory into the root directory -- it is no longer installed with music21 from pip -- this change was necessitated by the move to retina quality graphics, but reduces the installation size from 90MB to 15MB for the full corpus version and 6MB for the no-corpus version.  All documents are now tested with nbval, to ensure they stay up to date.

Other new features

• Stream.measures() now optionally allows for indexes (where 0 always is the first measure; -1 is the last, and so on) making .getElementsByClass('Measure') not necessary in most cases.
• much better metadata processing in musicxml, humdrum, and braille
• improved braille translation (tuplets) -- thanks Bo-Chen
• better beaming, meter, and tuplets in ABC
• output directly to PDF if MuseScore is installed.
• Nested Tuplets! including in MusicXML.
• Non-traditional key signatures
• New works by Clara Schumann in the corpus.
• stream.iterators.OffsetIterator() -- iterate groups of objects by offset.
• improvements to analysis.discrete
• demos/build_melody shows how to build MidiFile directly (thanks PeterMitrano!)
• corpus paths are now searchable in corpus.search()
• matplotlib and musescore graphics in Jupyter notebook are now retina quality.
• Chord.add() and Chord.remove() allow for direct manipulation of chords.
• Improvements to parallel processing in music21.
• Ottava spanners now come in two types, transposing and non-transposing -- reflecting whether the pitches under the spanner already reflect the transposition (non-transposing) or not.
• Palestrina humdrum has been reorganized and parses completely.
• Many improvements to spanners and RomanNumerals.
• Beams work much better and transfer in and out of MusicXML more completely.
• Every MusicXML 3.0 articulation is now supported.
• Core routines in stream.core have now been exposed publicly.  They are dangerous to use, but for anyone working on their own parsers, they can speed up insertions and appends by an order of magnitude.

Others, including bugs squashed:

• Warning on Python 2 that music21 v. 4 is the last version to support Py 2.
• Stream.template() is a great way to get an empty stream that otherwise matches the current Stream.  Replaces the obsolete .measureTemplate()
• ABC key signature and mode error fixed.
• RecursiveIterator gets a .currentOffsetInHierarchy, which can let even more places remove the dependence on .flat.  In fact, .flat uses the .recurse() method internally, because recurse() is now so fast.
• AudioSearch bugs fixed (thanks jjrob13)
• Chord.normalOrderString (thanks emzhang)
• Removed lots of old crutches including the "analysisData" on Stream, Note.editorial, and others. Style fixes most of this.
• fix to Bach BWV 386 (thanks alexcoplan) and to Beethoven Opus 59 no 3, movement 1.
• Note.pitches returns a tuple not list, just like Chord.pitches
• Converter can deal with some wrong file extensions now.
• Instrument reprs are fixed
• configure finds many more notation programs.
• ties are imported better between elements in and out of voices in musicxml
• configure works on macOS when user directory contains spaces.
• Bugs in ending and restarting a recursiveIterator fixed.
• doc errors fixed (thanks Andrew Sanchez)
• bestClef() has been moved to the clef module where it belongs.
• MusicXML sound tag now is placed properly (thanks Almog Cohen)
• MIDI output from transposed scores now plays in concert pitch.
• Breves are now acceptable as full measure rests.

Deprecations and deprecated elements removed

• (this list does not contain changes to the alpha/ directory which can change at any time)
• Note.ps, and Note.accidental, Note.pitchClass, Note.pitchClassString, Note.diatonicNoteNum, and Note.microtone are all deprecated. Use Note.pitch.ps, etc. instead.
• Chord.normalForm is deprecated because it gave the wrong answer. use normalOrder instead; same with normalFormString
• SpannerBundle.list is deprecated; use list(SpannerBundle) instead
• with the advent of .style, el.color is deprecated, use el.style.color instead
• Stream.stream() is deprecated -- now that the transition to iterators is done, there should be no need for this.
• REMOVED stream.getOffsetByElement; use s.elementOffset(el) instead.
• REMOVED stream.haveBeamsBeenMade; use stream.streamStatus.haveBeamsBeenMade
• REMOVED stream.makeTupletBrackets(); use stream.makeNotation.makeTupletBrackets(s)
• REMOVED stream.realizeOrnaments; use stream.makeNotation.realizeOrnaments(s)
• REMOVED VirtualCorpus -- it may return at some point but with a lot more features.
• nbconvert is no longer packaged with music21
• .exe files are no longer generated -- they were rarely used and pip is a better choice for Windows users now.

As the last version of music21 to run on Python 2, version 4 will have a longer support period for security patches and major bug fixes that render large parts of the system unusable for multiple users. This LTS (Long-term support) will run until the clock on python 2 runs out. (Currently two more years and eight months).  After that time, Python 2 will no longer be supported by the Python Foundation, and thus, by music21 either.  We're looking forward to joining the glorious Python 3-only feature and finally get to use some great features to make development faster and more stable.  Music21 v. 5 will be released at about the same time Django v. 2 (Py3 only) and a year after IPython/Jupyter made their first Python 3 only release.  Matplotlib will soon follow.  Python 3 is the future of humanistic and scientific programming.

As always, we thank the community for great support.  We'd always love to hear how you use music21 via the Google Groups mailing list.  Music21 was made possible by grants from the Seaver Institute, the National Endowment for the Humanities, and the School of Humanities, Arts, and Social Sciences / Music and Theater Arts Section at MIT.

Music21 v1.9 released

We are proud to release music21 v1.9.3, the latest and last release in the 1.x series.

There have been 147 commits in the two months since v1.8; here are some of the highlights:

• MUCH faster .getContextByClass (KUDOS to Josiah Oberholtzer for this). Even if you don't use .getContextByClass in your own code, you're definitely calling something that calls it. This method figures out where the most recent key signature, time signature, clef, etc. is for any given object, finds relationships between notes in different voices, etc. For analysis of medium-sized scores (say, 3 voices, 100 measures) expect a 10-fold speedup. For larger pieces, the speedup can be over 100-fold.
• A new stream/timespans module that makes the previous speedup possible by representing m21 Streams as AVL trees -- it's used in a few places (needs more docs), forthcoming releases will use it in a lot more places
• Python3 support (3.3 and later). The entire test/multiprocessTest.py suite passes on Python 3. N.B. to contributors -- from now on all contributions need to pass tests on both Python 2.7 and 3.3 and later. Negative -- in the past you could have made music21 run on unsupported older systems (2.6 and sometimes 2.5); now from music21 import * will fail on pre-2.7. 2.7 has been a requirement since Music21 1.7. Fewer than 30% of Macs still in use are running Lion or earlier and thus will need to update to 2.7. This version of music21 runs about 25% faster on Python 3 than Python 2, but otherwise no new features of Python3 are used. Python 2.7 will be supported throughout the Music21 2.x cycle so no panicking -- it'll be years (if ever) before Python 3.3+ is a requirement.
• Improvements to reductions of scores. And to analyzing voiceleading motion (some of this is backwards incompatible)
• Better, faster, and more consistent sorting of elements in a Stream
• Changes to the derivations module that I doubt anyone else was using anyhow...
• Removed obsolete files.
• Stafflines import and export from musicxml (thanks Metalmike!)
• Complete refactoring of converter.py to make it easier for users to write their own Subconverter formats (that can eventually be put into the system)
• Complete serialization of Streams via a new version of jsonpickle. This has big implications down the line; for now it affects...
• Vexflow output is much improved (unless you were counting on Voices; in which case do not upgrade) using the alpha version of music21j -- Javascript reimplementation of music21's core features.
• IPython improvements, allowing for robust and persistent communication between Javascript and Python. This will eventually (once I document it...) let you use the web browser as a UI for music21 python apps including live updating of music notation. It's too complex for most users right now, but I can attest that this will be one of the biggest perks of the 2.x development.

The usual bug fixes, documentation improvements and fixes, etc. are implemented. Thanks to MIT, the NEH, and the Seaver Institute for funding the project. (and to MIT for tenuring me in part on the basis of music21). This is the last release that Josiah Oberholtzer was lead programmer for; his considerable talents will still be on display in Abjad and many other projects he works on, and the implications of the new storage system he has developed will continue to pay off for years.

What's next?

Starting work on music21 2.0 today. That release will have some backwards incompatible changes that developers will need to deal with -- just as the path to 1.0 meant that some things that were originally thought of as good ideas were thrown out, the path to 2.0 will rely on 8 years of using music21 to fix some things that really should've been done differently from the beginning. Having just spent 2 weeks making m21 compatible with Python 3, I will give my assurance that as few incompatibilities as possible will be introduced. Most of the major changes will be on the core -- so if you've never messed with Sites, SpannerStorage, etc., you'll be fine.

• Problems with 5 quintuplets = .99999999 of a beat will disappear. Music21 2.X will store offsets and quarterLengths internally as rational numbers (actually a custom MixedNumeral class, so that the __repr__ is nicer...). All music21 objects will gain four properties: ".offsetRational, .duration.quarterLengthRational, .offsetFloat, and .duration.quarterLengthFloat" -- in music21 2.0, .offset and .duration.quarterLength will be aliases for offsetFloat and .duration.quarterLengthFloat -- so no changes will be needed to existing code. This will give a period of time (6 months?) to switch .offset either to .offsetFloat or .offsetRational. We'll have a tool to make the switch automatically. Then at a certain point, .offset will become an alias for .offsetRational. By music21 3.0 .offset will only support Rational numbers.
• Streams will store the position of notes, etc. in them. Right now this is all stored in the Note object itself. There are some great reasons for doing it that way, but significant speedups will take place by shifting this.
• inPlace will be False by default for all operations on Notes, Streams, etc. -- you can plan for the migration by explicitly setting inPlace for every call now.
• Some changes to boundary cases in .getElementsByOffset will take place -- it will not change much, but for a few users this will be crucial.
• NamedTuples and OrderedDicts will appear in a lot of places

that's all for now, but more examples to come soon. - Myke

Python reimports

We've been working a lot recently on two kinds of optimization in music21: improving speed and then using some of the speed increases to add functionality and stability, so that new features can be added without slowing down the process. One of the places we found where we could make changes is in our over-cautious use of imports. 

While everyone says that in Python you can import a module inside a function without it going through the overhead of actually reimporting, there is some real overhead still, especially if the function is called a lot of times:

Here I compare ten million calls to reference an object vs. doing the same while also importing a module that is already imported:

>>> from timeit import timeit as t # number = ten million; output in secs to 3 decimals

>>> t('x', setup='import weakref; x=5', number=10000000)

0.278

>>> t('import weakref; x', setup='import weakref; x=5', number=10000000)

7.810

So it's approximately two orders of magnitude slower than direct access alone.  Even with using the module and creating the weakref itself, the check-for-reimport timedominates five-fold over the creation of the weakref:

>>> t('weakref.ref(x)', setup='import weakref; from music21 import pitch; x=pitch.Pitch()', number=10000000)

2.098

>>> t('import weakref; weakref.ref(x)', setup='import weakref; from music21 import pitch; x=pitch.Pitch()', number=10000000)

9.823

for historical reasons (porting to systems without weakref, etc.) the “common.wrapWeakref” function of music21 (which does a try: except to see if a weakref could be made) did the import within the function.  Moving it outside the function sped it up considerably and made it only half the speed of calling weakref.ref(x) directly -- worth it for the extra safety--and only an order of magnitude slower than direct access to x itself:

before, with common.wrapWeakref doing a safety "import weakRef" call 

>>> t('common.wrapWeakref(x)', setup='from music21 import common,pitch; x=pitch.Pitch()', number=10000000)

17.112

after, without it:

>>> t('common.wrapWeakref(x)', setup='from music21 import common,pitch; x=pitch.Pitch()', number=10000000)

4.171

So this is the speedup in music21 that you'd find if you managed to grab the GitHub repository right now.  But we're planning on using the speedup to make things more functional.

As a practical consideration, one of the things that I’ve never been able to fix in music21 is the ability of elements embedded in a Stream to change their duration without telling their sites that things have changed for an element. There are expensive operations such as calculating that the length of a Measure, the last object, etc. which we cache as long as no .append(), .insert(), .remove() etc. are called.  But a Note inside the measure may have changed length so that the information in the cache is no longer accurate. I've been wanting to fix this for a while.

The problem is that the Note object itself has no idea that its duration has changed, because while the Note has a reference to the Duration, the Duration does not have a reference to Note -- it can't have a normal reference because this would create a circular reference (Note.duration = Duration; Duration.client = Note). With a circular reference, neither the Note nor the Duration will ever disappear even after they're not needed anymore, causing memory leaks. The obvious solution is to use a weak reference which behaves mostly like a normal reference but does not cause circular references. If the Note should disappear then the Duration.client weakref is not strong enough to keep the two objects alive.

With the speed increases, it should be possible to store a weakref on Duration and also Pitch to the object they’re attached to so that they can inform their “client” that they’ve changed.  The client can then inform its Sites (measures, etc.) that it has changed and clear the appropriate cache.  The extra overhead of creating the weakref ends up being only about 20% of object creation time; a small price to pay for the security of knowing that nothing can change and screw up the overall system:

>>> t('d=duration.Duration();', setup='from music21 import common,duration,pitch; x=pitch.Pitch()', number=10000000)

19.382

>>> t('d=duration.Duration(); pitchRef = common.wrapWeakref(x)', setup='from music21 import common,duration,pitch; x=pitch.Pitch()', number=10000000)

23.787

Expect to see more functionality like this in a forthcoming release of music21.

Abjad v.2 released

Woke up this morning to the great news that our friends Trevor Bača and Víctor Adán have released v.2 of their python-based, lilypond-powered, flexible music notation system Abjad (docs and installation instructions here). Abjad is a system for composers to build up scores from reusable, flexible elements and have precise control over notational elements. Some music21 users may have already noticed an "abj" directory in music21 and seen documentation at moduleAbjTranslate.html for how notes and simple streams can be translated from music21 to abjad. Since both projects use similar hierarchies including spanners and containers (Streams in music21), there is a lot of compatibility between the two. If a complete translator could be implemented (volunteers?), abjad would offer to music21 users high quality lilypond output and better tools for working with tuplets, staves with independent time signatures, and other rhythmic and layout tools that music21 does not yet have. Abjad users would gain access to music21's ability to parse many notational formats, work natively with intervals and harmonics, extensive scale collections (including Scala microtonal scales), and more. Getting these two projects closer to work more closely with each other is a win-win for everyone. Congrats to Trevor and Victor!

(Updated 2026-Feb: updated links. The music21/Abjad converter did not ever happen, unfortunately, and the alpha was removed.)