Thursday, January 15, 2015

Ten(+) Years of music21 (pt. 1)

An important milestone for music21 passed unnoticed by me a few weeks ago: ten years since the first time that code (well mostly documentation and tests) that still remains somewhere in the music21 codebase was run and did something useful (a scrambled loop for a composition project I was working on called Fine dead sound, after Faulkner).  It seemed worth saying a bit about the history of the project and things learned along the way.

Music21 started as a Perl project -- the oldest code I can find that even vaguely resembles the current music21  comes from November 2003, a 30k Perl script to label a Bach chorale with Roman numerals and correct keys (I still need to publish the algorithm somewhere since I haven't seen a better one). It had no graphical output except using David Rakowski's Lassus font. I was able to get it back up and running and through the modern miracle of web fonts, everyone (except IE users) can play with the oldest  music21  system at: If I remember correctly, chords in red had only a passing functionality, the small letters are the roots of chords and the second line of text is the locally active key. Pivot chords are labeled in brackets. This was done for an assignment in Avi Pfeffer's AI and Music class; others in my group had already produced the code (in C I believe) for removing passing tones from the score.

I had begun that project using Humdrum, a system I knew well and still have great admiration for, but I began to need to encapsulate more and more items into individual classes.  I googled around, because I figured someone would have already gotten far in an object-oriented replacement for Humdrum. But I found nothing, so when I got back to the project about a year later I started creating generalized objects which I incorporated into a package called "PMusic" for "Perl Music." (I think I first called it just "Music" but realized that this was going to be a problem.)  Here is part of the PMusic code for Chords, written on 12/31/2004.  Good documentation was already a music21 principle, but names of notes were still based on Kern/Humdrum:

=head1 PMusic::Chord (class)

class for dealing with chords

  $ch1 = PMusic::Chord->new(['CC', 'E', 'g', 'b-']);

Note double set of brackets, indicating a perl array reference.
Doesnt deal with rests yet.  Chords of more or less than four notes might
be odd...

Can also give an arrayRef of Note objects if you're that perverse...


package PMusic::Chord;
use strict;
use Carp;

sub new {
  my $proto = shift;
  my $class = ref($proto) || $proto;
  my %args;

  ## if only one arg, assume it's an arrayref of Note names
  if (scalar @_ == 1) { $args{Notes} = $_[0] }
  else { %args = @_ };
  my $self  = \%args;
  bless $self, $class;

### for now, first note is always bass of chord (does tenor ever go below bass? sometimes!)

=item NoteObjs()

Returns an array ref of PMusic::Note objects


sub NoteObjs {
  my $self = shift;
  return $self->{NoteObjs} if $self->{NoteObjs};
  my @objs = ();
  foreach (@{$self->{Notes}}) {
    if (ref($_)) { push @objs, $_; }
    else         { push @objs, Note->new($_); }
  $self->{NoteObjs} = \@objs;

=item Bass([note PMusic::Note])

returns the bass note or sets it to note.  Usually defined as the first noteObj
but we want to be able to override this.  You might want an implied
bass for instance...  v o9.


sub Bass {
  my $self = shift;
  $self->{Bass} = $_[0] if defined $_[0];
  $self->{Bass} ||= $self->NoteObjs->[0];
  return $self->{Bass};


At this point I was living in Rome at the American Academy as a Rome Prize fellow in Medieval Studies, and halfway through my fellowship I realized that none of this code was going to write a six-hundred page dissertation on fourteenth-century music fragments, and besides, I had job interviews to prepare for, and later, students to teach at Smith College, Mount Holyoke College, and then MIT, none of which (yes, including MIT) cared that I was working on generalized software for musical analysis as a side project. I also saw how long this project was going to take.  And besides, I was still convinced that someone had already written a generalized OO toolkit for musical analysis, I just wasn't searching well enough.  So the project was put aside for almost two years before it reemerged in Python, as the (to be written) Part 2 explains.

Sunday, January 11, 2015

Music21 v.2.0.0 (alpha) Released

We're happy to announce that the first public alpha of music21 v.2 has been released!

Version 2 is the first version of music21 since v.1 to make substantial changes in the code base that introduce backwards incompatibilities in order to make going forward faster and smoother.  It doesn't change anything super fundamental à la Python 3's print function, so most code should still run fine, but definitely test in a separate environment before upgrading on any code you have that needs to run without problems.  The system is still changing and more backward-incompatible changes could be included until v.2.1.

We have had 420 commits since the last release, so there is a lot that is new!

Substantial changes include:

  • Offsets and quarterLengths are now stored internally as Fractions if they cannot be exactly represented as floating point numbers. A lot of work went into making this conversion extremely fast; you probably won't ever notice the speed difference, but you can now be sure that seven septuplets will add up to exactly a half note.  For instance:
  • >>> n = note.Note()
    >>> n.duration.appendTuplet(duration.Tuplet(3,2))
    >>> n.fullName
    'C in octave 4 Quarter Triplet (2/3 QL) Note'
    >>> n.quarterLength
    Fraction(2, 3)
    >>> n.quarterLengthFloat # if you need it...
  • Converter structure has been overhauled for more easily adding new converters in the future. If you've wanted to write a converter or already have one for a format not supported but have been daunted by how to include it in music21 now is a great time to do it.  Speaking of which...
  • MEI format is supported for import (thanks to Chris Antila and the ELVIS team at McGill university for this great enhancement)
  • Python 2.6 is no longer supported. All tests and demos pass and run on Python 2.7, 3.3, and 3.4. (3.2 and older are not supported)
  • FreezeThaw on Streams works much better and caching loaded scores works great (some of this was included in 1.9, so definitely upgrade at least to that.
  • Much improved Vexflow output using music21j, Javascript/Vexflow rendering engine. Was in 1.9, but improved here.
  • Lots of places that used to return anonymous tuples, now return namedtuples for more easily understanding what the return values mean.
  • Integrated Travis-CI testing and Coverage tests will keep many more bugs out of music21 in the future.
  • Many small problems with Sorting and stream handling fixed.
  • Corpus changed: for various licensing reasons, v.2.0 does not include the scores from the MuseData corpus anymore. This change mostly affects Haydn string quartets and Handel's Messiah.  However, new replacement scores are being included and 2.1 will have as many great works as before.  The MuseData scores are still available online.  MuseData is now a deprecated format and no further testing on it will be conducted; only bug fixes that are easily implemented will be accepted.
  • music21 is now available under the BSD license in addition to LGPL! 

We will try to stick to something close to the semantic versioning model in the future once v.2.1 is released. In other words, after 2.1, we'll try very hard not to break anything that worked in v.2.1 until v. 3.0 is released.  This will probably mean that the version numbers are going to creep up faster in the future.

Still todo before v.2.1 is a major change in how elements are stored in Streams.  Stay tuned if you care about performance tweaks etc., otherwise ignore it -- we'll keep the interface the same so you might not notice anything except speed improvements.

Smaller backward-incompatible changes include:
  • Stream __repr__ now includes a pointer rather than a number if .id is not set.  This change will make filtering out doctests easier in the future.
  • TinyNotation no longer allows for a two-element tuple where the second element is the time signature.  Replace: ("C4 D E", "3/4") with ("tinynotation: 3/4 C4 D E")
  • Obscure calls in SpannerBundle have been removed: spannerBundle.getByClassComplete etc.
  • Convenience classes: WholeNote, HalfNote, etc. have been removed.  Replace with Note(type='whole') etc.
  • Old convenience classes for moving from Perl to Python (DefaultHash, defList) have been removed or renamed (defaultlist) 
  • Articulations are marked as equal if they are of the same class, regardless of other attributes.
  • common.almostLessThan, etc. are gone; were only needed for float rounding, and that problem is fixed.
  • duration.aggregateTupletRatio is now aggregateTupletMultiplier, which is more correct.
  • scala.ScalaStorage renamed scala.ScalaData
  • common.nearestMultiplier now returns the signed difference.
  • layout -- names changed for consistency (some layout objects had "startMeasure" and some "measureStart" - now they're all the same); now all use namedtuples.
  • rarely used functions in Sites, base, Duration, SpannerStorage, VariantStorage, have been removed or renamed.  I'd be surprised if these affect anyone outside the development team.
Improvements and bug fixes:
  • common.mixedNumeral for working with large rational numbers
  • common.opFrac() optionally converts a float, int, or Fraction to a float, int, or Fraction depending on what is necessary to get exact representations. This is a highly optimized function responsible for music21 working with Fractions at about 10x the speed of normal Fraction work.
  • Rest objects get a .lineShift attribute for layout.
  • staffDetails, printObject MXL had a bug, writing out "True" instead of "yes"
  • staffLines is now an int not float. (duh!)
  • better checks for reused pointers.
  • lots of private methods are now ready for public hacking!
  • Lyric.rawText() will return "hel-" instead of "hel" for "hel-lo".