Durations and DurationTuples

If you don’t do a lot with Tuplets (Triplets, etc.) and have never heard of a DurationUnit, this is a post to skim. :-)

Music21 has the ability to consider two different incompatible ideas of a note as a note.Note (or note.Rest, chord.Chord, etc.): a note on a page and a note as heard.

(1) When we write a note on a page, it has certain properties: stems, flags, beams, dots, ties, tuplets, etc. Consider a half note tied to an eighth note.  On the page these are definitely two notes — we read them as two notes and then sound them together because of the tie.

(2) Now consider a single sound from a trumpet, at quarter note = 60, which lasts 2.5 seconds.  This is a note.  One note.  But try to notate it.  You’ll find that it takes two notes on the page to notate it: a half note and an eighth note, tied together.  But when we hear this sound, it sounds like one note.

How can we represent the second type of note in music21 even though it can’t be notated as a single note (well, not normally; more on that below)? Simple, we create a single Note object, which has a single Duration object.  That Duration object, however, has two elements in its “.components” list — a component corresponding to a half note, and a component corresponding to an eighth note.  The note’s overall duration has a type of “complex”.  When sending this Note out to MusicXML, Lilypond, etc., we split it into two notes (you guessed it, a half note and an eighth note).  We then look at something called “linkage” to see that each of these notes should be connected by a Tie.  (Rests have no linkage, for instance).  When we send it out to MIDI, on the other hand, we can leave it alone as one Note, since MIDI doesn’t support ties, but does support arbitrary lengths of notes.

So, this has been the music21 model since alpha 1, and in general it remains the model.

What has changed in the newest GitHub repository and will change in the next 2.X release is what these “.components” are.  Up until now they’ve been an object called DurationUnit — an amazingly flexible object created by Chris Ariza that can represent everything that a “simple” duration does; DurationUnits can have tuplets, dot-groups (an obscure medieval term), and just about everything else you can think of.  They’re extremely cool, and I’m going to miss them.

In music21 2.X, the components of a complex duration are called DurationTuples.  They are much simpler objects that only store three pieces of information: the type (‘whole’, ‘half’, ‘16th’, etc., plus ‘zero’), the number of dots, and the quarter length of that component.  They don’t have tuplets, dot groups, etc.  And they’re called Tuples because they derive from namedtuple which derives from the Python tuple object — in other words they are immutable.  Once a DurationTuple is set, it can’t be changed.  To change a note’s duration from whole to half, the DurationTuple needs to be deleted from .components and a new one created and inserted.  So they do everything a DurationUnit does and much less.

So, why the change?  Amazing flexibility and power, such as DurationUnits offer, comes at a price: speed.  And complexity.  The new DurationTuple makes creating the most common type of Duration with a single component much faster.  The amount of time to do: “d = duration.Duration(1.0); d.type” has been cut by over half.  This makes the creation of Notes about 20% faster than before (well, after the first check of durations), which is a pretty substantial improvement.  And as Dmitri and others have noticed, there are a lot of ways to change the duration of a Note that can affect other things, such as Streams.  This change reduces the complexity by making it so that only the Duration object itself can change its duration.  Changes to underlying .components are impossible to make since DurationTuples are immutable.

The only practical effect that most users are likely to see is in the use of Tuplets.  In the past tuplets lived on DurationUnits.  This meant for instance that a Duration could represent a single duration of “half-note-tied-to-eight-note-triplet” (QL: = 2.333333333).  Now, all the components of a Duration need to have the same tuplet or nested tuplets.  So this duration can be represented as a (dotted-half-note + eighth note) triplet.  Or it can be represented as (do the math, but only when you have time) a single whole-note in an 12:7 tuplet (because the latter is easier to determine for the computer to do, that’s what is done right now, but that could change).  

The other practical change in Tuplets is that there are generally three aspects to a Tuplet (well, four, but we’ll keep it simpler): the number of “actual” notes (3 for a triplet), the number of “normal” notes (2), and the durationNormal (‘eighth’, no dots, for instance).  In theory, once a tuplet was attached to a note, it became immutable (frozen), but because normal note was a DurationUnit, it was possible to create the tuplet and then to change the normal note type, or dots, or whatever.  Now that durationNormal references a DurationTuple, it is immutable; so instead of this:

   t = duration.Tuplet()

   t.durationNormal.type = ‘quarter’

or do this:

   t = duration.Tuplet(‘durationNormal’ = ‘eighth')

or if you must:

   t = duration.Tuplet()

   t.durationNormal = duration.durationTupleFromTypeDots("eighth", 0))

(this may go away, tuplets might become fully immutable in the future)

I hope these changes make using the system faster without much trouble.

Python reimports

We've been working a lot recently on two kinds of optimization in music21: improving speed and then using some of the speed increases to add functionality and stability, so that new features can be added without slowing down the process. One of the places we found where we could make changes is in our over-cautious use of imports. 

While everyone says that in Python you can import a module inside a function without it going through the overhead of actually reimporting, there is some real overhead still, especially if the function is called a lot of times:

Here I compare ten million calls to reference an object vs. doing the same while also importing a module that is already imported:

>>> from timeit import timeit as t # number = ten million; output in secs to 3 decimals

>>> t('x', setup='import weakref; x=5', number=10000000)

0.278

>>> t('import weakref; x', setup='import weakref; x=5', number=10000000)

7.810

So it's approximately two orders of magnitude slower than direct access alone.  Even with using the module and creating the weakref itself, the check-for-reimport timedominates five-fold over the creation of the weakref:

>>> t('weakref.ref(x)', setup='import weakref; from music21 import pitch; x=pitch.Pitch()', number=10000000)

2.098

>>> t('import weakref; weakref.ref(x)', setup='import weakref; from music21 import pitch; x=pitch.Pitch()', number=10000000)

9.823

for historical reasons (porting to systems without weakref, etc.) the “common.wrapWeakref” function of music21 (which does a try: except to see if a weakref could be made) did the import within the function.  Moving it outside the function sped it up considerably and made it only half the speed of calling weakref.ref(x) directly -- worth it for the extra safety--and only an order of magnitude slower than direct access to x itself:

before, with common.wrapWeakref doing a safety "import weakRef" call 

>>> t('common.wrapWeakref(x)', setup='from music21 import common,pitch; x=pitch.Pitch()', number=10000000)

17.112

after, without it:

>>> t('common.wrapWeakref(x)', setup='from music21 import common,pitch; x=pitch.Pitch()', number=10000000)

4.171

So this is the speedup in music21 that you'd find if you managed to grab the GitHub repository right now.  But we're planning on using the speedup to make things more functional.

As a practical consideration, one of the things that I’ve never been able to fix in music21 is the ability of elements embedded in a Stream to change their duration without telling their sites that things have changed for an element. There are expensive operations such as calculating that the length of a Measure, the last object, etc. which we cache as long as no .append(), .insert(), .remove() etc. are called.  But a Note inside the measure may have changed length so that the information in the cache is no longer accurate. I've been wanting to fix this for a while.

The problem is that the Note object itself has no idea that its duration has changed, because while the Note has a reference to the Duration, the Duration does not have a reference to Note -- it can't have a normal reference because this would create a circular reference (Note.duration = Duration; Duration.client = Note). With a circular reference, neither the Note nor the Duration will ever disappear even after they're not needed anymore, causing memory leaks. The obvious solution is to use a weak reference which behaves mostly like a normal reference but does not cause circular references. If the Note should disappear then the Duration.client weakref is not strong enough to keep the two objects alive.

With the speed increases, it should be possible to store a weakref on Duration and also Pitch to the object they’re attached to so that they can inform their “client” that they’ve changed.  The client can then inform its Sites (measures, etc.) that it has changed and clear the appropriate cache.  The extra overhead of creating the weakref ends up being only about 20% of object creation time; a small price to pay for the security of knowing that nothing can change and screw up the overall system:

>>> t('d=duration.Duration();', setup='from music21 import common,duration,pitch; x=pitch.Pitch()', number=10000000)

19.382

>>> t('d=duration.Duration(); pitchRef = common.wrapWeakref(x)', setup='from music21 import common,duration,pitch; x=pitch.Pitch()', number=10000000)

23.787

Expect to see more functionality like this in a forthcoming release of music21.