A Virtual Piano Concerto

Coupling of the Mathews/Boie Radio Drum and the Yamaha Disklavier Grand Piano in "The Seven Wonders of the Ancient World"


David A. Jaffe                                                                         W. Andrew Schloss

295 Purdue Ave.                                                                    School of Music

Kensington, CA                                                                     University of Victoria

94708 U.S.A.                                                                            Victoria, B.C. V8W 2Y2

david@jaffe.com                                                                    CANADA




1   Introduction


We describe here the process of collaboration that went into the creation of a The Seven Wonders of the Ancient World a seventy-minute, seven-movement concerto by Jaffe, scored for remote-control piano and an acoustic ensemble of eight instruments.  The solo piano part, written specially for Schloss and developed in collaboration, for the Yamaha Disklavier C7 Grand Piano Mark II (a piano that can "play itself" under computer control) and the Mathews/Boie Radio Drum [Boie et al, 1989] (a device that conveys three-dimensional gestures to a  computer.)  The Drum and Disklavier are connected via a Macintosh computer running software created with MAX (Puckette/Zicarelli.)  Thus, the final result of this work is entirely acoustic.


The piece was developed in four stages: initial research on the "percussionistic" use of the Drum (Schloss), exploration of the Disklavier and of the Drum-Disklavier interaction (Jaffe, with extensive input from Schloss), composition of the work itself (Jaffe), and a period  of refinement of the solo part (Schloss, with extensive input from Jaffe.)  This mode of collaboration was markedly different from the authors' previous collaboration, Wildlife[Jaffe and Schloss, 1992 and 1994], a collaboratively-composed and -performed work  [Jaffe and Schloss, 1994].  In this paper, we distinguish between the contributions of the authors by referring to them as "the composer" (Jaffe) and "the performer" (Schloss).  The reader is requested to keep in mind that this division simplifies to some extent the actual collaboration that took place, as well as the fact that in a quasi-improvisational context the boundary between composition and performance is somewhat blurred.


The first four movements of The Seven Wonders have been completed and were recorded at the Banff Centre for the Arts in 1993 and premiered at the University of California at San Diego in May 1994.  These three movements, taken as a group, are known as "Part I" of the work.  The remaining three movements are in progress.


2   The Seven Wonders of the Ancient World


Two statues, a temple, a roof-top garden, two tombs and a lighthouse.  This rather odd collection of monuments has come down through history as "The Seven Wonders of the Ancient World."  All but one (the Pyramids) have been destroyed, either by Nature or by human hands.  A closer look at the Wonders reveals a crosshatch of parallels and oppositions.  Two deal with death the Pyramids and the Mausoleum.  The Hanging Gardens glorify cultivated nature, while Artemis was the goddess of wilderness and wild animals.  The two statues are of the heavens Zeus, the god of thunder and rain; and the sun god of the Colossus of Rhodes.     How can the essence of these monuments be conveyed in music?  In searching for an answer, the composer turned to the Disklavier and the Radio Drum, the latter used in a percussive manner.  Note that this differs from the conductor paradigm followed by Max Mathew's "Conductor" software [Mathews, 1988], and used in the composer's Terra Non Firma [Jaffe, 1994].       


The sound of the Drum-Piano is further expanded by an unusual ensemble consisting of instruments that extend the piano's sound five plucked string instruments (harp, mandolin, guitar, harpsichord, and contrabass), two percussionists who play primarily pitched instruments, and one instrument capable of sustained sound (harmonium).  In addition, the contrabass sometimes plays arco, providing another sustained sound.  Since these instruments are used in a predominantly orchestral role, we refer to them as the "orchestra," despite the relatively small size of the ensemble.  Finally, an improvisational approach to extended cadenzas in the solo part allows the performer to respond and communicate intimately with his unusual instrument.  The result is a new kind of piano concerto.


3   Methodology


            3.1   Building an Instrument, Part 1: The Controller       


The Radio Drum part is based on the foundation developed by Schloss during the period 1988-1990 [Mathews and Schloss, 1989].  After much experimentation, the percussionist's apparatus stabilized as follows:  He performs on the Radio Drum with two mallets, allowing six degrees of freedom, while his feet control an Elka MIDI organ foot-pedal, a polyphonic device consisting of 18 velocity-sensitive pedals.   Both devices send information to a Macintosh computer.  Each of the Drum mallets may be in one of two modes.  In "whack mode", the Drum sends data each time its surface is struck (more precisely, each time the mallet changes direction.)  In "continuous mode", the Drum sends position data, whenever it is polled by the computer, usually continuously.


3.2   Building an Instrument, Part 2: Radio Drum Meets Piano


Work on this piece began with a series of experiments (or "sketches") in combining the Drum and Disklavier, during the summer of 1992 at the Banff Centre for the Arts, where the authors were Resident Artists.  The composer discovered that the flexible and seemingly-magical mapping of percussion gestures onto piano mechanism makes possible the grand, monumental, yet very uncharacteristically "pianistic", sounds the Seven Wonders idea called for.  We created a set of quasi-improvisational scenarios, each with a strong immutable identity, but allowing the performer an opportunity to interact with a strong sense of immediacy with the Drum-Piano [Schloss and Jaffe, 1993].       


A variety of approaches were used in exploring the rich space of possibilities.  First, the composer explored compositional ideas on the Disklavier alone, focusing on effects that would be difficult or impossible if the piano were played in the conventional manner.  Later, the parameters were refined and mapped onto the Drum.  Since the effect of a given gesture on the Drum is entirely up to the composer, the Drum often controls aspects of compositional algorithms.       


Other effects were conceived with the Drum itself in mind.  One of the most intriguing aspects of the Drum-to-Piano transformation is that it enables the idioms, conventions and vocabulary studied by percussionists to be applied to the alien world of piano sound production.  In one case, we created a "drum set" in which regions of the Drum surface correspond to musical gestures on the piano.  For example, one region might play a rapid trill, another might play a staccato chord, a third might play a quick series of notes.  The gestures are all short enough to allow the performer to play the Drum as if it were a trap set, but one in which each "instrument" is actually a piano motive.  Often, the composer would implement a configuration, the performer would try it, and then the performer would suggest a modification (see section 4.1).  Occasionally, this interaction was reversed, as in the "rhythmic grid" of movement III (see section 3.3)which was suggested by the performer, then implemented and refined by the composer, who played the Drum part himself as a way of exploring its potential.  This illustrates the fluidity of roles typical of the collaborative process, with the performer assuming aspects of the composer's role and vice versa.


3.3   Compositional Strategy


The third phase involved the composition of the orchestral parts and was carried out by the composer alone during the period January 1993-May 1993.  He returned to California armed with a set of some twenty completely different approaches to interaction between the Drum and Piano, as well as several hours of recordings of improvisations with various Drum-Piano configurations.   He then began structuring this material into a concerto, proceeding by selecting one or two highly-characteristic approaches for each movement of the piece and structuring the movement around them.   Examples are given below from the first four movements.       


The first movement, The Pyramids, is based almost entirely on 88-note effects in which every note of the piano is played on each gesture (see section 4.1).  The orchestral material is complementary, much of it based on chromatic tone clusters.  For example, an extended passage for harpsichord, mandolin, guitar and harp rhythmically explores an 18-note tone cluster melody, with each instrument playing some portion of that cluster and changing which portion it plays as the melody moves.  The harmonium and arco bass also figure prominently in this movement, playing tone clusters and minor seconds that swell and fade, growing and contracting.  The second half of the movement is an extended Drum-Piano solo with commentary by the timpani and glockenspiel, leading eventually to a climax whose resonance is allowed to ring throughout a coda section, in which this resonance is taken up by the harmonium, guitar, low harp and bass playing harmonics.      


The second movement, The Hanging Gardens of Babylon, is built on a simple melody, ornamented in a variety of ways.  Here, the Radio Drum controls a "time map" that influence show far cascading melodic strands drift with respect to one another, floating between homophonic and canonic textures.  The orchestral material is similarly conceived, with melodies that proceed with like melodic contour, but not in lock-step synchronization.       


The third movement, The Statue of Zeus in the Great Temple of the Sacred Grove, is based on pulsations at various conflicting tempi.  The Drum performer improvises in the context of polyrhythms by moving within a "metrical grid," performing cross-rhythms and creating  metric modulations as each new rhythm is "latched" to become the dominant pulse.  The orchestral music is similarly based on conflicting pulsations.  At one point, the orchestra's multi-rhythmic patterns suddenly become homogeneous as the Radio Drum breaks into an improvised solo that attempts to contradict the pulsations of the orchestra.       


The fourth movement, The Colossus of Rhodes, is entirely based on trill-derived material, performed by the Radio Drummer (see section 4.2)   and taken up by the orchestra using a technique of gradual disintegration of ensemble, in which the conductor begins conducting and then stops, allowing the instruments to drift out of synchronization until they all arrive at the final trilled fermata [Brant, 1980].  Eventually, the trills morph into a wild fiddle tune, performed by the Radio Drummer over a string band accompaniment,  eventually disintegrating into a rhythmic chaos that finally leads back to the trills.


3.4   Shaping the Performance


The final phase involved refining the improvisational cadenzas and structuring the software into a series of sequential cues.  This work was carried out at CNMAT (U.C. Berkeley) and the Banff Centre for the Arts in July 1993.  Before the solo part could be shaped, we had to devise a system of sequential cues so that the various configurations of the Radio Drum could be accessed as specified in the score.  This was done by introducing a global "current cue" variable that selects the appropriate functionality.  One pedal is used to advance to the next queue, while another is reserved for stepping back a queue for rehearsal purposes.  There is also a graphical user-interface that allowed for random access to any queue.      


 Shaping the solo part required extensive practicing with each of the Drum-Piano's improvisational scenarios.  The performer began by executing the instructions in the score as written, asking questions of the composer when necessary.  In most cases, coaching by the composer, plus long sessions in which the performer alone explored the possibilities of the scenarios, were sufficient to refine the solos.  Sometimes the performer would add detailed annotations to the composer's general instructions, to remind himself to focus on aspects of the improvisational space that he found particularly effective.  In one or two places, these additions took on a structural significance and were added to the score itself.  The final result still left the performer a good deal of room to improvise in the performance itself, but with "guide-posts" enabling him to find his way and allowing the conductor to fit in the orchestral part.


4   Challenges and Opportunities


A number of challenges were involved in the creation of this piece.  Each posed a problem, but also presented opportunities for innovation.  The first was to discover ways of using the Drum-Piano combination idiomatically so that the strengths of each instrument are maximized, while the weaknesses are minimized.  The second challenge was to work within the physical limitations of the Disklavier, which are entirely different from those of a synthesizer or sampler.  In approaching this issue, we were led to some fascinating algorithms that we would have been very unlikely to find had we not been required to work within the constraints of the piano mechanism.  A third challenge was to provide the performer with enough freedom to take advantage of the expressive potential of the RadioDrum, while remaining in the context of a fully-composed concerto.



            4.1   What is "Idiomatic" for the Drum-Piano?


The Radio Drum is excellent for playing rapidly and for controlling multiple independent variables at the same time.  However, it is not very good at random access to a large set of discrete values, such as the pitches of a piano.  Therefore, when exact pitches were required, we used a "sequential drum" mode, in which each note of a stored sequence is splayed each time the Drum is struck.  A different version of the sequential approach is used in "The Hanging Garden of Babylon."  Here, a pre-composed melody is played, duplicated at several transposition levels, while the Drum controls the drifting of the voices with respect to one another, using a real-time performance "time map" [Jaffe, 1985].  The Drum's  left mallet is in continuous mode here and implements the following mapping:  The x axis determines how far the voices move out of synchronization.  As the performer moves the mallet to the right, the voices move from unison rhythm to a kind of heterophony, and eventually to a canonic texture.  The y axis controls dynamics.  The right mallet restarts the melody.       


Similarly, the piano is constrained by a number of factors, not the least of which is the geometry of the human hand.  In an effort to exploit the Drum-Piano combination in an idiomatic manner,  we explored techniques that would be awkward or impossible if the piano were played in a conventional manner.  For example, for "The Pyramids", the composer was interested in creating ponderous massive blocks of sound, each comprising all 88 notes of the piano.  We devised a mapping of the Drum in which all 88 notes of the piano are played each time the Drum surface is struck with the left mallet, with the speed and loudness of these notes corresponding to the y  value and how hard the Drum is struck, respectively, and the  x axis controlling the "correlation" of these notes as follows:  The right side of the Drum corresponds to the most correlated effect a single 88-note chromatic glissando.  The left side of the Drum corresponds to the least correlated effect 88 notes in random order.  Between these points are such combinations as 4 groups of 22-note glissandi, 8 groups of 11-note glissandi, etc.  Multiple instances of this process can be active at once.        


The score calls for the performer to improvise with this mapping for several minutes.  After extensive experimentation, we decided the performer needed more control over the configuration.  This led to a modification of the original plan:  As originally implemented, the direction of each glissando was randomly determined.  However, the performer wanted to be able to create multiple cascading glissandi in the same direction.  Therefore, wemodified the algorithm to allow assignment of glissando direction from a foot pedal.  Also, the performer found that slow 88-note gestures could continue for an annoyingly long time, leading to an undesirable plodding effect; he wanted a way to stop the gesture midstream.  So we gave the right mallet a special role; it stops the most recently-begun instance.  For example, the performer can start a slow 88-note glissando, play several other gestures on top of it and then stop several of the latter gestures while the initial gesture continues.    This mapping allows for a great deal of improvisational variety, despite the severe constraint that each Drum stroke produces 88 notes.


            4.2   Physical Limitations of the Piano as a Source of Inspiration


The physical limitations of the Disklavier include the duration required for a

 piano key to bounce back before it can re-strike the strings; the number of notes that can be played simultaneously; the maximum duration a note or the pedals can be held down; and thedelay between the reception by the piano of a MIDI message and the sounding of the corresponding note, which depends on the velocity with which the string is struck.  The process of working around these limitations led to discovery of an interesting set of effective mappings from the Drum to the Disklavier that we would have never found were not working within the confines of the piano's limitations. 


For example, the score called for an improvisational scenario in which the Drum's X axis is mapped to pitch and the performer's left hand played four-octave chords in a D major scale, while the right hand played the same chords in a D minor scale.   As soon as the performer started playing quickly, the limitation in the piano's action became problematic.  The solution we discovered was to constrain the piano to never repeat the same pitch twice.  If the performer attempts to strike the same location on the Drum surface twice, the pitch "dithers" in the vicinity of the pitch corresponding to the location where he struck. This dithering is in the context of the scale of the hand with which the performer played. For example, if the left hand plays a G and the right hand then strikes the same location on the Drum, the resulting pitch is F-natural.   However, if the right hand plays a G and the left hand then strikes the same location on the Drum, the resulting pitch is F-sharp.  This simple variation on the original design was an enormous improvement.  The performer could now play as fast as he wanted and the piano kept up beautifully.  More significantly, the resulting musical effect was substantially more effective and intriguing than that of the original design.  This is especially apparent when the Drum is switched to the continuous mode.  In this mode, even when the mallet is held in one position, an interesting constellation of  pitches result.  If, as the score calls for, the mallet is swept slowly from left to right, the effect is a generally-ascending set of gestures, but with a local elaboration that greatly enhances the melodic interest.


            4.3   Performance Freedom in the Context of a Fully-Composed Concerto


Since the Drum is a fully-configurable instrument, it is up to the composer how it is used to affect a musical result.  In The Seven Wonders of the Ancient World, it is used in a wide variety of ways that often cross the boundary where expression meets improvisation.  The model that seems to come closest to the way we worked is that of  the concerto with improvised cadenzas.  The programming of the Drum is where the composer and performer meet.  The composer provides a virtual world or language in which the performer can express himself.  The freedom is not limited to unaccompanied passages, but appears often in conjunction with the orchestra.  In such cases, the composer was especially careful to guarantee that no undesirable orchestrational collisions could result.       


For example, in "The Colossus of Rhodes", the piano plays twelve simultaneous trills.  One trill is considered "selected".  The selected trill is chosen continuously according to the performer's left hand X location.   The speed and loudness of the selected trill corresponds to the left hand z location as the hand approaches the surface, the trill speed and loudness increase.  The right hand controls the speed of the non-selected trills. The score includes passages where the approximate range of the selected trill pitch isspecified, along with the dynamics, while it is up to the performer to select an appropriate trill pitch.  In other passages, the performer has freedom to improvise in the context of this highly-constrained "trill world."


5   Summary and Acknowledgments       


Mappings from percussive and continuous gestures to piano sound production make possible a new realm of piano music.  The approach of using quasi-improvisational scenarios for the Radio Drum part can allow a composer sufficient influence over musical materials, while simultaneously allowing the performer freedom to realize the potential of his instrument.  Work on this piece was supported in part by a Collaborative Composer Fellowship fromthe N.E.A., a federal agency. Further support was provided by the Banff Centre.  Special thanks to Yamaha for providing the Disklavier used in developing the piece.  Thanks also to Max Mathews and Bob Boie for their marvelous inventions.



6   References


Boie, R.A. et al.  Gesture Sensing via Capacitive Moments.  Work  Project No.311401-(2099,2399) AT&T Bell    Laboratories, 1989.


Brant, H.  Personal communication, 1980.


Jaffe, D. and A. Schloss.  The Computer-Extended Ensemble.  Computer Music Journal, 1994 .


Jaffe, D.  Ensemble Timing in Computer Music.  1985.    Computer Music Journal, 9(4):38-48.


Jaffe, D. and A. Schloss.  Wildlife. CDCM Vol. 15, 1994


Jaffe, D. Terra Non Firma.  Ibid.


Jaffe, D. and A. Schloss.  The Making of Wildlife.   Proc. of the 1992 ICMC.


Mathews, M.V.  and A. Schloss  The Radio Drum as a Synthesizer Controller, Proc.ofthe 1989 ICMC.


Mathews, M.V.  The Conductor Program and Mechanical Baton.  1988.  Current Directions in Computer Music, MIT Press.


Schloss, A. and D. Jaffe.  Intelligent Musical Instruments...   Interface, 22(3):183-1