The Music
Plant is a media art piece that uses PC and electronic musical
instruments. This page introduces the concept and technical
information of Music Plant.
About Music Plant
The
Music Plant is a mysterious virtual life form
that grow with music and generates new music.
Play any song on an electronic instrument such
as an electric piano or guitar, and the plants
on the monitor will grow. When you finish
playing, a new song will be generated based on
the song you played and Playback will begin.
Before
you start playing, a primitive plant consisting
of only a stem is displayed in the center. As
you start playing, the virtual plant begins to
grow. The shape of the plant is not fixed, but
reflects your performance at that time. Hitting
a key on the digital piano adds a branch, and a
certain rule is assigned between the note on the
scale and the direction in which the branch
grows. There is also a relationship between the
duration of the note and the length of the
branch. Furthermore, if you hit two notes
simultaneously to create a chord, two branches
will grow simultaneously. The final shape of the
plant reflects the musical input. Some songs may
generate complex shapes, while others may
generate relatively simple shapes depending on
the musical style of the song.
A short time after you stop playing, the plants
will be redrawn and the song will begin playing.
When a branch is redrawn, the sound assigned to
that branch will be played, but this sound is
not the same as the sound when the branch was
created; it has been converted according to
certain rules. The branches are redrawn in the
same order as when they were created, but the
sound played when they are redrawn is the
inverse of the original pitch (high notes become
low notes, and low notes become high notes).
(Note: The function to reverse the time axis
that was available in previous versions has been
discontinued.)
In the Music Plant system, this music inversion
function is called "Music Mirror." The details
of Music Mirror will be explained later, but
what is important to note here is that the more
musical the original performance, the more
musical the tune that is played back when
redrawing. If you play it randomly, the music
that is returned when redrawing will also be
random.
The analogy of music plant offers a way to
visualize a performance through the process of
construction and deconstruction. For example,
pieces that are perceived to share musical
characteristics tend to generate similar plant
shapes. Reversing pitches can also help
determine the key elements of a "good tune," as
evidenced by the fact that many well-known
pieces still sound pleasant when played in
reverse.
Concept
(1) Visualize the structure of music by
representing the history of notes as a shape.
(2) Show that musical characteristics remain
intact even when pitches are inverted.
Musical Algorithm (Music
Mirror) - Music Inversion
As mentioned above, the
mechanism for flipping notes is called Music
Mirror. Specifically, it is the vertical and
horizontal flipping functions explained below.
Horizontal flipping reverses the time axis, so
the first note is played last and the last note
is played first. Vertical flipping reverses the
pitch.
Music Plant performs vertical flipping. (The
current version does not have a horizontal flip
function.)）
Vertical Inversion
Vertical flipping is like mirroring
a musical score upside down, where high notes
are played as low notes and low notes as high
notes. Music Plant adjusts the octave up or down
to prevent notes from sounding too high or too
low and sounding unnatural when flipped.
Original Melody
Vertical inversion of melody
By flipping the position of each
note in the chord, the chord structure is also
flipped. For example, if you flip the intervals
of a major chord, it becomes a minor chord. This
is because a "major chord" made up of a minor
third and a major third is converted into a
"minor chord" made up of a major third and a
minor third. For example, a C (triad) chord
becomes an Fm chord. A particular note (the C
note in the example below) is used as the pivot
point around which the inversion will take
place. (In actual processing, the Fm note will
be one octave lower.)
Vertical inversion of chord
As a whole, tunes in major keys are converted
into tunes in minor keys, and tunes in minor
keys are converted into tunes in major keys, so
bright music is played back as dark (melancory)
music and dark music is played back as bright
music.
The vertical Music Mirror processing can also
create different scales. For example, if you
apply inversion processing to the Okinawan scale
(Typical Asian scale), you will get the
Miyakobushi scale (Japanese Traditional scale)
that you might hear in Japanese samurai movies.
Vertical inversion of Scale
Diatonic Scale
Pentatonic Scale
Okinawan Scale and
Miyakobushi Scale
Whole tone scale
Vertical inversion - movie
Horizontal Inversion
In a
horizontal flip, the notes are played in reverse
order, just like mirroring left and right of
musical score. The first note played becomes the
last, and the last note played becomes the
first.
Original Medoly
Horizontal inversion of melody
Horizontal inversion - movie
This feature is not available in the current version.
Drawing Algorithms
- Plant Growth
When you strike a key, one branch is added to the plant for each
note, and the plant grows as you continue to play.
Each note in the scale (A, B, C, E, E, F, G and sharp and flat
notes in between) is assigned a specific angle, and when you
play the same note (including different octaves), branches at
the same angle grow.
As a result, the shape of the plant reflects the frequency of
each note in the song.
System
Configuration
The system of Music Plant consists of a
PC, monitor, and electronic instrument. Electronic instruments
include devices called MIDI controllers, such as electronic
pianos, guitars, and other devices that can send and receive
MIDI signals. If the MIDI controller does not have the ability
to produce sound, a device called a MIDI tone generator is also
required. The PC and electronic instrument are connected with a
USB cable. The left half of the monitor displays the input and
playback sounds in piano roll format, and the right half
displays a virtual plant.
When the first version of this work was released
in 2000, it used Microsoft's DirectX technology
for rendering. (This version was finally
exhibited in 2013.) Almost a quarter century
later,
when development of a new prototype began,
support for DirectX technology had ended, so the
entire work was redeveloped from scratch using
the same concept. After several trial and error
experiments and feature additions during the
development of the prototype version, extensive
modifications were made to complete version 2.0
for the exhibit at the "Seto Line Art Gallery
Project." The rendering portion was redesigned
to fit the event concept of the "Seto Line Art
Gallery Project," and several functional
improvements have also been made, including the
ability to play multiple instruments
simultaneously, such as piano and guitar.
As part of the modifications, the function to
reverse the time of the performance (horizontal
inversion) was removed.
