Multimediality

Multimediality of Hieronymous Bosch's Garden of Earthly delights

Approaching arts – and human experience in general – from the perspective of multimediality can be fruitful in opening different ways of understanding these experiences. We perceive the world first through our bodily senses and then construct various understandings and experiences of this sensory data through complex cognitive processes. While much of these experiences are non-conceptual I will here discuss mostly the ways in which music is conceptualised by using terminology from other artistic media. I. e we understand – or communicate our understandings of – music using words originally, or more often, used to describe other artistic media or realms of human experience.

Multimediality in music begins with one of the oldest ways of music-making; singing.  Although, as discussed before, singing may actually have preceded language and been a sort of “protolanguage”, singing as we usually think of it includes text, lyrics.

Intermediality and intertextuality

Multimediality cannot really be discussed without also addressing some neighbouring terms. Intertextuality became hip in the academic discussions of arts since at least in 1980s. It’s a helpful tool in analysing and understanding the ways in which meanings are created in multifaceted ways by various techniques such as quotation or some sort of reference. As discussed before, these techniques have been central to black American music-making since the times of slavery to the contemporary hip-hop.

Intertextuality tends to fall short when applied to performing arts. While there are certain benefits in reducing everything to “texts”, two dimensional layers of meaning, this comes with a cost when studying music as a performative phenomenon, e.g. through Christopher Small’s “musicking”. Multimedality is a more helpful concept in helping us study and understand how different artistic media are used, and can be used, to reflect and create rich human experiences by drawing from the tools and strengths of the different media in our disposal.

Multimediality in music

Multimediality in music is an old idea as music has always been a part of some “extramusical” performance or context such as a ritual. In fact “pure” music is one of those 19th century Romantic ideas still to some extent holding our experience of music captive. But more about that another time.

The gesamtkunstwerk of Richard Wagner‘s opera remain perhaps the most iconic examples of effort to bring all the art forms together. Whereas Wagner’s operatic works might stand as the ultimate artistic expression of modernity, the 21st century post-modern artists produce more fragmented works.

Whereas black American music has got from the cotton fields to White House (see below), western Classical music is now performed by native orchestras and singers all over the world – here also conducted by a woman, something which in Wagner’s time was quite unthinkable. Multimediality here includes also video projections and TV production.

Earlier I discussed how Jacob Collier presents his multifaceted talent in his YouTube videos and how Janelle Monáe implies multiple – or perhaps fragmented – identities in her performances of the song Tightrope with means of music production, the “music itself” (e.g. melody, harmony, groove), lyrics, video, live performance, etc. The Dutch group Tin Men and the Telephone is also a very interesting example of musical art that draws from multiple media in a very interactive way on and off stage.

Janelle Monáe’s performance in the White House by Barak Obama’s invitation has various multimedial layers. As discussed earlier, her performance style is rich in references to other black American artists, perhaps most notably in the James Brown steps in her dance moves. In this performance the “Funkiest horn section of Metropolis” becomes that of White House, opening up a myriad of interpretations.

Here’s Jacob Collier embracing the social medium of music making in a contemporary digital manner enabling music-making together across temporal and spacial boundaries.

Tin Men and the Telephone do various things with different media from “musicalising” recorded speech and other sounds to typing with the piano keyboard and collaboration with their audience through a special app.

Music in literature – Toni Morrison’s Jazz

One interesting form of multimediality is that of music in literature; the use of description of music in literature and use of musical techniques in writing. Describing music in words requires quite an effort from the writer and reader alike to convey and share an artistic experience across the media. To describe art of one medium with the means of another requires sharing cultural understanding on a deep level and the ability to imagine, in this case, music described with words.

One interesting form of multimediality is that of music in literature; the use of description of music in literature and use of musical techniques in writing. Describing music in words requires quite an effort from the writer and reader alike to convey and share an artistic experience Jazz by Toni Morrison, 1st edition cover across the media. To describe art of one medium with the means of another requires sharing cultural understanding on a deep level and the ability to imagine, in this case, music described with words.

When I first tried to read Toni Morrison’s Jazz, in the age of around 15 or so, I expected it to be “about jazz”. I didn’t understand much about it and quickly gave up.

Source: https://www.amazon.co.uk/Jazz-Toni-Morrison/dp/0679411674?SubscriptionId=AKIAILSHYYTFIVPWUY6Q&tag=duckduckgo-ffsb-uk-21&linkCode=xm2&camp=2025&creative=165953&creativeASIN=0679411674
Jazz by Toni Morrison, 1st edition cover

Recently I picked up the book again and was better able to appreciate the ways in which Morrison took jazz as a metaphor and method and used its compositional and performative techniques to tell the story of her book.

Source: https://www.abebooks.co.uk/servlet/BookDetailsPL?bi=15926347144
Amiri Baraka’s Blues People elaborates on the idea of music as the history of black Americans.

Like a jazz performance the book has a main theme, a story it wants to tell. However, the main characters are also given “solo spots” to elaborate on their personal stories giving depth to the main story and enabling the reader to approach – perhaps even experience – the story from the individual perspectives of the characters; much like in jazz performance the “tune” is approached differently by each of the soloists.

Jazz in Morrison’s book is also a metaphor for the black American struggle and experience. As briefly discussed before, jazz has come a long way from an unappreciated folk music symbolising the worst of human kind – even among some black Americans – to be heralded as the “American Classical music”. Whereas Amiri Baraka in his Blues People elaborated on the idea of “music as the history of black Americans”, Morrison gives the bones of this history the flesh of her characters.

At the time I’m typing this the first black American president has just stepped aside to make space for yet another white male, one whose rhetoric and first deeds clearly show how the struggle for human rights is far from over. Morrison’s story takes place in a period prior to the Civil Rights era when many – as some of the characters in the book – still had vivid personal memories of the Jim Crow treatment of blacks.

Music and visual arts

The painting on top of this article is the Garden of Earthly Pleasures by Hieronymus Bosch from 1500.  As sound is difficult to picture music in visual arts is mainly pictured through instruments and musical acts such as dancing and singing. Bosch’s painting is a classic one portraying music as a sinful – or at least not respectable – activity through placing some instruments of the time together with people busy with Earthly orgies.

The pianist Bill Evans wrote liner notes for the 1959 Miles Davis quintet album Kind of Blue, I’ve also discussed earlier. In his text Evans makes an analogy between the Japanese calligraphy shodō and jazz improvisation. He stresses the temporal nature of both media; just as the stroke of a brush leaves its mark on the paper and cannot be undone or altered, a musical sound cannot be taken back. Further challenge in jazz improvisation is the group setting in which it most often happens; there are in fact many “brushes” making strokes simultaneously to the “canvas” of temporal framework set, in this case, by Miles Davis.

Musical (un)talent

H. Jon Benjamin. Source: http://media.npr.org/assets/img/2016/01/19/jb_wide-c72b0ed1d2cabcc75fcc50b5b8affd24b8813315-s800-c85.jpg

I’ve previously touched the topic of musical talent in my discussion about the young Jacob Collier. I recently came across a sort of opposite case in terms of musical talent. H. Jon Benjamin is an American voice actor and comedian who’s got into the music world from the “wrong end”, so to say. He has a record deal and brought out his first album before he actually learned to play [Junkee].

As always I’d urge you to listen for yourself before reading on to make up your own mind about Benjamin’s undertaking.

First, let me say that I haven’t heard the whole record, just the clips available online, which I’ve included here as well. Frankly, I’m not sure if I could endure listening to the whole album 😉 But I do think these clips give a pretty good idea of the nature of Benjamin’s project.

Benjamin’s Jazz Daredevil raises many questions and thoughts of the nature of jazz as a musical genre but in a more general level also about what the philosopher Pierre Bourdieu has called “cultural capital”. More specifically, it challenges the notions of skills and talent required to perform music we call jazz and raises questions as to where lies the value of artistic enterprises in general and in jazz in particular.

Music as language

I’d like to here approach these questions through the common comparison between music and language. I’ll leave a more thorough discussion of this comparison for later and pick out a few themes from the music-as-language discourse that apply to the present discussion.

As discussed here before, jazz can – at least be attempted to – be defined by naming some essential characteristics such as specific rhythmic (swing) and melodic (blues) “vocabulary”, if you will. To be able to play jazz, then, would entail a command of these vocabularies in a way that others familiar with this musical language understand “what you’re saying”.

Music as language

The comments in the clip above by some of the musicians involved in Benjamin’s project suggest that they didn’t feel like they were exactly “speaking the same language” musically. In the clip it’s also suggested that Benjamin’s album is what jazz sounds like to the uninitiated. Some online commentators [NPR] also share this view.

Cultural capital

The narrator of the clip has a view in between the above two hearing Benjamin’s efforts as “fresh” approach to jazz. This, in my view, speaks of one of the biggest problems with some avant-garde art, it’s view of artistic novelty as cultural capital; trying to do something “new”, something nobody has done before as the main value of their art.

The underlying ideal of this aspiration to constantly reinvent the western musical tradition comes from one of the backbones of modernism, progressivism; the idea that the human condition is on a linear, constantly improving path (for the history of this line of thinking see here [Stanford Encyclopedia of Philosophy]).

In musical avant-garde this first took the forms of expanding the conventional parameters of music, mainly harmony, in musical impressionism and eventually attempting to do away with them entirely in expressionism. The latter also began experimenting with instrumentation such as the Italian musical Futurism of the early 20th century.

The group Performa discussing and performing parts of Luigi Rulloso’s Art of Noises.

While Benjamin undoubtedly was not particularly inspired by the pioneers of “Art of Noises”, the musical programme of the Futurists did include avoiding conventional musical training in favour of auto-didactic learning. However, both share the aim of creating something “new” and unconventional by setting themselves consciously apart from the “mainstream”.

loopool reading Russolo’s manifesto and providing some context to it 100 years after.

Unorthodox musicians are of course not a new phenomenon in jazz. Already in 1950s Ornette Coleman stirred the scene with his plastic saxophone and was told “You can’t play that” (Charlie Haden in the BBC documentary). However, Coleman found other musicians who shared his musical vision – or they found him – such as Charlie Haden and went on to “rewrite the language of jazz” with his playing that was rhythmically and harmonically “out of the box”.

Blues Connotation is one of Coleman’s classics revealing his rootedness in the Black American musical tradition and showing how he springs from it.

A quick listening experiment, however, reveals – even to the uninitiated, I believe – how Coleman is in fact very well at home in the musical language of jazz – or black music in general – and chooses to make his own version of it. Benjamin, on the other hand, seems to have a very rudimentary idea of how to “speak the jazz language”. Some of his rhythms and melodic shapes suggest that he’s not completely at loss in his musical environment, but in absence of any technical command of his instrument of choice or any knowledge of the “grammar” of the language he’s trying to convers in, his playing does sound quite as random as it actually is.

To say that this is how jazz sounds like to people who don’t know it says more about how people listen to music than about anything else. You don’t need to know a language to be able to tell when someone’s faking it. You just need to listen a bit more carefully and compare to something you know to be the “real thing”. In Benjamin’s case it’s actually quite easy if you listen to the exchanges – or dialogue, if you will – between the saxophonist and Benjamin.

In terms of cultural capital, I think the value of Benjamin’s project lies elsewhere than in its musical merits. As one of the online commentators mentioned this could be an “Andy Kauffman/Borat kind of ‘meta-gag’”. Benjamin himself is not admitting it, but then again that would spoil the gag, wouldn’t it?

It’ll be, however, interesting to see what comes of this if Benjamin actually pursues with his musical career as he says in the interview. Although jazz is no longer in the centre of Black American culture like it was in Coleman’s early days – or it’s even considered “dead” by some – high jacking it for such a “meta-gag” could be seen as controversial in perhaps a broader sense than Benjamin thought of. Him actually learning the language and then finding his way to speak it would bridge that gap. But whether that’s his end game with this project we’ll have to wait and see what his “untapped un-talent” brings us.

Jacob Collier

Jacob Collier in his music room

I recently came a cross this amazing talent in a newsletter email highlighting an upcoming Quincy Jones’ concert with Mr Collier accompanied by raving endorsements by Jones and other top jazz performers of our time such as Pat Metheny, Herbie Hancock and Chick Corea. The introduction of this rising star mentioned that he’s a successful YouTuber so naturally I looked him up. And here’s what I came across first.

That really blew my mind! It’s easy to hear why he’s received such praise. Not only does young Mr Collier has some very interesting and fresh sounding harmonic, melodic and rhythmic ideas but he’s also found his own way of expressing them – his performance style, if you will. And listen to the way he builds up this old classic first with rather traditional, although extended, harmonies and (more) conventional rhythms and saving his more unique ideas later on in the performance. I.e. he doesn’t go straight for the “acrobatics”, but first bows to the tradition before springing from it to his own adventurous exploration of the song.

In this age of fast flying YouTube stars and child prodigies Mr Collier shows not only musical maturity – at the age of 19 – but he’s put considerable effort and thought into his presentation. His videos are very well produced with fine audio and video quality and skilful editing. He even creates slightly different personages for the many “Jacobs” singing the multiple parts. They each have a unique attire and different hairstyle (although, frankly, he’s not much of a stylist 😉 ). What I especially like is that he moves his head in rhythm with the particular voice he’s singing making it easier for the listener to recognise which of the Jacobs is singing which part.

Technology and/in performance

After my university studies I was sketching a plan for a PhD research on music and technology, especially in the ways various technologies are used in live music performance. Collier seems to be interested in this topic as well and searching for technological means to express his musical ideas.

I recommend listening to this one with heaphones or well positioned speakers to experience the live mixing.

What I find especially intriguing in this video that Collier actually demonstrates us what DJs do when they mix records and samples and manipulate them live. Only Collier produces all the “samples” himself, live. In an interview he has indeed mentioned that honing his performance solo practices is one of is major goals. For this end he’s already got a project with MIT lined up. I’m very curious to hear and see the fruits of that endeavour!

In the above interview Collier makes an interesting comment about his predilection to acoustic piano against various keyboards and synthesisers. That’s an approach of a musician as a craftsman in the traditional sense. Playing acoustic instruments gives one a whole different sense of “doing” and control of the sounds one produces. Getting the desired sound(s) out of an acoustic instrument also requires sufficient technical command on it. On a keyboard, synthesiser, or some other “post-mechanical” instrument, the sound is much more – if not entirely – predetermined and the player merely triggers it with a press of a key or hitting the electric drum. Of course technique is still required to make music with such instruments but more towards executing than producing the music.

Collier’s choices of instruments are eclectic and apparently arise from his upbringing in a family of musicians and abundant and varied musical activities from an early age (see the interview linked to above). He has a particular preference to melodica, an instrument originally designed for educational use. In Collier’s hands this simple instrument doesn’t seem to lack anything but is able to rise for the musical occasion at hand. Outside his music room he also makes use of its portability and e.g. joins the horn section of Snarky Puppy and walks to the soloist’s mic to take his solo in the manner of horn players.

Talent in the Internet age

As mentioned above, Jacob Collier really stands out among the YouTube child prodigies playing e.g. classical piano, guitar or conducting an orchestra at a young age. While this comparison might be a bit far fetched as Mr Collier is not a child any more, there’s a similarity with the immense concentration of talent around a person (Anakin Skywalker anyone? 😉 ).

Significance of the medium in which we get these talents presented to us is also not to be neglected. While the YouTube stardom of most of the above mentioned young talents doesn’t reach the “Internet Phenomenon” stage, with Mr Collier it not only does reach it but delving into his world through the available clips makes quickly apparent that he already has a vision of his career that won’t allow his star to descend any time soon. At least that’s what I sincerely wish for him and am eager to witness in the years and, with all likelihood, decades to come.

Mixer 2

 

Actually Collier’s undertakings remind me of Prince in many ways. Prince has also always (or at least mostly) played all the instruments except horns on his records. But in fact also on the business side. In the 90s Prince infamously quarrelled with his record label at the time Warner Brothers about the ownership of his master records and began sporting a “Slave” text written on his face and changed his name to an “unpronounceable symbol” – all acts with multiple meanings as well as causing headaches to his record label trying to promote the brand formerly known as Prince.

Prince is attributed to have been one of the first artists to realise the power of internet in music business, which Collier seems to have mastered quite well. In addition to his YouTube channel, which having over 44 000 subscribers (at the time of writing this) probably brings him more than pocket money in Google Adds compensations, Collier has also launched crowd sourcing project on Patreon. I’d say he’s at least got his tuition fees at the Royal Academy of Music covered and he’s still left with some to maintain his sizeable instrument collection 🙂

Another thing about talent I’d like to bring up is the “nature-nurture” aspect of it. Statements about talented people like “It’s in his/her genes” are common enough implying that the talent is inherited. Coming from a musical family, this seems to fit the bill in Collier’s case. It is, however, a fact of biology that learned traits are not simply passed on. We can’t change our genes by learning new stuff and expect our offspring to get that “in the mother’s milk”. Species evolution takes place on the gene level due to (significant) changes in the living environment causing some genes to mutate. And this doesn’t happen in a generation or two of e.g. musicality in the family.

For more about human musicality check this short and concise presentation by Professor Henkjan Honing.

While “everyone is musical” to an extent as Professor Honing has shown [English translation of his book Iedereen is muzikaal], this musicality is of a different kind than what we witness in Collier. His musical family does have everything to do with it but not in a way of “passing on” the family trait in their genes. Rather, musical families support each other should there be interest in pursuing musical activities. Such interest is apparent in Collier and the support of his family has enabled him to reach a level of musical maturity in a young age (Check out also the bass player Victor Wooten‘s an multi-instrumentalist Usman Riaz‘s similar stories). What we hear in Collier’s undertakings is not merely talent but passion and drive to put the countless hours into practising and working on his music required to reach this level. And he just seems to be getting started so I’m really looking forward to follow his musical adventures to come 😀

And to wrap up here’s one more gem from Jacob Collier.

Scroll to top