Computers are both tool and myth. Their silicon centers are made from a natural element, which is then infused with symbols and codes in order to create a machine that is ultimately made of ideas. We have formed symbiotic relationships with them. As they co-opt various existing media, they have become powerful tools of expression and understanding. On the one hand, they allow us to express ourselves in novel; on the other, they allow us to better understand other points of view. This paper explores the various sub-media that make up the new digital media; the new paradigms that are forming in the new media; and proposes a model for best utilizing these media and paradigms.
Media
There are obviously many sources that inform the new media. There is the medium of logic and calculation -- the original purpose, and still the primary function, of computers. The opening and closing of many individual logic gates, resulting in ones and zeroes, is what allows computers to envelop and transform other media. Behind this, even, is the human brain -- a vastly more complex medium for processing information and melding various media into meaning.
The visual, artistic media are often referred to as source media for computers. Like painting or sculpture, a computer is a tool that allows us to realize our visualizations, to "photograph our dreams." Marshall McLuhan said, "An abstract painting represents direct manifestation of creative thought processes as they might appear in computer designs"[1]
Symbolic language is our most pervasive and powerful medium for expressing ourselves, and it informs digital media at a very basic level. We teach and speak to computers in languages -- "machine language" and "programming languages." Symbols represent the mathematical processes that take place inside the computer. Heinz Pagels believes that mathematical equations are a language that stands apart from humans, having en existence unto themselves.[2] And the computer itself is made of symbols and ideas. The same goes for media, and particularly digital media. It could be said that pieces of media are like words in the digital world; when combined, they form phrases of meaning.
But this ignores the ambiguities of language. Language is deliberately vague. When we speak or write, the intended and resulting meanings are not always found in the words themselves -- they are also on the margins, hidden "between the lines" or masked in intonation, inflection and "body language." Also, language is simply incapable of expressing many of the things we wish to. It is difficult to describe a feeling or a piece of music precisely, using only words.
Still, symbolic language is a convenient shorthand for communication. Jaron Lanier calls symbols "a quick hack to refer to things you can't realize because you're not powerful enough"[3] And communication is what is truly important about digital media. It may be communication between two people, from one person to many, among many people, between a person and a computer, between computers, between different media, and so on. In each case, a commonly shared language is necessary.
An important part of this is the concept of coding. Communication theories relate how we encode and decode messages. "Coding" is another word for computer programming. We try to "decode" the human genome. We encode and decode private messages on the Internet. Quantum equations are seen by Pagels as "cosmic code." Coding is a method of creating (or obscuring) meanings. It is a means of control, and of quantification. And this is what digital media are about as well -- dissecting, reorganizing previous media, digitizing (quantizing) them, harnessing control of them, and changing their meanings, or combining them to create new meanings.
Because communication is paramount in digital media, networks are another of the main sub-media. A "network" can refer to an artery within the body, the web of highways across the country, or a linkage of information. And digital media make extensive use of networks, whether a conversation between two people over the telephone network, a local area network of PCs, a national one-to-many TV network, or the global Internet.
Combining networks with the concept of coding, digital media take on new forms. People are able to mask their identity. A single medium (like music) can be "coded" into another medium through programming. A television broadcast could be encoded with a subtle, perhaps subliminal message.
Subtle meanings are no doubt already encoded into media messages, between the lines." This is often a charge directed at journalism. Yet journalism remains, even in the digital age, our primary means of getting news information. Journalism itself is a strong part of digital media. (It is telling that the epithet "The Media" often means journalism in general.) Journalism's influence can be seen in a slew of new digital projects, most obviously electronic and online publications, but also in the format of informational CD-ROMs, online databases, and Mosaic pages.
More specifically, the form seen most often is magazine journalism -- with its colorful page layouts and time-based publication schedule -- and secondarily television -- with its "screen" and "network" sub-parts, its rectangular frame, its dynamic movement and paradigm of conflict. (Since I have covered television's contribution in a previous paper, I will not go into depth here.)
It is important to consider some aspects of journalism that inform digital media. For instance, the notion of news "filters" applies directly. Information (data) passes through a number of journalistic filters -- reporters, editors, producers, transmission media, and so on -- before it reaches news "consumers." Digital media could allow us to choose our own filters, to create our own, or perhaps to dispense with them altogether (provided that pure, "unfiltered" information -- if such a thing is possible -- is available).
Another interesting aspect of journalism is that of point of view. Individual reporters, producers and editors inevitably impose their individual points of view upon a story. A principal tenet of journalism is to take readers "inside" of an event or story -- to make them feel like "insiders," as the reporters themselves are. With digital media, point of view is a major concern. Presumably, a person can change points of view, viewing an event or thing from the view of different characters, or from different physical places, or different conceptual stances. Often, as with journalism, one is allowed the illusion of going "behind the scenes." This can be seen, for example, in Voyager's Freak Show CD-ROM, and Peter Gabriel's Explora.
Transmission media, like people, impose their own points of view upon information. In obvious terms, an event, for example, is seen differently depending on whether it is experienced in a newspaper, via radio, or on television. But also, media impose deeper constraints upon data -- in McLuhanesque terms, they have ecological effects, and change the content they carry, the way we perceive it, and our very lives. (Neil Postman details this phenomenon in his book Technopoly.[4]) Digital media, by adopting, processing, disassembling and reorganizing individual media, change points of view and create new ones.
Digital media make it easy to transcend not only space but also time -- to read an email message left for you a few days ago, for example. But I believe that digital media really excel at live, real-time communication. It is the performative aspects of digital media that interest me. Brenda Laurel has detailed this idea in Computers as Theater.[5] Note the metaphor used in the predominant digital authoring program, Macromedia Director, with its stage, cast, score and script.
An interesting subset of performance is masked theater. With digital identities, anyone can become virtually anything. In "MUDs" on the Internet, people become furry creatures, medieval warriors, and things more imaginative, all through the coding of language. In Jaron Lanier's vision of virtual reality, people can transform themselves to anything they can imagine, using some form of "post-symbolic" communication.
Lanier's vision extends to perceive computers as musical instruments. This is where, I believe, the most fruitful inspiration lies, and it seems to be the direction we are heading in. (Note that Lanier's recent musical release -- involving solely non-electronic instruments -- is called Instruments of Change. This moniker could easily be used to describe digital personal computers.)
There are some obvious evidence of this influence -- computers used in digital sampling and music production, clumsy performances combining computers and traditional instruments. But more interesting are the increasing instances of computers used as instruments in live performance, playing not just music but images, animations, text, video. David Rokeby's Very Nervous System utilizes a computer to accompany a dancer. A recent performance by an ITP student allowed participants to "play" video clips along with musicians.[6] In a recent Neal Stephenson story, radical teenagers plug their guitars and "media processors" into the network "spew" to subvert commercialism and jam in a live, multimedia performance.[7] These days seem comparable to the 1940s in jazz, when enthusiastic improvisation really took off. Previous media, by comparison, are like structured classical music.
Jazz is a fruitful analogy, because in jazz, performers improvise off of each other with their instruments. Imagine a live improvisatory digital performance in which participants (perhaps in remote locations) are combining various media on the fly, pressing the keys on their computers like keys on a piano. This is close to what Lanier imagines in his ideal virtual world -- having cathedrals come out of a piano, and so on. As he says, playing an instrument far exceeds the experience of using a computer. He uses the piano as an example of a "button box" that exceeds its digital-ness -- the performer "reaches through" the keys to create the music.
So the goal of Lanier and others is to make digital media more like music -- to capture the mystical quality, the unspoken (and unspeakable) aspect of experience. Just as meaning in symbolic communication is between the lines, music is what is between the notes. For notation is merely an after-the-fact shorthand code, adequate but incapable of fully describing music. Lanier's goal is to cut out symbolic language altogether, to be able to create virtual worlds as fast as we create language. The only way to do this is to make computers more intuitive, more like musical instruments, since our bodies know more than our minds in terms of instantaneous action. And indeed, you feel powerful music in your body. Jazz musicians "in a groove" nod and sway to the tune.
Music as a medium, is, like light, pure information -- it doesn't carry another medium as its message. What digital media can add to a performance is meaning. Pure music may create a certain feeling or evoke images, but a mix of media could add actual images, sounds and information: they add other media. Goethe said that "architecture is frozen music."[8] We now have the chance to create liquid architecture.
Emergent Properties
Given these and other media that inform the digital paradigm, several properties emerge as the media converge. These boil down to the following concepts: process and processing, the cyborg principle, systems theory, and boundary/interface.
Process is evolution -- descent with modification over time. Processing is the modification of component parts. "News" is processed information, a process that evolves into meaning and understanding over time. We process sounds and images and video in order to modify them so that, in their next generation, they have evolved into something better -- or at least different -- than they were.
The performance aspect is inherently processual. A computer bulletin board system, such as Echo or The Well, evolves over time, as bonds are made and broken, a community forms and attains critical mass, and begins to regulate itself. A live multimedia performance evolves as the performers try to hit a "groove," that unspoken, mystical feeling. A collaborative artwork, such as the Internet's "Graffiti Wall" grows and grows like a colorful fungus. The Internet itself has become a huge, self-replicating organism.
The Internet is an "artificial life" system, subject to shaping forces of selection, mutation, drift and environmental effects. Others abound: programs such as "viruses" and "worms" that replicate themselves digitally, fractal art patterns, and so on. All highlight the fact that computers are active players ("agents" or "actors") in our everyday lives.
This is not to overlook the influence of our culture, and our own free will. Any performative digital project is necessarily shaped by intuitive, nonspoken, mystical impulses, as well as subconscious and surrounding cultural influences, and the active decisions each of the participants make.
Sergei Eisenstein, interestingly, took a similar dynamic, processual approach toward the film form.[9] He saw film as an extension of language, rather than staged theater or painting, and used a dialectical approach -- utilizing the conflict between "purposive initiative" (more concrete, rational motives) and "organic inertia" (abstract impulses).[10] Art, he said, is produced by the clashing forces of nature and industry. This is similar to the performative approach to digital media I am proposing, where digital media are seen as being produced by the clash of rational language and abstract painting or music, with the best parts of computers and humans being utilized in the process.
Because we are so intimately tied to computers, Donna Haraway proposes the "cyborg manifesto."[11] Her philosophy recognizes our joint kinship with animals and machines -- our organic and artificial selves -- and incorporates the oft-overlooked third aspect, free will, when she calls for the active creation of our own consciousness.
The cyborg is "committed to partiality, irony, intimacy, and perversity." It is "oppositional, utopian, and without innocence"[12] The cyborg is against a common language -- it speaks different codes to machines, and various human groups. It recognizes that we are increasingly relegating our mental life to the digital world (an extension of McLuhan's self-amputated collective consciousness) yet sees no evil in this; in fact revels in it.
We are putting our senses and memories into machines that are becoming increasingly small, powerful and ubiquitous. Laptop computers, personal digital assistants, cellular phones all carry our thoughts, (while we go to gyms and remake our bodies into more "animal" form, or is it more artificial?). Xerox PARC is developing the idea of "ubiquitous computing" (as opposed to "personal computing") in which we don't go to our computers, but computers are wherever we go. Pagels relates a story about a "mentally disturbed young man" who thought that aliens were controlling humans through their technology.[13] But clearly, the "alien" is merely the cyborg -- the joint being created by human animals and their technology.
With machines becoming smaller, more ubiquitous and more powerful, the best machines are nonmaterial. They are made of light, or electromagnetic pulses across a network. They are super-machines, parallel processors made up of many small components, whether those components are human or machine. As Haraway says, the realm of the cyborg is pure consciousness. The cyborg lives in digital-human performances -- in the light (and the shadows), between the words (and code), between the musical notes and between the video scan lines, between the ones and zeroes.
The binary principle subverts itself. The endless manipulation of ones and zeros allows us to transcend the distinctions between nature and culture, public and private, primitive and civilized. The cyborg principle allows for contradictory standpoints and partial identities.
Phenomena are no longer understandable by absolutes, but as statistical percentages. Quantum dynamics remakes the world in probabilities; fractal math breaks down component systems into endlessly repeating patterns; and systems theory helps us understand how each of the component systems act and interact. There is one exception to traditional systems theory, however -- there are virtually no "closed systems," for all systems (particularly media) are open and interacting with each other.
Component parts of digital media can be broken down, into "words" or pieces of code, processed, then reassembled. The components reach their lower limit at the binary level -- a single one or zero is meaningless, but a system of several represents a word of text, or an image, or a sound. Back in 1945, Vannevar Bush celebrated the coming of machines that could be created with easily produced and manipulated interchangeable parts.[14] Forty years later, Haraway states that "no objects, spaces or bodies are sacred in themselves; any component can be interfaced with any other if the proper standard, the proper code, can be constructed for processing signals in a common language."[15]
So there are the component parts of the system, and there is the interface between them -- the boundary, the margin, the place that is no place, which is the realm of the cyborg. The cyborg takes "pleasure in the confusion of boundaries, and responsibility in their construction."[16] And indeed, the boundary between science fiction and social reality is blurred to nonexistence (as with a digital filter in PhotoShop). Music, as Jaron Lanier says, takes place on the boundaries or the margins, and so too does digital performance. The boundary between natural and artificial is breached, for silicon is a natural element, and we code it with artificial language. It allows us to transcend the boundaries of time and space.
Model
Taking all of these concepts into account, the task, then is to put them into practice. Many are already seen in various component systems, but to bring them together into a larger system would be really interesting.
I see the Internet evolving further to incorporate many of the component digital systems and principles described previously. It is made up of millions of component computers, each of which uses the basic binary principal and mathematical calculations (cosmic code?); and human operators (the organic, animal component), which, in the "ether," on and between the wires and signals, become cyborg entities.
These cyborg nodes are known by their dualistic (or multiple) identity as both human and machine; perhaps they mask their identity, or are known be a certain code name, image or idea. They use linguistic codes, and nonlinguistic intuition -- clashing the two to create dynamic meaning. Communication is always the main purpose, and shared or personal meanings are the satisfying result, whether the meanings are rational and expressible in words, or irrational "grooves" that are merely felt.
Pieces of existing media are dissected, digitized (coded) and processed, then fed into the "spew," where they are reassembled into a larger system. Imagine a complex news event seen in this system. During the Los Angeles earthquake, the political crises in Russia and China, and other major events, people used the Internet to communicate and distribute information, bypassing mass media journalistic filters. But mostly, this was in the limited form of text. The technology is almost ready for all existing media to enter the network.
A major news event happens. Cyborg entities -- some human-computer hybrids, some merely computers -- enter the network. Text is exchanged as news is learned. Sound is fed in -- someone is live on the scene with a microphone; a local radio news station is fed into the net. Then comes video: with the myriad camcorders around, someone is inevitably filming the event; perhaps security cameras capture some part of the event.
The various media are mixed live by anyone/anything who wishes to jack in. Filters and points of view are variable. Individuals become known by their skill at digital performance -- like skilled television directors who mix shots on the fly, they are adept at "playing" various media components via their computer as quasi-musical instrument. They are skilled at improvising, "jamming" with others in real time. They are respected because, with their coding, processing and mixing skills, they create a fluid architecture of mystical meaning. We join their groove and revel in it.
Spread over geographic space, many points of view are added to the mix. Spread over time, the event-as-process (and as processed) evolves into a piece of history. Historical memories are often medium-specific. We know the Roman Empire, for example, mostly from spoken/written accounts; perhaps from sculpture and architecture. A contemporary event would become a fluid, dynamic process whose complexity is captured in the individual viewpoints and multiple media and multiple filters.
There are some key assumptions I am making in this model. For instance, I am assuming vast democratic access to this vast network. Also, there are hardware and bandwidth requirements which are only now coming into existence. Additionally, I am assuming that a myriad of media will be freely fed into the network. This means radio stations, TV stations, uplinks from individual homes, remote satellite feeds, security and surveillance cameras.
Ideally, all these media would be injected into the Net not just during events or crises, but all the time. Imagine being able to receive any radio station anywhere in the world at any time, via the Net. Or tuning into (and perhaps controlling) a camera trained constantly on the West Bank. Or looking in on your local bank, to see how long the lines are. Naturally, such visions of universal surveillance bring frightening possibilities. But if democratic access is allowed to already existing media, and anyone can tune into any of them at any time, the risk of oppressive control is lessened. Further, I am not proposing having live cameras installed in private homes, although some people will choose to broadcast from their homes onto the Net if they desire.
The news event itself and its performances on the network are components of a larger system, and so too have their own component parts. It is the parts interacting in concert that produce meaning.
This vision approaches that seen in science fiction -- a vast, multimedia network, ubiquitous digital computers and cameras, cyborgs and self-replicating artificial organisms. But as Haraway states, science fiction and social reality are no longer apart. Computers are becoming part of us -- they are turning from tools of calculation to tools of expression, and as they adopt and co-opt other media, they become fantastic tools for digital performance and expression.