Through the Touching Glass: Literature for Haptic Inter[(surf)aces]

Article d'une publication

Through the Touching Glass: Literature for Haptic Inter[(surf)aces]

Diogo Marques

Article paru dans Poétiques et esthétiques numériques tactiles: littérature et arts, sous la responsabilité de Anaïs Guilet et Emmanuelle Pelard (2016)

Leaps and take-offs

The blue sky above us is the optical layer of the atmosphere, the great lens of the terrestrial globe, its brilliant retina.
From ultra-marine, beyond the sea, to ultra-sky, the horizon divides opacity from transparency. It is just one small step from earth-matter to space-light – a leap or a take-off able to free us for a moment from gravity.

Paul Virilio, Open Sky

As I read Virilio’s introduction to Open Sky (1997 [1995]), I decide to open the Google Earth app on my iPad. By sliding my forefinger over its glassy surface, I notice that I am coming closer and closer to what corresponds to my current geographical position, but still, at the same time I am able to travel around the world in just a few taps and swipes on the screen. As reminiscent as it may be of David Bowie’s “Planet Earth is blue/and there’s nothing I can do”, this apparently insignificant manipulation also reminds me of Canadian astronaut Chris Hadfield’s version of “Space Oddity”, recorded inside the International Space Station and enabling more than 23 million people to witness Earth’s blueness through Hadfield’s camera lenses.

What all of these artefacts – videos, music, lyrics, quotations – have in common is the fact that they are affected by a series of interface mediations, all of which, of course, can be seen and touched through the Internet. However, one questions the real significance of this touch and why do we find the idea of holding the whole world in our hands so phenomenal. If what we need now to free us from gravity is just a leap or a take-off, which might be done by a simple touch of the hand or a snap of the fingers, what becomes of the eye?

The intensification of research around digital media devices that require tactile/haptic functions,¹ such as touch and gesture, along with efforts to increase tangibility in the Human-Machine Interface (Gallace & Spence, 2014: 229), is giving way to a whole new rhetoric of bodies and surfaces (as well as interfaces). Not only touch and gesture are anything but superficial, but also these “new” processes of writing and reading tend to amplify the primacy of vision over other sensory modalities. This is a paradoxical situation that Wendy Hui Kyong Chun defines as a “compensatory gesture” by “the current prominence of transparency in product design and political and scholarly discourse”:

As our machines increasingly read and write without us, as our machines become more and more unreadable, so that seeing no longer guarantees knowing (if it ever did), we the so-called users are offered more to see, more to read. The computer – that most nonvisual and nontransparent device – has paradoxically fostered ‘visual culture’ and ‘transparency’. (2005: 2)

In addition, as ubiquitous computing turns into a naturalized process in our lives, the opacity/transparency paradox becomes even stronger, which is a natural consequence of its attachment to ubi-comp.

Extending an avant-garde countercultural tradition that started questioning visual culture as early as the beginning of the last century, there is also evidence of an increasing number of digital literary works channelling its countercultural and metamedial poetics towards the aforementioned phenomena. These “technotexts” (to borrow Katherine Hayles’ term) may or may not include multi-touch devices such as tablets and smartphones. Nevertheless, the one who do, often self-reflectively question the specificities of these digital devices and media, as well as the apparatuses enclosing them.² I argue that such “machimanipulations”, manipulations of the device by both humans and machines, tend to defy the general assumption of surfaces as something superficial, recovering Deleuze’s idea of surfaces as double-fold and profound (1990: 4-11). If, in fact, we are now living in a “Glass Age” governed by a culture of transparency, to what extent are these “transparent” touching glass surfaces becoming an opaque looking glass?

To surf the surface of an interface

Why, in the triad of medium, substances, and surfaces, are surfaces so important? The surface is where most of the action is. The surface is where light is reflected or absorbed, not the interior of the substance. The surface is what touches the animal, not the interior. The surface is where chemical reaction mostly takes place. The surface is where vaporization or diffusion of substances into the medium occurs. And the surface is where vibrations of the substances are transmitted into the medium.

James J. Gibson, The Ecological Approach to Visual Perception

While surfing on the Internet, I came across a zine titled Some Ideas About Surfing (2010), by graphic designer Joel Colover. Arranged as a series of xeroxed sketches preceded and combined with phrases in bold, this zine’s aesthetics presents a creative use of the “surfing” metaphor in order to rethink cyberspace and hypertextuality. For instance, by putting together several of its bold sentences, one reads that:

Tab based browsing is surfing different waves at the same time. All the time. […] On the Internet we surf the currents left by other users. Old lost content and new data supply us with the momentum we need… The Internet is not a world, it is a city the size of a universe. It can be split into different quarters, not just based on content, but also on presentation, design, architecture of the pages. This is what the invisible walls are made of. (2010: 12-16)

Figures 1a & 1b: Some Ideas About Surfing. Joel Colover. Screenshots.

Taking into account its etymology, the verb “to surf” can be related with “the swell of the sea that breaks upon the shore” or “the foam, splash, and sound of breaking waves” (Source). If we relate tab based browsing to the act of riding a wave using a special board, it becomes clear that what we are actually riding is this “surf”. In Joel Colover’s zine, the action of surfing also seems to be used with reference to today’s exploration of the Internet. But not in a completely uninteresting, flavorless or superficial way. Ultimately, what is being discussed is the issue of digital multimodal environments (“surfing different waves at the same time”, minding the proper skills which are necessary to do this particularly complex task), where we drift on issues left by others (there are “good” and “bad” waves, and one needs to learn how to catch a perfect one), while simultaneously talking of a medium (the computer) that is able to gather a series of other media in its multiple layers. Finally, it is “a city the size of a universe” (not the Earth’s roundness, but the horizontal profoundness of infinite expansion), split into different quarters, more precisely, a city made of “invisible walls”. If, on the one hand, we may find it tempting to make use of these surface metaphors in order to describe what is generally known as a digital interface, on the other hand, what we find in the haptic motion of surfing a surface of an interface is a complex process of mediation between surfaces that is capable of destabilizing traditional uses of the HMI. That is to say, taking into account some of the terminology surrounding the HMI – “feedback loops”, “distributed cognition”, “telepresence”, “enactive cognition”, “cybrid bodies”, “enactive cognition”, among others – we find that there are surfaces and interfaces on both sides of the equation. Hence my decision to create this odd portmanteau-like word in the first place, i.e. inter[(surf)aces], in order to describe the ebbs and flows of a haptic motion between surfaces of different interfaced bodies.³ In this way, if an interface, a “productively open-ended, cross-disciplinary term”, cannot be restricted to a “point of interaction between any combination of hardware/software components”, as Lori Emerson already pointed out, (2014: x) and if we consider at least the eight different kinds of interface listed by Florian Cramer, (Emerson, 2014: x), the surface of our skin is no less of an interface than a digital screen.

Rather than a window or a mirror, the interface can also be described as a threshold (Emerson, 2014: x) between mediated layered surfaces, either actual or virtual. In short, it is what Alexander Galloway, in his understanding of the “interface as an effect”, describes as “a point of transition between different mediatic layers within any nested system” (2012: 31). This does not mean, again, that the threshold is to be found outside ourselves. Jean-Luc Nancy’s remarkable definition of “threshold”, given on the occasion of a conference on touch and the primacy of touch (at the Louvre Museum), already points to this particular kind of embodiment. Specifically by means of an analysis of Caravaggio’s “Death of the Virgin” (1604-1606), he remarks that:

[…] we have entered there where we will never enter, into this scene painted on a canvas. […] We can’t exactly say that we have penetrated there, but neither can we say that we are outside. We are there in a manner older and simpler than by any movement, displacement, or penetration. We are there without leaving the threshold, on the threshold, neither inside nor outside — and perhaps we are, ourselves, the threshold, just as our eye conforms to the plane of the canvas and weaves itself into its fabric. (Nancy, 1996: 57)

Taken these words into account, if we may be, in fact, this threshold by means of the eye, what can we say of the finger, especially taking into account digital devices that are able to enhance tactile/haptic touch and gestures?

It is perhaps relevant to state that a threshold is also a surface, albeit a complex one, as can be read in Gilles Deleuze’s The Logic of Sense (1990). On analyzing Lewis Carroll’s decision of changing the title from Alice’s Adventures Underground to Through the Looking Glass, Deleuze is able to denote a change from a depth perspective to a surface perspective:

By sliding, one passes to the other side, since the other side is nothing but the opposite direction. If there is nothing to see behind the curtain, it is because everything is visible, or rather all possible science is along the length of the curtain. It suffices to follow it far enough, precisely enough, and superficially enough, in order to reverse sides and to make the right side become the left or vice versa. It is not therefore a question of the adventures of Alice, but of Alice’s adventure: her climb to the surface, her disavowal of false depth and her discovery that everything happens at the border. […] This is the case –even more so– in Through the Looking Glass. […] Alice is no longer able to make her way through to the depths. Instead, she releases her incorporeal double. It is by following the border, by skirting the surface, that one passes from bodies to the incorporeal. (9-10)

Deleuze’s effective considerations on the double-foldness of surfaces manage to turn the rabbit’s hole completely upside down, or better, by spreading it across a horizontal depth that is far from being superficial. Nonetheless, while his theory of surfaces seems to work well when applied to the articulation of the plane of the page with the three-dimensional space of the book in the psycophysical mechanics of the codex, how does it behave in the realm of the digital?

Bearing in mind “writing on complex surfaces” John Cayley does not argue that “print-based textuality is incapable of delivering writing with a complex surface”, but he does say that “in so far as this is achieved it is achieved as concept, in the familiar and comfortable realm of literary virtuality, in the ‘mind’ and in the ‘imagination,’ but not in the material experience of the text and its language.” However, regarding writing in programmable media,

The screen should not simply be cast as the bearer, for example, of multiple (flat) surfaces or successive ‘states’ of text, it must be viewed as a monitor for complex processes, processes which, if they are linguistic, will be textual and symbolic, with a specific materiality as such. We must be able to see and read what the screen presents rather than recasting what passes before our eyes as the emulation of a ‘transparent’ medium. (Cayley, 2005: §6)

By seeing “materially and conceptually complex” surfaces as “a liminal symbolically interpenetrated membrane, a fractal coast –or borderline, a chaoticand complex structure with depth and history”, (2005: §1) Cayley develops a theory of surfaces that is not to be separated from his work as an artist. In his and Daniel C. Howe’s The Reader’s Project (2011–), a work in progress described by its authors as a “collection of distributed, performative, quasi-autonomous poetic ‘readers’ –active, procedural entities with distinct reading behaviors and strategies”, a series of automatic readers are released “onto inscribed surfaces that are explicitly or implicitly, visibly or invisibly, constituted by their texts”.⁴ Resembling the structure of a double-page book-like opening, these several programmed readers present multiple behaviors, enabling the possibility to “reveal certain contours and outlines of linguistic materiality”. Composed of surfaces of reading and writing that are constantly intertwining between each other, these readers also enhance a two-fold perspective which can be associated with the dualities “explicitly or implicitly, visibly or invisibly”, in a form which is “significant and affective, aesthetic and literary”. In their own way, each performing reader constitutes itself as a threshold to be crossed by metareaders that read the reading process.

Touchable inter[(surf)aces]

In television there occurs an extension of the sense of active, exploratory touch which involves all the senses simultaneously, rather than that of sight alone. You have to be “with” it. But in all electric phenomena, the visual is only one component in a complex interplay. Since, in the age of information, most transactions are managed electrically, the electric technology has meant for Western man a considerable drop in the visual component, in his experience, and a corresponding increase in the activity of his other senses.

Marshall McLuhan, The Medium is the Massage

Merleau-Ponty’s phenomenological observations on the “body schema” and the “body image” lead Mark B. N. Hansen to rethink the former’s notion of écart (a “gap” or a “divide”), explained by the latter in terms of a “transduction between embodiment and specularity […] that informs the emergence of the visual from primordial tactility” (2006: ix). Defining this first touch as a “protosensory (amodal) power”, different from “touch as a distinct sense” (2006: 68), Hansen manages to go beyond the idea of “haptic visuality” recovered by Deleuze and Guattari from Alois Riegl (Deleuze & Guattari, 1987: 492-93), a metaphor that, in spite of its significance when applied to the cinematic image, seems to be inadequate when it comes to “interactive art” (Huhtamo, 2007: 73-4). If what is now at stake is the idea of a “seeing hand” rather than a “touching eye”, what can be the implications of this chiasmatic game of inversions? If applied to the actions of touch and gesture in digital “interactive” interfaces, the problems with the “seeing hand” metaphor are, firstly, the danger of a regressive interpretation of tactility towards what Derrida called “humanualism”⁵; secondly, the continuity and reinforcement of a fetishization around the metonymy of the hand⁶, where the latter often comes paired with an invisible wall. In other words, where there are “invisible walls”, and minding the inverted commas in this expression, we will certainly find hands and fingers, human or, if you prefer, cyborguean ones.

Figure 2: Touch in the form of hands, as well as interfaces in the form of invisible walls, a double trope found on several covers of essays in digital media.

The recurrent use of the hand as a synecdoche of the biological body⁷ is far from being recent, as it may be attested by painted images found in cave paintings (Dur, 2015: 22). Notwithstanding, such a mystification and fetishization of the hand, despite also belonging to an ancestral collective imagination, seem to be intensified by the ubiquity of screens in both private and public spaces. Take for instance several Hollywood productions, from the 1980’s to the present century. In My Stepmother is an Alien (1988), the main character (Kim Basinger) is an alien creature, who takes the form of a woman to seduce an earthling and steal his scientific project as a means to save her planet. At a certain point, she manages to get access to a personal computer and “read” a series of data in a matter of seconds, by putting the palm of her right hand in contact with the screen.⁸ Not satisfied with her first “digital” results, she also approaches some bookshelves, in order to perform a reading of the Complete Works of Shakespeare by placing her forearm between folios.⁹ Regardless of the ergonomics enhanced by both surfaces, what this co-dependent relationship between body extremities used as prosthesis and surfaces seems to represent is the desire of having direct access to knowledge through cutaneous perception, (almost) without having to use vision.

Figure 3: Microsoft Word Processing Program for Apple Macintosh. Cover. Touching the screen with our hands and fingers, even when we know that this particular kind of interaction is useless.

But the fetishization continues along the history of media in popular culture. An example being Steven Spielberg’s Minority Report (2002), a film that according to some bloggers seemed to function as an experiment for the development of Oblong industries’ g-speak^TM interface. In the film, one is confronted with a futuristic chief of police (Tom Cruise) moving his hands and fingers (actually, a pair of haptic gloves) around a transparent glass full of data provided by three “precogs” able to “previsualize” crimes that are about to happen. Seen as a co-effort between the fields of science and arts, according to blogger Steve Anderson, this HCI is much more than a simple “visual effect”, as the whole sequence seemed to be intended to anticipate “a real system that would operate according to a gestural logic that was not yet technically feasible”.

Finally, reference can be made to a more recent example, the remake of Total Recall (2012), specifically the part where the renewed Douglas Quaid (Colin Farrell) senses a ringing cellphone encapsulated in his right palm. After answering this call from a supposedly unknown person, Quaid is told to look for any kind of glass in order to consummate the communication. And never mind where the audio speakers are, because the sequence is so seemingly “transparent” that we tend to forget the actual constraints of such a “marvelous” prosthesis. As these three examples can be representative of a certain archaeology of the screen (monitors, walls and gadgets embedded in our skin), all of them seem to share the same desire for transparency through tactile/haptic perception.

According to Erkki Huhtamo this desire for transparency goes back to, at least, the 19^th century, where the “screen served as a veil, hiding the secret tricks and the machinery used to conjure them up” (2004: 35-6), which presupposes a much bigger emphasis on visuality. But the real difference, made possible by progress in dig-tech, is that we are now offered the illusion of being able to unveil for ourselves these secret tricks with a simple “Midas touch”. If the hands-on-screen as the ultimate interface can be explained by both ubi-comp and the dream for transparency (plus a desire to access knowledge through a more “direct” way of perception), what to say of our relationship with these gadgets made of cold glass and aluminum, as they are capable of giving us a paradoxical sensation of warmth while we hold them in our hands? Is it just for the fantasy of the image in which we see the reflection of our desires? Considering the projection of desire and fantasy on screens, in his book Interface Fantasy: a Lacanian Cyborg Ontology, André Nusselder provides another possible explanation for the reason we tend to see the screen as the ultimate interface, by recovering Lacan’s theory of the mirror stage in order to draw a parallel with the computer screen. Being a “psychological space” for the projection of desire and fantasy in computer screens, Nusselder tells us that “there is a desire for an ecstasy of the real” (2009: 29). Furthermore, despite his emphasis on virtual worlds and avatar – phenomena that may give us “a unified form to tendencies otherwise experienced as discordant and disturbing, just as the identification with the virtual image does in Lacan’s theory of the mirror stage” (2009: 91) –, Nusselder’s considerations on the consciousness and unconsciousness of the self are not far from the idea of double-fold surfaces:

By picking an avatar, I can formalize certain tendencies (for example, eroticism, aggression, animality) that remain otherwise dark and obscure. Lacan’s point is, precisely, that the unconscious is not this ‘dark and obscure’ inside of the self, but comes to being only in externalization. It is only in the form of, for instance, an avatar that I come to recognize my ‘unconscious intentions’; they do not exist as such before their ‘materialization’. Therefore, the unconscious ‘happens’ at the interface. (2009: 91)

The problem being that along with the search for “virtuosic immersion and connectivity […] these transparent and ubiquitous thresholds become simultaneously invisible and inoperable” (Galloway, 2012: 25). A feeling corroborated by Lori Emerson, stating that “All of these interfaces share a common goal underlying their designs: to efface the interface altogether and so also efface our ability to read, let alone write, the interface, definitely turning us into consumers rather than producers of content” (2014: 1). Still, for Emerson there is some light at the end of the tunnel, namely by means of “a growing body of digital literature” that courts “difficulty, defamiliarization, and glitch as antidotes […] against what ubicomp has become” (2), the “nearly pervasive multi-touch interface” (4) included.

The collection of digital literary works created by the i-Trace collective are a poignant example of the way digital literature may perform an exercise of self-reflection over the triad “device, medium and concept”. And if the very name of this collective already makes us think of the tension between the ephemeral and permanent condition of our digital paths, the physical interaction provided by the i-Trace’s webpage shows us how this tension can become even more pronounced. Designed as a digital literary work, its home page invites the user to unveil it in order to reveal, to lose grasp in order to be aware of, two equivalent formulas that serve as an initiation to the works listed below the headline. An example of this unveiling are the iterations made possible by our interaction. By sliding our mouse/finger across the area surrounding and comprising the headline we start to notice some changes in behavior. Some of them are visual, as dark and grey glitches start to cover the previously white screen (minding that visual glitch can also be a loss of grasp); others are textual, as the line “No trace can last forever” changes into “Press any key to reset”. Yet, by resetting, we are again confronted with the same white surface concealing another possible surface: its reverse side.

Opacity (2012) is one of the works to be explored at the i-Trace’s webpage.¹⁰ Presenting an interactive story divided into four connected parts, the piece is described by its authors as “a journey from a dream of transparency to a desire for opacity”. It is as much a story about human relationships (love, politics, business) as it is a story about transparency and opacity between double-fold surfaces. Each part of the work asks for a specific physical interaction with distinct interfaces, in which the more or less direct actions of hand and fingers – depending on if we are using a PC, a tablet or a smartphone – drive the subject through a state of “an in-between”.

As we open the first part of the piece we are faced with a black screen presenting the contours of a rectangle in its center. Apart from this dark background, there is also a phrase in white saying “This morning everything seems opaque”. The impulse of a need for transparency immediately leads us to unveil what lies behind this rectangle. By moving the mouse/finger across it in all possible directions, bits of metallic pieces are added to the sound, as grey-to-white phrases and the fragmented image of a motherboard’s parts start to be gradually revealed: “I would like to see through my computer/I dream of transparency/I want to have a clear understanding of myself/And of others”. To this desire of using the computer as a hypothetical solution to all of his problems, the subject gets nothing apart from a female voice asking “Where are you?”, a question already denoting a sense of enclosure and division, as well as a lack of contact.

As we are immediately taken to the second part, a picture of a woman’s barely naked body stands before us. Once again, moving the mouse over the image gradually discloses this woman’s interior organs, a sequence that is accompanied by a sound resembling a magnetic resonance (intensified as we “touch” the woman’s body), but also the gradual appearance of a verbal instantiation next to the image. Moreover, as this double gradualism mediating the interaction with the work is the only possible way of progressing in the narrative, the information we need in order to disclose the story is given piece by piece, in a linear fashion: “My wife”, for the first time he refers to the woman in the story as his wife; “What could she be hiding from me”, typical feelings of suspicion between couples, we might think; “I want to see through her”, a desire that the subject already knows is impossible to fulfill and yet one that he previously attempted with himself albeit without success; “Deep down beneath the surface”, a final line pointing to a game of depths and surfaces that seems to be the central key to the inversion of transparent/opaque experiences. Still, as the subject is able to unveil the figurative image of his wife’s internal organs, there seems to be no answer apart from her anatomy, a frustration that is interrupted by his wife’s voice asking “What are you looking for?”.

Part three opens with the phrase “I am nothing without you”, a text that, by following the same iterative movement of dragging the mouse/finger across the screen, we are capable of mutating into two blocks of dispersed sentences, like a dialogue between lovers. Being composed of verbal instantiations only, this third part enables us to think of language in several possible ways. First, as a system composed of several units working together to perform meaning; second, as a technology and a medium in order to communicate with ourselves and with others. However, like every mediated technology, language also has its layers, its transparencies and opacities, its fusions and divisions. Believing that this third part could function as a digital literary work on its own, what we have is the constant attempt of making sense through a continuous tension inherent in the conditions imposed by language. In other words, in order to fix, in a more or less stable way, the sentences that are being created, there is a need for dissolution, in this case, located in the central mutating line. In sum, by means of a climax, part three makes use of language to clear up some of the ambiguous sensations and oblique connections between both parts of the story (also, of course, between the reader and the work, the separate parts of the work, etc.), always keeping in mind that there are certain things that cannot be as transparent as one would wish.

Figure 4: Screenshot. Opacity, by the I-Trace collective. Screenshot.

By the time we arrive at part four of our/his gradual exploration, the sense of disclosure becomes stronger, reaching a turning point (notice that the sound is now much calmer, less disturbing). Confronted with the misted glass of a shower door hiding his wife taking a bath, and with the impossibility of unveiling his wife’s thoughts by himself, the subject faces the paradox’s punchline: “I am not looking for transparency anymore/But for opaque/interactions and sensations”. In brief, there is always a need for a certain sense of opacity in order to reveal illusory transparencies, through literature, in this case. Moreover, it is a final disclosure of the impossibilities of fully touching the other, reminiscent of what Derrida, drawing on Jean-Luc Nancy’s philosophy of touch, states in the following way: “One should understand tact, not in the common sense of the tactile, but in the sense of knowing how to touch without touching, without touching too much, where touching is already too much.” (2005: 67)

A Stratification of Surfaces

Let us note that the depth of the Mystic Pad is simultaneously a depth without bottom, an infinite allusion, and a perfectly superficial exteriority: a stratification of surfaces each of whose relation to itself, each of whose interior, is but the implication of another similarly exposed surface.

— Jacques Derrida, Writing and Difference

In her article “Convergent Devices, Dissonant Genres: Tracking the ‘Future’ of Electronic Literature on the iPad”, Anastasia Salter analyzes the impact of the iPad interface in the context of literature. Drawing a line from pure remediations of print to children’s “interactive” books, adapted graphic novels/motion comics and finally the niche(s) of a more conceptual/experimental electronic literature, her contribution to the state of the art of e-lit points to a series of inescapable issues. An example of this is the potential of the iPad’s interface to “transform our point of entry into electronic literature at the physical level”, or its power as a site of tension and convergence, “tension between faithful remediation of the codex and the breaking of the page, between genres of fiction and genres of play, between turning the page and reinventing the text” (2013: §1). While Salter sees the interface’s “active physicality” as being “essential to its potential to redefine our interactions with books” (2013: section “Interfacing with Poetics”, §2), her emphasis on one particular multi-touch device, i.e., Apple’s iPad, seems to open a pitfall in her argument, specifically concerning the experimentation allowed by the iPad’s interface as the alleged key of its potential (this experimentation being what makes the iPad a “revolutionary device”). While Salter is, of course, aware of its current stages of experimentation and predictions of a near-future obsolescence, we could still reinvent here Sandy Baldwin and Rui Torres’ view of literature as a technology that “includes the computer and the web, not the other way round”, (2014: xvi) by stating that literature may (or may not) include tablets, smartphones and other similar devices. Regardless of all the groundbreaking features that may come with an iPad, tactility does not seem to be one of them, as demonstrated by previous technological and artistic experiences that summoned this complex sensory modality. Take the case of Alan Kay’s work on object-oriented programming and windowing GUI design, both synthesized in the cardboard mock-up of the Dynabook (1968) – one of his several projects imbued with the McLuhanist idea of media as extensions of man and accessible to everybody, especially children. Furthermore, Myron Krueger’s “responsive environments’s experiences, e.g. VideoPlace (1970-1984), an interactive system that, according to Lori Emerson, and particularly concerning multi-touch interfaces, “came to include such a remarkably rich collection of gestures and multifinger, multihand, and multiperson interaction that by comparison contemporary devices such as the iPad seem like nothing more than pale imitations” (2014: 21).

The whole new experience of intimacy provided by this touch-based equipment is also one of the strongest claims in Salter’s article, particularly by focusing on the “shift of the screen from the laptop profile to the form of a notepad or book”, a change from “typical models of digital interfaces” asking “the user to manipulate an object removed from the display” to one that does not require “a mental leap” where the actions may now “intersect the corresponding digital movement” (2015: section “The Formless Screen?”, §5). Yet, considering notepads that change “our relationship with the medium,” what can be said of Freud’s “Note Upon the ‘Mystic Writing Pad’” (also known as the Wunderblock)? In his analogy between this “children’s toy” and the “functioning of the perceptual apparatus of our mind”, Freud starts by saying that,

All the forms of auxiliary apparatus which we have invented for the improvement or intensification of our sensory functions are built on the same model as the sense organs themselves or portions of them: for instance, spectacles, photographic cameras, ear-trumpets. (1925: 212)

As for his description of the Mystic Writing Pad’s composition, while everything seems to happen at the surface, his idea of surface is far from having the connotation of superficial:

[…] a small contrivance […] nothing more than a writing tablet from which notes can be erased by an easy movement of the hand”, [composed of three layers] “a slab of dark brown resin or wax with a paper edging […]” [and] “a thin transparent sheet” [containing two other layers], “a transparent piece of celluloid” [and a lower layer made of] “thin translucent waxed paper”. (1925: 209)

Derrida called it “a stratification of surfaces” (1978, 281), another way of talking of a complex surface composed of a double-fold nature, that Freud, in his notes, identifies with relation to the perceptual system:

I showed that the perceptive apparatus of our mind consists of two layers, of an external protective shield against stimuli whose task it is to diminish the strength of excitations coming in, and of a surface behind it which receives the stimuli, namely the system Pcpt.-Cs.[Perception-Consciousness] (1925: 210)

And he ends with the example of “one hand writing upon the surface of the Mystic Writing-Pad while another periodically raises its covering sheet from the wax slab”, (1925: 212) an ergonomic observation that seems to anticipate some of our current uses of digital tablets, as mentioned in Salter’s article (here quoting Janez Strehovec):

With a stylus or touch screen we can come into very direct, although virtual, contact with the word, contact that is much more immediate and intimate than using a typewriter, which means that these devices once again establish an immediate relation between the body (in fact, the hand) and the word. (2015: section “Interfacing with Poetics”, §1)

And yet, as tempting as this remark may look, considering this alleged return to immediacy, we may ask in what ways are we now able to touch a word, beyond its figurative meaning, even if materialized as a tridimensional object and not just a kinetic touchable glass screen alphabet. It seems that as much as we talk about perceptual immediacy, it doesn’t seem plausible to escape the fact that words are chains of differential abstractions maintained by a system of traces. And, in this sense, the surface is no more than a trace of the word, not the word itself.

Nonetheless, as multi-touch devices (particularly, the iPad) were able to change the way we interact with computers, it is only natural to assume an increment of research and artistic experiences around these devices. A reality recently observed by Jason Edward Lewis, director of the Obx Labs, at the ELO2015 conference (Bergen, Norway), while presenting his and Bruno Nadeau’s P.o.E.M.M. project, a “series of poems written and designed to be read on touch devices, from large-scale exhibition surfaces to mobile screen” [http://www.poemm.net/]. Being confronted with the question of how he saw the increase of tactile/haptic processes by multi-touch devices (and of electronic literature metamedially exploring this increment), in the context of all the artwork and research made by the Obx Labs since its first artistic experiments, his answer was much more pragmatic than one would expect. Specifically, by stating that such an increase came naturally with the availability of these devices in the market, an opportunity he and his team believed to meet their demands. Making use of the intimacy provided by the iPad’s portability and scale (“It wasn’t small enough to fit into my pocket but it was still mine, it still felt intimate”), Jason Lewis sees the iPad as currently the better technological option to deal with poetry (also, “an intimate experience”)¹¹. Yet, his pragmatism reveals something more than a technological status quo and trendy features, as his equation between poetry and intimacy points less to a necessary relation between the iPad and poetry, than to a certain idea of poetry, that is directly connected to a historical and cultural model of intimacy associated with a certain reading habitus already provided by pocket books from both manuscript and typographical cultures.

However, without directing attention at all kinds of artistic experimentation focused on multisensory perception, I believe that the difference lies less in the “active physicality” and in its supposed individualization “marketed as a more personal computing experience than the ‘personal’ computer has ever conveyed” (Salter, 2015: section “The Formless Screen”, §5), than in the materialities of the text conveyed by the device. As stated by John Cayley,

We speak of the ‘materiality of the text’ or the ‘materiality of language’ in general, as if this might be an abstract characteristic when, in fact, it is the critical marker of linguistic and literary embodiment, recognizable only in terms of that embodiment. As N. Katherine Hayles puts it, ‘The materiality of an embodied text is the interaction of its physical characteristics with its signifying strategies’ (2003, 277; emphasis in original). (Cayley, 2006: 307)

Regarding digital literature that focuses its auto-reflexive poetics on the relationship across certain multisensory modalities such as tactile/haptic perception, we find that sometimes there is no need for multi-touch devices in order to offer a “touching” and “touchable” experience. For instance, Serge Bouchardon, Kevin Carpentier and Stéphanie Spenlé’s Touch: Six scenes on the paradox of screen touching (2009), a Flash and Javascript poem that explores “touch” in its multiple meanings¹²).

“Through a glass, darkly”

First, there’s the room you can see through the glass—that’s just the same as our drawing room, only the things go the other way. I can see all of it when I get upon a chair—all but the bit behind the fireplace. Oh! I do so wish I could see THAT bit! I want so much to know whether they’ve a fire in the winter: you never CAN tell, you know, unless our fire smokes, and then smoke comes up in that room too—but that may be only pretence, just to make it look as if they had a fire.

— Lewis Carroll, Through the Looking Glass

In “Performing Apps Touch and Gesture as Aesthetic Experience” (2013), Maria Engberg makes use of a multisensory¹³ perspective in order to present several digital literary works specifically designed for the iPad. Regarding the ways we touch and feel these “immediate” experiences, Engberg states that “[t]he alluring and directly responsive interface hides of course high-tech machinery and while the promise is that of an immediate and immersive experience, it is nevertheless a mediated one” (2013: 23). Yet, while the argument of the black-boxed device seems to be prevalent in several research papers from the field of Media Archaeology (e.g. Emerson, 2014), the last part of Engberg’s conclusion – “([…] we are touching interfaces, perhaps even code, as we are touching glass after all” [my emphasis] (2013: 27) – is ambiguous enough to occlude the materialities of surfaces involved in these mediations. That is to say that we cannot touch code anymore than we can touch words. Being the abstract representation of voltage differences (not the voltage values in themselves), code is as symbolic as natural language, and thus it cannot be touched. Plus, it is part of a “series of cascading abstractions with several layers nested on each other” (Portela, 2013: 169), with folds and depths that go beyond a possible representation on a complex surface. Symbols manipulating symbols manipulating other symbols, with the verb to manipulate directly referring to the machine, not to our hands.

Modern tablets and smartphones may come equipped with either a resistive or a capacitive touchscreen. Resistive touchscreens respond to the mechanical pressure of a finger or a stylus, as it is composed of two thin layers, each one with a coating, separated by a small gap. The voltage is passed through as the two coatings touch each other, processing this pressure as a touch. As for capacitive sensing (the one we find on iPads and iPhones), it relies on the electrical properties of our body. Capacitive touchscreens come with an insulator, typically made of glass, with an interior coating of transparent conductive material, which is the reason for the production of changes in the screen’s electrical field as we touch them with one or several fingers.¹⁴ As both types of touchscreens need direct physical contact in order to function, the electrical charges involved in these human-machine interchanges makes us think of the more “direct” ways we may sense to be touching and manipulating code. And, in this sense, much of this illusion seems to be provided by glassy surfaces and the effects they are able to cause.

Figure 5: Alicia 3.0 (2015). Ink-jet on double plexiglas. © Mário Lisboa Duarte and Bruno Martins. Reproduced with permission.

However, even though it may seem almost impossible not to retain the Flusserian metaphor of the “black box” at the level of the screen, it may be worth giving a closer look at what lies in the middle of the transparency/opacity paradox: i.e. translucency.

Taking into account the ways some of the architectural vocabulary is shared by both buildings and computers, and especially having in mind glass as its common material surface, glass may be anything but innocuous. Multi-touch devices now seem to congregate all the potentialities that modern architecture saw as a groundbreaking future, for instance, transmission, reflection, diffraction or refraction of light (plus resistance, lightness, among other characteristics). In fact, the so-called Glass Age prospered with these promises of transparency. Notwithstanding, there was always a catch along with these promises, easily illustrated by the move from small transparent surfaces in the shape of windows into large opaque panes of dark glass. For Mark Hansen, glass, as a “preeminent technology of modernist architecture (…) has opened the path to our massively surveillant society” (2006: 214), a premise that was latent in Marshall McLuhan’s considerations on the “reason why the story of glass is so closely related to the history of housing.” Not to mention the mirror as “a main chapter in the history of dress and manners and the sense of the self.” (McLuhan, 1964: Chapter 13, §8) As for Walter Benjamin’s exposé of 1935, “Paris, the Capital of the Nineteenth Century”, the prevalence of glass already seems to be a reality (despite the fact that it was considered by many as an utopia). In addition, since much of what Benjamin says of glass came by influence of Paul Scheerbart’s Glass Architecture (1914), it is worth recovering an excerpt of this historical account:

We live for the most part in closed rooms. These form the environment from which our culture grows. Our culture is to a certain extent the product of our architecture. If we want our culture to rise to a higher level, we are obliged, for better or for worse, to change our architecture. And this only becomes possible if we take away the closed character from the rooms in which we live. We can only do that by introducing glass architecture, which lets in the light of the sun, the moon, and the stars, not merely through a few windows, but through every possible wall, which will be made entirely of glass –of coloured glass. The new environment, which we thus create, must bring us a new culture. (Chapter 1, §1)

Figure 6: Front page of Keppler Glass Constructions’ catalogue (1914). The beginning of a dream for transparency? Or a more translucent one?

Finally, we could finish the list of references on the uses and misuses of glass in modern architecture with Fredric Jameson’s account of

[…] the great reflective glass skin of the Bonaventure, whose function might first be interpreted as developing a thematics of reproductive technology. Now, on a second reading, one would want to stress the way in which the glass skin repels the city outside (…). In a similar way, the glass skin achieves a peculiar and placeless dissociation of the Bonaventure from its neighbourhood: it is not even an exterior, inasmuch as when you seek to look at the hotel’s outer walls you cannot see the hotel itself, but only the distorted images of everything that surrounds it. (1998: 13)

It is not difficult to foresee a near-future entirely made of glass more or less transparent/opaque, which we can only imagine.

What we know for sure is that glass is able to change our connection to the world (for instance its use in optical fiber), and above all, the way we see the world, which can explain much of its early industrial reputation, but also its uses among early twentieth-century artists. Search any avant-garde exhibition catalog or manifesto and glass will be found among the list of cherished materials, in several of its possible shapes and forms. Take for example, Boccioni’s Technical Manifesto of Futurist Sculpture (1912), in which the Italian painter and sculptor sees glass and other less noble materials as the proper tools to translate “those atmospheric planes that link and intersect things” into a “new plastic art” that refuses “the wholly literary and traditional nobility of marble and bronze” (§4).

Nevertheless, when it comes to the use of glass by early avant-garde proposals, Marcel Duchamp’s La mariée mise à nu par ses célibataires, même, also known as The Large Glass (1915-1923), seems to overshadow everything else. Certainly the most intriguing work of Duchamp and definitely a work where glass is the prominent material. However, given Duchamp’s choice for this material and the hermeneutics around this work, there are few approaches directly concerned with its materialities. According to Calvin Tomkins, the reason for Duchamp’s choice was both pragmatic and idealistic. Not only did he notice that “a painting on glass could be sealed hermetically, which would prevent or at least delay the gradual oxidation that causes pigments to fade and change color”, but he also found in this new medium a way of breaking with what he called “retinal art”. Progressively less interested in “art for the eye alone” and determined to erase all traces of “the artist’s personal touch”, Duchamp decided to go for glass instead of the traditional canvas (1966: 34).¹⁵

Being less transparent than translucent, The Large Glass contains a plurality of dimensions. Duchamp’s descriptions of the work’s four-dimensionality open the window to a game of surfaces made possible by the material in question. Through the glass one is not only able to sense the series of “cogs” set by Duchamp in order to put the gear into motion, but also one’s reflection mixed with the space lying before and behind the glass planes. And yet, as it is not a work open to rational description and interpretation, by placing less emphasis on the visual result than in its process, is it viable to go beyond these vitreous layers of translucent surface? If we apply the formula to the glassy surfaces of an iPad, is it possible that, in the same way the Large Glass “resists a consistently transparent view because it includes the reflection of the observer and his or her environment in its image” (Shanken, 2003: 78), everything is destined to happen on the screen, “the site of interaction and negotiation for meaning”? (Ascott, 2003: 235)

This pertains to what Roy Ascott referred to as “screens of operations” rather than “screens of representation” (2003: 235), a potential counterpoint to Duchamp’s idea of “retinal art”, as it is possible to claim that most screen displays tend to open onto visual representations that are “retinal”, instead of “procedural”. Apart from a few exceptions, screens have been predominantly places of representation rather than fields of operation, wherein the transparency provided by its glassy surfaces frequently has its corollary in the retinal display of images that attempt to substitute our optical perception of the world. In other words, where there is a promise of transparency there is also a promise of convergence between technical image and perceived image. In short, the perfect cyborg human-machine interface, where the surfaces of the interface are coupled into a single perceptual system, an externalized body internalized as a natural interface.

Alex Gibney’s documentary Steve Jobs: The Man in the Machine (2015) talks of a dream of glass that came into life, more specifically, Steve Job’s dream of changing the world. As Apple’s transparent cube in NYC tends to be seen as a representation of the company’s interpretation of transparency, it is only a small fraction of the bigger building. In fact, Gibney’s personal disquietude on Job’s gradual sacralization by society (and consequently, on Apple) tells the opposite of transparency. This becomes even clearer as we reach the final sequence, in which we are confronted with a zoom-in of realistic Japanese gardens that, in the end, reveal themselves to be only an image on the screen of an iPad. Meanwhile, we hear Gibney’s thoughts:

[…] He offered us freedom, but only within a closed garden to which he held the key. […] As Jobs wanted it, the screen of my iPhone is dark, a zen-landscape of the unseen. If I stare into it, I see the obscure reflection of myself. But this impression lasts just a fleeting moment, before I press the home key and the screen lights up.

By lighting the screen, the world seems to be in our hands, with all of our decisions being conducted by the tip of one’s finger. But such a light is also the reason why the transparency/opacity dichotomy is impossible to be dissociated from a visuocentrist perspective. In Paul Levinson’s adaptation of Marshall McLuhan’s ideas on “media as extensions of man” to the digital age, we are told that:

The simple transparent properties of glass, in contrast to the reflective qualities of opaque surfaces, render this common technology –and the more sophisticated technologies like TV and computer screens that employ it– as profoundly informative in its own way as a particle chamber. Of course, glass backed-up by silver serves as a mirror, and therein works as a deliberate exception to its otherwise light-through performance. McLuhan seizes on this technological commonplace too, and finds meaning in it for our understanding (or misunderstanding) of media: ‘The youth Narcissus mistook his own reflection in the water for another person. This extension of himself by mirror numbed his perceptions’ (McLuhan, 1964, p. 51), with the result that he drowned. Looking at the surface of the water, rather than through it, thus proved not only superficial and misleading but deadly in this mythical instance. Not that looking through things is easy or without risk. But it does provide us with an opportunity to get beyond our own reflections. (1999: 98)

Through Levinson’s distinction between transparent glass and mirrored glass, it becomes clear that McLuhan’s use of the myth of Narcissus (curiously, in a chapter called “The Gadget Lover”), is only to say that what is at stake is not the image of Narcissus falling in love with himself (and consequently drowning) but rather the “narcosis”, or “numbness”, that this reflection was/is able to cause. If we add to this numbness a glassy surface like the ones that are being marketed by Apple – “magical” and “invisible” interfaces –, we may literally and metaphorically sense the perverse image of our personal reflection that this device is able to provoke.¹⁶ Considering that the use of the Greek word “narcosis” by McLuhan was no more than a way of saying that our biological body finds its balance for these extensions with self-amputation, we may also ask if these intimate experiences through touch are not subject to the same numbness McLuhan referred to? And if so, how can we manage to see through its touching glass? That is to say, the more we touch, the less we see (an observation that does not contradict the paradoxical emphasis on visuality). What may be seen as a conscious touch or manipulation is, instead, an unconscious, almost automatic immersive experience towards alienation. In order to revert the process, one must indubitably question these surfaces (starting by glass), digital literature being one of the possibilities of doing it. On the one hand, through the creation of auto-reflexive literary works focused on deliberate intentions of provoking a certain loss of grasp in order to raise awareness; and on the other hand, by accepting these reading experiences as an operation of estrangement that allows us to move back and forth on both sides of the surface. In a way, like a wary Alice on the threshold of her looking glass.

To recover Lori Emerson’s thoughts on the role of digital literature, there may be more behind the “gloss of the surface” than one would think at first glimpse (and touch). Among others, Emerson gives the example of Erik Loyer’s Strange Rain (2011), a digital literary work that questions a familiarity that is often taken for granted. Described by Leonardo Flores as being “made of equal parts narrative, game and poem”, this universal app developed by opertoon for iPad, iPhone and iPod Touch, is an aesthetic experience that is able to defy common assumptions of sensory perception. In the end, by putting all the pieces together, one may find that it is all about defamiliarization, an assumption that confirms opertoon’s synopsis of the work: “before your eyes and beneath your fingers, the familiar becomes strange, and the strange familiar.” (Source)

Strange Rain may be experienced in three gradual modes: “Wordless”, “Whispers” and “Story” mode. The first one provides us with the work’s ambience, a skylight on a rainy day, with raindrops hitting the screen and the possibility of being guided by our fingers. The second mode “adds another layer to the experience” (Loyer, 2011), with raindrops changing into words, as we tap the screen. The third mode presents us with a narrative, “Convergence”, in which a character who is having a family crisis (after his sister’s car crash), goes to a backyard “to just think things through a bit, and get rained on, and contemplate what’s going on” (Loyer, 2011). Single taps on the screen allow us to activate his thoughts, one by one, or as a sequence if we drag our fingers across the screen. As we tap and drag the main character’s (Alphonse) rainy thoughts, we start to sense unfamiliar phenomena, such as temporarily layered skies, an airplane passing over, “visual anomalies and shifts in speed and color” (Loyer, 2011), and a literal haptic rainstorm that truly makes us brace ourselves to the device. Since it is also a game, in order to complete all of the achievements/levels, one needs to be constantly alternating between the actions of slow and fast tapping, a constant tension that seems to be the only way of revealing Alphonse’s own anxiety between going inside or staying outside, which can also mean the acceptance or the refusal of his/our crisis. This tension we sense – not only as we progress through the story, but also through force feedback triggered by the device – is defined by Mark Sample in terms of “haptic density – because it reveals the outer edges of the interface of the system” (Sample, 2012).

As we gradually negotiate these complex surfaces, making use of touch and gesture as a necessary mediation in order to experience hidden layers lying behind the visible surface of glass, we also experience not only Alphonse’s anxious thoughts, but, above all, our own anxiety towards the device. Transmitted as a reflection of thoughts that start to become our own, Strange Rain is also a reflection of the ways we read and write through touch and gesture in these haptic inter[(surf)aces].

To grasp a thought

Tactility is involved with thought whether in our minds or in our machines, as a participant in the thinking process.

Derrick De Kerckhove, The Skin of Culture

The double-fold nature of haptic interfaces is, primarily, a temporal stratification of surfaces. In this sense, the traces left on the glassy surfaces of digital screens are no less of a palimpsest than the Wunderblock. Yet, an inversion of visible and invisible, opaque and transparent layers is necessary in order to fully understand the scope and nature of these traces. In fact, for the purposes of tactile/haptic perception, other dichotomies might be more appropriate. Concerning the layered nature of complex surfaces discussed in the beginning of this article, I offer to consideration the idea of an approach/departure paradox, possible to be represented, for instance, in the haptic motion of zooming in and zooming out.

Through the analysis of four different kinds of “digital” literary works (one that mixes print and screen-based texts, one created to be plastically experienced on a PC, a tablet or a smartphone, another one requiring the use of distinct interfaces in conflation with a PC, and finally one specifically designed for multi-touch devices such as the iPad and the iPhone), I tried to show that the specific metamedial poetics that define these works of electronic literature can assume different forms of rethinking the cultural effects of interfaces.

In addition, as digital touch and gestures are becoming an increasing part of our lives, and as they tend to be indistinguishable from actual touch and gestures, so too are the literalizations/instrumentalizations of tactile/haptic perception. The previously proposed definition of a haptic reading experience may be a way of subverting the process, particularly by rethinking the whole circuitry. In fact, by means of what I call a “machimanipulation”, a manipulation shared by both human and machine, digital literary works are questioning the device, perhaps like never before, in order to consider the bigger apparatuses enclosed between us and the machine – including language.¹⁷

Bibliographie

AGAMBEN, GIORGIO. 2009. WHAT IS AN APPARATUS? AND OTHER ESSAYS.
ASCOTT, ROY. 2003. TELEMATIC EMBRACE: VISIONARY THEORIES OF ART, TECHNOLOGY, AND CONSCIOUSNESS.
BENJAMIN, RICHARD. 1988. MY STEPMOTHER IS AN ALIEN.
BENJAMIN, WALTER, MICHAEL W JENNINGS, BRIGID DOHERTY ET THOMAS Y LEVIN. 2008. PARIS, THE CAPITAL OF THE NINETEENTH CENTURY.
BOCCIONI, UMBERTO. TECHNICAL MANIFESTO OF FUTURIST SCULPTURE (1912).
CAYLEY, JOHN. 2006. “TIME CODE LANGUAGE: NEW MEDIA POETICS AND PROGRAMMED SIGNIFICATION”.
CAYLEY, JOHN. 2005. WRITING ON COMPLEX SURFACES.
CHUN, WENDY HUI KYONK. 2005. ON SOFTWARE, OR THE PERSISTENCE OF VISUAL KNOWLEDGE.
COLOVER, JOEL. 2010. SOME IDEAS ABOUT SURFING.
DELEUZE, GILLES. 1990. THE LOGIC OF SENSE.
DELEUZE, GILLES ET FÉLIX GUATTARI. 1987. A THOUSAND PLATEAUS.
DERRIDA, JACQUES. 2005. ON TOUCHING – JEAN-LUC NANCY.
DERRIDA, JACQUES. 1978. WRITING AND DIFFERENCE.
DUR, B. U. 2015. HAND IMAGE AS A METAPHOR AND ITS USAGE IN POSTER DESIGN.
EMERSON, LORI. 2014. READING WRITING INTERFACES: FROM THE DIGITAL TO THE BOOKBOUND.
ENGBERG, MARIA. 2013. PERFORMING APPS TOUCH AND GESTURE AS AESTHETIC EXPERIENCE.
FLORES, LEONARDO. 2012. REVIEW OF ‘STRANGE RAIN’ BY ERIC LOYER.
FREUD, SIGMUND. 1925. A NOTE UPON THE ‘MYSTIC WRITING PAD.
FULKERSON, MATTHEW. 2014. THE FIRST SENSE. A PHILOSOPHICAL STUDY OF THE HUMAN TOUCH.
FUNKHOUSER, C. T. 2012. NEW DIRECTIONS IN DIGITAL POETRY.
GALLACE, ALBERTO ET CHARLES SPENCE. 2014. IN TOUCH WITH THE FUTURE: THE SENSE OF TOUCH FROM COGNITIVE NEUROSCIENCE TO VIRTUAL REALITY.
GALLOWAY, ALEXANDER R. 2012. THE INTERFACE EFFECT.
GIBNEY, ALEX. 2015. STEVE JOBS: THE MAN IN THE MACHINE.
HANSEN, MARK BN. 2006. BODIES IN CODE: INTERFACES WITH DIGITAL MEDIA.
HUHTAMO, ERKKI. 2007. TWIN–TOUCH–TEST–REDUX.
HUHTAMO, ERKKI. 2004. ELEMENTS OF SCREENOLOGY: TOWARD AN ARCHAELOGY OF THE SCREEN.
JAMESON, FREDERIC. 1998. THE CULTURAL TURN: SELECTED WRITINGS ON THE POSTMODERN (1983-1998).
KAY, ALAN. 2002. USER INTERFACE: A PERSONAL VIEW (1989).
LEVINSON, PAUL. 1999. DIGITAL MCLUHAN: A GUIDE TO THE INFORMATION MILLENIUM.
LOYER, ERIK. 2011. STRANGE RAIN PREVIEW.
MARQUIS, ALICE GOLDFARB. 2002. MARCEL DUCHAMP: THE BACHELOR STRIPPED BARE.
NANCY, JEAN-LUC. 1996. ON THE THRESHOLD.
NUSSELDER, ANDRÉ. 2009. INTERFACE FANTASY: A LACANIAN CYBORG ONTOLOGY.
PATERSON, MARK. 2007. THE SENSES OF TOUCH: HAPTICS, AFFECTS AND TECHNOLOGIES.
PORTELA, MANUEL. 2013. SCRIPTING READING MOTIONS: THE CODEX AND THE COMPUTER AS SELF-REFLEXIVE MACHINES.
PORTELA, MANUEL. 2010. THE MACHINE IN THE TEXT, AND THE TEXT IN THE MACHINE.
SALTER, ANASTASIA. 2015. CONVERGENT DEVICES, DISSONANT GENRES: TRACKING THE “FUTURE” OF ELECTRONIC LITERATURE ON THE IPAD.
SAMPLE, MARK. 2012. STRANGE RAIN AND THE POETICS OF MOTION AND TOUCH.
SCHEERBART, PAUL. 1914. GLASS ARCHITECTURE (EXCERPT).
SHANKEN, EDWARD A. 2003. FROM CYBERNETICS TO TELEMATICS: THE ART, PEDAGOGY, AND THEORY OF ROY ASCOTT.
2002. MINORITY REPORT.
TOMKINS, CALVIN. 1966. THE WORLD OF MARCEL DUCHAMP.
TORRES, RUI ET SANDY BALDWIN. 2014. PO.EX: AN INTRODUCTION.
WISEMAN, LEN. 2012. TOTAL RECALL.
VIRILIO, PAUL. 1997. OPEN SKY.

1
Differences between the notions of “tactile” and “haptic” were never consensual among different fields of knowledge. According to Gallace and Spence, “in much of the literature on ergonomics and human interface design, the term ‘haptic’ has been used to describe the stimulation of the participant’s skin/body […]. However, in the cognitive psychology/psycophysics literature […] this term has a very specific meaning, one that is restricted to describing those tactile stimuli that impinge on the skin, and which are perceived by means of a person actively palpating an object or surface, such as when actively exploring an object held in the hand. By contrast, the term ‘tactile’ is used to describe those stimuli that are delivered passively to the skin surface” (Gallace & Spence, 2014: 162). Nonetheless, there is also a long tradition in philosophy drawing attention to the aporetic qualities of these notions. For instance, Jean-Luc Nancy tells us that the etimology of the verb haptein goes beyond its common association with the idea of being able to touch, to grab, to grasp, to attach, to fasten, as it can also mean “to hold back, to stop” (Nancy, 2008[2003]: 15). In turn, a more empirical approach like the one given by Matthew Fulkerson (2014: 6), associates tactile sensations with the cutaneous (“those which are processed solely by sensory receptors located in the skin […] mostly passive”), distinguishing it from haptic perception (“a combination of cutaneous stimulation along with kinesthetic and other motor feedback […] typically active, controlled, and almost always object directed in their character”). For the purposes of this article, we prefer to follow Mark Paterson, relating “haptic” “to the sense of touch in all its forms”, including proprioceptive, vestibular, kinaesthetic, cutaneous and tactile perception, the latter described as “pertaining to the cutaneous sense, but more specifically the sensation of pressure (from mechanoreceptors) rather than temperature (thermoceptors) or pain (nociceptors)” (Paterson, 2007: ix).
2
Drawing from its common Foucauldian use as a “set of practices and mechanisms (both linguistic and nonlinguistic, juridical, technical, and military) that aim to face an urgent need and to obtain an effect that is more or less immediate”, Giorgio Agamben redefines the term “apparatus” as “anything that has in some way the capacity to capture, orient, determine, intercept, model, control, or secure the gestures, behaviors, opinions, or discourses of living beings. Not only, therefore, prisons, madhouses, the panopticon, schools, confession, factories, disciplines, juridical measures, and so forth (whose connection with power is in a certain sense evident), but also the pen, writing, literature, philosophy, agriculture, cigarettes, navigation, computers, cellular telephones and – why not – language itself, which is perhaps the most ancient of apparatuses […].” (2009: 14)
3
In order to better understand the broad definition of “bodies” I try to convey in this article (broad, in the sense that it goes beyond the general assumption, inherited from cartesian dualism, of body as in biological body), it may be helpful to recover N. Katherine Hayles’s definitions of “intermediation” and “dynamic heterarchy”, here wonderfully summed up by Manuel Portela: “Hayles adopts the concept of human-machine intermediation […] as a literal and metaphorical description of the ‘dynamics of human-computer interaction’ (Hayles, 2008: 51). ‘Intermediation’ means the emergence of complex patterns from local interactions resulting from ‘recursive feedback and feedforward loops’ (Hayles, 2008: 48) in a dynamic heterarchy. Quoting research in the fields of artificial life, genetics, and computer programming, she argues for the co-evolution of embodiment and technology, rather than for any strict form of social or technical determinism. Human-machine intermediation is also modeled on what recent research has uncovered about human brain development. Hayles sees such intermediation as similar to the co-evolution of brain and language or to the synergy between neural plasticity and learning. The recursive loops that take place inside the machine and the recursive loops that constitute a human subject’s cognition processes become linked in human-computer interaction. In electronic literature this dynamic between body and machine takes place both at the level of writing and reading. A series of cascading and interacting processes makes human and machines part of a single system of emergent cognition” (Portela, 2010: §9).
4
A more detailed analysis of these readers’ behaviors and characteristics can be found in Emerson, 2014: 183-84, as well as in Portela, 2013: 343-47.
5
In his book On Touching–Jean Luc Nancy, Derrida identifies a chain of haptic philosophies that he sees as a result of a post-cartesian “haptocentrism”, where touch often tends to be primary to vision. Going from Maine de Biran’s statements to Husserl’s ideas on perception, Derrida also detects a firmly rooted “humanualism” in a certain branch of philosophy that goes back to Aristotle. (Derrida, 2005: 181)
6
For instance, the Hamsa, or Hand of Fatma, a symbol representing an open eye in the palm of a hand, frequently used as a talisman in many cultures. (Dur, 2015: 22) Curiously enough, this symbol is also the image chosen by Linden Lab for the VR platform Second Life.
7
For the recurrence of the hand trope in poster design, see also Dur, 2015: 19-28.
8
See Benjamin, 1988: min. 44:00-44:25. The full video is available on Youtube. Accessed on May 15, 2015.
9
Id.: min. 44:25-45:26.
10
Co-authored by Serge Bouchardon, Leonard Dumas, Vincent Volckaert and Hervé Zénouda (sound design), the work also had the contribution of Giovanna di Rosario (Italian version), Valerie Bouchardon (English version) and the Laboratorio de literaturas extendidas y otras materialidades (Spanish version).
11
The full video of Jason E. Lewis presentation can be accessed here: [https://mediasite.uib.no/Mediasite/Play/c380c8f4a9f742bba8bd51268b93dfb91d?catalog=32d41cb3-5cd7-489f-bd55-f8f2b08528f9, accessed on September 11, 2015]
12
For the purposes of this article, I believe that it can be more productive to make a reflection based in some of the work’s general characteristics than to do an extensive close reading of all of the multiple possibilities for each iteration. For an accurate close reading of Touch, see Funkhouser, 2012: 107-114.[/fn].

As we enter the home page of Touch (the English version was used) we are faced with the outline of an open hand against a black background, wherein each finger corresponds not to each one of the five exteroceptive senses, but to five different scenes representing five different ways of touching –“Move”, “Caress”, “Hit”, “Spread”, “Blow”, plus a sixth one, “Brush”, “dissimulated in the interface”. Moving the mouse over the tip of a finger activates a different scene, some of which ask for special multisensory equipment: mouse, headphones, microphone, and a webcam for the hidden scene. This need for specific interfaces seems to be one of the main gateways to raise awareness of the interfaces as constitutive of the work. Two examples of this include, firstly, the need for a webcam in the “Brush” scene that, according to Bouchardon, “enables the reader to touch with his eyes” (Bouchardon, 2009: 1); secondly, the suggestion for the use of headphones in the “Spread” scene, as it allows “the reader to touch the music” in a “more user-friendly” way (emphasis in original). As these “technical mediators” call our attention to the paradox of creating “an artistic web-based creation on touching […] multimedia content on screen”, according to Serge Bouchardon, it also “reveals a lot about the way we touch people, objects in everyday life” (2009: 1).

In a growing number of research studies dedicated to the sense of touch, namely from the fields of cognitive neuroscience and psychophysics, we find that touch is far from being a “direct” way of perception. Without attention to the multiple intersections between our sensory receptors and our neurological system (most of them yet to be found), Gallace and Spence remind us that the complex system of our skin is “the most powerful interface ever designed between our self and the objects that surround us” (2014: 19). As stated by Manuel Portela, concerning his analysis of Touch, “[h]uman-machine interactions mediated by the sense of touch are recontextualized through those literary uses of the sense of touch as metaphorical expression of the possibilities of connecting with others and with oneself” (2013: 43) – a trope that can also be found in Opacity, as well as in other works proposed by Bouchardon et al. These “intermediations” between complex surfaces of distinct interfaces are clearly another way of describing the “haptic inter[(surf)aces]” which I mentioned in the main title of this article. This much is suggested when we reach the “Blow” scene, since one of the possible ambiguous textual iterations reads “I can only touch a surface”.

The lack of verbal content in some of the scenes (e.g. “Spread” and “Brush”) should not stop us from classifying Touch as a literary/poetic artefact in its own right. According to C. T. Funkhouser, “[T]he absence of language reflects electronic literature’s variability in formation, where expressivity emphasizes tactility over syntactical reasoning” (2012: 111). By means of an intense questioning of the ways we tend to literalize touch as an exteroceptive sensory modality, Touch ultimately points towards an interrelationship between sensory modalities that, regarding literature, could be classified as a haptic reading experience (not to be confused with the science of haptics nor with the notion of haptic perception often used by cognitive neurosciences¹²Cf. note 1
13
Maria Engberg uses the term “polyaesthetics”, explained on her personal webpage in the following way: “Originally, polyaesthesia was a rarely used medical term that described an abnormality of sensation in which a single stimulus is felt in several places. I use polyaesthetics to connect to that older term, in a way that retains the original Greek understanding of aiesthesis as the perception of the external world by the senses. I also want to expand the term to refer to the desire in contemporary media culture to foreground the use of multiple tools, practices, and modes of reception.” [http://polyaesthetics.net/what-is-polyaesthetics/, accessed on September 11, 2015]
14
For more on resistive and capacitive touchscreens, please visit the following webpage http://www.makeuseof.com/tag/differences-capacitive-resistive-touchscreens-si/, accessed on November 14, 2015]
15
Concerning Duchamp’s choice of glass as an alternative surface to canvas, in her biography of Duchamp, The Bachelor Stripped Bare, Alice Goldfarb Marquis tells a slightly different story from the one told by Tomkins. Establishing a connection between Duchamp and the Czech painter Frank Kupka, Marquis states that, not only was Kupka neighbor to two of his brothers, he also saw glass (particularly, stained glass) as “the best solution for what he was seeking” (2002: 39-60).
16
Manuel Portela suggests that the verbs “to invoke”, “to evoke” and “to revoke” could also be considered here, all of them, he says, possible to be argued. [M. Portela. personal communication. December 1, 2015]
17
This article is based on my presentations at the “Digital Material conference, held at NUI, Galway (May 21-22, 2015) and at the “Writing Wide, Reading Wide in the Digital Age” conference, held at Universidad Complutense, Madrid (October 22-23, 2015). The article was sponsored by the Centre for Portuguese Literature at the University of Coimbra under project PD/BD/52249/2013, funded by FCT-Foundation for Science and Technology. I would like to thank Manuel Portela for his insightful comments to my arguments, Sandra Bettencourt and Ana Marques da Silva for their comments on early drafts, Matthew Kirschenbaum for his reminding that language is a system, Vilashini Cooppan and Alexandra Saum Pascual for their questions and suggestions on temporality, Michelle Amado for her proofreading, and finally, Emmanuelle Pelard and Anaïs Guilet for their editing contribution and for organizing this exceptional cahier virtuel.

Type d'article:

Article d'une publication

Mots-clés:

Pour citer ce document