照片的声音是什么样的?——作为联觉视听数字人文方法的数字图像可听化

What Does A Photograph Sound Like?Digital Image Sonification As Synesthetic Audio Visual Digital Humanities

计算机具有将像素、形状和其他视觉材料特征转化为声音的能力。通过视觉与听觉之间的数据关联行为,一个以视觉为来源的声波组合得以生成,这是一种新的人工制品(artifact)。它精确地与原作数据相关联,从而提供了一种新的方式感知原作的形式、内容和背景。虽然它看起来似乎是将视觉对象扭转为一个听觉对象,但矛盾的是,它使观察者能够以更精确的方式重新观察视觉证据。通过跨越影像与音频、视觉与听觉之间的典型界线,一种具生产性的、联觉(synesthetic)的批判成为可能。通过对数据进行数字转置,听取和观察视觉作品,可以更好地进行近读,实现更具说服力的解释及更深入的理解背景。基于作者早期关于图像故障、可听化处理的学术工作,文章对美国著名歌手琼·贝兹1960年代初在加州伯克利希腊露天剧场演出的一张照片进行了研究。这张照片出自作者对伯克利民谣音乐节和美国西海岸民谣音乐复兴历史的研究项目。在此,数字图像可听化处理的使用尤为有趣。尽管不能魔术般地还原出照片里的音乐,但是我们可以更切近地关注照片中的“声音幽灵”。数字图像的可听化技术虽然无法还原音乐本身,但是在观察照片中贝兹如何创作音乐时,它确实有助于放大性别、权力、具化、场面、表演、等级制度和表现等问题。通过耳朵和眼睛来对图像进行多重意义的扫描,可以产生一些意想不到的感知,从而支持更具启发性的分析。由数据、信号、图像、声音、历史和人类感知构成的“赛博格舞蹈”在数字图像可听化中出现了,这激发了对视觉材料的新审视。照此,视听数字人文的联觉模式为激活学术想象力提供了一条充满希望的新途径。

Computers have the capacity to transpose the pixels,shapes,and other features of visualmaterial into sound. This act of data correlation between the visual and the audial produces anew artifact,a sonic composition created from the visual source. The new artifact,however,correlates precisely to data in the original,thus allowing for fresh ways of perceiving its form,content,and context. Seeming to distort the visual object into an aural one paradoxically allowsan observer to observe the visual evidence anew,with more accuracy. A kind of generative,synesthetic criticism becomes possible by cutting across typical boundaries between the visualand the audio,the optic and the aural. Listening to as well as looking at visual artifacts by wayof digital transpositions of data enables better close readings,more compelling interpretations, and deeper contextual understandings. Building on my earlier scholarship into image glitching,remixing,and sonification,this essay investigates a photograph of Joan Baez performing at theGreek Amphitheater in Berkeley,California,during the early 1960s. The image comes from myproject on the Berkeley Folk Music Festival and the history of the folk music revival on the WestCoast of the United States. Here,the use of digital image sonification becomes particularlyintriguing. While we cannot magically recover the music being made in the photograph,we canmore closely attend to the ghosts of sound within the silent snapshot. Digital image sonificationdoes not recover the music itself, but it does help to amplify issues of gender, power, embodiment, spectacle, performance, hierarchy, and performance in my perceptions of Baezmaking music in the photograph. Using the ear as well as the eye to scan the image for itsmultiple levels of meaning leads to unsuspected perceptions,which then support morerevealing analysis. In digital image sonification,a cyborgian dance of data,signal,image,sound,history,and human perception emerges,activating visual materials for renewed scrutiny. In doing so,this mode of AudioVisual DH activates the scholarly imagination in promising newways.