At its core, Carrie Sijia Wang’s video work Lost and Found (2022) charts an absurd breakdown in machine translation. Facing the camera in split-screen, Wang and two other performers—Stephen Kwok and Yiran Sun—each verbalize, in their native tongues, deeply personal thoughts about existing between Chinese and American cultures. Wang and Sun speak about having to change their identities, while Kwok describes finding a new community by learning a language he heard while growing up but couldn’t speak. With a slight lag, a simple speech translation interface Wang designed—a program that connects Web Speech Recognition to Google Translate—imperfectly translates what’s said by the three performers into either Mandarin or English, which are then superimposed onto their faces, like a running dialogue on screen. After a break, and in a subsequent segment of the video, the performers read back the flawed transcripts of their words–which are, in turn, translated yet again with Wang’s program. To create Lost and Found, Wang repeated this process between six and ten times; her and her collaborators’ thoughts and words degrade further with each round. The finished video, seven minutes long, compiles the final few rounds of this process into one seven minute piece (the full transcripts can be found here).

It’s not surprising that the utterances Wang’s piece depicts, when translated by software that fails to register accents, grow more garbled with each pass. Speech recognition systems are, according to the New York Times, “far less accurate when trying to identify women and people of color.” And as Lost and Found continues, phrases continually erode into what becomes a syntaxical soup of nonsense. For instance the line, “Then in this observation and experiment” morphs after eight rounds into, “Reactions from others in Tennessee.” We watch these transcripts literally pile up and accumulate over the performers’ faces, partially obscuring them in a confusion of copy. One of the piece’s implicit points, as Wang has explained, was “the rules on which these choices of words are based remain unclear, as are the databases the speech recognition system was trained on.”

Pioneering Swiss linguist Ferdinand de Saussure put forward the idea that all words derive meaning from their relationships with other words; Wang’s Lost and Found makes abundantly clear what happens when this syntaxical system—the system of language—falls apart. While the performers occasionally laugh at their translation’s absurdities, such reactions also underscore how vulnerable language is to manipulation, and the extent to which computer translation programs reduce complex identities to simple categories—especially for identities outside the mainstream. During the making of Lost and Found, Wang found herself contemplating her own relationship with language and her position within the larger technological society she occupies.“The computer program translating my speech,” she says, “starts to feel like a metaphor for social systems that fail to understand me.” Lost and Found highlights the insufficiencies of these systems, and how far we must go to address their shortcomings.

- David Everitt Howe ♦