Buddhism Interactive Installation | Commissioned for San Diego Museum of Art

This project is an interactive interpretive installation developed for the Temple Room gallery in the San Diego Museum of Art that supports public understanding of Buddhist sculpture through touch interaction and embodied gesture. Buddhist figures communicate meaning through visual attributes such as hand gestures, posture, drapery, and ornament, yet these symbolic cues often remain difficult to interpret through wall labels alone. The installation addresses this gap by transforming looking into an active, bodily experience where visitors learn through attention, touch, and movement rather than passive viewing.

The system consists of two interconnected modes.

‍Learn presents a real time rendered three dimensional model of a sculpture from the museum collection. Visitors explore the figure by touching visual hotspots placed on key iconographic regions. Each hotspot reveals zoomed details, visual highlights, and concise interpretive text that directs attention to meaningful sculptural features. This supports close looking and helps visitors recognize how symbolic meaning is embedded in form.

Play shifts learning into the body. Visitors perform Buddhist mudras in front of a camera. When a gesture is recognized, the system responds with short generative animations and interpretive captions connected to specific sculptures in the gallery. Gesture becomes a pathway to meaning, allowing visitors to experience iconography through physical enactment rather than observation alone.

Technically, the project combines museum oriented digitization, computer vision, and generative media. A mobile photogrammetry workflow was developed to digitize an immovable sculpture directly in the gallery using smartphone capture optimized for real time deployment. Gesture recognition extends MediaPipe hand tracking through custom training and a supervised two hand classifier capable of handling overlap and occlusion, implemented as real time inference within TouchDesigner. Recognized gestures trigger collection specific generative animations created through an AI and human collaborative workflow that translates symbolic meaning into dynamic visual metaphors grounded in the museum’s objects.

Conceptually, the work proposes interaction as interpretation. Rather than using technology for spectacle or navigation, the installation links attention and embodiment as complementary forms of learning. Touch supports analytical looking. Gesture supports experiential understanding. Together they make culturally encoded visual knowledge more accessible across ages and backgrounds while remaining anchored in the physical artworks in the gallery.

Full Experience:

Installation Documentation:

Installation view: the station embedded in the Temple Room's viewing sequence, between the gallery's collection case, the gold-lacquer painting, and the seventeenth-century Japanese Amida Buddha sculpture in the foreground. — Installation view: the station embedded in the Temple Room's viewing sequence, between the gallery's collection case, the gold-lacquer painting, and the seventeenth-century Japanese Amida Buddha sculpture.

Wide view of the Temple Room: the station's Learn mode in the foreground, the collection of Buddhist sculptures it interprets across the gallery.

Learn mode: a visitor explores the Buddha's robe attribute — guided looking as a form of close reading.

Stills from Learn Mode: touch hotspots on the digitized Amida Buddha, and detail views of key iconographic attributes — the usnisa, lotus pedestal, and svastika.

Play mode: a visitor holds a mudra before the integrated camera, and the system responds with a collection-specific generative animation — embodied gesture translated into symbolic meaning, with the gallery's sculptures sharing the frame

Stills from Play Mode: the guided mudra sequence — hand tracking, gesture matching, recognition, and the collection-specific generative animations it triggers.

Design & Technical Process:

Process: the mobile photogrammetry workflow for digitizing an immovable collection sculpture under spatial constraints — on-site smartphone capture and reconstruction (top), and the processed three-dimensional model views optimized for real-time rendering (bottom). — Mobile photogrammetry workflow for digitizing an immovable collection sculpture under spatial constraints — on-site smartphone capture and reconstruction (top), and the processed three-dimensional model views optimized for real-time rendering (bottom).

‍

Credits:

This commission is made possible with special thanks to Polly Liew (金蓉蓉) for their sponsorship, and to JoJo Ni (張瓊芳) for initially proposing the project.

SDMA Project Manager & Community Engagement Manager: Lucas Perez

Installation Development Team:

Art Director/Creative Technologist: Mingyong Cheng (程銘泳)

Creative Producer: Cameron Surh

Educational Content Editor: Xuexi Dang (黨雪茜)

Photogrammetry Support: Zetao Yu (郁澤韜)

Creative Consultant: JoJo Ni (張瓊芳)

Buddhism Consultant: Venerate Zhi Ching (知晴法師)