Buddhism Interactive Installation | Commissioned for San Diego Museum of Art

This project is an interactive interpretive installation developed for the Temple Room gallery in the San Diego Museum of Art that supports public understanding of Buddhist sculpture through touch interaction and embodied gesture. Buddhist figures communicate meaning through visual attributes such as hand gestures, posture, drapery, and ornament, yet these symbolic cues often remain difficult to interpret through wall labels alone. The installation addresses this gap by transforming looking into an active, bodily experience where visitors learn through attention, touch, and movement rather than passive viewing.

The system consists of two interconnected modes.

‍Learn presents a real time rendered three dimensional model of a sculpture from the museum collection. Visitors explore the figure by touching visual hotspots placed on key iconographic regions. Each hotspot reveals zoomed details, visual highlights, and concise interpretive text that directs attention to meaningful sculptural features. This supports close looking and helps visitors recognize how symbolic meaning is embedded in form.

Play shifts learning into the body. Visitors perform Buddhist mudras in front of a camera. When a gesture is recognized, the system responds with short generative animations and interpretive captions connected to specific sculptures in the gallery. Gesture becomes a pathway to meaning, allowing visitors to experience iconography through physical enactment rather than observation alone.

Technically, the project combines museum oriented digitization, computer vision, and generative media. A mobile photogrammetry workflow was developed to digitize an immovable sculpture directly in the gallery using smartphone capture optimized for real time deployment. Gesture recognition extends MediaPipe hand tracking through custom training and a supervised two hand classifier capable of handling overlap and occlusion, implemented as real time inference within TouchDesigner. Recognized gestures trigger collection specific generative animations created through an AI and human collaborative workflow that translates symbolic meaning into dynamic visual metaphors grounded in the museum’s objects.

Conceptually, the work proposes interaction as interpretation. Rather than using technology for spectacle or navigation, the installation links attention and embodiment as complementary forms of learning. Touch supports analytical looking. Gesture supports experiential understanding. Together they make culturally encoded visual knowledge more accessible across ages and backgrounds while remaining anchored in the physical artworks in the gallery.

Credits:

This commission is made possible with special thanks to Polly Liew (金蓉蓉) for their sponsorship, and to JoJo Ni (張瓊芳) for initially proposing the project.

SDMA Project Manager & Community Engagement Manager: Lucas Perez

Installation Development Team:

Art Director/Creative Technologist: Mingyong Cheng (程銘泳)

Creative Producer: Cameron Surh

Educational Content Editor: Xuexi Dang (黨雪茜)

Photogrammetry Support: Zetao Yu (郁澤韜)

Creative Consultant: JoJo Ni (張瓊芳)

Buddhism Consultant: Venerate Zhi Ching (知晴法師)