Everyone was there, but everyone was filming
We all leave with the same shaky video that never gets watched again.
Interactive, but only for one
We looked at the interactive art that already exists, like photo booths that drop you into a scene, walls that react to your hand and floors that ripple when you walk. They’re fun, but almost all of them are made for one person at a time, and when you walk away, your part disappears.
How might we use AI to turn individual expression into a collective momento?”
One memory, painted by everyone
Momento is a shared canvas for a space or an event. Each person steps up, answers one question, and leaves their own brushstroke on the same painting. By the end of the night, the canvas holds everyone who was there.
Then, for everyone who comes:
Sketching the booth first
Before we wrote any code, I sketched the whole visit on paper so we agreed on what a person would actually do: start, answer the question, count down, paint with your hand, close your fist to stop, add your name, and then hand it off to the next person.
Those sketches turned into our screens, plus the buttons we knew we’d need for a live demo, like reset, simulate and finish.
Teaching AI what a feeling looks like
The hardest part was making the painting actually feel like the person. Our first version used a general AI model to pick paintings, and it sent 25 out of 40 different feelings to The Scream. Grief, excitement and calm all ended up looking the same.
So we tested before we committed. We ran three approaches side by side, including a palette method I built that mapped moods straight to color, and rated the rendered paintings blind, with the columns shuffled so nobody knew which was which.
The winner learned from people. We used a library of real paintings where people had voted on how each one made them feel, then added a second score so “first snow” would find snowy paintings, not just paintings that feel snowy.
- Grief→Cézanne’s Sorrow and Fouquet’s Pietà
- First snow→Sisley’s Road under Snow, Louveciennes
- In love, first date→Toulouse Lautrec’s The Kiss
How it works technically
Momento runs in four steps. The browser senses your words, voice and movement. A small server reads the feeling and picks three paintings. A cloud GPU paints a frame in their style while keeping the scene lined up, and then the browser reveals it only along your path, with a ragged edge like a real brush.
- MediaPipe
- Web Speech and Web Audio
- CLIP
- Gemini on Vertex AI
- SDXL Turbo
- IP Adapter and ControlNet
- Google Cloud GPU
- Python
Pitching it to the whole room
Jonathan and I ended up winning the Entertainment track and pitching in the final round.
For the demo, we didn’t just show slides. We had the judges step up and paint on the canvas live, so they became part of the momento themselves.
Several other judges talked about how you all did such a great job clearly stating a real problem, laying out the value proposition, and showing the evolution of your work.”Amanda Stockwell, one of the judges
AI that brings people together
We usually talk about AI personalizing things for one person. Momento showed me it can do the opposite. It felt collective instead of isolating because every person’s mark stayed visible inside something we all made.
I also learned to test the riskiest idea first. A blind board of rendered paintings told us more in an hour than any number did, and it’s how we knew what to build.
Next, we’d love to pilot it at a class, a museum, a concert or a game, and send everyone home with their own copy of the night.
