SUNNY LIU

Grey Matter

HackathonClaude

Grey Matter pipes predicted brain activations from Meta's TRIBE V2 model through Claude Sonnet 4 and produces three-layer reports — one analysis serving performance marketers, creative directors, and content creators at once.

Year
2026
Duration
24 hours
Hackathon
NYU × Columbia · Claude
Place
2nd
Stack
TypeScript · Next.js · Claude API · TRIBE V2
Overview

Every ad platform tells you what happened — clicks, views, scroll depth. None of them tell you why. A viewer can watch a 30-second ad while their brain checked out at second 12, and watch time calls that a success.

Grey Matter closes that gap by making brain activity legible to the people who actually make ads. Upload a video, and three layers of analysis come back — one for the performance marketer, one for the creative director, one for the content creator.

The Problem

Performance marketers, creative directors, and content creators all need different things from the same ad. Existing analytics serve none of them well — they report signals that are downstream of attention (clicks, dwell time), not attention itself.

Meta's TRIBE V2 model can predict what a viewer's brain does during a video. But its output is dense activation arrays across 15+ brain regions per timestep — locked behind numerical infrastructure that nobody on a creative team can read.

Method

A Next.js streaming API route pipes TRIBE V2's predicted neural responses through the Claude Sonnet 4 API, along with a brain-region-to-function reference sheet and a structured prompt that produces three distinct output layers in a single pass.

The LLM interprets the raw neuroscience and the UI presents results in a tabbed interface, so each user type sees the layer most relevant to them without filtering through information meant for someone else.

Output

Three reports from one analysis. (1) Scorecard for performance marketers — attention score, peak moment, drop-off point, and a recommended edit. (2) Emotional Arc for creative directors — the narrative of the viewer's emotional journey through the spot. (3) Timestep Timeline for content creators — second-by-second engagement scores and feeling tags at each moment.

Challenges

TRIBE V2 is slow — a 5-second video takes roughly 20 minutes to process. Translating dense activation arrays into language that feels natural and actionable without losing neuroscientific accuracy was a constant balancing act. Designing a single prompt architecture that produces three distinct, audience-specific outputs in one streamed response was the trickiest piece.

What's Next

Real-time processing as TRIBE V2 inference speeds improve. A/B comparison mode to stack two ad variants side by side at the neural level. Benchmark libraries so teams can compare their content against category baselines.

Longer term, we want to move upstream — using brain predictions to guide creative decisions before production, not just evaluate them after.

Pics from the hackathon4 images
Grey Matter — 1Grey Matter — 2Grey Matter — 3Grey Matter — 4Grey Matter — 1Grey Matter — 2Grey Matter — 3Grey Matter — 4
OTHER PROJECTSSee all work →