Tacos

At the table in Amaya's Taco, taken as the recording began

Overview

A lunch conversation with David Avalon at Amaya’s Taco. It opens with a demo of the recording device and the pipeline behind it, turns to how David uses AI image tools at IBM, and settles on the main topic: how to run an AI film pipeline from story to finished video. One speaker described getting lost in a seven-stage, many-gated command-line process. The other argued that once the story is locked, only two things matter: the prompt for the image and the prompt for the motion, both built from a “shooting script” and reusable elements. They agreed to work through a new script together after lunch.

Key points

The recording device and pipeline

  • The device runs a special Android build; it has no power-off option yet and a SIM slot for use away from Wi-Fi.
  • Recordings upload over Tailscale to a server, which transcribes them and writes three versions: a synopsis, a thorough version and an executive summary. All three prompts can be edited in settings.
  • The server now has the MD Docs workflow built in, so it produces styled HTML without MD Docs itself. The same HTML feature drives the daily briefing.
  • The other option on the table was recording on a phone: one speaker records sermons at church with Granola, started from a watch. The other doesn’t want to carry two phones or tie up the phone. The B-Wearable, which records on a tap, also came up.
  • One speaker wished the device could take photos or short video while travelling. The reply was “I can add that.”

AI images at IBM

  • One speaker built a thumbnail and graphics generator for a teammate in Dublin by combining IBM’s brand photography guidance, which he helped write, with his “filmic” prompt engine.
  • IBM work has to render in Adobe Firefly, not outside tools, so the images are legally safe. Firefly “finally caught up,” and his boss took the results for photographs.
  • The filmic guide holds real film knowledge: lens choice (40–85 mm for portraits), film stock, lighting (butterfly lighting), and camera (Arri, Fujifilm).
  • People think “all he did was press a button”; the work sits in the knowledge built into the prompts.

Where the film pipeline gets stuck

  • The pipeline runs as a deterministic command-line tool with seven stages: script from story, assets, concept art, storyboards and asset boards, first frames, then video. So many gates make it easy to lose track.
  • The MD Questions skill has Claude (“Cod”) write an MD Docs web page with checkboxes, radio buttons and fields, up to 50 questions. A Copy Answers button sends the answers back. This works as an interface without building an app.
  • The goal is to keep context through every choice, such as scene and shot context when judging a storyboard.

David’s approach: prompts and elements

  • A film has three versions: the one you write, the one you shoot and the one you edit. Only the edited one matters, and AI work follows the same pattern.
  • Lock the story, then aim for a shooting script. In the end you need only a prompt for the image and a prompt for the motion; concept art and the rest are sandbox work.
  • BoxBox takes the story’s beats from Story Sherpa (“the algebra of storytelling”), then parses the script into prompt language, “the only language the screen speaks.”
  • The script breaks into elements: characters, locations and props. A prop becomes an element when the camera must look at it (Eric’s specific guitar, the R1 on the table) or it recurs across shots. Background items like a desk mouse do not.
  • Look choices made while testing (Arri, Cooke 40 mm lens, Kodak Vista 100 stock for a 1970s look, butterfly lighting) become fixed parts of the prompt.
  • A character needs only four images: a close-up of the head, profile or quarter profile, back and front. David and Eric tested this on Max for Mission Avalon.
  • First frames are mostly out; “elements are king.” Element-based rendering knows what lies around the subject, so the camera can move freely.
  • Eric has mastered motion prompts, for example “the camera rotates around Camille” rather than “Camille moves.” He has started using the BoxBox center prompt.
  • A fully automatic “render everything and come back in three hours” run can’t replace a human directing each shot, though BoxBox tries to write some of that judgment in.

Action items

  • Pick a new script for the working session, not the one already sent (too much computer graphics); the new one already has some concept art.
  • Walk through the BoxBox shooting-script and element workflow together on that script.
  • Add a power-off option to the recording device.
  • Consider adding photo or short-video capture to the recording device.