Sep 25, 15:59
Overview
This is a long working session between two filmmakers. It covers step two of an AI film workflow: turning a 60-second Story Sherpa script about Jordan, a “vibe coder” whose NewScan app exposes Maria Lopez’s home address, into shots in BoxBox. They cut the time spent on screens, gave the coding agent a voice, and put the script into Hollywood screenwriting format (Final Draft). They then tuned the script until the CinePrompt parser in BoxBox read it cleanly: 6 scenes and 16 shots in place of the first 30 scenes and 38 shots. Next they tested cinematic Midjourney prompts, built character sheets for Jordan and Maria and night and dawn plates of Jordan’s apartment, and rendered a first test shot through Kling 3.0 and a local LTX model. The visitor’s main lessons: fix problems in the script first, work shot by shot rather than all at once, and use a prompt engine like CinePrompt to turn a script into shot prompts.
Key points
Reworking the script
- Story Sherpa’s first draft spent about two thirds of its 60 seconds on screens and end cards, so they asked for about 30% screen time.
- Version four added three changes: the agent speaks rather than replying in text, the NewScan run becomes a set piece (a glowing map of the app fills the dark room), and a more human montage (hands, coffee cups piling up, day-night time-lapse, a dolly zoom on Jordan).
- The AI said none of the new shots would be hard for AI video, including see-through screen overlays.
- “Fix it in script”: sorting problems out at script stage saves work later.
Final Draft format and the parser
- Hollywood screenwriting format is the “visual language of cinematography” and brings the script one step closer to what BoxBox needs.
- The Fountain file would not open in Final Draft; an FDX file (structured XML) did.
- Final Draft counts scenes (11 at first) but not shots, and drops the timing the first script had.
- Scene headings confused the parser: “IN MONTAGE” and “INSERT LAPTOP” read as new locations, and “BACK TO SCENE” was unclear.
- After fixes, every scene uses the same heading, dawn is its own scene, scenes run 1–6 with no gaps, and inserts and cards are shot lines. BoxBox then found 16 shots and 3 characters.
- The breakdown found one real character (Jordan) and two voice-only ones (the agent and the narrator). Maria Lopez appears only as a profile photo. NewScan acts as a VFX character. There is one location (Jordan’s home office) in three lighting states.
Tools and versions
- BoxBox Studios is version 3.0; BoxBox CinePrompt was 2.0. Eric uses the older one, cut out parts (including the character builder) and has started using the new one.
- CinePrompt is a prompt engine with fixed rules that an AI wrote, not an AI itself. It speaks Midjourney’s language, but it needs film style and character details from the user.
- BoxBox is a website in a GitHub repo that the visitor already has access to. The visitor hopes to move BoxBox Studios to Electron with an MCP link to Claude or omp.
- The LTX video step in BoxBox Filmmaker is tested but not yet coded in.
Look and cinematic style
- For style research, use ShotDeck (You’ve Got Mail, Moneyball, The Matrix) and add picks to the BoxBox mood board. The mood board shapes every prompt it makes.
- Earlier Midjourney concept images looked burnt, drawn and too centered, “not cinematic”. Cinematic means photographic light, asymmetric framing and motivated screen light.
- The ChatGPT “filmic cinematic prompt generator” rewrote the prompt. Midjourney 8.2 settings: 2:1 aspect ratio, weirdness 0, variety 10, stylization 10, default profile.
- Midjourney prompts need their best length; the first rewrite was too long. The generator also wrongly added “AI Story Sherpa” text to the prompt.
- Jordan’s casting: late 20s to early 30s, Caucasian, short scruffy beard, worn baseball cap, dark Henley, no hoodie, skinny rather than muscly.
- Jordan’s apartment was redone as a modern high-rise in a big city, with blinds and clean lines, a teal-and-orange grade and one desk light at night. A bright dawn version was made too.
Rendering the test shot
- ChatGPT made character sheets for Jordan and Maria (face only) and cut Jordan’s sheet into seven images for Kling elements.
- Nano Banana Pro placed Jordan in the chair at the desk for an Omni reference.
- In Kling 3.0, the @ names in the prompt must match the uploaded element names (Jordan, Jordan apartment night). Kling 3.0 multi-shot splits one prompt into several cuts. The first test ran as one 720p shot.
- The same prompt ran through the local LTX model, which built a first frame on its own.
- First frame versus Omni reference: first frames can drift in location and block big camera moves. Location sheets with several room angles help. Use Omni references only when a shot needs them.
- The final polish happens in Premiere or Final Cut: a LUT, film grain, contrast, vignette and lens glare, at 24 fps. The usual grading tool failed after an Apple update. Blender’s compositor also came up as a good option.
Action items
- Build a way to go from a Story Sherpa script to a Final Draft script that BoxBox can read directly.
- Consider teaching Story Sherpa about Final Draft / Hollywood screenwriting format.
- Put the character builder back in BoxBox (Eric removed it).
- Set up the film style and mood board in BoxBox before generating shots.
- Add an element for the laptop screen so shots of it stay under control.
- Finish coding the LTX video step in BoxBox Filmmaker, and make a skill for the LTX model.
- Consider moving BoxBox Studios to Electron so the visitor can fork it.
- Update the color-grading tool that broke after the Apple update.
- Finish the rest of the scenes in a second session.