Sep 25, 15:59

Step two: from Story Sherpa script to production

Concern: too much screen time

  • Script heavy on shots of screens
  • Software is the hero, hence the screens
  • Read instead as Jordan personifying the tension
    • Focus on his face reacting to the screen
    • Screen shown only to make a point, e.g. Maria’s email

Creating the screens in AI

  • Product itself not shown; the vibe coder’s own app is
  • Wanted: extreme close-up screens, glowing dots, maybe scan lines
  • Partly handled through the mood board

First import into BoxBox Studios

  • New project “NewsCam” from a Markdown script
  • Broke it into 30 scenes, 38 shots, very fast
  • Script not in usual screenplay format; needs checking

Running BoxBox on own infrastructure

  • Code in a GitHub repo; uses an API
  • Just a website; could become an Electron app
  • Grab from repo and build own version

Screen-time analysis and rebalance

Analysis

  • About 50% of the 60 seconds on screens
    • Roughly 33 of 60 seconds
    • End cards take the last 8–13 seconds
  • About two-thirds graphics rather than people with end cards

First rebalance suggestion

  • Put Jordan in console section; cut to his face
  • Shots showing both screen and face; more human montage
  • Result: about 40–45% screen, 40–45% human
  • Needs compositing; not the point of this test

Target: about 30% screens

  • “Fix it in script” before trouble downstream
  • Three moves proposed
    • Give the agent a voice instead of on-screen text
    • Turn the scan into a set piece: glowing map of app fills dark room
    • Revised beat sheet
  • Revised beats
    • Deploy cursor, then face in screen glow
    • Physical montage: hands, coffee piling up, day/night time lapse
    • Music dies; URL tick and Maria’s face on screen
    • Slow dolly zoom on Jordan; glance at sticky note
    • Jordan asks aloud “are we secure?”; agent’s voice answers

Feasibility for AI video

  • Question: any shots impossible for AI video?
  • Worry: split screens, translucent screens
  • Answer: all fine, no problem

Version 4 in Hollywood screenplay format

  • Ask for v4 with agent voice, in Final Draft format
  • Screenplay format is the visual language of cinematography
  • Sherpa does not mention Final Draft format explicitly
    • Built working with engineers
  • Why it matters: one step closer to BoxBox translation
  • Notes for Sherpa
    • Export scripts as Markdown
    • Understand Hollywood screenwriting format

Aside: chair mat for hardwood floors

  • Rolling chair killed engineered hardwood after a year and a half
  • Office Depot fabric floor mats, about $150
    • Hard enough to roll, not sticky plastic

Opening the script in Final Draft

  • Fountain file would not open in Final Draft (paid app)
  • Renaming to FDX failed
  • Asked for a new file with FDX ending
  • FDX is structured XML, not proprietary binary
    • Could build a reader for it
  • Fountain viewer only views, no editing

Reading the script in Final Draft

  • Easy to read
  • Add scene numbers: 11 scenes, end cards counted as scenes
  • No shot numbers; would like them
  • Scene 1: Jordan’s apartment, no dialogue
  • Scene 2 montage: agent lines appear
    • “Build a login flow” / “Done”; “Add social” / “Done”
    • Coffee cups multiply; time lapse day/night

Where the treatment comes in

  • Question: where do color, lighting, camera, casting come in?
    • Example: scruffy beard, about 35, white, baseball cap
  • Answer: in the “world” step of the process
  • Aspect ratio pulled automatically from script: 1.85:1
    • More cinematic feel

Scene-heading problems

  • Delete old project; re-import new script
  • “In montage” wrongly a scene heading; should be a transition
  • Smash cut marks scene two
  • FDX has no timing; original script did
  • “Back to scene” leaves unclear which scene
  • Tool still learning; not perfect yet

Filmmaker’s view

  • As director of photography: count locations
  • Count shot setups; build shooting schedule
  • End goal: pure prompts for Midjourney

Making the script read as one location

  • Ask Claude: whole script is one scene, one location
  • Done in the CinePrompt project; it knows CinePrompt
  • Older version preferred; Eric uses it

CinePrompt breakdown of the script

  • Asked for breakdown: characters, locations, props
  • CinePrompt is the prompting engine, part of BoxBox
  • GitHub access already given; repo updates push latest

Characters

  • Jordan: only on-screen performer
    • “Late 20s, hoodie”; no hoodie wanted, fix later
  • Agent: voice only, no visual form
    • Tool expects a reference image
  • Narrator: voice only, same problem
  • Maria Lopez: photo on a profile page only
  • NewsCam: VFX entity behaving like a character

Locations

  • One real location: Jordan’s home office
  • Three lighting states
    • Night by blue screen
    • Scan architecture overlay (unclear to both)
    • Dawn

Problems found

  • Scene headings don’t match
  • Montage has no location; extraction returns nothing
  • Inserts

BoxBox Filmmaker parser review

  • BoxBox Filmmaker: working version; new version just made
  • Older one has some better aspects
  • Ask Claude in browser to read the parser code
    • Match the script to the parser
  • Claude permission prompts; switch off manual mode
  • Eric just starting to use BoxBox

What CinePrompt does

  • Translates client-friendly script into a shooting script
    • Final Draft script lets client understand and sign off
    • Then the real work: “shooting,” with Midjourney as cinematographer
  • Not an AI at heart; deterministic rules written by AI
    • Knows Midjourney’s prompt language
  • Style and character age must come from the script or extra info
  • Parser code still unreadable; stopped there
  • Focus on first three scenes; goal is workflow, not a perfect ad

Researching a style

ShotDeck

  • Search “computer” for screen shots
  • You’ve Got Mail: not right
  • Moneyball; The Matrix feels good, cinematic

Existing concept images

  • Drive folder: Anthony > concepts > screen plates v1
  • Have mood but not a cinematic feel
  • Made in Midjourney; prompts available

Midjourney filmic prompt generator

  • ChatGPT project: filmic cinematic prompt generator
  • Rewrite existing prompt for cinematic look
    • Stopped it from generating an image instead
    • Removed unasked “AI Story Sherpa” text
  • Settings
    • Aspect 2:1
    • Weirdness 0; variety about 10; stylization 10
    • Version 8.2
    • Default profile tuned on cinematic images; no mood board

Why the original was not cinematic

  • Burnt lighting; looks drawn, like an illustration
  • Not photographic despite prompt

Revised prompt

  • Casting: mid-20s to early 30s, Caucasian male
    • Baseball cap, scruffy beard, no hoodie
  • Asked for optimal prompt length for Midjourney 8.2
  • Result read aloud
    • Engineer at open laptop, 2:11 a.m., age 28–32
    • Worn cap, charcoal Henley, minimalist wooden desk
    • Laptop screen only light; cold blue-white
    • Eye-level three-quarter, asymmetrical, medium shot
    • Low-key, David Fincher, cool palette
  • One image with hat liked most: good style, good face
  • Link to generator shared by text

Goal: fast shooting script and prompts

  • Story narrowed; next narrow the shooting script
  • Not perfected yet: shooting script and Midjourney/Kling prompts
  • Automated Midjourney generation in new BoxBox Filmmaker
    • No API; browser automation logs in with subscription
  • Critique of results
    • Too centered; asymmetry matters
    • Looks sad; too muscly, developers skinnier

Earlier commercial shown

  • Prostate cancer PSA blood-test ad
    • Fighter pilot testimonial; biopsy statistics
  • Real men’s faces used
  • Midjourney for concepts; Kling for finals
    • One or two Midjourney videos
  • A huge hit

Optimizing the script for the parser

  • Ask Claude to adjust Final Draft script for BoxBox/CinePrompt
    • Parser must read scenes, characters, locations, props
  • Further tweaks possible inside BoxBox
  • Write, shoot, edit: the edit is the real film
    • Reach the prompting phase as soon as possible
  • CinePrompt converts script into video prompts

Style and assets first

  • Lock look and feel
  • Consistent characters: build character LoRAs/elements

Capturing a real person as a character

  • Also needed for the podcast
  • Shots taken: front, semi-profile, back, facing

Parser results: script v5

  • Every scene has the same heading
  • Last scene its own dawn scene
  • Scenes numbered 1–6, no gaps
  • Montage has a real location heading
  • Three insert-laptop lines and cards now shot lines
  • “Back to scene” means back to home office, night
  • Jordan matches casting: scruffy beard, worn cap, no hoodie
  • Imported into CinePrompt: 16 shots, much better
    • Features from previous version; why Eric likes it

Shot Builder, elements and film style

  • Prompts serve both images and video
  • Elements: character group
  • Film style panel sets the feel

Mood board

  • Reference image risks copying its look
  • Find images with right color instead
  • ShotDeck searches
    • Hacker screen, data screen, CRT screen
    • Digital display, sci-fi screen
  • Mood board influences every prompt, including Kling

Purpose of film style

  • Question: how does film style affect output?
    • No storyboards, no establishing shots
  • Answer: applies when Shot Builder writes prompts

Multi-shot prompts

  • Scene 1 split into shots 1.1, 1.2
  • Generate multi-shot for Kling 3.0
  • Format mimics what Higgsfield wants
  • Three multi-shots

Creating the Jordan character

  • Add character element “Jordan”; upload reference image
  • Description: age 22, Caucasian, baseball cap, t-shirt, vibe programmer
  • Eric removed the character builder sheet
    • Restore the character image builder prompt
  • Claude can’t generate images
  • ChatGPT with prompt generator, character builder, script 4.5
    • Builds Jordan’s sheet without a reference image
    • Skips choosing among candidate characters

Comparing workflows

  • Both spend time in the same areas
  • CinePrompt runs on own LLM
  • LTX tests done; video part not coded
  • Plan to compare both approaches

BoxBox versions

  • BoxBox Studios: 3.0, newest
  • CinePrompt: 2.0; also 1.0, simple but still works
  • Other tools: script manager, poster maker
  • Commercial built in 3.0
  • Mood board with generated hacker characters
  • Jordan character sheet; Maria sheet, face only

Electron and MCP setup

  • HTML apps keep ending up as Electron apps
  • Electron with MCP interface to Claude or OMP
    • Render videos, infographics
    • About 10 models: ChatGPT, GLM, Grok
  • Proposal: push BoxBox Studios into Electron; fork it

Generating locations

  • Use filmic Midjourney generator for script locations
    • Output is a ChatGPT image, not Midjourney
  • Night, scan and dawn versions
  • Name: “Jordan’s apartment night”

Revising the night location

  • Modern high-rise apartment, big city, blinds, clean lines
  • Keep teal and orange cinematic grade
  • Darker, foreboding, tension
    • City light through window
    • One desk light; bright monitor

Time check

  • Session may split in two
  • Doing two things: the work and capturing the workflow
  • Aim: one scene with Jordan at the computer

Aside: 8mm film and summer camps

  • Movie Maker Pro digitizes 8mm reels
  • First film made at 11 in a gifted-and-talented summer at UT
    • Friend’s mom saved the film
  • Fifth-grade art camp: charcoal, drawing boats in Annapolis

Dawn location and reference images

  • Dawn version: bright, happy
  • Still not quite cinematic; fix later
  • Sticky note reads “ship it”
  • Ran image prompt through filmic generator
    • Result worse
  • Images serve as omni references for video
  • “Shot on re flex” wording may be part of the issue; try ChatGPT

Character sheet cutouts for Kling

  • ChatGPT cuts sheet into seven separate images
  • Needed to build the element in Kling
  • Grok and Seedance can use a full character sheet
  • New Kling version expected soon
  • Same done for Maria

Placing Jordan with Nano Banana Pro

  • Full character sheet plus night location
  • Jordan sitting at desk facing laptop
  • Omni reference, not first frame
  • Kling 3.0 elements: upload face and body images
    • Kling one of the cheaper models
    • Upload limit and bad aspect ratio
  • Voices added with Grok
  • “Turn to video” would use a first frame

CinePrompt 3.0 run on the script

  • Re-import: 16 shots, three characters
  • Shot list with camera moves; Dutch angle
  • Question: does Kling want the metadata at top?
  • Elements referenced by @name
    • Jordan; Jordan apartment night
    • Misspelled references red, fixed to green
  • Multi-shot splits a shot into several
  • Chose one single shot, about 8–10 seconds, 720p
  • Next shot: zoom to laptop, Maria Lopez profile with home address
    • Needs a laptop element to control it

How CinePrompt chose shots

  • Multi-shot prompts follow the shot list
  • Uses the shared camera reference
  • Style not set up yet; film style keywords and ratio
  • Render a shot at a time; assemble in Premiere

LTX test through Claude

  • LTX not yet plugged into BoxBox Filmmaker
  • Paste prompt and elements into Claude terminal
  • LTX model render; generates a first frame

Takeaways

  • Story Sherpa work very valuable
  • Go shot by shot, not all at once
  • Own process: storyboard per shot → initial frame → video start
  • Missing piece: CinePrompt

First frame vs omni references

  • Own tools mostly need a first frame
  • First frame blocks big cinematic moves
    • Example: around character, through car, into city
    • Causes location drift
  • Omni reference with location sheets of all angles
  • Multi-view sheet like a LoRA, but not a trained LoRA
  • Use only needed omni references
    • Foyer walked through once: no reference
    • A pen he drops: needs one

Rendered video

  • Cinematic move; good image

Final color grade

  • Final layer in Final Cut
  • Own grading tool broke; needs update
  • Always 24 fps
  • Apply LUT/CLUT; film noir option
  • Typical grade
    • Film grain; more contrast and color intensity
    • Lens glare, not flare
    • Vignetting
  • Blender compositor also strong