test
Recorded 2026-09-25 16:39 UTC · 12m 22s. A conversation with Grok about the Scribeboard pipeline spec. Grok passes changes on to a second agent, which the transcript calls “Wipe” (later “White”).
Output format
Structured JSON output. Output is script.json.
Product name: Scribeboard
Name check
I asked: “What’s the name of our product again? Scribeboard? Is that what it is?” Grok confirmed: yes, Scribeboard.
Then I asked Grok to check whether Scribeboard already exists, because I don’t want to take somebody else’s name.
Grok’s findings:
- Scribeboard as one word doesn’t appear to be taken.
- There are some close cousins, though:
- Logitech makes a Scribe whiteboard camera for video conferencing.
- Noel has a Scribe mobile marker board.
- There’s an old Mac app called Scribeboard from 2005 that lets you draw on screen.
- Scribe Animator is a whiteboard explainer animation studio. That one’s probably the closest in spirit to what we’re building.
Domain
Scribeboard.com doesn’t resolve to anything: no active site, no parked page, nothing. It looks available, and Grok suggested grabbing it before someone else does.
Who controls the story and scenes
I generate a story or a synopsis, but the AI gets to generate the final narration. So the AI has complete control over the scene breakup and what’s in each scene.
Grok agreed: the synopsis is just the seed. The AI decides how many scenes there are, how long each one runs, what gets drawn in each, and how complex each image needs to be based on its own narration length. That gives it full creative control over pacing and density, which is what we want for a hands-off pipeline.
Pacing: scale image complexity to scene length
The flaw
Now that Grok has been explaining this, I see a flaw: we need to be able to generate the original images. Say we have six scenes, each with an original scene image. They need to be generated at the pace of the scene, and the pace of the scene is defined by the recorded voice. So part of image generation needs to be an understanding of how complex each image must be, based on that pace.
Example: six scenes, where one scene is 20 seconds and another is 40 seconds. We want to draw the image for scene one at the same speed as the image for scene two. But scene two takes much longer. I don’t want to slow down the rendering; I’d much rather make the image for scene two much more complex.
The fix
Grok agreed. Right now the spec assumes a fixed drawing speed and just stretches or compresses to fit the time. That would make a short scene feel rushed and a long one feel sluggish.
Instead:
- The image generator takes the scene’s duration as an input and scales complexity to match.
- The 40-second scene gets more elements, more strokes and more detail; the 20-second one gets fewer.
- Every scene draws at the same natural pace, and longer narration just gets a richer picture.
- A complexity budget in the script phase estimates stroke count per second and tells the image generator how dense each scene’s art should be.
I asked Grok to insert this into our spec. I don’t need the technical spec part, just the overall basic algorithm we’re working on. Grok sent the update to Wipe, which revised the outline with the complexity scaling rule built in.
The revised flow
The big change: images now get generated after the voice track, not before, because each image’s density depends on its scene’s real narration length, and you only know that once the audio exists.
The flow:
- Synopsis in.
- AI writes the script and decides scene count and lengths.
- Voice gets recorded and aligned.
- A complexity budget calculates how dense each image should be, based on actual drawing time.
- Images generate to that budget. If any come out too dense or too sparse, they get regenerated.
- Drawing speed stays constant across every scene. Longer scenes just get richer pictures.
Step 7: virtual unmasking test
I asked whether, after an image is generated, we should do a quick virtual test of unmasking it against the time code. Grok said yes, it’s worth adding:
- A fast dry run that simulates the unmasking against the real time code.
- No full render: a lightweight check that the drawing finishes on time, nothing is left undrawn, and the camera hits the right spots.
- If it fails, you catch it cheaply and regenerate before the expensive full render.
Wipe added it to the outline as the new Step 7: a quick simulated playback that reveals strokes at the standard speed against the real time code and checks that drawing finishes with the right hold. If it fails, it loops back to regenerate the image or fix the stroke order before anything expensive gets rendered.
Styles as modules
The next thing: we want to modularize the styles. The one style we talked about is the RSA Animate style. But we could also make a sketchnote style, which I think is different; if you look at them, they’re different styles. The idea is to create several styles and choose from them.
(Part of the recording here didn’t transcribe. It covered the rest of my point about resolution and the start of Grok’s reply. The transcript picks up partway through.)
From Grok’s reply: each style sets things like drawing speed and reveal moves, and you pick one at the start of a project.
Resolution
One decision, not three
As we do this project, we also need to be able to choose the final resolution. Grok agreed it has to be one decision, not three: pick your final output, say 4K, and that sets the canvas size, the video resolution and the per-scene image generation resolution all at once, so every image has enough density to land cleanly on the final canvas.
Wipe folded both styles and resolution into the app. The big additions to the outline:
- A project setup step where you pick your style and resolution up front.
- A style module library, with RSA Animate and SketchNote as the first two.
- A resolution plan that calculates canvas size, output video size and per-scene image size from that one choice.
- Everything downstream (script vocabulary, image density, segmentation rules, animation behavior) reads from whichever style module you picked.
The final image is larger than the video
I wanted to be sure: when I talk about resolution, we’re choosing the final video resolution, and the final image that’s used will be much larger than that, but calculated from the final video resolution.
Grok confirmed. You pick the final video resolution and that drives everything. The canvas is three to four times wider than the camera’s close-up view, so the source images on it are much larger than the output. But every size in the pipeline (canvas, per-scene image, output) traces back to that one number.
Can image models produce that?
I asked whether something like Claude or Codex would have a problem with this. Grok said yes, that’s the real bottleneck: current image models top out around 1500 × 1000 pixels. If the canvas is 8K and a single scene needs to cover a quarter of it, you’re asking for something like 4000 pixels wide, and no model does that natively today.
My correction: generate, upscale, stitch
No. If we create six scenes, that’s six images. To run at 4K, we’d render each of those six images at the highest resolution ChatGPT can manage. Then we’d AI-upscale them with a decent upscaler (find one that doesn’t distort too much) so each image is 4K. Then they can be stitched together.
And remember, the masking is all deterministic. The masking should be controlled by AI, but we’re not generating an image that way. The video masking should probably be done in FFmpeg.
FFmpeg is the thing that might choke on something like a 16K image. In fact, thinking about it, the destination, the final video size, should be something like 1920 × 1080. If you have six scenes, that’s still a large source image. FFmpeg can probably handle it at that resolution, but it can’t handle a 20K or 30K image.
Grok’s restatement
Grok agreed and said it’s a cleaner design:
- Each scene image gets generated at max ChatGPT resolution, upscaled cleanly to 4K, then composited onto one big working canvas. That could be 16K or whatever the layout needs.
- Masking and stroke ordering stay fully deterministic, driven by the segmentation data, not by generating new pixels.
- FFmpeg does the actual video work: sliding the viewport across the canvas, revealing strokes in order, and outputting a final 1920 × 1080 video. FFmpeg handles that easily at 1080p.
- This sidesteps the problem of rendering a massive canvas directly. The big canvas is just source material, not output.
Grok sent this correction to Wipe, which is flipping the resolution model now; the updated outline will follow shortly.
Okay, we’re good. Thanks.