test

2026-09-25 11:39 · 12:23

Structured JSON output. Output is script.json.

Hey, Grock. Now that you’ve been explaining this, I see that we have a bit of a flaw in this.

And that is that we need to be able to generate the original images, right?

So let’s say we have six, we’re going to call them scenes, original scene images.

They need to be generated at the pace of the scene. And the pace of the scene is defined by the recorded voice.

And so part of the image generation needs to be an understanding of the complexity required for that image based on the pace.

So let’s say we have six scenes and one scene is 20 seconds and one scene is 40 seconds.

And so in that case, the image generated for scene one needs, we want to draw it at the same speed as the image generated at scene two.

But scene two takes a lot longer. So I don’t want to slow down the rendering.

I’d much rather make the rendering for scene two much more complex. Does that make sense?

Yeah, that makes total sense.

Right now, the spec assumes a fixed drawing speed and just stretches or compresses to fit the time,

which would make a short scene feel rushed and a long one feel sluggish.

Instead, the image generator should take the scene’s duration as an input and scale complexity accordingly.

More elements, more strokes, more detail for the 40 second scene, fewer for the 20 second one.

That way, every scene draws at the same natural pace and the longer narration just gets a richer picture to match.

We’d add a complexity budget to the script phase that estimates stroke count per second and tells the image generator how dense each scene’s art should be.

Yes, that’s a good idea. And don’t forget that I generate a story or a synopsis, you know, but the AI gets to generate the final narration.

So the AI has complete control over the scene breakup and what’s in the scene. Does that make sense?

Absolutely. So your synopsis is just the seed. The AI decides how many scenes, how long each one runs, what gets drawn in each, and how complex each image needs to be based on its own narration length.

That gives it full creative control over pacing and density, which is exactly what you want for a hands-off pipeline.

Right. So go ahead and insert that into our spec. And I don’t need the technical spec part. I just want the overall, you know, the basic algorithm part that we’re working on.

Sending the update to wipe now. Done. Wipe’s revising the outline now with the complexity scaling rule baked in.

And what’s the name of our product again? Scribeboard? Is that what it is?

Yes. Scribeboard. And wipe just sent…

Anyway, the revised outline is back. The big change is that images now get generated after the voice track, not before.

Because each image’s density depends on its scene’s real narration length, which you only know once the audio exists.

So the flow is, synopsis in, AI writes the script and decides scene count and links, voice gets recorded and aligned,

then a complexity budget calculates how dense each image should be based on actual drawing time.

Images generate to that budget, and if any come out too dense or too sparse, they get regenerated.

Drawing speed stays constant across every scene. Longer scenes just get richer pictures.

Do we want to test the… After an image is generated, do we want to do a quick virtual test of unmasking it based on the time code?

Yes. That’s worth adding.

A fast dry run that simulates the unmasking against the real time code.

No full render, just a lightweight check that the drawing finishes on time, nothing’s left undrawn, and the camera hits the right spots.

If it fails, you catch it cheap and regenerate before the expensive full render.

Wipes adding that to the outline.

Wipes updated the outline.

The new Step 7 is a virtual unmasking test.

A quick simulated playback that reveals strokes at the standard speed against the real time code.

Checking that drawing finishes with the right hold.

If it fails, it loops back to regenerate the image or fix the stroke order before anything expensive gets rendered.

Great. So…

Now, do me a favor and check and see if Scribeboard is something that already exists.

I don’t want to try and take somebody’s name that already exists.

Let me check that for you.

Good news. Scribeboard as one word doesn’t appear to be taken.

There are some close cousins, though.

Logitech makes a Scribe whiteboard camera for video conferencing.

Noel has a Scribe mobile marker board.

There’s an old Mac app called Scribeboard from 2005 that lets you draw on screen.

And Scribe Animator is a whiteboard explainer animation studio.

That one’s probably the closest in spirit to what you’re building.

What about Scribeboard.com?

Scribeboard.com doesn’t resolve to anything.

No active site, no parked page, nothing.

Looks like it’s available.

You should grab it before someone else does.

Good. Okay, that’s wonderful.

So…

So, uh…

Now, the next thing is we’re going to want to be able to modularize the styles, right?

So the one style that we talked about that we can create is the RSA Animate style.

But we can also create more of a…

Like a…

A sketch note style, if we want.

Um…

Uh…

Which is, I think, different.

If you look at them, they’re different styles.

So the idea is that we might be able to also, you know, create some different styles and be able to choose from them.

Uh…

As we, uh…

As we, uh…

As we do this project, we also need to be able to choose the final resolution of the image.

And that final resolution will be, uh…

Uh…

Uh…

Uh…

Uh…

Uh…

Uh…

Uh…

Uh…

Uh…

Uh…

Uh…

Uh…

Uh…

Uh…

Uh…

Uh…

Uh…

Uh…

Uh…

Uh…

Uh…

Uh…

Uh…

Uh…

Uh…

drawing speed and reveal moves, and you pick one at the start of a project. On resolution, you’re

right that it has to be one decision, not three. Pick your final output, say 4k, and that sets the

canvas size, the video resolution, and the per scene image generation resolution all at once. So

every image has enough density to land cleanly on the final canvas. Wipes folding both into the

app. Wipes back with the updated outline. The big additions are a project setup step where you pick

your style and resolution up front, a style module library with RSA Animate and SketchNote as the first

two, and a resolution plan that calculates canvas size, output video size, and per scene image size

all from that one choice. Everything downstream, script vocabulary, image density, segmentation rules,

animation behavior reads from whichever style module you picked. Okay, so I just want to be sure that when

I’m talking about resolution, we’re choosing the final video resolution, and the final image that is used

is actually going to be much larger than that, right? So, but it is calculated based on the final video

resolution. Does that make sense? Yes, exactly. You pick the final video resolution, say 4k, and that drives

everything. The canvas is three to four times wider than your camera’s close-up view, so the source images

sitting on it are much larger than 4k. But every size in the pipeline, canvas, per scene image, output,

traces back to that one number you chose.

Okay, so then, if that’s the case, uh,

do you think

as something like Claude or Codex

are going to have a problem?

Honestly, yes, that’s the real bottleneck. Current image models top out around 1500 by 1000 pixels. And if your canvas is 8k, and a single scene needs to cover a quarter of it, you’re asking for something like 4000 pixels wide. No model does that natively today.

No, no, no, no, no, no, no, no, no, no, no, no, no, no. So remember, remember, so. Go ahead. If we create, you know, six scenes, that will be six images. And to, and if we want them to run at 4k, then we would basically render those six images at each at the highest resolution that Canon chat GPT.

Then we would upscale, do a, you know, an AI upscaling of the images, you know, a decent one, you find one that doesn’t distort it too much. And then, so that each image is 4k, and then those can be stitched together. And don’t forget, remember, when we’re masking, that’s all deterministic, right? The masking should be controlled by AI, but it’s not a, we’re not generating an image this way. When we, when we do the video masking, that should be done probably in

And FFMPEG. And so FFMPEG would be the thing that might choke on a, you know, a 16k or, you know, in fact, I probably should say that the, the, the, the, the destination should be something like 1920 by 1080. That should be the destination, the final video size, when I think about it.

If you have more than, if you have more than, if you have six, let’s just say you have six, six, six scenes, that’s still going to be a significantly large image.

So, uh, an FFMPEG probably can handle it at that resolution, but it can’t handle a, a 20k or 30k image. Does that make sense?

That makes total sense. And it’s a cleaner design. Each scene image gets generated at max chat GPT resolution, upscaled cleanly to 4k, then composited onto one big working canvas. Could be 16k or whatever the layout needs.

The masking and stroke ordering stay fully deterministic driven by the segmentation data, not by generating new pixels. Then FFMPEG does the actual video work, sliding the viewport across that canvas, revealing strokes in order and outputting a final one night 20 by 1080 video.

FFMPEG handles that easily at 1080p. And you sidestep the whole problem of trying to render a massive canvas directly. The big canvas is just your source material, not your output. Want me to send this correction to White?

Yes.

Done. White’s flipping the resolution model now. You’ll get the updated outline shortly.

Okay, we’re good. Thanks.

*sad music*

*sad music*