Sep 24, 17:03

2026-09-24 16:03 · 38:41

Hey!

Okay, so we’re going to have a discussion with Grok, and we’re going to talk about

a couple different things.

Specifically, one is going to be Grok’s

or the ability for us to build a server inside of Record.

And the second is going to be

the

some research on

how we can do

kind of a chalkboard style

rapid viz, you know, kind of an animated drawing with a hand and the

pencils and stuff like that.

So,

so let’s start off with the first one.

Okay, Grok.

I want to talk about

my Record server.

I have a server app that fits on a desktop

that communicates with my rabbit that I custom programmed

to be able to send files to my desktop.

and I do it through CloudFlare and a tunnel approach and it’s done from the internet.

So, tell me you understand what I’m talking about.

Yeah, I follow you.

You’ve got a rabbit R1 you’ve custom programmed to push files out over the internet.

And instead of exposing your desktop directly, you’re routing it through a CloudFlare tunnel.

So the connection comes in securely without opening ports on your home network.

The desktop runs a server app that receives those files on the other end.

That’s a solid setup.

CloudFlare tunnel handles the ingress, keeps your machine off the public internet,

and the rabbit just needs outbound access to reach the tunnel endpoint.

What do you want to dig into?

The rabbit side?

The tunnel config?

The server app?

Or something else?

So I want to talk about the server app.

And I want to think, I’d like to make this so that I could,

so the server would be secure enough to not require CloudFlare.

Because asking a user to configure CloudFlare to create a tunnel for them

is kind of ridiculous in terms of user-friendliness.

And so I’d like, ideally, to be able to just run the server as a service as it is now

and somehow lock it down enough that people can, that a user,

that the user of the rabbit can get to it.

Typically, this is a one-to-one deal.

Tunnels are powerful, but they’re a pain to set up for a regular person.

For a one-to-one setup like yours,

something like tail scale or zero tier is probably the sweet spot.

You install a tiny client on the rabbit, run a node on the desktop,

and you just see…

No, no, I don’t want to do that either.

So it’s just, it’s got to be,

you got to basically have the executable on the PC

that you can install and run.

And the rabbit is just an APK, right?

It’s just an APK.

So it’s not, I don’t want to go through a complete configuration mode if I don’t have to.

Now, let’s back up the situation.

Is it possible to open a port on my desktop without having to go through my router,

my Wi-Fi router, or whatever?

Is it possible to open up a port on my desktop?

Is there a program like, I don’t know?

Short answer, no.

Not from inside the desktop alone.

Your desktop sits behind the router,

and the router is doing that.

It translates your private IP to the public one the internet sees.

So any incoming connection has to hit the router first,

and the router has to know to forward it to your desktop.

There’s no way for the desktop to punch through that on its own.

The closest thing to just run it without manual router config is UPnP.

The app asks the router to open the port automatically.

Most home routers support it,

but a lot of people have it disabled or don’t know it’s on.

It’s not guaranteed, and some ISPs block it entirely.

What about, what if you truly…

So tell me what Cloudflare does then,

because I don’t mess with the router that I’m aware of.

Cloudflare just bypasses the router somehow.

How does that work?

Cloudflare doesn’t bypass your router at all.

Your router still has to allow the outbound connection.

What Cloudflare does is flip the direction.

Instead of the internet reaching into your network,

your desktop reaches out to Cloudflare’s edge servers first.

An outbound connection.

Routers almost always allow outbound traffic,

so nothing on the router needs to change.

Once that connection is up,

Cloudflare’s edge holds it open,

and any incoming request that hits your Cloudflare hostname

gets routed back down

through that same outbound pipe.

So the tunnel is really just a long-lived outbound connection

that Cloudflare multiplexes return traffic over.

That’s why you never touch the router.

You’re not opening anything inbound.

You’re just making a normal outbound call,

and Cloudflare does the rest on their side.

So how does a program like Dropbox,

or, you know, how does it handle it?

Because, you know, I’m basically accessing files on the Dropbox server

without having to set up a Cloudflare or any other tunneling software.

Dropbox flips the model completely.

Your desktop never receives an inbound connection.

Instead, the Dropbox client on your PC makes an outbound connection to Dropbox’s servers

and just sits there waiting.

When you want a file,

your phone or another device talks to Dropbox’s cloud,

and Dropbox pushes the file down to your desktop

through that already open outbound pipe.

The key difference is where the files live.

They’re on Dropbox’s servers, not on your machine.

So there’s no server running on your desktop

that the outside world needs to reach.

Your desktop is just a client pulling and pushing data

to a central server that everyone else also talks to.

That’s the fundamental trade-off.

With Cloudflare Tunnel,

your desktop is the server,

and Cloudflare is the middleman.

With Dropbox,

Dropbox is the server,

and your desktop is just another client.

If you want the files to stay on your own hardware,

but still be reachable without touching the router,

you’d need something in between,

a relay that both sides connect outbound to,

kind of like a mini Dropbox,

but pointed at your machine instead of a cloud store.

So, what if I were to configure…

Go ahead.

Can I configure any…

Just for the sake of discussion,

is there a way to configure Google Drive

to handle any of this?

Like, can I use, like,

ACK files on Google Drive,

or…

or…

even ACK files on Dropbox

to do

any of this?

Yeah, absolutely.

And it’s actually a really clean fit for what you’re doing.

You could treat a shared Drive or Dropbox folder as your relay.

The Rabbit uploads files to a designated folder.

Your desktop app watches that folder,

either through the Drive or Dropbox API,

or even just by syncing the folder locally

and picking up new files as they land.

No server,

no tunnel,

no router config.

Both sides just talk outbound to Google or Dropbox,

and the folder is the meeting point.

The trade-offs are latency and API limits.

You’re not getting a real-time push.

You’re pulling or running on syncing.

I don’t need the real-time push.

I don’t need that.

That’s not something that I’m worried about.

But,

here’s what I am worried about.

It’s privacy, right?

So, the whole point of this product

is that you can have discussions

with

this converted rabbit.

It’s an audio…

It’s a real-time audio recorder, right?

So, that’s what it does.

It’s R1 Cord,

is what the name of the product is.

You can look it up.

But,

the point of it is that

you just don’t want

your conversations

in the cloud

on somebody else’s server.

So,

I understand.

I asked that question

about

Dropbox

and Google Drive

because I just wanted to understand

a little bit about how,

you know,

what kind of architectures

may work

or may not work.

So,

what about

if you had your own,

let’s see,

what if you had your,

your own PHP server

like on a cPanel

or something like that,

you could obviously do it there,

but most people don’t have that.

I’m trying to think of

what most people have

that they can,

that they can use.

I don’t really have a…

Let’s back up.

Let’s go and talk about

Tailscale now.

Tailscale is worth a serious look

for exactly your privacy concern.

It’s a mesh VPN built on WireGuard,

and the privacy story

is actually pretty good

for your use case.

Both the Rabbit

and your desktop

install a tiny Tailscale client.

On Android,

that’s straightforward,

and on the desktop,

it’s one installer.

They both authenticate

to Tailscale’s coordination servers

just long enough

to exchange keys

and find each other.

After that,

the actual audio data

flows directly

between the two devices,

encrypted end-to-end.

Tailscale servers

never see your conversations.

They only know

that two nodes are connected.

If a direct peer-to-peer path

isn’t possible,

it falls back

to encrypted relay servers.

But even then,

the data is encrypted

and Tailscale can’t read it.

The big win for you

is that your desktop server app

just listens

on the Tailscale IP address,

and the Rabbit

connects to that same address.

No router config,

no port forwarding,

no Cloudflare,

no third-party file storage

holding your audio.

The desktop is the server,

the Rabbit is the client,

and Tailscale is just

the private network

pipe between them.

And the free tier

covers up to 100 devices

and three users,

which is more than enough

for a one-to-one product

like R1 Cord.

The one thing to think through

is the initial setup experience.

The user has to install Tailscale

on both ends

and log in once.

That’s a couple of extra steps

compared to just installing an APK.

But it’s a one-time thing,

and it’s about as painless

as mesh networking gets.

Want to dig into

how the Rabbit side

would actually connect,

or the desktop server config?

No, I want to talk

a little bit more

about what the difference

between Tailscale

and Cloudflare is.

The core difference

is architecture

and who you’re trusting.

With Cloudflare Tunnel,

your desktop connects

outbound to Cloudflare’s edge,

and every request

from the Rabbit

has to pass through

Cloudflare servers

on the way in

and back out.

Cloudflare is sitting

in the middle

of every single packet.

Even though the connection

to Cloudflare is encrypted,

Cloudflare is the one

terminating that connection

at their edge.

They’re the one deciding

what gets forwarded

to your machine.

So you’re trusting Cloudflare

not just with routing,

but with being the gatekeeper

for your service.

Tailscale works differently.

It’s peer-to-peer

by default.

The Rabbit

and your desktop

find each other,

usually directly

over the internet,

sometimes through a relay

if a direct path

is impossible.

But the actual data

is encrypted end-to-end

between the two devices.

Tailscale’s coordination

servers only handle

the initial handshake,

exchanging keys,

figuring out

how the two nodes

can reach each other.

After that,

Tailscale is out

of the data path entirely.

They can’t read your audio

even if they wanted to

because they never

have the keys.

So the practical difference

for your product,

with Cloudflare,

you’re building on top

of someone else’s

public-facing infrastructure,

and your service

is reachable

from the entire internet

through that

Cloudflare host name.

With Tailscale,

you’re building

a private network

that only your two devices

are part of.

Nothing is exposed

to the public internet

at all.

The Rabbit

connects to a Tailscale IP,

not a public domain.

There’s no URL.

So tell me,

one,

which is more secure,

and number two,

which is easier

to set up

for the user?

For security,

Tailscale wins,

and it’s not close

for your use case.

With Tailscale,

the data path

is end-to-end encrypted

between the Rabbit

and your desktop.

Tailscale servers

never touch your audio.

With Cloudflare,

Cloudflare’s edge

is actively proxying

every request.

So even though

the tunnel itself

is encrypted,

Cloudflare is a trusted

intermediary in the middle.

For a product

whose whole selling point

is that your conversations

stay private,

that distinction

matters a lot.

For ease of setup,

Cloudflare is simpler

on paper.

You install one thing

on the desktop,

get a tunnel token,

and you’re done.

The Rabbit just hits

a URL.

But Tailscale

has gotten really close.

The desktop install

is one click,

the Rabbit just needs

the Tailscale app

from the Play Store

and a login,

and after that,

it’s automatic.

The difference is maybe

two extra taps

on the Rabbit side.

Honestly,

for a non-technical user,

both are doable,

but Tailscale’s setup

is more of a login

once and forget it experience,

whereas Cloudflare requires

you to manage tunnel credentials

and a hostname.

So if I had to pick

for R1 Cord,

Tailscale gives you

better security

with nearly the same

setup effort.

The only real cost

is that extra login

step on the Rabbit,

and that’s a one-time thing.

Yeah,

so let’s talk about that

for a couple reasons.

One is I want to understand

the memory footprint

of it,

and if it’s going to

chew up any cycles,

because that’s an underpowered

device,

the Rabbit is.

And number two is

this is a very minimum,

I think it’s an Android 12,

maybe,

or 13 OS,

that goes on the Rabbit

that you put on the Rabbit.

That’s for hacking the Rabbit.

I think it’s maybe Android 13

that they give you

to hack it with.

And it’s got to work with that.

And it’s not,

you can’t go to a Play Store

to download it.

So can you download Tailscale

and install it

without going to the Play Store?

is it open source?

Hey, Grok.

Hey, Grok.

Hey, Grok.

So we were talking about

Tailscale versus Cloudflare,

and I’ve got a Rabbit

that we’ve hacked

that has a recording app,

and I’m trying to send data

from the Rabbit

using Android 13.

And my own APK,

it’s the only device,

and I want to understand,

one,

the memory footprint

of Tailscale on the Rabbit,

which is what you recommended

we use Tailscale

instead of Cloudflare,

what the memory footprint is,

and how we install it

on the Rabbit

if we don’t have the Play Store

and we don’t have

a user ID

on the Rabbit.

The Rabbit is just going to be

this little utility device

that’s just a recorder,

just an offline recorder,

and when it goes online,

it’ll upload its files.

So I’m trying to understand

is Tailscale

is that open source?

If so,

can I put it

inside my APK,

or do I,

does it have its own APK

that I can install

without having to go through

the Play Store?

Hey, Grok.

Hey, Grok.

Okay, Grok.

Can you hear me?

Can you hear me?

Hello?

Can you hear me?

Hello?

Hey, I’m here.

What can I help you with?

Okay, we were just talking

and got interrupted

to get some kind of issue,

but we were talking about

Tailscale on the Rabbit device

when you convert a Rabbit

to an Android 13 machine.

what is the memory footprint

of Tailscale?

And the machine I’m putting on

is a dedicated voice recorder,

offline, online voice recorder,

and I want it

to be able to upload files

to my computer at home.

And I want to understand

how I install,

if I don’t have a Google account

to install from

on the device,

or can I just,

is Tailscale open source?

And if so,

can I just install it

as an APK directly?

Let me check the latest

on Tailscale’s footprint

and sideloading.

Yes,

Tailscale is fully open source,

and you can absolutely install it

as a plain APK

with no Google account involved.

Official stable APKs

are at

pkgs.tailscale.com

slash stable

slash hash android.

And the same builds

are mirrored

on the Tailscale Android

GitHub releases page,

so you can grab them

with Uptanium

or just sideload directly.

FDroid also builds

from the same source

if you prefer that route.

On memory,

Tailscale’s baseline

on mobile

is roughly 10 to 15 megabytes

of RAM at idle.

That’s from their own profiling

on iOS,

where they were capped

at 15 megabytes total

and sat around 13.

On Android,

it runs a bit higher,

but still in that all-perk,

well under 20 megabytes

steady state.

For a dedicated voice recorder,

that’s plenty,

especially since

the Rabbit R1

has around 4 gigabytes

of RAM

even after the Android

conversion eats some of it.

For authentication

without a Google account,

the app has a sign-in

with other option

that supports

any SSO provider,

or you can use

an auth key

from the Tailscale

admin console

for headless

or kiosk-style setup.

There’s also a QR code flow

where the device

displays a code

and you approve it

from your admin console

on another machine.

One thing to keep

in mind,

though,

Tailscale only gives you

the network tunnel.

For actually pushing files

to your home computer,

you’ll still want something

like an SFTP server,

an SMB share,

or a simple upload endpoint

running on the receiving end.

Tailscale just makes

that endpoint reachable

as if it were

on your local network.

Want me to dig into

any of those

file transfer options?

No, no.

What I want to do

is I want to understand

better

exactly

how I register

the Tailscale app

on the Android.

So I can install it.

The server app

on the PC

has a USB-C cord

connected to the

Rabbit.

So I’m assuming

I can install Tailscale

using that USB cord.

Can I also authenticate it

at the same time

I install it

if I have the servers

already set up

with Tailscale?

Yes.

You can do both

over that USB-C connection.

Since your PC

already has Tailscale running,

here’s the cleanest path.

First,

on the Rabbit,

enable developer options

and turn on USB debugging.

Plug it into the PC,

then from a terminal

on the PC,

run ADB devices

to confirm it shows up.

Next,

grab the official APK

from PKG.

I know.

I think I know

how to do all that.

I just wanted to understand.

So Tailscale

looks like it’s

a better approach

than using

a Cloudflare tunnel.

It’s going to be

more secure.

Yeah.

For your setup,

Tailscale

is the better fit.

A Cloudflare tunnel

works by punching

a hole through

Cloudflare’s edge network

to expose a service

on your home machine,

which means that machine

has to run a tunnel daemon

and you’re relying

on Cloudflare’s infrastructure

as a middleman.

Tailscale instead

builds a private mesh

directly between your devices

using WireGuard,

so the connection

is end-to-end encrypted

with no third party

in the path,

and your home PC

doesn’t need to expose

anything at all.

For a dedicated recorder

just pushing files

to one specific machine,

that direct peer-to-peer approach

is simpler,

lighter,

and more secure.

Can I have multiple servers

on my computer

and each of them

have a different

Tailscale endpoint?

Yes.

Each instance just needs

its own state directory

and socket path

so they don’t collide.

On Linux,

you’d launch separate

Tailscale processes

with different

hyphen state directory

and hyphen socket flags,

and each one registers

as its own known

with its own Tailscale IP.

You can tag them differently

or put them on

different tailnets entirely.

On Windows,

it’s a bit more awkward

since there’s no native

multi-instance support,

but you can run them

in separate containers

or WSL instances

to get the same effect.

Can I serve up,

is there any kind

of server,

can I create

a tiny little server

and serve up

a web page

from within my desktop

computer using Tailscale?

Absolutely.

Tailscale has a built-in feature

called Serve

that does exactly that.

You run a tiny web server

on your desktop,

something like a Python HTTP server

or Nginx,

then point Tailscale Serve

at it with a command like

Tailscale Serve 8080.

That makes the page available

at your machine’s

Tailscale IP

or magic DNS name,

but only to devices

on your Tailnet.

So your rabbit,

your phone,

anything connected

to your Tailscale network

can hit it

and nothing outside can.

If you ever wanted it public,

you’d use Funnel instead.

But for your use case,

Serve is the right call.

Hey, Grok,

let’s talk about

a video,

it’s kind of a sketch

drawing technique

you see in videos.

I’m trying to think

of the company that does it,

who created it,

but the whole idea

was that,

you know,

you see a hand

at a whiteboard

and it starts drawing something

and there’s a narrator

in the background

doing something.

What is that called?

That’s usually called

a whiteboard animation,

also known as

a sketch animation

or doodle video.

The big names

in that space

are Video Scribe,

Doodly,

and Common Craft.

Common Craft

basically pioneered

the explainer style

with simple hand-drawn visuals

and a calm narrator.

RSA Animate

took it further

with more artistic,

fast-paced

whiteboard sketches.

Any of those ring a bell?

Yes,

RSA Animate

does a really good job.

So,

let’s say I want to create

a video

and I want it to be

in the style

of RSA Animate.

I don’t really even need

the hand

and the pencil

in the picture.

I just want to see it

being drawn

on a whiteboard.

You know,

how would I go about

creating a video

like that with AI?

Would I need to create

a custom tool first

or are there any custom tools

out there that do that?

Let me check

what’s out there right now.

Good news,

you don’t need

to build anything custom.

There are several

solid options right now.

The closest match

to what you’re describing

is Ancon AI.

You give it a topic

or script,

it writes the narration,

sketches each scene

stroke by stroke

on a clean whiteboard

and syncs the voiceover.

It has a no-hand mode

so you get just

the drawing effect

without a visible hand.

animation.ai

does something similar

but starts from images

you already have

and animates them

with an outline-first,

no-hand option.

Are these open source

or are these

subscription-based?

Mostly subscription-based.

Ancon AI,

Colpo,

and Animaker

all run on monthly plans.

Okay,

I want to make my own.

Okay,

so that’s first.

Or use open source,

you know,

open source parts,

using open source parts

but I definitely

want to make my own.

And I’ve got,

you know,

access to Claude

and Codex

and that can use

the frontier AIs

to drive this thing

but I want,

this is not

a commercial product,

this is a product

for me to create

my own,

my own talking,

you know,

my own videos.

I have a YouTube account

so I can just,

you know,

put them on there.

So,

that being said,

that’s,

how would I go about,

you know,

what are some ways

I go about

creating something like that?

Let me pull up

the open source options.

There’s a really

clean path here

and the good news is

you don’t have

to start from zero.

The best open source

render engine out there

is called

Whiteboard-Animator

on GitHub.

MIT Licensed,

runs on CPU only,

no GPU or API keys

needed for the core rendering.

You feed it a clean

whiteboard-style image

and it traces every stroke,

writes text word by word,

fills shapes with brush strokes,

and paces it all

to a narration audio file.

That’s your drawing engine.

Then you wrap it

with an AI pipeline.

The architecture

looks like this.

First, Claude or Codex

takes your topic

and writes a scene-by-scene script.

Each scene is one idea,

roughly 12 to 15 words,

with a visual description.

Second,

you send each scene description

to an image model,

something like Flux or GPT image,

and ask it to draw a clean,

black ink on white illustration.

Third,

you run each image

through Whiteboard-Animator

with a matching narration clip.

Fourth,

FMPG stitches the scenes together

into one video.

For the narration,

you can use 11 labs

if you want a polished voice

or Edge TTS,

which is free

and needs no API key.

There’s also a project

called HandDraw-Skill

that does almost exactly

this end-to-end.

It uses SVG,

Remotion,

FMPG,

and free Edge TTS,

and it’s designed

to be driven

by a coding agent

like Claude Code or Codex.

You could fork that

as your starting point

and customize the visual style

to match RSA Animate

more closely.

The one thing

Whiteboard-Animator

doesn’t do

is generate the images for you.

It only animates

what you give it.

So the image generation step

is where you lean on

Claude or Codex

to craft good prompts

or call an image API directly.

Want me to dig deeper

into any one of those pieces?

Yeah,

I’d like you to try

to figure out

how they

understand

A,

how to

know where to put

the pencil to draw

based on

the script.

Meaning,

so if I have like

a picture of an apple

and an orange

and a pear

and I first start talking

and I start with a white screen

and I first start talking

about the apple,

does it draw,

how does it know

to draw the apple?

and B,

how does it even,

is it drawing

by just removing

a mask?

Is it drawing,

removing a mask,

creating a raster mask

and only exposing

the parts of the line

where its cursor

or the pen

is going over?

Or is it vectorizing

the whole thing

and then drawing it

using vector shapes,

you know,

basically using

the vector shapes

as paths?

Great questions.

Those are the two

hardest parts

of the whole thing.

Here’s the honest answer

on both.

On the first question,

it doesn’t actually

understand the script.

It’s not doing

semantic matching

between apple

in the narration

and the apple

in the image.

What it does

is spatial heuristics.

Containers get drawn

before their contents,

shapes before their labels,

text and reading order,

small dots attached

to whatever glyph

they belong to.

So if your apple,

orange,

and pear

are just three separate blobs

on a white background,

it’ll pick one

based on position,

probably left to right,

top to bottom.

Not because it knows

you’re talking

about the apple first.

You can override that

with a region plan,

which is a JSON file

where you explicitly list

each region

and its reveal order.

There’s also

an optional Gemini step

where you pass in the image

plus your narration text

and Gemini proposes

which regions to draw

and in what order.

But even then,

the final order

comes from that explicit plan,

not from real-time

speech understanding.

So the smart part

is really just

a one-time LLM pass

that annotates your image,

not the renderer itself.

On the second question,

it’s raster,

not vector.

No SVG,

no path tracing

in the vector sense.

Here’s what actually happens.

It scans the image

for connected blobs

of non-white pixels.

Anything darker

than about 240 gray

becomes a component.

A small text detector model

flags which blobs

are words

so those get written

left to right

instead of traced

as outlines.

Then for each component,

it computes a skeleton,

basically the centerline

of the stroke,

and starts revealing

from a real endpoint,

like the tip of a V.

Closed shapes

get a single traveling front

that walks around

the boundary.

Fills get an outline pass

first,

then either an angled sweep

or bristled brush strokes

depending on size.

And anything with junctions,

like an X or a grid,

gets decomposed

into sequential pen paths

so it doesn’t just bloom

outward from the center.

Each pixel gets assigned

a reveal timestamp.

As frames render,

finished pixels

get committed permanently,

and only the pixels

currently being drawn

get a fade blend.

Everything streams out

through FMFPEG,

so it’s really

a pixel-level mask reveal

guided by skeleton geometry,

not vector paths,

but not a dumb

left-to-right wipe either.

The skeleton tracing

is what makes it feel

like an actual hand drawing

rather than a curtain opening.

The trade-off

is it only works well

on clean ink-on-white

illustrations.

Photos,

gradients,

textured backgrounds,

those fall apart

because the connected

component detection

can’t separate

meaningful strokes

from noise.

Okay, I got that.

I got that.

So, let me ask,

I’m just brainstorming here.

What,

has anyone tried

to do this

using an AI video?

So, with AI video,

you could give it

a picture and say,

I want you,

I think,

maybe you can say this,

I want you to

progressively disclose this

in this order.

Draw the apple first,

and then shade it red,

and then draw the pear second,

and shade it,

you know,

kind of a golden yellow,

and then finally,

in the bottom right-hand corner,

I want you to zoom in

and draw the orange last,

and shade it orange.

Yes,

people have tried

exactly that,

and there’s even

a research project

built for it.

It’s called VideoSketcher.

An LLM takes your prompt,

breaks the subject

into semantic parts,

and decides

the drawing order,

like cake base first,

then frosting,

then candles.

That plan gets fed

to a video diffusion model

trained on artist SVGs,

where each stroke

animates along its path.

So the model literally

generates a video

of strokes

appearing progressively

on a blank canvas

in a meaningful order,

not just a random wipe.

The catch is,

it’s a research prototype,

not a polished product

you can just open and use.

And the control is semantic,

not pixel-precise.

You can’t say,

draw the apple,

then shade it red,

then zoom to the bottom right

for the orange.

You get a rough order of parts,

but the model decides

the exact strokes.

What creators are actually

doing in practice

is the start and end frame trick.

You generate a blank whiteboard

as frame one,

a whiteboard with just

the apple outline

as frame two,

then a version

with the apple shaded red,

then one with the pear added,

and so on.

You chain those as keyframes

through C-dance or VO,

and the model interpolates

the drawing motion

between each state.

Kling’s motion brush

helps too.

You paint over the region

you want to animate

and leave the rest static,

so only the apple moves

while everything else

holds still.

So the honest answer is,

the idea works,

but nobody’s nailed

fine-grained,

script-driven,

stroke-level control

with a pure video model yet.

The deterministic engines

like whiteboard animator

still win on precision.

The video models win

on organic,

hand-drawn feel.

They just can’t follow

a strict order.

I see.

So,

uh…

So,

the practical takeaway

is this.

If you want pixel-perfect

control over drawing order,

stick with a deterministic

renderer like

whiteboard-animator

and let an LLM

just plan the regions.

If you want that

organic,

hand-drawn feel

where the strokes look alive,

lean on a video model,

but accept you’ll be

chaining keyframes

and babysitting the order.

Most people building

this for real

end up doing a hybrid.

Generate the clean illustration

with an image model,

run it through a stroke renderer

for the drawing motion,

then use a video model

only for the camera moves

or shading transitions

where you don’t need

exact stroke control.

That gets you maybe

80% of the RSA animate look

without fighting the model

on every frame.

Want me to sketch out

what that hybrid pipeline

would actually look like in code?

No, no.

Here’s what I want you to do.

Let’s think about this

for a second.

Let’s think about

a complete RSA animate type

drawing,

whiteboard drawing.

And when you zoom out

at the end,

you see like,

I don’t know,

there’s probably 30 or 40

different drawings

on the whiteboard.

But you started

at the top left

and you kept drawing

and you moved

and you panned

and scanned

and went back,

you know,

zoomed in a little bit,

zoomed out a little bit.

but the end

is in way out.

So if you wanted

to create that type

of video,

then it seems to me

that what you’d want

to do is

you’d want to

create the video

at a super high resolution

first

and then add

the camera movements

based on the narration

second.

Does that make sense?

That makes total sense.

And it’s actually

how the real RSA animate

videos were built.

cognitive,

the studio behind

RSA animate

worked on a giant canvas,

sometimes several times

the final frame size

and the camera

was a separate layer

that moved around it.

So your instinct

is exactly right.

You render the full

whiteboard at say

4K or 8K

with every drawing

placed in its final position.

Then you define

a camera path,

a sequence of keyframes

with timestamps,

each one saying

at three seconds,

be zoomed

into the top left

at 200%.

At seven seconds,

pan right to the apple.

At 12 seconds,

zoom out to show everything.

That camera path

gets driven

by your narration timestamps

so the drawing

and the voice

stay locked together.

Tools like Remotion

handle this natively.

It’s basically

a React component

where you animate

where you animate

a transform

with a big image.

I know Remotion.

So how do I,

what tool would I use

to script

creating that

extremely large

frame video?

You generate it,

right,

a grid of tiles.

Each tile

gets generated

at normal resolution,

say 1024x1024,

with prompts

that describe

what’s in that region

and what bleeds

into the neighboring tiles

so the seams match up.

Then you composite them

with a small overlap

and a soft blend

at the edges.

And you’ve got

one giant canvas,

maybe 8K

or 16K total.

The trick

is the prompts.

You feed each tile generation

the context

of its neighbors,

so a drawing

that straddles

two tiles

doesn’t get cut

in half.

Wait, wait, wait, wait, wait, wait, wait.

So are you saying,

let me back up.

Are you saying

you create a finished,

the finished render first?

Is that what you’re saying?

Yes, exactly.

You generate

the entire finished whiteboard,

all 30 or 40 drawings,

fully inked and shaded,

sitting in their final positions,

as one static image first.

Nothing is animated yet.

Then in Remotion,

you load that single image

and animate

a virtual camera window

that pans,

zooms,

and reveals pieces of it

over time.

Synced to your narration.

The drawing motion itself

comes from the camera

revealing what’s already there,

not from generating

strokes on the fly.

That’s what gives you

that RSA animate feel

where the whole board

exists from the start

and the camera

just discovers it

piece by piece.