Sep 25, 12:20

Overview

The speaker continues a talk with Grok about building an AI pipeline that turns a single concept into an RSA Animate-style scribe video with no human in the loop. They cover how to lock the drawing style from a real RSA video, how to tie each part of an image to the narration so elements draw in the right order, how to stitch six chapter images into one 8K whiteboard that a small camera window pans across, and how to end on a zoom-out to the full picture. Grok has a bot draft an outline and spec, the speaker names the skill “Scribe Board”, and Grok starts reading the spec back. The recording ends partway through Phase 0 of the technical spec.

Key points

Style extraction

  • Lock the style first, from real keyframes, instead of guessing at a look.
  • Download an RSA video, take a stretch the speaker likes (a quarter to a third, or about five minutes), and pull every frame (about 150).
  • Use Codex to break the video out, and ChatGPT to turn keyframes into a style prompt.
  • Study how the sketches are drawn (stroke order, layering, motion) and write it up in a Markdown file for later video generation. The hand and pen don’t matter.

Drawing and panning

  • RSA Animate fills one blank section, starting top left and spreading out, then pans gently to the next section while still drawing.
  • A pan should never land on pure white; some finished drawing should stay in view.
  • Once drawn, lines stay fully opaque, like a real whiteboard. The speaker rejected Grok’s idea of dimming finished sections.
  • One way to animate: lay a white image over the finished drawing and erase it where the lines are.

Narration-to-drawing sync

  • Break the narration into segments; each segment has a drawing, which may be complex.
  • Segment each image into its parts and reveal them in the order the words name them. Example: Father Time figure first, then the calendar, then the “two months” arrow and label.
  • Record the narration in the speaker’s voice, timestamp it, and use the timestamps to pace each segment’s reveal.
  • Structure: about six scenes, each with five or six segments, joined into one video.

One big whiteboard

  • Combine the six chapter images into one 8K image that reads as a single flowing whiteboard, with no visible square edges.
  • The speaker thinks AI can cut, move and splice the pieces without an image generator.
  • The big image keeps its segmentation, so a small sliding window can animate across it by timestamp.
  • At the end, zoom out so the whole image fills an HD frame.

Scribe Board spec

  • The speaker wanted “scribe” in the name and chose “Scribe Board” over Scribe Flow and Scribe Sync.
  • The outline has nine phases: style extraction, script, voice and timing, image generation, segmentation, composition, timeline, render, and orchestration and QA.
  • The canvas is 7680 by 4320 (16:9, four times 1080p); masks and strokes move with their images.
  • The only input is the concept.
  • Phase 0 inputs are the reference video URL and chunk start and end times. The tools are yt-dlp, FFmpeg, OpenCV, a vision model and Codex. Sample about 150 frames by scene-change detection plus a fixed interval, compare frame to frame to isolate new ink, and mask out the hand.

Action items

  • Finish reading back the Scribe Board technical spec, starting from Phase 0.
  • Pick a reference RSA video and the stretch to use for style extraction.
  • Build the Scribe Board skill from the outline and spec.