![Build Your Own Dev-Brain — sketchnote overview](dev-brain-setup-guide_images/dev-brain-sketchnote.png)

# How to Build Your Own "dev-brain"

A dev-brain is a **folder of Markdown notes that an AI agent writes and keeps up to date**, opened in Obsidian so you can read and browse it. It collects what you've learned across all your projects: shared patterns, gotchas, decisions, and one page per project. Each project's own `AGENTS.md`/`CLAUDE.md` covers how that project works. The dev-brain covers what links the projects together.

> **A note on Obsidian:** it isn't strictly necessary. The wiki is plain Markdown that your agent reads and writes on its own. But Obsidian is well worth having if you want to inspect and review the wiki yourself, and to use its crazy graph view, shown below for a real dev-brain: every dot is a wiki page and every line is a `[[link]]` between pages.

![Obsidian graph view of a dev-brain wiki](dev-brain-setup-guide_images/obsidian-graph-view.png)

## It's Fully Autonomous

Once it's installed, **you never touch it.** Every morning a scheduled task:

1. scans the folders you told it to watch,
2. finds what changed since yesterday (new commits, edited files, brand-new projects, deleted ones),
3. runs your AI agent headless to update the wiki pages,
4. commits and pushes the result to your private GitHub repo.

No prompts, no approvals, no copy-paste. You just open Obsidian (or ask your agent a question) and the knowledge is there, current as of this morning.

## Install It by Handing This File to Your Agent

You don't have to build any of this by hand. **Get this guide's `.md` file and feed it to whatever agent harness you use**: Claude Code, Codex, Grok Build, omp, and so on.

- **Folder:** https://www.widgetgadget.com/cw1/other/dev-brain-install/
- **The `.md` file itself:** https://www.widgetgadget.com/cw1/other/dev-brain-install/dev-brain-setup-guide.md

Either download the `.md` and attach it, or just paste the link into your agent. Say something like:

> Read https://www.widgetgadget.com/cw1/other/dev-brain-install/dev-brain-setup-guide.md and set up a dev-brain for me on this machine, following it exactly. Ask me anything you need first.

The agent reads the guide, asks you a few questions (below), creates the vault, writes the scripts, seeds the wiki from your projects, and registers the daily task. The stages further down are there so you (and the agent) understand what's being built; you don't need to type them in yourself.

## Tell It Which Folders to Monitor

The one thing the agent can't guess is **where your projects live.** You tell it which folders to watch, and the daily sync updates the dev-brain from anything in them.

The simplest setup, and the one this guide assumes: **keep all your repos (projects) in one parent folder**, for example `C:\Code\` or `D:\GitHub\`. Point the dev-brain at that folder and you're done:

- **Every project folder inside it is watched automatically.** Clone or create a new project there and the next morning's run notices it and writes a page for it. No extra setup.
- **Folders elsewhere** (a client workspace on another drive, a notes folder) can be added to an "extra folders" list.
- **Folders you want kept out** (taxes, credentials, archives, anything private) go in an "ignore" list and are never read.

**It never copies your files.** The dev-brain only *reads* the watched folders: git history, READMEs, `AGENTS.md`/`CLAUDE.md`, config files. It writes its own short summaries into the wiki. Your code stays where it is, and nothing from those folders is duplicated into the dev-brain repo.

## Instructions for the AI Agent Installing This

*If you are an agent reading this on behalf of the user, follow this section.*

1. **Ask the user first** (one batch of questions):
   - Which parent folder holds all their projects? (Default suggestion: the folder most of their git repos live in.)
   - Any extra folders outside it to watch?
   - Any folders inside it to **ignore** (private, archives, logs)?
   - Where should the `dev-brain` vault live? (Default: inside the projects folder, e.g. `C:\Code\dev-brain`.)
   - The URL of an empty **private** GitHub repo for it (or permission to create one with `gh`).
   - Which agent CLI should the daily run use in headless mode, and which (cheap) model?
   - What time should the daily run happen? (Default 06:07.)
   - Do they want Stage 6 (learning from agent chat sessions)?
   - Is Obsidian installed? If not, may you install it for them? (Optional: the wiki works without it, but Obsidian is the nicest way to browse it.)
2. **Install Obsidian if the user said yes** and it isn't already installed. On Windows: `winget install --id Obsidian.Obsidian --exact` (macOS: `brew install --cask obsidian`). Confirm the install succeeded.
3. **Build Stages 1–5** (and 6 if wanted) exactly as described below, substituting the user's answers for `C:\Code\`, the folder lists, the CLI command, and the time. Write real, working scripts; don't leave placeholders.
4. **Seed the wiki** (Stage 2's seed prompt) and run a lint pass.
5. **Verify before handing off:** run `detect-drift.ps1` and show the report; run `daily-autosync.ps1` once by hand and confirm the commit reached GitHub; confirm the scheduled task is registered and points at `daily-autosync.ps1`.
6. **Tell the user** where the vault is, how to open it in Obsidian, and what time the daily run happens.

**What you need**

- One parent folder holding all your projects (this guide uses `C:\Code\`; swap in your own)
- Git and a **private** GitHub repo for the wiki
- [Obsidian](https://obsidian.md) to read it
- An agent harness with a headless mode: Claude Code, Codex, Grok Build, omp, …
- Windows (the scripts are PowerShell and use Task Scheduler; on macOS/Linux, ask your agent to port them to a shell script and `cron`/`launchd`)

## How It Works

```mermaid
flowchart LR
  R[Your repos] -->|git log / file dates| D[detect-drift script]
  S[Agent session transcripts] -->|scrubbed digests| X[distill script]
  D --> M[drift report .md]
  M --> H[headless agent run]
  X --> H
  H --> W[wiki pages + index + log]
  W -->|git push| G[GitHub]
  W --> O[Obsidian]
```

The rest of this guide describes what gets built, in stages. Stages 1–3 are already useful by themselves. Stages 4–6 make it update itself every day.

---

## Stage 1: Set Up the Vault (≈30 min)

Create this layout:

```
dev-brain/
  AGENTS.md          # the schema: rules the agent follows (copy to CLAUDE.md too)
  .gitignore
  raw/               # source docs you drop in; the agent never edits them
    assets/          # images referenced by sources
  wiki/
    index.md         # catalog: one line per page; agents read this first
    log.md           # append-only activity log
    sources/         # one summary per ingested raw/ document
    entities/        # one page per project (and per tool/service)
    concepts/        # reusable techniques and patterns
    topics/          # platform-level overviews (e.g. "Electron Development")
    comparisons/     # side-by-side analyses
    meta/            # drift reports, health checks
```

`.gitignore`:

```
*.log
.obsidian/
.sessions/
desktop.ini
Thumbs.db
.DS_Store
```

Open the `dev-brain` folder in Obsidian as its own vault (**Open folder as vault**).

The three layers:

- **`raw/`**: documents you curate (articles, reports, notes). Treated as unchangeable.
- **`wiki/`**: pages the agent writes, all following a strict schema.
- **`scripts/`**: automation (stages 4–6).

---

## Stage 2: Write `AGENTS.md`, the Schema

This file is the most important part, because the agent obeys it. Codex reads `AGENTS.md`; Claude Code reads `CLAUDE.md`. Keep the two files identical. Here's a minimal starter based on a working one:

````markdown
# dev-brain — LLM Wiki Schema

A developer second brain. The agent owns the wiki layer; the human curates
sources, asks questions, and directs analysis.

## Focus Areas
- (list your platforms, e.g. web apps, Python tools, game mods)

## Repo Location
All project repos live in `C:/Code/`. Each has its own AGENTS.md.
This wiki synthesizes across all of them.

## Page Format
Every page:
```markdown
---
title: Page Title
aliases: [Page Title]
type: source | entity | concept | topic | comparison | meta
created: YYYY-MM-DD
updated: YYYY-MM-DD
sources: []
tags: [tag-one, tag-two]
---
# Page Title
Content. Use [[wikilinks]] for cross-references.
## See Also
- [[Related Page]]
```
- `aliases` MUST contain the title verbatim (double-quote it if it contains
  `:` `,` `[` `]` or `\`). Obsidian resolves [[Title]] only by filename or
  alias; without it, clicking a link creates an empty stub file.
- Project entity pages carry `repo: C:/Code/<repo>`.
- `updated:` is load-bearing: drift detection compares it to the repo's last commit.
- Source references point to filenames in `raw/`.
- One subject per page; prefer many small interlinked pages.

## Workflows
### Ingest from Repo
1. Read the repo's AGENTS.md / README / package.json / pyproject.toml.
2. Create/update `wiki/entities/<kebab-name>.md`.
3. Update topic pages if the project's scope changed.
4. Create/update concept pages for notable shared patterns.
5. Update `wiki/index.md`; append to `wiki/log.md`.

### Ingest (raw source)
1. Read `raw/<doc>` fully; discuss key takeaways with the user.
2. Write `wiki/sources/<doc>.md`.
3. Update affected entities, concepts, topics; add [[wikilinks]] both ways.
4. Update index and log. One source may touch 10-15 pages; that's expected.

### Query
Read `wiki/index.md` first, then relevant pages; answer with [[wikilink]] citations.

### Lint
Report contradictions, stale claims, orphan pages (no inbound links),
broken wikilinks, mentioned-but-missing pages, missing cross-references.

## Conventions
- Filenames kebab-case; wikilinks use the page title.
- Tags lowercase-hyphenated. Dates ISO 8601.
- When sources disagree, record both positions with citations.
- If a claim rests on one source or is speculative, say so.
- Fail loud, never fake.

## Index and Log
index.md: sections by type (Topics, Entities by platform, Concepts, Sources,
Comparisons, Meta); each entry `- [[Title]] — one-line summary`.
log.md entry format:
## [YYYY-MM-DD] action | Subject
Brief description. Pages created/updated.
Actions: ingest, query, lint, update, create.
````

### Seed the wiki

Open your agent CLI inside `dev-brain` and run:

> Read AGENTS.md. Ingest every repo under C:/Code that has an AGENTS.md, CLAUDE.md or README: one entity page each. Then create topic pages for the platforms they share, and concept pages for patterns used in 2+ repos. Build index.md and log.md.

Then ask for a **lint** pass and fix what it finds. A good second pass: "Deep enrichment pass: for each entity, scan key files and add conventions, gotchas and dependencies."

---

## Stage 3: Using It From Other Projects

Add a read-only rule (or skill) to your global agent instructions (e.g. `~/.claude/CLAUDE.md` or `~/.codex/AGENTS.md`):

> For cross-project knowledge, read `C:\Code\dev-brain\wiki\index.md` first, then grep `C:\Code\dev-brain\wiki\` for keywords (frontmatter `tags:` first), follow `[[wikilinks]]` one hop, and cite full page paths. Never edit the wiki from a normal session. Don't duplicate wiki content into project docs.

**Make it opt-in.** Loading the wiki on every session start wastes tokens. Only consult it when you ask, or when the answer really isn't in the current repo.

---

## Stage 4: Drift Detection (PowerShell, No AI Involved)

`scripts/detect-drift.ps1` is how the dev-brain **monitors your folders.** It **only reads** them (it never copies or changes a file) and writes one dated report to `wiki/meta/drift-YYYY-MM-DD.md` with these sections:

1. **Drifted entities**: the page's `repo:` has a newer commit (`git log -1 --format=%cI`) than the page's `updated:`. List the commits since that date. For folders without git, compare the newest file modification time instead.
2. **Orphan repos**: folders under the root with no entity page and activity in the last 30 days (commits, file changes, or a freshly created `.git`).
3. **Missing repos**: the entity's `repo:` folder no longer exists.
4. *(Optional)* **Agent config changes**: files in `~/.claude` (skills, settings) newer than a "workspace" topic page.
5. *(Optional)* **Link hygiene**: pages missing `aliases`, unresolved `[[links]]`.

Settings to keep at the top of the script. These three lists are the "which folders to monitor" answers from the install questions:

```powershell
$Wiki       = 'C:\Code\dev-brain\wiki'
$ReposRoot  = 'C:\Code'    # the ONE folder holding all your projects; every subfolder is watched
$ExtraRoots = @()          # folders OUTSIDE the root, watched by file mtime
$IgnoreDirs = @(           # top-level folders never scanned
    'archive'
    'taxes'                # private folders: must never reach the pushed wiki
)
$DocExt   = @('.md','.txt','.json','.js','.ts','.py','.ps1','.html','.css','.yml','.yaml','.toml')
$SkipDirs = @('.git','node_modules','__pycache__','.venv','venv','dist','build','out')
$MaxFiles = 40             # cap on changed files listed per non-git folder
```

Core logic: parse each `wiki/entities/*.md` frontmatter for `repo:` and `updated:`, normalize paths (forward slashes, lowercase), compare against the repo, render the sections into Markdown.

Test it by hand first:

```powershell
pwsh -NoProfile -ExecutionPolicy Bypass -File C:\Code\dev-brain\scripts\detect-drift.ps1
```

---

## Stage 5: Daily Autonomous Sync

### The wrapper: `scripts/daily-autosync.ps1`

```powershell
$Repo  = 'C:\Code\dev-brain'
$Log   = "$Repo\scripts\daily-sync.log"
$Today = Get-Date -Format 'yyyy-MM-dd'
function Log($m) { Add-Content $Log ("[{0}] {1}" -f (Get-Date -Format 's'), $m) }

Log '=== start ==='
# 1. Drift report (non-fatal)
try { & "$Repo\scripts\detect-drift.ps1" *>&1 | ForEach-Object { Log "drift: $_" } }
catch { Log "drift ERROR: $($_.Exception.Message)" }

# 2. Headless agent edits pages only (no git)
if (Test-Path "$Repo\wiki\meta\drift-$Today.md") {
  $prompt = "You are the daily scheduled dev-brain run, the wiki's only writer. " +
    "Process wiki/meta/drift-$Today.md following AGENTS.md: update drifted entity pages " +
    "(signal = features, breaking changes, gotchas, dependency changes; noise = chores, " +
    "typos -> just bump updated:), read changed README/AGENTS.md/package.json rather than " +
    "trusting commit subjects, ingest orphan repos (skip anything that looks private and " +
    "print 'SKIPPED orphan <name>'), archive missing ones to wiki/entities/_archived " +
    "(never delete), update index.md, add one log.md entry. No confirmation. " +
    "Do NOT run git. Print a one-line summary."
  Push-Location $Repo
  '' | & claude -p $prompt --model <cheap-model> --permission-mode bypassPermissions --add-dir $Repo 2>&1 |
    ForEach-Object { Log "agent: $_" }
  Pop-Location
}

# 3. Commit + push (deterministic)
git -C $Repo add -A
$leak = git -C $Repo diff --cached --name-only | Where-Object { $_ -match '^\.sessions/|\.jsonl$' }
if ($leak) { Log "REFUSING: session material staged: $leak"; git -C $Repo reset; exit 1 }
if (git -C $Repo status --porcelain) { git -C $Repo commit -m "chore(daily): autonomous wiki sync $Today" }
git -C $Repo push origin main 2>&1 | ForEach-Object { Log "push: $_" }
if ($LASTEXITCODE -ne 0) { Log 'push FAILED'; exit 1 }   # fail loud: shows in Task Scheduler
Log '=== end ==='
```

Swap `claude -p` for your harness's headless command (`codex exec`, `omp -p`, Grok Build's headless mode, …).

### Register the scheduled task

```powershell
$action    = New-ScheduledTaskAction -Execute 'pwsh.exe' -Argument '-NoProfile -ExecutionPolicy Bypass -File C:\Code\dev-brain\scripts\daily-autosync.ps1'
$trigger   = New-ScheduledTaskTrigger -Daily -At 6:07am
$principal = New-ScheduledTaskPrincipal -UserId $env:USERNAME -LogonType Interactive -RunLevel Limited
$settings  = New-ScheduledTaskSettingsSet -StartWhenAvailable -DontStopIfGoingOnBatteries -AllowStartIfOnBatteries -ExecutionTimeLimit (New-TimeSpan -Hours 1)
Register-ScheduledTask -TaskName 'dev-brain-drift' -Action $action -Trigger $trigger -Principal $principal -Settings $settings -Force
```

Point the task at the **wrapper**, not at `detect-drift.ps1`; pointing it at the detector silently disables the sync. An off-the-hour time like 06:07 avoids colliding with other jobs scheduled on the hour.

### Manage it

```powershell
Get-ScheduledTaskInfo -TaskName 'dev-brain-drift' | Format-List NextRunTime, LastRunTime, LastTaskResult
Start-ScheduledTask   -TaskName 'dev-brain-drift'     # run now
Disable-ScheduledTask -TaskName 'dev-brain-drift'     # pause
Enable-ScheduledTask  -TaskName 'dev-brain-drift'
```

`LastTaskResult` of `0` = success; `267011` = never run yet. To sync from inside an agent session, start the task rather than letting the session edit pages.

---

## Stage 6 (Optional): Learn From Your Agent Sessions

`scripts/distill-sessions.py` reads the agent transcript files written since its last successful run:

| Harness | Transcript location |
|---|---|
| Claude Code | `~/.claude/projects/<project>/<session>.jsonl` |
| Codex | `~/.codex/sessions/YYYY/MM/DD/rollout-*.jsonl` |
| omp | `~/.omp/agent/sessions/<project>/<session>.jsonl` (plus subagent transcripts) |

It:

- keeps **only your prompts and the agent's final reply to each**; skips tool calls, tool output and thinking
- skips headless runs (`claude -p`, `codex exec`), sessions in Temp folders, and private folders (same list as `$IgnoreDirs`)
- redacts secrets (API keys, GitHub/HF/AWS/Google tokens, JWTs, private keys) and email addresses
- writes `.sessions/digest-YYYY-MM-DD-partN.md` (git-ignored), with each part capped at ~60k characters / 1,500 lines
- saves the "last processed" time to `.sessions/state.json` only when called with `--commit-state`, which the wrapper does after every part succeeds
- deletes digests older than 14 days

The wrapper then runs the agent once per digest part with these rules:

**Read the whole digest first.** If you didn't, make the first output line `INCOMPLETE: <reason>` (the wrapper treats that as failure and retries).

**Extract only knowledge someone would want weeks later:**

- decisions and why (chose X over Y because Z)
- gotchas: bugs with their root cause, things that look right but are wrong
- measured results: timings, costs, sizes, failure rates
- how a model or agent performed on a kind of work
- conventions or workflow rules you set
- new tools/services wired into a project

**Never write:** secrets or `[REDACTED]` placeholders, emails, personal/health/financial details, unfixed security weaknesses, long quotes, transcript or digest paths.

**Where it goes:** map the session's working folder to the entity page with that `repo:`; add `(session YYYY-MM-DD)` after each new fact; put cross-project knowledge in a concept/topic page; if the folder has no entity page and the knowledge isn't cross-project, leave it out. One log entry per part, even if nothing was worth keeping.

---

## Lessons Learned

| Lesson | Why |
|---|---|
| **Only the scheduled run edits the wiki**; interactive sessions just read it | Stops random sessions from scattering edits and contradicting each other |
| Every page needs `aliases: [Title]` | Otherwise Obsidian `[[Title]]` links break and create empty stub files |
| Always bump `updated:` when editing a page | Otherwise drift detection re-flags the page forever, or misses real changes |
| Run the task with **Interactive logon** | Git Credential Manager and the agent CLI's login need your user session. It catches up at next login (`StartWhenAvailable`) |
| Keep each digest under ~60k chars / 1,500 lines | A small model silently skipped most of a 200k-char input and still exited 0 |
| Use the `INCOMPLETE:` marker and only advance the "last processed" time when every part succeeded | Failed runs get retried instead of lost |
| Wrapper refuses to commit if `.sessions/` or any `.jsonl` gets staged | Raw transcripts must never reach GitHub |
| The agent run never touches git; the wrapper commits and pushes | Commits stay predictable and auditable (`git log --grep "chore(daily)"`) |
| Use a cheap model (Haiku-class) for the daily run | It's summarizing and filing work, not hard reasoning |
| Keep the GitHub repo **private** | It pushes to `main` every day with no review |
| Put private folders in `$IgnoreDirs` *and* the distiller's private list | Two separate paths can leak them: repo scan and session transcripts |
| Never delete pages; archive to `wiki/entities/_archived/` | History stays recoverable when a repo is renamed or removed |
| Drift reports pile up in `wiki/meta/` | Prune them now and then, or have the sync archive old ones |
| `raw/` is unchangeable only because the instructions say so | Nothing technical stops an agent from editing it; repeat the rule in prompts |
| The scripts hardcode paths | Moving your repo root means editing the config at the top of each script |

## Quick Start Checklist (what your agent does for you)

- [ ] Create the folder layout, `.gitignore`, and a private GitHub repo
- [ ] Write `AGENTS.md` (copy to `CLAUDE.md`)
- [ ] Open the folder as an Obsidian vault
- [ ] Run the seed prompt, then a lint pass
- [ ] Add the read-only rule to your global agent instructions
- [ ] Write `detect-drift.ps1`; run it by hand and check the report
- [ ] Write `daily-autosync.ps1`; run it by hand once
- [ ] Register the `dev-brain-drift` scheduled task
- [ ] *(Optional)* add `distill-sessions.py` and the per-digest runs
