Files

gongbaoxiong a67be62996 1

2026-07-22 09:34:39 +08:00

12 KiB

Raw Blame History

Mimik

Open-source Chrome extension that auto-captures browser workflows and generates step-by-step guides. No backend, no account, no data leaves the browser.

What It Does

You click "Record," perform a workflow in your browser, and Mimik automatically captures each action as a step with an annotated screenshot and description. You can edit the guide, replay it on a live page, or export it as a file.

Core loop: Record → Edit → Replay or Export.

Architecture

Everything runs in the Chrome extension. No backend.

Storage: IndexedDB via Dexie.js (browser-local)
AI descriptions: optional, user provides their own API key in settings
Export: generated client-side (no server rendering)
No auth, no database, no hosting, no Docker

Directory Structure

src/
├── core/                    # Business logic (no UI dependencies)
│   ├── capture/             # Recording pipeline
│   │   ├── ai/              # AI description + title generation (Vercel AI SDK)
│   │   │   ├── description.ts   # getAIDescription (DOM context → AI → step text)
│   │   │   ├── title.ts         # generateGuideTitle (steps → AI → guide name)
│   │   │   ├── models.ts        # AI_PROVIDERS config (OpenAI/Anthropic model lists)
│   │   │   ├── prompts.ts       # Prompt templates
│   │   │   └── provider.ts      # createModel factory (OpenAI/Anthropic)
│   │   ├── dom/              # DOM extraction utilities
│   │   │   ├── context.ts       # DOMContext extraction + serialization
│   │   │   ├── element-meta.ts  # extractElementMeta (selector, text, aria, rect)
│   │   │   └── element-utils.ts # findFocusableAncestor, isTextField, etc.
│   │   ├── events/           # Event capture system
│   │   │   ├── handlers.ts      # CaptureController class + startCapture
│   │   │   ├── highlight.ts     # HighlightManager (dashed border overlay)
│   │   │   └── input-session.ts # InputSession (typing lifecycle)
│   │   ├── machine.ts        # xstate capture state machine
│   │   ├── rrweb-recorder.ts # DOM recording for replay
│   │   ├── session.ts        # CaptureSession (lifecycle manager)
│   │   ├── spa-nav.ts        # SPA navigation tracking
│   │   ├── start-notification.ts # Recording notification overlay
│   │   └── step-description.ts   # Fallback rule-based descriptions
│   ├── blur/                # Smart blur: regex presets, DOM scanner, element picker, panel UI
│   ├── export/              # HTML, PDF, Markdown export generators + shared utils
│   └── guides/              # Data layer: types, Dexie DB, CRUD service
├── entrypoints/             # Chrome extension entry points (WXT)
│   ├── background/          # Service worker: state machine, message handlers, tab management
│   ├── content.ts           # Content script: CaptureSession, event listeners, rrweb
│   ├── sidepanel/           # Side panel React mount
│   ├── fullview/            # Full-page view React mount
│   ├── onboarding/          # Onboarding wizard (opens on first install)
│   └── options/             # Settings page React mount
├── lib/                     # Shared utilities
│   ├── messaging.ts         # Extension messaging protocol (webext-core)
│   ├── port.ts              # Long-lived port: background ↔ sidepanel
│   ├── browser-api.ts       # Chrome API wrappers
│   ├── tab-messages.ts      # Content script message types
│   ├── logger.ts            # Logging utility
│   └── utils.ts             # Shared helpers (dates, URLs, cn)
├── stores/                  # Zustand state stores
│   └── fullview.ts          # Fullview UI state (search, counts, guide data)
└── ui/                      # React components
    ├── components/ui/       # shadcn/ui primitives (button, input, dialog, badge)
    ├── fullview/            # Full-page dashboard
    │   ├── components/      # Extracted sub-components (grid, list, search, etc.)
    │   ├── App.tsx
    │   ├── TopNav.tsx
    │   ├── SearchModal.tsx
    │   ├── GuideContent.tsx
    │   ├── LibraryContent.tsx
    │   └── router.ts
    ├── sidepanel/           # Side panel UI
    │   ├── App.tsx
    │   ├── LibraryView.tsx
    │   ├── GuideEditor.tsx
    │   ├── RecordingView.tsx
    │   ├── StepCard.tsx
    │   ├── ExportMenu.tsx
    │   ├── BlurCanvas.tsx
    │   └── ZoomScreenshot.tsx
    ├── onboarding/          # Onboarding wizard UI
    │   └── App.tsx          # 5-step wizard (welcome, AI, blur, pin, done)
    ├── shared/              # Shared UI components
    │   └── SettingsView.tsx  # AI settings (provider, model, API key)
    └── options/             # Settings page
        └── App.tsx

State Management

Layer	Tool	Purpose
Capture lifecycle	xstate	State machine (IDLE ↔ RECORDING) in background service worker
Fullview UI	Zustand	Search modal, guide counts, active guide data
Persistence	Dexie (IndexedDB)	Guides, steps, screenshots, rrweb chunks
Service worker recovery	sessionStorage	xstate machine snapshot persistence
Background → Sidepanel	Port messaging	Real-time state broadcast
Cross-context sync	BroadcastChannel	Guide mutations (star, delete) across sidepanel/fullview

Extension Entry Points

Entry Point	File	Purpose
Background	`entrypoints/background/`	Service worker: xstate actor, message handlers, tab management
Content Script	`entrypoints/content.ts`	Injected into all tabs: CaptureSession, event listeners, rrweb
Side Panel	`entrypoints/sidepanel/`	Recording controls, library, guide editor, settings
Full View	`entrypoints/fullview/`	Dashboard: library browse, guide viewer, Ctrl+K search
Onboarding	`entrypoints/onboarding/`	First-install wizard: AI setup, smart blur, pin extension
Options	`entrypoints/options/`	Settings page (shared SettingsView in centered card)

Messaging

Content Script ←→ Background Service Worker ←→ Sidepanel / Fullview

Extension messages (webext-core, lib/messaging.ts):

getState → current capture state, step count, guide ID
startRecording({url}) → creates guide, returns guideId
stopRecording() → finalizes guide, generates AI title
captureStep({guideId, action, elementMeta, domContext}) → screenshots + creates step
updateInputStep({stepId, description}) → updates typing step description
finalizeInputStep({stepId, elementMeta, domContext}) → final screenshot + AI description
rrwebChunk({guideId, events, timestamp}) → stores DOM recording chunk

Tab messages (content script ↔ background, lib/tab-messages.ts):

PING / START_CAPTURE / STOP_CAPTURE — lifecycle
HIDE_OVERLAY / SHOW_OVERLAY — overlay toggle (fallback, used for non-click captures)
SHOW_NOTIFICATION — "Recording started" overlay
URL_CHANGED / GET_ROUTE — SPA navigation tracking

Capture Pipeline

Start recording:

User clicks "Start Capture" in sidepanel
Background transitions xstate machine IDLE → RECORDING
Creates Guide in IndexedDB, broadcasts START_CAPTURE to all tabs
Content scripts create CaptureSession → CaptureController (event listeners) + rrweb
Shows recording notification overlay on active tab

Capture a click:

Content script's CaptureController detects click via DOM event listener
Click handler pushes async work into PQueue (concurrency: 1)
Queue processes: hides overlay instantly → sends captureStep to background → waits
Background calls captureVisibleTab (overlay is hidden, page hasn't reacted yet)
Saves Screenshot + Step to IndexedDB
Optionally generates AI description from DOM context text (not screenshot)
Returns { stepId } → content script shows overlay → queue processes next event

Capture text input (typing):

Click on text field → CaptureController starts InputSession → captureStep with initial screenshot
Each keystroke → InputSession.update() → updateInputStep (description only, fire-and-forget)
Enter/Escape/focusout → InputSession.finalize() → finalizeInputStep → final screenshot replaces initial + AI description
Result: one step for entire typing interaction

Stop recording:

Background transitions RECORDING → IDLE
Broadcasts STOP_CAPTURE, content scripts flush pending input sessions + rrweb events
Background generates guide title from step descriptions + URLs via AI
Opens fullview dashboard with the guide

DOM Context (AI Input)

Instead of sending screenshots to the AI for step descriptions, Mimik extracts a lightweight DOM context (~50-100 tokens) from around the target element:

Page: "Public profile - Settings" /settings/profile
Container: form "Public profile"
Heading: "Public profile"
Siblings: input "Name", input "Email", textarea "Bio", button "Update profile"
→ Target: button "Update profile" (click)

Extraction walks up from the target element to find:

Page title + URL path
Nearest semantic container (form, nav, dialog, section) or 3 levels up
Nearest heading
Sibling interactive elements in the same container (max 10)

Export Formats

Format	Generator	Details
HTML	`core/export/html-export.ts`	Self-contained, base64 images, inline CSS
PDF	`core/export/pdf-export.ts`	jsPDF, A4 portrait, auto page breaks
Markdown	`core/export/markdown-export.ts`	Standard MD with base64 image data URLs

Tech Stack

Layer	Technology
Extension framework	WXT (Manifest V3)
Language	TypeScript
UI	React 19
Styling	Tailwind CSS v4
Components	shadcn/ui
State (capture)	xstate
State (UI)	Zustand
Storage	Dexie.js (IndexedDB)
Messaging	webext-core
Session recording	rrweb
Export	jsPDF, client-side HTML/Markdown
AI (optional)	Vercel AI SDK (`ai`, `@ai-sdk/openai`, `@ai-sdk/anthropic`)
Event queue	p-queue (concurrency: 1)
Icons	Lucide React
Dates	dayjs
DOM utils	css-selector-generator

Design System

All colors are defined as CSS variables in src/ui/global.css and used via Tailwind classes:

Token	Color	Usage
`--color-foreground`	`#1E1B4B`	Primary text (deep navy)
`--color-muted-foreground`	`#6B7280`	Secondary text
`--color-border`	`#C7D2FE`	Borders, dividers (lavender)
`--color-secondary`	`#EEF2FF`	Light wash backgrounds
`--color-accent`	`#4F46E5`	Accent / interactive (indigo)
`--color-primary`	`#1E1B4B`	Dark backgrounds, badges
`--color-primary-foreground`	`#C7D2FE`	Text on dark backgrounds
`--color-lavender`	`#C7D2FE`	Soft accent
`--color-purple`	`#4F46E5`	Primary indigo
`--color-deep`	`#1E1B4B`	Deepest navy
`--color-violet`	`#38BDF8`	Sky blue accent
`--color-success`	`#059669`	Success green

Font: Poppins (loaded via @fontsource/poppins).

Key Technical Details

Async event queue in content script (PQueue, concurrency 1) serializes capture work — each action awaits the full background round-trip before the next starts
Overlay hidden from content script side (instant display:none) before sending capture message — no round-trip delay for overlay toggle
Input session aggregates all typing on a field into one step — click creates it, keystrokes update description, finalize takes final screenshot
DOM context sent as text to AI instead of screenshots — 15-30x cheaper per step
Highlight overlay uses a custom web component (<mimik-highlight>) with closed Shadow DOM at max z-index
Content script injection pings first, falls back to chrome.scripting.executeScript() for tabs without the script
xstate snapshot persisted to sessionStorage so the state machine survives service worker restarts
Recording notification uses animationend event (not hardcoded delays) for timing
Font loading uses @fontsource/poppins (CSP-safe, no CDN dependency)
Cross-context sync via BroadcastChannel — star/delete events update other views without full reload

12 KiB Raw Blame History