Skip to content
BranchesTags

Latest commit

 

History

2 Commits

Folders and files

NameName
Last commit message
Last commit date
client
client
 
 
server
server
 
 
.env.example
.env.example
 
 
.gitattributes
.gitattributes
 
 
.gitignore
.gitignore
 
 
README.md
README.md
 
 
package-lock.json
package-lock.json
 
 
package.json
package.json
 
 

Repository files navigation

AI Comic Book Generator

An AI-assisted comic production studio for turning game character assets, lore, and styling rules into a multi-page comic book. The app combines a React wizard, an Express API server, local asset storage, Google Gemini text agents, and an image-generation endpoint that can target Gemini/Imagen-style models.

The system is intentionally hybrid:

  • Deterministic code handles asset scanning, profile persistence, API routing, saved volumes, layout, copy controls, and fallback prompt generation.
  • AI agents handle story writing, script critique, art direction, page critique, and cover concept generation.
  • The UI keeps a human in the loop through page-by-page approval, tweak requests, final prompt copying, and export/print support.

Project Structure

Comic Book Agent/
|-- client/
|   |-- src/
|   |   |-- App.jsx
|   |   |-- main.jsx
|   |   |-- index.css
|   |   |-- components/
|   |   |   |-- Phase1AssetDiscovery.jsx
|   |   |   |-- Phase2NarrativeScoping.jsx
|   |   |   |-- Phase3StoryGeneration.jsx
|   |   |   |-- Phase4Production.jsx
|   |   |   `-- ComicViewer.jsx
|   |   `-- utils/
|   |       `-- promptCompiler.js
|   |-- vite.config.js
|   `-- package.json
|
|-- server/
|   |-- server.js
|   |-- agents/
|   |   `-- index.js
|   |-- routes/
|   |   |-- aiRoutes.js
|   |   `-- dataRoutes.js
|   `-- utils/
|       `-- storage.js
|
|-- game_assets/
|   |-- uploads/
|   `-- data/
|       |-- characters.json
|       |-- games.json
|       `-- stories.json
|
|-- .env
|-- .env.example
|-- package.json
`-- README.md

Runtime Architecture

flowchart LR
  User["User"] --> Client["React client\nVite dev server :3000"]
  Client --> Proxy["Vite proxy\n/api and /game_assets"]
  Proxy --> Server["Express server\n:5000"]
  Server --> DataRoutes["dataRoutes.js\nlocal asset + JSON APIs"]
  Server --> AIRoutes["aiRoutes.js\nAI orchestration APIs"]
  DataRoutes --> Storage["game_assets/\nuploads + data JSON"]
  AIRoutes --> Agents["agents/index.js\nGemini text agents"]
  AIRoutes --> ImageGen["/api/image/generate\nimage model call"]
  Agents --> Gemini["Gemini Developer API\ngenerativelanguage.googleapis.com"]
  ImageGen --> ImageAPI["Imagen/Gemini image endpoint\nconfigured by IMAGE_GEN_MODEL"]
  Client --> Assets["Static assets\n/game_assets/*"]
  Assets --> Storage
Loading

Client Flow

The frontend is a four-step wizard controlled by client/src/App.jsx. All step state is held in buildData and passed down to the phase components.

flowchart TD
  A["Phase 1\nAsset Ingestion"] --> B["Phase 2\nLore and Scoping"]
  B --> C["Phase 3\nStory Drafting"]
  C --> D["Phase 4\nArt Production"]
  D --> E["ComicViewer\nRead, copy prompts, print"]
Loading

Phase 1: Asset Ingestion

Component: client/src/components/Phase1AssetDiscovery.jsx

Responsibilities:

  • Search local assets in game_assets/.
  • Upload character images to game_assets/uploads/.
  • Save character profiles.
  • Add selected characters to buildData.characters.

Relevant API calls:

Method Path Purpose
POST /api/scan Find local image files matching requested character names.
POST /api/search-assets Search local assets by query.
POST /api/upload Upload one character image using multipart form data.
GET /api/characters Load saved character profiles.
POST /api/characters Save or update one character profile.

Phase 2: Narrative Scoping

Component: client/src/components/Phase2NarrativeScoping.jsx

Responsibilities:

  • Capture story theme, plot guidelines, tone, brand constraints, panel density, lighting style, and evaluation preference.
  • Capture game profile and previous-volume context.
  • Store all values in buildData.

Relevant API calls:

Method Path Purpose
GET /api/games Load saved game profiles.
POST /api/games Save or update a game profile.
GET /api/stories Load previous saved volumes for continuity.

Phase 3: Story Drafting

Component: client/src/components/Phase3StoryGeneration.jsx

Responsibilities:

  • Compile buildData into a structured XML-like payload using client/src/utils/promptCompiler.js.
  • Send the payload to the Story Agent.
  • Display and allow revision of the generated script.
  • Store the accepted script in buildData.comicScript.

Relevant API calls:

Method Path Purpose
POST /api/story/generate Generate or revise a comic script.

Phase 4: Art Production

Component: client/src/components/Phase4Production.jsx

Responsibilities:

  • Ask the Art Direction Agent for panel prompts.
  • Evaluate pages with the Page Critic.
  • Render page previews.
  • Generate cover concept.
  • Save the completed volume.
  • Pass script, prompts, cover, style, and character assets into ComicViewer.

Relevant API calls:

Method Path Purpose
POST /api/art-direction Generate per-panel image prompts.
POST /api/page/evaluate Critique a page and optionally regenerate prompts.
POST /api/story/cover Generate cover concept and cover image prompt.
POST /api/stories Save completed volume.

ComicViewer

Component: client/src/components/ComicViewer.jsx

Responsibilities:

  • Render the comic page layout.
  • Build the exact final prompt for each panel image request.
  • Render image backgrounds through /api/image/generate.
  • Overlay local character art in default mode.
  • Copy final panel prompts to the clipboard.
  • Render print/PDF layout.

Panel image prompt rules:

Mode Final prompt shape
Default overlay mode First 500 characters of the art-direction prompt plus comic book background environment no characters.
Immersive mode First 700 characters of the art-direction prompt plus featured character names and unified-panel instructions.

Cover prompt rule:

first 500 characters of coverDesign.imagePrompt + " comic book art style"

Server Flow

The Express server starts in server/server.js.

Server responsibilities:

  • Load .env from the project root.
  • Serve /game_assets statically.
  • Mount data routes at /api.
  • Mount AI routes at /api.
  • Serve client/dist in production if the build output exists.
flowchart TD
  Start["server/server.js"] --> Env["Load root .env"]
  Env --> Middleware["CORS + JSON middleware"]
  Middleware --> StaticAssets["Serve /game_assets"]
  StaticAssets --> DataAPI["Mount dataRoutes"]
  DataAPI --> AIAPI["Mount aiRoutes"]
  AIAPI --> Dist{"client/dist exists?"}
  Dist -- yes --> ServeClient["Serve production client"]
  Dist -- no --> Listen["Listen on PORT"]
  ServeClient --> Listen
Loading

API Reference

Data APIs

Defined in server/routes/dataRoutes.js.

Method Path Request Body Response
POST /api/scan { "characters": ["Mage", "Archer"] } { success, latencyMs, assets }
POST /api/search-assets { "query": "mage" } { success, results }
POST /api/upload Multipart field characterImage, plus characterName { success, characterName, url, fileName, source }
GET /api/characters none { success, characters }
GET /api/characters/:name none { success, character }
POST /api/characters { "character": { ... } } { success, character }
GET /api/games none { success, games }
GET /api/games/:name none { success, game }
POST /api/games { "game": { ... } } { success, game }
GET /api/stories none { success, stories }
POST /api/stories { "story": { ... } } { success, volumeNumber }

AI APIs

Defined in server/routes/aiRoutes.js.

Method Path Request Body / Query Primary Agent or Model
POST /api/story/generate { payload, revisionPrompt? } Story Agent, then Story Critic on first drafts
POST /api/art-direction { script, styling } Art Direction Agent, deterministic fallback on transient failure
POST /api/page/evaluate { page, prompts, styling } Page Critic, then Art Direction Agent for prompt retry
POST /api/story/cover { script, characters, styling, gameProfile } Cover Designer Agent
GET /api/image/generate?prompt=... Query string prompt IMAGE_GEN_MODEL

Model Call Flow

Story Generation

sequenceDiagram
  participant C as Client Phase3
  participant API as POST /api/story/generate
  participant Story as Story Agent
  participant Judge as Story Critic
  participant G as Gemini API

  C->>API: payload + optional revisionPrompt
  API->>API: Parse payload and build prioritized prompt
  API->>Story: run(task, input)
  Story->>G: generateContent
  G-->>Story: strict JSON script text
  Story-->>API: output
  API->>API: parse script JSON
  alt first draft and critic available
    API->>Judge: evaluate script
    Judge->>G: generateContent
    G-->>Judge: JSON scorecard
    Judge-->>API: pass/fail + feedback
    alt failed and retries remain
      API->>Story: regenerate with critic feedback
      Story->>G: generateContent
      G-->>Story: corrected script JSON
    end
  end
  API-->>C: script + eval score + feedback
Loading

Art Direction

sequenceDiagram
  participant C as Client Phase4
  participant API as POST /api/art-direction
  participant Art as Art Direction Agent
  participant G as Gemini API

  C->>API: script + styling
  alt live agent available
    API->>Art: translate panels to prompts
    Art->>G: generateContent
    alt success
      G-->>Art: JSON panelPrompts
      Art-->>API: output
      API-->>C: mode=production, panelPrompts
    else transient 429/500/502/503
      API->>API: buildDeterministicPanelPrompts
      API-->>C: mode=fallback, warning, panelPrompts
    end
  else demo mode
    API->>API: buildDeterministicPanelPrompts
    API-->>C: mode=simulation, panelPrompts
  end
Loading

Page Evaluation

sequenceDiagram
  participant C as Client Phase4
  participant API as POST /api/page/evaluate
  participant Critic as Page Critic Agent
  participant Art as Art Direction Agent
  participant G as Gemini API

  C->>API: page + prompts + styling
  API->>Critic: evaluate visual translation
  Critic->>G: generateContent
  G-->>Critic: JSON scorecard
  Critic-->>API: pass/fail + feedback
  alt failed and retries remain
    API->>Art: regenerate prompts with corrective feedback
    Art->>G: generateContent
    G-->>Art: corrected panelPrompts
  end
  API-->>C: pass, overall, scores, feedback, prompts
Loading

Image Generation

sequenceDiagram
  participant V as ComicViewer
  participant API as GET /api/image/generate
  participant IMG as Image Model API

  V->>V: Build exact final prompt
  V->>API: prompt query string
  API->>API: Read IMAGE_GEN_MODEL
  alt model name contains "imagen"
    API->>IMG: REST predict call
    IMG-->>API: bytesBase64 image
  else gemini image model
    API->>IMG: generateContent with responseModalities TEXT,IMAGE
    IMG-->>API: inlineData image part
  end
  API-->>V: image bytes with Content-Type
Loading

Agents

All text agents live in server/agents/index.js.

Agent Env var Role
Story Generation Agent STORY_AGENT_MODEL Writes the structured comic script.
Story Critic Agent JUDGE_AGENT_MODEL Scores story tone, flow, and character coverage.
Page Critic Agent CRITIC_AGENT_MODEL Scores visual prompt composition, style, and fidelity.
Art Direction Agent ART_AGENT_MODEL Converts script panels into image prompts.
Cover Designer Agent COVER_AGENT_MODEL Creates cover composition and cover prompt.

The shared Agent.run() method retries transient Gemini errors:

  • 429
  • 500
  • 502
  • 503
  • messages containing high-demand / temporary retry language

Configure retry count with:

GEMINI_MAX_RETRIES=3

Environment Setup

Create .env in the project root:

PORT=5000

# Gemini Developer API key from Google AI Studio.
# This code path calls generativelanguage.googleapis.com.
GEMINI_API_KEY=your_gemini_api_key_here

# Text agents
STORY_AGENT_MODEL=gemini-3.5-flash
ART_AGENT_MODEL=gemini-2.5-flash-lite
COVER_AGENT_MODEL=gemini-2.5-flash-lite
JUDGE_AGENT_MODEL=gemini-3.1-flash-lite
CRITIC_AGENT_MODEL=gemini-3.1-flash-lite

# Shared fallback if an agent-specific model is missing
GEMINI_MODEL=gemini-2.5-flash
GEMINI_MAX_RETRIES=3

# Image generation model.
# Imagen models use REST predict.
# Gemini image models use generateContent with image response modalities.
IMAGE_GEN_MODEL=imagen-4.0-fast-generate-001

Important API key note:

  • The current server uses the Gemini Developer API host: generativelanguage.googleapis.com.
  • A Vertex AI key or service account setup targets aiplatform.googleapis.com and is not automatically compatible with the current @google/generative-ai code path.
  • If you see API_KEY_SERVICE_BLOCKED for generativelanguage.googleapis.com, either use an AI Studio key that is allowed to call the Gemini Developer API or migrate the server to Vertex AI SDK/API calls.

Important image-model note:

  • Google model availability and free-tier access change over time.
  • If IMAGE_GEN_MODEL points to an Imagen model, the server sends a REST predict request to generativelanguage.googleapis.com/v1beta/models/<model>:predict.
  • The image route returns the active model in the X-Image-Model response header and includes model in JSON errors.

Installation and Running

Prerequisites:

  • Node.js 18+
  • npm
  • Google AI Studio API key for the current Gemini Developer API integration

Install dependencies:

npm install

Create assets folders if needed:

mkdir -p game_assets/uploads game_assets/data

Start both client and server:

npm run dev

Open:

http://localhost:3000

The Vite dev server proxies /api and /game_assets to:

http://localhost:5000

Local Storage Layout

game_assets/
|-- uploads/
|   `-- characterImage-<timestamp>-<random>.png
`-- data/
    |-- characters.json
    |-- games.json
    `-- stories.json

Storage behavior:

  • Uploads are stored through Multer in game_assets/uploads.
  • Character, game, and story data are stored as local JSON files.
  • Missing JSON files are treated as empty arrays.
  • JSON write operations overwrite the full file.

Prompt Lifecycle

flowchart TD
  A["User inputs\ncharacters, lore, style"] --> B["compilePayload()\nXML-like story payload"]
  B --> C["Story Agent prompt"]
  C --> D["Comic script JSON"]
  D --> E["Art Direction Agent prompt"]
  E --> F["Panel imagePrompt values"]
  F --> G{"ComicViewer mode"}
  G -- "Default overlay" --> H["Final prompt:\nimagePrompt + background/no characters suffix"]
  G -- "Immersive" --> I["Final prompt:\nimagePrompt + character names + unified panel instructions"]
  H --> J["/api/image/generate"]
  I --> J
  J --> K["Generated image bytes"]
Loading

Error Handling and Fallbacks

Area Failure Behavior
Missing GEMINI_API_KEY No live Gemini key Server enters simulation mode for agent routes.
Story agent failure Invalid model, blocked API, quota, malformed output /api/story/generate returns 500.
Story critic failure Judge unavailable Server logs the judge error and returns the script rather than blocking the user.
Art direction transient failure 429, 500, 502, 503, high demand Server returns deterministic fallback prompts with mode: "fallback".
Page critic unavailable in demo mode No critic agent Server returns simulated pass scores.
Image generation failure Model unavailable, quota, no image bytes /api/image/generate returns 500 JSON with active model.

Development Notes

Useful commands:

npm run dev
npm run build --workspace=client
npm start --workspace=server

Syntax-check server files:

node --check server/server.js
node --check server/agents/index.js
node --check server/routes/aiRoutes.js
node --check server/routes/dataRoutes.js
node --check server/utils/storage.js

On Windows PowerShell, if npm is blocked by execution policy, use:

npm.cmd run build --workspace=client

Security Notes

  • Do not commit .env.
  • Rotate any Gemini or Google Cloud key that has been pasted into logs, screenshots, chats, or committed history.
  • Restrict API keys to only the APIs they need.
  • The current app is designed for local development and does not include authentication. Add auth before exposing it to a network.
  • Uploaded files are written to local disk. For production, add stricter file validation, size limits, and storage isolation.

Tech Stack

Layer Technology
Frontend React 18, Vite, Tailwind CSS, Lucide Icons
Backend Node.js, Express
Text AI Google Gemini via @google/generative-ai
Image AI Configured by IMAGE_GEN_MODEL in /api/image/generate
Uploads Multer
Persistence Local JSON files
Workspace runner npm workspaces + concurrently

About

An AI-powered tool that transforms game characters into visually rich comics with compelling narratives tailored to their personalities and abilities.

Resources

Readme
Activity

Stars

0 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases

Packages

Contributors

Languages