Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 21 additions & 0 deletions integrations/exa/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
# Exa + Smallest AI

Integrations combining Exa semantic web search with Smallest AI's voice and audio APIs.

## Projects

### Saturn — AI Meeting Copilot

Real-time meeting intelligence app that transcribes audio with Smallest AI Pulse STT, detects questions in conversation, and automatically searches the web via Exa to surface relevant answers — all summarized by Claude in a live side panel.

**Stack:** Next.js, Smallest AI Pulse STT, Exa Search, Anthropic Claude

**Features:**
- Live transcription via Pulse STT
- Auto question detection → Exa semantic search
- AI-synthesized answers in real-time
- Push-to-talk manual search
- Meeting summary generation
- Google Meet Chrome extension

[View project →](../../speech-to-text/saturn-meeting-copilot)
18 changes: 18 additions & 0 deletions speech-to-text/saturn-meeting-copilot/.env.example
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
# Smallest.ai — Speech-to-Text
# Get your key at: https://waves.smallest.ai
SMALLEST_API_KEY=your_key_here

# Exa — Semantic web search
# Get your key at: https://exa.ai
EXA_API_KEY=your_key_here

# Anthropic — Claude for AI summaries and insights
# Get your key at: https://console.anthropic.com
ANTHROPIC_API_KEY=your_key_here

# App config (leave as-is for local development)
NEXT_PUBLIC_APP_URL=http://localhost:3000

# Optional: shared secret to authenticate extension → server transcript pushes.
# Set this and enter the same value in the extension popup's "Push token" field.
SATURN_PUSH_TOKEN=
6 changes: 6 additions & 0 deletions speech-to-text/saturn-meeting-copilot/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
.env
.env.local
node_modules/
.next/
.turbo/
tsconfig.tsbuildinfo
145 changes: 145 additions & 0 deletions speech-to-text/saturn-meeting-copilot/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,145 @@
# Saturn — AI Meeting Intelligence

Saturn is a real-time AI meeting copilot. It transcribes your calls, detects questions, searches the web for answers, and generates a full meeting summary with action items when you're done.

---

## What it does

- **Live transcription** — captures your mic via Smallest.ai (falls back to browser speech if needed)
- **Google Meet integration** — Chrome extension reads live captions or tab audio from any Meet call
- **Auto research** — detects questions in conversation and searches Exa for answers in real time
- **AI insights** — Claude synthesizes clean answers shown in a side panel
- **Push-to-talk search** — hold `Tab` and speak to search anything directly
- **Meeting summary** — Claude writes a full summary, decisions, and action items when the meeting ends

---

## Setup

### 1. Clone and install

```bash
git clone <repo-url>
cd Saturn
npm install
```

### 2. Add your API keys

Create a `.env.local` file in the project root:

```bash
cp .env.example .env.local
```

Then open `.env.local` and fill in your keys:

```env
# Smallest.ai — Speech-to-Text (required for mic transcription)
# Get your key at: https://waves.smallest.ai
SMALLEST_API_KEY=your_key_here

# Exa — Semantic web search (required for AI research)
# Get your key at: https://exa.ai
EXA_API_KEY=your_key_here

# Anthropic — Claude for summarization and meeting notes (required)
# Get your key at: https://console.anthropic.com
ANTHROPIC_API_KEY=your_key_here

# App config (leave as-is for local dev)
NEXT_PUBLIC_APP_URL=http://localhost:3000
```

| Key | Where to get it | Used for |
|-----|----------------|----------|
| `SMALLEST_API_KEY` | [waves.smallest.ai](https://waves.smallest.ai) | Mic → text transcription |
| `EXA_API_KEY` | [exa.ai](https://exa.ai) | Web search for questions |
| `ANTHROPIC_API_KEY` | [console.anthropic.com](https://console.anthropic.com) | AI summaries & insights |

### 3. Start the app

```bash
npm run dev
```

Open [http://localhost:3000](http://localhost:3000) in your browser.

---

## Google Meet Extension (optional but recommended)

The Chrome extension lets Saturn hear **all participants** in a Google Meet call — not just your mic.

### Install

1. Open Chrome and go to `chrome://extensions`
2. Enable **Developer mode** (top right toggle)
3. Click **Load unpacked**
4. Select the `extensions/google-meet/` folder from this repo

### Use

1. Join a Google Meet call
2. Click the **Saturn** icon in your Chrome toolbar
3. Click **Side Panel ⊞** — Saturn opens as a sidebar alongside Meet
4. Enable **CC (captions)** in Google Meet for best results (all participants captured)
5. Saturn auto-starts when it hears the first word

> **Tip:** Enabling Google Meet's live captions (CC button in the Meet toolbar) switches the extension to caption mode, which captures every participant and doesn't need Smallest.ai at all.

---

## How to use

| Action | How |
|--------|-----|
| Start a meeting | Click **Start Meeting** on the landing page |
| Ask a question | Just speak — Saturn detects `?` or question words and auto-researches |
| Manual search | Hold `Tab` + speak → release to search |
| View insights | AI Insights panel on the right (or full-width in the side panel) |
| End meeting | Click **Stop** — Claude generates your meeting notes automatically |

---

## Project structure

```
Saturn/
├── app/
│ ├── api/
│ │ ├── research/ # Exa search + Claude summarization
│ │ ├── summary/ # End-of-meeting Claude summary
│ │ ├── transcribe/ # Smallest.ai STT endpoint
│ │ └── transcript/ # SSE stream + push endpoint
│ └── page.tsx # Main app page
├── components/
│ ├── controls/ # BottomBar with push-to-talk
│ ├── insights/ # AI Insights panel + cards
│ └── transcript/ # Live transcript panel
├── extensions/
│ └── google-meet/ # Chrome extension (MV3)
├── hooks/
│ ├── useExaBot.ts # Auto question detection + research
│ ├── useDirectSearch.ts # Push-to-talk Tab search
│ └── useGoogleMeetTranscript.ts # SSE + mic capture
├── services/
│ └── researchAgent.ts # Question detection logic
└── store/
└── meetingStore.ts # Zustand state
```

---

## Requirements

- Node.js 18+
- Chrome (for the extension)
- Microphone access

---

## Production Notes

This demo uses open CORS and no auth on API routes (required for the Chrome extension on localhost). For production, add authentication, restrict CORS origins, and update `SATURN_URL` in the extension.
109 changes: 109 additions & 0 deletions speech-to-text/saturn-meeting-copilot/app/api/research/route.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,109 @@
/**
* POST /api/research
* Runs a research job: searches Exa and summarizes with OpenAI.
*
* Body: { query: string, segmentId?: string }
* Returns: Insight object
*/

import { NextRequest, NextResponse } from "next/server";
import Anthropic from "@anthropic-ai/sdk";
import { runResearch, buildInsight } from "@/services/researchAgent";

const anthropic = new Anthropic();

async function classifyResearchIntent(text: string): Promise<string | null> {
const message = await anthropic.messages.create({
model: "claude-opus-4-6",
max_tokens: 80,
messages: [
{
role: "user",
content:
`You are classifying transcript utterances for web research triggering.\n` +
`Input sentence: "${text}"\n\n` +
`Rules:\n` +
`1) Trigger only if this is an actual information-seeking question.\n` +
`2) Do NOT trigger for conversational/meta prompts like: "can I ask you something", "are you there", greetings, confirmations.\n` +
`3) Do NOT trigger if the sentence is not a question.\n` +
`4) If triggering, return a concise cleaned query.\n\n` +
`Respond in EXACTLY one of these formats:\n` +
`TRIGGER|<clean query>\n` +
`SKIP`,
},
],
});

const textOut =
message.content.find((b): b is Anthropic.TextBlock => b.type === "text")?.text?.trim() ??
"";

if (!textOut.toUpperCase().startsWith("TRIGGER|")) return null;

const query = textOut.slice("TRIGGER|".length).trim();
return query.length > 0 ? query : null;
}

async function summarizeWithClaude(query: string, snippets: string[]): Promise<string[]> {
const context = snippets.join("\n\n---\n\n");
const message = await anthropic.messages.create({
model: "claude-opus-4-6",
max_tokens: 256,
messages: [
{
role: "user",
content: `Query: "${query}"\n\nWeb results:\n${context}\n\nRespond in this exact format:\nLINE1: <ultra-short direct answer, 1-6 words max, e.g. "Pittsburgh, Pennsylvania" or "42 million trillion stars">\nLINE2: <one sentence of key context>\nLINE3: <one more interesting detail sentence>\n\nNo headers, no bullets, no labels — just the 3 plain lines.`,
},
],
});
const text = message.content.find((b): b is Anthropic.TextBlock => b.type === "text")?.text ?? "";
return text
.split("\n")
.map((l) => l.replace(/^LINE\d:\s*/i, "").replace(/^[•\-*]\s*/, "").trim())
.filter((l) => l.length > 2)
.slice(0, 3);
}

export async function POST(req: NextRequest) {
try {
const { query, segmentId, numResults, detectOnly } = await req.json();

if (!query || typeof query !== "string") {
return NextResponse.json({ error: "query is required" }, { status: 400 });
}

if (detectOnly) {
const classified = await classifyResearchIntent(query.trim());
return NextResponse.json({
shouldResearch: Boolean(classified),
query: classified,
});
}

const result = await runResearch(query.trim(), typeof numResults === "number" ? Math.min(10, Math.max(1, numResults)) : 5);

// Summarize raw snippets with Claude (server-side only)
try {
// Truncate each snippet to avoid overloading Claude's context
const trimmed = result.bullets.map((s) => s.slice(0, 500));
result.bullets = await summarizeWithClaude(query, trimmed);
} catch (err) {
console.error("[/api/research] Claude summarization failed:", err);
// Show a brief fallback instead of the raw wall of text
result.bullets = [result.bullets[0]?.slice(0, 120) ?? "No summary available."];
}

const insight = buildInsight(result, query, segmentId);

return NextResponse.json({
insight: {
...insight,
id: `insight-${Date.now()}`,
},
});
} catch (err) {
const message = err instanceof Error ? err.message : "Research failed";
console.error("[/api/research]", message);
return NextResponse.json({ error: message }, { status: 500 });
}
}
Loading