diff --git a/.cursor/plans/gradio_sdk_deploy_58daaf6e.plan.md b/.cursor/plans/gradio_sdk_deploy_58daaf6e.plan.md new file mode 100644 index 0000000000000000000000000000000000000000..c4f052db188c1c1469e6cbc7e52ae82437df3999 --- /dev/null +++ b/.cursor/plans/gradio_sdk_deploy_58daaf6e.plan.md @@ -0,0 +1,268 @@ +--- +name: Gradio SDK Deploy +overview: "Add Gradio SDK deployment files on main alongside the existing Dockerfile, switch README to `sdk: gradio` for ZeroGPU Spaces, and add `@spaces.GPU` wrappers on LLM entry points so the full Studio + Classic app runs on HF without Docker." +todos: + - id: root-gradio-files + content: Add root app.py, requirements.txt, packages.txt with editable workspace installs and Debian deps + status: completed + - id: readme-gradio-sdk + content: "Fix README YAML frontmatter and switch to sdk: gradio (sdk_version 6.16.0, app_file: app.py)" + status: completed + - id: spaces-runtime + content: Add gradio_space/spaces_runtime.py with gpu_task decorator and is_hf_gradio_runtime() + status: completed + - id: zerogpu-decorators + content: Apply @gpu_task to LLM entry points in model_loading, research_helpers, and tab handlers; skip preload on HF Gradio runtime in server.py + status: completed + - id: usage-docs + content: Update USAGE.md with Gradio SDK + ZeroGPU deploy steps; demote Docker section to later phase + status: completed + - id: local-smoke + content: Validate pip install + python app.py locally before pushing to HF Space + status: completed + - id: hf-space-create + content: Create Gradio Space under build-small-hackathon with ZeroGPU hardware and env vars; verify Studio + Classic smoke tests + status: cancelled +isProject: false +--- + +# Gradio SDK + ZeroGPU deployment (same branch as Docker) + +## Goal + +Ship the **full app** (Studio at `/`, Classic at `/classic`, all tabs) via **Gradio SDK** on Hugging Face with **ZeroGPU**, while keeping [`Dockerfile`](Dockerfile) on `main` untouched for a later Docker Space phase. + +## Same-branch constraint (important) + +HF reads **one** `sdk:` value from root [`README.md`](README.md). Both deploy paths can live on the same branch as files, but **only one SDK is active per branch at a time**: + +| Files on `main` | Active when README says | +|-----------------|-------------------------| +| `app.py`, `requirements.txt`, `packages.txt` | `sdk: gradio` | +| [`Dockerfile`](Dockerfile) | `sdk: docker` + `app_port: 7860` | + +**Phase 1 (now):** set `sdk: gradio` β€” Gradio Space builds from `app.py`. +**Phase 2 (later):** flip README to `sdk: docker` for Docker Space, or use a **second HF Space on a second branch** if you need both live at once. + +```mermaid +flowchart TB + subgraph repo [main branch] + AppPy[app.py] + ReqTxt[requirements.txt] + DockerFile[Dockerfile] + Shared[apps/gradio-space + libs + skills] + end + subgraph phase1 [Phase 1 active] + ReadmeG[sdk: gradio in README] + HFGradio[HF Gradio SDK build] + ZeroGPU[ZeroGPU hardware] + end + subgraph phase2 [Phase 2 later] + ReadmeD[sdk: docker in README] + HFDocker[HF Docker build] + GPUBasic[GPU Basic hardware] + end + Shared --> AppPy + Shared --> DockerFile + ReadmeG --> HFGradio --> ZeroGPU + ReadmeD --> HFDocker --> GPUBasic +``` + +--- + +## Phase 1 β€” Add Gradio SDK root files + +### 1. Root [`app.py`](app.py) + +Thin entry point that reuses the existing server (no UI rewrite): + +```python +from gradio_space.server import main + +if __name__ == "__main__": + main() +``` + +HF Gradio SDK executes `app.py`; [`server.py`](apps/gradio-space/src/gradio_space/server.py) already calls `server.launch()` on port 7860 with Studio + `/classic`. + +### 2. Root [`requirements.txt`](requirements.txt) + +Pip-install workspace packages via editable paths (HF clones the full repo): + +```text +-e ./libs/inference +-e ./libs/researchmind +-e ./libs/agent +-e ./libs/echocoach[piper,whisper] +-e ./apps/gradio-space +# plus transitive deps from libs/*/pyproject.toml (torch, transformers, sentence-transformers, python-pptx, etc.) +``` + +Rules (per [HF Spaces dependencies](https://huggingface.co/docs/hub/spaces-dependencies)): + +- **Do not pin** `gradio`, `spaces`, or `huggingface_hub` β€” HF preinstalls them. +- Pin heavy libs that matter for reproducibility: `torch`, `transformers`, `accelerate`, `sentence-transformers`, etc. +- Keep `llama-cpp-python` for preset parity (HF image has `cmake`; build may be slow). + +Optional: add [`scripts/sync-requirements.sh`](scripts/sync-requirements.sh) later to regenerate from `pyproject.toml` files β€” not required for v1. + +### 3. Root [`packages.txt`](packages.txt) + +Debian deps beyond HF defaults (mirror [`Dockerfile`](Dockerfile) apt lines): + +```text +ffmpeg +libsndfile1 +``` + +### 4. Fix + switch README frontmatter + +Current [`README.md`](README.md) has a blank line after `---` and still declares Docker. Update to: + +```yaml +--- +title: Lesson Agent +emoji: πŸ“š +colorFrom: blue +colorTo: green +sdk: gradio +sdk_version: "6.16.0" +app_file: app.py +python_version: "3.12" +pinned: false +license: apache-2.0 +--- +``` + +Remove `app_port` (Docker-only). Keep [`Dockerfile`](Dockerfile) in repo for phase 2. + +--- + +## Phase 2 β€” ZeroGPU runtime hooks + +ZeroGPU requires all CUDA work inside `@spaces.GPU`. The decorator is a **no-op** locally and on dedicated GPU Spaces, so it is safe to apply everywhere. + +### New module: [`apps/gradio-space/src/gradio_space/spaces_runtime.py`](apps/gradio-space/src/gradio_space/spaces_runtime.py) + +```python +def gpu_task(*, duration: int = 180, size: str = "large"): + """Apply @spaces.GPU when the HF spaces runtime is present.""" + ... + +def is_hf_gradio_runtime() -> bool: + """True on HF Gradio SDK Spaces (skip startup model preload).""" + ... +``` + +Use `duration=180`–`300` for agent/slide flows; `duration=60` for simple chat. + +### Skip startup preload on HF Gradio runtime + +[`server.py`](apps/gradio-space/src/gradio_space/server.py) currently calls `preload_active_model()` before launch β€” this fails on ZeroGPU (no GPU at process start): + +```69:69:apps/gradio-space/src/gradio_space/server.py + preload_active_model() +``` + +Change to: + +```python +if not is_hf_gradio_runtime(): + preload_active_model() +``` + +First user request lazy-loads inside a `@spaces.GPU`-decorated handler. + +### Decorate LLM entry points (not every `backend.chat` call) + +Wrap **top-level handlers** so multi-step agent loops run inside one GPU allocation: + +| Module | Functions to decorate | +|--------|----------------------| +| [`model_loading.py`](apps/gradio-space/src/gradio_space/model_loading.py) | `chat`, `reload_model` | +| [`research_helpers.py`](apps/gradio-space/src/gradio_space/research_helpers.py) | `run_research_question`, `rag_aware_chat` | +| [`tabs/education_pptx.py`](apps/gradio-space/src/gradio_space/tabs/education_pptx.py) | `generate_lesson_slides`, `discover_lesson_sources` | +| [`tabs/research_mind.py`](apps/gradio-space/src/gradio_space/tabs/research_mind.py) | `discover_sources`, `ask_question`, `auto_search_ingest` | +| [`tabs/echo_coach.py`](apps/gradio-space/src/gradio_space/tabs/echo_coach.py) | `analyze_pitch` | +| [`tabs/teacher_voice.py`](apps/gradio-space/src/gradio_space/tabs/teacher_voice.py) | text/audio turn handlers | + +Studio APIs in [`api/studio.py`](apps/gradio-space/src/gradio_space/api/studio.py) call these helpers β€” decorating the tab/helper layer avoids duplicating decorators on ~20 API wrappers. + +**Generator caveat:** `generate_lesson_slides` uses `yield` for progress. If ZeroGPU rejects generator handlers, extract GPU work into a plain `@gpu_task` function and keep the outer generator for UI progress only (test on Space Logs after first deploy). + +**Embeddings (ResearchMind ingest):** sentence-transformers can stay on CPU for v1; only LLM paths need `@spaces.GPU` initially. + +--- + +## Phase 3 β€” Space configuration + +Create Space under [build-small-hackathon](https://huggingface.co/build-small-hackathon): + +| Setting | Value | +|---------|-------| +| SDK | **Gradio** (Blank template) | +| Hardware | **ZeroGPU** (creator needs PRO/Team) | +| Repo | GitHub `main` (or push to Space git) | + +**Environment variables** (Settings β†’ Variables): + +| Variable | Value | +|----------|-------| +| `ACTIVE_MODEL` | `minicpm5-1b` | +| `ALLOW_MODEL_SWITCH` | `false` | +| `RESEARCHMIND_DATA_DIR` | `/tmp/researchmind` | + +Default preset in [`models.yaml`](models.yaml) is already `minicpm5-1b` (transformers) β€” good fit for ZeroGPU. + +--- + +## Phase 4 β€” Docs and local smoke test + +Update [`USAGE.md`](USAGE.md): + +- New **Gradio SDK deployment** section (primary path): `app.py`, `requirements.txt`, ZeroGPU, env vars. +- Move existing Docker section to **"Docker SDK (later)"** β€” note README must switch to `sdk: docker` + `app_port: 7860`. +- Local Gradio SDK smoke test: + +```bash +python -m venv .venv && source .venv/bin/activate +pip install -r requirements.txt +ACTIVE_MODEL=minicpm5-1b ALLOW_MODEL_SWITCH=false python app.py +``` + +Keep existing `uv run` workflow for day-to-day dev unchanged. + +Update [`.cursor/plans/hf_space_publish_e8a57bab.plan.md`](.cursor/plans/hf_space_publish_e8a57bab.plan.md) todos to reflect Gradio-first ordering. + +--- + +## Phase 5 β€” Verify on Space + +1. **Logs** β€” pip install succeeds; app starts on `0.0.0.0:7860`. +2. **`/` Studio** β€” loads static UI. +3. **`/classic`** β€” all tabs render. +4. **Smoke flows** β€” slides generation, research chat, EchoCoach sample clip, teacher voice text turn. +5. **ZeroGPU** β€” first LLM request allocates GPU (may be slow on cold start); watch for "No CUDA GPUs" (means handler is outside `@spaces.GPU`). + +--- + +## Phase 6 β€” Docker later (no code removal) + +When ready for Docker Space: + +1. Change README to `sdk: docker`, `app_port: 7860` (remove `sdk_version` / `app_file`). +2. Create a **second Space** (or reuse after README flip) with **GPU Basic** hardware. +3. Existing [`Dockerfile`](Dockerfile) + `uv sync` path unchanged; no `@spaces.GPU` needed on dedicated GPU. + +Both file sets remain on `main`; only README `sdk:` toggles which build HF runs. + +--- + +## Risk notes + +| Risk | Mitigation | +|------|------------| +| `pip install llama-cpp-python` slow/fails on HF | Accept slow build; default `minicpm5-1b` avoids GGUF at runtime | +| EchoCoach deps (piper, whisper) heavy | Full scope requested; pin versions; fix from Space Logs if needed | +| ZeroGPU + generator slide progress | Refactor GPU block to non-generator helper if build succeeds but inference fails | +| Two live Spaces same branch | Not supported with different SDKs β€” use README flip or second branch for concurrent Docker + Gradio | diff --git a/.cursor/plans/hf_space_publish_e8a57bab.plan.md b/.cursor/plans/hf_space_publish_e8a57bab.plan.md new file mode 100644 index 0000000000000000000000000000000000000000..c98475533a2514e06cf19e2a58bb4300aca19324 --- /dev/null +++ b/.cursor/plans/hf_space_publish_e8a57bab.plan.md @@ -0,0 +1,208 @@ +--- +name: HF Space Publish +overview: Fix two repo blockers (README Space card YAML and missing `researchmind` in Dockerfile), validate locally with Docker, push to GitHub, then create a Docker Space under build-small-hackathon linked to GitHub with GPU hardware and MiniCPM5-1B env vars. +todos: + - id: fix-readme-yaml + content: "Fix root README.md frontmatter: change `## title:` to `title:`" + status: completed + - id: fix-dockerfile-researchmind + content: Add libs/researchmind COPY lines to Dockerfile + status: completed + - id: local-docker-smoke + content: Run docker build + docker run locally on port 7860 with ACTIVE_MODEL=minicpm5-1b + status: in_progress + - id: push-github + content: Push fixed branch to GitHub repo + status: pending + - id: create-space + content: Create Docker Space under build-small-hackathon, link GitHub, set GPU basic + status: pending + - id: configure-env + content: "Set Space secrets: ACTIVE_MODEL=minicpm5-1b, ALLOW_MODEL_SWITCH=false, RESEARCHMIND_DATA_DIR=/tmp/researchmind" + status: pending + - id: verify-live + content: Check Space Logs, test / and /classic, confirm slide generation works + status: pending +isProject: false +--- + +# Publish Gradio app to Hugging Face Space + +## Current state + +Your repo is **mostly ready** for a Docker Space: + +- Root [`Dockerfile`](Dockerfile) exposes port **7860** and runs `python -m gradio_space.app` +- Root [`README.md`](README.md) has Space metadata (`sdk: docker`, `app_port: 7860`) +- Default model in [`models.yaml`](models.yaml) is **`minicpm5-1b`** (transformers, `openbmb/MiniCPM5-1B`) + +Two issues will likely **break the Space build or card** until fixed: + +### Blocker 1 β€” README YAML is malformed + +The Space card frontmatter must use `title:`, not a markdown heading: + +```yaml +# Current (wrong) +## title: Lesson Agent + +# Required (correct) +title: Lesson Agent +``` + +HF reads YAML from the **root** [`README.md`](README.md) only. Keep [`apps/gradio-space/README.md`](apps/gradio-space/README.md) as dev docs. + +### Blocker 2 β€” Dockerfile missing `researchmind` + +[`libs/agent`](libs/agent/pyproject.toml) depends on `researchmind`, but the Dockerfile only copies `inference`, `agent`, and `echocoach`. `uv sync` inside the image will fail without: + +```dockerfile +COPY libs/researchmind/pyproject.toml libs/researchmind/README.md libs/researchmind/ +COPY libs/researchmind/src libs/researchmind/src +``` + +Add these lines alongside the other `libs/*` COPY blocks in [`Dockerfile`](Dockerfile). + +--- + +## Architecture (what gets deployed) + +```mermaid +flowchart LR + subgraph hf [HuggingFaceSpace] + DockerBuild[DockerBuild] + Container[Container_port7860] + end + GitHub[GitHub_repo] --> DockerBuild + DockerBuild --> Container + Container --> StudioUI["/ Studio UI"] + Container --> ClassicUI["/classic Gradio tabs"] + Container --> HubModel["Hub: openbmb/MiniCPM5-1B"] + HubModel --> Container +``` + +Entrypoint (unchanged): + +```44:44:Dockerfile +CMD ["uv", "run", "--package", "gradio-space", "python", "-m", "gradio_space.app"] +``` + +This launches [`gradio_space.server`](apps/gradio-space/src/gradio_space/server.py): Studio at `/`, Classic tabs at `/classic`. + +--- + +## Phase 1 β€” Fix repo files (before push) + +| File | Change | +|------|--------| +| [`README.md`](README.md) | Fix frontmatter: `title: Lesson Agent` (remove `##`); keep `sdk: docker`, `app_port: 7860` | +| [`Dockerfile`](Dockerfile) | Add `libs/researchmind` pyproject + src COPY lines | + +Optional but recommended in README frontmatter (already present except title): + +```yaml +--- +title: Lesson Agent +emoji: πŸ“š +colorFrom: blue +colorTo: green +sdk: docker +app_port: 7860 +pinned: false +license: apache-2.0 +--- +``` + +--- + +## Phase 2 β€” Validate locally with Docker + +From repo root: + +```bash +docker build -t hackathon-space . +docker run --rm -p 7860:7860 \ + -e ACTIVE_MODEL=minicpm5-1b \ + -e ALLOW_MODEL_SWITCH=false \ + hackathon-space +``` + +Open [http://localhost:7860](http://localhost:7860) (`/` Studio, `/classic` tabs). First model load downloads weights from Hub β€” expect several minutes on first run. + +If build fails, check Logs for `researchmind` or `uv sync` errors (confirms Blocker 2 fix). + +--- + +## Phase 3 β€” Push to GitHub + +1. Create a GitHub repo (if not already linked) +2. Push `main` with at minimum: + - `Dockerfile`, `README.md`, `pyproject.toml`, `uv.lock` + - `apps/gradio-space/`, `libs/`, `skills/`, `models.yaml`, `voice_models.yaml` + +Do **not** commit `.env`, local `models/*.gguf`, or large artifacts (`.dockerignore` already excludes these). + +--- + +## Phase 4 β€” Create and link the Space + +1. Go to [build-small-hackathon](https://huggingface.co/build-small-hackathon) β†’ **New Space** +2. Settings: + - **Name:** e.g. `lesson-agent` or `small-model-hackathon` + - **SDK:** **Docker** (not Gradio SDK β€” monorepo needs root Dockerfile) + - **Hardware:** **GPU basic** (required for transformers `minicpm5-1b`) +3. Under **Repository** β†’ connect your GitHub repo and branch (`main`) +4. HF will auto-build from root `Dockerfile` on each push + +--- + +## Phase 5 β€” Space environment variables + +In Space **Settings β†’ Variables and secrets** (Repository secrets, not `.env` in git): + +| Variable | Value | Why | +|----------|-------|-----| +| `ACTIVE_MODEL` | `minicpm5-1b` | Pins model for visitors | +| `ALLOW_MODEL_SWITCH` | `false` | Hides dev model dropdown | +| `AGENT_OUTPUTS_DIR` | `/tmp/agent_outputs` | Already set in Dockerfile; optional override | +| `RESEARCHMIND_DATA_DIR` | `/tmp/researchmind` | Ephemeral RAG store on Space (recommended) | + +No secrets required for the default MiniCPM5 preset unless you switch to a gated model. + +--- + +## Phase 6 β€” Verify publish + +1. Open Space **Logs** β€” wait for `Running on local URL: 0.0.0.0:7860` +2. Open the Space URL +3. Smoke test: + - `/` β€” Studio loads + - Generate slides with a simple topic (e.g. "Photosynthesis, grade 8, 5 slides") + - `/classic` β€” tabs render +4. First inference may be slow while `openbmb/MiniCPM5-1B` downloads + +### Optional: faster restarts + +If cold starts are painful, add a **Storage Bucket** in Space settings so Hub model cache persists across restarts. + +--- + +## Troubleshooting + +| Symptom | Fix | +|---------|-----| +| Space card shows wrong title / no Docker | Fix README YAML (`title:` not `## title:`) | +| Docker build fails at `uv sync` | Add `researchmind` to Dockerfile | +| Build OK but app crashes on Research tab | Confirm `researchmind` src is copied | +| First request very slow | Normal β€” model download; use Storage Bucket | +| OOM on GPU | Try smaller batch or switch preset to GGUF on CPU | + +Full reference: [`USAGE.md`](USAGE.md) sections "Docker smoke test" and "Hugging Face Space deployment". + +--- + +## What you do NOT need + +- Plain Gradio SDK (`app.py` + `requirements.txt` at root) β€” wrong fit for this monorepo +- Committing GGUF files β€” models download from Hub at runtime via `ACTIVE_MODEL` / `models.yaml` +- Changing the CMD β€” current entrypoint already serves Studio + Classic diff --git a/.env.example b/.env.example index f24ca4c656e6ce6e26a9427fe25776917c6b6685..36fa66818eb3532926ae5a5dee5e2d78d13e2778 100644 --- a/.env.example +++ b/.env.example @@ -66,14 +66,4 @@ ALLOW_MODEL_SWITCH=false # For Cohere Transcribe ASR: huggingface-cli login + accept model terms, then: # ECHOCOACH_ASR_PRESET=cohere-transcribe -# --- Ensemble research (research/ensemble/) --- -# Base LLM resolution (first match wins): ENSEMBLE_LLM, LLM_PATH, BASE, MODEL_ID, ACTIVE_MODEL -# LLM_PATH=./models/finetuned/minicpm5-1b-lora-merged -# ENSEMBLE_LLM=Qwen/Qwen2.5-0.5B-Instruct -# ENSEMBLE_PRESET=minicpm5-1b -# ENSEMBLE_OUT=./models/ensemble/minicpm5-1b-jepa-pretrain -# ENSEMBLE_QA=./research/data/benchmark-qa.jsonl -# ENSEMBLE_KB=./research/data/benchmark-kb.jsonl -# ENSEMBLE_CKPT=./models/ensemble/jepa-lesson-pretrain - BASE=openbmb/MiniCPM5-1B \ No newline at end of file diff --git a/.gitignore b/.gitignore index 668acb3a13c1061872be775a12856971af8f15f8..68ea945c9ad415967919616ff91a729058c1f1c6 100644 --- a/.gitignore +++ b/.gitignore @@ -1,4 +1,5 @@ .venv/ +.venv-gradio/ __pycache__/ *.py[cod] .env diff --git a/Dockerfile b/Dockerfile index c6e914b46347165c850276f476ea8a11ea628dd6..9d7f8aa09d1f6cb1339c8c85d33b5f56881b7fe8 100644 --- a/Dockerfile +++ b/Dockerfile @@ -20,11 +20,13 @@ COPY apps/gradio-space/pyproject.toml apps/gradio-space/README.md apps/gradio-sp COPY libs/inference/pyproject.toml libs/inference/README.md libs/inference/ COPY libs/agent/pyproject.toml libs/agent/README.md libs/agent/ COPY libs/echocoach/pyproject.toml libs/echocoach/README.md libs/echocoach/ +COPY libs/researchmind/pyproject.toml libs/researchmind/README.md libs/researchmind/ COPY apps/gradio-space/src apps/gradio-space/src COPY apps/gradio-space/static apps/gradio-space/static COPY libs/inference/src libs/inference/src COPY libs/agent/src libs/agent/src COPY libs/echocoach/src libs/echocoach/src +COPY libs/researchmind/src libs/researchmind/src COPY skills skills RUN useradd -m -u 1000 user && \ diff --git a/README.md b/README.md index 4cf8a24fb0f2bae36ba2d3f01e77051959787eed..6e7d769273c719e7d08e80d7618d486bf5cb06af 100644 --- a/README.md +++ b/README.md @@ -1,13 +1,15 @@ --- - -## title: Lesson Agent +title: Lesson Agent emoji: πŸ“š colorFrom: blue colorTo: green -sdk: docker -app_port: 7860 +sdk: gradio +sdk_version: "6.16.0" +app_file: app.py +python_version: "3.12" pinned: false license: apache-2.0 +--- # Lesson Agent @@ -15,7 +17,7 @@ license: apache-2.0 A local skill-based agent helps a teacher you know turn a **topic + grade level** into a downloadable **PowerPoint** β€” powered by a small transformers model (`MiniCPM5-1B` by default), no cloud LLM API. -See **[USAGE.md](USAGE.md)** for local run, Docker smoke test, and HF Space deployment. +See **[USAGE.md](USAGE.md)** for local run, Gradio SDK / ZeroGPU Space deployment, and Docker (later). ## Prerequisites @@ -59,7 +61,7 @@ libs/agent/ # Skill agent runner, tools, trace recorder libs/researchmind/ # Scraper, chunk/embed, MemRAG SQLite store, retrieval libs/inference/ # Transformers + llama.cpp backends skills/ # SKILL.md + references/ + scripts/ per task -research/ # Fine-tune, ensemble experiments, agentic evals (optional) +research/ # Fine-tune and agentic evals (optional) ``` ### ResearchMind (offline after ingest) @@ -87,15 +89,12 @@ See [`.env.example`](.env.example) and [`models.yaml`](models.yaml) for model pr ## Hugging Face Space deployment -1. Create a Space under [build-small-hackathon](https://huggingface.co/build-small-hackathon) with **Docker** SDK. -2. Link this repository (root `Dockerfile` + root `README.md` YAML above). -3. Hardware: **GPU basic** recommended for transformers (`minicpm5-1b`). -4. Optional secrets: `ACTIVE_MODEL`, `N_GPU_LAYERS` (if using GGUF preset). +1. Create a Space under [build-small-hackathon](https://huggingface.co/build-small-hackathon) with **Gradio** SDK (Blank template). +2. Link this repository β€” HF builds from root `app.py` + `requirements.txt` (README YAML above). +3. Hardware: **ZeroGPU** for burst GPU inference, or **GPU basic** for always-on GPU. +4. Set `ACTIVE_MODEL=minicpm5-1b`, `ALLOW_MODEL_SWITCH=false`, `RESEARCHMIND_DATA_DIR=/tmp/researchmind`. -```bash -docker build -t hackathon-space . -docker run --rm -p 7860:7860 -e ACTIVE_MODEL=minicpm5-1b hackathon-space -``` +A root `Dockerfile` is kept for a later **Docker SDK** deploy (flip README to `sdk: docker`). See [USAGE.md](USAGE.md). ## Hackathon checklist diff --git a/USAGE.md b/USAGE.md index 0366297792430079b0ae82800fbf9488bae65ee3..f24f2bb2d4d49d7f49ca973c7e54f5fb321129df 100644 --- a/USAGE.md +++ b/USAGE.md @@ -1,6 +1,6 @@ # Usage -How to run the **Lesson Agent** Gradio app locally, test it in Docker, and deploy to a Hugging Face Space for the [Build Small Hackathon](https://huggingface.co/build-small-hackathon). +How to run the **Lesson Agent** Gradio app locally, deploy to a Hugging Face Space (Gradio SDK + ZeroGPU), and optionally test with Docker later for the [Build Small Hackathon](https://huggingface.co/build-small-hackathon). The primary UI is the **Lesson slides** tab (topic β†’ local model outline β†’ downloadable `.pptx`). Use **ResearchMind** for corpus Q&A, **TeacherVoice** for spoken back-and-forth tutoring, **EchoCoach** for one-shot pitch analysis, or ground lessons directly from the Lesson tab. The **Chat (debug)** tab tests the underlying model. @@ -223,98 +223,121 @@ INFERENCE_BACKEND=transformers MODEL_ID=Qwen/Qwen2.5-3B-Instruct \ --- -## Docker (local prod-like test) +## Gradio SDK local smoke test (matches HF Space build) -Run the same container image HF Spaces will build: +Before pushing to Hugging Face, verify the Gradio SDK entry point: ```bash -docker build -t hackathon-space . -docker run --rm -p 7860:7860 \ - -e MODEL_REPO=Qwen/Qwen2.5-3B-Instruct-GGUF \ - -e MODEL_FILE=qwen2.5-3b-instruct-q4_k_m.gguf \ - -e N_CTX=4096 \ - -e N_GPU_LAYERS=0 \ - hackathon-space +python -m venv .venv-gradio && source .venv-gradio/bin/activate +pip install -r requirements.txt +ACTIVE_MODEL=minicpm5-1b ALLOW_MODEL_SWITCH=false python app.py ``` -Open [http://localhost:7860](http://localhost:7860) β€” Studio at `/`, Classic tabs at `/classic`. Stop with `Ctrl+C`. - -To use a pre-downloaded local model inside Docker, mount it and set `MODEL_PATH`: +Open [http://localhost:7860](http://localhost:7860) β€” Studio at `/`, Classic at `/classic`. -```bash -docker run --rm -p 7860:7860 \ - -v "$(pwd)/models:/app/models:ro" \ - -e MODEL_PATH=/app/models/qwen2.5-3b-instruct-q4_k_m.gguf \ - hackathon-space -``` +Day-to-day development can still use `uv run` (see above); this path mirrors what HF installs from `requirements.txt`. --- -## Hugging Face Space deployment +## Hugging Face Space deployment (Gradio SDK + ZeroGPU) -This repo uses the **Docker SDK**. The Space card metadata lives in the YAML frontmatter at the top of [README.md](README.md). +The Space card metadata lives in the YAML frontmatter at the top of [README.md](README.md) (`sdk: gradio`, `app_file: app.py`). ### 1. Push code to GitHub -Make sure `main` (or your deploy branch) contains at minimum: +Make sure `main` contains at minimum: -- `Dockerfile` -- `README.md` (with `sdk: docker` and `app_port: 7860`) -- `pyproject.toml`, `uv.lock` -- `apps/gradio-space/` and `libs/inference/` +- `app.py`, `requirements.txt`, `packages.txt` +- `README.md` (with `sdk: gradio`, `sdk_version`, `app_file: app.py`) +- `models.yaml`, `skills/` +- `apps/gradio-space/` and all `libs/*` packages + +The root `Dockerfile` stays in the repo for a later Docker SDK deploy (see below). ### 2. Create the Space 1. Go to [build-small-hackathon](https://huggingface.co/build-small-hackathon) 2. **New Space** -3. Name: e.g. `small-model-hackathon` -4. SDK: **Docker** -5. Link your GitHub repo, or push directly to the Space repo +3. Name: e.g. `lesson-agent` or `small-model-hackathon` +4. SDK: **Gradio** (Blank template) +5. Hardware: **ZeroGPU** (creator needs PRO/Team) or **GPU basic** +6. Link your GitHub repo, or push directly to the Space git remote CLI alternative (if you have `hf` installed and org access): ```bash hf repo create build-small-hackathon/ \ --repo-type space \ - --space_sdk docker + --space_sdk gradio ``` -### 3. Configure hardware +### 3. Set Space environment variables + +In the Space **Settings β†’ Variables and secrets**: + +| Variable | Value | +| -------- | ----- | +| `ACTIVE_MODEL` | `minicpm5-1b` | +| `ALLOW_MODEL_SWITCH` | `false` | +| `RESEARCHMIND_DATA_DIR` | `/tmp/researchmind` | +Default preset in [`models.yaml`](models.yaml) is `minicpm5-1b` (transformers) β€” suitable for ZeroGPU. -| Setting | Recommendation | -| -------- | ------------------------------------------------------------ | -| Hardware | **CPU basic** to start (llama.cpp with `N_GPU_LAYERS=0`) | -| Upgrade | GPU Space if you set `N_GPU_LAYERS > 0` for faster inference | +### 4. Build and verify +HF installs from `requirements.txt` and runs root `app.py`. Check the **Logs** tab for: -### 4. Set Space environment variables +- Successful pip install (first build may take several minutes β€” `llama-cpp-python` compiles) +- `Running on local URL: 0.0.0.0:7860` -In the Space **Settings β†’ Variables and secrets**: +Smoke test on the live Space: +1. **`/`** β€” Studio UI loads +2. **`/classic`** β€” all tabs render +3. Generate slides with a simple topic (e.g. "Photosynthesis, grade 8, 5 slides") +4. First LLM request may be slow (model download + ZeroGPU queue) -| Variable | Value | -| ------------------- | --------------------------------- | -| `INFERENCE_BACKEND` | `llama_cpp` | -| `MODEL_REPO` | `Qwen/Qwen2.5-3B-Instruct-GGUF` | -| `MODEL_FILE` | `qwen2.5-3b-instruct-q4_k_m.gguf` | -| `N_CTX` | `4096` | -| `N_GPU_LAYERS` | `0` (or higher on GPU hardware) | +### 5. ZeroGPU notes +LLM handlers use `@spaces.GPU` via [`gradio_space/spaces_runtime.py`](apps/gradio-space/src/gradio_space/spaces_runtime.py). If you see **No CUDA GPUs are available**, an inference path is running outside a decorated handler. -### 5. Build and verify +Startup model preload is skipped on HF Gradio runtime; the first user request loads the model inside a GPU task. + +### 6. Optional: persistent model cache + +Attach a **Storage Bucket** in Space settings so Hub model weights survive restarts. + +--- -HF builds from the root `Dockerfile` and runs: +## Docker SDK deployment (later) + +Both deploy paths live on the same branch. HF reads **one** `sdk:` from README β€” switch to Docker when you are ready for a dedicated-GPU Space. + +1. Change [README.md](README.md) frontmatter to `sdk: docker`, `app_port: 7860` (remove `sdk_version` / `app_file`) +2. Create or reconfigure a Space with **Docker** SDK and **GPU basic** hardware +3. Set the same env vars (`ACTIVE_MODEL=minicpm5-1b`, etc.) + +### Local Docker smoke test ```bash -uv run --package gradio-space python -m gradio_space.app +docker build -t hackathon-space . +docker run --rm -p 7860:7860 \ + -e ACTIVE_MODEL=minicpm5-1b \ + -e ALLOW_MODEL_SWITCH=false \ + -e RESEARCHMIND_DATA_DIR=/tmp/researchmind \ + hackathon-space ``` -Check the **Logs** tab while the Space builds. Once running, open the Space URL and send a test chat message. The first message may take several minutes on CPU while the GGUF downloads. +Open [http://localhost:7860](http://localhost:7860) β€” Studio at `/`, Classic tabs at `/classic`. Stop with `Ctrl+C`. -### 6. Optional: persistent model cache +To use a pre-downloaded local GGUF model inside Docker, mount it and set `MODEL_PATH`: -If cold starts are too slow, attach a **Storage Bucket** in Space settings so downloaded GGUF files survive restarts. +```bash +docker run --rm -p 7860:7860 \ + -v "$(pwd)/models:/app/models:ro" \ + -e MODEL_PATH=/app/models/qwen2.5-3b-instruct-q4_k_m.gguf \ + hackathon-space +``` --- @@ -323,29 +346,24 @@ If cold starts are too slow, attach a **Storage Bucket** in Space settings so do | Symptom | Likely cause | Fix | | ---------------------------------------- | --------------------------------- | -------------------------------------------------------------------- | -| First chat hangs / slow | GGUF downloading from Hub | Pre-download locally; on Space, wait or use Storage Bucket | -| `Failed to load model` in chat | Wrong `MODEL_REPO` / `MODEL_FILE` | Check env vars match a valid GGUF on Hub | -| Docker build fails on `llama-cpp-python` | Missing build tools | Dockerfile already installs `build-essential` and `cmake` | -| Space build fails | Missing `uv.lock` or README YAML | Ensure `sdk: docker` is in root `README.md` frontmatter | -| `transformers` backend error | Optional deps not installed | Run `uv sync --package inference --extra transformers` | -| Port already in use locally | Another process on 7860 | `PORT=7861 uv run --package gradio-space python -m gradio_space.app` | +| First chat hangs / slow | Model downloading from Hub | Wait on Space; use Storage Bucket for cache | +| `Failed to load model` in chat | Wrong `ACTIVE_MODEL` preset | Use `minicpm5-1b` or valid key from `models.yaml` | +| Space build fails on pip install | `llama-cpp-python` compile | Check Logs; default preset avoids GGUF at runtime | +| Space build fails | Malformed README YAML | Ensure `sdk: gradio` and `app_file: app.py` in README frontmatter | +| No CUDA GPUs on ZeroGPU | Handler outside `@spaces.GPU` | LLM entry points must use `gpu_task` in `spaces_runtime.py` | +| Docker build fails on `llama-cpp-python` | Missing build tools | Dockerfile installs `build-essential` and `cmake` | +| Port already in use locally | Another process on 7860 | `PORT=7861 python app.py` or `uv run ...` | --- ## Entrypoint summary -All three environments use the same command: - -```bash -uv run --package gradio-space python -m gradio_space.app -``` - - -| Environment | How to run | -| ----------- | ---------------------------------------------------------- | -| Local dev | `uv run --package gradio-space python -m gradio_space.app` | -| Docker | `docker run -p 7860:7860 hackathon-space` | -| HF Space | Built and started automatically from `Dockerfile` `CMD` | +| Environment | How to run | +| ----------- | ---------- | +| Local dev (uv) | `uv run --package gradio-space python -m gradio_space.app` | +| Local Gradio SDK smoke | `pip install -r requirements.txt && python app.py` | +| HF Gradio Space | HF runs root `app.py` automatically | +| Docker (later) | `docker run -p 7860:7860 hackathon-space` (after README `sdk: docker`) | diff --git a/app.py b/app.py new file mode 100644 index 0000000000000000000000000000000000000000..29b20a8277f30d6e332dc84300e4f0ae374e6bd4 --- /dev/null +++ b/app.py @@ -0,0 +1,6 @@ +"""Hugging Face Gradio SDK entry point (ZeroGPU / Gradio Spaces).""" + +from gradio_space.server import main + +if __name__ == "__main__": + main() diff --git a/apps/gradio-space/src/gradio_space/model_loading.py b/apps/gradio-space/src/gradio_space/model_loading.py index a220ac5bdaebacc463d06fcf283de9bfd291c97a..2d064ac106950aeef861b8e5b98dc7e481578d72 100644 --- a/apps/gradio-space/src/gradio_space/model_loading.py +++ b/apps/gradio-space/src/gradio_space/model_loading.py @@ -1,3 +1,4 @@ +from gradio_space.spaces_runtime import gpu_task from inference.config import get_app_config, get_model_config from inference.factory import get_backend, reset_backend from inference.response_clean import strip_reasoning_output @@ -74,6 +75,7 @@ def warmup(model_key: str | None = None) -> str: ) +@gpu_task(duration=120) def reload_model(model_key: str) -> str: """Clear cached backend and reload weights for settings panel.""" global _current_model_key @@ -120,6 +122,7 @@ def _history_to_messages(history: list) -> list[dict[str, str]]: return messages +@gpu_task(duration=60) def chat(message: str, history: list, model_key: str) -> str: load_error = ensure_model_loaded(model_key) if load_error: diff --git a/apps/gradio-space/src/gradio_space/research_helpers.py b/apps/gradio-space/src/gradio_space/research_helpers.py index 51b8e7ced2d2388ad9f641dc3c2531581058cbd2..1c3407aeb865d8642f948d2b3d0fde61af51d2ed 100644 --- a/apps/gradio-space/src/gradio_space/research_helpers.py +++ b/apps/gradio-space/src/gradio_space/research_helpers.py @@ -8,6 +8,7 @@ import gradio as gr from agent.models import ResearchIngestResult from agent.runner import AgentRunner from gradio_space.model_loading import chat, ensure_model_loaded, get_active_model_key +from gradio_space.spaces_runtime import gpu_task from inference.factory import get_backend from researchmind.ingest import IngestPipeline @@ -209,6 +210,7 @@ def rag_scope_hint(session_id: str, doc_ids: list[str] | None) -> str: return "RAG scope: **entire** indexed corpus (all sessions)." +@gpu_task(duration=180) def run_research_question( question: str, *, diff --git a/apps/gradio-space/src/gradio_space/server.py b/apps/gradio-space/src/gradio_space/server.py index 67af26ba9434df4df06b4227f74ca334d87158cf..402e69d28dde51a0c16c4affa1557f68a8b3196d 100644 --- a/apps/gradio-space/src/gradio_space/server.py +++ b/apps/gradio-space/src/gradio_space/server.py @@ -12,6 +12,7 @@ from gradio import mount_gradio_app from gradio_space.api.studio import register_studio_apis from gradio_space.app import build_demo from gradio_space.model_loading import preload_active_model +from gradio_space.spaces_runtime import is_hf_gradio_runtime from gradio_space.tabs.education_pptx import gradio_allowed_paths from gradio_space.tabs.echo_coach import echo_coach_allowed_paths from gradio_space.tabs.research_mind import researchmind_allowed_paths @@ -66,7 +67,8 @@ def create_server() -> gr.Server: def main() -> None: - preload_active_model() + if not is_hf_gradio_runtime(): + preload_active_model() server = create_server() port = int(os.environ.get("PORT", "7860")) server_name = os.environ.get("GRADIO_SERVER_NAME", "0.0.0.0") diff --git a/apps/gradio-space/src/gradio_space/spaces_runtime.py b/apps/gradio-space/src/gradio_space/spaces_runtime.py new file mode 100644 index 0000000000000000000000000000000000000000..f5de1ea9c9ebe78adb3558f186ecbbc7af10ec07 --- /dev/null +++ b/apps/gradio-space/src/gradio_space/spaces_runtime.py @@ -0,0 +1,37 @@ +"""Hugging Face Spaces ZeroGPU helpers.""" + +from __future__ import annotations + +import os +from collections.abc import Callable +from typing import ParamSpec, TypeVar + +P = ParamSpec("P") +R = TypeVar("R") + + +def is_hf_gradio_runtime() -> bool: + """True on Hugging Face Gradio SDK Spaces (skip startup model preload).""" + try: + import spaces # noqa: F401 + except ImportError: + return False + return bool(os.environ.get("SPACE_ID")) + + +def gpu_task( + *, + duration: int = 180, + size: str = "large", +) -> Callable[[Callable[P, R]], Callable[P, R]]: + """Apply @spaces.GPU when the HF spaces runtime is present (no-op elsewhere).""" + + def decorator(fn: Callable[P, R]) -> Callable[P, R]: + try: + import spaces + + return spaces.GPU(duration=duration, size=size)(fn) + except ImportError: + return fn + + return decorator diff --git a/apps/gradio-space/src/gradio_space/tabs/echo_coach.py b/apps/gradio-space/src/gradio_space/tabs/echo_coach.py index 1486994908d6ce65e682177b234626d3c2a9faa1..98083da328e0592f11e02057a04d2eddd7e42091 100644 --- a/apps/gradio-space/src/gradio_space/tabs/echo_coach.py +++ b/apps/gradio-space/src/gradio_space/tabs/echo_coach.py @@ -7,6 +7,7 @@ import gradio as gr from echocoach.config import get_echo_coach_config from echocoach.pipeline import run_echo_coach from gradio_space.model_loading import ensure_model_loaded, get_active_model_key +from gradio_space.spaces_runtime import gpu_task from gradio_space.ui.components import ( build_advanced_panel, build_recording_block, @@ -64,6 +65,7 @@ def load_sample_pitch() -> tuple[str | None, str]: ) +@gpu_task(duration=180) def analyze_pitch( audio_path: str | None, language: str, diff --git a/apps/gradio-space/src/gradio_space/tabs/education_pptx.py b/apps/gradio-space/src/gradio_space/tabs/education_pptx.py index 94ec993b583359541e57cabe345dc1909309b909..fc070b380c97b7ca7c712965910fdd6b4d46387c 100644 --- a/apps/gradio-space/src/gradio_space/tabs/education_pptx.py +++ b/apps/gradio-space/src/gradio_space/tabs/education_pptx.py @@ -16,6 +16,7 @@ from gradio_space.research_helpers import ( resolve_session, resolve_topic, ) +from gradio_space.spaces_runtime import gpu_task from gradio_space.ui.components import build_advanced_panel, DOC_CHOICE_LIST_CLASSES, WorkspaceWidgets from inference.factory import get_backend from researchmind.config import get_config @@ -158,6 +159,7 @@ def update_source_visibility(source_mode_label: str, search_workflow_label: str) ) +@gpu_task(duration=120) def discover_lesson_sources( topic: str, session_id: str, @@ -208,6 +210,7 @@ def discover_lesson_sources( return msg, gr.update(choices=[], value=[]), refresh_sessions(session_id) +@gpu_task(duration=300) def generate_lesson_slides( topic: str, grade: str, diff --git a/apps/gradio-space/src/gradio_space/tabs/research_mind.py b/apps/gradio-space/src/gradio_space/tabs/research_mind.py index 62e727f49869ba6f27e313f1f63257fd56e257f6..94591230eaef0daa2ed8102f9c3e6611747b4312 100644 --- a/apps/gradio-space/src/gradio_space/tabs/research_mind.py +++ b/apps/gradio-space/src/gradio_space/tabs/research_mind.py @@ -23,6 +23,7 @@ from gradio_space.research_helpers import ( run_research_question, trace_summary_markdown, ) +from gradio_space.spaces_runtime import gpu_task from gradio_space.ui.components import build_advanced_panel, DOC_CHOICE_LIST_CLASSES, WorkspaceWidgets from inference.factory import get_backend @@ -35,6 +36,7 @@ def _require_topic(topic: str | None) -> str | None: return None +@gpu_task(duration=120) def discover_sources( topic: str, session_id: str, @@ -118,6 +120,7 @@ def discover_sources( ) +@gpu_task(duration=180) def auto_search_ingest( topic: str, session_id: str, @@ -279,6 +282,7 @@ def ingest_selected( ) +@gpu_task(duration=180) def ask_question( question: str, session_id: str, diff --git a/apps/gradio-space/src/gradio_space/tabs/teacher_voice.py b/apps/gradio-space/src/gradio_space/tabs/teacher_voice.py index c108a6c05ff6e681834413109c1c5aace7d37129..39c182a608ed599e89aabd667080c2daaebea2b1 100644 --- a/apps/gradio-space/src/gradio_space/tabs/teacher_voice.py +++ b/apps/gradio-space/src/gradio_space/tabs/teacher_voice.py @@ -18,6 +18,7 @@ from gradio_space.research_helpers import ( resolve_topic, trace_as_dict, ) +from gradio_space.spaces_runtime import gpu_task from gradio_space.tabs.research_mind import ( auto_search_ingest, discover_sources, @@ -87,6 +88,7 @@ def _turn_error(history: list | None, message: str) -> tuple: ) +@gpu_task(duration=180) def send_turn( audio_path: str | None, history: list, @@ -142,6 +144,7 @@ def send_turn( return _turn_result(result) +@gpu_task(duration=180) def send_text_turn( message: str, history: list, diff --git a/apps/gradio-space/static/studio/index.html b/apps/gradio-space/static/studio/index.html index f7490b61e4645e0fe6b5f83668e55a1cc7002d5f..6f7a54c81b583e064055b07fd700c6bb572b31c4 100644 --- a/apps/gradio-space/static/studio/index.html +++ b/apps/gradio-space/static/studio/index.html @@ -291,83 +291,107 @@
- -
-

RAG Scope

- -

Session and documents use workspace defaults above unless overridden per tool.

-
-
-

Teacher Voice Mode

-
- - - -
- -
- ResearchMind sources (optional) -

Set focus topic, then discover or ingest sources. Enable RAG above to ground answers in your library.

-
- - +
+
-
-

Type a message or record audio, then send.

-
-
-
- -

Plain chat or corpus-grounded answers β€” traces appear below when RAG is on.

+
+
+ +

Plain chat or corpus-grounded answers β€” traces appear below when RAG is on.

+

Plain chat or corpus-grounded answers β€” traces appear below when RAG is on.

+