oops
/
iron
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
Browse Source

Streamline open source documentation

codex/vue-opencloud-design-system
Rain Mark 3 months ago
parent
17f1530328
commit
a32943be95
16 changed files with 623 additions and 3734 deletions
Whitespace
Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL
Split View
Diff Options
Show Stats Download Patch File Download Diff File
  1. 82
      AGENTS.md
  2. 62
      CONTRIBUTING.md
  3. 130
      README.md
  4. 14
      docs/README.md
  5. 726
      docs/api-design.md
  6. 84
      docs/api.md
  7. 122
      docs/architecture.md
  8. 507
      docs/database-schema.md
  9. 105
      docs/development.md
  10. 233
      docs/handoff.md
  11. 473
      docs/mvp-status.md
  12. 69
      docs/product.md
  13. 397
      docs/requirements.md
  14. 874
      docs/technical-architecture-draft.md
  15. 137
      docs/ui-playwright-standard.md
  16. 342
      docs/web-ui-technical-design.md

82
AGENTS.md
View File

@ -0,0 +1,82 @@
# AGENTS.md
Guidance for coding agents and automated contributors working in this repository.
## Project Snapshot
Iron is a self-hosted personal cloud drive gateway with:
- FastAPI backend under `app/`
- Vite + React + TypeScript frontend under `frontend/`
- built frontend assets served from `app/web/dist/`
- Alembic migrations under `alembic/`
- tests under `tests/`
- project docs under `docs/`
The current stage is **Product MVP Candidate**. Preserve the existing
desktop-first drive UX and avoid drifting back toward a generic admin dashboard.
## Required Setup
Use the local virtualenv and `uv` cache:
```bash
UV_CACHE_DIR=.uv-cache uv venv .venv
UV_CACHE_DIR=.uv-cache uv pip install --python .venv/bin/python -e '.[dev]'
npm install
```
## Common Commands
```bash
npm run build
.venv/bin/python -m pytest
.venv/bin/python scripts/ui_playwright_smoke.py
.venv/bin/python -m pytest tests/e2e/test_web_ui_playwright.py
```
## Development Rules
- Keep runtime artifacts out of Git: `.venv/`, `node_modules/`, `output/`,
`.iron-storage/`, `.iron-temp/`, caches, and local databases.
- Add or update tests in the same change as behavior changes.
- For UI work, run the real-service Playwright flow and review screenshots.
- Do not use naked `/api/files/...` media URLs for browser preview/download.
Fetch file content with authorization, create an object URL, and revoke it.
- Keep docs current when changing setup, architecture, test counts, or user flows.
- Prefer small, focused changes over broad rewrites.
## UI Quality Standard
For any meaningful Web UI change:
1. build frontend assets with `npm run build`
2. run `./.venv/bin/python scripts/ui_playwright_smoke.py`
3. inspect screenshots in `output/playwright/`
4. fix layout or interaction issues found in screenshots
5. rerun the Playwright flow
See [docs/development.md](docs/development.md).
## Documentation Expectations
- `README.md` should stay user-facing and concise.
- `CONTRIBUTING.md` should describe contributor workflow.
- `docs/README.md` should index deeper docs.
- `docs/development.md` should describe test and UI validation rules.
- Avoid local absolute paths in committed Markdown.
## Verification Before Finishing
At minimum, run:
```bash
npm run build
.venv/bin/python -m pytest
```
If UI behavior changed, also run:
```bash
.venv/bin/python scripts/ui_playwright_smoke.py
```

62
CONTRIBUTING.md
View File

@ -0,0 +1,62 @@
# Contributing
Thanks for helping improve Iron. The project is still early, so the most useful
contributions are focused changes with tests and clear documentation updates.
## Development Setup
```bash
UV_CACHE_DIR=.uv-cache uv venv .venv
UV_CACHE_DIR=.uv-cache uv pip install --python .venv/bin/python -e '.[dev]'
npm install
npm run build
```
Run the API locally:
```bash
.venv/bin/python -m uvicorn app.main:app --reload
```
Open `http://127.0.0.1:8000/app`.
## Test Before Submitting
Run the full suite:
```bash
.venv/bin/python -m pytest
```
For UI changes, also run:
```bash
npm run build
.venv/bin/python scripts/ui_playwright_smoke.py
```
Review screenshots in `output/playwright/` before considering UI work complete.
## Pull Request Checklist
- The change is focused and described clearly.
- Tests were added or updated for behavior changes.
- UI changes were validated with Playwright when applicable.
- Documentation was updated if setup, behavior, or architecture changed.
- Generated/runtime files are not included.
## Code Style
- Backend code lives in `app/` and uses async FastAPI + SQLAlchemy patterns.
- Frontend code lives in `frontend/` and uses React, React Router, and
TanStack Query.
- Keep browser file preview/download flows authenticated through the API client.
- Prefer clear names and small modules over clever abstractions.
## Project Status
Iron is a Product MVP Candidate. See:
- [docs/product.md](docs/product.md)
- [docs/architecture.md](docs/architecture.md)
- [docs/development.md](docs/development.md)

130
README.md
View File

@ -1,44 +1,54 @@
# Iron
Personal cloud drive gateway and Web app.
Iron is a self-hosted personal cloud drive gateway. It provides one logical file
namespace over local storage and pluggable backends, with a browser UI for daily
file management and operational visibility.
## Current Status
The project is currently a **Product MVP Candidate**: the core backend and Web
app exist, but the release still needs hardening, broader browser regression
coverage, and more real-world usage.
- product requirements drafted in `docs/`
- technical architecture drafted in `docs/`
- Python backend foundation implemented
- product Web UI implemented with `Vite + React + TypeScript`
- local auth, upload, local persistence, S3 reconciliation, policy-driven replica placement, declarative background jobs, full-system reconcile, preview artifact generation, metadata export/validate/integrity/import, and a product-oriented browser app are available
- the current Web app includes files, uploads, recycle bin, storage, and jobs pages with a desktop-first drive layout
- authenticated download and inline preview flows are covered by Python Playwright E2E tests
- current automated baseline: `66` passing tests
- next primary implementation focus should be hardening the product experience through real-browser regression, not broad backend expansion
## Features
## Backend
- FastAPI backend with SQLite, SQLAlchemy, and Alembic migrations
- Local authentication with persisted bearer-token sessions
- Directory and file operations: browse, create folder, rename, move, delete,
restore, download, and preview
- `tus` upload flow with local object persistence
- Desktop-first React Web app at `/app`
- Storage backend management for local storage and S3 runtime paths
- Placement policies, reconcile jobs, health checks, and retryable background jobs
- Metadata export, validation, integrity checks, restore plans, and guarded import
- Python Playwright end-to-end coverage for core browser flows
Create the local environment with `uv`:
## Quick Start
Iron uses `uv` for local Python environment setup.
```bash
UV_CACHE_DIR=.uv-cache uv venv .venv
UV_CACHE_DIR=.uv-cache uv pip install --python .venv/bin/python -e '.[dev]'
```
Run the API locally:
```bash
npm install
npm run build
.venv/bin/python -m uvicorn app.main:app --reload
```
Open the current browser app at:
Open the Web app:
- [http://127.0.0.1:8000/app](http://127.0.0.1:8000/app)
```text
http://127.0.0.1:8000/app
```
Default local bootstrap credentials:
- username: `admin`
- password: `changeme-iron`
```text
username: admin
password: changeme-iron
```
## Configuration
Environment overrides:
Common environment variables:
- `IRON_DATABASE_URL`
- `IRON_BOOTSTRAP_USERNAME`
@ -50,12 +60,18 @@ Environment overrides:
- `IRON_JOB_POLL_INTERVAL_SECONDS`
- `IRON_JOB_BATCH_SIZE`
Restore safety:
Runtime data is intentionally ignored by Git. By default, local development may
create `.iron-storage/`, `.iron-temp/`, `iron.db`, and `output/`.
## Development
- metadata import requires `confirm_replace=true`
- metadata import also requires a fresh validation token from `POST /api/exports/metadata/restore-plan`
Build the frontend:
```bash
npm run build
```
## Tests
Run the full test suite:
```bash
.venv/bin/python -m pytest
@ -63,9 +79,11 @@ Restore safety:
Current expected result:
- `66 passed`
```text
66 passed
```
Run the browser E2E flow directly with:
Run the browser E2E flow directly:
```bash
.venv/bin/python scripts/ui_playwright_smoke.py
@ -77,23 +95,45 @@ Or through pytest:
.venv/bin/python -m pytest tests/e2e/test_web_ui_playwright.py
```
## Implemented Foundations
Playwright screenshots and runtime artifacts are written to `output/playwright/`.
## Repository Layout
```text
app/ FastAPI application, services, repositories, schemas
alembic/ Database migrations
frontend/ Vite + React + TypeScript source
app/web/dist/ Built frontend assets served by FastAPI
scripts/ Local automation and E2E scripts
tests/ Backend, API, and browser E2E tests
docs/ Product, architecture, API, and maintenance docs
```
## Documentation
Start with:
- [Documentation index](docs/README.md)
- [Product](docs/product.md)
- [Architecture](docs/architecture.md)
- [API](docs/api.md)
- [Development](docs/development.md)
For contribution workflow, see [CONTRIBUTING.md](CONTRIBUTING.md).
For AI/coding-agent guidance, see [AGENTS.md](AGENTS.md).
## Security Notes
Iron is not yet a hardened public Internet service. Treat the current credentials
and local bearer-token session model as development-oriented defaults. Change the
bootstrap password and review deployment boundaries before exposing the service.
- FastAPI application bootstrap
- async SQLAlchemy engine setup
- initial ORM entity definitions
- Alembic migrations
- local auth bootstrap and bearer-token sessions
- React Web app at `/app`
- authenticated browser download and preview using bearer-token-backed blob fetches
- local + S3 runtime storage foundation
- persisted placement policy controls and placement preview API
- SQLite-backed job records, declarative reconcile flows, and metadata export/validate/integrity/import APIs
- health and readiness endpoints
- baseline automated tests
Metadata import is guarded and requires both:
## Handoff
- `confirm_replace=true`
- a fresh validation token from `POST /api/exports/metadata/restore-plan`
For the most useful handoff view before continuing development, start here:
## License
- [docs/handoff.md](/Users/bytedance/iron/docs/handoff.md:1)
No open source license has been selected yet. Add a `LICENSE` file before
publishing or accepting external contributions.

14
docs/README.md
View File

@ -0,0 +1,14 @@
# Documentation
This directory contains the stable project documentation for Iron.
- [Product](product.md): product scope, current stage, MVP cutline, and roadmap.
- [Architecture](architecture.md): backend, frontend, data model, storage, and job design.
- [API](api.md): route map and API groups.
- [Development](development.md): setup, test commands, Playwright workflow, and documentation rules.
Root-level project docs:
- [README](../README.md): user-facing overview and quick start.
- [Contributing](../CONTRIBUTING.md): contributor workflow.
- [Agents](../AGENTS.md): coding-agent and automation guidance.

726
docs/api-design.md
View File

@ -1,726 +0,0 @@
# Iron API Design
## 1. Overview
This document defines the MVP API surface for the Iron gateway.
Principles:
- REST-first
- browser-first product usage, with backend auth that can evolve
- `tus` for resumable upload transport
- stable logical file identifiers
- backend topology hidden from normal users
Base prefix:
- `/api`
Non-API upload transport prefix:
- `/files`
Notes:
- `tus` endpoints intentionally live outside the normal JSON API shape because they follow protocol-specific request and response semantics.
- All timestamps should use ISO 8601 UTC strings.
- All entity IDs should use UUIDv7 or another sortable unique ID format.
## 2. Authentication
MVP auth model:
- single local user
- bearer token session for the current backend implementation
- Web UI can later wrap this with cookie-based session handling if desired
### 2.1 Login
- `POST /api/auth/login`
Request:
```json
{
"username": "admin",
"password": "secret"
}
```
Response:
```json
{
"access_token": "iron_xxx",
"token_type": "bearer",
"expires_at": "2026-04-20T00:00:00Z",
"user": {
"id": "usr_01",
"username": "admin"
}
}
```
Behavior:
- implemented in current codebase
- creates a persisted auth session and returns a bearer token
- returns `401` for invalid credentials
### 2.2 Logout
- `POST /api/auth/logout`
Response:
```json
{
"success": true
}
```
Behavior:
- implemented in current codebase
- revokes the current bearer-token session
### 2.3 Current Session
- `GET /api/auth/me`
Response:
```json
{
"user": {
"id": "usr_01",
"username": "admin"
}
}
```
Behavior:
- implemented in current codebase
- requires `Authorization: Bearer <token>`
## 3. Directories
Route protection:
- all directory, file, upload, and backend routes require authentication in the current codebase
### 3.1 List Directory Children
- `GET /api/directories/{directory_id}/children`
Query params:
- `cursor`
- `limit`
- `sort`
- `order`
Response:
```json
{
"directory": {
"id": "dir_root",
"name": "/",
"parent_id": null
},
"items": [
{
"kind": "directory",
"id": "dir_01",
"name": "photos",
"created_at": "2026-04-13T00:00:00Z",
"updated_at": "2026-04-13T00:00:00Z"
},
{
"kind": "file",
"id": "fil_01",
"name": "movie.mp4",
"mime_type": "video/mp4",
"size_bytes": 1048576,
"cache_status": "partial",
"preview_status": "ready",
"created_at": "2026-04-13T00:00:00Z",
"updated_at": "2026-04-13T00:00:00Z"
}
],
"next_cursor": null
}
```
### 3.2 Create Directory
- `POST /api/directories`
Request:
```json
{
"parent_id": "dir_root",
"name": "photos"
}
```
Implemented status:
- implemented in current codebase
- currently supports create under an existing parent directory
- duplicate sibling names are rejected
### 3.3 Rename Directory
- `POST /api/directories/{directory_id}/rename`
Request:
```json
{
"name": "vacation-photos"
}
```
Implemented status:
- implemented in current codebase
### 3.4 Move Directory
- `POST /api/directories/{directory_id}/move`
Request:
```json
{
"target_parent_id": "dir_02"
}
```
Implemented status:
- implemented in current codebase
- moving a directory updates descendant logical path keys
### 3.5 Delete Directory
- `DELETE /api/directories/{directory_id}`
Behavior:
- soft delete in MVP
- reject if non-empty unless `recursive=true` is explicitly supported later
Current implementation status:
- `GET /api/directories/{directory_id}/children` is implemented
- current response can contain both directory items and file items
- items are currently returned with directories first, then files
## 4. Files
### 4.1 Get File Details
- `GET /api/files/{file_id}`
Response:
```json
{
"file": {
"id": "fil_01",
"directory_id": "dir_root",
"name": "movie.mp4",
"mime_type": "video/mp4",
"size_bytes": 1048576,
"current_version_id": "ver_01",
"preview_status": "ready",
"cache_status": "partial",
"created_at": "2026-04-13T00:00:00Z",
"updated_at": "2026-04-13T00:00:00Z"
},
"preview_artifacts": [
{
"id": "prv_01",
"artifact_type": "thumbnail",
"blob_id": "blb_01",
"status": "ready"
}
]
}
```
Implemented status:
- implemented in current codebase
- returns logical file metadata from the gateway database
- now includes preview artifact metadata when background generation has run
### 4.2 Rename File
- `POST /api/files/{file_id}/rename`
Request:
```json
{
"name": "movie-final.mp4"
}
```
Implemented status:
- implemented in current codebase
### 4.3 Move File
- `POST /api/files/{file_id}/move`
Request:
```json
{
"target_parent_id": "dir_02"
}
```
Implemented status:
- implemented in current codebase
### 4.4 Delete File
- `DELETE /api/files/{file_id}`
Behavior:
- current implementation moves the file to a recycle-bin state
- deleted files disappear from normal directory listings
- deleted files can be listed and restored through recycle-bin APIs
### 4.5 Download File
- `GET /api/files/{file_id}/download`
Behavior:
- returns attachment response
- gateway resolves best available replica and streams to client
Implemented status:
- implemented in current codebase for files persisted to the default local backend
- current implementation resolves the ready local replica and serves it directly
### 4.6 Stream File
- `GET /api/files/{file_id}/stream`
Behavior:
- supports `Range` requests
- intended for video and browser-native preview flows
Implemented status:
- implemented in current codebase for local persisted objects
- supports full reads and single-range byte requests
### 4.7 File Thumbnail
- `GET /api/files/{file_id}/thumbnail`
Behavior:
- returns preview derivative if present
- returns `404` if not yet available
### 4.7a File Preview
- `GET /api/files/{file_id}/preview`
Behavior:
- returns an inline response for preview-supported file types
Implemented status:
- implemented in current codebase for images and PDFs persisted to the default local backend
- unsupported media types currently return a validation-style error
### 4.8 Recycle Bin
- `GET /api/files/recycle-bin`
- `POST /api/files/{file_id}/restore`
Current implementation status:
- both endpoints are implemented
- restore returns a conflict if the original directory now contains an active item with the same file name
## 5. Tus Upload Flow
Iron uses `tus` as the upload transport and adds one gateway-specific finalize step.
### 5.1 Create Upload Resource
- `POST /files`
Required headers:
- `Tus-Resumable`
- `Upload-Length`
- `Upload-Metadata`
Behavior:
- creates a new upload session
- returns `201` with `Location`
Implemented status:
- implemented in current codebase
- currently supports sequential append uploads with declared total length
- returns a gateway-managed upload resource under `/files/{upload_id}`
### 5.2 Query Upload Offset
- `HEAD /files/{upload_id}`
Behavior:
- returns current `Upload-Offset`
Implemented status:
- implemented in current codebase
### 5.3 Upload Bytes
- `PATCH /files/{upload_id}`
Required headers:
- `Tus-Resumable`
- `Upload-Offset`
- `Content-Type: application/offset+octet-stream`
Behavior:
- appends bytes to temporary ingest file
- updates upload progress
Implemented status:
- implemented in current codebase
- currently requires exact offset matching and sequential writes
### 5.4 Finalize Upload
- `POST /api/uploads/{upload_id}/finalize`
Request:
```json
{
"directory_id": "dir_root",
"filename": "movie.mp4"
}
```
Response:
```json
{
"file": {
"id": "fil_01",
"name": "movie.mp4"
},
"upload": {
"id": "upl_01",
"status": "finalized"
}
}
```
Behavior:
- verifies upload completion
- computes content metadata
- creates file and file version records
- performs first durable write
- only exposes the logical file after minimum durability is met
Implemented status:
- implemented in current codebase
- currently finalizes into metadata records and persists the blob into the default local backend
- creates a `blob_replicas` record for the local persisted object
- enqueues replication jobs for enabled non-local backends such as `s3`
## 6. Backends
### 6.1 List Backends
- `GET /api/backends`
### 6.2 Create Backend
- `POST /api/backends`
Request example:
```json
{
"name": "local-main",
"type": "local",
"stability_class": "local",
"read_priority": 100,
"write_priority": 100,
"config": {
"base_path": "/srv/iron/storage"
}
}
```
### 6.3 Update Backend
- `PATCH /api/backends/{backend_id}`
### 6.4 Check Backend Health
- `POST /api/backends/{backend_id}/check`
Response:
```json
{
"backend_id": "bkd_01",
"status": "healthy",
"checked_at": "2026-04-13T00:00:00Z"
}
```
### 6.5 Disable Backend
- `POST /api/backends/{backend_id}/disable`
## 7. Placement Policies
### 7.1 List Placement Policies
- `GET /api/policies/placement`
Behavior:
- implemented in current codebase
- returns file-class placement policy rows
- current policy shape includes:
- `require_local`
- `stable_replica_count`
- `opportunistic_replica_count`
- ordered `preferred_backend_ids`
- explicit `excluded_backend_ids`
- optional `max_non_local_size_bytes`
### 7.2 Upsert Placement Policy
- `PUT /api/policies/placement/{file_class}`
Request example:
```json
{
"require_local": true,
"stable_replica_count": 1,
"opportunistic_replica_count": 1,
"preferred_backend_ids": ["bkd_s3_primary", "bkd_s3_archive"],
"excluded_backend_ids": ["bkd_s3_legacy"],
"max_non_local_size_bytes": 104857600
}
```
Behavior:
- implemented in current codebase
- validates that referenced backend ids exist
- rejects overlap between preferred and excluded backend ids
- rejects negative replica counts
### 7.3 Preview Placement Decision
- `GET /api/policies/placement/preview`
Query params:
- `mime_type`
- `size_bytes`
Behavior:
- implemented in current codebase
- returns the matched file class, active policy, and selected backends for the given mime type and optional size
- marks whether non-local replicas are currently allowed under the policy
## 8. Jobs
### 8.1 List Jobs
- `GET /api/jobs`
Query params:
- `kind`
- `status`
- `limit`
### 8.2 Get Job
- `GET /api/jobs/{job_id}`
### 8.3 Retry Job
- `POST /api/jobs/{job_id}/retry`
### 8.4 Run Pending Jobs
- `POST /api/jobs/run-pending`
### 8.5 Enqueue Backend Health Checks
- `POST /api/jobs/enqueue-health-checks`
### 8.6 Enqueue Full Reconcile
- `POST /api/jobs/enqueue-full-reconcile`
## 9. Exports
### 9.1 Metadata Export
- `GET /api/exports/metadata`
### 9.2 Metadata Import
- `POST /api/exports/metadata/import`
Notes:
- current backend requires `confirm_replace=true` for destructive import
- current backend also requires a fresh `validation_token` from `POST /api/exports/metadata/restore-plan`
- successful import currently schedules follow-up reconcile and backend health-check jobs
## 9.3 Reconciliation
- `POST /api/files/{file_id}/reconcile`
Implemented status:
- implemented in current codebase
- the backend intention is "make this file converge to policy-defined replica state"
## 9.4 Metadata Validation and Integrity
- `POST /api/exports/metadata/validate`
- `GET /api/exports/metadata/integrity`
## 9.5 Restore Plan
- `POST /api/exports/metadata/restore-plan`
## 10. System
### 10.1 Health Check
- `GET /api/system/health`
Response:
```json
{
"status": "ok"
}
```
### 10.2 Readiness
- `GET /api/system/ready`
Response:
```json
{
"status": "ready",
"database": "ok"
}
```
## 10.3 Backends
- `GET /api/backends`
- `POST /api/backends`
- `POST /api/backends/{backend_id}/check`
- `POST /api/backends/{backend_id}/disable`
Implemented status:
- implemented in current codebase
- supports local and s3 backend configuration metadata
- local and s3 backend health checks are implemented
Additional implemented status:
- `POST /api/jobs/run-pending` is implemented in current codebase
- `POST /api/jobs/enqueue-health-checks` is implemented in current codebase
- `GET /api/exports/metadata` is implemented in current codebase
- `POST /api/exports/metadata/validate` is implemented in current codebase
- `GET /api/exports/metadata/integrity` is implemented in current codebase
- `POST /api/exports/metadata/import` is implemented in current codebase
- `POST /api/files/{file_id}/reconcile` is implemented in current codebase
## 11. Error Shape
All JSON API errors should use one consistent envelope:
```json
{
"error": {
"code": "file_not_found",
"message": "File does not exist.",
"details": null
}
}
```
Suggested error codes:
- `unauthorized`
- `forbidden`
- `validation_error`
- `directory_not_found`
- `file_not_found`
- `upload_not_found`
- `backend_unavailable`
- `backend_config_invalid`
- `conflict`
- `internal_error`
## 12. Versioning Approach
MVP can stay unversioned internally if the API is changing quickly.
Recommendation:
- keep `/api` in code
- be prepared to introduce `/api/v1` before external client surface becomes stable
## 13. Implementation Notes
- `tus` transport handlers should be separated from normal JSON routers.
- Finalization logic must stay in application services, not inside protocol handlers.
- Download and stream endpoints should rely on the same replica resolution service.
- Avoid exposing backend IDs in normal file responses unless operationally useful.

84
docs/api.md
View File

@ -0,0 +1,84 @@
# API
Iron exposes JSON APIs under `/api` and `tus` upload transport under `/files`.
Most `/api` routes require:
```text
Authorization: Bearer <token>
```
## Auth
- `POST /api/auth/login`
- `POST /api/auth/logout`
- `GET /api/auth/me`
## Directories
- `GET /api/directories/{directory_id}/children`
- `POST /api/directories`
- `POST /api/directories/{directory_id}/rename`
- `POST /api/directories/{directory_id}/move`
## Files
- `GET /api/files/{file_id}`
- `GET /api/files/{file_id}/download`
- `GET /api/files/{file_id}/preview`
- `GET /api/files/{file_id}/stream`
- `POST /api/files/{file_id}/rename`
- `POST /api/files/{file_id}/move`
- `DELETE /api/files/{file_id}`
- `POST /api/files/{file_id}/restore`
- `GET /api/files/recycle-bin`
## Uploads
Protocol endpoints:
- `POST /files`
- `HEAD /files/{upload_id}`
- `PATCH /files/{upload_id}`
Finalize endpoint:
- `POST /api/uploads/{upload_id}/finalize`
## Backends
- `GET /api/backends`
- `POST /api/backends`
- `POST /api/backends/{backend_id}/check`
- `POST /api/backends/{backend_id}/disable`
## Jobs
- `GET /api/jobs`
- `GET /api/jobs/{job_id}`
- `POST /api/jobs/{job_id}/retry`
- `POST /api/jobs/run-pending`
- `POST /api/jobs/enqueue-health-checks`
- `POST /api/jobs/enqueue-full-reconcile`
## Placement Policies
- `GET /api/policies/placement`
- `PUT /api/policies/placement/{file_class}`
- `GET /api/policies/placement/preview`
## Metadata Export And Recovery
- `GET /api/exports/metadata`
- `POST /api/exports/metadata/validate`
- `GET /api/exports/metadata/integrity`
- `POST /api/exports/metadata/restore-plan`
- `POST /api/exports/metadata/import`
Metadata import requires `confirm_replace=true` and a validation token from the
restore-plan endpoint.
## System
- `GET /api/system/health`
- `GET /api/system/ready`

122
docs/architecture.md
View File

@ -0,0 +1,122 @@
# Architecture
Iron is an async Python modular monolith with a React single-page Web app.
## System Overview
```text
Browser UI
|
FastAPI app
|
Services
|
Repositories
|
SQLite metadata database
|
Storage adapters: local, S3
```
## Backend
Core stack:
- FastAPI
- SQLAlchemy 2.x async ORM
- SQLite via `aiosqlite`
- Alembic migrations
- durable in-process jobs stored in SQLite
Important packages:
- `app/api/`: route handlers and dependency wiring
- `app/services/`: business logic
- `app/repositories/`: database access
- `app/models/`: SQLAlchemy entities
- `app/schemas/`: API schemas
- `app/adapters/storage/`: storage backends
## Frontend
Core stack:
- Vite
- React
- TypeScript
- React Router
- TanStack Query
Source lives in `frontend/`. Built assets are emitted into `app/web/dist/` and
served by FastAPI at `/app`.
## Data Model
Main tables:
- `users`
- `auth_sessions`
- `directories`
- `file_entries`
- `file_versions`
- `blobs`
- `blob_replicas`
- `backends`
- `upload_sessions`
- `upload_session_parts`
- `jobs`
- `placement_policies`
- `preview_artifacts`
- `cache_entries`
Design principles:
- logical namespace is separate from physical storage placement
- files have immutable content versions
- physical content is represented as blobs and replicas
- user-facing deletion uses recycle-bin semantics
- jobs are durable and retryable
- backend-specific configuration stays behind adapter boundaries
## Upload And Read Path
Uploads use `tus` transport under `/files`.
Flow:
1. create upload session
2. append bytes with `PATCH`
3. finalize through the API
4. create file metadata, version, blob, and local replica
5. enqueue replication or preview jobs when needed
Read path:
1. resolve file and current version
2. find the best local or cached blob
3. if needed, materialize a ready remote replica into cache
4. serve download, preview, or range stream
## Browser File Access
JSON APIs use bearer-token authorization through the shared frontend API client.
Browser preview and download must not use naked `/api/files/...` media URLs in
`img`, `iframe`, `video`, or `window.open`. Fetch file content with
authorization, create an object URL, use it, then revoke it.
This is covered by the Playwright E2E flow.
## Jobs
Jobs are stored in SQLite and executed by the in-process job loop.
Current job types include:
- blob replication
- preview artifact generation
- backend health checks
- full reconcile
- file reconcile
Every job should be safe to retry.

507
docs/database-schema.md
View File

@ -1,507 +0,0 @@
# Iron Database Schema
## 1. Overview
This document defines the initial SQLite schema for Iron MVP.
Goals:
- support a single logical file namespace
- separate logical metadata from physical storage placement
- track replicas and background jobs durably
- support `tus` resumable uploads
- keep the schema portable enough for future migration to PostgreSQL
Conventions:
- IDs are stored as `TEXT`
- timestamps are stored as UTC datetimes
- soft delete is preferred for logical user-facing entities in MVP
- provider-specific details belong in JSON columns only when no stable relational shape exists
## 2. Entity Overview
Main tables:
- `users`
- `auth_sessions`
- `directories`
- `file_entries`
- `file_versions`
- `blobs`
- `blob_replicas`
- `backends`
- `upload_sessions`
- `upload_session_parts`
- `jobs`
- `placement_policies`
- `preview_artifacts`
- `cache_entries`
## 3. Table Definitions
### 3.1 `users`
Purpose:
- local gateway login identity
Columns:
- `id` TEXT PRIMARY KEY
- `username` TEXT NOT NULL UNIQUE
- `password_hash` TEXT NOT NULL
- `created_at` DATETIME NOT NULL
- `updated_at` DATETIME NOT NULL
### 3.2 `auth_sessions`
Purpose:
- persisted login sessions for bearer-token auth
Columns:
- `id` TEXT PRIMARY KEY
- `user_id` TEXT NOT NULL REFERENCES `users`(`id`)
- `token_hash` TEXT NOT NULL UNIQUE
- `expires_at` DATETIME NOT NULL
- `revoked_at` DATETIME NULL
- `created_at` DATETIME NOT NULL
- `updated_at` DATETIME NOT NULL
Indexes:
- `idx_auth_sessions_user_status` on (`user_id`, `revoked_at`, `expires_at`)
- unique `idx_auth_sessions_token_hash` on `token_hash`
Notes:
- raw session tokens should never be stored directly
- the current implementation stores a SHA-256 token hash and returns the raw bearer token only at login time
### 3.3 `directories`
Purpose:
- logical folder hierarchy
Columns:
- `id` TEXT PRIMARY KEY
- `parent_id` TEXT NULL REFERENCES `directories`(`id`)
- `name` TEXT NOT NULL
- `path_key` TEXT NOT NULL UNIQUE
- `created_at` DATETIME NOT NULL
- `updated_at` DATETIME NOT NULL
- `deleted_at` DATETIME NULL
Constraints:
- unique sibling name constraint should be enforced in application logic or by an additional composite unique index later
Indexes:
- `idx_directories_parent_id` on `parent_id`
- `idx_directories_path_key` on `path_key`
### 3.4 `file_entries`
Purpose:
- logical files visible to users
Columns:
- `id` TEXT PRIMARY KEY
- `directory_id` TEXT NOT NULL REFERENCES `directories`(`id`)
- `name` TEXT NOT NULL
- `mime_type` TEXT NULL
- `size_bytes` INTEGER NOT NULL
- `current_version_id` TEXT NULL
- `is_deleted` BOOLEAN NOT NULL DEFAULT 0
- `deleted_at` DATETIME NULL
- `created_at` DATETIME NOT NULL
- `updated_at` DATETIME NOT NULL
Constraints:
- unique active filename per directory
Indexes:
- unique `idx_file_entries_directory_name` on (`directory_id`, `name`, `is_deleted`)
- `idx_file_entries_current_version_id` on `current_version_id`
- `idx_file_entries_deleted_at` on `deleted_at`
Notes:
- file deletion uses recycle-bin semantics in MVP
- active and deleted files may temporarily share the same logical name in one directory because the uniqueness rule only protects active entries
### 3.5 `file_versions`
Purpose:
- immutable snapshots of file content
Columns:
- `id` TEXT PRIMARY KEY
- `file_entry_id` TEXT NOT NULL REFERENCES `file_entries`(`id`)
- `content_hash` TEXT NOT NULL
- `size_bytes` INTEGER NOT NULL
- `storage_layout` TEXT NOT NULL
- `created_at` DATETIME NOT NULL
Recommended enum values:
- `single`
- `chunked`
Indexes:
- `idx_file_versions_file_entry_id` on `file_entry_id`
- `idx_file_versions_content_hash` on `content_hash`
### 3.5 `blobs`
Purpose:
- physical storage units referenced by file versions
Columns:
- `id` TEXT PRIMARY KEY
- `file_version_id` TEXT NOT NULL REFERENCES `file_versions`(`id`)
- `blob_index` INTEGER NOT NULL
- `kind` TEXT NOT NULL
- `content_hash` TEXT NOT NULL
- `size_bytes` INTEGER NOT NULL
- `logical_offset` INTEGER NOT NULL DEFAULT 0
- `created_at` DATETIME NOT NULL
Recommended enum values:
- `file`
- `chunk`
- `thumbnail`
- `poster`
Indexes:
- unique `idx_blobs_file_version_index` on (`file_version_id`, `blob_index`)
- `idx_blobs_content_hash` on `content_hash`
### 3.6 `blob_replicas`
Purpose:
- one row per physical copy of a blob on one backend
Columns:
- `id` TEXT PRIMARY KEY
- `blob_id` TEXT NOT NULL REFERENCES `blobs`(`id`)
- `backend_id` TEXT NOT NULL REFERENCES `backends`(`id`)
- `storage_key` TEXT NOT NULL
- `status` TEXT NOT NULL
- `etag` TEXT NULL
- `checksum` TEXT NULL
- `size_bytes` INTEGER NOT NULL
- `last_verified_at` DATETIME NULL
- `created_at` DATETIME NOT NULL
- `updated_at` DATETIME NOT NULL
Recommended enum values:
- `pending`
- `ready`
- `missing`
- `corrupt`
- `offline`
- `failed`
Indexes:
- unique `idx_blob_replicas_blob_backend` on (`blob_id`, `backend_id`)
- `idx_blob_replicas_backend_status` on (`backend_id`, `status`)
- `idx_blob_replicas_storage_key` on `storage_key`
Current implementation status:
- local backend writes now create `blob_replicas` rows during upload finalize
- initial persisted replica status is `ready`
### 3.7 `backends`
Purpose:
- configured storage targets and their policy attributes
Columns:
- `id` TEXT PRIMARY KEY
- `name` TEXT NOT NULL UNIQUE
- `type` TEXT NOT NULL
- `stability_class` TEXT NOT NULL
- `read_priority` INTEGER NOT NULL
- `write_priority` INTEGER NOT NULL
- `is_enabled` BOOLEAN NOT NULL DEFAULT 1
- `config_json` TEXT NOT NULL
- `capacity_hint_bytes` INTEGER NULL
- `last_health_status` TEXT NULL
- `last_health_checked_at` DATETIME NULL
- `created_at` DATETIME NOT NULL
- `updated_at` DATETIME NOT NULL
Recommended `type` values:
- `local`
- `s3`
### 3.8 `placement_policies`
Purpose:
- persist file-class placement intent separately from backend rows
Columns:
- `id` TEXT PRIMARY KEY
- `file_class` TEXT NOT NULL UNIQUE
- `require_local` BOOLEAN NOT NULL DEFAULT 1
- `stable_replica_count` INTEGER NOT NULL DEFAULT 1
- `opportunistic_replica_count` INTEGER NOT NULL DEFAULT 0
- `preferred_backend_ids_json` TEXT NOT NULL DEFAULT `[]`
- `excluded_backend_ids_json` TEXT NOT NULL DEFAULT `[]`
- `max_non_local_size_bytes` INTEGER NULL
- `created_at` DATETIME NOT NULL
- `updated_at` DATETIME NOT NULL
Notes:
- the current backend persists preferred and excluded backend ids as JSON lists
- `max_non_local_size_bytes` can force large files to remain local-only even when stable backends exist
- this table is actively used by reconcile jobs and by `/api/policies/placement/preview`
- `aliyun`
- `baidu`
- `bridge`
Recommended `stability_class` values:
- `local`
- `stable`
- `opportunistic`
Indexes:
- `idx_backends_enabled` on `is_enabled`
- `idx_backends_priority` on (`write_priority`, `read_priority`)
Current implementation status:
- the gateway currently bootstraps one default local backend record automatically
### 3.8 `upload_sessions`
Purpose:
- durable record for `tus` upload lifecycle
Columns:
- `id` TEXT PRIMARY KEY
- `directory_id` TEXT NULL REFERENCES `directories`(`id`)
- `filename` TEXT NULL
- `total_size_bytes` INTEGER NOT NULL
- `received_size_bytes` INTEGER NOT NULL
- `status` TEXT NOT NULL
- `temp_path` TEXT NOT NULL
- `tus_upload_url` TEXT NULL
- `upload_metadata_json` TEXT NULL
- `created_at` DATETIME NOT NULL
- `updated_at` DATETIME NOT NULL
Recommended enum values:
- `created`
- `uploading`
- `uploaded`
- `finalizing`
- `finalized`
- `failed`
- `expired`
Indexes:
- `idx_upload_sessions_status` on `status`
- `idx_upload_sessions_updated_at` on `updated_at`
### 3.9 `upload_session_parts`
Purpose:
- optional upload-part detail for resumable recovery and diagnostics
Columns:
- `id` TEXT PRIMARY KEY
- `upload_session_id` TEXT NOT NULL REFERENCES `upload_sessions`(`id`)
- `part_number` INTEGER NOT NULL
- `byte_offset` INTEGER NOT NULL
- `size_bytes` INTEGER NOT NULL
- `checksum` TEXT NULL
- `status` TEXT NOT NULL
- `created_at` DATETIME NOT NULL
- `updated_at` DATETIME NOT NULL
Indexes:
- unique `idx_upload_session_parts_unique` on (`upload_session_id`, `part_number`)
- `idx_upload_session_parts_upload_id` on `upload_session_id`
### 3.10 `jobs`
Purpose:
- durable background task queue
Columns:
- `id` TEXT PRIMARY KEY
- `kind` TEXT NOT NULL
- `status` TEXT NOT NULL
- `payload_json` TEXT NOT NULL
- `attempt_count` INTEGER NOT NULL DEFAULT 0
- `max_attempts` INTEGER NOT NULL DEFAULT 5
- `run_after` DATETIME NOT NULL
- `last_error` TEXT NULL
- `created_at` DATETIME NOT NULL
- `updated_at` DATETIME NOT NULL
Recommended enum values:
- kinds: `replicate_blob`, `generate_thumbnail`, `verify_replica`, `repair_blob`, `health_check_backend`
- status: `queued`, `running`, `succeeded`, `failed`, `dead`
Indexes:
- `idx_jobs_status_run_after` on (`status`, `run_after`)
- `idx_jobs_kind_status` on (`kind`, `status`)
### 3.11 `preview_artifacts`
Purpose:
- preview resources such as thumbnails and poster images
Columns:
- `id` TEXT PRIMARY KEY
- `file_version_id` TEXT NOT NULL REFERENCES `file_versions`(`id`)
- `artifact_type` TEXT NOT NULL
- `blob_id` TEXT NOT NULL REFERENCES `blobs`(`id`)
- `status` TEXT NOT NULL
- `created_at` DATETIME NOT NULL
- `updated_at` DATETIME NOT NULL
Recommended enum values:
- `thumbnail`
- `poster`
- `pdf_preview`
Indexes:
- unique `idx_preview_artifacts_version_type` on (`file_version_id`, `artifact_type`)
### 3.12 `cache_entries`
Purpose:
- local cache state for blobs and derivatives
Columns:
- `id` TEXT PRIMARY KEY
- `blob_id` TEXT NOT NULL REFERENCES `blobs`(`id`)
- `local_path` TEXT NOT NULL
- `cache_kind` TEXT NOT NULL
- `size_bytes` INTEGER NOT NULL
- `status` TEXT NOT NULL
- `last_accessed_at` DATETIME NOT NULL
- `expires_at` DATETIME NULL
- `created_at` DATETIME NOT NULL
- `updated_at` DATETIME NOT NULL
Recommended enum values:
- cache kinds: `blob`, `preview`, `temp`
- status: `ready`, `partial`, `evicted`, `invalid`
Indexes:
- `idx_cache_entries_blob_id` on `blob_id`
- `idx_cache_entries_last_accessed_at` on `last_accessed_at`
- `idx_cache_entries_expires_at` on `expires_at`
## 4. Relationship Summary
- one `directory` has many child `directories`
- one `directory` has many `file_entries`
- one `file_entry` has many `file_versions`
- one `file_version` has many `blobs`
- one `blob` has many `blob_replicas`
- one `backend` has many `blob_replicas`
- one `upload_session` may become one `file_entry`
- one `job` targets one logical task payload
## 5. Migration Guidance
Recommended migration order:
1. `users`
2. `directories`
3. `file_entries`
4. `file_versions`
5. `backends`
6. `blobs`
7. `blob_replicas`
8. `upload_sessions`
9. `upload_session_parts`
10. `jobs`
11. `placement_policies`
12. `preview_artifacts`
13. `cache_entries`
Guidelines:
- create foreign keys only after referenced tables exist
- add indexes in the same migration as table creation where practical
- keep enum values in application constants for SQLite MVP
## 6. SQLite Notes
- enable foreign keys explicitly
- use WAL mode for better concurrent read behavior
- keep transactions short around upload finalization and job claiming
- avoid long-lived write transactions during file streaming
## 7. Future Schema Evolution
Likely future additions:
- `file_tags`
- `share_links`
- `device_clients`
- `sync_rules`
- `search_documents`
- `backend_metrics`
- `replica_verification_events`
Likely future changes:
- introduce true version history controls
- move some JSON config into typed backend-specific tables if the surface becomes stable
- expand `placement_policies` if policy shape grows beyond file-class controls, preferred/excluded backend sets, and size-aware replica caps

105
docs/development.md
View File

@ -0,0 +1,105 @@
# Development
This document captures the development workflow, test expectations, and UI
quality rules for Iron.
## Setup
```bash
UV_CACHE_DIR=.uv-cache uv venv .venv
UV_CACHE_DIR=.uv-cache uv pip install --python .venv/bin/python -e '.[dev]'
npm install
```
## Run Locally
```bash
npm run build
.venv/bin/python -m uvicorn app.main:app --reload
```
Open:
```text
http://127.0.0.1:8000/app
```
Default development credentials:
```text
username: admin
password: changeme-iron
```
## Tests
Full suite:
```bash
.venv/bin/python -m pytest
```
Current expected result:
```text
66 passed
```
Browser E2E:
```bash
.venv/bin/python scripts/ui_playwright_smoke.py
```
Pytest E2E entry:
```bash
.venv/bin/python -m pytest tests/e2e/test_web_ui_playwright.py
```
## UI Change Standard
Any meaningful Web UI change must be validated against a real running service.
Required workflow:
1. build frontend assets with `npm run build`
2. run `./.venv/bin/python scripts/ui_playwright_smoke.py`
3. review screenshots in `output/playwright/`
4. fix layout, spacing, overflow, or interaction issues
5. rerun the Playwright flow
The E2E flow currently covers:
- login
- files page render
- create folder
- upload text file
- authenticated file download
- upload image file
- authenticated inline image preview
- search
- uploads, recycle bin, storage, and jobs page render
- browser `/api` 4xx/5xx failures during the flow
## Runtime Artifacts
Do not commit:
- `.venv/`
- `.uv-cache/`
- `node_modules/`
- `output/`
- `.iron-storage/`
- `.iron-temp/`
- local databases
- Python caches
## Documentation Rules
- Keep `README.md` concise and user-facing.
- Keep contributor instructions in `CONTRIBUTING.md`.
- Keep coding-agent instructions in `AGENTS.md`.
- Keep deeper project docs in `docs/`.
- Use relative links in Markdown.
- Update docs when changing setup, commands, architecture, or user-visible behavior.

233
docs/handoff.md
View File

@ -1,233 +0,0 @@
# Iron Engineering Handoff
Last updated: 2026-04-15
## 1. What This Document Is For
This is the fastest way for another AI or engineer to continue work on Iron without re-reading the full project history.
Read this first, then use the other docs as references:
- product scope: [requirements.md](/Users/bytedance/iron/docs/requirements.md:1)
- gap analysis: [mvp-status.md](/Users/bytedance/iron/docs/mvp-status.md:1)
- implementation architecture: [technical-architecture-draft.md](/Users/bytedance/iron/docs/technical-architecture-draft.md:1)
- API details: [api-design.md](/Users/bytedance/iron/docs/api-design.md:1)
- schema details: [database-schema.md](/Users/bytedance/iron/docs/database-schema.md:1)
## 2. Current Truth
The project is no longer architecture-only.
It already has:
- async FastAPI backend
- SQLite + SQLAlchemy ORM
- Alembic migrations
- local auth with persisted bearer-token sessions
- directories, files, recycle bin, rename, move
- `tus` uploads
- local backend persistence
- S3 runtime replication and remote fallback reads
- declarative reconcile jobs
- metadata export, validate, integrity, restore-plan, and guarded import
- placement policy persistence and placement preview API
- a frontend app scaffold at `/app` built with `Vite + React + TypeScript`
- product routes for files, uploads, recycle bin, storage, and jobs
- a desktop-first Web UI with file list, contextual inspector, upload queue, recycle bin, storage, and jobs surfaces
- Python Playwright E2E coverage for login, create folder, upload, authenticated download, authenticated image preview, search, and page rendering
It now has a product-usable Web frontend foundation, but it still needs user-facing hardening before calling it a polished release.
The most honest stage label is still:
- `Product MVP Candidate`
## 3. Current Verified Baseline
Use `uv` and the local virtualenv:
```bash
UV_CACHE_DIR=.uv-cache uv venv .venv
UV_CACHE_DIR=.uv-cache uv pip install --python .venv/bin/python -e '.[dev]'
```
Run tests with:
```bash
export UV_CACHE_DIR=.uv-cache && .venv/bin/python -m pytest
```
Current expected result:
- `66 passed`
If new work changes behavior, update this number in:
- [README.md](/Users/bytedance/iron/README.md:1)
- [mvp-status.md](/Users/bytedance/iron/docs/mvp-status.md:1)
- this file
## 4. What Is Implemented
### 4.1 Backend Capabilities
- auth:
- bootstrap admin user
- login
- logout
- current user
- namespace:
- create/list directories
- mixed children listing
- file detail
- file and directory rename/move
- recycle-bin delete/restore for files
- upload:
- `tus` create/head/patch
- finalize to file metadata
- storage:
- local backend object persistence
- S3 adapter runtime path
- remote fallback read into local cache
- preview:
- image/PDF inline preview
- generic video stream through HTTP Range
- preview artifact rows and generation jobs
- ops:
- backend list/create/check/disable
- jobs list/detail/retry/run
- enqueue full reconcile
- enqueue backend health checks
- file reconcile
- recovery:
- metadata export
- metadata validate
- metadata integrity
- restore plan with validation token
- guarded metadata import
### 4.2 Placement Controls
This area is complete enough for backend MVP work.
Current placement behavior supports:
- policies by file class
- local requirement flag
- stable and opportunistic replica counts
- ordered preferred backend ids
- excluded backend ids
- optional `max_non_local_size_bytes`
- placement preview API
- reconcile jobs converging toward policy-selected targets
Main files:
- [app/services/policy_service.py](/Users/bytedance/iron/app/services/policy_service.py:1)
- [app/api/routes/policies.py](/Users/bytedance/iron/app/api/routes/policies.py:1)
- [tests/test_policy_routes.py](/Users/bytedance/iron/tests/test_policy_routes.py:1)
- [tests/test_placement_jobs.py](/Users/bytedance/iron/tests/test_placement_jobs.py:1)
## 5. Main Remaining Work
### 5.1 P0: Product Frontend
This is the main next focus.
Needed next:
- deepen the new frontend foundation into a more polished daily-usable product
- improve directory navigation and breadcrumbs
- harden upload progress UX and directory targeting
- improve preview and detail-panel behavior
- refine mobile behavior and browser session handling
- extend Playwright E2E coverage as each product flow becomes real
### 5.2 P1: Backend Hardening
These are still useful, but they are no longer the main gating path:
- stronger preview derivative generation
- post-restore operational tooling
- richer replica verification and repair visibility
- more sophisticated cache observability and eviction
### 5.3 Post-MVP / Roadmap
- Aliyun adapter
- Baidu adapter
- desktop client
- mobile client
- share links
- search
## 6. Recommended Next Sequence
If another AI continues from here, the recommended order is:
1. preserve the current desktop-first drive UX and avoid reverting to dashboard-style empty panels
2. keep browser file access going through authenticated blob fetches, not naked `/api/files/...` media URLs
3. extend Playwright E2E coverage when adding preview, download, upload, or mutation flows
4. only after product hardening, return to backend polish such as richer preview derivatives
## 7. Important Repo Realities
These details are easy to get wrong if someone only reads the older architecture docs:
- config is currently a lightweight cached dataclass module, not `pydantic-settings`
- the frontend source now lives in `frontend/` and builds into `app/web/dist`
- `tus` endpoints live under `/files`, outside the normal `/api` JSON namespace
- the backend uses persisted bearer tokens today
- metadata import is intentionally guarded by:
- `confirm_replace=true`
- `validation_token` from restore-plan
## 8. Where To Look In Code
High-signal entry points:
- app bootstrap: [app/main.py](/Users/bytedance/iron/app/main.py:1)
- API composition: [app/api/router.py](/Users/bytedance/iron/app/api/router.py:1)
- auth: [app/services/auth_service.py](/Users/bytedance/iron/app/services/auth_service.py:1)
- files: [app/services/file_service.py](/Users/bytedance/iron/app/services/file_service.py:1)
- uploads: [app/services/upload_service.py](/Users/bytedance/iron/app/services/upload_service.py:1)
- jobs: [app/services/job_service.py](/Users/bytedance/iron/app/services/job_service.py:1)
- storage: [app/services/storage_service.py](/Users/bytedance/iron/app/services/storage_service.py:1)
- placement: [app/services/policy_service.py](/Users/bytedance/iron/app/services/policy_service.py:1)
- frontend entry: [frontend/src/app/App.tsx](/Users/bytedance/iron/frontend/src/app/App.tsx:1)
- frontend routes: [frontend/src/app/router.tsx](/Users/bytedance/iron/frontend/src/app/router.tsx:1)
- frontend API client: [frontend/src/lib/api.ts](/Users/bytedance/iron/frontend/src/lib/api.ts:1)
- frontend styles: [frontend/src/styles/app.css](/Users/bytedance/iron/frontend/src/styles/app.css:1)
- Playwright E2E: [scripts/ui_playwright_smoke.py](/Users/bytedance/iron/scripts/ui_playwright_smoke.py:1)
- pytest E2E entry: [tests/e2e/test_web_ui_playwright.py](/Users/bytedance/iron/tests/e2e/test_web_ui_playwright.py:1)
- built app entry: [app/web/dist/index.html](/Users/bytedance/iron/app/web/dist/index.html:1)
## 9. Testing Guidance
When continuing development:
- keep using `uv` + `.venv`
- prefer adding tests in the same slice as each feature
- run Python Playwright for UI-affecting work
- update docs after meaningful milestone changes
- do not let README, MVP status, and actual test count drift apart
Current test files:
- [tests/test_auth_routes.py](/Users/bytedance/iron/tests/test_auth_routes.py:1)
- [tests/test_backend_routes.py](/Users/bytedance/iron/tests/test_backend_routes.py:1)
- [tests/test_directory_routes.py](/Users/bytedance/iron/tests/test_directory_routes.py:1)
- [tests/test_export_routes.py](/Users/bytedance/iron/tests/test_export_routes.py:1)
- [tests/test_file_routes.py](/Users/bytedance/iron/tests/test_file_routes.py:1)
- [tests/test_job_routes.py](/Users/bytedance/iron/tests/test_job_routes.py:1)
- [tests/test_placement_jobs.py](/Users/bytedance/iron/tests/test_placement_jobs.py:1)
- [tests/test_policy_routes.py](/Users/bytedance/iron/tests/test_policy_routes.py:1)
- [tests/test_system_routes.py](/Users/bytedance/iron/tests/test_system_routes.py:1)
- [tests/test_upload_routes.py](/Users/bytedance/iron/tests/test_upload_routes.py:1)
- [tests/test_web_routes.py](/Users/bytedance/iron/tests/test_web_routes.py:1)
## 10. Suggested Handoff Prompt
If another AI is about to continue, a good short prompt is:
> Read `/Users/bytedance/iron/docs/handoff.md` first, treat it as the current source of truth, then continue from the frontend-first sequence unless new user instructions override it.

473
docs/mvp-status.md
View File

@ -1,473 +0,0 @@
# Iron MVP Status
Last updated: 2026-04-15
## 1. Purpose
This document fixes one question in a concrete way:
- what should count as Iron's **usable MVP**
- what is already implemented
- what still blocks that MVP today
It is the live checkpoint between:
- [requirements.md](/Users/bytedance/iron/docs/requirements.md:1)
- [technical-architecture-draft.md](/Users/bytedance/iron/docs/technical-architecture-draft.md:1)
- the actual code and tests in `app/` and `tests/`
## 2. Current Conclusion
The honest current answer is:
- Iron already has a **real backend foundation**, not just design documents
- Iron now has a **browser-usable product foundation**
- the current code is best described as a **Product MVP Candidate**
Why it is not yet a polished release:
- the Web UI supports the core drive flows, but still needs real-user hardening and polish
- browser auth exists, but session setup, password management, and onboarding are still basic
- `local + S3` both have real runtime behavior, but advanced repair/visibility remains basic
- declarative reconcile, health-check, and preview-generation jobs now exist
- metadata export, validation, integrity checks, restore-plan, and guarded import now exist
Priority decision after re-evaluation:
- the remaining **true MVP blockers** are now mostly product-surface blockers
- the remaining backend work is important, but most of it is now `P1 polish / hardening`, not `P0 blocker`
## 3. What "Usable MVP" Means
For this project, a usable MVP should mean:
- one person can deploy a single gateway node
- the person can log in from a browser
- the person can browse, upload, rename, move, delete, restore, download, and preview files
- the person does not need to care which backend stores a file
- at least two backend types work at runtime, not only in configuration
- the system can survive one non-primary backend failure without losing metadata control
- the system exposes enough operational visibility to inspect backend health and failed tasks
This is still smaller than the long-term vision. It does **not** require yet:
- desktop sync client
- mobile-native app
- public share links
- full-text search
- cross-file deduplication
- erasure coding
- advanced media indexing
## 4. Current System Capability
As of today, Iron already has these working backend slices.
### 4.1 Core Platform
- FastAPI service bootstrap
- async SQLAlchemy ORM with SQLite
- Alembic migrations
- `uv`-based local development flow
- automated test suite
### 4.2 Namespace and Metadata
- root directory bootstrap
- create directory
- list mixed directory and file children
- file detail lookup
- rename directory
- move directory with descendant `path_key` updates
- rename file
- move file
- recycle-bin file deletion
- restore file from recycle bin
### 4.3 Upload and Persistence
- `tus` upload creation
- upload offset inspection
- sequential `PATCH` upload
- finalize upload into file metadata
- persist finalized content into the default `local` backend
- create `blob` and `blob_replica` records
### 4.4 Read Path
- file download
- file streaming
- HTTP Range support for local persisted objects
- inline preview for images and PDFs stored in the local backend
### 4.5 Backend Management
- list backends
- create backend config entries for `local` and `s3`
- local backend health check
- S3 backend health check
- backend disable flow
- default local backend bootstrap
### 4.6 Multi-Backend and Operations
- S3 runtime replication path
- fallback reads from ready remote replicas
- remote blob cache materialization
- policy-based replication target selection
- durable SQLite job records
- in-process job polling loop
- job list/detail/retry/run APIs
- declarative file reconcile API
- backend health-check enqueue API
- full-system reconcile enqueue API
- metadata export, validate, integrity, and import APIs
- staged restore-plan workflow
- preview artifact generation jobs
- preview artifacts are exposed in file detail
### 4.7 Test Baseline
Current verified state:
- `66` automated tests passing
- route coverage exists for the currently implemented backend slices
- Python Playwright E2E covers login, create folder, upload, authenticated download, authenticated inline image preview, search, and major page rendering
### 4.8 Web UI Shell
Implemented:
- browser entry at `/app`
- frontend app scaffold with `Vite + React + TypeScript`
- local login form
- token-backed session bootstrap in browser storage
- files route with listing, selection, preview panel, and basic file actions
- uploads route with browser queue on top of existing `tus` flow
- recycle-bin route
- storage route
- jobs route
Status:
- real product foundation and MVP candidate
- still needs user hardening, mobile refinement, richer media behavior, and broader E2E coverage before a release label
## 5. MVP Gap Matrix
The table below is the most useful way to read the project right now.
| Capability | MVP expectation | Current state | Gap level |
| --- | --- | --- | --- |
| Web login and session | browser-usable auth flow | login route and route guard implemented, still local-storage based | medium |
| Web file browser | daily-usable browser entry | desktop-first files workspace with list/grid, pagination, inspector, dialogs, menus | low |
| Namespace operations | browse, rename, move, delete, restore | backend API implemented | low |
| Upload flow | resumable upload from browser | browser upload route and files-page upload implemented, targeting still needs polish | medium |
| Download and preview | browser download and preview | authenticated browser download and image preview covered by E2E; PDF/video paths still need deeper coverage | medium |
| Runtime multi-backend storage | at least `local + S3` really work | `local + S3` both operate at runtime | low |
| Backend abstraction | provider records and management | partially implemented | medium |
| On-demand remote retrieval | read from another backend when local copy is absent | implemented for ready remote replicas | medium |
| Background tasks | replication, repair, health polling | declarative reconcile, health-check, and preview jobs implemented | low |
| Metadata export and restore | backup and recovery path | export, validate, integrity, restore-plan, and import implemented | low |
| Video-first experience | stream-friendly playback, poster/thumbnail | generic stream exists, preview artifact pipeline is still basic | medium |
| Cache model | explicit local cache behavior | remote-fetch cache exists, policy is still basic | medium |
| Repair visibility | inspect and retry failed work | job visibility, retry, and file reconcile enqueue are implemented | low |
| Aliyun/Baidu adapters | roadmap target backends | not implemented | roadmap gap |
## 6. What Is Already Good Enough
These areas should no longer be treated as blank or hypothetical:
- backend service skeleton
- metadata model direction
- upload protocol choice
- recycle-bin semantics
- local object persistence
- local read, stream, and basic preview path
- namespace mutation rules
- baseline test discipline
This matters because Iron is already past the "architecture-only" phase. The remaining work is now mostly about turning a solid backend core into a user-usable product slice.
## 7. Biggest MVP Blockers
If the question is "what prevents someone from really using Iron as their personal cloud drive today?", the answer is below.
### 7.1 Web UI Exists And Is Now The Main Hardening Surface
Iron now has a browser entry point with real drive workflows.
Implemented:
- login page
- session bootstrap
- root directory listing
- backend listing
- logout
- files workspace with desktop drive layout
- create folder, upload, rename, move, delete, download
- image preview through authenticated blob URLs
- uploads, recycle bin, storage, and jobs pages
- Playwright screenshots and E2E coverage
Still missing:
- backend management forms
- better browser-side session handling
- broader PDF/video preview E2E
- mobile/narrow-screen polish
- richer empty-state and operational guidance
MVP priority:
- `P0 hardening`
### 7.2 Authentication Is Browser-Usable But Still Basic
The backend has a real local auth slice, and the Web app now has a browser-facing login/session flow.
Implemented:
- bootstrap local admin user
- login API
- bearer-token session issuance
- current-user API
- logout API
- protected file, directory, backend, upload routes
- browser login page
- authenticated API client
- authenticated blob fetches for download and preview
Still missing:
- password rotation or setup UX
- clearer bootstrap credential onboarding
MVP priority:
- `P0 blocker`, but much smaller than before because the backend auth layer already exists
### 7.3 Multi-Backend Runtime Is Real But Still Basic
This area now has a real first implementation, but it is still early.
Implemented:
- real S3 read/write support
- remote fallback reads that can materialize a missing local blob into cache
- S3 health checks
- persisted placement policies by file class
- placement preview API for product-facing inspection
- preferred backend ordering and explicit backend exclusion
- size-aware non-local replica caps
- reconcile jobs now converge toward policy-selected targets instead of a single hardcoded target shape
Still missing:
- richer verification and replica selection rules
MVP priority:
- `P1 important`
- no longer a first-order blocker for the first usable release
### 7.4 Background Task Engine Exists But Is Not Complete
Iron now has a real first task engine, but it is still narrower than the intended MVP.
Implemented:
- declarative reconcile jobs
- file reconcile enqueue flow
- backend health-check enqueue flow
- task inspection and retry flow
- in-process polling loop
Still missing:
- richer preview derivative generation beyond the current basic artifact flow
MVP priority:
- `P1 important`
- the task substrate itself is no longer a blocker
### 7.5 Metadata Recovery Exists And Has Basic Safety Checks
This is now substantially addressed.
Implemented:
- export command or API
- validation API for metadata snapshots
- integrity check API for live runtime state
- restore-plan dry-run workflow
- import/restore API for metadata snapshots
- restore-triggered follow-up reconcile and health-check jobs
Still missing:
- richer post-restore operational tooling
MVP priority:
- `P1 important`
- the restore contract already exists; remaining work is hardening
## 8. Priority Reset
After re-evaluating the current repository, the most useful classification is:
### 8.1 P0: Must Finish Before Release Label
- harden Web UI flows with Playwright-backed regression
- browser-facing session UX beyond local-storage basics
- broader PDF/video/download E2E coverage
### 8.2 P1: Should Finish Immediately Before or After MVP Cut
- stronger preview derivative generation
- post-restore operational hardening and checks
### 8.3 P2: Important But No Longer MVP-Critical
- Aliyun backend
- Baidu backend
- richer cache policy
- stronger replica verification sophistication
- advanced media workflows beyond current preview artifacts
## 9. Secondary Gaps
These do not block the first shippable MVP as hard as the items above, but they still matter.
### 9.1 Media Experience Is Only Basic
Current state:
- image preview works
- PDF preview works
- generic video stream works through HTTP Range
- preview artifacts are tracked and generated through background jobs
Still missing:
- richer thumbnails
- richer video posters
- better unsupported-codec fallback behavior
### 9.2 Cache Behavior Is Not Yet a Product Feature
The architecture talks about on-demand retrieval and local caching, but current runtime behavior does not yet expose a real cache policy.
Still missing:
- explicit cache store
- retention policy
- eviction policy
- cache observability
### 9.3 Consumer Cloud Backends Are Still Roadmap Work
Aliyun and Baidu remain important to the long-term goal, but they should be treated as **post-first-MVP** unless they become easier than S3 in practice.
## 10. Recommended MVP Cutline
To avoid drifting scope, the recommended first **product-usable MVP** should be defined as:
### 10.1 Must Have
- local authentication
- Web UI for file browsing and basic backend management
- `tus` upload from the browser
- upload, rename, move, delete, restore, download
- image and PDF preview
- video playback through range-capable gateway streaming
- `local` backend runtime support
- `S3` backend runtime support
- one policy-driven placement flow with:
- file-class placement rules
- preferred and excluded backend controls
- local plus optional non-local replicas
- backend health check visibility
- metadata export command or API
- one repair or retry path for failed replicas
### 10.2 Can Wait Until After MVP
- Aliyun runtime adapter
- Baidu runtime adapter
- thumbnail/poster generation pipeline
- richer cache intelligence
- share links
- search
- desktop client shell
- mobile app shell
## 11. Stage Labels
To keep internal communication honest, use these labels:
### 11.1 Current Stage
`Product MVP Candidate`
Meaning:
- backend APIs exist
- local auth exists
- local persistence works
- core file lifecycle works
- browser product flows exist and are E2E-tested
- still needs polish, hardening, and more coverage before release
### 11.2 Target Next Stage
`Product MVP`
Meaning:
- browser-usable
- authenticated
- local + S3 both operational
- at least one simple replication flow
- metadata export and basic repair visibility
### 11.3 Later Stage
`Multi-Backend Personal Drive`
Meaning:
- consumer cloud backends join the runtime pool
- remote fallback retrieval works
- background orchestration is richer
- media workflows are stronger
## 12. Recommended Gap-Closing Order
If the goal is to reach the first usable MVP with the shortest path, the order should be:
1. harden the current browser product flows through Playwright-backed regression
2. add deeper PDF/video preview and download coverage
3. deepen browser session handling on top of the current auth backend
4. improve preview derivative generation and post-restore hardening
5. refine operational pages based on real user tasks
6. add the first consumer-cloud backend
## 13. Final Assessment
The most accurate one-line summary today is:
> Iron has a credible backend and a browser-usable Web product foundation; the next work is hardening and polish.
The shortest path to changing that is not "more backend breadth everywhere". It is:
- browser session hardening
- deeper Web UI regression coverage
- stronger media and restore hardening
- repair and restore
For implementation sequencing and repo-level handoff notes, see [handoff.md](/Users/bytedance/iron/docs/handoff.md:1).
Once those land, Iron can honestly be called a first usable personal cloud-drive MVP.

69
docs/product.md
View File

@ -0,0 +1,69 @@
# Product
Iron is a self-hosted personal cloud drive gateway. The product goal is to give
one person a familiar browser drive experience while keeping storage placement,
repair, and metadata ownership under their control.
## Current Stage
Iron is a **Product MVP Candidate**.
Implemented:
- local authentication and browser login
- file browser with folders, search, sort, pagination, and an inspector panel
- create folder, upload, rename, move, delete, restore, download, and image preview
- uploads, recycle bin, storage, and jobs pages
- local backend persistence
- S3 runtime replication and remote fallback reads
- placement policies and reconcile jobs
- metadata export, validate, integrity, restore-plan, and guarded import
- Python Playwright E2E coverage for the core browser flow
Still needs hardening before a polished release:
- better browser session and bootstrap credential UX
- deeper PDF/video preview coverage
- richer preview derivative generation
- stronger post-restore and repair visibility
- mobile and narrow-screen refinement
## MVP Cutline
A first usable MVP should let one person:
- deploy a single gateway node
- log in from a browser
- browse, upload, preview, download, rename, move, delete, and restore files
- use local storage and at least one remote backend
- inspect backend health and failed work
- export and validate metadata for recovery
Not required for the first MVP:
- desktop sync client
- mobile native app
- public share links
- full-text search
- cross-file deduplication
- erasure coding
- advanced media indexing
## Roadmap
Near-term:
- harden Web UI flows with Playwright-backed regression
- improve session management and onboarding
- expand PDF/video preview tests
- improve upload targeting and progress feedback
- refine storage and jobs pages around real user tasks
Later:
- Aliyun and Baidu adapters
- richer cache policy
- share links
- desktop and mobile clients
- stronger media workflows
- advanced repair orchestration

397
docs/requirements.md
View File

@ -1,397 +0,0 @@
# Iron Requirements
## 1. Product Overview
`Iron` is a personal cloud drive system built around a self-hosted `gateway` service.
Users access a unified file space through a Web application in the first phase. The gateway manages metadata, local cache, upload/download pipelines, preview streaming, and data placement across multiple storage backends.
The product goal is not to be a backup tool first. It is a daily-use personal drive focused on:
- ease of use
- resistance to storage provider failure
- low storage cost
- optional privacy protections
The system should allow storage across a mix of:
- Baidu Netdisk
- Aliyun Drive
- S3-compatible object storage
- local directories
The long-term vision is to expose the same core capabilities to desktop and mobile clients, while keeping storage control and metadata ownership in the user's hands.
## 2. Product Goals
### 2.1 Primary Goals
- Provide a single virtual drive space across multiple heterogeneous storage backends.
- Support daily file management, not only backup and restore.
- Allow large media files and documents to coexist in one product.
- Support on-demand cloud retrieval so files do not need to be fully cached locally.
- Support preview-first workflows, especially for photos, videos, and PDFs.
- Keep system-critical metadata under user control instead of delegating it to any one storage vendor.
- Allow unstable or free providers to participate as secondary or cold-tier storage.
### 2.2 Priority Order
1. Ease of use
2. Resistance to provider shutdown or lock-in
3. Low cost
4. Privacy and encryption
## 3. Target Users and Usage Context
### 3.1 Primary User
- a single user or household
- has one always-on host running the gateway
- accesses files from PCs and phones in a home network, with possible future remote access
### 3.2 Main Data Types
- large video files
- large photo libraries
- documents
- PDFs
- mixed personal archives
### 3.3 Usage Style
- browse files from a Web UI
- upload from browser or client
- preview before download when possible
- let the system decide actual storage placement
- tolerate heavier local infrastructure if it improves experience
## 4. Product Principles
- Users see one logical drive, not multiple backend silos.
- Storage placement is system-managed, not user-managed in normal workflows.
- Metadata must remain recoverable even if one backend becomes unavailable.
- Stable backends and unstable backends must be treated differently by policy.
- Media experience should be optimized explicitly, not treated as a side effect of generic file storage.
- The MVP should prefer understandable replication over complex erasure coding.
## 5. Scope Definition
### 5.1 In Scope for the Product
- file browsing and directory navigation
- upload and download
- delete, rename, move, copy
- on-demand retrieval from remote storage
- local cache management
- image preview
- video preview via streaming
- PDF preview
- multi-backend storage placement
- backend health monitoring
- metadata backup and recovery tooling
- background jobs for sync, repair, and replication
### 5.2 Out of Scope for MVP
- real-time collaborative editing
- multi-tenant enterprise permissions
- public sharing links with strong abuse controls
- fully offline-first desktop synchronization
- true distributed peer-to-peer storage nodes
- POSIX-complete filesystem semantics
- erasure coding as a required storage strategy
- full-text OCR and semantic search
## 6. System Context
### 6.1 Phase 1 Deployment Model
- one self-hosted gateway process
- one Web client
- one SQLite database
- one local cache directory
- multiple configured storage backends
### 6.2 Future Deployment Model
- gateway remains the control plane
- desktop clients may add richer local sync and mount capabilities
- mobile clients may provide browsing, preview, upload, and camera ingestion
- metadata database may move from SQLite to a networked database if needed
## 7. Functional Requirements
### 7.1 File Space and Navigation
- The system must expose a single hierarchical directory tree.
- The system must allow creating, renaming, moving, and deleting directories.
- The system must allow listing files with pagination and sorting.
- The system must show key metadata including name, size, type, timestamps, and cache status.
- The system must decouple logical path from backend storage location.
### 7.2 Upload
- The system must support browser-based upload.
- The system must support resumable upload at least at the gateway boundary.
- The system must support large-file upload without requiring full in-memory buffering.
- The system must classify uploaded files by size and media type to choose storage strategy.
- The system must allow asynchronous replication after initial successful ingestion.
### 7.3 Download
- The system must allow direct file download from the Web UI.
- The system must support on-demand fetch from remote backends if data is not in local cache.
- The system must expose download progress and failure states.
### 7.4 Preview
- The system must support image preview.
- The system must support PDF preview.
- The system must support video preview through HTTP range-compatible streaming.
- The system should generate thumbnails for common media types.
- The system should avoid full-file fetch when previewing large video files if the backend layout permits partial reads.
### 7.5 Cache Management
- The gateway must maintain a local cache for hot files and preview artifacts.
- The system must track cache state separately from logical file state.
- The system must support configurable cache eviction policies.
- The system should allow pinning specific files or directories for local retention in future versions.
### 7.6 Multi-Backend Storage
- The system must support multiple backend types at once.
- The system must support backend policies such as `stable`, `opportunistic`, and `local`.
- The system must support at least one primary copy and optional secondary copies.
- The system must track where each object or file replica is stored.
- The system must detect backend unavailability and mark replicas accordingly.
### 7.7 Metadata and Recovery
- The system must maintain its own metadata database independent of backend-native directory layouts.
- The system must support export and backup of metadata.
- The system must support integrity verification between metadata records and backend objects.
- The system must support repair jobs that recreate missing replicas when another valid replica exists.
### 7.8 Jobs and Operations
- The system must support background jobs for uploads, replication, thumbnail generation, and repair.
- The system must expose job status in the UI or API.
- The system must retain structured logs for user-visible failures.
### 7.9 Backend Management
- The system must allow adding, updating, disabling, and deleting backend configurations.
- The system must display backend status, capacity hints if available, and last health-check result.
- The system should support backend weighting and placement rules.
## 8. Non-Functional Requirements
### 8.1 Usability
- A user should not need to understand backend topology during normal file operations.
- Common file operations should be understandable from a single Web interface.
- Preview should be first-class for photo, video, and PDF content.
### 8.2 Reliability
- Metadata must not depend on a single unstable third-party backend.
- The system should tolerate temporary backend outages and recover later.
- Background jobs must be restart-safe.
### 8.3 Performance
- Listing a directory from metadata should not require live backend enumeration.
- Upload and download pipelines should stream rather than fully buffer large files.
- Preview start latency for cached or stable-backend media should be low enough for normal personal use.
### 8.4 Extensibility
- Backend integrations must be pluggable.
- Data model must allow migration from SQLite to another relational database.
- API design must allow future desktop and mobile clients.
### 8.5 Security
- Authentication is required for the Web UI and APIs.
- Secrets for backend credentials must not be stored in plaintext in user-facing logs.
- Encryption should be supported in design even if not mandatory for every backend path in MVP.
## 9. Storage Model Strategy
### 9.1 MVP Strategy
- documents and small files are stored as whole-file objects
- photos are stored as whole-file originals plus generated thumbnails
- large videos may use chunked storage if needed for upload resilience and streamability
- replication is preferred over erasure coding
### 9.2 Future Strategy
- content-addressed chunks for large files
- optional object packing for small files
- deduplication across identical content
- tier-aware migration policies
- optional erasure coding for selected backend groups
## 10. Backend Policy Model
Each backend should have policy metadata, for example:
- backend type
- stability class
- cost class
- write priority
- read priority
- capacity hint
- health status
- credential state
Suggested policy classes:
- `stable`: trusted for primary storage, such as S3 or a managed local directory on a reliable host
- `opportunistic`: useful for secondary replicas or cold storage, such as free consumer cloud drives
- `local`: fast cache-adjacent storage on the gateway host
## 11. MVP Definition
### 11.1 MVP Objective
Deliver a working single-user personal cloud drive that feels usable every day through a Web UI, with support for multi-backend storage, on-demand file retrieval, and media preview.
### 11.2 MVP Feature Set
- Web authentication
- file and directory browsing
- upload, download, rename, move, delete
- backend management for local directory, S3, Aliyun Drive, and Baidu Netdisk
- SQLite metadata store
- local cache directory
- background job queue inside the gateway process
- image preview and thumbnail generation
- PDF preview
- video preview through range-capable gateway streaming
- replica tracking
- backend health checks
- metadata export
- repair job for missing replicas
### 11.3 MVP Constraints
- single-user only
- one gateway node only
- no desktop sync client yet
- no mobile-native app yet
- no public share links
- no full-text search
- no cross-file deduplication requirement
- no erasure coding requirement
### 11.4 MVP Success Criteria
- A user can upload files and browse them without caring which backend stores them.
- A user can preview photos, PDFs, and many common videos from the browser.
- A user can survive loss of one configured secondary backend without losing metadata control.
- A user can inspect and retry failed background tasks.
- A user can back up metadata and recover system state onto a new gateway host.
Current implementation status:
- see [mvp-status.md](/Users/bytedance/iron/docs/mvp-status.md:1) for the live gap analysis against this MVP definition
- the current repository should be described as a `Product MVP Candidate`, not yet a polished release
- after re-evaluation, the main remaining `P0` items are Web UI and browser-facing auth/session UX
- the backend now has persisted placement policy controls for file classes, preferred/excluded backends, and size-aware non-local replica caps
- most remaining backend work has moved into `P1 hardening`
## 12. End-State Vision
### 12.1 Product Vision
Iron becomes a full personal storage control plane for heterogeneous storage providers, with one logical namespace, rich media workflows, policy-driven placement, and resilient metadata ownership.
### 12.2 End-State Capabilities
- Web, desktop, and mobile clients
- remote access support
- selective offline sync
- folder pinning
- advanced media indexing and timeline views
- content deduplication
- optional end-to-end encryption modes
- policy-based lifecycle management
- automatic migration between hot, warm, and cold tiers
- stronger repair and self-healing workflows
- share links and fine-grained permissions
- import and migration tools from common cloud drives
- optional mount support on desktop systems
## 13. Version Roadmap
### 13.1 V0: Foundation
- repository setup
- gateway skeleton
- metadata schema
- local backend
- basic Web UI shell
Current status:
- mostly achieved for backend foundation
- a browser-usable Web UI now exists, but it still needs real-user hardening, broader E2E coverage, and polish
### 13.2 V1: MVP
- file operations
- local cache
- S3 backend
- Aliyun backend
- Baidu backend
- preview pipeline
- jobs and repair
Current interpretation:
- the first shippable V1 should prioritize `auth + Web UI + local + S3 + simple replication + metadata export`
- the backend side of `local + S3 + simple replication + metadata recovery contract` is already largely in place
- Aliyun and Baidu remain in the roadmap, but are not required to call the first public version a usable MVP
### 13.3 V2: Usability Expansion
- better backend policies
- desktop client shell
- remote access hardening
- richer media views
- pinning and offline support
### 13.4 V3: Advanced Storage Intelligence
- deduplication
- packed small-object storage
- lifecycle movement
- stronger repair orchestration
- optional erasure-coded backend groups
## 14. Risks and Product Assumptions
### 14.1 Assumptions
- The user accepts a heavy local gateway service.
- Free or consumer cloud backends may be unstable and are therefore not trusted as the only source of critical metadata.
- Web-first access is sufficient for MVP.
### 14.2 Risks
- Consumer cloud providers may change APIs, rate limits, or anti-abuse rules.
- Preview latency may be inconsistent for files that only exist on slow or unstable backends.
- Browser upload behavior for very large files may require resumable protocols early.
- Media streaming behavior can vary by codec and container format.
## 15. Open Questions for Technical Design
- What auth model should the first gateway use: simple local account, reverse-proxy auth, or pluggable auth?
- Should large video files use chunking from day one, or only after file size thresholds are exceeded?
- Should backend adapters be implemented natively or through an abstraction layer such as rclone-compatible gateways?
- How should local cache quotas and eviction policies be tuned?
- What metadata export format should be treated as the recovery contract?
- How much encryption should be mandatory in MVP versus optional by backend policy?

874
docs/technical-architecture-draft.md
View File

@ -1,874 +0,0 @@
# Iron Technical Architecture Draft
## 1. Design Goals
This draft translates the product requirements into an implementation-oriented architecture for the first development phase.
The design aims to:
- support a Web-first product with a self-hosted gateway
- keep the codebase simple enough for rapid iteration
- isolate backend-specific complexity behind adapters
- use SQLite first without blocking future database migration
- optimize for media preview and daily file operations
## 2. Recommended MVP Technical Stack
## 2.1 Backend
Recommended:
- language: `Python`
- HTTP framework: `FastAPI`
- data access: `SQLAlchemy 2.0 ORM`
- migrations: `Alembic`
- schema validation: `Pydantic v2`
- background jobs: in-process worker pool backed by database job records
- object hashing: standard SHA-256
- config: env-driven settings via a small cached settings module
Why:
- Python fits rapid iteration well and has a strong ecosystem for Web APIs, file tooling, storage SDKs, and media processing.
- `FastAPI` provides typed request and response models, automatic OpenAPI generation, and a clean async-friendly API layer.
- MVP is a single-node gateway, so Python is a good fit as long as upload, download, and preview paths are implemented with async streaming semantics instead of full buffering.
- The architecture should keep storage I/O and worker logic explicit so hot paths remain understandable and optimizable.
## 2.2 Frontend
Recommended:
- framework: `React` with `Next.js` App Router or `Vite + React`
- UI library: minimal component primitives such as `shadcn/ui` or `Radix UI`
- data fetching: `TanStack Query`
- video and file preview: native browser capabilities first
Recommendation between the two:
- choose `Vite + React` if the Web app is a pure SPA against the gateway API
- choose `Next.js` only if server-rendered routing or later integrated auth flows are clearly desired
For this project, the current recommendation is:
- `Vite + React`
Reason:
- the gateway already owns the API
- the app is operational rather than content-heavy
- simpler deployment and fewer moving parts for MVP
## 2.3 Storage Integrations
Recommended strategy:
- implement a common backend adapter interface in the gateway
- use native SDKs when stable and practical
- explicitly allow a bridge adapter layer when consumer cloud providers are unstable or poorly documented
- keep bridge usage behind the same storage adapter interface so the rest of the system stays readable
Proposed initial adapter strategy:
- local directory: native file operations
- S3: AWS SDK compatible implementation
- Aliyun Drive: adapter with bridge mode support
- Baidu Netdisk: adapter with bridge mode support
Important note:
- consumer cloud providers are the highest churn part of the system
- keep their adapters in isolated packages and do not let provider-specific concepts leak into core domain models
- bridge mode is preferred over contaminating core services with provider-specific protocol quirks
## 3. System Architecture
```text
Browser
|
v
Web UI
|
v
Gateway API
|
+--> Auth Module
+--> File Service
+--> Preview Service
+--> Backend Service
+--> Job Service
|
+--> Metadata Repository (SQLite)
+--> Cache Manager (local disk)
+--> Storage Engine
|
+--> Placement Planner
+--> Transfer Manager
+--> Replica Manager
+--> Backend Adapters
|- Local
|- S3
|- Aliyun
|- Baidu
```
## 4. Layering Recommendation
Use a layered async modular monolith for MVP.
Suggested top-level structure:
```text
/app
/api
/core
/domain
/services
/repositories
/models
/schemas
/adapters
/db
/storage
/jobs
/preview
/cache
/placement
/workers
/utils
/alembic
/scripts
/tests
/web
/docs
```
Alternative package layout if you prefer stricter layering:
```text
/app
/config
/domain
/service
/repository
/adapter
/http
/db
/storage
/job
/preview
/cache
/placement
```
Layer responsibilities:
- `domain`: core entities, enums, policies, service contracts
- `services`: business orchestration such as upload, move, preview, repair
- `repositories`: persistence operations over SQLite
- `models`: SQLAlchemy ORM models
- `schemas`: API request and response models
- `adapter/db`: SQLite implementations
- `adapter/storage`: backend-specific storage adapters
- `api`: FastAPI routers, dependencies, and HTTP entrypoints
- `jobs`: durable job definitions and scheduling logic
- `cache`: local cache and preview artifact management
- `placement`: backend selection and replication policy logic
This keeps the codebase simple and still allows later extraction.
Async design rules:
- API handlers must be async.
- Repository methods should be async-facing.
- Backend adapters should expose async APIs.
- Blocking SDKs or filesystem-heavy operations must be isolated behind explicit threadpool boundaries.
- No request path should rely on hidden blocking I/O in utility code.
## 5. Core Domain Model
The system should not model files as mere backend paths. Use stable IDs.
Core entities:
- `User`
- `Directory`
- `FileEntry`
- `FileVersion`
- `Blob`
- `BlobReplica`
- `Backend`
- `UploadSession`
- `Job`
- `PreviewArtifact`
- `CacheEntry`
### 5.1 Entity Semantics
- `Directory`: logical folder node in the virtual filesystem
- `FileEntry`: logical file identity tied to a path parent and display name
- `FileVersion`: immutable content snapshot of a file entry
- `Blob`: the stored binary unit, whole-file or chunk object
- `BlobReplica`: one physical copy of a blob on one backend
- `Backend`: configured storage target and its policy metadata
- `PreviewArtifact`: thumbnail, poster frame, or derivative preview asset
- `CacheEntry`: local materialization status for blobs or preview artifacts
This separation matters because:
- rename and move should not create a new file identity
- future version history should be possible
- one logical file may map to multiple blobs
- one blob may have multiple replicas
## 6. SQLite Schema Draft
Below is the recommended MVP schema direction. Naming can still change, but the concepts should remain stable.
### 6.1 `users`
Purpose:
- local authentication identity for MVP
Suggested columns:
- `id` TEXT PRIMARY KEY
- `username` TEXT NOT NULL UNIQUE
- `password_hash` TEXT NOT NULL
- `created_at` DATETIME NOT NULL
- `updated_at` DATETIME NOT NULL
### 6.2 `directories`
Purpose:
- logical folder tree
Suggested columns:
- `id` TEXT PRIMARY KEY
- `parent_id` TEXT NULL
- `name` TEXT NOT NULL
- `path_key` TEXT NOT NULL UNIQUE
- `created_at` DATETIME NOT NULL
- `updated_at` DATETIME NOT NULL
- `deleted_at` DATETIME NULL
Notes:
- `path_key` is a normalized path index for fast lookups
- root directory can be a special fixed row
### 6.3 `file_entries`
Purpose:
- logical file objects visible in the namespace
Suggested columns:
- `id` TEXT PRIMARY KEY
- `directory_id` TEXT NOT NULL
- `name` TEXT NOT NULL
- `mime_type` TEXT NULL
- `size_bytes` INTEGER NOT NULL
- `current_version_id` TEXT NOT NULL
- `is_deleted` BOOLEAN NOT NULL DEFAULT 0
- `created_at` DATETIME NOT NULL
- `updated_at` DATETIME NOT NULL
- UNIQUE (`directory_id`, `name`)
### 6.4 `file_versions`
Purpose:
- immutable content version records
Suggested columns:
- `id` TEXT PRIMARY KEY
- `file_entry_id` TEXT NOT NULL
- `content_hash` TEXT NOT NULL
- `size_bytes` INTEGER NOT NULL
- `storage_layout` TEXT NOT NULL
- `created_at` DATETIME NOT NULL
Suggested `storage_layout` values:
- `single`
- `chunked`
### 6.5 `blobs`
Purpose:
- binary storage units
Suggested columns:
- `id` TEXT PRIMARY KEY
- `file_version_id` TEXT NOT NULL
- `blob_index` INTEGER NOT NULL
- `kind` TEXT NOT NULL
- `content_hash` TEXT NOT NULL
- `size_bytes` INTEGER NOT NULL
- `logical_offset` INTEGER NOT NULL DEFAULT 0
- `created_at` DATETIME NOT NULL
Suggested `kind` values:
- `file`
- `chunk`
- `thumbnail`
- `poster`
### 6.6 `blob_replicas`
Purpose:
- track every physical copy
Suggested columns:
- `id` TEXT PRIMARY KEY
- `blob_id` TEXT NOT NULL
- `backend_id` TEXT NOT NULL
- `storage_key` TEXT NOT NULL
- `status` TEXT NOT NULL
- `etag` TEXT NULL
- `checksum` TEXT NULL
- `size_bytes` INTEGER NOT NULL
- `last_verified_at` DATETIME NULL
- `created_at` DATETIME NOT NULL
- `updated_at` DATETIME NOT NULL
Suggested `status` values:
- `pending`
- `ready`
- `missing`
- `corrupt`
- `offline`
- `failed`
### 6.7 `backends`
Purpose:
- backend configuration and policy metadata
Suggested columns:
- `id` TEXT PRIMARY KEY
- `name` TEXT NOT NULL UNIQUE
- `type` TEXT NOT NULL
- `stability_class` TEXT NOT NULL
- `read_priority` INTEGER NOT NULL
- `write_priority` INTEGER NOT NULL
- `is_enabled` BOOLEAN NOT NULL DEFAULT 1
- `config_json` TEXT NOT NULL
- `capacity_hint_bytes` INTEGER NULL
- `last_health_status` TEXT NULL
- `last_health_checked_at` DATETIME NULL
- `created_at` DATETIME NOT NULL
- `updated_at` DATETIME NOT NULL
Notes:
- secrets should be encrypted before being stored in `config_json`
### 6.8 `upload_sessions`
Purpose:
- tus-based resumable upload tracking between browser and gateway
Suggested columns:
- `id` TEXT PRIMARY KEY
- `directory_id` TEXT NOT NULL
- `filename` TEXT NOT NULL
- `total_size_bytes` INTEGER NOT NULL
- `received_size_bytes` INTEGER NOT NULL
- `status` TEXT NOT NULL
- `temp_path` TEXT NOT NULL
- `tus_upload_url` TEXT NULL
- `upload_metadata_json` TEXT NULL
- `created_at` DATETIME NOT NULL
- `updated_at` DATETIME NOT NULL
### 6.8a `upload_session_parts`
Purpose:
- track uploaded ranges or parts when local resumable state is needed
Suggested columns:
- `id` TEXT PRIMARY KEY
- `upload_session_id` TEXT NOT NULL
- `part_number` INTEGER NOT NULL
- `byte_offset` INTEGER NOT NULL
- `size_bytes` INTEGER NOT NULL
- `checksum` TEXT NULL
- `status` TEXT NOT NULL
- `created_at` DATETIME NOT NULL
- `updated_at` DATETIME NOT NULL
- UNIQUE (`upload_session_id`, `part_number`)
### 6.9 `jobs`
Purpose:
- durable background work tracking
Suggested columns:
- `id` TEXT PRIMARY KEY
- `kind` TEXT NOT NULL
- `status` TEXT NOT NULL
- `payload_json` TEXT NOT NULL
- `attempt_count` INTEGER NOT NULL DEFAULT 0
- `max_attempts` INTEGER NOT NULL DEFAULT 5
- `run_after` DATETIME NOT NULL
- `last_error` TEXT NULL
- `created_at` DATETIME NOT NULL
- `updated_at` DATETIME NOT NULL
Suggested `kind` values:
- `replicate_blob`
- `generate_thumbnail`
- `verify_replica`
- `repair_blob`
- `health_check_backend`
### 6.10 `preview_artifacts`
Purpose:
- preview resources mapped back to files or file versions
Suggested columns:
- `id` TEXT PRIMARY KEY
- `file_version_id` TEXT NOT NULL
- `artifact_type` TEXT NOT NULL
- `blob_id` TEXT NOT NULL
- `status` TEXT NOT NULL
- `created_at` DATETIME NOT NULL
- `updated_at` DATETIME NOT NULL
### 6.11 `cache_entries`
Purpose:
- local cache tracking
Suggested columns:
- `id` TEXT PRIMARY KEY
- `blob_id` TEXT NOT NULL
- `local_path` TEXT NOT NULL
- `cache_kind` TEXT NOT NULL
- `size_bytes` INTEGER NOT NULL
- `status` TEXT NOT NULL
- `last_accessed_at` DATETIME NOT NULL
- `expires_at` DATETIME NULL
- `created_at` DATETIME NOT NULL
- `updated_at` DATETIME NOT NULL
## 7. API Design Direction
Use a clean REST API for MVP. gRPC is unnecessary at this stage.
Suggested route groups:
- `/api/auth`
- `/api/files`
- `/api/directories`
- `/api/uploads`
- `/api/previews`
- `/api/backends`
- `/api/jobs`
- `/api/system`
### 7.1 Key Endpoints
Examples:
- `POST /api/auth/login`
- `GET /api/directories/:id/children`
- `POST /api/directories`
- `POST /api/uploads`
- `HEAD /api/uploads/:id`
- `PATCH /api/uploads/:id`
- `POST /api/uploads/:id/finalize`
- `GET /api/files/:id`
- `GET /api/files/:id/download`
- `GET /api/files/:id/stream`
- `GET /api/files/:id/thumbnail`
- `POST /api/files/:id/move`
- `POST /api/files/:id/rename`
- `DELETE /api/files/:id`
- `GET /api/backends`
- `POST /api/backends`
- `POST /api/backends/:id/check`
- `GET /api/jobs`
- `POST /api/jobs/:id/retry`
## 8. Upload Pipeline
Recommended MVP flow:
1. Browser creates a `tus` upload resource on the gateway.
2. Browser uploads file content through `tus` patch requests.
3. Gateway writes incoming bytes to a temporary local ingest path.
4. Gateway tracks upload progress in the database.
5. When upload completes, gateway computes metadata such as mime type, hash, and size.
6. Gateway creates logical records in the database.
7. Gateway writes the first required replica synchronously to a chosen primary backend.
8. Gateway marks the file visible when minimum durability criteria are met.
9. Gateway schedules secondary replication and preview generation jobs.
Important MVP rule:
- visibility should happen only after at least one durable backend write succeeds
Recommended upload protocol choice:
- adopt `tus` for resumable uploads in MVP
- keep upload session state explicit in the database so the gateway can coordinate file creation, placement, and recovery
- treat `tus` as the ingress protocol, while keeping storage placement and replica logic as internal concerns
- add a gateway-side finalize step so a completed `tus` upload becomes a managed logical file only after validation and first durable write
## 9. Download and Stream Pipeline
### 9.1 Download
1. User requests file download.
2. Gateway resolves current file version and blob layout.
3. Gateway checks local cache.
4. If cache misses, gateway chooses the best available replica based on backend health and priority.
5. Gateway streams the response to the client while optionally filling cache.
### 9.2 Video Streaming
Recommended strategy:
- support HTTP Range requests on the gateway
- map byte ranges to full-file or chunked blob reads
- prefer local cache, then stable backends, then opportunistic backends
Why:
- this is the most standard Web-compatible approach
- native browser video players work with it
- it avoids inventing a custom preview transport too early
## 10. Preview Pipeline
Recommended preview support for MVP:
- images: generate thumbnails and serve original or scaled versions
- PDFs: browser inline preview using direct file or proxy response
- videos: poster image plus range-based playback
Suggested preview job flow:
1. file ingest completes
2. preview worker inspects mime type
3. thumbnail or poster extraction runs if applicable
4. derivative asset is stored as a blob
5. preview artifact record is created
Tooling options:
- `ffmpeg` for video poster generation and optional transcoding later
- image library for thumbnails
Recommendation:
- do not include full video transcoding in MVP
- stick to poster extraction and native playback for supported source formats
## 11. Placement and Replica Policy
Recommended MVP policy engine:
- every backend has a policy score
- backend selection uses stability class, enabled state, health state, and priority
- every file has a placement rule selected by file class
Suggested first placement classes:
- `critical-metadata`
- `document`
- `photo`
- `video`
- `cold-archive`
Suggested default behavior:
- metadata exports: local + stable backend
- documents: stable primary, optional opportunistic secondary
- photos: stable primary, opportunistic secondary if available
- videos: stable primary, optional async secondary depending on size and backend capacity
## 12. Authentication Recommendation
For MVP:
- one local user table
- session cookie auth or signed token auth
Recommendation:
- use a simple local auth model first
- the current backend implementation uses persisted bearer-token sessions
- the future Web UI may wrap that with cookie handling or translate it into cookie-based sessions
- avoid introducing OAuth or external identity providers in MVP
Reason:
- local self-hosted single-user setup does not need auth complexity yet
## 13. Secrets and Security
Recommended MVP measures:
- encrypt backend credentials before storing them in SQLite
- use a gateway master key from environment or local config file
- redact sensitive fields in logs
- require login for all file and backend management routes
## 14. Caching Strategy
Suggested MVP cache layers:
- ingest temp area
- blob cache
- preview cache
Suggested cache policy:
- LRU-like eviction based on size budget and recent access
- separate quotas for blob cache and preview cache
Do not make cache semantics too smart in MVP. Keep them observable and debuggable.
## 15. Background Jobs
Recommended architecture:
- jobs stored in SQLite
- one scheduler loop claims runnable jobs
- small worker pool executes jobs
- failed jobs are retried with backoff
- workers may call synchronous SDKs through thread pools where async-native clients are unavailable
Why not a separate queue system yet:
- too much operational complexity for single-node MVP
- SQLite durability is enough for this stage if jobs are idempotent
Job design rule:
- every job must be safe to retry
## 16. Adapter Interface Draft
Define a storage adapter interface around core object operations.
Suggested interface shape:
```python
from typing import Protocol
class StorageAdapter(Protocol):
adapter_type: str
async def check(self) -> None: ...
async def stat(self, key: str) -> "ObjectInfo": ...
async def put(
self,
key: str,
stream,
size: int,
options: "PutOptions",
) -> "ObjectInfo": ...
async def get(
self,
key: str,
byte_range: "ByteRange | None" = None,
) -> tuple[object, "ObjectInfo"]: ...
async def delete(self, key: str) -> None: ...
```
Notes:
- use `key` as the physical storage identifier
- keep adapters object-oriented even for local filesystem storage
- do not let adapters know about directories, files, or logical paths
- if a provider SDK is synchronous, isolate it inside the adapter and offload blocking work explicitly
Suggested adapter split:
- `NativeStorageAdapter`: direct provider SDK or protocol implementation
- `BridgeStorageAdapter`: talks to an external bridge service or compatibility layer
Core services should not need to know which one is in use.
## 17. Physical Key Strategy
Do not store files in backends by original user path.
Recommended key pattern:
- `blobs/sha256_prefix/full_hash`
- `previews/file_version_id/artifact_type`
- `exports/date/export_id`
Why:
- avoids backend path rename costs
- reduces coupling to logical namespace changes
- makes replication and repair much simpler
## 18. Suggested First Implementation Order
1. config loading and app bootstrap
2. SQLite migrations
3. domain models and repository interfaces
4. local backend adapter
5. file ingest pipeline
6. file listing and basic Web UI
7. blob cache
8. download pipeline
9. image and PDF preview
10. S3 adapter
11. job system
12. Aliyun adapter
13. Baidu adapter
14. video poster generation and range streaming polish
## 19. Key Architecture Decisions To Confirm
These are the most important choices still worth discussing before coding:
- Backend language: `Python` is confirmed.
- Frontend stack: confirm `Vite + React`.
- Database access: `SQLAlchemy ORM` is confirmed.
- Consumer cloud adapters: bridge-capable adapter strategy is confirmed.
- Upload protocol: `tus` is confirmed.
- Credential encryption: define whether to use a local master key file or environment variable only.
- Async model: async-first architecture is confirmed; blocking operations must be isolated explicitly.
## 20. Recommendation Summary
For the first implementation phase, the strongest recommendation is:
- backend: `Python + FastAPI`
- frontend: `Vite + React`
- database: `SQLite`
- data access: `SQLAlchemy ORM`
- jobs: in-process durable workers
- architecture: layered async modular monolith
- storage strategy: replication first, chunking only where beneficial
- upload strategy: `tus` resumable upload with gateway-managed session records
- preview strategy: native browser preview plus range-based streaming
- backend order: local, S3, Aliyun, Baidu
- adapter policy: native where clean, bridge where provider complexity would otherwise damage maintainability
- Python libraries to prefer: `SQLAlchemy 2.0`, `Alembic`, `Pydantic v2`, `httpx`, `boto3` or compatible S3 SDK, and `ffmpeg` integration for previews
This combination is the best balance between simplicity, stream handling, self-hosted deployment, and future extensibility.
## 21. Implemented Foundation Status
For a product-level view of what is still missing before Iron can be called a usable MVP, see [mvp-status.md](/Users/bytedance/iron/docs/mvp-status.md:1).
The repository now includes the following implemented baseline pieces:
- `uv`-managed local virtual environment workflow
- FastAPI application bootstrap and routing shell
- async SQLAlchemy engine setup for SQLite
- initial ORM entity definitions for the core schema
- first Alembic migration for the initial metadata model
- readiness check backed by a real database ping
- baseline automated tests for config, system routes, and migration application
- first auth slice:
- bootstrap local admin user
- login, logout, and current-user APIs
- bearer-token session persistence and route protection
- first Web shell slice:
- `/app` browser entry
- login form and session bootstrap
- root directory listing
- backend listing
- first metadata feature slice for directories:
- root directory bootstrap
- create directory
- list child directories
- first file metadata slice:
- internal file metadata creation service for future upload finalization
- mixed directory and file child listings
- file detail API
- recycle-bin file deletion and restore flow
- first upload slice:
- tus upload session creation
- offset inspection
- sequential patch uploads into temp ingest storage
- finalize flow wired into file metadata creation
- finalize flow persists the primary blob into the default local backend
- local `blob_replicas` records are created for finalized uploads
- finalize enqueues non-local replication jobs
- first download slice:
- file download endpoint resolves local ready replicas
- downloaded content is served from persisted local backend objects
- remote fallback can materialize ready replicas into local cache
- first stream slice:
- file stream endpoint resolves local ready replicas
- HTTP Range support for partial content reads
- first preview slice:
- inline preview endpoint for images and PDFs persisted to local backend
- first backend-visibility slice:
- backend list endpoint for configured backends
- first backend-management slice:
- backend creation for local and s3 config metadata
- local and s3 backend health checks
- backend disable flow
- first jobs-and-ops slice:
- durable SQLite job records
- in-process polling loop
- job list/detail/retry/run APIs
- backend health-check enqueue flow
- declarative file reconcile enqueue flow
- metadata export and import APIs
- first policy slice:
- persisted placement policies by file class
- preferred and excluded backend controls
- size-aware non-local replica caps
- placement preview API for product-facing inspection
- upload finalize uses policy-selected secondary targets
- reconciliation jobs converge actual replicas toward desired state
- first namespace-mutation slice:
- directory rename
- directory move with descendant path updates
- file rename
- file move
From a product perspective, the next logical step is no longer more basic metadata CRUD. It is:
- deepen the browser shell into a fuller Web UI
- deepen browser-facing session handling on top of the current auth backend
- add post-restore hardening and stronger preview derivative generation
- add the first consumer-cloud backend
For the product-level gap assessment, use [mvp-status.md](/Users/bytedance/iron/docs/mvp-status.md:1) as the source of truth.
For day-to-day continuation guidance and current repo reality, use [handoff.md](/Users/bytedance/iron/docs/handoff.md:1) first.

137
docs/ui-playwright-standard.md
View File

@ -1,137 +0,0 @@
# Iron UI Playwright Standard
Last updated: 2026-04-14
## 1. Purpose
All future Iron Web UI changes must be validated against a real running service, using Python Playwright flows and screenshot review.
This is the default UI modification standard for the project.
## 2. Required Workflow For Any UI Change
Before considering a UI change complete:
1. build the frontend assets
2. start the real local service
3. run the Python Playwright regression flow
4. generate screenshots into `output/playwright/`
5. review screenshots for layout, spacing, responsiveness, and interaction states
6. fix issues found in the screenshots or browser flow
7. rerun the Playwright flow after the fixes
Do not treat UI work as complete based only on code review or unit tests.
## 3. Required Validation Levels
Every meaningful UI change should validate:
- login flow
- files workspace render
- primary file operation flow affected by the change
- upload flow if the change touches file workflows
- screenshots of the affected page states
## 4. Tooling Standard
Use:
- Python Playwright code for repeatable regression checks
- a real locally running Iron service
- screenshot artifacts for visual inspection
Preferred command:
```bash
./.venv/bin/python scripts/ui_playwright_smoke.py
```
Pytest entrypoint for CI or full local regression:
```bash
./.venv/bin/python -m pytest tests/e2e/test_web_ui_playwright.py
```
Artifacts must be written to:
```text
output/playwright/
```
## 5. Screenshot Review Checklist
Every screenshot review should check:
- overall balance of the layout
- spacing consistency
- text truncation or wrapping problems
- button grouping and hierarchy
- panel heights and overflow behavior
- empty-state clarity
- table/list readability
- dialog layout and action placement
- mobile or narrow-width resilience when the changed surface is responsive
## 6. Interaction Review Checklist
Every flow review should check:
- click targets are obvious
- primary actions are easy to find
- selection state is clear
- navigation state is preserved where expected
- success feedback is visible
- error feedback is actionable
- dialogs do not rely on browser-native prompts
- repetitive tasks do not require unnecessary clicks
## 7. Regression Scope For Iron
Current baseline regression should cover:
- login
- files page
- create folder
- upload from the files page
- authenticated file download
- authenticated inline image preview
- no failing `/api` browser requests during the flow
- files search/filter visibility
- uploads page render
As new UI capabilities are added, extend the Python Playwright script to cover them.
## 8. Expected Outputs
Each run should produce:
- pass/fail console output
- screenshots
- clear error exit code on failure
Recommended screenshot set:
- `login.png`
- `files-empty-or-initial.png`
- `files-after-create-folder.png`
- `files-after-upload.png`
- `uploads-page.png`
## 9. Standard For Future UI Optimization
For future layout or interaction polish:
- first reproduce the current state in Playwright
- capture before screenshots
- make a focused UI change
- capture after screenshots
- compare the changed states visually
- keep the regression script updated if the flow meaningfully changes
## 10. Non-Negotiable Rule
No future UI optimization should be merged or considered complete without:
- a real service run
- Python Playwright flow validation
- screenshot review

342
docs/web-ui-technical-design.md
View File

@ -1,342 +0,0 @@
# Iron Web UI Technical Design
Last updated: 2026-04-15
## 1. Goal
This document defines the first real Web UI implementation for Iron as a product, not as a backend demo shell.
The design target is:
- a browser-based personal cloud drive that feels familiar to users of mainstream drive products
- a file-first experience, with storage and operational controls available when needed
- a frontend architecture that can grow without forcing a backend rewrite
## 2. Product Direction
Iron should feel like:
- a calm personal drive for everyday browsing, upload, preview, and organization
- a storage-aware control surface when the user wants to inspect health, jobs, or replica state
Iron should not feel like:
- a generic admin dashboard
- a developer-only operations console
- a backend demo that exposes storage complexity too early
The default user mental model should be:
- there is one drive
- files live in one namespace
- storage placement is handled by the system
- operational details are inspectable but not required for normal use
## 3. UX Principles
- file browsing is the center of the product
- preview before download whenever possible
- complex storage details stay secondary
- important system state stays visible but low-noise
- common actions should be reachable in one or two clicks
- the desktop layout should resemble familiar drive products, while mobile should collapse into simple list-detail flows
## 4. Primary Information Architecture
Recommended primary navigation:
1. Files
2. Uploads
3. Recycle Bin
4. Storage
5. Jobs
Secondary entry points:
- search
- account/session menu
- future settings/policies entry
## 5. Layout Model
The app should use a three-zone desktop layout:
- left sidebar: primary navigation and directory shortcuts
- main workspace: listing, toolbar, breadcrumbs, upload flows
- right detail panel: preview, metadata, storage status, contextual actions
Mobile and narrow tablet should collapse to:
- top bar
- main content
- bottom sheet or dedicated route for file detail
## 6. Visual Direction
Reference class:
- mainstream Web drive layout
- product-first, not enterprise-heavy
- spacious, legible, and calm
Current visual language:
- ownCloud-inspired file-product structure, without copying exact visual assets
- desktop-first left navigation, central file workspace, and right contextual panels
- compact first-screen layouts with pagination for long lists instead of page-length scrolling
- restrained neutral surfaces, blue primary actions, high-contrast text, and light borders
- UI changes must be reviewed from real Playwright screenshots before being considered complete
## 7. Frontend Technical Stack
Recommended stack:
- `Vite`
- `React`
- `TypeScript`
- `React Router`
- `TanStack Query`
- `Radix UI` primitives only when needed later
State strategy:
- server state: `TanStack Query`
- local UI state: React state and context
- persistent auth/session state: small local storage wrapper
Rationale:
- Iron is a long-lived SPA-style product
- the backend already owns API and routing responsibility
- Vite keeps build and iteration simple
- React Router and TanStack Query are enough for current complexity
## 8. App Structure
Recommended source layout:
```text
frontend/
src/
app/
App.tsx
router.tsx
providers.tsx
components/
layout/
feedback/
files/
storage/
features/
auth/
files/
uploads/
recycle-bin/
storage/
jobs/
lib/
api/
auth/
format/
utils/
routes/
styles/
```
Guidance:
- keep route-level pages under `routes/`
- keep API wrappers under `lib/api/`
- keep reusable visual building blocks under `components/`
- keep feature-specific hooks and rendering helpers in each feature folder
## 9. Route Plan
Initial route map:
- `/app/login`
- `/app/files`
- `/app/files/:directoryId`
- `/app/uploads`
- `/app/recycle-bin`
- `/app/storage`
- `/app/jobs`
Behavior:
- `/app` should redirect authenticated users to `/app/files`
- unauthenticated users should be sent to `/app/login`
## 10. Data and API Integration
The first Web UI should reuse the current backend API surface.
Auth:
- `POST /api/auth/login`
- `GET /api/auth/me`
- `POST /api/auth/logout`
Files and directories:
- `GET /api/directories/{directory_id}/children`
- `GET /api/files/{file_id}`
- `GET /api/files/{file_id}/download`
- `GET /api/files/{file_id}/preview`
- `GET /api/files/{file_id}/stream`
- `POST /api/files/{file_id}/rename`
- `POST /api/files/{file_id}/move`
- `DELETE /api/files/{file_id}`
- `POST /api/files/{file_id}/restore`
- `GET /api/files/recycle-bin`
- `POST /api/directories`
- `POST /api/directories/{directory_id}/rename`
- `POST /api/directories/{directory_id}/move`
Uploads:
- `POST /files`
- `HEAD /files/{upload_id}`
- `PATCH /files/{upload_id}`
- `POST /api/uploads/{upload_id}/finalize`
Operations:
- `GET /api/backends`
- `POST /api/backends/{backend_id}/check`
- `POST /api/backends/{backend_id}/disable`
- `GET /api/jobs`
- `GET /api/jobs/{job_id}`
- `POST /api/jobs/{job_id}/retry`
- `POST /api/jobs/run-pending`
- `POST /api/jobs/enqueue-health-checks`
- `POST /api/jobs/enqueue-full-reconcile`
Browser content access:
- JSON APIs use bearer-token `Authorization` headers through the shared API client
- file download, preview, and stream actions must not use naked `/api/files/...` URLs in media tags or `window.open`
- browser preview/download should fetch the file with authorization, create an object URL, and revoke it after use
- Playwright E2E must cover authenticated file access whenever these flows change
## 11. MVP Page Scope
### 11.1 Files
Must include:
- breadcrumb navigation
- file/folder listing
- list and grid view toggle
- select item
- right-side detail panel
- preview for image, PDF, and video-capable files
- rename, move, delete, download actions
- create folder action
### 11.2 Uploads
Must include:
- file picker
- drag and drop
- upload queue
- progress display
- finalize into current directory
### 11.3 Recycle Bin
Must include:
- deleted file list
- restore action
- metadata visibility
### 11.4 Storage
Must include:
- backend cards/list
- health state
- last checked time
- disable action
- trigger check action
### 11.5 Jobs
Must include:
- recent jobs list
- status emphasis
- retry for retryable/failed jobs
- trigger run pending
- enqueue health checks
- enqueue full reconcile
## 12. Interaction Model
Toolbar behavior in Files:
- primary action: upload
- secondary actions: new folder, refresh, view toggle
- selection actions move into detail panel and contextual menus
Detail panel behavior:
- opens on item selection
- shows metadata immediately
- shows preview when supported
- keeps actions grouped by safety
Feedback behavior:
- use inline toasts or banners for successful mutations
- keep destructive confirmations explicit
- show loading skeletons instead of blank content
## 13. Accessibility and Responsiveness
The first Web UI should include:
- keyboard reachable navigation
- visible focus states
- semantic buttons, forms, lists, and headings
- color contrast suitable for long sessions
- responsive behavior for widths down to mobile phone layouts
## 14. Delivery Sequence
Recommended implementation order:
1. frontend scaffold and shared layout
2. auth flow and route guards
3. files page and detail panel
4. uploads page and upload client
5. recycle bin
6. storage and jobs pages
7. polish, responsiveness, and interaction refinement
## 15. Non-Goals For This Slice
Do not block the first Web UI on:
- advanced settings UI
- full-text search
- share links
- policy editor UX
- multi-window drag/drop complexity
- perfect media derivative support
The first goal is a strong file-first personal drive UI built on the current backend.
## 16. Current Implementation Snapshot
Implemented as of 2026-04-15:
- login and authenticated route guard
- files workspace with local folder navigation, breadcrumbs, search, sort, list/grid toggle, pagination, contextual row actions, dialogs, and inspector
- authenticated browser download and inline image preview via blob-backed object URLs
- uploads page with drag/drop, file picker, queue, target folder tree, and pagination
- recycle bin, storage, and jobs pages with compact desktop workspaces
- Python Playwright smoke/E2E flow with screenshots under `output/playwright/`
- pytest E2E entry at `tests/e2e/test_web_ui_playwright.py`
Write Preview
Loading…
Cancel
Save