iron/docs/database-schema.md

Iron Database Schema

1. Overview

This document defines the initial SQLite schema for Iron MVP.

Goals:

• support a single logical file namespace
• separate logical metadata from physical storage placement
• track replicas and background jobs durably
• support tus resumable uploads
• keep the schema portable enough for future migration to PostgreSQL

Conventions:

• IDs are stored as TEXT
• timestamps are stored as UTC datetimes
• soft delete is preferred for logical user-facing entities in MVP
• provider-specific details belong in JSON columns only when no stable relational shape exists

2. Entity Overview

Main tables:

• users
• auth_sessions
• directories
• file_entries
• file_versions
• blobs
• blob_replicas
• backends
• upload_sessions
• upload_session_parts
• jobs
• placement_policies
• preview_artifacts
• cache_entries

3. Table Definitions

3.1 users

Purpose:

• local gateway login identity

Columns:

• id TEXT PRIMARY KEY
• username TEXT NOT NULL UNIQUE
• password_hash TEXT NOT NULL
• created_at DATETIME NOT NULL
• updated_at DATETIME NOT NULL

3.2 auth_sessions

Purpose:

• persisted login sessions for bearer-token auth

Columns:

• id TEXT PRIMARY KEY
• user_id TEXT NOT NULL REFERENCES users(id)
• token_hash TEXT NOT NULL UNIQUE
• expires_at DATETIME NOT NULL
• revoked_at DATETIME NULL
• created_at DATETIME NOT NULL
• updated_at DATETIME NOT NULL

Indexes:

• idx_auth_sessions_user_status on (user_id, revoked_at, expires_at)
• unique idx_auth_sessions_token_hash on token_hash

Notes:

• raw session tokens should never be stored directly
• the current implementation stores a SHA-256 token hash and returns the raw bearer token only at login time

3.3 directories

Purpose:

• logical folder hierarchy

Columns:

• id TEXT PRIMARY KEY
• parent_id TEXT NULL REFERENCES directories(id)
• name TEXT NOT NULL
• path_key TEXT NOT NULL UNIQUE
• created_at DATETIME NOT NULL
• updated_at DATETIME NOT NULL
• deleted_at DATETIME NULL

Constraints:

• unique sibling name constraint should be enforced in application logic or by an additional composite unique index later

Indexes:

• idx_directories_parent_id on parent_id
• idx_directories_path_key on path_key

3.4 file_entries

Purpose:

• logical files visible to users

Columns:

• id TEXT PRIMARY KEY
• directory_id TEXT NOT NULL REFERENCES directories(id)
• name TEXT NOT NULL
• mime_type TEXT NULL
• size_bytes INTEGER NOT NULL
• current_version_id TEXT NULL
• is_deleted BOOLEAN NOT NULL DEFAULT 0
• deleted_at DATETIME NULL
• created_at DATETIME NOT NULL
• updated_at DATETIME NOT NULL

Constraints:

• unique active filename per directory

Indexes:

• unique idx_file_entries_directory_name on (directory_id, name, is_deleted)
• idx_file_entries_current_version_id on current_version_id
• idx_file_entries_deleted_at on deleted_at

Notes:

• file deletion uses recycle-bin semantics in MVP
• active and deleted files may temporarily share the same logical name in one directory because the uniqueness rule only protects active entries

3.5 file_versions

Purpose:

• immutable snapshots of file content

Columns:

• id TEXT PRIMARY KEY
• file_entry_id TEXT NOT NULL REFERENCES file_entries(id)
• content_hash TEXT NOT NULL
• size_bytes INTEGER NOT NULL
• storage_layout TEXT NOT NULL
• created_at DATETIME NOT NULL

Recommended enum values:

• single
• chunked

Indexes:

• idx_file_versions_file_entry_id on file_entry_id
• idx_file_versions_content_hash on content_hash

3.5 blobs

Purpose:

• physical storage units referenced by file versions

Columns:

• id TEXT PRIMARY KEY
• file_version_id TEXT NOT NULL REFERENCES file_versions(id)
• blob_index INTEGER NOT NULL
• kind TEXT NOT NULL
• content_hash TEXT NOT NULL
• size_bytes INTEGER NOT NULL
• logical_offset INTEGER NOT NULL DEFAULT 0
• created_at DATETIME NOT NULL

Recommended enum values:

• file
• chunk
• thumbnail
• poster

Indexes:

• unique idx_blobs_file_version_index on (file_version_id, blob_index)
• idx_blobs_content_hash on content_hash

3.6 blob_replicas

Purpose:

• one row per physical copy of a blob on one backend

Columns:

• id TEXT PRIMARY KEY
• blob_id TEXT NOT NULL REFERENCES blobs(id)
• backend_id TEXT NOT NULL REFERENCES backends(id)
• storage_key TEXT NOT NULL
• status TEXT NOT NULL
• etag TEXT NULL
• checksum TEXT NULL
• size_bytes INTEGER NOT NULL
• last_verified_at DATETIME NULL
• created_at DATETIME NOT NULL
• updated_at DATETIME NOT NULL

Recommended enum values:

• pending
• ready
• missing
• corrupt
• offline
• failed

Indexes:

• unique idx_blob_replicas_blob_backend on (blob_id, backend_id)
• idx_blob_replicas_backend_status on (backend_id, status)
• idx_blob_replicas_storage_key on storage_key

Current implementation status:

• local backend writes now create blob_replicas rows during upload finalize
• initial persisted replica status is ready

3.7 backends

Purpose:

• configured storage targets and their policy attributes

Columns:

• id TEXT PRIMARY KEY
• name TEXT NOT NULL UNIQUE
• type TEXT NOT NULL
• stability_class TEXT NOT NULL
• read_priority INTEGER NOT NULL
• write_priority INTEGER NOT NULL
• is_enabled BOOLEAN NOT NULL DEFAULT 1
• config_json TEXT NOT NULL
• capacity_hint_bytes INTEGER NULL
• last_health_status TEXT NULL
• last_health_checked_at DATETIME NULL
• created_at DATETIME NOT NULL
• updated_at DATETIME NOT NULL

Recommended type values:

• local
• s3

3.8 placement_policies

Purpose:

• persist file-class placement intent separately from backend rows

Columns:

• id TEXT PRIMARY KEY
• file_class TEXT NOT NULL UNIQUE
• require_local BOOLEAN NOT NULL DEFAULT 1
• stable_replica_count INTEGER NOT NULL DEFAULT 1
• opportunistic_replica_count INTEGER NOT NULL DEFAULT 0
• preferred_backend_ids_json TEXT NOT NULL DEFAULT []
• excluded_backend_ids_json TEXT NOT NULL DEFAULT []
• max_non_local_size_bytes INTEGER NULL
• created_at DATETIME NOT NULL
• updated_at DATETIME NOT NULL

Notes:

• the current backend persists preferred and excluded backend ids as JSON lists
• max_non_local_size_bytes can force large files to remain local-only even when stable backends exist
• this table is actively used by reconcile jobs and by /api/policies/placement/preview
• aliyun
• baidu
• bridge

Recommended stability_class values:

• local
• stable
• opportunistic

Indexes:

• idx_backends_enabled on is_enabled
• idx_backends_priority on (write_priority, read_priority)

Current implementation status:

• the gateway currently bootstraps one default local backend record automatically

3.8 upload_sessions

Purpose:

• durable record for tus upload lifecycle

Columns:

• id TEXT PRIMARY KEY
• directory_id TEXT NULL REFERENCES directories(id)
• filename TEXT NULL
• total_size_bytes INTEGER NOT NULL
• received_size_bytes INTEGER NOT NULL
• status TEXT NOT NULL
• temp_path TEXT NOT NULL
• tus_upload_url TEXT NULL
• upload_metadata_json TEXT NULL
• created_at DATETIME NOT NULL
• updated_at DATETIME NOT NULL

Recommended enum values:

• created
• uploading
• uploaded
• finalizing
• finalized
• failed
• expired

Indexes:

• idx_upload_sessions_status on status
• idx_upload_sessions_updated_at on updated_at

3.9 upload_session_parts

Purpose:

• optional upload-part detail for resumable recovery and diagnostics

Columns:

• id TEXT PRIMARY KEY
• upload_session_id TEXT NOT NULL REFERENCES upload_sessions(id)
• part_number INTEGER NOT NULL
• byte_offset INTEGER NOT NULL
• size_bytes INTEGER NOT NULL
• checksum TEXT NULL
• status TEXT NOT NULL
• created_at DATETIME NOT NULL
• updated_at DATETIME NOT NULL

Indexes:

• unique idx_upload_session_parts_unique on (upload_session_id, part_number)
• idx_upload_session_parts_upload_id on upload_session_id

3.10 jobs

Purpose:

• durable background task queue

Columns:

• id TEXT PRIMARY KEY
• kind TEXT NOT NULL
• status TEXT NOT NULL
• payload_json TEXT NOT NULL
• attempt_count INTEGER NOT NULL DEFAULT 0
• max_attempts INTEGER NOT NULL DEFAULT 5
• run_after DATETIME NOT NULL
• last_error TEXT NULL
• created_at DATETIME NOT NULL
• updated_at DATETIME NOT NULL

Recommended enum values:

• kinds: replicate_blob, generate_thumbnail, verify_replica, repair_blob, health_check_backend
• status: queued, running, succeeded, failed, dead

Indexes:

• idx_jobs_status_run_after on (status, run_after)
• idx_jobs_kind_status on (kind, status)

3.11 preview_artifacts

Purpose:

• preview resources such as thumbnails and poster images

Columns:

• id TEXT PRIMARY KEY
• file_version_id TEXT NOT NULL REFERENCES file_versions(id)
• artifact_type TEXT NOT NULL
• blob_id TEXT NOT NULL REFERENCES blobs(id)
• status TEXT NOT NULL
• created_at DATETIME NOT NULL
• updated_at DATETIME NOT NULL

Recommended enum values:

• thumbnail
• poster
• pdf_preview

Indexes:

• unique idx_preview_artifacts_version_type on (file_version_id, artifact_type)

3.12 cache_entries

Purpose:

• local cache state for blobs and derivatives

Columns:

• id TEXT PRIMARY KEY
• blob_id TEXT NOT NULL REFERENCES blobs(id)
• local_path TEXT NOT NULL
• cache_kind TEXT NOT NULL
• size_bytes INTEGER NOT NULL
• status TEXT NOT NULL
• last_accessed_at DATETIME NOT NULL
• expires_at DATETIME NULL
• created_at DATETIME NOT NULL
• updated_at DATETIME NOT NULL

Recommended enum values:

• cache kinds: blob, preview, temp
• status: ready, partial, evicted, invalid

Indexes:

• idx_cache_entries_blob_id on blob_id
• idx_cache_entries_last_accessed_at on last_accessed_at
• idx_cache_entries_expires_at on expires_at

4. Relationship Summary

• one directory has many child directories
• one directory has many file_entries
• one file_entry has many file_versions
• one file_version has many blobs
• one blob has many blob_replicas
• one backend has many blob_replicas
• one upload_session may become one file_entry
• one job targets one logical task payload

5. Migration Guidance

Recommended migration order:

1. users
2. directories
3. file_entries
4. file_versions
5. backends
6. blobs
7. blob_replicas
8. upload_sessions
9. upload_session_parts
10. jobs
11. placement_policies
12. preview_artifacts
13. cache_entries

Guidelines:

• create foreign keys only after referenced tables exist
• add indexes in the same migration as table creation where practical
• keep enum values in application constants for SQLite MVP

6. SQLite Notes

• enable foreign keys explicitly
• use WAL mode for better concurrent read behavior
• keep transactions short around upload finalization and job claiming
• avoid long-lived write transactions during file streaming

7. Future Schema Evolution

Likely future additions:

• file_tags
• share_links
• device_clients
• sync_rules
• search_documents
• backend_metrics
• replica_verification_events

Likely future changes:

• introduce true version history controls
• move some JSON config into typed backend-specific tables if the surface becomes stable
• expand placement_policies if policy shape grows beyond file-class controls, preferred/excluded backend sets, and size-aware replica caps