# Iron Database Schema ## 1. Overview This document defines the initial SQLite schema for Iron MVP. Goals: - support a single logical file namespace - separate logical metadata from physical storage placement - track replicas and background jobs durably - support `tus` resumable uploads - keep the schema portable enough for future migration to PostgreSQL Conventions: - IDs are stored as `TEXT` - timestamps are stored as UTC datetimes - soft delete is preferred for logical user-facing entities in MVP - provider-specific details belong in JSON columns only when no stable relational shape exists ## 2. Entity Overview Main tables: - `users` - `auth_sessions` - `directories` - `file_entries` - `file_versions` - `blobs` - `blob_replicas` - `backends` - `upload_sessions` - `upload_session_parts` - `jobs` - `placement_policies` - `preview_artifacts` - `cache_entries` ## 3. Table Definitions ### 3.1 `users` Purpose: - local gateway login identity Columns: - `id` TEXT PRIMARY KEY - `username` TEXT NOT NULL UNIQUE - `password_hash` TEXT NOT NULL - `created_at` DATETIME NOT NULL - `updated_at` DATETIME NOT NULL ### 3.2 `auth_sessions` Purpose: - persisted login sessions for bearer-token auth Columns: - `id` TEXT PRIMARY KEY - `user_id` TEXT NOT NULL REFERENCES `users`(`id`) - `token_hash` TEXT NOT NULL UNIQUE - `expires_at` DATETIME NOT NULL - `revoked_at` DATETIME NULL - `created_at` DATETIME NOT NULL - `updated_at` DATETIME NOT NULL Indexes: - `idx_auth_sessions_user_status` on (`user_id`, `revoked_at`, `expires_at`) - unique `idx_auth_sessions_token_hash` on `token_hash` Notes: - raw session tokens should never be stored directly - the current implementation stores a SHA-256 token hash and returns the raw bearer token only at login time ### 3.3 `directories` Purpose: - logical folder hierarchy Columns: - `id` TEXT PRIMARY KEY - `parent_id` TEXT NULL REFERENCES `directories`(`id`) - `name` TEXT NOT NULL - `path_key` TEXT NOT NULL UNIQUE - `created_at` DATETIME NOT NULL - `updated_at` DATETIME NOT NULL - `deleted_at` DATETIME NULL Constraints: - unique sibling name constraint should be enforced in application logic or by an additional composite unique index later Indexes: - `idx_directories_parent_id` on `parent_id` - `idx_directories_path_key` on `path_key` ### 3.4 `file_entries` Purpose: - logical files visible to users Columns: - `id` TEXT PRIMARY KEY - `directory_id` TEXT NOT NULL REFERENCES `directories`(`id`) - `name` TEXT NOT NULL - `mime_type` TEXT NULL - `size_bytes` INTEGER NOT NULL - `current_version_id` TEXT NULL - `is_deleted` BOOLEAN NOT NULL DEFAULT 0 - `deleted_at` DATETIME NULL - `created_at` DATETIME NOT NULL - `updated_at` DATETIME NOT NULL Constraints: - unique active filename per directory Indexes: - unique `idx_file_entries_directory_name` on (`directory_id`, `name`, `is_deleted`) - `idx_file_entries_current_version_id` on `current_version_id` - `idx_file_entries_deleted_at` on `deleted_at` Notes: - file deletion uses recycle-bin semantics in MVP - active and deleted files may temporarily share the same logical name in one directory because the uniqueness rule only protects active entries ### 3.5 `file_versions` Purpose: - immutable snapshots of file content Columns: - `id` TEXT PRIMARY KEY - `file_entry_id` TEXT NOT NULL REFERENCES `file_entries`(`id`) - `content_hash` TEXT NOT NULL - `size_bytes` INTEGER NOT NULL - `storage_layout` TEXT NOT NULL - `created_at` DATETIME NOT NULL Recommended enum values: - `single` - `chunked` Indexes: - `idx_file_versions_file_entry_id` on `file_entry_id` - `idx_file_versions_content_hash` on `content_hash` ### 3.5 `blobs` Purpose: - physical storage units referenced by file versions Columns: - `id` TEXT PRIMARY KEY - `file_version_id` TEXT NOT NULL REFERENCES `file_versions`(`id`) - `blob_index` INTEGER NOT NULL - `kind` TEXT NOT NULL - `content_hash` TEXT NOT NULL - `size_bytes` INTEGER NOT NULL - `logical_offset` INTEGER NOT NULL DEFAULT 0 - `created_at` DATETIME NOT NULL Recommended enum values: - `file` - `chunk` - `thumbnail` - `poster` Indexes: - unique `idx_blobs_file_version_index` on (`file_version_id`, `blob_index`) - `idx_blobs_content_hash` on `content_hash` ### 3.6 `blob_replicas` Purpose: - one row per physical copy of a blob on one backend Columns: - `id` TEXT PRIMARY KEY - `blob_id` TEXT NOT NULL REFERENCES `blobs`(`id`) - `backend_id` TEXT NOT NULL REFERENCES `backends`(`id`) - `storage_key` TEXT NOT NULL - `status` TEXT NOT NULL - `etag` TEXT NULL - `checksum` TEXT NULL - `size_bytes` INTEGER NOT NULL - `last_verified_at` DATETIME NULL - `created_at` DATETIME NOT NULL - `updated_at` DATETIME NOT NULL Recommended enum values: - `pending` - `ready` - `missing` - `corrupt` - `offline` - `failed` Indexes: - unique `idx_blob_replicas_blob_backend` on (`blob_id`, `backend_id`) - `idx_blob_replicas_backend_status` on (`backend_id`, `status`) - `idx_blob_replicas_storage_key` on `storage_key` Current implementation status: - local backend writes now create `blob_replicas` rows during upload finalize - initial persisted replica status is `ready` ### 3.7 `backends` Purpose: - configured storage targets and their policy attributes Columns: - `id` TEXT PRIMARY KEY - `name` TEXT NOT NULL UNIQUE - `type` TEXT NOT NULL - `stability_class` TEXT NOT NULL - `read_priority` INTEGER NOT NULL - `write_priority` INTEGER NOT NULL - `is_enabled` BOOLEAN NOT NULL DEFAULT 1 - `config_json` TEXT NOT NULL - `capacity_hint_bytes` INTEGER NULL - `last_health_status` TEXT NULL - `last_health_checked_at` DATETIME NULL - `created_at` DATETIME NOT NULL - `updated_at` DATETIME NOT NULL Recommended `type` values: - `local` - `s3` ### 3.8 `placement_policies` Purpose: - persist file-class placement intent separately from backend rows Columns: - `id` TEXT PRIMARY KEY - `file_class` TEXT NOT NULL UNIQUE - `require_local` BOOLEAN NOT NULL DEFAULT 1 - `stable_replica_count` INTEGER NOT NULL DEFAULT 1 - `opportunistic_replica_count` INTEGER NOT NULL DEFAULT 0 - `preferred_backend_ids_json` TEXT NOT NULL DEFAULT `[]` - `excluded_backend_ids_json` TEXT NOT NULL DEFAULT `[]` - `max_non_local_size_bytes` INTEGER NULL - `created_at` DATETIME NOT NULL - `updated_at` DATETIME NOT NULL Notes: - the current backend persists preferred and excluded backend ids as JSON lists - `max_non_local_size_bytes` can force large files to remain local-only even when stable backends exist - this table is actively used by reconcile jobs and by `/api/policies/placement/preview` - `aliyun` - `baidu` - `bridge` Recommended `stability_class` values: - `local` - `stable` - `opportunistic` Indexes: - `idx_backends_enabled` on `is_enabled` - `idx_backends_priority` on (`write_priority`, `read_priority`) Current implementation status: - the gateway currently bootstraps one default local backend record automatically ### 3.8 `upload_sessions` Purpose: - durable record for `tus` upload lifecycle Columns: - `id` TEXT PRIMARY KEY - `directory_id` TEXT NULL REFERENCES `directories`(`id`) - `filename` TEXT NULL - `total_size_bytes` INTEGER NOT NULL - `received_size_bytes` INTEGER NOT NULL - `status` TEXT NOT NULL - `temp_path` TEXT NOT NULL - `tus_upload_url` TEXT NULL - `upload_metadata_json` TEXT NULL - `created_at` DATETIME NOT NULL - `updated_at` DATETIME NOT NULL Recommended enum values: - `created` - `uploading` - `uploaded` - `finalizing` - `finalized` - `failed` - `expired` Indexes: - `idx_upload_sessions_status` on `status` - `idx_upload_sessions_updated_at` on `updated_at` ### 3.9 `upload_session_parts` Purpose: - optional upload-part detail for resumable recovery and diagnostics Columns: - `id` TEXT PRIMARY KEY - `upload_session_id` TEXT NOT NULL REFERENCES `upload_sessions`(`id`) - `part_number` INTEGER NOT NULL - `byte_offset` INTEGER NOT NULL - `size_bytes` INTEGER NOT NULL - `checksum` TEXT NULL - `status` TEXT NOT NULL - `created_at` DATETIME NOT NULL - `updated_at` DATETIME NOT NULL Indexes: - unique `idx_upload_session_parts_unique` on (`upload_session_id`, `part_number`) - `idx_upload_session_parts_upload_id` on `upload_session_id` ### 3.10 `jobs` Purpose: - durable background task queue Columns: - `id` TEXT PRIMARY KEY - `kind` TEXT NOT NULL - `status` TEXT NOT NULL - `payload_json` TEXT NOT NULL - `attempt_count` INTEGER NOT NULL DEFAULT 0 - `max_attempts` INTEGER NOT NULL DEFAULT 5 - `run_after` DATETIME NOT NULL - `last_error` TEXT NULL - `created_at` DATETIME NOT NULL - `updated_at` DATETIME NOT NULL Recommended enum values: - kinds: `replicate_blob`, `generate_thumbnail`, `verify_replica`, `repair_blob`, `health_check_backend` - status: `queued`, `running`, `succeeded`, `failed`, `dead` Indexes: - `idx_jobs_status_run_after` on (`status`, `run_after`) - `idx_jobs_kind_status` on (`kind`, `status`) ### 3.11 `preview_artifacts` Purpose: - preview resources such as thumbnails and poster images Columns: - `id` TEXT PRIMARY KEY - `file_version_id` TEXT NOT NULL REFERENCES `file_versions`(`id`) - `artifact_type` TEXT NOT NULL - `blob_id` TEXT NOT NULL REFERENCES `blobs`(`id`) - `status` TEXT NOT NULL - `created_at` DATETIME NOT NULL - `updated_at` DATETIME NOT NULL Recommended enum values: - `thumbnail` - `poster` - `pdf_preview` Indexes: - unique `idx_preview_artifacts_version_type` on (`file_version_id`, `artifact_type`) ### 3.12 `cache_entries` Purpose: - local cache state for blobs and derivatives Columns: - `id` TEXT PRIMARY KEY - `blob_id` TEXT NOT NULL REFERENCES `blobs`(`id`) - `local_path` TEXT NOT NULL - `cache_kind` TEXT NOT NULL - `size_bytes` INTEGER NOT NULL - `status` TEXT NOT NULL - `last_accessed_at` DATETIME NOT NULL - `expires_at` DATETIME NULL - `created_at` DATETIME NOT NULL - `updated_at` DATETIME NOT NULL Recommended enum values: - cache kinds: `blob`, `preview`, `temp` - status: `ready`, `partial`, `evicted`, `invalid` Indexes: - `idx_cache_entries_blob_id` on `blob_id` - `idx_cache_entries_last_accessed_at` on `last_accessed_at` - `idx_cache_entries_expires_at` on `expires_at` ## 4. Relationship Summary - one `directory` has many child `directories` - one `directory` has many `file_entries` - one `file_entry` has many `file_versions` - one `file_version` has many `blobs` - one `blob` has many `blob_replicas` - one `backend` has many `blob_replicas` - one `upload_session` may become one `file_entry` - one `job` targets one logical task payload ## 5. Migration Guidance Recommended migration order: 1. `users` 2. `directories` 3. `file_entries` 4. `file_versions` 5. `backends` 6. `blobs` 7. `blob_replicas` 8. `upload_sessions` 9. `upload_session_parts` 10. `jobs` 11. `placement_policies` 12. `preview_artifacts` 13. `cache_entries` Guidelines: - create foreign keys only after referenced tables exist - add indexes in the same migration as table creation where practical - keep enum values in application constants for SQLite MVP ## 6. SQLite Notes - enable foreign keys explicitly - use WAL mode for better concurrent read behavior - keep transactions short around upload finalization and job claiming - avoid long-lived write transactions during file streaming ## 7. Future Schema Evolution Likely future additions: - `file_tags` - `share_links` - `device_clients` - `sync_rules` - `search_documents` - `backend_metrics` - `replica_verification_events` Likely future changes: - introduce true version history controls - move some JSON config into typed backend-specific tables if the surface becomes stable - expand `placement_policies` if policy shape grows beyond file-class controls, preferred/excluded backend sets, and size-aware replica caps