Skip to main content
ChameleonDB is a schema-governed database platform with explicit integrity guarantees. This page explains the architecture, components, and design decisions.

​
System Overview

System Overview diagram ChameleonDB governs schemas at runtime through versioning, cryptographic integrity, and explicit operational modes.

​
Architecture Layers

The system is organized into five distinct layers: 
┌─────────────────────────────────────────────┐
│  Application Layer                          │
│  - Query Builder API                        │
│  - Mutations (Insert/Update/Delete)         │
│  - Debug Mode                               │
└─────────────────────────────────────────────┘
                    ↓
┌─────────────────────────────────────────────┐
│  Security Layer                             │
│  - Integrity Modes (readonly/standard/...)  │
│  - Password-protected upgrades              │
│  - Mode enforcement                         │
└─────────────────────────────────────────────┘
                    ↓
┌─────────────────────────────────────────────┐
│  Schema Vault                               │
│  - Versioned schemas (v001, v002, ...)      │
│  - SHA256 integrity verification           │
│  - Immutable snapshots                      │
│  - Append-only audit log                    │
└─────────────────────────────────────────────┘
                    ↓
┌─────────────────────────────────────────────┐
│  Runtime Layer                              │
│  - Query Executor                           │
│  - Connection Pool (pgx)                    │
│  - Error Mapping                            │
│  - IdentityMap (deduplication)              │
└─────────────────────────────────────────────┘
                    ↓
┌─────────────────────────────────────────────┐
│  Database Backend                           │
│  - PostgreSQL (v1.0)                        │
│  - MySQL (planned v1.2)                     │
│  - DuckDB (planned v1.5)                    │
└─────────────────────────────────────────────┘
Each layer has a specific responsibility. Security and governance are built into the core, not bolted on.

​
Core Components

​
1. Rust Core (chameleon-core)

Parser
Transforms .cham source into an AST using LALRPOP grammar. Handles entities, fields, relations, type modifiers, default values, and backend annotations. Type Checker
Validates the AST before runtime. Organized in three modules:
  • relations.rs - Entity references and foreign key consistency
  • constraints.rs - Primary keys and annotation rules
  • mod.rs - Pipeline orchestration and error reporting
Query Optimizer (Planned v1.5)
Rule-based optimization for query execution plans. Deterministic, no ML in v1.x. Status: ✅ Complete

​
2. Schema Vault

Purpose: Immutable, versioned schema storage with integrity guarantees. See the Schema Vault page for complete details. Structure: 
.chameleon/vault/
├── manifest.json       # Current version + history
├── integrity.log       # Append-only audit trail
├── versions/
│   ├── v001.json      # Immutable schema snapshot
│   └── v002.json
└── hashes/
    ├── v001.hash      # SHA256 verification
    └── v002.hash
Features:
  • ✅ Automatic version registration on every migration
  • ✅ SHA256 hash verification before every operation
  • ✅ Tamper detection (if hash mismatch → abort)
  • ✅ Lineage tracking (parent versions)
  • ✅ Complete audit trail (who, what, when)
Status: ✅ Complete (v1.0)

​
3. Integrity Modes

Purpose: Ring-based access control for schema governance. See the Integrity Modes page for complete details.
ModeRingUse CaseSchema Changes
readonlyR3Production (default)❌ Blocked
standardR2Development teams✅ Controlled
privilegedR1DBAs✅ Direct (logged)
emergencyR0Incident recovery✅ No checks (audited)
Mode Enforcement:
  • Application code checks mode before operations
  • Mode upgrades require password authentication
  • Downgrades don’t require password
  • All mode changes logged
Status: ✅ Complete (v1.0)

​
4. Go Runtime (chameleon)

Engine
Entry point for Go applications. Loads schemas from vault, verifies integrity, enforces modes, and exposes public API. Key change in v1.0: 
// ❌ OLD (insecure - bypasses vault)
eng := engine.NewEngine()
eng.LoadSchemaFromFile("any.cham")

// ✅ NEW (secure - vault-enforced)
eng, err := engine.NewEngine()
// ↑ Loads ONLY from .chameleon/state/schema.merged.cham
// ↑ Verifies integrity automatically
// ↑ Enforces mode restrictions
Query Executor
Translates validated queries into backend-specific SQL. Handles field projection (.Select()), eager loading (.Include()), and filters. Connection Pool
pgx-based PostgreSQL connection management with automatic reconnection and health checks. IdentityMap (NEW in v1.0)
Automatic object deduplication in memory. When querying with .Include(), parent objects are deduplicated to save memory. Example: 
// User with 100 posts
result := db.Query("User").
    Include("posts").
    Execute(ctx)

// Without IdentityMap: User duplicated 100 times (wasteful)
// With IdentityMap: User appears only once (efficient)
Error Mapping
Comprehensive PostgreSQL error → ChameleonDB error mapping with clear messages and suggestions. Status: ✅ Complete (v1.0)

​
5. CLI Tools

Commands:
CommandPurposeStatus
initInitialize project + vault✅ v1.0
migrateGenerate & apply migrations✅ v1.0
validateValidate schema syntax✅ v1.0
verifyVerify vault integrity✅ v1.0
statusShow vault + mode status✅ v1.0
journal schemaView version history✅ v1.0
configManage modes & settings✅ v1.0
introspectDB → Schema generation✅ v1.0
Status: ✅ Complete (v1.0)

​
6. FFI Boundary

Communication between Rust core and Go runtime via C ABI. How it works:
  • Schemas serialized to JSON in Rust
  • Passed to Go via C strings
  • Go deserializes and uses
  • Memory managed explicitly (Rust allocates, Go frees)
Overhead: ~100ns per call (negligible for DB operations) Status: ✅ Complete

​
Compilation & Validation Flow

Compilation Flow 
.cham files
    ↓
Parser (LALRPOP)
    ↓
AST
    ↓
Type Checker
  - Relations validation
  - Constraints validation
  - Cycle detection
    ↓
Validated Schema
    ↓
JSON Serialization
    ↓
FFI Boundary (C ABI)
    ↓
Go Runtime
    ↓
Schema Vault Registration
  - Compute SHA256
  - Save snapshot
  - Update manifest
    ↓
Migration Generation
    ↓
SQL Execution (PostgreSQL)
Validation happens at compile time. Invalid schemas never reach the database.

​
Execution Flow

The complete flow from migration to database: 
1. User: chameleon migrate --apply
         ↓
2. Vault: Check integrity (verify all hashes)
         ↓
3. Mode:  Check if changes allowed (readonly blocks)
         ↓
4. Detect: Compare schema hash with current version
         ↓
5. Register: Create v002 snapshot + hash
         ↓
6. Execute: Apply SQL migration
         ↓
7. Log: Record in integrity.log + journal

​
Security Model

See SECURITY.md in the source for complete details. Layers:
  1. OS Permissions - File access control (0700)
  2. Hash Integrity - SHA256 tamper detection
  3. Integrity Modes - Runtime access control
  4. Vault Enforcement - No schema bypass
  5. Audit Trail - Complete forensics
All security layers work together. Bypassing one layer doesn’t compromise the system.

​
Design Decisions

​
Why Rust for Core?

  • True lambdas and closures (essential for query API)
  • Extreme type safety (catch errors at compile time)
  • Operator overloading (natural query syntax)
  • Excellent performance on parser hot paths

​
Why Go for Runtime?

  • Simple concurrency (goroutines for connection pooling)
  • Excellent PostgreSQL driver (pgx)
  • Single-binary deployment
  • Great debugging tools

​
Why FFI?

  • Each language does what it does best
  • Minimal overhead (~100ns)
  • Future-proof (easy to add Node, Python, Java bindings)

​
Why Schema Vault?

  • Treats schemas as first-class artifacts
  • Immutability prevents silent drift
  • Cryptographic integrity (SHA256)
  • Complete audit trail for compliance

​
Why Integrity Modes?

  • Explicit governance (not just config)
  • Runtime enforcement (not optional)
  • Password-protected escalation
  • Compliance-ready out of the box

​
Performance Targets (v1.0)

OperationTargetNotes
Schema parse< 10msOne-time, cold start
Type check< 5msPer schema validation
FFI call< 100nsPer boundary crossing
Hash verification< 1msSHA256 computation
Query compilation< 1msPer query
Query executionDB-boundOptimized SQL generation
Performance overhead from governance is negligible compared to database I/O.

​
Component Status

ComponentStatusVersionNotes
Parser✅ Completev1.0LALRPOP, all types
Type Checker✅ Completev1.0Relations, constraints, cycles
Schema Vault✅ Completev1.0Versioning, hashing, audit
Integrity Modes✅ Completev1.04 modes, password-protected
FFI Layer✅ Completev1.0C ABI bridge
Query Builder✅ Completev1.0Filter, Include, Select
Mutations✅ Completev1.0Insert, Update, Delete
IdentityMap✅ Completev1.0Object deduplication
Connection Pool✅ Completev1.0pgx-based
Error Mapping✅ Completev1.0Comprehensive
Migration Gen✅ Completev1.0PostgreSQL DDL
Introspection✅ Completev1.0DB → Schema
Debug Mode✅ Completev1.0SQL visibility
CLI Tools✅ Completev1.08 commands
Backend Registry⏳ Plannedv2.0Multi-backend routing
Code Generator⏳ Plannedv1.1+Boilerplate generation
Query Optimizer⏳ Plannedv1.5+Rule-based optimization

​
Project Structure

chameleondb/
├── chameleon-core/          # Rust core
│   ├── src/
│   │   ├── ast/             # Schema structures
│   │   ├── parser/          # LALRPOP grammar
│   │   ├── typechecker/     # Validation
│   │   └── ffi/             # C ABI bridge
│   └── tests/               # Integration tests
│
├── chameleon/               # Go runtime
│   ├── cmd/chameleon/       # CLI tool
│   ├── pkg/
│   │   ├── engine/          # Public API
│   │   └── vault/           # Schema Vault (NEW)
│   └── internal/
│       ├── admin/           # Journal, state tracking
│       └── schema/          # Schema merge
│
├── examples/                # Example apps
│   └── todo-app/            # Complete CRUD example
│
└── docs/                    # Documentation
    ├── architecture.md      # System design
    ├── SECURITY.md          # Security model
    ├── QUICK_START.md       # 5 min tutorial
    └── ...

​
Testing

Test coverage:
  • Rust: 96 tests ✅
  • Go: 188 tests ✅
  • Integration: 30 tests ✅
  • Total: 314 tests passing
Test categories:
  • Parser tests (syntax, error handling)
  • Type checker tests (relations, cycles, constraints)
  • Vault tests (versioning, integrity, modes)
  • Query tests (filters, includes, selects)
  • Mutation tests (CRUD operations)
  • Error mapping tests (PostgreSQL → ChameleonDB)

​
Future Architecture (v2.0+)

Future Architecture Planned features (not in v1.x):
  • Multi-backend routing (PostgreSQL + DuckDB + Redis)
  • ML-based query optimization
  • Visual schema editor
  • Distributed vault (multi-node)
  • Advanced observability
v2.0 features are not part of the open-source v1.x releases.

​
Summary

ChameleonDB v1.0 provides:
  • ✅ Schema Vault - Versioned, hash-verified schemas
  • ✅ Integrity Modes - Explicit runtime governance
  • ✅ Type-safe queries - Validated before execution
  • ✅ Complete audit trail - Who, what, when
  • ✅ Zero-config security - Fail-safe defaults
  • ✅ Production-ready - 314 tests passing
Philosophy: Explicit over implicit, safety over convenience, governance over magic.

​
Next Steps