docs: rewrite README as concise landing page and add architecture diagrams

docs/README.md

@@ -35,7 +35,55 @@ Complete documentation for madengine - AI model automation and distributed bench
 
 ## 🏗️ Architecture
 
-The architecture diagram (Orchestration, Infrastructure, and Launcher layers) is in the [main README](../README.md#-architecture). Summary:
+The CLI drives orchestrators that discover and build models, then hand off to a local or distributed execution target, which runs the model under the appropriate launcher and emits performance data for reporting. (Same diagram as the [main README](../README.md#-architecture).)
+
+```mermaid
+flowchart TB
+    subgraph CLI["CLI Layer — Typer + Rich"]
+        C1[discover]
+        C2[build]
+        C3[run]
+        C4[report]
+        C5[database]
+    end
+
+    subgraph ORC["Orchestration Layer"]
+        O1[DiscoverModels]
+        O2[BuildOrchestrator]
+        O3[RunOrchestrator]
+        MAN[(build_manifest.json)]
+    end
+
+    subgraph EXEC["Execution / Deployment Layer"]
+        E1[ContainerRunner<br/>local Docker]
+        E2[DeploymentFactory]
+        K8S[Kubernetes Jobs]
+        SLURM[SLURM Jobs]
+    end
+
+    subgraph LAUNCH["Launcher Layer"]
+        T[Train: torchrun · DeepSpeed<br/>Megatron-LM · TorchTitan · Primus]
+        I[Infer: vLLM · SGLang · SGLang Disagg]
+    end
+
+    OUT[(perf.csv / JSON)]
+
+    C1 --> O1
+    C2 --> O2
+    C3 --> O3
+    O2 --> MAN --> O3
+    O1 --> O2
+    O3 --> E1
+    O3 --> E2
+    E2 --> K8S
+    E2 --> SLURM
+    E1 --> LAUNCH
+    K8S --> LAUNCH
+    SLURM --> LAUNCH
+    LAUNCH --> OUT
+    OUT --> C4
+    OUT --> C5
+```
 
 1. **CLI Layer** - User interface with 5 commands (discover, build, run, report, database)
 2. **Model Discovery** - Find and validate models from MAD package
@@ -106,7 +154,7 @@ madengine operates within the MAD (Model Automation and Dashboarding) ecosystem.
 
 - **torchrun** - PyTorch DDP/FSDP
 - **deepspeed** - ZeRO optimization
-- **megatron** - Large transformers (K8s + SLURM)
+- **megatron-lm** - Large transformers (K8s + SLURM)
 - **torchtitan** - LLM pre-training
 - **vllm** - LLM inference
 - **sglang** - Structured generation
@@ -119,7 +167,7 @@ This documentation follows these principles:
 2. **Progressive disclosure** - Start simple, add complexity as needed
 3. **Examples first** - Show working examples before explaining details
 4. **Consistent naming** - Files follow simple naming pattern (no prefixes)
-5. **Up-to-date** - Reflects current implementation (v2.0)
+5. **Up-to-date** - Tracks the current implementation; see [CHANGELOG.md](../CHANGELOG.md) for release history
 
 ## 🤝 Contributing to Documentation
 

docs/batch-build.md

@@ -224,11 +224,15 @@ Creates `build_manifest.json` with:
     }
   },
   "built_models": {...},
+  "context": {...},
+  "credentials_required": {...},
   "deployment_config": {...},
   "summary": {...}
 }
 ```
 
+> `deployment_config` is only written when `--additional-context` resolves to a non-local deployment (e.g. `slurm`, `k8s`/`kubernetes`, `distributed`, `vllm`, or non-empty `env_vars`). Plain local builds omit this key entirely.
+
 ## Best Practices
 
 1. **Version Control**: Keep batch manifests in version control for reproducibility

docs/cli-reference.md

@@ -233,10 +233,10 @@ madengine run [OPTIONS]
 | `--manifest-output` | | TEXT | `build_manifest.json` | Output file for build manifest (full workflow) |
 | `--summary-output` | `-s` | TEXT | `None` | Output file for summary JSON |
 | `--live-output` | `-l` | FLAG | `False` | Print output in real-time |
-| `--output` | `-o` | TEXT | `perf_entry.csv` | Performance output file |
+| `--output` | `-o` | TEXT | `perf.csv` | Performance output file |
 | `--ignore-deprecated` | | FLAG | `False` | Force run deprecated models |
 | `--data-config` | | TEXT | `data.json` | Custom data configuration file |
-| `--tools-config` | | TEXT | `tools.json` | Custom tools JSON configuration |
+| `--tools-config` | | TEXT | `./scripts/common/tools.json` | Custom tools JSON configuration |
 | `--sys-env-details` | | FLAG | `True` | Generate system config env details |
 | `--force-mirror-local` | | TEXT | `None` | Path to force local data mirroring |
 | `--disable-skip-gpu-arch` | | FLAG | `False` | Disable skipping models based on GPU architecture |
@@ -356,7 +356,7 @@ madengine run --tags model \
 
 **Performance Output:**
 
-Results are saved to CSV file (default: `perf_entry.csv`) with metrics including:
+Results are saved to CSV file (default: `perf.csv`) with metrics including:
 - Execution time
 - GPU utilization
 - Memory usage
@@ -384,20 +384,20 @@ madengine report to-html [OPTIONS]
 
 | Option | Short | Type | Required | Description |
 |--------|-------|------|----------|-------------|
-| `--csv-file` | | TEXT | **Yes** | Path to the CSV file to convert |
+| `--csv-file-path` | | TEXT | **Yes** | Path to the CSV file to convert |
 | `--verbose` | `-v` | FLAG | No | Enable verbose logging |
 
 **Examples:**
 
 ```bash
 # Convert CSV to HTML
-madengine report to-html --csv-file perf_entry.csv
+madengine report to-html --csv-file-path perf_entry.csv
 
 # With custom CSV file
-madengine report to-html --csv-file results/perf_mi300.csv
+madengine report to-html --csv-file-path results/perf_mi300.csv
 
 # Verbose output
-madengine report to-html --csv-file perf.csv --verbose
+madengine report to-html --csv-file-path perf.csv --verbose
 ```
 
 **Output:** Creates `{filename}.html` in the same directory as the CSV file.
@@ -444,7 +444,7 @@ madengine report to-email --directory ./results --verbose
 
 ### `database` - Upload to MongoDB
 
-Upload CSV performance data to MongoDB database.
+Upload CSV or JSON performance data to MongoDB (format is auto-detected).
 
 **Usage:**
 
@@ -456,32 +456,30 @@ madengine database [OPTIONS]
 
 | Option | Short | Type | Default | Required | Description |
 |--------|-------|------|---------|----------|-------------|
-| `--csv-file` | | TEXT | `perf_entry.csv` | No | Path to the CSV file to upload |
-| `--database-name` | `--db` | TEXT | `None` | **Yes** | Name of the MongoDB database |
-| `--collection-name` | `--collection` | TEXT | `None` | **Yes** | Name of the MongoDB collection |
-| `--verbose` | `-v` | FLAG | `False` | No | Enable verbose logging |
+| `--file` | `-f` | TEXT | `None` | **Yes** | Path to file (CSV or JSON, auto-detected) |
+| `--database` | `--db` | TEXT | `None` | **Yes** | MongoDB database name |
+| `--collection` | `-c` | TEXT | `None` | **Yes** | MongoDB collection name |
+| `--unique-key` | `-k` | TEXT | `None` | No | Unique field(s) for deduplication (comma-separated, auto-detected if not specified) |
+| `--batch-size` | | INT | `1000` | No | Batch size for bulk operations |
+| `--no-upsert` | | FLAG | `False` | No | Insert only (don't update existing documents) |
+| `--no-index` | | FLAG | `False` | No | Skip automatic index creation |
+| `--dry-run` | | FLAG | `False` | No | Validate without uploading |
+| `--verbose` | `-v` | FLAG | `False` | No | Verbose output |
 
 **Examples:**
 
 ```bash
-# Upload to MongoDB
-madengine database \
-  --csv-file perf_entry.csv \
-  --database-name mydb \
-  --collection-name results
+# Upload JSON with auto-detection
+madengine database -f perf_entry_super.json --db mydb -c perf_super
 
-# Short option names
-madengine database \
-  --csv-file perf.csv \
-  --db test \
-  --collection perf_data
+# Upload CSV with custom unique key
+madengine database -f perf.csv --db test -c results -k model,timestamp
+
+# Dry run to validate
+madengine database -f data.json --db test -c data --dry-run
 
 # With verbose output
-madengine database \
-  --csv-file perf.csv \
-  --db mydb \
-  --collection results \
-  --verbose
+madengine database -f perf.csv --db mydb -c results --verbose
 ```
 
 **Environment Variables:**
@@ -490,10 +488,12 @@ MongoDB connection details are read from environment variables:
 
 | Variable | Description | Example |
 |----------|-------------|---------|
-| `MONGO_HOST` | MongoDB host address | `localhost` or `mongodb.example.com` |
-| `MONGO_PORT` | MongoDB port | `27017` |
+| `MONGO_HOST` | MongoDB host address (default: `localhost`) | `localhost` or `mongodb.example.com` |
+| `MONGO_PORT` | MongoDB port (default: `27017`) | `27017` |
 | `MONGO_USER` | MongoDB username | `admin` |
 | `MONGO_PASSWORD` | MongoDB password | `secretpassword` |
+| `MONGO_AUTH_SOURCE` | MongoDB authentication database (default: `admin`) | `admin` |
+| `MONGO_TIMEOUT_MS` | Server selection timeout in milliseconds (default: `5000`) | `5000` |
 
 **Example Setup:**
 
@@ -504,7 +504,7 @@ export MONGO_USER=myuser
 export MONGO_PASSWORD=mypassword
 
 madengine database \
-  --csv-file perf_entry.csv \
+  --file perf_entry.csv \
   --db performance_db \
   --collection model_runs
 ```
@@ -637,12 +637,17 @@ madengine recognizes these environment variables:
 | `MAD_DOCKERHUB_USER` | Docker Hub username | None |
 | `MAD_DOCKERHUB_PASSWORD` | Docker Hub password/token | None |
 | `MAD_DOCKERHUB_REPO` | Docker Hub repository | None |
-| `MAD_CONTAINER_IMAGE` | Pre-built container image to use | None |
+| `DOCKER_CONFIG` | Directory holding the Docker `config.json` whose existing login madengine reuses | `~/.docker` |
+| `MAD_SKIP_DOCKER_LOGIN` | Set to `1` to never run `docker login`; always defer to the machine's existing credentials | Unset |
 | `MONGO_HOST` | MongoDB host for database command | `localhost` |
 | `MONGO_PORT` | MongoDB port for database command | `27017` |
 | `MONGO_USER` | MongoDB username | None |
 | `MONGO_PASSWORD` | MongoDB password | None |
 
+> `MAD_CONTAINER_IMAGE` is **not** an environment variable. It is an
+> `--additional-context` key that selects a pre-built image and skips the build
+> phase — see [Configuration](configuration.md#pre-built-container-images).
+
 ---
 
 ## Best Practices
@@ -669,6 +674,6 @@ madengine recognizes these environment variables:
 
 ---
 
-**Version:** 2.1.0  
-**Last Updated:** May 2026
+Run `madengine --version` for the installed version (derived from git tags via
+versioningit). Release history is in [CHANGELOG.md](../CHANGELOG.md).