Compare commits
44 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 72e62bb2ff | |||
| 236ed8ef97 | |||
| 0d2dbf0139 | |||
| a94117c696 | |||
| 5f20bd3193 | |||
| 1b35702b1b | |||
| d226ee0b1e | |||
| 42a5d231b8 | |||
| 100213de2d | |||
| eb340eb664 | |||
| 5cdf645865 | |||
| ed75613a08 | |||
| 0aabfc8a72 | |||
| 231aba08b0 | |||
| d6650108e8 | |||
| 6e1720891f | |||
| e239867ce3 | |||
| 6c8c570239 | |||
| bb6d36ad7a | |||
| 2f1babc681 | |||
| 0e7c188845 | |||
| 38676de168 | |||
| 9c5ea400fb | |||
| 9c4265b429 | |||
| 2346d5a096 | |||
| 5b15a0aedd | |||
| 4b43e80f06 | |||
| 97396afe88 | |||
| 3d9a599f71 | |||
| 237be77014 | |||
| afea257f86 | |||
| 55dc745a5e | |||
| 20323f2ecd | |||
| b1f74d4495 | |||
| 56dfa7a653 | |||
| 6baad76e0d | |||
| f5580b3be3 | |||
| 18bf5b2736 | |||
| 84ee052d71 | |||
| e9834a1997 | |||
| a6d429255c | |||
| c1e347e5e0 | |||
| 57545e661c | |||
| 69011271fc |
@@ -0,0 +1,7 @@
|
|||||||
|
*.agent_wip
|
||||||
|
handoff.md
|
||||||
|
artifacts/unraid-images-live.txt
|
||||||
|
artifacts/unraid-images-with-names.tsv
|
||||||
|
artifacts/test-zima-inventory.yaml
|
||||||
|
artifacts/test-zima-map.yaml
|
||||||
|
artifacts/unraid-to-public-stores-map.yaml
|
||||||
@@ -0,0 +1,3 @@
|
|||||||
|
[submodule "Apps/ark-survival-ascended-linux-container-image"]
|
||||||
|
path = Apps/ark-survival-ascended-linux-container-image
|
||||||
|
url = https://github.com/mschnitzer/ark-survival-ascended-linux-container-image.git
|
||||||
@@ -0,0 +1,110 @@
|
|||||||
|
# Plan: Add Snacks app to zima-apps
|
||||||
|
|
||||||
|
## Context
|
||||||
|
|
||||||
|
`apps.md` lists "Snacks" (https://github.com/derekshreds/snacks) as a pending app. It is an automated video library encoder with hardware acceleration (NVENC, QSV, VAAPI, AMF).
|
||||||
|
|
||||||
|
This plan has two parts:
|
||||||
|
1. Create the Snacks app definition
|
||||||
|
2. Update `apps.md` with agent instructions for future additions
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Part 1: Create `Apps/snacks/`
|
||||||
|
|
||||||
|
### Steps
|
||||||
|
|
||||||
|
1. **Create `Apps/snacks/` directory** from `_template`
|
||||||
|
|
||||||
|
2. **`docker-compose.yaml`** — adapt upstream `deploy-compose.yml`:
|
||||||
|
|
||||||
|
| Field | Upstream value | ZimaOS target |
|
||||||
|
|---|---|---|
|
||||||
|
| `name` | n/a | `snacks` |
|
||||||
|
| `image` | `derekshreds/snacks-docker:latest` | **Pinned version** — fetch latest release tag from GitHub, verify manifest exists |
|
||||||
|
| `network_mode` | `host` | Keep `host` — required for cluster UDP broadcast discovery |
|
||||||
|
| `privileged` | `true` | Keep `true` — required for `/dev/dri` access on QNAP/ZimaOS |
|
||||||
|
| `devices` | `/dev/dri:/dev/dri` | Keep — VAAPI/QSV hardware acceleration |
|
||||||
|
| `ports` | none (host mode) | Add `6767:6767` for web UI |
|
||||||
|
| `volumes` | QNAP-specific paths | Parameterize as `/DATA/AppData/$AppID/...` |
|
||||||
|
| `environment` | QNAP-specific ffmpeg path | Use default ffmpeg path; make jellyfin-ffmpeg path configurable |
|
||||||
|
|
||||||
|
Security baseline (MUST):
|
||||||
|
- `security_opt: ["no-new-privileges:true"]`
|
||||||
|
- `cap_drop: ["ALL"]`
|
||||||
|
- `deploy.resources.reservations` set to appropriate value
|
||||||
|
|
||||||
|
High-risk settings that MUST be documented in README:
|
||||||
|
- `network_mode: host` — required for cluster UDP broadcast
|
||||||
|
- `privileged: true` — required for `/dev/dri` access
|
||||||
|
- Device mount `/dev/dri` — GPU acceleration
|
||||||
|
|
||||||
|
3. **`README.md`** — document:
|
||||||
|
- Purpose: automated video library encoder with hardware acceleration
|
||||||
|
- Port: 6767 (web UI)
|
||||||
|
- Volumes: media library, logs, config
|
||||||
|
- High-risk settings with justification, alternatives evaluated, and risks
|
||||||
|
- Hardware acceleration options (VAAPI, QSV, NVENC)
|
||||||
|
- Cluster mode (UDP broadcast requirement)
|
||||||
|
- Health check endpoint
|
||||||
|
|
||||||
|
4. **Image pinning**: Before merge, verify the image tag exists in Docker Hub registry (manifest check)
|
||||||
|
|
||||||
|
5. **Run validation**: `./scripts/validate-appstore.sh`
|
||||||
|
|
||||||
|
6. **Optional**: `HOW_TO_VERIFY.md` with integration test cases
|
||||||
|
|
||||||
|
### Risk Assessment
|
||||||
|
|
||||||
|
- **High risk** due to `network_mode: host`, `privileged: true`, and device mounts
|
||||||
|
- Must document all three in README per AGENTS.md §3
|
||||||
|
- Image must be pinned — no `:latest`
|
||||||
|
|
||||||
|
### Branch name
|
||||||
|
|
||||||
|
`snacks/initial/add-video-encoder`
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Part 2: Update `apps.md`
|
||||||
|
|
||||||
|
### Changes
|
||||||
|
|
||||||
|
Replace current content with a table format that includes:
|
||||||
|
- Done/pending checkbox
|
||||||
|
- App name
|
||||||
|
- Source URL
|
||||||
|
- Brief description
|
||||||
|
- Agent instructions column (how to pick up this item)
|
||||||
|
|
||||||
|
The file should serve as an agent-facing backlog — clear enough that an agent can read it, understand what is needed, and execute without additional prompting.
|
||||||
|
|
||||||
|
### Suggested format
|
||||||
|
|
||||||
|
```markdown
|
||||||
|
## Backlog
|
||||||
|
|
||||||
|
| # | Done | Name | Source | What | Agent instructions |
|
||||||
|
|---|---|---|---|---|---|
|
||||||
|
| 1 | [ ] | Snacks | https://github.com/derekshreds/snacks | Automated video library encoder | Pick up, follow AGENTS.md §9 workflow, branch `snacks/initial/add-video-encoder` |
|
||||||
|
|
||||||
|
## Adding a new app
|
||||||
|
|
||||||
|
1. Copy `Apps/_template/` → `Apps/<app-id>/`
|
||||||
|
2. Set `name` in compose (lowercase + hyphen only)
|
||||||
|
3. Pin image to explicit version/tag (no `:latest`)
|
||||||
|
4. Add `x-casaos` metadata
|
||||||
|
5. Write `README.md` with purpose, ports, volumes, and risk justifications
|
||||||
|
6. Validate: `./scripts/validate-appstore.sh`
|
||||||
|
7. Run final validation before release: `./scripts/validate-appstore.sh --enforce-risk-docs`
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Verification
|
||||||
|
|
||||||
|
- `docker-compose -f Apps/snacks/docker-compose.yaml config` passes (no syntax errors)
|
||||||
|
- No `:latest` references
|
||||||
|
- `x-casaos` metadata complete
|
||||||
|
- README documents all high-risk settings with justification
|
||||||
|
- `./scripts/validate-appstore.sh` reports `Validation OK`
|
||||||
@@ -0,0 +1,192 @@
|
|||||||
|
# Plan: Local LLM Zima App (Intel NUC8)
|
||||||
|
|
||||||
|
## Context
|
||||||
|
- **Hardware**: Intel NUC8 i7, 16GB RAM, 500GB SSD
|
||||||
|
- **Goal**: Zima app for local LLM inference with web UI
|
||||||
|
- **Constraints**: Intel Iris GPU cannot be used for LLM offload; CPU-only inference
|
||||||
|
- **Decisions**:
|
||||||
|
- Include OpenWebUI (two-container solution)
|
||||||
|
- 8G memory reservation (allows 7B Q4 models)
|
||||||
|
- App name: `llama-server`
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Technology Decision
|
||||||
|
|
||||||
|
### vLLM — **REJECTED**
|
||||||
|
- Requires NVIDIA CUDA GPU
|
||||||
|
- Cannot run on Intel NUC
|
||||||
|
|
||||||
|
### llama.cpp (llama-server) — **SELECTED**
|
||||||
|
- CPU-only, AVX2/AVX512 optimized
|
||||||
|
- Built-in REST API server
|
||||||
|
- Minimal footprint, fast for quantized models
|
||||||
|
- Best fit for NUC8 constraints
|
||||||
|
|
||||||
|
### LocalAI — **BACKUP OPTION**
|
||||||
|
- More features (TTS, image gen, multi-model)
|
||||||
|
- Can backend to llama.cpp
|
||||||
|
- Heavier; only choose if extra features needed
|
||||||
|
|
||||||
|
### OpenWebUI — **RECOMMENDED COMPANION**
|
||||||
|
- Modern chat UI for LLM
|
||||||
|
- Docker-based, easy to deploy alongside
|
||||||
|
- Can be separate Zima app or documented companion
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Architecture: Two Zima Apps
|
||||||
|
|
||||||
|
```
|
||||||
|
┌─────────────────────────┐ ┌─────────────────────────┐
|
||||||
|
│ llama-server │ │ open-webui │
|
||||||
|
│ - REST API :8080 │────▶│ - Chat UI :3000 │
|
||||||
|
│ - Serves model │ │ - Connects to LLM API │
|
||||||
|
└─────────────────────────┘ └─────────────────────────┘
|
||||||
|
```
|
||||||
|
|
||||||
|
Both are separate Zima apps, deployed independently. OpenWebUI references `http://llama-server:8080` via Docker internal networking.
|
||||||
|
|
||||||
|
### App 1: `llama-server`
|
||||||
|
- Container: `ghcr.io/ggerganov/llama.cpp:server`
|
||||||
|
- Port: 8080
|
||||||
|
- Memory: 8G reservation
|
||||||
|
|
||||||
|
### App 2: `open-webui`
|
||||||
|
- Container: `ghcr.io/open-webui/open-webui:main`
|
||||||
|
- Port: 3000
|
||||||
|
- Memory: 2G reservation
|
||||||
|
- Environment: `OLLAMA_BASE_URL=http://llama-server:8080`
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## App: `llama-server`
|
||||||
|
|
||||||
|
### Container: `ghcr.io/ggerganov/llama.cpp:server`
|
||||||
|
|
||||||
|
**Environment Variables**:
|
||||||
|
| Variable | Default | Description |
|
||||||
|
|----------|---------|-------------|
|
||||||
|
| `MODEL` | (required) | Model filename in `/models` |
|
||||||
|
| `CTX_SIZE` | 2048 | Context window size |
|
||||||
|
| `N_THREADS` | auto | CPU threads (auto = all) |
|
||||||
|
| `HOST` | 0.0.0.0 | Listen address |
|
||||||
|
| `PORT` | 8080 | API port |
|
||||||
|
| `MAX_TOKENS` | 512 | Max tokens to generate |
|
||||||
|
|
||||||
|
**Volumes**:
|
||||||
|
| Container | Description |
|
||||||
|
|-----------|-------------|
|
||||||
|
| `/models` | Model files (GGUF format) |
|
||||||
|
| `/DATA/AppData/$AppID/logs` | Server logs |
|
||||||
|
|
||||||
|
**Ports**:
|
||||||
|
| Container | Protocol | Description |
|
||||||
|
|-----------|----------|-------------|
|
||||||
|
| 8080 | TCP | llama.cpp REST API |
|
||||||
|
|
||||||
|
**Resources**:
|
||||||
|
- Memory reservation: **8G** (allows 7B Q4 models)
|
||||||
|
|
||||||
|
**Security**:
|
||||||
|
- `security_opt: no-new-privileges:true`
|
||||||
|
- `cap_drop: ALL`
|
||||||
|
- No privileged needed (CPU-only)
|
||||||
|
|
||||||
|
### Model Download (Documented in README)
|
||||||
|
Users download models manually:
|
||||||
|
```bash
|
||||||
|
# Example: Download Llama 3.2 3B Q4_K_M
|
||||||
|
curl -L -o /DATA/AppData/llama-server/models/llama-3.2-3b-q4_k_m.gguf \
|
||||||
|
"https://huggingface.co/QuantFactory/Llama-3.2-3B-Instruct-GGUF/resolve/main/Llama-3.2-3B-Instruct.Q4_K_M.gguf"
|
||||||
|
```
|
||||||
|
|
||||||
|
**Recommended Models for 16GB RAM**:
|
||||||
|
| Model | Size | Quant | RAM Needed | Speed (est) |
|
||||||
|
|-------|------|-------|------------|-------------|
|
||||||
|
| Llama 3.2 3B | 1.8GB | Q4_K_M | ~4GB | ~15-20 tok/s |
|
||||||
|
| Phi-3.5 Mini 3B | 1.8GB | Q4_K_M | ~4GB | ~15-20 tok/s |
|
||||||
|
| Mistral 7B | 4.1GB | Q4_K_M | ~6-7GB | ~8-12 tok/s |
|
||||||
|
| Qwen 2.5 7B | 4.4GB | Q4_K_M | ~6-7GB | ~8-12 tok/s |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## App: `open-webui`
|
||||||
|
|
||||||
|
### Container: `ghcr.io/open-webui/open-webui:main`
|
||||||
|
|
||||||
|
**Environment Variables**:
|
||||||
|
| Variable | Default | Description |
|
||||||
|
|----------|---------|-------------|
|
||||||
|
| `OLLAMA_BASE_URL` | http://llama-server:8080 | LLM API endpoint |
|
||||||
|
| `WEBUI_PORT` | 3000 | Web UI port |
|
||||||
|
|
||||||
|
**Ports**:
|
||||||
|
| Container | Protocol | Description |
|
||||||
|
|-----------|----------|-------------|
|
||||||
|
| 3000 | TCP | OpenWebUI |
|
||||||
|
|
||||||
|
**Resources**:
|
||||||
|
- Memory reservation: **2G**
|
||||||
|
|
||||||
|
**Notes**:
|
||||||
|
- Connects to `http://llama-server:8080` via Docker internal networking
|
||||||
|
- Requires `llama-server` app to be running first
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## File Structure
|
||||||
|
```
|
||||||
|
Apps/llama-server/
|
||||||
|
├── docker-compose.yaml
|
||||||
|
├── README.md
|
||||||
|
└── HOW_TO_VERIFY.md (optional)
|
||||||
|
|
||||||
|
Apps/open-webui/
|
||||||
|
├── docker-compose.yaml
|
||||||
|
├── README.md
|
||||||
|
└── HOW_TO_VERIFY.md (optional)
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Implementation Steps
|
||||||
|
|
||||||
|
### llama-server
|
||||||
|
1. Create `Apps/llama-server/` directory
|
||||||
|
2. Write `docker-compose.yaml` with:
|
||||||
|
- Image: `ghcr.io/ggerganov/llama.cpp:server`
|
||||||
|
- 8G memory reservation
|
||||||
|
- Port 8080
|
||||||
|
- Model volume at `/models`
|
||||||
|
- Env vars: MODEL, CTX_SIZE, N_THREADS, HOST, PORT
|
||||||
|
3. Write `README.md` with:
|
||||||
|
- Model download instructions
|
||||||
|
- First-run setup
|
||||||
|
- API testing examples
|
||||||
|
- Performance tips for NUC8
|
||||||
|
4. Validate with `./scripts/validate-appstore.sh`
|
||||||
|
|
||||||
|
### open-webui
|
||||||
|
1. Create `Apps/open-webui/` directory
|
||||||
|
2. Write `docker-compose.yaml` with:
|
||||||
|
- Image: `ghcr.io/open-webui/open-webui:main`
|
||||||
|
- 2G memory reservation
|
||||||
|
- Port 3000
|
||||||
|
- Environment: `OLLAMA_BASE_URL=http://llama-server:8080`
|
||||||
|
3. Write `README.md` with:
|
||||||
|
- Prerequisites (llama-server must be running first)
|
||||||
|
- How to access
|
||||||
|
- Troubleshooting connection issues
|
||||||
|
4. Validate with `./scripts/validate-appstore.sh`
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Risk Assessment
|
||||||
|
|
||||||
|
| Risk | Level | Mitigation |
|
||||||
|
|------|-------|------------|
|
||||||
|
| NUC8 RAM insufficient for 7B with other apps | Medium | 8G reservation; close other apps for 7B |
|
||||||
|
| Model download issues | Low | Provide direct HF links in README |
|
||||||
|
| OpenWebUI API compatibility | Low | llama.cpp v1 API is OpenAI-compatible |
|
||||||
|
| Intel AVX2 performance | Low | llama.cpp auto-detects and uses AVX2 |
|
||||||
@@ -0,0 +1,93 @@
|
|||||||
|
# Plan: Update AGENTS.md with Commit/Test/Build/Push Workflow
|
||||||
|
|
||||||
|
## Context
|
||||||
|
|
||||||
|
The repo's AGENTS.md (section 4 "Arbetsflöde för ändringar") currently only mentions `./scripts/validate-appstore.sh` as a loose recommendation. It lacks detailed guidance on the full development lifecycle: committing, testing images, building the appstore zip, and pushing.
|
||||||
|
|
||||||
|
## Goal
|
||||||
|
|
||||||
|
Add a new section to AGENTS.md (or expand section 4) covering the full workflow:
|
||||||
|
|
||||||
|
### A. Branch & Commit Workflow
|
||||||
|
- Branch naming per existing section 8
|
||||||
|
- Single-focus commits (one logical change per commit)
|
||||||
|
- Commit message format: short summary + bullet points for details
|
||||||
|
- What files can be committed (Apps/ scope rules from section 9)
|
||||||
|
|
||||||
|
### B. Image Verification (Pre-commit)
|
||||||
|
- Before committing compose changes, verify images are online:
|
||||||
|
```bash
|
||||||
|
docker manifest inspect <image:tag@sha256:...>
|
||||||
|
```
|
||||||
|
- Alternative: use build-appstore-zip.sh which does this automatically
|
||||||
|
|
||||||
|
### C. Local Validation
|
||||||
|
- Always run before push/PR:
|
||||||
|
```bash
|
||||||
|
./scripts/validate-appstore.sh
|
||||||
|
```
|
||||||
|
|
||||||
|
### D. Building the Appstore Package
|
||||||
|
- Script: `./scripts/build-appstore-zip.sh`
|
||||||
|
- Outputs to `dist/phirna-appstore.zip`
|
||||||
|
- Auto-generates SHA256 checksum
|
||||||
|
- Verifies all images online before building
|
||||||
|
- Commits `dist/` separately from app changes
|
||||||
|
|
||||||
|
### E. Push & Release
|
||||||
|
- Push order: app commits → build zip → commit zip → push
|
||||||
|
- PR description must include (per existing section 6):
|
||||||
|
- Affected app IDs
|
||||||
|
- Security risk level
|
||||||
|
- High-risk settings changes
|
||||||
|
|
||||||
|
## Proposed New Section (12) in AGENTS.md
|
||||||
|
|
||||||
|
```
|
||||||
|
## 12) Release- och publiceringsarbetsflöde
|
||||||
|
|
||||||
|
### Steg 1: Branch
|
||||||
|
Skapa branch enligt format i sektion 8:
|
||||||
|
<appnamn>/<initial|bugfix|update>/<detalj>
|
||||||
|
|
||||||
|
### Steg 2: Verifiera images (innan commit)
|
||||||
|
Kontrollera att alla Docker-images är tillgängliga online:
|
||||||
|
docker manifest inspect <image:tag@sha256:...>
|
||||||
|
|
||||||
|
### Steg 3: Validera lokalt
|
||||||
|
Kör validering innan commit:
|
||||||
|
./scripts/validate-appstore.sh
|
||||||
|
|
||||||
|
### Steg 4: Committa ändringar
|
||||||
|
- Små, reviewbara commits
|
||||||
|
- Separera appfiler från dist/-filer
|
||||||
|
- Commit-meddelande: rubrik + bulletpunkter
|
||||||
|
|
||||||
|
### Steg 5: Bygg appstore-zip
|
||||||
|
./scripts/build-appstore-zip.sh
|
||||||
|
- Skapar dist/phirna-appstore.zip
|
||||||
|
- Verifierar alla images
|
||||||
|
- Genererar SHA256
|
||||||
|
|
||||||
|
### Steg 6: Committa dist/
|
||||||
|
Separer commit för dist/ från appfiler:
|
||||||
|
git add dist/ && git commit -m "Build appstore zip"
|
||||||
|
|
||||||
|
### Steg 7: Push och PR
|
||||||
|
git push -u origin <branch>
|
||||||
|
Skapa PR med:
|
||||||
|
- Vilka app-id som påverkas
|
||||||
|
- Säkerhetsrisk (låg/medel/hög)
|
||||||
|
- Högrisk-inställningar vid introduktion
|
||||||
|
```
|
||||||
|
|
||||||
|
## Implementation
|
||||||
|
|
||||||
|
1. Read current AGENTS.md
|
||||||
|
2. Insert new section 12 after existing section 11
|
||||||
|
3. Renumber subsequent sections (12 → 13, etc.)
|
||||||
|
|
||||||
|
## Questions for User
|
||||||
|
|
||||||
|
- Should this be a new numbered section or expand existing section 4?
|
||||||
|
- Is `dist/phirna-appstore.zip` the correct output name for all repos, or should this be configurable?
|
||||||
@@ -0,0 +1,170 @@
|
|||||||
|
# Plan: Gitea Bot User Setup for Tea CLI
|
||||||
|
|
||||||
|
## Context
|
||||||
|
|
||||||
|
Enable the agent (Kilo) to interact with Gitea (git.phirna.uk) via the `tea` CLI for:
|
||||||
|
- Creating branches
|
||||||
|
- Committing and pushing changes
|
||||||
|
- Creating pull requests
|
||||||
|
- Managing issues and labels
|
||||||
|
|
||||||
|
## Step 1: Username Suggestion
|
||||||
|
|
||||||
|
**Suggested username: `kilo-bot`**
|
||||||
|
|
||||||
|
While not directly Norse mythology, "Kilo" evokes the Norse root meaning "coal" or "torch". Alternatives if you prefer pure mythology:
|
||||||
|
|
||||||
|
| Username | Origin |
|
||||||
|
|----------|--------|
|
||||||
|
| `kilo-bot` | Kilo = "torch of life" from Old Norse "Kjöl" |
|
||||||
|
| `mimir-bot` | Mimir - Norse god of wisdom, keeper of knowledge |
|
||||||
|
| `hnir-bio` | Hnir - "breath" in Old Norse |
|
||||||
|
| `sowilo-bot` | Sowilo - the S rune, meaning "sun" |
|
||||||
|
|
||||||
|
**Recommendation**: `kilo-bot` — maintains brand consistency with the agent name "Kilo".
|
||||||
|
|
||||||
|
## Step 2: Required Permissions
|
||||||
|
|
||||||
|
Based on Gitea granular scopes, the bot needs:
|
||||||
|
|
||||||
|
| Scope | Reason |
|
||||||
|
|-------|--------|
|
||||||
|
| `write:repository` | Create branches, push commits, create PRs |
|
||||||
|
| `read:repository` | Read branches, commits, repos |
|
||||||
|
| `read:user` | Identify authenticated user |
|
||||||
|
| `write:issue` | Create/update issues if needed |
|
||||||
|
| `read:org` | Read org membership if needed |
|
||||||
|
|
||||||
|
**Alternative**: Use `write:repository, read:user` for minimal permissions covering all git operations.
|
||||||
|
|
||||||
|
**NOT needed**: `admin` (would allow deleting repos, managing orgs, etc.)
|
||||||
|
|
||||||
|
## Step 3: Create the Bot User
|
||||||
|
|
||||||
|
Requires admin access on git.phirna.uk. Steps:
|
||||||
|
|
||||||
|
1. Go to `https://git.phirna.uk/admin/users/new` (or use `tea admin`)
|
||||||
|
2. Create user `kilo-bot` with email (e.g., `kilo-bot@phirna.uk`)
|
||||||
|
3. Set a strong random password (store in password manager)
|
||||||
|
4. Optionally add to relevant organization(s)
|
||||||
|
|
||||||
|
## Step 4: Generate Access Token
|
||||||
|
|
||||||
|
1. Login as `kilo-bot`
|
||||||
|
2. Go to Settings → Applications → "Manage Access Tokens"
|
||||||
|
3. Create token with name `kilo-cli` and scopes:
|
||||||
|
- `repository:write`
|
||||||
|
- `user:read`
|
||||||
|
4. Copy the generated token securely
|
||||||
|
|
||||||
|
## Step 5: Configure Tea
|
||||||
|
|
||||||
|
```bash
|
||||||
|
tea logins add --name kilo-bot --url https://git.phirna.uk --token <generated-token>
|
||||||
|
```
|
||||||
|
|
||||||
|
Or set environment variable `GITEA_TOKEN` or configure in `~/.config/tea/config.yml`.
|
||||||
|
|
||||||
|
## Step 6: Update AGENTS.md
|
||||||
|
|
||||||
|
Add new section or subsection covering:
|
||||||
|
|
||||||
|
- Bot user credentials (token) storage approach
|
||||||
|
- Expected token scopes
|
||||||
|
- tea command patterns for common operations
|
||||||
|
- Security considerations (bot has limited scope)
|
||||||
|
|
||||||
|
## Step 7: Create Skill (optional but recommended)
|
||||||
|
|
||||||
|
Create `.kilo/.skills/gitea-agent.md`:
|
||||||
|
|
||||||
|
- Standardized tea commands for branch creation
|
||||||
|
- Commit/push workflow via tea
|
||||||
|
- PR creation commands
|
||||||
|
- Issue management shortcuts
|
||||||
|
|
||||||
|
## Security Considerations
|
||||||
|
|
||||||
|
- Bot should **NOT** be admin
|
||||||
|
- Token should be scoped to `write:repository` + `read:user` only
|
||||||
|
- Token stored in environment or secured config, NOT in repo
|
||||||
|
- Consider setting bot's `MaxTokenPermissions` at org level if supported
|
||||||
|
|
||||||
|
## Verification Steps
|
||||||
|
|
||||||
|
After setup, test:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Verify identity
|
||||||
|
tea whoami --login kilo-bot
|
||||||
|
|
||||||
|
# List repos (should see assigned repos)
|
||||||
|
tea repos list --login kilo-bot
|
||||||
|
|
||||||
|
# Create a test branch
|
||||||
|
git checkout -b test/tea-cli-test
|
||||||
|
git push -u origin test/tea-cli-test --force
|
||||||
|
```
|
||||||
|
|
||||||
|
## Confirmed Decisions
|
||||||
|
|
||||||
|
| Decision | Choice |
|
||||||
|
|----------|--------|
|
||||||
|
| Username | `mimir` (already created on Gitea) |
|
||||||
|
| Token storage | Tea login system (`tea logins add`) |
|
||||||
|
| Access scope | All repos on git.phirna.uk |
|
||||||
|
|
||||||
|
## Implementation Steps
|
||||||
|
|
||||||
|
### Step 1: Generate Access Token for mimir
|
||||||
|
|
||||||
|
1. Login to `https://git.phirna.uk` as `mimir`
|
||||||
|
2. Go to Settings → Applications → "Manage Access Tokens"
|
||||||
|
3. Create token with name `kilo-cli` and scopes:
|
||||||
|
- `repository:write`
|
||||||
|
- `user:read`
|
||||||
|
4. Copy the generated token securely
|
||||||
|
|
||||||
|
### Step 2: Add Tea Login
|
||||||
|
|
||||||
|
```bash
|
||||||
|
tea logins add --name mimir --url https://git.phirna.uk --token <generated-token>
|
||||||
|
```
|
||||||
|
|
||||||
|
Verify with:
|
||||||
|
```bash
|
||||||
|
tea whoami --login mimir
|
||||||
|
```
|
||||||
|
|
||||||
|
### Step 3: Update AGENTS.md
|
||||||
|
|
||||||
|
Add new section under "Repo-invarianter" or create new section:
|
||||||
|
|
||||||
|
```
|
||||||
|
## X) Gitea Bot (mimir)
|
||||||
|
|
||||||
|
För att kunna skapa branches, commits och PRs via tea:
|
||||||
|
|
||||||
|
- Användare: `mimir` på git.phirna.uk
|
||||||
|
- Token lagras i tea's login system (`tea logins add`)
|
||||||
|
- Scopes: `repository:write`, `user:read`
|
||||||
|
- Säkerhet: mimir är inte admin, token har begränsade scopes
|
||||||
|
|
||||||
|
Exempel-kommandon:
|
||||||
|
tea pr create --login mimir --owner <owner> --repo <repo>
|
||||||
|
tea pulls list --login mimir --owner <owner> --repo <repo>
|
||||||
|
```
|
||||||
|
|
||||||
|
### Step 4: Create Skill (optional)
|
||||||
|
|
||||||
|
Create `.kilo/.skills/gitea-agent.md` with tea command patterns.
|
||||||
|
|
||||||
|
### Step 5: Verify Access
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Should show mimir's identity
|
||||||
|
tea whoami --login mimir
|
||||||
|
|
||||||
|
# Should list repos mimir can access
|
||||||
|
tea repos list --login mimir
|
||||||
|
```
|
||||||
@@ -16,7 +16,9 @@ Fokus: korrekthet, låg risk och underhållbarhet.
|
|||||||
- `docker-compose.yaml`
|
- `docker-compose.yaml`
|
||||||
- `README.md` (syfte, portar, volymer, privilegier, risker)
|
- `README.md` (syfte, portar, volymer, privilegier, risker)
|
||||||
- Compose ska ha giltig top-level `name` (gemener + bindestreck).
|
- Compose ska ha giltig top-level `name` (gemener + bindestreck).
|
||||||
|
- Om `container_name` används måste den vara max 32 tecken och bara innehålla `0-9`, `a-z`, `A-Z`, `_`, `-`.
|
||||||
- Endast `.yaml` används i repot (aldrig `.yml`).
|
- Endast `.yaml` används i repot (aldrig `.yml`).
|
||||||
|
- Swap-filer (t.ex. `*.swp`, `*.swo`) ska alltid ignoreras och får aldrig committas.
|
||||||
|
|
||||||
## 3) Säkerhetsbaseline (Compose)
|
## 3) Säkerhetsbaseline (Compose)
|
||||||
|
|
||||||
@@ -24,6 +26,9 @@ MUST:
|
|||||||
- Pinna images till explicit version eller digest.
|
- Pinna images till explicit version eller digest.
|
||||||
- Inte använda `:latest`.
|
- Inte använda `:latest`.
|
||||||
- Hålla volymer snäva (`/DATA/AppData/$AppID/...`).
|
- Hålla volymer snäva (`/DATA/AppData/$AppID/...`).
|
||||||
|
- Vid förslag/byte av `image:` måste imagen verifieras online (manifest/tag finns i registry) innan merge.
|
||||||
|
- Använd App/_template som grund om inget annat anges.
|
||||||
|
- deply.resources.reservations ska sättas till något passande. Detta kommer även parsas av ZimaOS webui
|
||||||
|
|
||||||
SHOULD:
|
SHOULD:
|
||||||
- `security_opt: ["no-new-privileges:true"]`
|
- `security_opt: ["no-new-privileges:true"]`
|
||||||
@@ -97,6 +102,11 @@ Regler för `<detalj>`:
|
|||||||
- När en ny app skapas ska en ny mapp alltid skapas under `Apps/<app-id>/`.
|
- När en ny app skapas ska en ny mapp alltid skapas under `Apps/<app-id>/`.
|
||||||
- Flera agenter kan arbeta samtidigt. Fråga alltid innan du skapar ny branch eller byter branch, för att undvika krockar.
|
- Flera agenter kan arbeta samtidigt. Fråga alltid innan du skapar ny branch eller byter branch, för att undvika krockar.
|
||||||
- `git add` och `git commit` får endast omfatta filer i den egna appens undermapp under `Apps/<app-id>/`.
|
- `git add` och `git commit` får endast omfatta filer i den egna appens undermapp under `Apps/<app-id>/`.
|
||||||
|
- För varje fil som en agent tänker ändra ska agenten först skapa en tom låsfil: `<filnamn>.agent_wip`.
|
||||||
|
- Om `<filnamn>.agent_wip` redan finns ska agenten vänta 60 sekunder och kontrollera igen.
|
||||||
|
- Upprepa väntan i upp till 10 minuter (10 försök). Om låsfilen fortfarande finns efter 10 minuter: avbryt och be användaren om beslut.
|
||||||
|
- När agenten är klar med filen ska motsvarande `<filnamn>.agent_wip` tas bort direkt.
|
||||||
|
- Låsfilen ska tas bort även vid fel/avbrott så långt det går, för att undvika falska lås.
|
||||||
|
|
||||||
## 10) Verifieringsdokument när app är redo
|
## 10) Verifieringsdokument när app är redo
|
||||||
|
|
||||||
@@ -112,9 +122,97 @@ Filen ska vara praktiskt körbar och innehålla:
|
|||||||
- exakta kommandon för att verifiera DNS, nät och TLS,
|
- exakta kommandon för att verifiera DNS, nät och TLS,
|
||||||
- en sektion \"data att samla\" för snabb Codex-felsökning.
|
- en sektion \"data att samla\" för snabb Codex-felsökning.
|
||||||
|
|
||||||
Sektionen \"data att samla\" ska minst täcka:
|
Sektionen "data att samla" ska minst täcka:
|
||||||
|
|
||||||
- versions-/buildinfo (appversion, branch/commit eller zip + checksum),
|
- versions-/buildinfo (appversion, branch/commit eller zip + checksum),
|
||||||
- relevant konfiguration (med maskade secrets),
|
- relevant konfiguration (med maskade secrets),
|
||||||
- loggar från berörda containers,
|
- loggar från berörda containers,
|
||||||
- konkreta felobservationer (hostname, tidpunkt, förväntat vs faktiskt beteende).
|
- konkreta felobservationer (hostname, tidpunkt, förväntat vs faktiskt beteende).
|
||||||
|
|
||||||
|
<<<<<<< HEAD
|
||||||
|
## 11) Release- och publiceringsarbetsflöde
|
||||||
|
|
||||||
|
### Steg 1: Branch
|
||||||
|
Skapa branch enligt format i sektion 8:
|
||||||
|
`<appnamn>/<initial|bugfix|update>/<detalj>`
|
||||||
|
|
||||||
|
### Steg 2: Verifiera images (innan commit)
|
||||||
|
Kontrollera att alla Docker-images är tillgängliga online. Scriptet `build-appstore-zip.sh` verifierar automatiskt -- kör det för att kontrollera, eller använd:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker manifest inspect <image:tag@sha256:...>
|
||||||
|
```
|
||||||
|
|
||||||
|
### Steg 3: Validera lokalt
|
||||||
|
Kör validering innan commit:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
./scripts/validate-appstore.sh
|
||||||
|
```
|
||||||
|
|
||||||
|
### Steg 4: Committa ändringar
|
||||||
|
- Små, reviewbara commits.
|
||||||
|
- Separera appfiler från `dist/`-filer.
|
||||||
|
- Commit-meddelande: rubrik + bulletpunkter.
|
||||||
|
|
||||||
|
### Steg 5: Bygg appstore-zip
|
||||||
|
|
||||||
|
```bash
|
||||||
|
./scripts/build-appstore-zip.sh
|
||||||
|
```
|
||||||
|
|
||||||
|
- Skapar `dist/phirna-appstore.zip`.
|
||||||
|
- Verifierar alla images online automatiskt.
|
||||||
|
- Genererar SHA256 checksum.
|
||||||
|
- Med `CI=true` eller `--strict-images` misslyckas bygget om en image saknas.
|
||||||
|
|
||||||
|
### Steg 6: Committa dist/
|
||||||
|
Separer commit för `dist/` från appfiler:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git add dist/ && git commit -m "Build appstore zip"
|
||||||
|
```
|
||||||
|
|
||||||
|
### Steg 7: Push och PR
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git push -u origin <branch>
|
||||||
|
```
|
||||||
|
|
||||||
|
PR ska inkludera:
|
||||||
|
- Vilka app-id som påverkas.
|
||||||
|
- Säkerhetsrisk (låg/medel/hög).
|
||||||
|
- Högrisk-inställningar vid introduktion eller förändring.
|
||||||
|
|
||||||
|
## 11) Gitea Bot (mimir)
|
||||||
|
|
||||||
|
För att kunna skapa branches, commits och PRs via tea-CLI:
|
||||||
|
|
||||||
|
- **Användare**: `mimir` på git.phirna.uk
|
||||||
|
- **Token**: Lagras i tea's login-system via `tea logins add`
|
||||||
|
- **Scopes**: `repository:write`, `user:read`
|
||||||
|
- **Säkerhet**: mimir är inte admin, token har begränsade scopes
|
||||||
|
|
||||||
|
### Vanliga kommandon
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Sätt aktiv login
|
||||||
|
export GITEA_LOGIN=mimir
|
||||||
|
|
||||||
|
# Lista repos
|
||||||
|
tea repos list --login mimir
|
||||||
|
|
||||||
|
# Skapa branch och push
|
||||||
|
git checkout -b <branch-name>
|
||||||
|
git push -u origin <branch-name>
|
||||||
|
|
||||||
|
# Skapa PR
|
||||||
|
tea pulls create --login mimir --owner <owner> --repo <repo> --head <branch> --base <target>
|
||||||
|
|
||||||
|
# Lista öppna PRs
|
||||||
|
tea pulls list --login mimir --owner <owner> --repo <repo>
|
||||||
|
|
||||||
|
# Hantera issues
|
||||||
|
tea issues list --login mimir --owner <owner> --repo <repo>
|
||||||
|
tea issues create --login mimir --owner <owner> --repo <repo> --title "Titel" --body "Body"
|
||||||
|
```
|
||||||
|
|||||||
@@ -2,19 +2,24 @@ name: sample-app
|
|||||||
|
|
||||||
services:
|
services:
|
||||||
app:
|
app:
|
||||||
image: ghcr.io/example/sample-app:1.0.0
|
# Setting this will allow the Zima webui to display the field "REQUIRE MEMORY"
|
||||||
|
deploy:
|
||||||
|
resources:
|
||||||
|
reservations:
|
||||||
|
memory: 64M
|
||||||
|
image: joafri/sample-app-app:main
|
||||||
container_name: sample-app
|
container_name: sample-app
|
||||||
restart: unless-stopped
|
restart: unless-stopped
|
||||||
|
|
||||||
environment:
|
environment:
|
||||||
TZ: ${TZ}
|
TZ: Europe/Stockholm
|
||||||
PUID: ${PUID}
|
PUID: "1000"
|
||||||
PGID: ${PGID}
|
PGID: "1000"
|
||||||
WEBUI_PORT: ${WEBUI_PORT:-8080}
|
WEBUI_PORT: "8080"
|
||||||
|
|
||||||
ports:
|
ports:
|
||||||
- target: 8080
|
- target: 8080
|
||||||
published: ${WEBUI_PORT:-8080}
|
published: "8080"
|
||||||
protocol: tcp
|
protocol: tcp
|
||||||
|
|
||||||
volumes:
|
volumes:
|
||||||
@@ -53,10 +58,10 @@ x-casaos:
|
|||||||
- amd64
|
- amd64
|
||||||
- arm64
|
- arm64
|
||||||
main: app
|
main: app
|
||||||
category: Utilities
|
category: phirna
|
||||||
author: Zima Apps Team
|
author: Joachim Friberg
|
||||||
developer: example
|
developer: Joachim Friberg
|
||||||
icon: https://example.invalid/icon.png
|
icon: https://cdn.simpleicons.org/docker
|
||||||
tagline:
|
tagline:
|
||||||
en_US: Replace with a short one-line value proposition
|
en_US: Replace with a short one-line value proposition
|
||||||
description:
|
description:
|
||||||
@@ -64,4 +69,4 @@ x-casaos:
|
|||||||
title:
|
title:
|
||||||
en_US: Sample App
|
en_US: Sample App
|
||||||
index: /
|
index: /
|
||||||
port_map: ${WEBUI_PORT:-8080}
|
port_map: "8080"
|
||||||
|
|||||||
@@ -0,0 +1,130 @@
|
|||||||
|
# ARK: Survival Ascended Server
|
||||||
|
|
||||||
|
ARK: Survival Ascended dedicated server container for ZimaOS.
|
||||||
|
|
||||||
|
## Overview
|
||||||
|
|
||||||
|
This app provides a dedicated server for ARK: Survival Ascended, allowing you to host your own game server with automatic updates and remote administration capabilities.
|
||||||
|
|
||||||
|
## Features
|
||||||
|
|
||||||
|
- **Automatic Updates**: Server files are automatically updated on startup
|
||||||
|
- **Cluster Support**: Configure multiple servers to work together
|
||||||
|
- **Remote Administration**: RCON access for server management
|
||||||
|
- **Customizable**: Adjust game parameters, player limits, and maps
|
||||||
|
- **Secure by Default**: Runs with minimal privileges and capabilities
|
||||||
|
|
||||||
|
## Configuration
|
||||||
|
|
||||||
|
### Ports
|
||||||
|
|
||||||
|
- **7777/udp**: Game port for player connections
|
||||||
|
- **27020/tcp**: RCON port for remote server administration
|
||||||
|
|
||||||
|
### Volumes
|
||||||
|
|
||||||
|
All volumes are mounted under `/DATA/AppData/ark-suac/`:
|
||||||
|
|
||||||
|
- `steam`: Steam runtime files
|
||||||
|
- `steamcmd`: SteamCMD installation
|
||||||
|
- `server-files`: ARK server files and save data
|
||||||
|
- `cluster-shared`: Shared data for cluster setups
|
||||||
|
|
||||||
|
### Environment Variables
|
||||||
|
|
||||||
|
- `TZ`: Timezone (default: Europe/Stockholm)
|
||||||
|
- `ASA_START_PARAMS`: Server start parameters and map configuration
|
||||||
|
- `ENABLE_DEBUG`: Set to "1" to enable debug mode (prevents server startup)
|
||||||
|
|
||||||
|
## Security
|
||||||
|
|
||||||
|
### Baseline Security
|
||||||
|
|
||||||
|
- Runs as non-root user `gameserver`
|
||||||
|
- All capabilities dropped (`cap_drop: ALL`)
|
||||||
|
- `no-new-privileges` security option enabled
|
||||||
|
- No privileged mode required
|
||||||
|
- No Docker socket access
|
||||||
|
|
||||||
|
### Permission Bootstrap
|
||||||
|
|
||||||
|
The compose includes a one-shot helper service `set-permissions` that runs as `root`
|
||||||
|
before the game server starts. It only performs `chown` on app-specific bind mounts so
|
||||||
|
the main `ark-asa-server` service can run as non-root `gameserver`.
|
||||||
|
|
||||||
|
### Data Persistence
|
||||||
|
|
||||||
|
All game data, including server files, save games, and configuration, is persisted in the bound volumes. The container will automatically download and update server files on startup.
|
||||||
|
|
||||||
|
## Usage
|
||||||
|
|
||||||
|
### First Start
|
||||||
|
|
||||||
|
1. Install the app through ZimaOS App Store
|
||||||
|
2. Configure desired map and settings via `ASA_START_PARAMS`
|
||||||
|
3. Start the container - first startup will download ~11GB of server files
|
||||||
|
4. Connect to your server via the ARK game client
|
||||||
|
|
||||||
|
### Map Configuration
|
||||||
|
|
||||||
|
Change the map by modifying `ASA_START_PARAMS`. Example maps:
|
||||||
|
|
||||||
|
- `TheIsland_WP` (default)
|
||||||
|
- `ScorchedEarth_WP`
|
||||||
|
- `Ragnarok_WP`
|
||||||
|
- `Valguero_WP`
|
||||||
|
|
||||||
|
### Cluster Setup
|
||||||
|
|
||||||
|
For multi-server clusters:
|
||||||
|
|
||||||
|
1. Use the same `clusterid` parameter for all servers
|
||||||
|
2. Share the `cluster-shared` volume between containers
|
||||||
|
3. Configure different ports for each server instance
|
||||||
|
|
||||||
|
### Remote Administration
|
||||||
|
|
||||||
|
Connect to RCON on port 27020 using tools like:
|
||||||
|
|
||||||
|
- [ARK Server API](https://ark-server-api.com/)
|
||||||
|
- [Battlemetrics RCON](https://www.battlemetrics.com/rcon)
|
||||||
|
- Command line: `nc hostname 27020`
|
||||||
|
|
||||||
|
## Performance Requirements
|
||||||
|
|
||||||
|
- **RAM**: ~13GB per server instance
|
||||||
|
- **Disk**: ~11GB for server files + space for save games
|
||||||
|
- **CPU**: Modern multi-core processor recommended
|
||||||
|
|
||||||
|
## Updates
|
||||||
|
|
||||||
|
Server files are automatically updated on container startup. To update the container image itself, pull the new version through ZimaOS.
|
||||||
|
|
||||||
|
## Troubleshooting
|
||||||
|
|
||||||
|
### Server Not Visible
|
||||||
|
|
||||||
|
- Verify port 7777/udp is properly forwarded
|
||||||
|
- Check firewall settings
|
||||||
|
- Ensure `ASA_START_PARAMS` includes `?listen` parameter
|
||||||
|
|
||||||
|
### Connection Timeouts
|
||||||
|
|
||||||
|
- Verify your network configuration
|
||||||
|
- Check that UDP traffic is not being blocked
|
||||||
|
- Ensure the server has completed initialization (can take several minutes)
|
||||||
|
|
||||||
|
### Debug Mode
|
||||||
|
|
||||||
|
Set `ENABLE_DEBUG=1` to prevent server startup and investigate container issues.
|
||||||
|
|
||||||
|
## References
|
||||||
|
|
||||||
|
- [Upstream Repository](https://github.com/mschnitzer/asa-linux-server)
|
||||||
|
- [ARK Official Website](https://survivetheark.com/)
|
||||||
|
- [SteamCMD Documentation](https://developer.valvesoftware.com/wiki/SteamCMD)
|
||||||
|
|
||||||
|
## Support
|
||||||
|
|
||||||
|
For issues specific to this ZimaOS integration, please contact the Zima Apps Team.
|
||||||
|
For game server issues, refer to the upstream repository or ARK official support.
|
||||||
@@ -0,0 +1 @@
|
|||||||
|
root/usr/share/proton/GE-Proton8-21.tar.gz filter=lfs diff=lfs merge=lfs -text
|
||||||
@@ -0,0 +1,20 @@
|
|||||||
|
.PHONY: prepare build build-development build-beta load
|
||||||
|
|
||||||
|
GLOBAL_BUILD_DIR = /tmp/.kiwi-build-results
|
||||||
|
TARGET_DIR = $(GLOBAL_BUILD_DIR)/ark-survival-ascended-linux-container-image
|
||||||
|
|
||||||
|
prepare:
|
||||||
|
- sudo rm -rf $(TARGET_DIR)
|
||||||
|
- mkdir -p $(GLOBAL_BUILD_DIR)
|
||||||
|
|
||||||
|
build: prepare
|
||||||
|
- sudo kiwi-ng --profile stable --color-output --debug system build --target-dir $(TARGET_DIR) --description .
|
||||||
|
|
||||||
|
build-development: prepare
|
||||||
|
- sudo kiwi-ng --profile development --color-output --debug system build --target-dir $(TARGET_DIR) --description .
|
||||||
|
|
||||||
|
build-beta: prepare
|
||||||
|
- sudo kiwi-ng --profile beta --color-output --debug system build --target-dir $(TARGET_DIR) --description .
|
||||||
|
|
||||||
|
load:
|
||||||
|
- sudo docker load -i $(TARGET_DIR)/*.xz
|
||||||
@@ -0,0 +1,592 @@
|
|||||||
|
# ARK: Survival Ascended - Dedicated Linux Server - Docker Image
|
||||||
|
|
||||||
|
This repository provides a step by step guide for Linux administrators to host ARK: Survival Ascended servers on Linux using a docker image.
|
||||||
|
|
||||||
|
## Table of Contents
|
||||||
|
|
||||||
|
* [Hardware Requirements](#hardware-requirements)
|
||||||
|
* [Installation](#installation)
|
||||||
|
* [Install Docker & Docker Compose](#1-install-docker--docker-compose)
|
||||||
|
* [Start docker daemon](#2-start-docker-daemon)
|
||||||
|
* [Create the Docker Compose config](#3-create-the-docker-compose-config)
|
||||||
|
* [First server start](#4-first-server-start)
|
||||||
|
* [Server configuration](#5-server-configuration)
|
||||||
|
* [Changing the start parameters AND the player limit](#6-changing-the-start-parameters-and-the-player-limit)
|
||||||
|
* [Port forwarding?](#port-forwarding)
|
||||||
|
* [Changing the game port and RCON port](#changing-the-game-port-and-rcon-port)
|
||||||
|
* [Start/Restart/Stop](#startrestartstop)
|
||||||
|
* [Server Administration](#server-administration)
|
||||||
|
* [Debug Mode](#debug-mode)
|
||||||
|
* [Applying server updates](#applying-server-updates)
|
||||||
|
* [Daily restarts](#daily-restarts)
|
||||||
|
* [Executing RCON commands](#executing-rcon-commands)
|
||||||
|
* [Setting up a second server / cluster](#setting-up-a-second-server--cluster)
|
||||||
|
* [Adding Mods](#adding-mods)
|
||||||
|
* [Adding Mod Maps](#adding-mod-maps)
|
||||||
|
* [Adding Plugins](#adding-plugins)
|
||||||
|
* [Map Names](#map-names)
|
||||||
|
* [Updating the Container Image](#updating-the-container-image)
|
||||||
|
* [Common Issues](#common-issues)
|
||||||
|
* [Server is not visible in server browser](#server-is-not-visible-in-server-browser)
|
||||||
|
* [Addressing "Connection Timeout" issues](#addressing-connection-timeout-issues)
|
||||||
|
* [Your server has multiple IPv4 addresses](#your-server-has-multiple-ipv4-addresses)
|
||||||
|
* [Debugging with curl](#debugging-with-curl)
|
||||||
|
* [How to customize your routing?](#how-to-customize-your-routing)
|
||||||
|
* [Making your iptable rules persistent](#making-your-iptable-rules-persistent)
|
||||||
|
* [Found an Issue or Bug?](#found-an-issue-or-bug)
|
||||||
|
* [Credits](#credits)
|
||||||
|
|
||||||
|
## Hardware Requirements
|
||||||
|
|
||||||
|
The hardware requirements might change over time, but as of today you can expect:
|
||||||
|
|
||||||
|
* ~13 GB RAM usage per server instance
|
||||||
|
* ~11 GB disk space (the server files alone, without any savegames)
|
||||||
|
|
||||||
|
I cannot tell you what CPU to use, as I didn't do any testing on this, but this is the hardware I'm running one ASA server on:
|
||||||
|
|
||||||
|
* Intel Xeon E3-1275v5
|
||||||
|
* 2x SSD M.2 NVMe 512 GB
|
||||||
|
* 4x RAM 16384 MB DDR4 ECC
|
||||||
|
|
||||||
|
The server runs next to other services and it runs pretty well.
|
||||||
|
|
||||||
|
## Installation
|
||||||
|
|
||||||
|
Required Linux experience: **Beginner**
|
||||||
|
|
||||||
|
In theory, you can use these steps on any Linux system where Docker is installed. It has been tested with:
|
||||||
|
|
||||||
|
* openSUSE Leap 15.6 (>= 16.0 won't work due to missing 32bit support)
|
||||||
|
* Debian 12 (bookworm)
|
||||||
|
* **NOT WORKING:** Ubuntu 22.04.x LTS (Jammy Jellyfish) [As of March 28th 2025, a recent distro update causes the container to have a constant high CPU usage, well beyond 400% and the server won't launch. Use Ubuntu 24.04.x if you can]
|
||||||
|
* Ubuntu 24.04.1 (Noble Numbat)
|
||||||
|
|
||||||
|
You need to be root user (`su root`) to perform these steps, but don't worry, the ASA server itself will run rootless.
|
||||||
|
|
||||||
|
### 1. Install Docker & Docker Compose
|
||||||
|
|
||||||
|
#### openSUSE Leap 15.6:
|
||||||
|
|
||||||
|
```
|
||||||
|
zypper in -y docker docker-compose
|
||||||
|
```
|
||||||
|
|
||||||
|
#### Debian 12
|
||||||
|
|
||||||
|
It is recommended to install the docker engine from Docker's official repository. Follow the instructions in [this guide](https://docs.docker.com/engine/install/debian/#install-using-the-repository)
|
||||||
|
and refer to the "Install using the apt repository" section.
|
||||||
|
|
||||||
|
#### Ubuntu (24.04.x):
|
||||||
|
|
||||||
|
The docker engine is not part of the official Ubuntu 24.x repositories, thus you need to install it from the Docker's repository instead. Please refer to
|
||||||
|
[this guide](https://docs.docker.com/engine/install/ubuntu/#install-using-the-repository) and follow the steps outlined in the "Install using the apt repository" section.
|
||||||
|
|
||||||
|
### 2. Start docker daemon
|
||||||
|
|
||||||
|
```
|
||||||
|
systemctl start docker
|
||||||
|
systemctl enable docker
|
||||||
|
```
|
||||||
|
|
||||||
|
### 3. Create the Docker Compose config
|
||||||
|
|
||||||
|
Create a directory called `asa-server` wherever you like and download [my docker-compose.yml](https://github.com/mschnitzer/ark-survival-ascended-linux-container-image/blob/main/docker-compose.yml) example.
|
||||||
|
|
||||||
|
```
|
||||||
|
mkdir asa-server
|
||||||
|
cd asa-server
|
||||||
|
wget https://raw.githubusercontent.com/mschnitzer/ark-survival-ascended-linux-container-image/main/docker-compose.yml
|
||||||
|
```
|
||||||
|
|
||||||
|
### 4. First server start
|
||||||
|
|
||||||
|
Now start the server for the first time. It will install Steam, Proton, and downloads the ARK: Survival Ascended server files.
|
||||||
|
|
||||||
|
Go to the directory of your `docker-compose.yml` file and execute the following command:
|
||||||
|
|
||||||
|
```
|
||||||
|
docker compose up -d
|
||||||
|
```
|
||||||
|
|
||||||
|
It will download my docker image and then spins up a container called `asa-server-1` (defined in `docker-compose.yml`). You can follow the installation and the start of your server by running:
|
||||||
|
|
||||||
|
```
|
||||||
|
docker logs -f asa-server-1
|
||||||
|
```
|
||||||
|
|
||||||
|
(Note: You can safely run `CTRL + C` to exit the log window again without causing the server to stop)
|
||||||
|
|
||||||
|
Once the log shows the following line:
|
||||||
|
|
||||||
|
```
|
||||||
|
Starting the ARK: Survival Ascended dedicated server...
|
||||||
|
```
|
||||||
|
|
||||||
|
... the server should be reachable and discoverable through the server browser in ~2-5 minutes.
|
||||||
|
|
||||||
|
The server name is randomly generated upon the first start. Please execute the following command to see under which name the server is discoverable in the server browser:
|
||||||
|
```
|
||||||
|
docker exec asa-server-1 cat server-files/ShooterGame/Saved/Config/WindowsServer/GameUserSettings.ini | grep SessionName
|
||||||
|
```
|
||||||
|
|
||||||
|
If the command fails in execution and reports an `No such file or directory` error, just wait some more minutes and it should eventually work. Once the command executed successfully, it should output something like this:
|
||||||
|
```
|
||||||
|
SessionName=ARK #334850
|
||||||
|
```
|
||||||
|
|
||||||
|
Now try to find the server by its name. Just search in the "Unofficial" section in ASA for the number of the server. In my case it is `334850`. If you are not able to connect to it right away, wait up to 5 more minutes and
|
||||||
|
try it again. If it's still not possible, [open an issue on GitHub](https://github.com/mschnitzer/ark-survival-ascended-linux-container-image/issues/new) to get help.
|
||||||
|
|
||||||
|
Once confirmed that you are able to connect, stop the server again:
|
||||||
|
|
||||||
|
```
|
||||||
|
docker stop asa-server-1
|
||||||
|
```
|
||||||
|
|
||||||
|
### 5. Server configuration
|
||||||
|
|
||||||
|
The `docker-compose.yml` config defines three docker volumes, which serve as a storage for your server files, Steam, and Proton. They are directly mounted to the docker container and can be edited outside of the container. The
|
||||||
|
location of these volumes is `/var/lib/docker/volumes`. If you followed the steps 1:1, then you should find the following directories at that location:
|
||||||
|
|
||||||
|
```
|
||||||
|
asa-server_cluster-shared/
|
||||||
|
asa-server_server-files-1/
|
||||||
|
asa-server_steam-1/
|
||||||
|
asa-server_steamcmd-1/
|
||||||
|
```
|
||||||
|
|
||||||
|
The prefix `asa-server` is defined by the directory name of your `docker-compose.yml` file.
|
||||||
|
|
||||||
|
You can ignore `asa-server_steam-1` and `asa-server_steamcmd-1`, these volumes are being used by the container to avoid setting up `Steam` and `steamcmd` on every launch again. Server files including config files are stored at `asa-server_server-files-1`. `asa-server_cluster-shared` provides support for server clusters, so that survivors can travel between your servers with their characters and dinos.
|
||||||
|
|
||||||
|
The `GameUserSettings.ini` and `Game.ini` file can be found at `/var/lib/docker/volumes/asa-server_server-files-1/_data/ShooterGame/Saved/Config/WindowsServer`. The `Game.ini` file is not there by default, so you might want to create it yourself.
|
||||||
|
|
||||||
|
You don't need to worry about file permissions. The `docker-compose.yml` is running a container before starting the ASA server and adjusts the file permissions to `25000:25000`, which is the user id and group id the server starts with. These ids are not bound to any user on your system and that's fine and not an issue.
|
||||||
|
|
||||||
|
### 6. Changing the start parameters AND the player limit
|
||||||
|
|
||||||
|
Start parameters are defined in the `docker-compose.yml`:
|
||||||
|
|
||||||
|
```yml
|
||||||
|
...
|
||||||
|
environment:
|
||||||
|
- ASA_START_PARAMS=TheIsland_WP?listen?Port=7777?RCONPort=27020?RCONEnabled=True -WinLiveMaxPlayers=50
|
||||||
|
...
|
||||||
|
```
|
||||||
|
|
||||||
|
Please note:
|
||||||
|
* The value before `?listen` is the name of the map the server launches with. ([See all official map names](#map-names))
|
||||||
|
* Please do not remove `?listen` from the parameters, otherwise the server is not binding ports
|
||||||
|
* `?Port=` is the server port players connect to
|
||||||
|
* `?RCONPort=` is the port of the RCON server that allows remote administration of the server
|
||||||
|
* The player limit is set by `-WinLiveMaxPlayers`. Please note that for ASA servers, editing the player limit via `GameUserSettings.ini` is not working.
|
||||||
|
|
||||||
|
## Port forwarding?
|
||||||
|
|
||||||
|
There should not be the need to forward any ports if your server runs in a public cloud. This is because docker configures `iptables` by itself. In a home setup, where a router is in between, it is very likely that you need to forward ports.
|
||||||
|
|
||||||
|
In any case, you ONLY need to forward the following ports:
|
||||||
|
|
||||||
|
```
|
||||||
|
7777 (UDP only - This is the game port to allow players to connect to the server)
|
||||||
|
27020 (TCP only - This is the port to connect through RCON and is therefore optional to forward)
|
||||||
|
```
|
||||||
|
|
||||||
|
As of today, ASA does no longer offer a way to query the server, so there's no query port and you won't be able to find your server through the Steam server browser, only via the ingame browser.
|
||||||
|
|
||||||
|
## Changing the game port and RCON port
|
||||||
|
|
||||||
|
You already learned that ports are defined by `ASA_START_PARAMS` in the `docker-compose.yml` file. This just tells the ASA server what ports to bind.
|
||||||
|
As a first step for port changes adjust the start parameters accordingly.
|
||||||
|
|
||||||
|
E. g. if you want to change the game port from `7777` to `7755` your new start parameters would look like this:
|
||||||
|
|
||||||
|
```yml
|
||||||
|
...
|
||||||
|
environment:
|
||||||
|
- ASA_START_PARAMS=TheIsland_WP?listen?Port=7755?RCONPort=27020?RCONEnabled=True -WinLiveMaxPlayers=50 -clusterid=default -ClusterDirOverride="/home/gameserver/cluster-shared"
|
||||||
|
- ENABLE_DEBUG=0
|
||||||
|
...
|
||||||
|
```
|
||||||
|
|
||||||
|
But this alone is not enough and you need to apply the following changes as well.
|
||||||
|
|
||||||
|
Open the `docker-compose.yml` file again and edit the lines where the container ports are defined:
|
||||||
|
|
||||||
|
```yml
|
||||||
|
...
|
||||||
|
ports:
|
||||||
|
# Game port for player connections through the server browser
|
||||||
|
- 0.0.0.0:7777:7777/udp
|
||||||
|
# RCON port for remote server administration
|
||||||
|
- 0.0.0.0:27020:27020/tcp
|
||||||
|
...
|
||||||
|
```
|
||||||
|
|
||||||
|
Adjust the port to your liking, but make sure that you change both numbers (the one before and after the `:`). Assuming the above game port changes to `7755` this would be the result:
|
||||||
|
|
||||||
|
```yml
|
||||||
|
...
|
||||||
|
ports:
|
||||||
|
# Game port for player connections through the server browser
|
||||||
|
- 0.0.0.0:7755:7755/udp
|
||||||
|
# RCON port for remote server administration
|
||||||
|
- 0.0.0.0:27020:27020/tcp
|
||||||
|
...
|
||||||
|
```
|
||||||
|
|
||||||
|
Now that your port changes are set, you have to recreate your container. Therefore you need to use `docker compose up -d` in order to apply your port changes.
|
||||||
|
|
||||||
|
|
||||||
|
## Start/Restart/Stop
|
||||||
|
|
||||||
|
To perform any of the actions, execute the following commands (you need to be in the directory of the `docker-compose.yml` file):
|
||||||
|
|
||||||
|
```
|
||||||
|
docker compose start asa-server-1
|
||||||
|
docker compose restart asa-server-1
|
||||||
|
docker compose stop asa-server-1
|
||||||
|
```
|
||||||
|
|
||||||
|
You can also use the native docker commands, where you do not need to be in the directory of the `docker-compose.yml` file. However using this method would not check for changes in your `docker-compose.yml` file.
|
||||||
|
So in case you edited the `docker-compose.yml` file (e.g. because you adjusted the start parameters), you need to use `docker compose` commands instead.
|
||||||
|
```
|
||||||
|
docker start/restart/stop asa-server-1
|
||||||
|
```
|
||||||
|
|
||||||
|
## Server Administration
|
||||||
|
|
||||||
|
### Debug Mode
|
||||||
|
|
||||||
|
Sometimes you want to test something inside the container without starting the ASA server. The debug mode can be enabled by changing `- ENABLE_DEBUG=0` to `1` in the `docker-compose.yml` file.
|
||||||
|
Once done, the result will look like this:
|
||||||
|
|
||||||
|
```yml
|
||||||
|
...
|
||||||
|
version: "3.3"
|
||||||
|
services:
|
||||||
|
asa-server-1:
|
||||||
|
container_name: asa-server-1
|
||||||
|
hostname: asa-server-1
|
||||||
|
entrypoint: "/usr/bin/start_server"
|
||||||
|
user: gameserver
|
||||||
|
image: "mschnitzer/asa-linux-server:latest"
|
||||||
|
environment:
|
||||||
|
- ASA_START_PARAMS=TheIsland_WP?listen?Port=7777?RCONPort=27020?RCONEnabled=True -WinLiveMaxPlayers=50
|
||||||
|
- ENABLE_DEBUG=1
|
||||||
|
...
|
||||||
|
```
|
||||||
|
|
||||||
|
Now run `docker compose up -d` and the container will just start without launching the server or validating server files.
|
||||||
|
|
||||||
|
Check if the container launched in debug mode by running `docker logs -f asa-server-1` and check whether it's saying "Entering debug mode...". If that's the case, you are good.
|
||||||
|
|
||||||
|
You can enter the shell of your server by running
|
||||||
|
|
||||||
|
```
|
||||||
|
docker exec -ti asa-server-1 bash
|
||||||
|
```
|
||||||
|
|
||||||
|
If you need root access run
|
||||||
|
|
||||||
|
```
|
||||||
|
docker exec -ti -u root asa-server-1 bash
|
||||||
|
```
|
||||||
|
|
||||||
|
### Applying server updates
|
||||||
|
|
||||||
|
Updates will be automatically downloaded or applied once you restart the container with ...
|
||||||
|
|
||||||
|
```
|
||||||
|
docker restart asa-server-1
|
||||||
|
```
|
||||||
|
|
||||||
|
It is totally possible that after a restart and applying all updates, the client is still one or more versions ahead. This is because Wildcard does sometimes run client-only updates, since not all
|
||||||
|
updates are affecting the server software. This is not a problem at all. As long as you can connect to your server, everything is fine. The server software checks for incompatible client
|
||||||
|
versions anyway.
|
||||||
|
|
||||||
|
In general you can check when the latest server update was published by Wildcard, by checking [this link](https://steamdb.info/app/2430930/depots/). The section mentioning the last update of the `public` branch
|
||||||
|
tells you when the last update was rolled out for the server software.
|
||||||
|
|
||||||
|
If you have any doubts on this, open a GitHub issue.
|
||||||
|
|
||||||
|
### Daily restarts
|
||||||
|
|
||||||
|
As `root` user of your server (or any other user that is member of the `docker` group) open your crontab configuration:
|
||||||
|
|
||||||
|
```
|
||||||
|
crontab -e
|
||||||
|
```
|
||||||
|
|
||||||
|
Add the following lines to it:
|
||||||
|
```
|
||||||
|
30 3 * * * docker exec asa-server-1 asa-ctrl rcon --exec 'serverchat Server restart in 30 minutes'
|
||||||
|
50 3 * * * docker exec asa-server-1 asa-ctrl rcon --exec 'serverchat Server restart in 10 minutes'
|
||||||
|
57 3 * * * docker exec asa-server-1 asa-ctrl rcon --exec 'serverchat Server restart in 3 minutes'
|
||||||
|
58 3 * * * docker exec asa-server-1 asa-ctrl rcon --exec 'saveworld'
|
||||||
|
0 4 * * * docker restart asa-server-1
|
||||||
|
```
|
||||||
|
|
||||||
|
Explanation:
|
||||||
|
* Line 1: Every day at 03:30am of your server's timezone, a message will be sent to all players announcing a restart in 30 minutes.
|
||||||
|
* Line 2: Every day at 03:50am of your server's timezone, a message will be sent to all players announcing a restart in 10 minutes.
|
||||||
|
* Line 3: Every day at 03:57am of your server's timezone, a message will be sent to all players announcing a restart in 3 minutes.
|
||||||
|
* Line 4: Every day at 03:58am of your server's timezone, the server saves the world before the restart happens.
|
||||||
|
* Line 5: Every day at 04:00am of your server's timezone, the ASA server gets restarted and installs pending updates from Steam.
|
||||||
|
|
||||||
|
Read more about the crontab syntax [here](https://www.adminschoice.com/crontab-quick-reference).
|
||||||
|
|
||||||
|
**NOTE:** The first 4 lines execute RCON commands, which requires you to have a working RCON setup. Please follow the instructions in section "[Executing RCON commands](#executing-rcon-commands)" to
|
||||||
|
ensure you can execute RCON commands.
|
||||||
|
|
||||||
|
### Executing RCON commands
|
||||||
|
|
||||||
|
You can run RCON commands by accessing the `rcon` subcommand of the `asa-ctrl` tool which is shipped with the container image. There's no need to enter your server password, IP, or RCON port manually. As long as
|
||||||
|
you have set your RCON password and port, either as a start parameter or in the `GameUserSettings.ini` file of your server, `asa-ctrl` is able to figure those details out by itself.
|
||||||
|
|
||||||
|
The following variables need to be present in `GameUserSettings.ini` under the `[ServerSettings]` section:
|
||||||
|
|
||||||
|
```
|
||||||
|
RCONEnabled=True
|
||||||
|
ServerAdminPassword=mysecretpass
|
||||||
|
RCONPort=27020
|
||||||
|
```
|
||||||
|
|
||||||
|
**NOTE:** There can be issues setting `ServerAdminPassword` as command line option. I'd suggest to set it in the `GameUserSettings.ini` file only.
|
||||||
|
|
||||||
|
Example:
|
||||||
|
|
||||||
|
```
|
||||||
|
docker exec -t asa-server-1 asa-ctrl rcon --exec 'saveworld'
|
||||||
|
```
|
||||||
|
|
||||||
|
**NOTE:** As opposed to ingame cheat commands, you must not put `admincheat` or `cheat` in front of the command.
|
||||||
|
|
||||||
|
## Setting up a second server / cluster
|
||||||
|
|
||||||
|
Setting up a second server is quite easy and you can easily add more if you want (given that your hardware is capable of running multiple instances). There's already a definition for a second server in the `docker-compose.yml` file,
|
||||||
|
but the definition is commented out by a leading `#`. If you remove these `#`, and run `docker compose up -d` again, then the second server should start and it will listen on the game port `7778` and the query port `27021`. Please note that
|
||||||
|
the server files, as well as Steam, and steamcmd will be downloaded again and the first start can take a while.
|
||||||
|
|
||||||
|
You can edit the start parameters in the same way like for the first server and the files of the second server are located at the same location, except that the second server has its suffix changed from `-1` to `-2`. The directories will therefore,
|
||||||
|
named like this:
|
||||||
|
|
||||||
|
```
|
||||||
|
asa-server_server-files-2/
|
||||||
|
asa-server_steam-2/
|
||||||
|
asa-server_steamcmd-2/
|
||||||
|
```
|
||||||
|
|
||||||
|
That's it! Your second server is now running in a cluster setup. This means that travelling between your servers is possible through Obelisks. If you do not want players to travel between your servers, you need to remove the `-clusterid` option
|
||||||
|
from the start parameters. It's advised to change the `-clusterid` parameter for all of your servers to a random string and keep it secret (e.g. `-clusterid=aSM42F6PLaPk` as opposed to `-clusterid=default`). The reason for that is that you will
|
||||||
|
end up seeing also other servers from the community that use `default` as their `clusterid`. If you only want players to travel between your own servers, then the `clusterid` must be different.
|
||||||
|
|
||||||
|
If you want to spin up more servers, you need to add more entries to the `docker-compose.yml` file. The following sections need to be edited: `services` and `volumes`. Make sure that you adjust all suffixes and replace them with a new one
|
||||||
|
(e.g. `-3` now) for the newly added entries.
|
||||||
|
|
||||||
|
## Adding Mods
|
||||||
|
|
||||||
|
Mods can be added by adjusting the `docker-compose.yml` file and adding a `-mods` option to the start parameters.
|
||||||
|
|
||||||
|
e.g.
|
||||||
|
|
||||||
|
```
|
||||||
|
[...]
|
||||||
|
- ASA_START_PARAMS=TheIsland_WP?listen?Port=7777?RCONPort=27020?RCONEnabled=True -WinLiveMaxPlayers=50 -mods=12345,67891
|
||||||
|
[...]
|
||||||
|
```
|
||||||
|
|
||||||
|
Once done, restart the server using `docker compose up -d`. It might take longer until the server comes up, because the server has to download the mods first.
|
||||||
|
|
||||||
|
Mod IDs are usually somewhere listed on the mod page of a mod on curseforge.com.
|
||||||
|
|
||||||
|
### Adding Mod Maps
|
||||||
|
|
||||||
|
Search for a map on curseforge.com and find out what mod id the map has and what the map name is. For the map [Svartalfheim](https://www.curseforge.com/ark-survival-ascended/mods/svartalfheim) the map name
|
||||||
|
is `Svartalfheim_WP` and the mod id is `893657`.
|
||||||
|
|
||||||
|
Once you found out the information you need, you need to adjust your start parameters in the `docker-compose.yml` file and add the map name, as well as the `-mods` option.
|
||||||
|
|
||||||
|
e.g.
|
||||||
|
|
||||||
|
```
|
||||||
|
[...]
|
||||||
|
- ASA_START_PARAMS=Svartalfheim_WP?listen?Port=7777?RCONPort=27020?RCONEnabled=True -WinLiveMaxPlayers=50 -mods=893657
|
||||||
|
[...]
|
||||||
|
```
|
||||||
|
|
||||||
|
Restart your server using `docker compose up -d`. It may take a while, as the server has to download the map, so be patient.
|
||||||
|
|
||||||
|
## Adding Plugins
|
||||||
|
|
||||||
|
Plugin support was introduced by version 1.4.0 of this container image. So make sure that you updated to the latest version of the container image or to version 1.4.0 as described [here](#updating-the-container-image).
|
||||||
|
|
||||||
|
There's a project ([see here](https://gameservershub.com/forums/resources/ark-survival-ascended-serverapi-crossplay-supported.683/)) that allows you to load plugins on your server (e.g. permission handling). To install the plugin loader, please visit [gameservershub.com](https://gameservershub.com/forums/resources/ark-survival-ascended-serverapi-crossplay-supported.683/) and refer to the "ServerAPI Installation Steps" section and download the zip archive. A `gameservershub.com` account is required in order to download the plugin loader.
|
||||||
|
|
||||||
|
When the download of the zip archive is completed, follow these steps to install the plugin loader:
|
||||||
|
|
||||||
|
1. Make sure that you launched the ASA server at least once without the plugin loader.
|
||||||
|
2. Stop the ASA server container by running `docker stop asa-server-1`
|
||||||
|
3. Enter the server files binary directory as `root` user: `cd /var/lib/docker/volumes/asa-server_server-files-1/_data/ShooterGame/Binaries/Win64`
|
||||||
|
4. Place the downloaded zip archive in that directory (the name of the archive must start with `AsaApi_`). Do not unzip the content.
|
||||||
|
5. Restart your server using `docker compose up -d`
|
||||||
|
|
||||||
|
The installation happens automatically by the container start script. You can follow the installation process by running `docker logs -f asa-server-1`. Once the log says "Detected ASA Server API loader. Launching server through AsaApiLoader.exe",
|
||||||
|
the installation is complete. In the following log lines your should see the start process of the plugin loader.
|
||||||
|
|
||||||
|
How to install plugins is described on gameservershub.com, from which you obtained the plugin loader. Please refer to their guide instead.
|
||||||
|
|
||||||
|
## Map Names
|
||||||
|
|
||||||
|
This is a list of all official map names with their map id. The map id is used as start parameter in the `docker-compose.yml` file. ([click](#6-changing-the-start-parameters-and-the-player-limit))
|
||||||
|
|
||||||
|
| Map Name | Map ID (for the start parameter) |
|
||||||
|
| ------------- | ------------- |
|
||||||
|
| The Island | TheIsland_WP |
|
||||||
|
| Scorched Earth | ScorchedEarth_WP |
|
||||||
|
| The Center | TheCenter_WP |
|
||||||
|
| Aberration | Aberration_WP |
|
||||||
|
| Extinction | Extinction_WP |
|
||||||
|
| Ragnarok | Ragnarok_WP |
|
||||||
|
| Astraeos | Astraeos_WP |
|
||||||
|
| Valguero | Valguero_WP |
|
||||||
|
| Lost Colony | LostColony_WP |
|
||||||
|
|
||||||
|
**NOTE:** Mod Maps have their own id! ([click](#adding-mod-maps))
|
||||||
|
|
||||||
|
## Updating the Container Image
|
||||||
|
|
||||||
|
The container image will be updated from time to time. In general, we try to not break previous installations by an update, but to add certain features, it might be necessary to introduce backward incompatibilities.
|
||||||
|
The default `docker-compose.yml` file suggests to use the `latest` branch of the container image. If you want to stay on one specific version, you can force the container image to launch with that said version, by
|
||||||
|
changing `image: "mschnitzer/asa-linux-server:latest"` in your `docker-compose.yml` file (as outlined below) to whatever version suits you. A list of all versions can be
|
||||||
|
found [here](https://hub.docker.com/r/mschnitzer/asa-linux-server/tags).
|
||||||
|
|
||||||
|
For example:
|
||||||
|
|
||||||
|
If you want to stay on version `1.4.0` for your ASA server, you must change `image: "mschnitzer/asa-linux-server:latest"` to `image: "mschnitzer/asa-linux-server:1.4.0"`.
|
||||||
|
|
||||||
|
Even if you stay on branch `latest`, your container image won't be updated automatically if we roll out an update. You explicitly need to run `docker pull mschnitzer/asa-linux-server:latest` to obtain the newest version.
|
||||||
|
|
||||||
|
We strongly suggest to read through the [releases page](https://github.com/mschnitzer/ark-survival-ascended-linux-container-image/releases) of this repository to see what has changed between versions. If there's
|
||||||
|
a backward incompatibility being introduced, it will be mentioned there with an explanation what to change.
|
||||||
|
|
||||||
|
## Common Issues
|
||||||
|
|
||||||
|
### Server is not visible in server browser
|
||||||
|
|
||||||
|
If you cannot discover your server in the server browser, it's most likely due to at least one of the following reasons:
|
||||||
|
|
||||||
|
* Your server is still booting up, give it ~5 minutes
|
||||||
|
* You are not looking at the "Unofficial" server browser list
|
||||||
|
* Your filter settings in the server browser exclude your server
|
||||||
|
* You forgot clicking the "Show player server settings". ([view screenshot](https://raw.githubusercontent.com/mschnitzer/ark-survival-ascended-linux-container-image/main/assets/show-player-servers.jpg)) By default, only Nitrado servers are shown to players when searching for unofficial servers, unfortunately.
|
||||||
|
|
||||||
|
## Addressing "Connection Timeout" issues
|
||||||
|
|
||||||
|
First of all, try to connect through the ingame console to your server. In many cases this works, but only connecting through the server browser causes an issue. Try to run the command `open $IP:$PORT` and test whether you
|
||||||
|
can connect to it.
|
||||||
|
|
||||||
|
If that is NOT working and you are having a home setup and not a VPS cloud setup, make sure your ports are REALLY open. This needs to be configured on your router. The ports that need to be opened are listed above in this README.
|
||||||
|
Please refer to the documentation of your router how to configure port forwarding properly.
|
||||||
|
|
||||||
|
If you can connect to your server through the console command, but not via the sever browser, it is very likely that you are running into one of these issues:
|
||||||
|
|
||||||
|
### Your server has multiple IPv4 addresses
|
||||||
|
|
||||||
|
If your server has multiple IPv4 addresses and you bound your ASA server to one of the secondary ones, by default, docker routes your traffic always through your primary network interface, which would cause the server browser to list your
|
||||||
|
server under the wrong IP address.
|
||||||
|
|
||||||
|
For example:
|
||||||
|
|
||||||
|
Your primary IP is: `255.255.300.300`
|
||||||
|
Your secondary one is: `255.255.400.400`
|
||||||
|
|
||||||
|
You adjusted the `docker-compose.yml` file in a way where it binds the ports on interface `255.255.400.400`. However, if your ASA server communicates with the internet and announces itself to the ASA server list, the ASA master server that manages the
|
||||||
|
server browser entries, would see the requests coming from `255.255.300.300` as this is your primary network interface.
|
||||||
|
|
||||||
|
This issue can be solved by forcing the traffic to be routed manually through your secondary network interface.
|
||||||
|
|
||||||
|
But before we start fixing it, you should make sure that this is really the issue.
|
||||||
|
|
||||||
|
#### Debugging with curl
|
||||||
|
|
||||||
|
1. Log in to the container `docker exec -ti -u root asa-server-1 bash`
|
||||||
|
2. Run `zypper --no-gpg-checks ref`
|
||||||
|
3. Install curl `zypper in -y curl`
|
||||||
|
4. Run `curl icanhazip.com` (`icanhazip.com` is a service that tells you from what ip address it received traffic from)
|
||||||
|
|
||||||
|
If the service responds with an IP that you have not assigned to the ASA server in the `docker-compose.yml` file, then it's very likely that this is the reason why you are getting a "Connection Timeout" error.
|
||||||
|
Please continue following the instructions below.
|
||||||
|
|
||||||
|
#### How to customize your routing?
|
||||||
|
|
||||||
|
You need to adjust the `docker-compose.yml` file and add `com.docker.network.bridge.enable_ip_masquerade: 'false'` to the `networks` section, so that it looks like this:
|
||||||
|
|
||||||
|
```yml
|
||||||
|
networks:
|
||||||
|
asa-network:
|
||||||
|
attachable: true
|
||||||
|
driver: bridge
|
||||||
|
driver_opts:
|
||||||
|
com.docker.network.bridge.name: 'asanet'
|
||||||
|
com.docker.network.bridge.enable_ip_masquerade: 'false'
|
||||||
|
```
|
||||||
|
|
||||||
|
Now stop the ASA server if it's running:
|
||||||
|
|
||||||
|
```
|
||||||
|
docker stop asa-server-1
|
||||||
|
```
|
||||||
|
|
||||||
|
Delete the docker network interface and the container, so that they can be recreated:
|
||||||
|
|
||||||
|
```
|
||||||
|
docker rm asa-server-1
|
||||||
|
docker network rm asa-server_asa-network
|
||||||
|
```
|
||||||
|
|
||||||
|
Now run `docker compose up -d` from within the directory where your `docker-compose.yml` is located at.
|
||||||
|
|
||||||
|
Once done and the container is up again, inspect the network to find its subnet:
|
||||||
|
|
||||||
|
```
|
||||||
|
docker network inspect asa-server_asa-network | grep Subnet
|
||||||
|
```
|
||||||
|
|
||||||
|
Now customize the routing of the container through `iptables`, where `$SUBNET` needs to be replaced with the subnet from the previous command (including the `/24` or `/16` - whatever it is in your case):
|
||||||
|
|
||||||
|
```
|
||||||
|
iptables -t nat -A POSTROUTING -s $SUBNET ! -o asanet -j SNAT --to-source $YOUR_SECONDARY_IP_USED_BY_ASA
|
||||||
|
```
|
||||||
|
|
||||||
|
Once done, connect to your container and test that the remote IP is the right one, by following the steps with `curl` again.
|
||||||
|
|
||||||
|
Now try to connect to your server through the server browser. If that is not solving your problem or if the IP is still the wrong one, open a GitHub issue. If it solves your problem, continue with the
|
||||||
|
next section to make the `iptables` adjustments persistent after reboot.
|
||||||
|
|
||||||
|
#### Making your iptable rules persistent
|
||||||
|
|
||||||
|
Changes to the `iptables` will get reverted after reboot. You can make them persistent by saving the current state:
|
||||||
|
|
||||||
|
```
|
||||||
|
iptables-save > /root/iptables
|
||||||
|
```
|
||||||
|
|
||||||
|
Now run `crontab -e` and add the following entry:
|
||||||
|
|
||||||
|
```
|
||||||
|
@reboot /bin/bash -c 'sleep 15 ; /usr/sbin/iptables-restore < /root/iptables'
|
||||||
|
```
|
||||||
|
|
||||||
|
Save the cronjob and test it by rebooting your system. You can test whether it has worked by following the `curl` steps from above again.
|
||||||
|
|
||||||
|
## Found an Issue or Bug?
|
||||||
|
|
||||||
|
Create a ticket on GitHub, I will do my best to fix it. Feel free to open a pull request as well.
|
||||||
|
|
||||||
|
## Credits
|
||||||
|
|
||||||
|
* Glorius Eggroll - For his version of Proton to run the ARK Windows binaries on Linux ([click](https://github.com/GloriousEggroll/proton-ge-custom))
|
||||||
|
* cdp1337 - For his Linux guide of installing Proton and running ARK on Linux ([click](https://github.com/cdp1337/ARKSurvivalAscended-Linux))
|
||||||
|
* tesfabpel - For his Valve RCON implementation in Ruby ([click](https://github.com/tesfabpel/srcon-rb))
|
||||||
@@ -0,0 +1,91 @@
|
|||||||
|
<?xml version="1.0" encoding="utf-8"?>
|
||||||
|
<image schemaversion="7.4" name="asa-linux-server">
|
||||||
|
<description type="system">
|
||||||
|
<author>Manuel Schnitzer</author>
|
||||||
|
<contact>github@mschnitzer.de</contact>
|
||||||
|
<specification>ARK: Survival Ascended - Dedicated Server (Linux)</specification>
|
||||||
|
</description>
|
||||||
|
<profiles>
|
||||||
|
<profile name="stable" description="building docker tag: latest"/>
|
||||||
|
<profile name="beta" description="building docker tag: beta"/>
|
||||||
|
<profile name="development" description="building docker tag: development"/>
|
||||||
|
</profiles>
|
||||||
|
<preferences>
|
||||||
|
<version>1.5.1</version>
|
||||||
|
<packagemanager>zypper</packagemanager>
|
||||||
|
<rpm-excludedocs>true</rpm-excludedocs>
|
||||||
|
<rpm-check-signatures>false</rpm-check-signatures>
|
||||||
|
<locale>en_US</locale>
|
||||||
|
<keytable>us</keytable>
|
||||||
|
</preferences>
|
||||||
|
<preferences profiles="stable">
|
||||||
|
<type image="docker">
|
||||||
|
<containerconfig tag="latest" workingdir="/home/gameserver" name="mschnitzer/asa-linux-server"/>
|
||||||
|
</type>
|
||||||
|
</preferences>
|
||||||
|
<preferences profiles="beta">
|
||||||
|
<type image="docker">
|
||||||
|
<containerconfig tag="beta" workingdir="/home/gameserver" name="mschnitzer/asa-linux-server"/>
|
||||||
|
</type>
|
||||||
|
</preferences>
|
||||||
|
<preferences profiles="development">
|
||||||
|
<type image="docker">
|
||||||
|
<containerconfig tag="development" workingdir="/home/gameserver" name="mschnitzer/asa-linux-server"/>
|
||||||
|
</type>
|
||||||
|
</preferences>
|
||||||
|
<users>
|
||||||
|
<user name="gameserver" password="" home="/home/gameserver" groups="gameserver" id="25000"/>
|
||||||
|
</users>
|
||||||
|
<repository type="rpm-md" alias="repo-oss" imageinclude="true">
|
||||||
|
<source path="http://cdn.opensuse.org/distribution/leap/16.0/repo/oss/x86_64"/>
|
||||||
|
</repository>
|
||||||
|
<packages type="image">
|
||||||
|
<package name="patterns-base-base"/>
|
||||||
|
<package name="glibc-locale-base"/>
|
||||||
|
<package name="timezone"/>
|
||||||
|
<!-- application dependencies: ASA -->
|
||||||
|
<package name="libgcc_s1-32bit"/>
|
||||||
|
<package name="python313"/>
|
||||||
|
<package name="wget"/>
|
||||||
|
<package name="tar"/>
|
||||||
|
<package name="libfreetype6"/>
|
||||||
|
<!-- application dependencies: ASA server api plugin loader -->
|
||||||
|
<package name="unzip"/>
|
||||||
|
<!-- application dependencies: asa-ctrl -->
|
||||||
|
<package name="ruby3.4"/>
|
||||||
|
<package name="ruby3.4-devel"/>
|
||||||
|
<package name="make"/>
|
||||||
|
<package name="gcc-c++"/>
|
||||||
|
</packages>
|
||||||
|
<packages type="image" profiles="development">
|
||||||
|
<package name="vim"/>
|
||||||
|
<package name="vim-data"/>
|
||||||
|
<package name="vim-data-common"/>
|
||||||
|
</packages>
|
||||||
|
<packages type="bootstrap">
|
||||||
|
<package name="udev"/>
|
||||||
|
<package name="filesystem"/>
|
||||||
|
<package name="glibc-locale"/>
|
||||||
|
<package name="cracklib-dict-small"/>
|
||||||
|
<package name="ca-certificates"/>
|
||||||
|
<package name="ca-certificates-mozilla"/>
|
||||||
|
</packages>
|
||||||
|
<packages type="delete">
|
||||||
|
<package name="dbus-1"/>
|
||||||
|
<package name="kbd"/>
|
||||||
|
<package name="kmod"/>
|
||||||
|
<package name="ncurses-utils"/>
|
||||||
|
<package name="pinentry"/>
|
||||||
|
<package name="udev"/>
|
||||||
|
<package name="glibc-locale"/>
|
||||||
|
<package name="systemd"/>
|
||||||
|
</packages>
|
||||||
|
<packages type="delete" profiles="stable,beta">
|
||||||
|
<package name="ruby3.4-devel"/>
|
||||||
|
<package name="make"/>
|
||||||
|
<package name="gcc-c++"/>
|
||||||
|
<package name="gcc15"/>
|
||||||
|
<package name="gcc15-c++"/>
|
||||||
|
<package name="libstdc++6-devel-gcc15"/>
|
||||||
|
</packages>
|
||||||
|
</image>
|
||||||
BIN
Binary file not shown.
|
After Width: | Height: | Size: 227 KiB |
@@ -0,0 +1,29 @@
|
|||||||
|
#!/bin/bash
|
||||||
|
rm -f /etc/localtime
|
||||||
|
zypper --gpg-auto-import-keys ref
|
||||||
|
|
||||||
|
groupmod -g 25000 gameserver
|
||||||
|
usermod -u 25000 gameserver
|
||||||
|
|
||||||
|
chmod 0755 /usr/bin/start_server
|
||||||
|
chmod 0755 /usr/bin/cli-asa-mods
|
||||||
|
|
||||||
|
# install ruby gems
|
||||||
|
cd /usr/share/asa-ctrl
|
||||||
|
bundle.ruby3.4
|
||||||
|
|
||||||
|
if [ "$kiwi_profiles" = "development" ]; then
|
||||||
|
# will be mounted to ease development
|
||||||
|
rm -r /usr/share/asa-ctrl
|
||||||
|
gem.ruby3.4 install byebug
|
||||||
|
else
|
||||||
|
chmod 0755 /usr/share/asa-ctrl/main.rb
|
||||||
|
fi
|
||||||
|
|
||||||
|
ln -s /usr/share/asa-ctrl/main.rb /usr/bin/asa-ctrl
|
||||||
|
|
||||||
|
# This fixes a warning with Proton. The warning confused some people, but it haven't had any effect on the ASA server.
|
||||||
|
echo "d5b7b5ed-1674-497d-ad98-7437a6543312" > /etc/machine-id
|
||||||
|
chmod 644 /etc/machine-id
|
||||||
|
|
||||||
|
exit 0
|
||||||
@@ -0,0 +1,89 @@
|
|||||||
|
###
|
||||||
|
### THIS IS THE DEVELOPMENT COMPOSE FILE!
|
||||||
|
### You are most likely looking for the 'docker-compose.yml' file instead!
|
||||||
|
###
|
||||||
|
version: "3.3"
|
||||||
|
services:
|
||||||
|
asa-server-1:
|
||||||
|
container_name: asa-server-1
|
||||||
|
hostname: asa-server-1
|
||||||
|
entrypoint: "/usr/bin/start_server"
|
||||||
|
user: gameserver
|
||||||
|
image: "mschnitzer/asa-linux-server:development"
|
||||||
|
tty: true
|
||||||
|
environment:
|
||||||
|
- ASA_START_PARAMS=TheIsland_WP?listen?Port=7777?RCONPort=27020?RCONEnabled=True -WinLiveMaxPlayers=50 -clusterid=default -ClusterDirOverride="/home/gameserver/cluster-shared"
|
||||||
|
- ENABLE_DEBUG=0
|
||||||
|
ports:
|
||||||
|
# Game port for player connections through the server browser
|
||||||
|
- 0.0.0.0:7777:7777/udp
|
||||||
|
# RCON port for remote server administration
|
||||||
|
- 0.0.0.0:27020:27020/tcp
|
||||||
|
depends_on:
|
||||||
|
- set-permissions-1
|
||||||
|
volumes:
|
||||||
|
- steam-1:/home/gameserver/Steam:rw
|
||||||
|
- steamcmd-1:/home/gameserver/steamcmd:rw
|
||||||
|
- server-files-1:/home/gameserver/server-files:rw
|
||||||
|
- cluster-shared:/home/gameserver/cluster-shared:rw
|
||||||
|
- /etc/localtime:/etc/localtime:ro
|
||||||
|
- ./root/usr/share/asa-ctrl:/usr/share/asa-ctrl:rw
|
||||||
|
networks:
|
||||||
|
asa-network:
|
||||||
|
set-permissions-1:
|
||||||
|
entrypoint: "/bin/bash -c 'chown -R 25000:25000 /steam ; chown -R 25000:25000 /steamcmd ; chown -R 25000:25000 /server-files ; chown -R 25000:25000 /cluster-shared'"
|
||||||
|
user: root
|
||||||
|
image: "opensuse/leap"
|
||||||
|
volumes:
|
||||||
|
- steam-1:/steam:rw
|
||||||
|
- steamcmd-1:/steamcmd:rw
|
||||||
|
- server-files-1:/server-files:rw
|
||||||
|
- cluster-shared:/cluster-shared:rw
|
||||||
|
# asa-server-2:
|
||||||
|
# container_name: asa-server-2
|
||||||
|
# hostname: asa-server-2
|
||||||
|
# entrypoint: "/usr/bin/start_server"
|
||||||
|
# user: gameserver
|
||||||
|
# image: "mschnitzer/asa-linux-server:development"
|
||||||
|
# tty: true
|
||||||
|
# environment:
|
||||||
|
# - ASA_START_PARAMS=ScorchedEarth_WP?listen?Port=7778?RCONPort=27021?RCONEnabled=True -WinLiveMaxPlayers=50 -clusterid=default -ClusterDirOverride="/home/gameserver/cluster-shared"
|
||||||
|
# ports:
|
||||||
|
# # Game port for player connections through the server browser
|
||||||
|
# - 0.0.0.0:7778:7778/udp
|
||||||
|
# # RCON port for remote server administration
|
||||||
|
# - 0.0.0.0:27021:27021/tcp
|
||||||
|
# depends_on:
|
||||||
|
# - set-permissions-2
|
||||||
|
# volumes:
|
||||||
|
# - steam-2:/home/gameserver/Steam:rw
|
||||||
|
# - steamcmd-2:/home/gameserver/steamcmd:rw
|
||||||
|
# - server-files-2:/home/gameserver/server-files:rw
|
||||||
|
# - cluster-shared:/home/gameserver/cluster-shared:rw
|
||||||
|
# - /etc/localtime:/etc/localtime:ro
|
||||||
|
# - ./root/usr/share/asa-ctrl:/usr/share/asa-ctrl:rw
|
||||||
|
# networks:
|
||||||
|
# asa-network:
|
||||||
|
# set-permissions-2:
|
||||||
|
# entrypoint: "/bin/bash -c 'chown -R 25000:25000 /steam ; chown -R 25000:25000 /steamcmd ; chown -R 25000:25000 /server-files ; chown -R 25000:25000 /cluster-shared'"
|
||||||
|
# user: root
|
||||||
|
# image: "opensuse/leap"
|
||||||
|
# volumes:
|
||||||
|
# - steam-2:/steam:rw
|
||||||
|
# - steamcmd-2:/steamcmd:rw
|
||||||
|
# - server-files-2:/server-files:rw
|
||||||
|
# - cluster-shared:/cluster-shared:rw
|
||||||
|
volumes:
|
||||||
|
cluster-shared:
|
||||||
|
steam-1:
|
||||||
|
steamcmd-1:
|
||||||
|
server-files-1:
|
||||||
|
# steam-2:
|
||||||
|
# steamcmd-2:
|
||||||
|
# server-files-2:
|
||||||
|
networks:
|
||||||
|
asa-network:
|
||||||
|
attachable: true
|
||||||
|
driver: bridge
|
||||||
|
driver_opts:
|
||||||
|
com.docker.network.bridge.name: 'asanet'
|
||||||
@@ -0,0 +1,83 @@
|
|||||||
|
version: "3.3"
|
||||||
|
services:
|
||||||
|
asa-server-1:
|
||||||
|
container_name: asa-server-1
|
||||||
|
hostname: asa-server-1
|
||||||
|
entrypoint: "/usr/bin/start_server"
|
||||||
|
user: gameserver
|
||||||
|
image: "mschnitzer/asa-linux-server:latest"
|
||||||
|
tty: true
|
||||||
|
environment:
|
||||||
|
- ASA_START_PARAMS=TheIsland_WP?listen?Port=7777?RCONPort=27020?RCONEnabled=True -WinLiveMaxPlayers=50 -clusterid=default -ClusterDirOverride="/home/gameserver/cluster-shared"
|
||||||
|
- ENABLE_DEBUG=0
|
||||||
|
ports:
|
||||||
|
# Game port for player connections through the server browser
|
||||||
|
- 0.0.0.0:7777:7777/udp
|
||||||
|
# RCON port for remote server administration
|
||||||
|
- 0.0.0.0:27020:27020/tcp
|
||||||
|
depends_on:
|
||||||
|
- set-permissions-1
|
||||||
|
volumes:
|
||||||
|
- steam-1:/home/gameserver/Steam:rw
|
||||||
|
- steamcmd-1:/home/gameserver/steamcmd:rw
|
||||||
|
- server-files-1:/home/gameserver/server-files:rw
|
||||||
|
- cluster-shared:/home/gameserver/cluster-shared:rw
|
||||||
|
- /etc/localtime:/etc/localtime:ro
|
||||||
|
networks:
|
||||||
|
asa-network:
|
||||||
|
set-permissions-1:
|
||||||
|
entrypoint: "/bin/bash -c 'chown -R 25000:25000 /steam ; chown -R 25000:25000 /steamcmd ; chown -R 25000:25000 /server-files ; chown -R 25000:25000 /cluster-shared'"
|
||||||
|
user: root
|
||||||
|
image: "opensuse/leap"
|
||||||
|
volumes:
|
||||||
|
- steam-1:/steam:rw
|
||||||
|
- steamcmd-1:/steamcmd:rw
|
||||||
|
- server-files-1:/server-files:rw
|
||||||
|
- cluster-shared:/cluster-shared:rw
|
||||||
|
# asa-server-2:
|
||||||
|
# container_name: asa-server-2
|
||||||
|
# hostname: asa-server-2
|
||||||
|
# entrypoint: "/usr/bin/start_server"
|
||||||
|
# user: gameserver
|
||||||
|
# image: "mschnitzer/asa-linux-server:latest"
|
||||||
|
# tty: true
|
||||||
|
# environment:
|
||||||
|
# - ASA_START_PARAMS=ScorchedEarth_WP?listen?Port=7778?RCONPort=27021?RCONEnabled=True -WinLiveMaxPlayers=50 -clusterid=default -ClusterDirOverride="/home/gameserver/cluster-shared"
|
||||||
|
# ports:
|
||||||
|
# # Game port for player connections through the server browser
|
||||||
|
# - 0.0.0.0:7778:7778/udp
|
||||||
|
# # RCON port for remote server administration
|
||||||
|
# - 0.0.0.0:27021:27021/tcp
|
||||||
|
# depends_on:
|
||||||
|
# - set-permissions-2
|
||||||
|
# volumes:
|
||||||
|
# - steam-2:/home/gameserver/Steam:rw
|
||||||
|
# - steamcmd-2:/home/gameserver/steamcmd:rw
|
||||||
|
# - server-files-2:/home/gameserver/server-files:rw
|
||||||
|
# - cluster-shared:/home/gameserver/cluster-shared:rw
|
||||||
|
# - /etc/localtime:/etc/localtime:ro
|
||||||
|
# networks:
|
||||||
|
# asa-network:
|
||||||
|
# set-permissions-2:
|
||||||
|
# entrypoint: "/bin/bash -c 'chown -R 25000:25000 /steam ; chown -R 25000:25000 /steamcmd ; chown -R 25000:25000 /server-files ; chown -R 25000:25000 /cluster-shared'"
|
||||||
|
# user: root
|
||||||
|
# image: "opensuse/leap"
|
||||||
|
# volumes:
|
||||||
|
# - steam-2:/steam:rw
|
||||||
|
# - steamcmd-2:/steamcmd:rw
|
||||||
|
# - server-files-2:/server-files:rw
|
||||||
|
# - cluster-shared:/cluster-shared:rw
|
||||||
|
volumes:
|
||||||
|
cluster-shared:
|
||||||
|
steam-1:
|
||||||
|
steamcmd-1:
|
||||||
|
server-files-1:
|
||||||
|
# steam-2:
|
||||||
|
# steamcmd-2:
|
||||||
|
# server-files-2:
|
||||||
|
networks:
|
||||||
|
asa-network:
|
||||||
|
attachable: true
|
||||||
|
driver: bridge
|
||||||
|
driver_opts:
|
||||||
|
com.docker.network.bridge.name: 'asanet'
|
||||||
@@ -0,0 +1,34 @@
|
|||||||
|
#!/usr/bin/ruby.ruby3.4
|
||||||
|
require 'json'
|
||||||
|
|
||||||
|
db_path = '/home/gameserver/server-files/mods.json'
|
||||||
|
|
||||||
|
unless File.exist?(db_path)
|
||||||
|
print ""
|
||||||
|
exit! 0
|
||||||
|
end
|
||||||
|
|
||||||
|
begin
|
||||||
|
mods = JSON.parse(File.read(db_path))
|
||||||
|
args = "-mods="
|
||||||
|
counter = 0
|
||||||
|
|
||||||
|
mods.each do |mod|
|
||||||
|
if mod['enabled']
|
||||||
|
args += ',' if counter > 0
|
||||||
|
args += mod['mod_id'].to_s
|
||||||
|
|
||||||
|
counter += 1
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
|
if counter > 0
|
||||||
|
print args
|
||||||
|
end
|
||||||
|
rescue JSON::ParserError
|
||||||
|
File.write('/tmp/mod-read-error', 'mods.json is corrupted')
|
||||||
|
print ""
|
||||||
|
rescue => err
|
||||||
|
File.write('/tmp/mod-read-error', err.to_s)
|
||||||
|
print ""
|
||||||
|
end
|
||||||
@@ -0,0 +1,91 @@
|
|||||||
|
#!/bin/bash
|
||||||
|
if [ "$ENABLE_DEBUG" = "1" ]; then
|
||||||
|
echo "Entering debug mode..."
|
||||||
|
sleep 999999999999
|
||||||
|
exit 0
|
||||||
|
fi
|
||||||
|
|
||||||
|
# download steamcmd if necessary
|
||||||
|
if [ ! -d "/home/gameserver/steamcmd/linux32" ]; then
|
||||||
|
cd /home/gameserver/steamcmd
|
||||||
|
wget https://steamcdn-a.akamaihd.net/client/installer/steamcmd_linux.tar.gz
|
||||||
|
tar xfvz steamcmd_linux.tar.gz
|
||||||
|
fi
|
||||||
|
|
||||||
|
# download/update server files
|
||||||
|
cd /home/gameserver/steamcmd
|
||||||
|
./steamcmd.sh +force_install_dir /home/gameserver/server-files +login anonymous +app_update 2430930 validate +quit
|
||||||
|
|
||||||
|
PROTON_VERSION="10-17"
|
||||||
|
PROTON_DIR_NAME="GE-Proton$PROTON_VERSION"
|
||||||
|
PROTON_ARCHIVE_NAME="$PROTON_DIR_NAME.tar.gz"
|
||||||
|
STEAM_COMPAT_DATA=/home/gameserver/server-files/steamapps/compatdata
|
||||||
|
STEAM_COMPAT_DIR=/home/gameserver/Steam/compatibilitytools.d
|
||||||
|
ASA_COMPAT_DATA=$STEAM_COMPAT_DATA/2430930
|
||||||
|
ASA_BINARY_DIR="/home/gameserver/server-files/ShooterGame/Binaries/Win64"
|
||||||
|
START_PARAMS_FILE="/home/gameserver/server-files/start-parameters"
|
||||||
|
MODS="$(/usr/bin/cli-asa-mods)"
|
||||||
|
ASA_START_PARAMS="$ASA_START_PARAMS $MODS"
|
||||||
|
ASA_BINARY_NAME="ArkAscendedServer.exe"
|
||||||
|
ASA_PLUGIN_BINARY_NAME="AsaApiLoader.exe"
|
||||||
|
ASA_PLUGIN_LOADER_ARCHIVE_NAME=$(basename $ASA_BINARY_DIR/AsaApi_*.zip)
|
||||||
|
ASA_PLUGIN_LOADER_ARCHIVE_PATH="$ASA_BINARY_DIR/$ASA_PLUGIN_LOADER_ARCHIVE_NAME"
|
||||||
|
ASA_PLUGIN_BINARY_PATH="$ASA_BINARY_DIR/$ASA_PLUGIN_BINARY_NAME"
|
||||||
|
LAUNCH_BINARY_NAME="$ASA_BINARY_NAME"
|
||||||
|
|
||||||
|
# install proton if necessary
|
||||||
|
if [ ! -d "$STEAM_COMPAT_DIR/$PROTON_DIR_NAME" ]; then
|
||||||
|
mkdir -p $STEAM_COMPAT_DIR
|
||||||
|
echo "Downloading Proton version $PROTON_VERSION... This might take a while"
|
||||||
|
wget -P /tmp https://github.com/GloriousEggroll/proton-ge-custom/releases/download/GE-Proton$PROTON_VERSION/GE-Proton$PROTON_VERSION.tar.gz
|
||||||
|
EXIT_CODE=$?
|
||||||
|
|
||||||
|
if [ $EXIT_CODE -ne 0 ]; then
|
||||||
|
echo "Error: Error while downloading Proton ($EXIT_CODE)"
|
||||||
|
exit 200
|
||||||
|
fi
|
||||||
|
|
||||||
|
echo "Download finished, comparing checksums..."
|
||||||
|
sha512sum -c /usr/share/proton/GE-Proton$PROTON_VERSION.sha512sum
|
||||||
|
|
||||||
|
if [ $? -ne 0 ]; then
|
||||||
|
echo "Error: Proton checksum mismatch!"
|
||||||
|
exit 201
|
||||||
|
fi
|
||||||
|
|
||||||
|
tar -xf /tmp/$PROTON_ARCHIVE_NAME -C $STEAM_COMPAT_DIR
|
||||||
|
rm /tmp/$PROTON_ARCHIVE_NAME
|
||||||
|
fi
|
||||||
|
|
||||||
|
# install proton compat game data
|
||||||
|
if [ ! -d "$ASA_COMPAT_DATA" ]; then
|
||||||
|
mkdir -p $STEAM_COMPAT_DATA
|
||||||
|
cp -r $STEAM_COMPAT_DIR/$PROTON_DIR_NAME/files/share/default_pfx $ASA_COMPAT_DATA
|
||||||
|
fi
|
||||||
|
|
||||||
|
echo "Starting the ARK: Survival Ascended dedicated server..."
|
||||||
|
echo "Start parameters: $ASA_START_PARAMS"
|
||||||
|
|
||||||
|
export XDG_RUNTIME_DIR=/run/user/$(id -u)
|
||||||
|
export STEAM_COMPAT_CLIENT_INSTALL_PATH=/home/gameserver/Steam
|
||||||
|
export STEAM_COMPAT_DATA_PATH=$ASA_COMPAT_DATA
|
||||||
|
|
||||||
|
cd "$ASA_BINARY_DIR"
|
||||||
|
|
||||||
|
# unzip the asa plugin api archive if it exists. delete it afterwards
|
||||||
|
if [ -f "$ASA_PLUGIN_LOADER_ARCHIVE_PATH" ]; then
|
||||||
|
unzip -o $ASA_PLUGIN_LOADER_ARCHIVE_NAME
|
||||||
|
rm $ASA_PLUGIN_LOADER_ARCHIVE_NAME
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [ -f "$ASA_PLUGIN_BINARY_PATH" ]; then
|
||||||
|
echo "Detected ASA Server API loader. Launching server through $ASA_PLUGIN_BINARY_NAME"
|
||||||
|
LAUNCH_BINARY_NAME="$ASA_PLUGIN_BINARY_NAME"
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Remove steamclient64.dll to prevent server from crashing.
|
||||||
|
# File is not needed and was probably accidentally committed to Steam.
|
||||||
|
# See: https://github.com/mschnitzer/ark-survival-ascended-linux-container-image/issues/123
|
||||||
|
rm -f /home/gameserver/server-files/ShooterGame/Binaries/Win64/steamclient64.dll
|
||||||
|
|
||||||
|
$STEAM_COMPAT_DIR/$PROTON_DIR_NAME/proton run $LAUNCH_BINARY_NAME $ASA_START_PARAMS
|
||||||
+4
@@ -0,0 +1,4 @@
|
|||||||
|
source 'https://rubygems.org'
|
||||||
|
|
||||||
|
gem 'slop', '= 4.10.1'
|
||||||
|
gem 'iniparse', '= 1.5.0'
|
||||||
+15
@@ -0,0 +1,15 @@
|
|||||||
|
GEM
|
||||||
|
remote: https://rubygems.org/
|
||||||
|
specs:
|
||||||
|
iniparse (1.5.0)
|
||||||
|
slop (4.10.1)
|
||||||
|
|
||||||
|
PLATFORMS
|
||||||
|
x86_64-linux-gnu
|
||||||
|
|
||||||
|
DEPENDENCIES
|
||||||
|
iniparse (= 1.5.0)
|
||||||
|
slop (= 4.10.1)
|
||||||
|
|
||||||
|
BUNDLED WITH
|
||||||
|
2.5.0.dev
|
||||||
+15
@@ -0,0 +1,15 @@
|
|||||||
|
module AsaCtrl
|
||||||
|
module Cli
|
||||||
|
class CliInterface
|
||||||
|
def initialize(opts)
|
||||||
|
@opts = opts
|
||||||
|
|
||||||
|
print_help! if opts[:help]
|
||||||
|
end
|
||||||
|
|
||||||
|
def print_help!
|
||||||
|
raise "Help not implemented!"
|
||||||
|
end
|
||||||
|
end
|
||||||
|
end
|
||||||
|
end
|
||||||
+34
@@ -0,0 +1,34 @@
|
|||||||
|
module AsaCtrl
|
||||||
|
module Cli
|
||||||
|
class ModsInterface < CliInterface
|
||||||
|
def initialize(opts)
|
||||||
|
super(opts)
|
||||||
|
|
||||||
|
execute!
|
||||||
|
end
|
||||||
|
|
||||||
|
def execute!
|
||||||
|
if @opts[:enable]
|
||||||
|
enable_mod!
|
||||||
|
end
|
||||||
|
|
||||||
|
exit! AsaCtrl::ExitCodes::OK
|
||||||
|
end
|
||||||
|
|
||||||
|
def enable_mod!
|
||||||
|
mod_id = @opts[:enable]
|
||||||
|
AsaCtrl::Mods::Database.get_instance.enable_mod!(mod_id)
|
||||||
|
|
||||||
|
puts "Enabled mod id '#{mod_id}' successfully. The server will download the mod upon startup."
|
||||||
|
rescue AsaCtrl::Errors::ModAlreadyEnabledError
|
||||||
|
AsaCtrl::Cli.exit_with_error!("This mod is already enabled! Use 'asa-ctrl mods --list' to see what mods are currently enabled.",
|
||||||
|
AsaCtrl::ExitCodes::MOD_ALREADY_ENABLED)
|
||||||
|
end
|
||||||
|
|
||||||
|
def print_help!
|
||||||
|
puts "Usage: asa-ctrl mods [--install] (--dry-run)"
|
||||||
|
exit! AsaCtrl::ExitCodes::OK
|
||||||
|
end
|
||||||
|
end
|
||||||
|
end
|
||||||
|
end
|
||||||
+42
@@ -0,0 +1,42 @@
|
|||||||
|
module AsaCtrl
|
||||||
|
module Cli
|
||||||
|
class RconInterface < CliInterface
|
||||||
|
def initialize(opts)
|
||||||
|
super(opts)
|
||||||
|
|
||||||
|
execute!
|
||||||
|
end
|
||||||
|
|
||||||
|
def execute!
|
||||||
|
if @opts[:exec]
|
||||||
|
run_command!
|
||||||
|
end
|
||||||
|
|
||||||
|
exit! AsaCtrl::ExitCodes::OK
|
||||||
|
end
|
||||||
|
|
||||||
|
def run_command!
|
||||||
|
rcon_command = @opts[:exec]
|
||||||
|
response = AsaCtrl::Rcon.exec_command!('127.0.0.1', AsaCtrl::Rcon.identify_port, rcon_command, AsaCtrl::Rcon.identify_password)
|
||||||
|
|
||||||
|
if response[:id] == AsaCtrl::Rcon::PacketTypes::RESPONSE_VALUE
|
||||||
|
puts response[:body]
|
||||||
|
else
|
||||||
|
AsaCtrl::Cli.exit_with_error!("Rcon command execution failed: #{response}",
|
||||||
|
AsaCtrl::ExitCodes::RCON_COMMAND_EXECUTION_FAILED)
|
||||||
|
end
|
||||||
|
rescue AsaCtrl::Errors::RconPasswordNotFoundError
|
||||||
|
AsaCtrl::Cli.exit_with_error!("Could not read RCON password. Make sure it is properly configured, either as start parameter ?ServerAdminPassword=mypass or " \
|
||||||
|
"in GameUserSettings.ini in the [ServerSettings] section as ServerAdminPassword=mypass", AsaCtrl::ExitCodes::RCON_PASSWORD_NOT_FOUND)
|
||||||
|
rescue AsaCtrl::Errors::RconAuthenticationError
|
||||||
|
AsaCtrl::Cli.exit_with_error!("Could not execute this RCON command. Authentication failed (wrong server password).",
|
||||||
|
AsaCtrl::ExitCodes::RCON_PASSWORD_WRONG)
|
||||||
|
end
|
||||||
|
|
||||||
|
def print_help!
|
||||||
|
puts "Usage: asa-ctrl rcon [--exec]"
|
||||||
|
exit! AsaCtrl::ExitCodes::OK
|
||||||
|
end
|
||||||
|
end
|
||||||
|
end
|
||||||
|
end
|
||||||
+23
@@ -0,0 +1,23 @@
|
|||||||
|
module AsaCtrl
|
||||||
|
module Cli
|
||||||
|
HELP_ARGUMENT = '--help'
|
||||||
|
HELP_DESCRIPTION = 'Prints a help message'
|
||||||
|
|
||||||
|
def self.passed_command(args)
|
||||||
|
if ARGV.size == 0
|
||||||
|
[]
|
||||||
|
else
|
||||||
|
[ARGV[0]]
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
|
def self.print_usage
|
||||||
|
puts "Usage: asa-ctrl [rcon] (--help)"
|
||||||
|
end
|
||||||
|
|
||||||
|
def self.exit_with_error!(message, code)
|
||||||
|
$stderr.puts "Error: #{message}"
|
||||||
|
exit! code
|
||||||
|
end
|
||||||
|
end
|
||||||
|
end
|
||||||
+5
@@ -0,0 +1,5 @@
|
|||||||
|
module AsaCtrl
|
||||||
|
module Errors
|
||||||
|
class BaseError < StandardError; end
|
||||||
|
end
|
||||||
|
end
|
||||||
+5
@@ -0,0 +1,5 @@
|
|||||||
|
require_relative './base_error.rb'
|
||||||
|
require_relative './mod_already_enabled_error.rb'
|
||||||
|
require_relative './rcon_authentication_error.rb'
|
||||||
|
require_relative './rcon_password_not_found_error.rb'
|
||||||
|
require_relative './rcon_port_not_found_error.rb'
|
||||||
+5
@@ -0,0 +1,5 @@
|
|||||||
|
module AsaCtrl
|
||||||
|
module Errors
|
||||||
|
class ModAlreadyEnabledError < BaseError; end
|
||||||
|
end
|
||||||
|
end
|
||||||
+5
@@ -0,0 +1,5 @@
|
|||||||
|
module AsaCtrl
|
||||||
|
module Errors
|
||||||
|
class RconAuthenticationError < BaseError; end
|
||||||
|
end
|
||||||
|
end
|
||||||
+5
@@ -0,0 +1,5 @@
|
|||||||
|
module AsaCtrl
|
||||||
|
module Errors
|
||||||
|
class RconPasswordNotFoundError < BaseError; end
|
||||||
|
end
|
||||||
|
end
|
||||||
+5
@@ -0,0 +1,5 @@
|
|||||||
|
module AsaCtrl
|
||||||
|
module Errors
|
||||||
|
class RconPortNotFoundError < BaseError; end
|
||||||
|
end
|
||||||
|
end
|
||||||
+10
@@ -0,0 +1,10 @@
|
|||||||
|
module AsaCtrl
|
||||||
|
module ExitCodes
|
||||||
|
OK = 0
|
||||||
|
CORRUPTED_MODS_DATABASE = 1
|
||||||
|
MOD_ALREADY_ENABLED = 2
|
||||||
|
RCON_PASSWORD_NOT_FOUND = 3
|
||||||
|
RCON_PASSWORD_WRONG = 4
|
||||||
|
RCON_COMMAND_EXECUTION_FAILED = 5
|
||||||
|
end
|
||||||
|
end
|
||||||
+2
@@ -0,0 +1,2 @@
|
|||||||
|
require_relative './start_params_helper.rb'
|
||||||
|
require_relative './ini_config_helper.rb'
|
||||||
+17
@@ -0,0 +1,17 @@
|
|||||||
|
module AsaCtrl
|
||||||
|
module IniConfigHelper
|
||||||
|
def self.game_user_settings_ini
|
||||||
|
self.parse('/home/gameserver/server-files/ShooterGame/Saved/Config/WindowsServer/GameUserSettings.ini')
|
||||||
|
end
|
||||||
|
|
||||||
|
def self.game_ini
|
||||||
|
self.parse('/home/gameserver/server-files/ShooterGame/Saved/Config/WindowsServer/Game.ini')
|
||||||
|
end
|
||||||
|
|
||||||
|
def self.parse(path)
|
||||||
|
return unless File.exist?(path)
|
||||||
|
|
||||||
|
IniParse.parse(File.read(path))
|
||||||
|
end
|
||||||
|
end
|
||||||
|
end
|
||||||
+22
@@ -0,0 +1,22 @@
|
|||||||
|
module AsaCtrl
|
||||||
|
module StartParamsHelper
|
||||||
|
def self.get_value(start_params, key)
|
||||||
|
return unless start_params
|
||||||
|
|
||||||
|
value = ''
|
||||||
|
offset = start_params.index("#{key}=")
|
||||||
|
|
||||||
|
return unless offset
|
||||||
|
|
||||||
|
offset += "#{key}=".length
|
||||||
|
|
||||||
|
start_params[offset..-1].each_char do |char|
|
||||||
|
break if char == ' ' || char == '?'
|
||||||
|
|
||||||
|
value += char
|
||||||
|
end
|
||||||
|
|
||||||
|
value
|
||||||
|
end
|
||||||
|
end
|
||||||
|
end
|
||||||
Executable
+36
@@ -0,0 +1,36 @@
|
|||||||
|
#!/usr/bin/ruby.ruby3.4
|
||||||
|
require 'json'
|
||||||
|
require 'slop'
|
||||||
|
require 'iniparse'
|
||||||
|
require 'socket'
|
||||||
|
|
||||||
|
if ENV['DEV'] == '1'
|
||||||
|
require 'byebug'
|
||||||
|
end
|
||||||
|
|
||||||
|
require_relative './exit_codes.rb'
|
||||||
|
require_relative './errors/errors.rb'
|
||||||
|
require_relative './helpers/helpers.rb'
|
||||||
|
require_relative './mods/database.rb'
|
||||||
|
require_relative './rcon/rcon.rb'
|
||||||
|
require_relative './cli/utils.rb'
|
||||||
|
require_relative './cli/interfaces/cli_interface.rb'
|
||||||
|
require_relative './cli/interfaces/mods_interface.rb'
|
||||||
|
require_relative './cli/interfaces/rcon_interface.rb'
|
||||||
|
|
||||||
|
main_args = Slop.parse(AsaCtrl::Cli.passed_command(ARGV)) do |args|
|
||||||
|
args.on 'rcon', 'Interface for RCON command execution' do
|
||||||
|
opts = Slop.parse(ARGV[1..-1]) do |opt|
|
||||||
|
opt.string '--exec', 'An RCON command to execute'
|
||||||
|
opt.bool AsaCtrl::Cli::HELP_ARGUMENT, AsaCtrl::Cli::HELP_DESCRIPTION
|
||||||
|
end
|
||||||
|
|
||||||
|
AsaCtrl::Cli::RconInterface.new(opts)
|
||||||
|
end
|
||||||
|
|
||||||
|
args.on AsaCtrl::Cli::HELP_ARGUMENT, AsaCtrl::Cli::HELP_DESCRIPTION do
|
||||||
|
# handled once slop exits
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
|
AsaCtrl::Cli.print_usage
|
||||||
+66
@@ -0,0 +1,66 @@
|
|||||||
|
module AsaCtrl
|
||||||
|
module Mods
|
||||||
|
MOD_DATABASE_PATH = '/home/gameserver/server-files/mods.json'
|
||||||
|
|
||||||
|
class Database
|
||||||
|
@@singleton_reference = nil
|
||||||
|
|
||||||
|
def initialize(database_path)
|
||||||
|
@database_path = database_path
|
||||||
|
|
||||||
|
ensure_database_presence!
|
||||||
|
load_database
|
||||||
|
end
|
||||||
|
|
||||||
|
def self.get_instance
|
||||||
|
return @@singleton_reference if @@singleton_reference
|
||||||
|
@@singleton_reference = Database.new(MOD_DATABASE_PATH)
|
||||||
|
end
|
||||||
|
|
||||||
|
def enable_mod!(mod_id)
|
||||||
|
@database.each do |record|
|
||||||
|
if record['mod_id'].to_i == mod_id.to_i
|
||||||
|
raise AsaCtrl::Errors::ModAlreadyEnabledError if record['enabled']
|
||||||
|
|
||||||
|
record['enabled'] = true
|
||||||
|
write_database!
|
||||||
|
|
||||||
|
return
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
|
add_new_record!(mod_id, 'unknown', true, false)
|
||||||
|
end
|
||||||
|
|
||||||
|
def add_new_record!(mod_id, name, enabled, scanned)
|
||||||
|
@database << {
|
||||||
|
mod_id: mod_id.to_i,
|
||||||
|
name: name,
|
||||||
|
enabled: enabled,
|
||||||
|
scanned: scanned
|
||||||
|
}
|
||||||
|
|
||||||
|
write_database!
|
||||||
|
end
|
||||||
|
|
||||||
|
def write_database!
|
||||||
|
File.write(@database_path, JSON.pretty_generate(@database))
|
||||||
|
end
|
||||||
|
|
||||||
|
def ensure_database_presence!
|
||||||
|
return if File.exist?(@database_path)
|
||||||
|
|
||||||
|
@database = []
|
||||||
|
write_database!
|
||||||
|
end
|
||||||
|
|
||||||
|
def load_database
|
||||||
|
@database = JSON.parse(File.read(@database_path))
|
||||||
|
rescue JSON::ParserError
|
||||||
|
# we do not want to delete the file for the user, as they might want to save its content first
|
||||||
|
AsaCtrl::Cli.exit_with_error!("mods.json file is corrupted and cannot be parsed, please delete this file " \
|
||||||
|
"manually. It can be found in the server files root directory.", AsaCtrl::ExitCodes::CORRUPTED_MODS_DATABASE)
|
||||||
|
end
|
||||||
|
end
|
||||||
|
end
|
||||||
|
end
|
||||||
+69
@@ -0,0 +1,69 @@
|
|||||||
|
module AsaCtrl
|
||||||
|
module Rcon
|
||||||
|
module PacketTypes
|
||||||
|
RESPONSE_VALUE = 0
|
||||||
|
EXEC_COMMAND = 2
|
||||||
|
AUTH_RESPONSE = 2
|
||||||
|
AUTH = 3
|
||||||
|
end
|
||||||
|
|
||||||
|
Packet = Struct.new(:size, :id, :type, :body)
|
||||||
|
|
||||||
|
def self.exec_command!(server_ip, rcon_port, rcon_command, password)
|
||||||
|
socket = TCPSocket.new(server_ip, rcon_port)
|
||||||
|
raise AsaCtrl::Errors::RconAuthenticationError unless self.authenticate!(socket, password)
|
||||||
|
|
||||||
|
self.send_packet!(socket, rcon_command, PacketTypes::EXEC_COMMAND)
|
||||||
|
end
|
||||||
|
|
||||||
|
def self.authenticate!(socket, password)
|
||||||
|
response = self.send_packet!(socket, password, PacketTypes::AUTH)
|
||||||
|
response[:id] != -1
|
||||||
|
end
|
||||||
|
|
||||||
|
def self.send_packet!(socket, data, packet_id)
|
||||||
|
packet = Packet.new(10+data.bytesize, 0, packet_id, data)
|
||||||
|
|
||||||
|
self.send_to(packet, socket)
|
||||||
|
self.recv_from(socket)
|
||||||
|
end
|
||||||
|
|
||||||
|
def self.send_to(packet, socket)
|
||||||
|
szb = [packet[:size]].pack 'l<'
|
||||||
|
idb = [packet[:id]].pack 'l<'
|
||||||
|
type_b = [packet[:type]].pack 'l<'
|
||||||
|
body_b = [packet[:body]].pack 'Z*'
|
||||||
|
data = szb + idb + type_b + body_b + "\0"
|
||||||
|
|
||||||
|
socket.sendmsg(data)
|
||||||
|
end
|
||||||
|
|
||||||
|
def self.recv_from(socket)
|
||||||
|
msg_ary = socket.recvmsg
|
||||||
|
msg = msg_ary[0]
|
||||||
|
ary = msg.unpack('l<l<l<Z*')
|
||||||
|
|
||||||
|
Packet.new(ary[0], ary[1], ary[2], ary[3])
|
||||||
|
end
|
||||||
|
|
||||||
|
def self.identify_password
|
||||||
|
password = AsaCtrl::StartParamsHelper.get_value(ENV['ASA_START_PARAMS'], 'ServerAdminPassword')
|
||||||
|
return password if password
|
||||||
|
|
||||||
|
config = AsaCtrl::IniConfigHelper.game_user_settings_ini
|
||||||
|
return config['ServerSettings']['ServerAdminPassword'] if config['ServerSettings'] && config['ServerSettings']['ServerAdminPassword']
|
||||||
|
|
||||||
|
raise AsaCtrl::Errors::RconPasswordNotFoundError
|
||||||
|
end
|
||||||
|
|
||||||
|
def self.identify_port
|
||||||
|
port = AsaCtrl::StartParamsHelper.get_value(ENV['ASA_START_PARAMS'], 'RCONPort')
|
||||||
|
return port.to_i if port
|
||||||
|
|
||||||
|
config = AsaCtrl::IniConfigHelper.game_user_settings_ini
|
||||||
|
return config['ServerSettings']['RCONPort'].to_i if config['ServerSettings'] && config['ServerSettings']['RCONPort']
|
||||||
|
|
||||||
|
raise AsaCtrl::Errors::RconPortNotFoundError
|
||||||
|
end
|
||||||
|
end
|
||||||
|
end
|
||||||
+1
@@ -0,0 +1 @@
|
|||||||
|
d21bd48479ab35213a7bf5b7eb87d0f156da16891cdc6bed3665e268811304c20cc9834ff901b9d4e31abc6d10bf6b8066276d32f7c487607270a3a9f975ae2c /tmp/GE-Proton10-17.tar.gz
|
||||||
@@ -0,0 +1,129 @@
|
|||||||
|
name: ark-suac
|
||||||
|
|
||||||
|
services:
|
||||||
|
ark-asa-server:
|
||||||
|
image: mschnitzer/asa-linux-server:1.4.0
|
||||||
|
container_name: ark-asa-server
|
||||||
|
hostname: ark-asa-server
|
||||||
|
entrypoint: /usr/bin/start_server
|
||||||
|
user: gameserver
|
||||||
|
tty: true
|
||||||
|
restart: unless-stopped
|
||||||
|
|
||||||
|
environment:
|
||||||
|
TZ: Europe/Stockholm
|
||||||
|
ASA_START_PARAMS: 'TheIsland_WP?listen?Port=7777?RCONPort=27020?RCONEnabled=True -WinLiveMaxPlayers=50 -clusterid=default -ClusterDirOverride="/home/gameserver/cluster-shared"'
|
||||||
|
ENABLE_DEBUG: "0"
|
||||||
|
|
||||||
|
ports:
|
||||||
|
- target: 7777
|
||||||
|
published: "7777"
|
||||||
|
protocol: udp
|
||||||
|
- target: 27020
|
||||||
|
published: "27020"
|
||||||
|
protocol: tcp
|
||||||
|
|
||||||
|
depends_on:
|
||||||
|
- set-permissions
|
||||||
|
|
||||||
|
volumes:
|
||||||
|
- type: bind
|
||||||
|
source: /DATA/AppData/$AppID/steam
|
||||||
|
target: /home/gameserver/Steam
|
||||||
|
- type: bind
|
||||||
|
source: /DATA/AppData/$AppID/steamcmd
|
||||||
|
target: /home/gameserver/steamcmd
|
||||||
|
- type: bind
|
||||||
|
source: /DATA/AppData/$AppID/server-files
|
||||||
|
target: /home/gameserver/server-files
|
||||||
|
- type: bind
|
||||||
|
source: /DATA/AppData/$AppID/cluster-shared
|
||||||
|
target: /home/gameserver/cluster-shared
|
||||||
|
- type: bind
|
||||||
|
source: /etc/localtime
|
||||||
|
target: /etc/localtime
|
||||||
|
read_only: true
|
||||||
|
|
||||||
|
# Secure-by-default baseline. Relax only if appen kräver det.
|
||||||
|
security_opt:
|
||||||
|
- no-new-privileges:true
|
||||||
|
cap_drop:
|
||||||
|
- ALL
|
||||||
|
|
||||||
|
x-casaos:
|
||||||
|
envs:
|
||||||
|
- container: TZ
|
||||||
|
description:
|
||||||
|
en_us: Timezone, for example Europe/Stockholm
|
||||||
|
- container: ASA_START_PARAMS
|
||||||
|
description:
|
||||||
|
en_us: ARK start params, including map and ports
|
||||||
|
- container: ENABLE_DEBUG
|
||||||
|
description:
|
||||||
|
en_us: Set 1 to enter debug mode without starting the server
|
||||||
|
ports:
|
||||||
|
- container: "7777"
|
||||||
|
description:
|
||||||
|
en_us: Game port for player connections (UDP)
|
||||||
|
- container: "27020"
|
||||||
|
description:
|
||||||
|
en_us: RCON admin port (TCP)
|
||||||
|
volumes:
|
||||||
|
- container: /home/gameserver/Steam
|
||||||
|
description:
|
||||||
|
en_us: Steam runtime cache
|
||||||
|
- container: /home/gameserver/steamcmd
|
||||||
|
description:
|
||||||
|
en_us: SteamCMD cache
|
||||||
|
- container: /home/gameserver/server-files
|
||||||
|
description:
|
||||||
|
en_us: ARK server files and saved data
|
||||||
|
- container: /home/gameserver/cluster-shared
|
||||||
|
description:
|
||||||
|
en_us: Shared cluster transfer data
|
||||||
|
|
||||||
|
set-permissions:
|
||||||
|
image: opensuse/leap:15.6
|
||||||
|
container_name: ark-asa-set-permissions
|
||||||
|
user: root
|
||||||
|
restart: "no"
|
||||||
|
entrypoint:
|
||||||
|
- /bin/bash
|
||||||
|
- -c
|
||||||
|
- chown -R 25000:25000 /steam /steamcmd /server-files /cluster-shared
|
||||||
|
volumes:
|
||||||
|
- type: bind
|
||||||
|
source: /DATA/AppData/$AppID/steam
|
||||||
|
target: /steam
|
||||||
|
- type: bind
|
||||||
|
source: /DATA/AppData/$AppID/steamcmd
|
||||||
|
target: /steamcmd
|
||||||
|
- type: bind
|
||||||
|
source: /DATA/AppData/$AppID/server-files
|
||||||
|
target: /server-files
|
||||||
|
- type: bind
|
||||||
|
source: /DATA/AppData/$AppID/cluster-shared
|
||||||
|
target: /cluster-shared
|
||||||
|
|
||||||
|
x-casaos:
|
||||||
|
architectures:
|
||||||
|
- amd64
|
||||||
|
- arm64
|
||||||
|
main: ark-asa-server
|
||||||
|
category: phirna
|
||||||
|
author: Joachim Friberg
|
||||||
|
developer: Joachim Friberg
|
||||||
|
icon: https://cdn.jsdelivr.net/gh/selfhst/icons/png/ark-survival-ascended.png
|
||||||
|
tagline:
|
||||||
|
en_us: ARK Survival Ascended dedicated server for ZimaOS
|
||||||
|
description:
|
||||||
|
en_us: |
|
||||||
|
Host your own ARK: Survival Ascended dedicated server.
|
||||||
|
|
||||||
|
Minimum requirements:
|
||||||
|
- RAM: 16 GB (24 GB recommended)
|
||||||
|
- Disk: 20 GB+ for server files and updates
|
||||||
|
title:
|
||||||
|
en_us: ARK Survival Ascended Server
|
||||||
|
index: /
|
||||||
|
port_map: "7777"
|
||||||
@@ -27,32 +27,22 @@ Konfigurera lokal DNS (AdGuard Home):
|
|||||||
- `A`/rewrite för `*.home.example.com` -> ZimaOS LAN-IP
|
- `A`/rewrite för `*.home.example.com` -> ZimaOS LAN-IP
|
||||||
- `A`/rewrite för `home.example.com` -> ZimaOS LAN-IP
|
- `A`/rewrite för `home.example.com` -> ZimaOS LAN-IP
|
||||||
|
|
||||||
## 3) Positivt test: endpoint skapas korrekt
|
## 3) Positivt test: status-UI + endpoint
|
||||||
|
|
||||||
Exempel med Frigate (web UI på 5000):
|
|
||||||
|
|
||||||
Sätt env-vars på Frigate-containern:
|
|
||||||
|
|
||||||
```text
|
|
||||||
LABEL_CADDY_ENABLE=true
|
|
||||||
LABEL_CADDY_TARGET_PORT=5000
|
|
||||||
LABEL_CADDY_HOST=frigate
|
|
||||||
LABEL_CADDY_SCHEME=http
|
|
||||||
LABEL_CADDY_PATH=/
|
|
||||||
```
|
|
||||||
|
|
||||||
Verifiera:
|
|
||||||
|
|
||||||
1. Vänta 15-30 sekunder (default polling).
|
1. Vänta 15-30 sekunder (default polling).
|
||||||
2. Öppna `https://frigate.home.example.com`.
|
2. Öppna `http://<zima-lan-ip>:31820/`.
|
||||||
3. Kontrollera att sidan svarar med giltigt certifikat.
|
3. Verifiera i UI:
|
||||||
4. Kontrollera att media-portar (8554/8555) inte blivit egna hostnamn/endpoints.
|
- `Caddy Apply = OK`
|
||||||
|
- `Cloudflare Reachability = OK`
|
||||||
|
- `Cloudflare Token = OK`
|
||||||
|
- minst en route visas under `Generated Hosts`.
|
||||||
|
4. Öppna en exponerad host, t.ex. `https://frigate.home.example.com`.
|
||||||
|
|
||||||
Förväntat resultat:
|
Förväntat resultat:
|
||||||
|
|
||||||
- Endpoint finns för web UI.
|
- UI laddar och visar aktuell status.
|
||||||
|
- Endpoint finns för web UI-hosten.
|
||||||
- Certifikat utfärdas via Let's Encrypt DNS-01.
|
- Certifikat utfärdas via Let's Encrypt DNS-01.
|
||||||
- Endast explicit målport routas.
|
|
||||||
|
|
||||||
## 4) Negativt test: fail-closed
|
## 4) Negativt test: fail-closed
|
||||||
|
|
||||||
@@ -60,24 +50,26 @@ Scenario A, ingen opt-in:
|
|||||||
|
|
||||||
1. Ta bort `LABEL_CADDY_ENABLE=true` från målcontainern.
|
1. Ta bort `LABEL_CADDY_ENABLE=true` från målcontainern.
|
||||||
2. Vänta en polling-cykel.
|
2. Vänta en polling-cykel.
|
||||||
3. Kontrollera att endpointen försvinner/slutar routas.
|
3. Verifiera i status-UI att routen försvinner.
|
||||||
|
|
||||||
Scenario B, saknad Cloudflare-token:
|
Scenario B, saknad/ogiltig Cloudflare-token:
|
||||||
|
|
||||||
1. Sätt `REQUIRE_CLOUDFLARE=true`.
|
1. Sätt `REQUIRE_CLOUDFLARE=true`.
|
||||||
2. Ta bort eller ogiltiggör `CLOUDFLARE_API_TOKEN`.
|
2. Ta bort eller ogiltiggör `CLOUDFLARE_API_TOKEN`.
|
||||||
3. Verifiera att ingen ny extern route publiceras.
|
3. Verifiera i status-UI att Cloudflare Token inte är OK.
|
||||||
|
4. Verifiera att ingen ny extern route publiceras.
|
||||||
|
|
||||||
Förväntat resultat:
|
Förväntat resultat:
|
||||||
|
|
||||||
- Appen exponerar inte tjänster oavsiktligt.
|
- Appen exponerar inte tjänster oavsiktligt.
|
||||||
- Loggar visar tydligt fel kring token/Cloudflare.
|
- Loggar visar tydligt token/Cloudflare-relaterat fel.
|
||||||
|
|
||||||
## 5) Kommandon för snabb verifiering
|
## 5) Kommandon för snabb verifiering
|
||||||
|
|
||||||
Byt `<domain>` och `<host>` enligt din miljö.
|
Byt `<domain>`, `<host>` och `<zima-lan-ip>` enligt din miljö.
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
|
curl -sS http://<zima-lan-ip>:31820/status.json | jq .
|
||||||
nslookup frigate.home.example.com
|
nslookup frigate.home.example.com
|
||||||
curl -vk https://frigate.home.example.com/
|
curl -vk https://frigate.home.example.com/
|
||||||
openssl s_client -connect frigate.home.example.com:443 -servername frigate.home.example.com </dev/null
|
openssl s_client -connect frigate.home.example.com:443 -servername frigate.home.example.com </dev/null
|
||||||
@@ -106,6 +98,7 @@ Samla detta först. Det kortar felsökningstiden avsevärt.
|
|||||||
- Målcontainerns publicerade portar.
|
- Målcontainerns publicerade portar.
|
||||||
|
|
||||||
3. Nät/TLS-bevis:
|
3. Nät/TLS-bevis:
|
||||||
|
- `curl -sS http://<zima-lan-ip>:31820/status.json | jq .`.
|
||||||
- `nslookup`-svar från en LAN-klient.
|
- `nslookup`-svar från en LAN-klient.
|
||||||
- `curl -vk` output mot endpoint.
|
- `curl -vk` output mot endpoint.
|
||||||
- `openssl s_client` sammanfattning av cert/issuer.
|
- `openssl s_client` sammanfattning av cert/issuer.
|
||||||
|
|||||||
@@ -8,6 +8,25 @@ Arkitektur:
|
|||||||
- Endast explicit opt-in exponeras (`LABEL_CADDY_ENABLE=true`).
|
- Endast explicit opt-in exponeras (`LABEL_CADDY_ENABLE=true`).
|
||||||
- Caddy config laddas dynamiskt via Caddy Admin API (`POST /load`).
|
- Caddy config laddas dynamiskt via Caddy Admin API (`POST /load`).
|
||||||
- TLS sker med Let's Encrypt DNS-01 via Cloudflare.
|
- TLS sker med Let's Encrypt DNS-01 via Cloudflare.
|
||||||
|
- Ett enkelt status-UI finns på port `31820` (LAN/private ranges only).
|
||||||
|
|
||||||
|
## Status-UI (v1)
|
||||||
|
|
||||||
|
Status-UI visar:
|
||||||
|
|
||||||
|
- senaste config apply-status,
|
||||||
|
- Cloudflare reachability + token-validitet,
|
||||||
|
- genererade hosts/routes,
|
||||||
|
- om Let's Encrypt-cert verkar finnas för varje host.
|
||||||
|
|
||||||
|
URL (default):
|
||||||
|
|
||||||
|
- `http://<zima-lan-ip>:31820/`
|
||||||
|
|
||||||
|
Obs:
|
||||||
|
|
||||||
|
- "Hosts" i UI betyder **genererade Caddy-routes**, inte att appen skapar A/CNAME-records i Cloudflare-zonen.
|
||||||
|
- Certstatus bygger på Caddys lokala certificate storage och är en praktisk v1-signal.
|
||||||
|
|
||||||
## Säkerhetsmodell
|
## Säkerhetsmodell
|
||||||
|
|
||||||
@@ -15,6 +34,7 @@ Arkitektur:
|
|||||||
- Ingen auto-exponering av alla portar.
|
- Ingen auto-exponering av alla portar.
|
||||||
- Endast HTTP(S)-upstreams stöds i v1.
|
- Endast HTTP(S)-upstreams stöds i v1.
|
||||||
- Docker socket exponeras inte direkt till agenten, endast via `socket-proxy` med begränsade endpoint-flaggor.
|
- Docker socket exponeras inte direkt till agenten, endast via `socket-proxy` med begränsade endpoint-flaggor.
|
||||||
|
- Status-UI är LAN-begränsat (`remote_ip private_ranges`).
|
||||||
|
|
||||||
## Miljövariabler (appnivå)
|
## Miljövariabler (appnivå)
|
||||||
|
|
||||||
@@ -24,6 +44,9 @@ Arkitektur:
|
|||||||
- `REQUIRE_CLOUDFLARE` (default `true`)
|
- `REQUIRE_CLOUDFLARE` (default `true`)
|
||||||
- `ALLOW_INTERNAL_TLS_FALLBACK` (default `false`)
|
- `ALLOW_INTERNAL_TLS_FALLBACK` (default `false`)
|
||||||
- `POLL_SECONDS` (default `15`)
|
- `POLL_SECONDS` (default `15`)
|
||||||
|
- `STATUS_BIND` (default `0.0.0.0:8089`, intern agent endpoint)
|
||||||
|
- `STATUS_UI_PORT` (default `31820`)
|
||||||
|
- `CF_VERIFY_URL` (default Cloudflare token verify endpoint)
|
||||||
|
|
||||||
Cloudflare-token bör vara scoped till minsta möjliga rättigheter:
|
Cloudflare-token bör vara scoped till minsta möjliga rättigheter:
|
||||||
|
|
||||||
@@ -97,10 +120,12 @@ Kör lokala integrationstester för discovery-agenten:
|
|||||||
./Apps/caddy-autogen/tests/run_integration_tests.sh
|
./Apps/caddy-autogen/tests/run_integration_tests.sh
|
||||||
```
|
```
|
||||||
|
|
||||||
Testerna mockar Docker API-svar och Caddy `/load`-anrop och verifierar:
|
Testerna mockar Docker API-svar och Cloudflare verify och verifierar:
|
||||||
|
|
||||||
- opt-in-regler (endast markerade containers exponeras),
|
- opt-in-regler (endast markerade containers exponeras),
|
||||||
- säker portselektionslogik (inga media/UDP-portar routas av misstag),
|
- säker portselektionslogik (inga media/UDP-portar routas av misstag),
|
||||||
- fail-closed beteende när Cloudflare-token saknas.
|
- fail-closed beteende när Cloudflare-token saknas,
|
||||||
|
- status-UI-block i genererad Caddy-konfiguration,
|
||||||
|
- cert-matchning mot lokal Caddy certificate storage.
|
||||||
|
|
||||||
För verifiering i riktig ZimaOS-miljö, se `HOW_TO_VERIFY.md` i samma mapp.
|
För verifiering i riktig ZimaOS-miljö, se `HOW_TO_VERIFY.md` i samma mapp.
|
||||||
|
|||||||
@@ -4,10 +4,12 @@ import json
|
|||||||
import os
|
import os
|
||||||
import re
|
import re
|
||||||
import sys
|
import sys
|
||||||
|
import threading
|
||||||
import time
|
import time
|
||||||
import urllib.error
|
import urllib.error
|
||||||
import urllib.parse
|
import urllib.parse
|
||||||
import urllib.request
|
import urllib.request
|
||||||
|
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
|
||||||
|
|
||||||
|
|
||||||
TRUE_VALUES = {"1", "true", "yes", "on"}
|
TRUE_VALUES = {"1", "true", "yes", "on"}
|
||||||
@@ -116,7 +118,15 @@ def _build_fqdn(host_hint: str, base_domain: str) -> str:
|
|||||||
return fqdn
|
return fqdn
|
||||||
|
|
||||||
|
|
||||||
def _collect_routes(docker_api_url: str, env_prefix: str, denylist: set, base_domain: str, default_scheme: str, default_path: str, default_health_uri: str):
|
def _collect_routes(
|
||||||
|
docker_api_url: str,
|
||||||
|
env_prefix: str,
|
||||||
|
denylist: set,
|
||||||
|
base_domain: str,
|
||||||
|
default_scheme: str,
|
||||||
|
default_path: str,
|
||||||
|
default_health_uri: str,
|
||||||
|
):
|
||||||
routes = []
|
routes = []
|
||||||
containers = _get_json(f"{docker_api_url}/containers/json?all=0")
|
containers = _get_json(f"{docker_api_url}/containers/json?all=0")
|
||||||
for c in containers:
|
for c in containers:
|
||||||
@@ -175,7 +185,32 @@ def _collect_routes(docker_api_url: str, env_prefix: str, denylist: set, base_do
|
|||||||
return routes
|
return routes
|
||||||
|
|
||||||
|
|
||||||
def _generate_caddyfile(routes, token: str, require_cloudflare: bool, allow_internal_tls_fallback: bool, wildcard_domain: str, cert_email: str):
|
def _append_status_site(out: list[str], status_ui_port: int, status_upstream: str) -> None:
|
||||||
|
out.append(f":{status_ui_port} {{")
|
||||||
|
out.append(" @allowed remote_ip private_ranges")
|
||||||
|
out.append(" handle @allowed {")
|
||||||
|
out.append(" @status_json path /status.json")
|
||||||
|
out.append(" handle @status_json {")
|
||||||
|
out.append(f" reverse_proxy {status_upstream}")
|
||||||
|
out.append(" }")
|
||||||
|
out.append(" root * /srv/status")
|
||||||
|
out.append(" file_server")
|
||||||
|
out.append(" }")
|
||||||
|
out.append(" respond \"forbidden\" 403")
|
||||||
|
out.append("}")
|
||||||
|
out.append("")
|
||||||
|
|
||||||
|
|
||||||
|
def _generate_caddyfile(
|
||||||
|
routes,
|
||||||
|
token: str,
|
||||||
|
require_cloudflare: bool,
|
||||||
|
allow_internal_tls_fallback: bool,
|
||||||
|
wildcard_domain: str,
|
||||||
|
cert_email: str,
|
||||||
|
status_ui_port: int,
|
||||||
|
status_upstream: str,
|
||||||
|
):
|
||||||
if require_cloudflare and not token:
|
if require_cloudflare and not token:
|
||||||
raise RuntimeError("CLOUDFLARE_API_TOKEN is required in fail-closed mode")
|
raise RuntimeError("CLOUDFLARE_API_TOKEN is required in fail-closed mode")
|
||||||
|
|
||||||
@@ -189,6 +224,8 @@ def _generate_caddyfile(routes, token: str, require_cloudflare: bool, allow_inte
|
|||||||
out.append("}")
|
out.append("}")
|
||||||
out.append("")
|
out.append("")
|
||||||
|
|
||||||
|
_append_status_site(out, status_ui_port=status_ui_port, status_upstream=status_upstream)
|
||||||
|
|
||||||
if wildcard_domain and token:
|
if wildcard_domain and token:
|
||||||
out.append(f"{wildcard_domain}, *.{wildcard_domain} {{")
|
out.append(f"{wildcard_domain}, *.{wildcard_domain} {{")
|
||||||
out.append(" tls {")
|
out.append(" tls {")
|
||||||
@@ -237,6 +274,147 @@ def _generate_caddyfile(routes, token: str, require_cloudflare: bool, allow_inte
|
|||||||
return "\n".join(out)
|
return "\n".join(out)
|
||||||
|
|
||||||
|
|
||||||
|
def _parse_bind_addr(value: str, default_host: str = "0.0.0.0", default_port: int = 8089) -> tuple[str, int]:
|
||||||
|
raw = str(value or "").strip()
|
||||||
|
if not raw:
|
||||||
|
return default_host, default_port
|
||||||
|
if ":" not in raw:
|
||||||
|
raise ValueError(f"invalid bind address '{raw}', expected host:port")
|
||||||
|
host, port_raw = raw.rsplit(":", 1)
|
||||||
|
if not host:
|
||||||
|
host = default_host
|
||||||
|
try:
|
||||||
|
port = int(port_raw)
|
||||||
|
except ValueError as exc:
|
||||||
|
raise ValueError(f"invalid port in bind address '{raw}'") from exc
|
||||||
|
if port < 1 or port > 65535:
|
||||||
|
raise ValueError(f"port out of range in bind address '{raw}'")
|
||||||
|
return host, port
|
||||||
|
|
||||||
|
|
||||||
|
def _verify_cloudflare_token(verify_url: str, token: str) -> dict:
|
||||||
|
now = int(time.time())
|
||||||
|
if not token:
|
||||||
|
return {
|
||||||
|
"reachable": False,
|
||||||
|
"token_valid": False,
|
||||||
|
"last_check_ts": now,
|
||||||
|
"error": "CLOUDFLARE_API_TOKEN is missing",
|
||||||
|
}
|
||||||
|
|
||||||
|
req = urllib.request.Request(
|
||||||
|
verify_url,
|
||||||
|
headers={
|
||||||
|
"Authorization": f"Bearer {token}",
|
||||||
|
"Accept": "application/json",
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
|
try:
|
||||||
|
with urllib.request.urlopen(req, timeout=10) as resp:
|
||||||
|
payload = json.loads(resp.read().decode("utf-8"))
|
||||||
|
return {
|
||||||
|
"reachable": True,
|
||||||
|
"token_valid": bool(payload.get("success", False)),
|
||||||
|
"last_check_ts": now,
|
||||||
|
"error": "",
|
||||||
|
}
|
||||||
|
except urllib.error.HTTPError as exc:
|
||||||
|
return {
|
||||||
|
"reachable": True,
|
||||||
|
"token_valid": False,
|
||||||
|
"last_check_ts": now,
|
||||||
|
"error": f"HTTP {exc.code}",
|
||||||
|
}
|
||||||
|
except urllib.error.URLError as exc:
|
||||||
|
return {
|
||||||
|
"reachable": False,
|
||||||
|
"token_valid": False,
|
||||||
|
"last_check_ts": now,
|
||||||
|
"error": f"connection failure: {exc}",
|
||||||
|
}
|
||||||
|
except Exception as exc:
|
||||||
|
return {
|
||||||
|
"reachable": False,
|
||||||
|
"token_valid": False,
|
||||||
|
"last_check_ts": now,
|
||||||
|
"error": str(exc),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _collect_letsencrypt_hosts(caddy_data_dir: str) -> set[str]:
|
||||||
|
results: set[str] = set()
|
||||||
|
cert_root = os.path.join(caddy_data_dir, "caddy", "certificates")
|
||||||
|
if not os.path.isdir(cert_root):
|
||||||
|
return results
|
||||||
|
|
||||||
|
for root, _dirs, files in os.walk(cert_root):
|
||||||
|
if "letsencrypt" not in root.lower():
|
||||||
|
continue
|
||||||
|
for filename in files:
|
||||||
|
if not filename.endswith(".crt"):
|
||||||
|
continue
|
||||||
|
host = filename[:-4].lower()
|
||||||
|
if host.startswith("_."):
|
||||||
|
host = "*." + host[2:]
|
||||||
|
if host and host != "*" and (host.startswith("*.") or HOST_RE.match(host)):
|
||||||
|
results.add(host)
|
||||||
|
return results
|
||||||
|
|
||||||
|
|
||||||
|
def _has_matching_le_cert(route_fqdn: str, cert_hosts: set[str]) -> bool:
|
||||||
|
if route_fqdn in cert_hosts:
|
||||||
|
return True
|
||||||
|
for cert_host in cert_hosts:
|
||||||
|
if not cert_host.startswith("*."):
|
||||||
|
continue
|
||||||
|
suffix = cert_host[2:]
|
||||||
|
if route_fqdn == suffix or route_fqdn.endswith("." + suffix):
|
||||||
|
return True
|
||||||
|
return False
|
||||||
|
|
||||||
|
|
||||||
|
def _build_status_payload(state: dict) -> bytes:
|
||||||
|
body = json.dumps(state, separators=(",", ":"), sort_keys=True).encode("utf-8")
|
||||||
|
return body
|
||||||
|
|
||||||
|
|
||||||
|
def _start_status_server(bind_addr: str, snapshot: dict, lock: threading.Lock):
|
||||||
|
host, port = _parse_bind_addr(bind_addr)
|
||||||
|
|
||||||
|
class _Handler(BaseHTTPRequestHandler):
|
||||||
|
def do_GET(self):
|
||||||
|
if self.path not in {"/status.json", "/healthz"}:
|
||||||
|
self.send_response(404)
|
||||||
|
self.end_headers()
|
||||||
|
return
|
||||||
|
|
||||||
|
if self.path == "/healthz":
|
||||||
|
self.send_response(200)
|
||||||
|
self.send_header("Content-Type", "text/plain; charset=utf-8")
|
||||||
|
self.end_headers()
|
||||||
|
self.wfile.write(b"ok")
|
||||||
|
return
|
||||||
|
|
||||||
|
with lock:
|
||||||
|
payload = _build_status_payload(snapshot)
|
||||||
|
self.send_response(200)
|
||||||
|
self.send_header("Content-Type", "application/json; charset=utf-8")
|
||||||
|
self.send_header("Cache-Control", "no-store")
|
||||||
|
self.send_header("Content-Length", str(len(payload)))
|
||||||
|
self.end_headers()
|
||||||
|
self.wfile.write(payload)
|
||||||
|
|
||||||
|
def log_message(self, fmt, *args):
|
||||||
|
return
|
||||||
|
|
||||||
|
server = ThreadingHTTPServer((host, port), _Handler)
|
||||||
|
thread = threading.Thread(target=server.serve_forever, daemon=True)
|
||||||
|
thread.start()
|
||||||
|
_log(f"INFO: status endpoint listening on {host}:{port}")
|
||||||
|
return server, thread
|
||||||
|
|
||||||
|
|
||||||
def main():
|
def main():
|
||||||
defaults = _read_simple_yaml(os.getenv("CONFIG_FILE", "/app/config/defaults.yaml"))
|
defaults = _read_simple_yaml(os.getenv("CONFIG_FILE", "/app/config/defaults.yaml"))
|
||||||
|
|
||||||
@@ -249,6 +427,25 @@ def main():
|
|||||||
default_scheme = str(_cfg("DEFAULT_SCHEME", defaults, "default_scheme", "http")).strip().lower()
|
default_scheme = str(_cfg("DEFAULT_SCHEME", defaults, "default_scheme", "http")).strip().lower()
|
||||||
default_path = str(_cfg("DEFAULT_PATH", defaults, "default_path", "/")).strip() or "/"
|
default_path = str(_cfg("DEFAULT_PATH", defaults, "default_path", "/")).strip() or "/"
|
||||||
default_health_uri = str(_cfg("DEFAULT_HEALTH_URI", defaults, "default_health_uri", "")).strip()
|
default_health_uri = str(_cfg("DEFAULT_HEALTH_URI", defaults, "default_health_uri", "")).strip()
|
||||||
|
status_bind = str(_cfg("STATUS_BIND", defaults, "status_bind", "0.0.0.0:8089")).strip()
|
||||||
|
status_ui_port_raw = _cfg("STATUS_UI_PORT", defaults, "status_ui_port", 31820)
|
||||||
|
status_upstream = str(_cfg("STATUS_UPSTREAM", defaults, "status_upstream", "discovery-agent:8089")).strip()
|
||||||
|
cf_verify_url = str(
|
||||||
|
_cfg(
|
||||||
|
"CF_VERIFY_URL",
|
||||||
|
defaults,
|
||||||
|
"cf_verify_url",
|
||||||
|
"https://api.cloudflare.com/client/v4/user/tokens/verify",
|
||||||
|
)
|
||||||
|
).strip()
|
||||||
|
caddy_data_dir = str(_cfg("CADDY_DATA_DIR", defaults, "caddy_data_dir", "/caddy-data")).strip()
|
||||||
|
|
||||||
|
try:
|
||||||
|
status_ui_port = int(status_ui_port_raw)
|
||||||
|
if status_ui_port < 1 or status_ui_port > 65535:
|
||||||
|
raise ValueError
|
||||||
|
except (TypeError, ValueError):
|
||||||
|
status_ui_port = 31820
|
||||||
|
|
||||||
poll_seconds_raw = _cfg("POLL_SECONDS", defaults, "poll_seconds", 15)
|
poll_seconds_raw = _cfg("POLL_SECONDS", defaults, "poll_seconds", 15)
|
||||||
try:
|
try:
|
||||||
@@ -268,12 +465,37 @@ def main():
|
|||||||
token = os.getenv("CLOUDFLARE_API_TOKEN", "").strip()
|
token = os.getenv("CLOUDFLARE_API_TOKEN", "").strip()
|
||||||
last_digest = ""
|
last_digest = ""
|
||||||
|
|
||||||
|
snapshot_lock = threading.Lock()
|
||||||
|
snapshot = {
|
||||||
|
"app": {
|
||||||
|
"name": "caddy-autogen",
|
||||||
|
"status_ui_port": status_ui_port,
|
||||||
|
"status_upstream": status_upstream,
|
||||||
|
"require_cloudflare": require_cloudflare,
|
||||||
|
"allow_internal_tls_fallback": allow_internal_tls_fallback,
|
||||||
|
},
|
||||||
|
"last_tick_ts": 0,
|
||||||
|
"last_apply_ok": False,
|
||||||
|
"last_apply_http_status": 0,
|
||||||
|
"last_error": "not started",
|
||||||
|
"routes": [],
|
||||||
|
"cloudflare": {
|
||||||
|
"reachable": False,
|
||||||
|
"token_valid": False,
|
||||||
|
"last_check_ts": 0,
|
||||||
|
"error": "not checked",
|
||||||
|
},
|
||||||
|
"certs": [],
|
||||||
|
}
|
||||||
|
_start_status_server(status_bind, snapshot, snapshot_lock)
|
||||||
|
|
||||||
_log(
|
_log(
|
||||||
"INFO: starting caddy-autogen discovery-agent "
|
"INFO: starting caddy-autogen discovery-agent "
|
||||||
f"(docker_api_url={docker_api_url}, caddy_load_url={caddy_load_url}, poll_seconds={poll_seconds})"
|
f"(docker_api_url={docker_api_url}, caddy_load_url={caddy_load_url}, poll_seconds={poll_seconds})"
|
||||||
)
|
)
|
||||||
|
|
||||||
while True:
|
while True:
|
||||||
|
tick_ts = int(time.time())
|
||||||
try:
|
try:
|
||||||
routes = _collect_routes(
|
routes = _collect_routes(
|
||||||
docker_api_url=docker_api_url,
|
docker_api_url=docker_api_url,
|
||||||
@@ -284,6 +506,16 @@ def main():
|
|||||||
default_path=default_path,
|
default_path=default_path,
|
||||||
default_health_uri=default_health_uri,
|
default_health_uri=default_health_uri,
|
||||||
)
|
)
|
||||||
|
cloudflare_status = _verify_cloudflare_token(cf_verify_url, token)
|
||||||
|
cert_hosts = _collect_letsencrypt_hosts(caddy_data_dir)
|
||||||
|
cert_rows = [
|
||||||
|
{
|
||||||
|
"fqdn": route["fqdn"],
|
||||||
|
"letsencrypt_present": _has_matching_le_cert(route["fqdn"], cert_hosts),
|
||||||
|
}
|
||||||
|
for route in routes
|
||||||
|
]
|
||||||
|
|
||||||
caddyfile = _generate_caddyfile(
|
caddyfile = _generate_caddyfile(
|
||||||
routes=routes,
|
routes=routes,
|
||||||
token=token,
|
token=token,
|
||||||
@@ -291,21 +523,53 @@ def main():
|
|||||||
allow_internal_tls_fallback=allow_internal_tls_fallback,
|
allow_internal_tls_fallback=allow_internal_tls_fallback,
|
||||||
wildcard_domain=wildcard_domain,
|
wildcard_domain=wildcard_domain,
|
||||||
cert_email=cert_email,
|
cert_email=cert_email,
|
||||||
|
status_ui_port=status_ui_port,
|
||||||
|
status_upstream=status_upstream,
|
||||||
)
|
)
|
||||||
|
|
||||||
digest = hashlib.sha256(caddyfile.encode("utf-8")).hexdigest()
|
digest = hashlib.sha256(caddyfile.encode("utf-8")).hexdigest()
|
||||||
|
apply_ok = True
|
||||||
|
apply_status = 0
|
||||||
|
last_error = ""
|
||||||
if digest != last_digest:
|
if digest != last_digest:
|
||||||
status = _post_caddyfile(caddy_load_url, caddyfile)
|
apply_status = _post_caddyfile(caddy_load_url, caddyfile)
|
||||||
_log(f"INFO: applied config (routes={len(routes)}, status={status})")
|
_log(f"INFO: applied config (routes={len(routes)}, status={apply_status})")
|
||||||
last_digest = digest
|
last_digest = digest
|
||||||
else:
|
else:
|
||||||
_log(f"INFO: no config changes (routes={len(routes)})")
|
_log(f"INFO: no config changes (routes={len(routes)})")
|
||||||
|
|
||||||
|
with snapshot_lock:
|
||||||
|
snapshot["last_tick_ts"] = tick_ts
|
||||||
|
snapshot["last_apply_ok"] = apply_ok
|
||||||
|
snapshot["last_apply_http_status"] = apply_status
|
||||||
|
snapshot["last_error"] = last_error
|
||||||
|
snapshot["routes"] = routes
|
||||||
|
snapshot["cloudflare"] = cloudflare_status
|
||||||
|
snapshot["certs"] = cert_rows
|
||||||
except urllib.error.HTTPError as e:
|
except urllib.error.HTTPError as e:
|
||||||
_log(f"ERROR: http failure {e.code} {e.reason}")
|
err = f"http failure {e.code} {e.reason}"
|
||||||
|
_log(f"ERROR: {err}")
|
||||||
|
with snapshot_lock:
|
||||||
|
snapshot["last_tick_ts"] = tick_ts
|
||||||
|
snapshot["last_apply_ok"] = False
|
||||||
|
snapshot["last_apply_http_status"] = 0
|
||||||
|
snapshot["last_error"] = err
|
||||||
except urllib.error.URLError as e:
|
except urllib.error.URLError as e:
|
||||||
_log(f"ERROR: connection failure: {e}")
|
err = f"connection failure: {e}"
|
||||||
|
_log(f"ERROR: {err}")
|
||||||
|
with snapshot_lock:
|
||||||
|
snapshot["last_tick_ts"] = tick_ts
|
||||||
|
snapshot["last_apply_ok"] = False
|
||||||
|
snapshot["last_apply_http_status"] = 0
|
||||||
|
snapshot["last_error"] = err
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
_log(f"ERROR: {e}")
|
err = str(e)
|
||||||
|
_log(f"ERROR: {err}")
|
||||||
|
with snapshot_lock:
|
||||||
|
snapshot["last_tick_ts"] = tick_ts
|
||||||
|
snapshot["last_apply_ok"] = False
|
||||||
|
snapshot["last_apply_http_status"] = 0
|
||||||
|
snapshot["last_error"] = err
|
||||||
|
|
||||||
time.sleep(poll_seconds)
|
time.sleep(poll_seconds)
|
||||||
|
|
||||||
|
|||||||
@@ -2,6 +2,19 @@
|
|||||||
admin {$CADDY_ADMIN}
|
admin {$CADDY_ADMIN}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
:31820 {
|
||||||
|
@allowed remote_ip private_ranges
|
||||||
|
handle @allowed {
|
||||||
|
@status_json path /status.json
|
||||||
|
handle @status_json {
|
||||||
|
reverse_proxy discovery-agent:8089
|
||||||
|
}
|
||||||
|
root * /srv/status
|
||||||
|
file_server
|
||||||
|
}
|
||||||
|
respond "forbidden" 403
|
||||||
|
}
|
||||||
|
|
||||||
:80 {
|
:80 {
|
||||||
respond "caddy-autogen initializing" 200
|
respond "caddy-autogen initializing" 200
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -5,3 +5,4 @@ RUN xcaddy build \
|
|||||||
FROM caddy:2.10.2-alpine
|
FROM caddy:2.10.2-alpine
|
||||||
COPY --from=builder /usr/bin/caddy /usr/bin/caddy
|
COPY --from=builder /usr/bin/caddy /usr/bin/caddy
|
||||||
COPY Caddyfile /etc/caddy/Caddyfile
|
COPY Caddyfile /etc/caddy/Caddyfile
|
||||||
|
COPY status/ /srv/status/
|
||||||
|
|||||||
@@ -0,0 +1,186 @@
|
|||||||
|
<!doctype html>
|
||||||
|
<html lang="en">
|
||||||
|
<head>
|
||||||
|
<meta charset="UTF-8" />
|
||||||
|
<meta name="viewport" content="width=device-width, initial-scale=1" />
|
||||||
|
<title>Caddy AutoGen Status</title>
|
||||||
|
<style>
|
||||||
|
:root {
|
||||||
|
--bg: #0f172a;
|
||||||
|
--panel: #111827;
|
||||||
|
--muted: #9ca3af;
|
||||||
|
--ok: #22c55e;
|
||||||
|
--warn: #f59e0b;
|
||||||
|
--bad: #ef4444;
|
||||||
|
--text: #e5e7eb;
|
||||||
|
}
|
||||||
|
body {
|
||||||
|
margin: 0;
|
||||||
|
font-family: "Iosevka", "Menlo", "Consolas", monospace;
|
||||||
|
background: radial-gradient(circle at top, #1e293b, #0b1020 60%);
|
||||||
|
color: var(--text);
|
||||||
|
}
|
||||||
|
.wrap {
|
||||||
|
max-width: 980px;
|
||||||
|
margin: 0 auto;
|
||||||
|
padding: 24px;
|
||||||
|
}
|
||||||
|
h1 { margin: 0 0 8px; }
|
||||||
|
.subtitle { color: var(--muted); margin-bottom: 16px; }
|
||||||
|
.grid {
|
||||||
|
display: grid;
|
||||||
|
grid-template-columns: repeat(auto-fit, minmax(240px, 1fr));
|
||||||
|
gap: 12px;
|
||||||
|
margin-bottom: 12px;
|
||||||
|
}
|
||||||
|
.card {
|
||||||
|
background: color-mix(in oklab, var(--panel), black 8%);
|
||||||
|
border: 1px solid #1f2937;
|
||||||
|
border-radius: 10px;
|
||||||
|
padding: 12px;
|
||||||
|
}
|
||||||
|
.label { color: var(--muted); font-size: 12px; text-transform: uppercase; letter-spacing: 0.07em; }
|
||||||
|
.value { font-size: 20px; margin-top: 6px; }
|
||||||
|
.ok { color: var(--ok); }
|
||||||
|
.warn { color: var(--warn); }
|
||||||
|
.bad { color: var(--bad); }
|
||||||
|
table {
|
||||||
|
width: 100%;
|
||||||
|
border-collapse: collapse;
|
||||||
|
}
|
||||||
|
th, td {
|
||||||
|
text-align: left;
|
||||||
|
border-bottom: 1px solid #1f2937;
|
||||||
|
padding: 8px 6px;
|
||||||
|
font-size: 14px;
|
||||||
|
}
|
||||||
|
.raw {
|
||||||
|
white-space: pre-wrap;
|
||||||
|
color: var(--muted);
|
||||||
|
font-size: 12px;
|
||||||
|
}
|
||||||
|
.topline {
|
||||||
|
display: flex;
|
||||||
|
justify-content: space-between;
|
||||||
|
align-items: baseline;
|
||||||
|
gap: 8px;
|
||||||
|
}
|
||||||
|
button {
|
||||||
|
background: #1d4ed8;
|
||||||
|
color: white;
|
||||||
|
border: 0;
|
||||||
|
border-radius: 8px;
|
||||||
|
padding: 8px 12px;
|
||||||
|
cursor: pointer;
|
||||||
|
}
|
||||||
|
button:hover { background: #2563eb; }
|
||||||
|
</style>
|
||||||
|
</head>
|
||||||
|
<body>
|
||||||
|
<div class="wrap">
|
||||||
|
<div class="topline">
|
||||||
|
<h1>Caddy AutoGen Status</h1>
|
||||||
|
<button id="refresh">Refresh</button>
|
||||||
|
</div>
|
||||||
|
<div class="subtitle" id="updated">Loading...</div>
|
||||||
|
|
||||||
|
<div class="grid">
|
||||||
|
<div class="card">
|
||||||
|
<div class="label">Caddy Apply</div>
|
||||||
|
<div class="value" id="apply-status">-</div>
|
||||||
|
</div>
|
||||||
|
<div class="card">
|
||||||
|
<div class="label">Cloudflare Reachability</div>
|
||||||
|
<div class="value" id="cf-reachable">-</div>
|
||||||
|
</div>
|
||||||
|
<div class="card">
|
||||||
|
<div class="label">Cloudflare Token</div>
|
||||||
|
<div class="value" id="cf-token">-</div>
|
||||||
|
</div>
|
||||||
|
<div class="card">
|
||||||
|
<div class="label">Routes</div>
|
||||||
|
<div class="value" id="route-count">-</div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div class="card" style="margin-bottom: 12px;">
|
||||||
|
<div class="label">Generated Hosts</div>
|
||||||
|
<table>
|
||||||
|
<thead>
|
||||||
|
<tr>
|
||||||
|
<th>Host</th>
|
||||||
|
<th>Upstream</th>
|
||||||
|
<th>Scheme</th>
|
||||||
|
<th>Path</th>
|
||||||
|
<th>LE Cert</th>
|
||||||
|
</tr>
|
||||||
|
</thead>
|
||||||
|
<tbody id="routes-body"></tbody>
|
||||||
|
</table>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div class="card">
|
||||||
|
<div class="label">Last Error</div>
|
||||||
|
<div id="last-error" class="raw">-</div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<script>
|
||||||
|
function stateText(ok) {
|
||||||
|
return ok ? ["ok", "OK"] : ["bad", "FAIL"];
|
||||||
|
}
|
||||||
|
|
||||||
|
function setEl(id, text, cls) {
|
||||||
|
const el = document.getElementById(id);
|
||||||
|
el.textContent = text;
|
||||||
|
el.classList.remove("ok", "warn", "bad");
|
||||||
|
if (cls) el.classList.add(cls);
|
||||||
|
}
|
||||||
|
|
||||||
|
function fmtTs(ts) {
|
||||||
|
if (!ts) return "-";
|
||||||
|
return new Date(ts * 1000).toLocaleString();
|
||||||
|
}
|
||||||
|
|
||||||
|
async function loadStatus() {
|
||||||
|
const res = await fetch('/status.json', { cache: 'no-store' });
|
||||||
|
if (!res.ok) throw new Error('status fetch failed: ' + res.status);
|
||||||
|
const data = await res.json();
|
||||||
|
|
||||||
|
const applyPair = stateText(Boolean(data.last_apply_ok));
|
||||||
|
setEl('apply-status', applyPair[1] + (data.last_apply_http_status ? ` (${data.last_apply_http_status})` : ''), applyPair[0]);
|
||||||
|
|
||||||
|
const reachablePair = stateText(Boolean(data.cloudflare && data.cloudflare.reachable));
|
||||||
|
setEl('cf-reachable', reachablePair[1], reachablePair[0]);
|
||||||
|
|
||||||
|
const tokenPair = stateText(Boolean(data.cloudflare && data.cloudflare.token_valid));
|
||||||
|
setEl('cf-token', tokenPair[1], tokenPair[0]);
|
||||||
|
|
||||||
|
setEl('route-count', String((data.routes || []).length));
|
||||||
|
setEl('updated', `Last tick: ${fmtTs(data.last_tick_ts)} | Cloudflare check: ${fmtTs(data.cloudflare && data.cloudflare.last_check_ts)}`);
|
||||||
|
|
||||||
|
const certByHost = new Map((data.certs || []).map((c) => [c.fqdn, c.letsencrypt_present]));
|
||||||
|
const rows = (data.routes || []).map((r) => {
|
||||||
|
const cert = certByHost.get(r.fqdn) ? 'yes' : 'no';
|
||||||
|
return `<tr><td>${r.fqdn}</td><td>${r.upstream}</td><td>${r.scheme}</td><td>${r.path}</td><td>${cert}</td></tr>`;
|
||||||
|
});
|
||||||
|
document.getElementById('routes-body').innerHTML = rows.join('') || '<tr><td colspan="5">No routes</td></tr>';
|
||||||
|
|
||||||
|
const err = data.last_error || data.cloudflare?.error || '';
|
||||||
|
document.getElementById('last-error').textContent = err || '(none)';
|
||||||
|
}
|
||||||
|
|
||||||
|
async function refresh() {
|
||||||
|
try {
|
||||||
|
await loadStatus();
|
||||||
|
} catch (err) {
|
||||||
|
setEl('updated', 'Error: ' + err.message, 'bad');
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
document.getElementById('refresh').addEventListener('click', refresh);
|
||||||
|
refresh();
|
||||||
|
setInterval(refresh, 15000);
|
||||||
|
</script>
|
||||||
|
</body>
|
||||||
|
</html>
|
||||||
@@ -10,3 +10,8 @@ poll_seconds: 15
|
|||||||
require_cloudflare: true
|
require_cloudflare: true
|
||||||
allow_internal_tls_fallback: false
|
allow_internal_tls_fallback: false
|
||||||
container_name_denylist: "caddy-autogen,caddy-autogen-discovery,caddy-autogen-socket-proxy"
|
container_name_denylist: "caddy-autogen,caddy-autogen-discovery,caddy-autogen-socket-proxy"
|
||||||
|
status_bind: "0.0.0.0:8089"
|
||||||
|
status_ui_port: 31820
|
||||||
|
status_upstream: "discovery-agent:8089"
|
||||||
|
cf_verify_url: "https://api.cloudflare.com/client/v4/user/tokens/verify"
|
||||||
|
caddy_data_dir: "/caddy-data"
|
||||||
|
|||||||
@@ -2,21 +2,25 @@ name: caddy-autogen
|
|||||||
|
|
||||||
services:
|
services:
|
||||||
caddy:
|
caddy:
|
||||||
|
image: joafri/caddy-autogen-caddy:main
|
||||||
build:
|
build:
|
||||||
context: ./caddy
|
context: ./caddy
|
||||||
dockerfile: Dockerfile
|
dockerfile: Dockerfile
|
||||||
container_name: caddy-autogen
|
container_name: caddy-autogen
|
||||||
restart: unless-stopped
|
restart: unless-stopped
|
||||||
environment:
|
environment:
|
||||||
TZ: ${TZ}
|
TZ: Europe/Stockholm
|
||||||
CADDY_ADMIN: ${CADDY_ADMIN:-0.0.0.0:2019}
|
CADDY_ADMIN: 0.0.0.0:2019
|
||||||
CLOUDFLARE_API_TOKEN: ${CLOUDFLARE_API_TOKEN}
|
CLOUDFLARE_API_TOKEN: CHANGE_ME
|
||||||
ports:
|
ports:
|
||||||
- target: 80
|
- target: 80
|
||||||
published: ${HTTP_PORT:-80}
|
published: 81
|
||||||
protocol: tcp
|
protocol: tcp
|
||||||
- target: 443
|
- target: 443
|
||||||
published: ${HTTPS_PORT:-443}
|
published: 4431
|
||||||
|
protocol: tcp
|
||||||
|
- target: 31820
|
||||||
|
published: 31820
|
||||||
protocol: tcp
|
protocol: tcp
|
||||||
volumes:
|
volumes:
|
||||||
- type: bind
|
- type: bind
|
||||||
@@ -44,6 +48,9 @@ services:
|
|||||||
- container: "443"
|
- container: "443"
|
||||||
description:
|
description:
|
||||||
en_us: HTTPS ingress
|
en_us: HTTPS ingress
|
||||||
|
- container: "31820"
|
||||||
|
description:
|
||||||
|
en_us: Local/LAN status UI
|
||||||
volumes:
|
volumes:
|
||||||
- container: /data
|
- container: /data
|
||||||
description:
|
description:
|
||||||
@@ -53,11 +60,11 @@ services:
|
|||||||
en_us: Caddy configuration state
|
en_us: Caddy configuration state
|
||||||
|
|
||||||
socket-proxy:
|
socket-proxy:
|
||||||
image: lscr.io/linuxserver/socket-proxy:version-24.02.26
|
image: lscr.io/linuxserver/socket-proxy:version-3.0.9-r0
|
||||||
container_name: caddy-autogen-socket-proxy
|
container_name: caddy-autogen-socket-proxy
|
||||||
restart: unless-stopped
|
restart: unless-stopped
|
||||||
environment:
|
environment:
|
||||||
TZ: ${TZ}
|
TZ: Europe/Stockholm
|
||||||
CONTAINERS: 1
|
CONTAINERS: 1
|
||||||
EVENTS: 1
|
EVENTS: 1
|
||||||
INFO: 1
|
INFO: 1
|
||||||
@@ -79,6 +86,7 @@ services:
|
|||||||
- ALL
|
- ALL
|
||||||
|
|
||||||
discovery-agent:
|
discovery-agent:
|
||||||
|
image: joafri/caddy-autogen-discovery-agent:main
|
||||||
build:
|
build:
|
||||||
context: ./agent
|
context: ./agent
|
||||||
dockerfile: Dockerfile
|
dockerfile: Dockerfile
|
||||||
@@ -88,27 +96,36 @@ services:
|
|||||||
- caddy
|
- caddy
|
||||||
- socket-proxy
|
- socket-proxy
|
||||||
environment:
|
environment:
|
||||||
TZ: ${TZ}
|
TZ: Europe/Stockholm
|
||||||
DOCKER_API_URL: ${DOCKER_API_URL:-http://socket-proxy:2375}
|
DOCKER_API_URL: http://socket-proxy:2375
|
||||||
CADDY_LOAD_URL: ${CADDY_LOAD_URL:-http://caddy:2019/load}
|
CADDY_LOAD_URL: http://caddy:2019/load
|
||||||
BASE_DOMAIN: ${BASE_DOMAIN}
|
BASE_DOMAIN: example.com
|
||||||
WILDCARD_DOMAIN: ${WILDCARD_DOMAIN:-}
|
WILDCARD_DOMAIN: ""
|
||||||
CLOUDFLARE_API_TOKEN: ${CLOUDFLARE_API_TOKEN}
|
CLOUDFLARE_API_TOKEN: CLOUDFLARE_API_TOKEN
|
||||||
CERT_EMAIL: ${CERT_EMAIL:-}
|
CERT_EMAIL: ""
|
||||||
REQUIRE_CLOUDFLARE: ${REQUIRE_CLOUDFLARE:-true}
|
REQUIRE_CLOUDFLARE: false
|
||||||
ALLOW_INTERNAL_TLS_FALLBACK: ${ALLOW_INTERNAL_TLS_FALLBACK:-false}
|
ALLOW_INTERNAL_TLS_FALLBACK: false
|
||||||
ENV_PREFIX: ${ENV_PREFIX:-LABEL_CADDY_}
|
ENV_PREFIX: LABEL_CADDY_
|
||||||
POLL_SECONDS: ${POLL_SECONDS:-15}
|
POLL_SECONDS: 15
|
||||||
CONTAINER_NAME_DENYLIST: ${CONTAINER_NAME_DENYLIST:-caddy-autogen,caddy-autogen-discovery,caddy-autogen-socket-proxy}
|
CONTAINER_NAME_DENYLIST: caddy-autogen,caddy-autogen-discovery,caddy-autogen-socket-proxy
|
||||||
DEFAULT_SCHEME: ${DEFAULT_SCHEME:-http}
|
DEFAULT_SCHEME: http
|
||||||
DEFAULT_PATH: ${DEFAULT_PATH:-/}
|
DEFAULT_PATH: /
|
||||||
DEFAULT_HEALTH_URI: ${DEFAULT_HEALTH_URI:-}
|
DEFAULT_HEALTH_URI:
|
||||||
CONFIG_FILE: ${CONFIG_FILE:-/app/config/defaults.yaml}
|
STATUS_BIND: 0.0.0.0:8089
|
||||||
|
STATUS_UI_PORT: 31820
|
||||||
|
STATUS_UPSTREAM: discovery-agent:8089
|
||||||
|
CF_VERIFY_URL: https://api.cloudflare.com/client/v4/user/tokens/verify
|
||||||
|
CADDY_DATA_DIR: /caddy-data
|
||||||
|
CONFIG_FILE: /app/config/defaults.yaml
|
||||||
volumes:
|
volumes:
|
||||||
- type: bind
|
- type: bind
|
||||||
source: /DATA/AppData/$AppID/config
|
source: /DATA/AppData/$AppID/config
|
||||||
target: /app/config
|
target: /app/config
|
||||||
read_only: true
|
read_only: true
|
||||||
|
- type: bind
|
||||||
|
source: /DATA/AppData/$AppID/caddy/data
|
||||||
|
target: /caddy-data
|
||||||
|
read_only: true
|
||||||
read_only: true
|
read_only: true
|
||||||
tmpfs:
|
tmpfs:
|
||||||
- /tmp
|
- /tmp
|
||||||
@@ -144,10 +161,10 @@ x-casaos:
|
|||||||
- arm64
|
- arm64
|
||||||
- arm
|
- arm
|
||||||
main: caddy
|
main: caddy
|
||||||
category: Network
|
category: phirna
|
||||||
author: Zima Apps Team
|
author: Joachim Friberg
|
||||||
developer: Zima Apps Team
|
developer: Joachim Friberg
|
||||||
icon: https://caddyserver.com/resources/images/caddy-circle-lock.svg
|
icon: https://cdn.simpleicons.org/caddy
|
||||||
tagline:
|
tagline:
|
||||||
en_us: Auto-generate Caddy endpoints from running containers
|
en_us: Auto-generate Caddy endpoints from running containers
|
||||||
description:
|
description:
|
||||||
@@ -158,5 +175,5 @@ x-casaos:
|
|||||||
title:
|
title:
|
||||||
en_us: Caddy AutoGen
|
en_us: Caddy AutoGen
|
||||||
index: /
|
index: /
|
||||||
port_map: ${HTTPS_PORT:-443}
|
port_map: "443"
|
||||||
scheme: https
|
scheme: https
|
||||||
|
|||||||
@@ -1,5 +1,7 @@
|
|||||||
#!/usr/bin/env python3
|
#!/usr/bin/env python3
|
||||||
import importlib.util
|
import importlib.util
|
||||||
|
import json
|
||||||
|
import tempfile
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
|
|
||||||
|
|
||||||
@@ -93,8 +95,13 @@ def test_optin_route_selection(module):
|
|||||||
allow_internal_tls_fallback=False,
|
allow_internal_tls_fallback=False,
|
||||||
wildcard_domain="home.example.test",
|
wildcard_domain="home.example.test",
|
||||||
cert_email="",
|
cert_email="",
|
||||||
|
status_ui_port=31820,
|
||||||
|
status_upstream="discovery-agent:8089",
|
||||||
)
|
)
|
||||||
|
|
||||||
|
assert_true(":31820" in caddyfile, "expected status UI server block")
|
||||||
|
assert_true("remote_ip private_ranges" in caddyfile, "expected LAN-only restriction")
|
||||||
|
assert_true("reverse_proxy discovery-agent:8089" in caddyfile, "expected status upstream")
|
||||||
assert_true("frigate.home.example.test" in caddyfile, "expected frigate host in caddyfile")
|
assert_true("frigate.home.example.test" in caddyfile, "expected frigate host in caddyfile")
|
||||||
assert_true("reverse_proxy http://host.docker.internal:5000" in caddyfile, "expected web port route")
|
assert_true("reverse_proxy http://host.docker.internal:5000" in caddyfile, "expected web port route")
|
||||||
assert_true("8554" not in caddyfile and "8555" not in caddyfile, "media ports must not be routed")
|
assert_true("8554" not in caddyfile and "8555" not in caddyfile, "media ports must not be routed")
|
||||||
@@ -122,6 +129,8 @@ def test_fail_closed_and_internal_fallback(module):
|
|||||||
allow_internal_tls_fallback=False,
|
allow_internal_tls_fallback=False,
|
||||||
wildcard_domain="",
|
wildcard_domain="",
|
||||||
cert_email="",
|
cert_email="",
|
||||||
|
status_ui_port=31820,
|
||||||
|
status_upstream="discovery-agent:8089",
|
||||||
)
|
)
|
||||||
except RuntimeError as exc:
|
except RuntimeError as exc:
|
||||||
failed = True
|
failed = True
|
||||||
@@ -135,15 +144,60 @@ def test_fail_closed_and_internal_fallback(module):
|
|||||||
allow_internal_tls_fallback=True,
|
allow_internal_tls_fallback=True,
|
||||||
wildcard_domain="",
|
wildcard_domain="",
|
||||||
cert_email="",
|
cert_email="",
|
||||||
|
status_ui_port=31820,
|
||||||
|
status_upstream="discovery-agent:8089",
|
||||||
)
|
)
|
||||||
assert_true("local_certs" in fallback_caddyfile, "expected local_certs in fallback mode")
|
assert_true("local_certs" in fallback_caddyfile, "expected local_certs in fallback mode")
|
||||||
assert_true("tls internal" in fallback_caddyfile, "expected internal tls in fallback mode")
|
assert_true("tls internal" in fallback_caddyfile, "expected internal tls in fallback mode")
|
||||||
|
|
||||||
|
|
||||||
|
def test_cloudflare_verify_and_cert_discovery(module):
|
||||||
|
class FakeResponse:
|
||||||
|
def __init__(self, payload):
|
||||||
|
self._payload = payload
|
||||||
|
|
||||||
|
def read(self):
|
||||||
|
return json.dumps(self._payload).encode("utf-8")
|
||||||
|
|
||||||
|
def __enter__(self):
|
||||||
|
return self
|
||||||
|
|
||||||
|
def __exit__(self, exc_type, exc, tb):
|
||||||
|
return False
|
||||||
|
|
||||||
|
def fake_urlopen(req, timeout=0):
|
||||||
|
_ = req
|
||||||
|
_ = timeout
|
||||||
|
return FakeResponse({"success": True})
|
||||||
|
|
||||||
|
original_urlopen = module.urllib.request.urlopen
|
||||||
|
module.urllib.request.urlopen = fake_urlopen
|
||||||
|
try:
|
||||||
|
status = module._verify_cloudflare_token("https://api.cloudflare.com/client/v4/user/tokens/verify", "token")
|
||||||
|
finally:
|
||||||
|
module.urllib.request.urlopen = original_urlopen
|
||||||
|
|
||||||
|
assert_true(status["reachable"] is True, "cloudflare should be reachable in mocked success")
|
||||||
|
assert_true(status["token_valid"] is True, "token should be valid in mocked success")
|
||||||
|
|
||||||
|
with tempfile.TemporaryDirectory() as td:
|
||||||
|
cert_dir = Path(td) / "caddy" / "certificates" / "acme-v02.api.letsencrypt.org-directory" / "example.com"
|
||||||
|
cert_dir.mkdir(parents=True, exist_ok=True)
|
||||||
|
(cert_dir / "demo.home.example.test.crt").write_text("fake", encoding="utf-8")
|
||||||
|
(cert_dir / "_.home.example.test.crt").write_text("fake", encoding="utf-8")
|
||||||
|
|
||||||
|
hosts = module._collect_letsencrypt_hosts(td)
|
||||||
|
assert_true("demo.home.example.test" in hosts, "expected concrete cert host")
|
||||||
|
assert_true("*.home.example.test" in hosts, "expected wildcard cert host conversion")
|
||||||
|
assert_true(module._has_matching_le_cert("api.home.example.test", hosts), "wildcard should match")
|
||||||
|
assert_true(module._has_matching_le_cert("demo.home.example.test", hosts), "exact cert should match")
|
||||||
|
|
||||||
|
|
||||||
def main():
|
def main():
|
||||||
module = load_agent_module()
|
module = load_agent_module()
|
||||||
test_optin_route_selection(module)
|
test_optin_route_selection(module)
|
||||||
test_fail_closed_and_internal_fallback(module)
|
test_fail_closed_and_internal_fallback(module)
|
||||||
|
test_cloudflare_verify_and_cert_discovery(module)
|
||||||
print("Integration tests passed")
|
print("Integration tests passed")
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,234 @@
|
|||||||
|
# HOW_TO_VERIFY
|
||||||
|
|
||||||
|
Detta dokument verifierar att `docker-ip-addr-manager` fungerar säkert i ZimaOS.
|
||||||
|
|
||||||
|
## 1) Förutsättningar
|
||||||
|
|
||||||
|
- ZimaOS-host med minst ett interface (ex `eth0`).
|
||||||
|
- Docker körs på hosten.
|
||||||
|
- Appen är installerad och startad.
|
||||||
|
- Webb-UI/API nås på `${APP_PORT:-31810}`.
|
||||||
|
|
||||||
|
Kontrollera att appen är uppe:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl -fsS http://127.0.0.1:31810/healthz
|
||||||
|
```
|
||||||
|
|
||||||
|
Förväntat resultat:
|
||||||
|
|
||||||
|
- JSON med `{"ok":true}`.
|
||||||
|
|
||||||
|
## 2) Positiva testfall
|
||||||
|
|
||||||
|
### Test A: Skapa och enable IP-post
|
||||||
|
|
||||||
|
Skapa post:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl -fsS -X POST http://127.0.0.1:31810/api/entries \
|
||||||
|
-H 'Content-Type: application/json' \
|
||||||
|
-d '{"name":"lan-test","ip":"10.0.4.2","cidr":16,"device":"eth0"}'
|
||||||
|
```
|
||||||
|
|
||||||
|
Hämta `id` från svaret och enable:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
ENTRY_ID="<id-fran-svaret>"
|
||||||
|
curl -fsS -X POST "http://127.0.0.1:31810/api/entries/${ENTRY_ID}/enable"
|
||||||
|
```
|
||||||
|
|
||||||
|
Verifiera på host:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
ip -4 -o addr show dev eth0 | rg '10\.0\.4\.2/16'
|
||||||
|
```
|
||||||
|
|
||||||
|
Förväntat resultat:
|
||||||
|
|
||||||
|
- IP-adressen finns på `eth0`.
|
||||||
|
|
||||||
|
### Test B: Used-detektion via Docker port binding
|
||||||
|
|
||||||
|
Starta testcontainer bunden till IP:n:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker run -d --rm --name ip-verify-nginx -p 10.0.4.2:18080:80 nginx:1.27.5
|
||||||
|
```
|
||||||
|
|
||||||
|
Refresh appdata:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl -fsS -X POST http://127.0.0.1:31810/api/refresh
|
||||||
|
```
|
||||||
|
|
||||||
|
Förväntat resultat:
|
||||||
|
|
||||||
|
- posten med `10.0.4.2` har `used=true`.
|
||||||
|
- `containers` innehåller `ip-verify-nginx`.
|
||||||
|
|
||||||
|
### Test C: DNS create när posten är enabled + used
|
||||||
|
|
||||||
|
Förutsätter DNS config i appen, exempel för AdGuard:
|
||||||
|
|
||||||
|
- `DNS_PROVIDER=adguard`
|
||||||
|
- `DNS_BASE_DOMAIN=home.arpa`
|
||||||
|
- `ADGUARD_URL=http://<adguard-ip>:3000`
|
||||||
|
- `ADGUARD_USERNAME=<user>`
|
||||||
|
- `ADGUARD_PASSWORD=<password>`
|
||||||
|
|
||||||
|
Verifiera att record skapats:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
dig +short lan-test.home.arpa @<adguard-ip>
|
||||||
|
```
|
||||||
|
|
||||||
|
Förväntat resultat:
|
||||||
|
|
||||||
|
- returnerar `10.0.4.2`.
|
||||||
|
|
||||||
|
### Test D: Disable/Delete efter frigöring
|
||||||
|
|
||||||
|
Stoppa testcontainer:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker stop ip-verify-nginx
|
||||||
|
```
|
||||||
|
|
||||||
|
Disable posten:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl -fsS -X POST "http://127.0.0.1:31810/api/entries/${ENTRY_ID}/disable"
|
||||||
|
```
|
||||||
|
|
||||||
|
Verifiera borttagen IP:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
ip -4 -o addr show dev eth0 | rg '10\.0\.4\.2/16' || true
|
||||||
|
```
|
||||||
|
|
||||||
|
Förväntat resultat:
|
||||||
|
|
||||||
|
- ingen träff för `10.0.4.2/16`.
|
||||||
|
|
||||||
|
Ta bort posten:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl -fsS -X DELETE "http://127.0.0.1:31810/api/entries/${ENTRY_ID}"
|
||||||
|
```
|
||||||
|
|
||||||
|
Förväntat resultat:
|
||||||
|
|
||||||
|
- svar med `{"deleted":true}`.
|
||||||
|
|
||||||
|
## 3) Negativt / fail-closed testfall
|
||||||
|
|
||||||
|
### Test E: Blockera disable när IP används
|
||||||
|
|
||||||
|
1. Skapa + enable som i Test A.
|
||||||
|
2. Starta container som i Test B.
|
||||||
|
3. Försök disable:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl -sS -o /tmp/disable.out -w '%{http_code}\n' \
|
||||||
|
-X POST "http://127.0.0.1:31810/api/entries/${ENTRY_ID}/disable"
|
||||||
|
cat /tmp/disable.out
|
||||||
|
```
|
||||||
|
|
||||||
|
Förväntat resultat:
|
||||||
|
|
||||||
|
- HTTP `409`.
|
||||||
|
- feltext som anger att posten används av container.
|
||||||
|
|
||||||
|
### Test F: Fail-closed vid DNS-fel
|
||||||
|
|
||||||
|
1. Se till att en post är `enabled` och `used` (Test A+B).
|
||||||
|
2. Sabotera DNS-auth tillfälligt, exempel:
|
||||||
|
- ändra `ADGUARD_PASSWORD` till fel värde och starta om appen.
|
||||||
|
3. Försök disable/delete på posten.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl -sS -o /tmp/dns-fail.out -w '%{http_code}\n' \
|
||||||
|
-X POST "http://127.0.0.1:31810/api/entries/${ENTRY_ID}/disable"
|
||||||
|
cat /tmp/dns-fail.out
|
||||||
|
```
|
||||||
|
|
||||||
|
Förväntat resultat:
|
||||||
|
|
||||||
|
- HTTP `409` eller `503`.
|
||||||
|
- feltext som indikerar DNS-synkfel.
|
||||||
|
- posten ska inte lämna systemet i delvis uppdaterat läge.
|
||||||
|
|
||||||
|
## 4) DNS / nät / TLS verifiering
|
||||||
|
|
||||||
|
### DNS (om hostname används i LAN)
|
||||||
|
|
||||||
|
```bash
|
||||||
|
DNS_SERVER="<dns-server-ip>"
|
||||||
|
HOSTNAME_TO_TEST="lan-test.home.arpa"
|
||||||
|
dig +short "${HOSTNAME_TO_TEST}" @"${DNS_SERVER}"
|
||||||
|
```
|
||||||
|
|
||||||
|
Förväntat resultat:
|
||||||
|
|
||||||
|
- returnerar avsedd LAN-IP när posten är `enabled && used`.
|
||||||
|
- ingen träff när posten inte längre är `used` eller är `disabled`.
|
||||||
|
|
||||||
|
### Nätverk (lyssning och routning)
|
||||||
|
|
||||||
|
```bash
|
||||||
|
ss -ltnp | rg ':31810'
|
||||||
|
ip route get 10.0.4.2
|
||||||
|
```
|
||||||
|
|
||||||
|
Förväntat resultat:
|
||||||
|
|
||||||
|
- appen lyssnar på `31810`.
|
||||||
|
- route lookup fungerar mot rätt interface.
|
||||||
|
|
||||||
|
### TLS
|
||||||
|
|
||||||
|
Appens UI/API i v1 kör HTTP, inte HTTPS.
|
||||||
|
|
||||||
|
Verifiera att HTTP fungerar:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl -fsS http://127.0.0.1:31810/healthz
|
||||||
|
```
|
||||||
|
|
||||||
|
Verifiera att HTTPS mot app-porten inte ska användas:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl -k https://127.0.0.1:31810/healthz || true
|
||||||
|
```
|
||||||
|
|
||||||
|
Förväntat resultat:
|
||||||
|
|
||||||
|
- HTTP fungerar.
|
||||||
|
- HTTPS-anrop misslyckas eller ger ogiltigt TLS-handslag.
|
||||||
|
|
||||||
|
## 5) Data att samla (för snabb felsökning)
|
||||||
|
|
||||||
|
- Versions-/buildinfo:
|
||||||
|
- app-id: `docker-ip-addr-manager`
|
||||||
|
- branch/commit eller zip + checksum.
|
||||||
|
- Relevant konfiguration (maska secrets):
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker inspect docker-ip-addr-manager | jq '.[0].Config.Env'
|
||||||
|
```
|
||||||
|
|
||||||
|
- Loggar från berörda containers:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker logs --tail 200 docker-ip-addr-manager
|
||||||
|
|
||||||
|
docker logs --tail 200 docker-ip-addr-manager-proxy
|
||||||
|
```
|
||||||
|
|
||||||
|
- Konkreta felobservationer:
|
||||||
|
- hostname/IP,
|
||||||
|
- exakt tidpunkt,
|
||||||
|
- förväntat beteende,
|
||||||
|
- faktiskt beteende,
|
||||||
|
- exakta kommandon som kördes.
|
||||||
@@ -0,0 +1,138 @@
|
|||||||
|
# Docker IP Addr Manager
|
||||||
|
|
||||||
|
Docker IP Addr Manager ger ett enkelt GUI i ZimaOS för att hantera host-IP-aliaser som används i portbindings (`IP:PORT`).
|
||||||
|
|
||||||
|
Exempel: istället för att köra `ip addr add 10.0.4.2/16 dev eth0` via SSH, kan du skapa en post i GUI och aktivera den.
|
||||||
|
|
||||||
|
## Funktioner (v1)
|
||||||
|
|
||||||
|
- CRUD för IP-poster: `name`, `ip`, `cidr`, `device`.
|
||||||
|
- Enable/disable per post:
|
||||||
|
- enable => `ip addr add <ip>/<cidr> dev <device>`
|
||||||
|
- disable => `ip addr del <ip>/<cidr> dev <device>`
|
||||||
|
- Sorterbar tabell: namn, IP-adress, used/unused, containernamn, device, enable/disable.
|
||||||
|
- Used/unused-kontroll via Docker API (`NetworkSettings.Ports`) med exakt `HostIp`-match.
|
||||||
|
- Include stopped containers i used-kontroll.
|
||||||
|
- DNS-livscykel (opt-in): skapar A-record när `enabled=true` och `used=true`, tar bort record när villkoret inte längre gäller.
|
||||||
|
- DNS-namn byggs från `name` + `DNS_BASE_DOMAIN` => `<name>.<base-domain>` (DNS-säkrad label).
|
||||||
|
- Fail-closed:
|
||||||
|
- disable blockeras om IP används av minst en container,
|
||||||
|
- delete blockeras om posten är enabled eller used,
|
||||||
|
- disable/delete blockeras om Docker-usage inte kan verifieras,
|
||||||
|
- state-ändringar blockeras om nödvändig DNS-synk misslyckas.
|
||||||
|
- Startup reconcile: enabled-poster återappliceras vid appstart.
|
||||||
|
- DNS reconcile körs i bakgrunden med poll-interval.
|
||||||
|
- Manuell refresh-knapp för UI-status (ingen websocket i v1).
|
||||||
|
|
||||||
|
## Portar
|
||||||
|
|
||||||
|
- `${APP_PORT:-31810}`: webbgränssnitt/API.
|
||||||
|
- `127.0.0.1:2375` (socket-proxy, valfritt): lokal Docker TCP proxy för alternativ endpoint.
|
||||||
|
|
||||||
|
## Volymer
|
||||||
|
|
||||||
|
- `/DATA/AppData/$AppID/data:/data`
|
||||||
|
- persistent lagring av IP-poster.
|
||||||
|
- `/var/run/docker.sock:/var/run/docker.sock:ro`
|
||||||
|
- Docker metadata för used/unused-kontroll.
|
||||||
|
|
||||||
|
## Privilegier
|
||||||
|
|
||||||
|
- `network_mode: host`
|
||||||
|
- `cap_add: [NET_ADMIN]`
|
||||||
|
- `security_opt: ["no-new-privileges:true"]`
|
||||||
|
|
||||||
|
## Säkerhetsavvikelser
|
||||||
|
|
||||||
|
Denna app använder högrisk-inställningar och de är avsiktliga:
|
||||||
|
|
||||||
|
- `network_mode: host`
|
||||||
|
- mount av `/var/run/docker.sock`
|
||||||
|
|
||||||
|
### Varför det behövs
|
||||||
|
|
||||||
|
- `ip addr add/del` måste köras i hostens nätverksnamespace för att påverka hostens interface (ex. `eth0`).
|
||||||
|
- Used/unused-status behöver läsas från Docker container metadata.
|
||||||
|
|
||||||
|
### Alternativ som utvärderats
|
||||||
|
|
||||||
|
- **Extern host-agent utanför Docker**: bättre isolering men kräver separat installation utanför appstore.
|
||||||
|
- **nsenter-helper**: mer komplex driftsmodell och högre implementationsoverhead för v1.
|
||||||
|
- **Endast Docker TCP endpoint utan socket**: stöds, men default är unix-socket för enklare baseline i ZimaOS.
|
||||||
|
|
||||||
|
### Risker
|
||||||
|
|
||||||
|
- Host network + `NET_ADMIN` kan påverka hostens nätverk direkt vid felaktig konfiguration.
|
||||||
|
- Docker socket-access innebär insyn i container-metadata och behöver hanteras med strikt kontroll.
|
||||||
|
|
||||||
|
### Riskreducering
|
||||||
|
|
||||||
|
- Minsta capability för nätverksoperationer (`NET_ADMIN`), inte `privileged: true`.
|
||||||
|
- `no-new-privileges:true`.
|
||||||
|
- Fail-closed blockering vid osäker Docker-usage.
|
||||||
|
- Valbar socket-proxy med read-only Docker socket och endpoint-begränsning.
|
||||||
|
|
||||||
|
## Konfiguration
|
||||||
|
|
||||||
|
Viktiga environment-variabler:
|
||||||
|
|
||||||
|
- `APP_PORT` (default `31810`)
|
||||||
|
- `DOCKER_API_URL` (default `unix:///var/run/docker.sock`)
|
||||||
|
- alternativt `http://127.0.0.1:2375` för socket-proxy.
|
||||||
|
- `DOCKER_TIMEOUT_SECONDS` (default `3`)
|
||||||
|
- `STATE_FILE` (default `/data/entries.json`)
|
||||||
|
- `DNS_PROVIDER` (`none`, `adguard`, `rfc2136`; default `none`)
|
||||||
|
- `DNS_BASE_DOMAIN` (exempel: `home.arpa`)
|
||||||
|
- `DNS_TTL_SECONDS` (default `120`)
|
||||||
|
- `DNS_SYNC_INTERVAL_SECONDS` (default `15`)
|
||||||
|
|
||||||
|
AdGuard (`DNS_PROVIDER=adguard`):
|
||||||
|
|
||||||
|
- `ADGUARD_URL` (exempel: `http://127.0.0.1:3000`)
|
||||||
|
- `ADGUARD_USERNAME`
|
||||||
|
- `ADGUARD_PASSWORD`
|
||||||
|
- `ADGUARD_API_TOKEN` (framtida alternativ, inte aktiv auth-väg i v1)
|
||||||
|
|
||||||
|
RFC2136 (`DNS_PROVIDER=rfc2136`):
|
||||||
|
|
||||||
|
- `RFC2136_SERVER`
|
||||||
|
- `RFC2136_ZONE`
|
||||||
|
- `RFC2136_PORT` (default `53`)
|
||||||
|
- `RFC2136_TSIG_KEY_NAME` (valfri om osignerade updates tillåts)
|
||||||
|
- `RFC2136_TSIG_SECRET` (base64, valfri utan TSIG)
|
||||||
|
- `RFC2136_TSIG_ALGORITHM` (default `hmac-sha256`)
|
||||||
|
|
||||||
|
## DNS-beteende
|
||||||
|
|
||||||
|
- Villkor för record: endast när posten är `enabled` och `used`.
|
||||||
|
- När posten inte längre är `used` tas DNS-record bort i bakgrundsreconcile.
|
||||||
|
- Vid enable/disable/delete görs direkt DNS-synk och operationen failar vid synkfel (fail-closed).
|
||||||
|
- Om Docker usage-kontroll är okänd i bakgrundsloop görs inga DNS-mutationer i den cykeln.
|
||||||
|
|
||||||
|
## Integrationstester
|
||||||
|
|
||||||
|
Kör:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
./Apps/docker-ip-addr-manager/tests/run_integration_tests.sh
|
||||||
|
```
|
||||||
|
|
||||||
|
Testerna mockar Docker API och `ip`-kommandoflöde och verifierar:
|
||||||
|
|
||||||
|
- exakt `HostIp`-matchning,
|
||||||
|
- fail-closed disable/delete,
|
||||||
|
- blockering vid enabled/used,
|
||||||
|
- startup reconcile av enabled-poster,
|
||||||
|
- DNS create/delete på `enabled && used`,
|
||||||
|
- fail-closed rollback vid DNS-synkfel.
|
||||||
|
|
||||||
|
## Auth-notis
|
||||||
|
|
||||||
|
Ingen auth ingår i v1.
|
||||||
|
|
||||||
|
Auth/autorisering ska implementeras i en senare version och är en uttalad roadmap-punkt.
|
||||||
|
|
||||||
|
## Roadmap (ej v1)
|
||||||
|
|
||||||
|
- WebSocket-baserad live-uppdatering av used-status.
|
||||||
|
- Alternativ auth för AdGuard via API-token.
|
||||||
@@ -0,0 +1,19 @@
|
|||||||
|
FROM python:3.12.9-slim
|
||||||
|
|
||||||
|
ENV PYTHONDONTWRITEBYTECODE=1 \
|
||||||
|
PYTHONUNBUFFERED=1
|
||||||
|
|
||||||
|
WORKDIR /app
|
||||||
|
|
||||||
|
RUN apt-get update \
|
||||||
|
&& apt-get install -y --no-install-recommends iproute2 \
|
||||||
|
&& rm -rf /var/lib/apt/lists/*
|
||||||
|
|
||||||
|
COPY requirements.txt /tmp/requirements.txt
|
||||||
|
RUN pip install --no-cache-dir -r /tmp/requirements.txt
|
||||||
|
|
||||||
|
COPY app /app/app
|
||||||
|
|
||||||
|
EXPOSE 31810
|
||||||
|
|
||||||
|
CMD ["sh", "-c", "uvicorn app.main:app --host 0.0.0.0 --port ${APP_PORT:-31810}"]
|
||||||
@@ -0,0 +1 @@
|
|||||||
|
"""docker-ip-addr-manager backend package."""
|
||||||
@@ -0,0 +1,49 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from dataclasses import dataclass
|
||||||
|
import os
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class Settings:
|
||||||
|
state_file: str
|
||||||
|
docker_api_url: str
|
||||||
|
docker_timeout_seconds: float
|
||||||
|
app_port: int
|
||||||
|
dns_provider: str
|
||||||
|
dns_base_domain: str
|
||||||
|
dns_ttl_seconds: int
|
||||||
|
dns_sync_interval_seconds: float
|
||||||
|
adguard_url: str
|
||||||
|
adguard_username: str
|
||||||
|
adguard_password: str
|
||||||
|
adguard_api_token: str
|
||||||
|
rfc2136_server: str
|
||||||
|
rfc2136_zone: str
|
||||||
|
rfc2136_port: int
|
||||||
|
rfc2136_tsig_key_name: str
|
||||||
|
rfc2136_tsig_secret: str
|
||||||
|
rfc2136_tsig_algorithm: str
|
||||||
|
|
||||||
|
|
||||||
|
def get_settings() -> Settings:
|
||||||
|
return Settings(
|
||||||
|
state_file=os.getenv("STATE_FILE", "/data/entries.json"),
|
||||||
|
docker_api_url=os.getenv("DOCKER_API_URL", "unix:///var/run/docker.sock"),
|
||||||
|
docker_timeout_seconds=float(os.getenv("DOCKER_TIMEOUT_SECONDS", "3")),
|
||||||
|
app_port=int(os.getenv("APP_PORT", "31810")),
|
||||||
|
dns_provider=os.getenv("DNS_PROVIDER", "none"),
|
||||||
|
dns_base_domain=os.getenv("DNS_BASE_DOMAIN", ""),
|
||||||
|
dns_ttl_seconds=int(os.getenv("DNS_TTL_SECONDS", "120")),
|
||||||
|
dns_sync_interval_seconds=float(os.getenv("DNS_SYNC_INTERVAL_SECONDS", "15")),
|
||||||
|
adguard_url=os.getenv("ADGUARD_URL", ""),
|
||||||
|
adguard_username=os.getenv("ADGUARD_USERNAME", ""),
|
||||||
|
adguard_password=os.getenv("ADGUARD_PASSWORD", ""),
|
||||||
|
adguard_api_token=os.getenv("ADGUARD_API_TOKEN", ""),
|
||||||
|
rfc2136_server=os.getenv("RFC2136_SERVER", ""),
|
||||||
|
rfc2136_zone=os.getenv("RFC2136_ZONE", ""),
|
||||||
|
rfc2136_port=int(os.getenv("RFC2136_PORT", "53")),
|
||||||
|
rfc2136_tsig_key_name=os.getenv("RFC2136_TSIG_KEY_NAME", ""),
|
||||||
|
rfc2136_tsig_secret=os.getenv("RFC2136_TSIG_SECRET", ""),
|
||||||
|
rfc2136_tsig_algorithm=os.getenv("RFC2136_TSIG_ALGORITHM", "hmac-sha256"),
|
||||||
|
)
|
||||||
@@ -0,0 +1,309 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from dataclasses import dataclass
|
||||||
|
import base64
|
||||||
|
import http.client
|
||||||
|
import json
|
||||||
|
from typing import Protocol
|
||||||
|
from urllib.parse import urlparse
|
||||||
|
|
||||||
|
|
||||||
|
class DnsSyncError(RuntimeError):
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
class DnsProvider(Protocol):
|
||||||
|
def upsert_a_record(self, fqdn: str, ip: str, ttl: int) -> None:
|
||||||
|
raise NotImplementedError
|
||||||
|
|
||||||
|
def delete_a_record(self, fqdn: str) -> None:
|
||||||
|
raise NotImplementedError
|
||||||
|
|
||||||
|
|
||||||
|
def to_fqdn(entry_name: str, base_domain: str) -> str:
|
||||||
|
label = _sanitize_label(entry_name)
|
||||||
|
domain = base_domain.strip().lower().strip(".")
|
||||||
|
if not domain:
|
||||||
|
raise DnsSyncError("DNS_BASE_DOMAIN is required when DNS is enabled")
|
||||||
|
return f"{label}.{domain}"
|
||||||
|
|
||||||
|
|
||||||
|
def _sanitize_label(value: str) -> str:
|
||||||
|
source = value.strip().lower()
|
||||||
|
if not source:
|
||||||
|
raise DnsSyncError("Entry name is required to create DNS record")
|
||||||
|
|
||||||
|
cleaned: list[str] = []
|
||||||
|
prev_dash = False
|
||||||
|
for ch in source:
|
||||||
|
if "a" <= ch <= "z" or "0" <= ch <= "9":
|
||||||
|
cleaned.append(ch)
|
||||||
|
prev_dash = False
|
||||||
|
continue
|
||||||
|
if ch in {" ", "_", "-"} and not prev_dash:
|
||||||
|
cleaned.append("-")
|
||||||
|
prev_dash = True
|
||||||
|
|
||||||
|
label = "".join(cleaned).strip("-")
|
||||||
|
if not label:
|
||||||
|
raise DnsSyncError(f"Entry name cannot produce DNS-safe label: {value!r}")
|
||||||
|
if len(label) > 63:
|
||||||
|
raise DnsSyncError("DNS label derived from entry name is too long (max 63)")
|
||||||
|
return label
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class AdguardConfig:
|
||||||
|
url: str
|
||||||
|
username: str
|
||||||
|
password: str
|
||||||
|
timeout_seconds: float
|
||||||
|
|
||||||
|
|
||||||
|
class AdguardDnsProvider:
|
||||||
|
def __init__(self, config: AdguardConfig):
|
||||||
|
parsed = urlparse(config.url)
|
||||||
|
if parsed.scheme not in {"http", "https"}:
|
||||||
|
raise ValueError("ADGUARD_URL must use http or https")
|
||||||
|
if not parsed.netloc:
|
||||||
|
raise ValueError("ADGUARD_URL must include host")
|
||||||
|
|
||||||
|
self._https = parsed.scheme == "https"
|
||||||
|
self._host = parsed.hostname or "localhost"
|
||||||
|
self._port = parsed.port
|
||||||
|
self._base_path = parsed.path.rstrip("/")
|
||||||
|
self._username = config.username
|
||||||
|
self._password = config.password
|
||||||
|
self._timeout = config.timeout_seconds
|
||||||
|
self._session_cookie: str | None = None
|
||||||
|
|
||||||
|
def upsert_a_record(self, fqdn: str, ip: str, ttl: int) -> None:
|
||||||
|
del ttl # AdGuard rewrite records do not expose TTL controls.
|
||||||
|
rewrites = self._list_rewrites()
|
||||||
|
for item in rewrites:
|
||||||
|
if item.get("domain") == fqdn and item.get("answer") == ip:
|
||||||
|
return
|
||||||
|
if item.get("domain") == fqdn and item.get("answer") != ip:
|
||||||
|
self._request("POST", "/control/rewrite/delete", {"domain": fqdn, "answer": item.get("answer", "")})
|
||||||
|
self._request("POST", "/control/rewrite/add", {"domain": fqdn, "answer": ip})
|
||||||
|
|
||||||
|
def delete_a_record(self, fqdn: str) -> None:
|
||||||
|
rewrites = self._list_rewrites()
|
||||||
|
for item in rewrites:
|
||||||
|
if item.get("domain") != fqdn:
|
||||||
|
continue
|
||||||
|
self._request("POST", "/control/rewrite/delete", {"domain": fqdn, "answer": item.get("answer", "")})
|
||||||
|
|
||||||
|
def _list_rewrites(self) -> list[dict]:
|
||||||
|
payload = self._request("GET", "/control/rewrite/list", None)
|
||||||
|
if not isinstance(payload, list):
|
||||||
|
raise DnsSyncError("AdGuard returned unexpected rewrite list format")
|
||||||
|
output: list[dict] = []
|
||||||
|
for item in payload:
|
||||||
|
if isinstance(item, dict):
|
||||||
|
output.append(item)
|
||||||
|
return output
|
||||||
|
|
||||||
|
def _request(self, method: str, path: str, payload: dict | None) -> object:
|
||||||
|
if self._session_cookie is None:
|
||||||
|
self._login()
|
||||||
|
return self._request_with_session(method, path, payload, retry_on_auth=True)
|
||||||
|
|
||||||
|
def _login(self) -> None:
|
||||||
|
body = {"name": self._username, "password": self._password}
|
||||||
|
payload, headers = self._raw_request("POST", "/control/login", body, include_auth=False)
|
||||||
|
if headers is None:
|
||||||
|
raise DnsSyncError("AdGuard login failed: missing response headers")
|
||||||
|
cookie = headers.get("set-cookie", "")
|
||||||
|
session = ""
|
||||||
|
for piece in cookie.split(";"):
|
||||||
|
piece = piece.strip()
|
||||||
|
if piece.startswith("agh_session="):
|
||||||
|
session = piece
|
||||||
|
break
|
||||||
|
if not session:
|
||||||
|
raise DnsSyncError("AdGuard login failed: no agh_session cookie")
|
||||||
|
self._session_cookie = session
|
||||||
|
del payload
|
||||||
|
|
||||||
|
def _request_with_session(self, method: str, path: str, payload: dict | None, retry_on_auth: bool) -> object:
|
||||||
|
body, _ = self._raw_request(method, path, payload, include_auth=True)
|
||||||
|
if isinstance(body, dict) and body.get("message") == "unauthorized":
|
||||||
|
if retry_on_auth:
|
||||||
|
self._session_cookie = None
|
||||||
|
self._login()
|
||||||
|
return self._request_with_session(method, path, payload, retry_on_auth=False)
|
||||||
|
raise DnsSyncError("AdGuard request unauthorized")
|
||||||
|
return body
|
||||||
|
|
||||||
|
def _raw_request(
|
||||||
|
self, method: str, path: str, payload: dict | None, include_auth: bool
|
||||||
|
) -> tuple[object, dict[str, str] | None]:
|
||||||
|
conn: http.client.HTTPConnection | http.client.HTTPSConnection
|
||||||
|
if self._https:
|
||||||
|
conn = http.client.HTTPSConnection(self._host, self._port, timeout=self._timeout)
|
||||||
|
else:
|
||||||
|
conn = http.client.HTTPConnection(self._host, self._port, timeout=self._timeout)
|
||||||
|
|
||||||
|
request_path = f"{self._base_path}{path}"
|
||||||
|
raw = ""
|
||||||
|
headers = {"Content-Type": "application/json"}
|
||||||
|
if include_auth and self._session_cookie:
|
||||||
|
headers["Cookie"] = self._session_cookie
|
||||||
|
if payload is not None:
|
||||||
|
raw = json.dumps(payload)
|
||||||
|
|
||||||
|
try:
|
||||||
|
conn.request(method, request_path, body=raw, headers=headers)
|
||||||
|
response = conn.getresponse()
|
||||||
|
body_text = response.read().decode("utf-8", errors="replace")
|
||||||
|
response_headers = {k.lower(): v for k, v in response.getheaders()}
|
||||||
|
except OSError as exc:
|
||||||
|
raise DnsSyncError(f"AdGuard request failed for {path}: {exc}") from exc
|
||||||
|
finally:
|
||||||
|
conn.close()
|
||||||
|
|
||||||
|
if response.status < 200 or response.status >= 300:
|
||||||
|
raise DnsSyncError(
|
||||||
|
f"AdGuard request failed for {path}: HTTP {response.status} {response.reason}; body={body_text[:400]}"
|
||||||
|
)
|
||||||
|
|
||||||
|
if not body_text.strip():
|
||||||
|
return {}, response_headers
|
||||||
|
try:
|
||||||
|
return json.loads(body_text), response_headers
|
||||||
|
except json.JSONDecodeError:
|
||||||
|
return body_text, response_headers
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class Rfc2136Config:
|
||||||
|
server: str
|
||||||
|
zone: str
|
||||||
|
port: int
|
||||||
|
timeout_seconds: float
|
||||||
|
tsig_key_name: str
|
||||||
|
tsig_secret: str
|
||||||
|
tsig_algorithm: str
|
||||||
|
|
||||||
|
|
||||||
|
class Rfc2136DnsProvider:
|
||||||
|
def __init__(self, config: Rfc2136Config):
|
||||||
|
if not config.server.strip():
|
||||||
|
raise ValueError("RFC2136_SERVER is required")
|
||||||
|
if not config.zone.strip():
|
||||||
|
raise ValueError("RFC2136_ZONE is required")
|
||||||
|
|
||||||
|
self._server = config.server.strip()
|
||||||
|
self._zone = config.zone.strip().rstrip(".")
|
||||||
|
self._port = config.port
|
||||||
|
self._timeout = config.timeout_seconds
|
||||||
|
self._key_name = config.tsig_key_name.strip()
|
||||||
|
self._secret = config.tsig_secret.strip()
|
||||||
|
self._algorithm = config.tsig_algorithm.strip() or "hmac-sha256"
|
||||||
|
|
||||||
|
def upsert_a_record(self, fqdn: str, ip: str, ttl: int) -> None:
|
||||||
|
rcode, tsigkeyring, update, query = self._dns_modules()
|
||||||
|
zone_text = self._zone_with_dot()
|
||||||
|
keyring = self._keyring_or_none(tsigkeyring)
|
||||||
|
target = self._absolute_name(fqdn)
|
||||||
|
try:
|
||||||
|
req = update.Update(zone_text, keyring=keyring, keyname=self._key_name or None, keyalgorithm=self._algorithm)
|
||||||
|
req.delete(target, "A")
|
||||||
|
req.add(target, int(ttl), "A", ip)
|
||||||
|
response = query.tcp(req, self._server, port=self._port, timeout=self._timeout)
|
||||||
|
except Exception as exc: # noqa: BLE001
|
||||||
|
raise DnsSyncError(f"RFC2136 upsert failed for {fqdn} -> {ip}: {exc}") from exc
|
||||||
|
if response.rcode() != rcode.NOERROR:
|
||||||
|
text = rcode.to_text(response.rcode())
|
||||||
|
raise DnsSyncError(f"RFC2136 upsert failed for {fqdn}: {text}")
|
||||||
|
|
||||||
|
def delete_a_record(self, fqdn: str) -> None:
|
||||||
|
rcode, tsigkeyring, update, query = self._dns_modules()
|
||||||
|
zone_text = self._zone_with_dot()
|
||||||
|
keyring = self._keyring_or_none(tsigkeyring)
|
||||||
|
target = self._absolute_name(fqdn)
|
||||||
|
try:
|
||||||
|
req = update.Update(zone_text, keyring=keyring, keyname=self._key_name or None, keyalgorithm=self._algorithm)
|
||||||
|
req.delete(target, "A")
|
||||||
|
response = query.tcp(req, self._server, port=self._port, timeout=self._timeout)
|
||||||
|
except Exception as exc: # noqa: BLE001
|
||||||
|
raise DnsSyncError(f"RFC2136 delete failed for {fqdn}: {exc}") from exc
|
||||||
|
if response.rcode() != rcode.NOERROR:
|
||||||
|
text = rcode.to_text(response.rcode())
|
||||||
|
raise DnsSyncError(f"RFC2136 delete failed for {fqdn}: {text}")
|
||||||
|
|
||||||
|
def _dns_modules(self):
|
||||||
|
try:
|
||||||
|
import dns.query as query
|
||||||
|
import dns.rcode as rcode
|
||||||
|
import dns.tsigkeyring as tsigkeyring
|
||||||
|
import dns.update as update
|
||||||
|
except ImportError as exc:
|
||||||
|
raise DnsSyncError("dnspython is required for RFC2136 mode") from exc
|
||||||
|
return rcode, tsigkeyring, update, query
|
||||||
|
|
||||||
|
def _keyring_or_none(self, tsigkeyring):
|
||||||
|
if not self._key_name and not self._secret:
|
||||||
|
return None
|
||||||
|
if not self._key_name or not self._secret:
|
||||||
|
raise DnsSyncError("RFC2136 TSIG requires both key name and secret")
|
||||||
|
key_name = self._key_name if self._key_name.endswith(".") else f"{self._key_name}."
|
||||||
|
try:
|
||||||
|
base64.b64decode(self._secret, validate=True)
|
||||||
|
except Exception as exc: # noqa: BLE001
|
||||||
|
raise DnsSyncError("RFC2136_TSIG_SECRET must be valid base64") from exc
|
||||||
|
if self._algorithm not in {"hmac-sha256", "hmac-sha512", "hmac-sha1", "hmac-md5.sig-alg.reg.int"}:
|
||||||
|
raise DnsSyncError(f"Unsupported TSIG algorithm: {self._algorithm}")
|
||||||
|
return tsigkeyring.from_text({key_name: self._secret})
|
||||||
|
|
||||||
|
def _zone_with_dot(self) -> str:
|
||||||
|
return self._zone if self._zone.endswith(".") else f"{self._zone}."
|
||||||
|
|
||||||
|
def _absolute_name(self, fqdn: str) -> str:
|
||||||
|
return fqdn if fqdn.endswith(".") else f"{fqdn}."
|
||||||
|
|
||||||
|
|
||||||
|
def build_dns_provider(
|
||||||
|
provider_name: str,
|
||||||
|
*,
|
||||||
|
adguard_url: str,
|
||||||
|
adguard_username: str,
|
||||||
|
adguard_password: str,
|
||||||
|
rfc2136_server: str,
|
||||||
|
rfc2136_zone: str,
|
||||||
|
rfc2136_port: int,
|
||||||
|
rfc2136_tsig_key_name: str,
|
||||||
|
rfc2136_tsig_secret: str,
|
||||||
|
rfc2136_tsig_algorithm: str,
|
||||||
|
timeout_seconds: float,
|
||||||
|
) -> DnsProvider | None:
|
||||||
|
mode = provider_name.strip().lower()
|
||||||
|
if not mode or mode == "none":
|
||||||
|
return None
|
||||||
|
if mode == "adguard":
|
||||||
|
if not adguard_url.strip():
|
||||||
|
raise DnsSyncError("ADGUARD_URL is required for DNS_PROVIDER=adguard")
|
||||||
|
if not adguard_username.strip() or not adguard_password.strip():
|
||||||
|
raise DnsSyncError("ADGUARD_USERNAME and ADGUARD_PASSWORD are required for DNS_PROVIDER=adguard")
|
||||||
|
return AdguardDnsProvider(
|
||||||
|
AdguardConfig(
|
||||||
|
url=adguard_url,
|
||||||
|
username=adguard_username,
|
||||||
|
password=adguard_password,
|
||||||
|
timeout_seconds=timeout_seconds,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
if mode == "rfc2136":
|
||||||
|
return Rfc2136DnsProvider(
|
||||||
|
Rfc2136Config(
|
||||||
|
server=rfc2136_server,
|
||||||
|
zone=rfc2136_zone,
|
||||||
|
port=rfc2136_port,
|
||||||
|
timeout_seconds=timeout_seconds,
|
||||||
|
tsig_key_name=rfc2136_tsig_key_name,
|
||||||
|
tsig_secret=rfc2136_tsig_secret,
|
||||||
|
tsig_algorithm=rfc2136_tsig_algorithm,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
raise DnsSyncError(f"Unsupported DNS_PROVIDER: {provider_name}")
|
||||||
@@ -0,0 +1,134 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import http.client
|
||||||
|
import json
|
||||||
|
import socket
|
||||||
|
from urllib.parse import urlencode, urlparse
|
||||||
|
|
||||||
|
|
||||||
|
class DockerApiError(RuntimeError):
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
class _UnixHTTPConnection(http.client.HTTPConnection):
|
||||||
|
def __init__(self, socket_path: str, timeout: float):
|
||||||
|
super().__init__(host="localhost", timeout=timeout)
|
||||||
|
self._socket_path = socket_path
|
||||||
|
|
||||||
|
def connect(self) -> None:
|
||||||
|
sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
|
||||||
|
sock.settimeout(self.timeout)
|
||||||
|
sock.connect(self._socket_path)
|
||||||
|
self.sock = sock
|
||||||
|
|
||||||
|
|
||||||
|
class DockerApiClient:
|
||||||
|
def __init__(self, base_url: str, timeout_seconds: float = 3.0):
|
||||||
|
parsed = urlparse(base_url)
|
||||||
|
self._timeout = timeout_seconds
|
||||||
|
|
||||||
|
if parsed.scheme == "unix":
|
||||||
|
self._mode = "unix"
|
||||||
|
self._socket_path = parsed.path or "/var/run/docker.sock"
|
||||||
|
self._base_path = ""
|
||||||
|
elif parsed.scheme in {"http", "https"}:
|
||||||
|
if not parsed.netloc:
|
||||||
|
raise ValueError(f"Invalid Docker URL (missing host): {base_url}")
|
||||||
|
self._mode = parsed.scheme
|
||||||
|
self._host = parsed.hostname or "localhost"
|
||||||
|
self._port = parsed.port
|
||||||
|
self._base_path = parsed.path.rstrip("/")
|
||||||
|
else:
|
||||||
|
raise ValueError(f"Unsupported Docker URL scheme: {parsed.scheme}")
|
||||||
|
|
||||||
|
def list_containers(self, include_stopped: bool = True) -> list[dict]:
|
||||||
|
query = urlencode({"all": "1" if include_stopped else "0"})
|
||||||
|
return self._get_json(f"/containers/json?{query}")
|
||||||
|
|
||||||
|
def inspect_container(self, container_id: str) -> dict:
|
||||||
|
return self._get_json(f"/containers/{container_id}/json")
|
||||||
|
|
||||||
|
def _get_json(self, path: str):
|
||||||
|
body = self._get(path)
|
||||||
|
try:
|
||||||
|
return json.loads(body)
|
||||||
|
except json.JSONDecodeError as exc:
|
||||||
|
raise DockerApiError(f"Docker API returned invalid JSON for {path}: {exc}") from exc
|
||||||
|
|
||||||
|
def _get(self, path: str) -> str:
|
||||||
|
if not path.startswith("/"):
|
||||||
|
path = "/" + path
|
||||||
|
|
||||||
|
if self._mode == "unix":
|
||||||
|
conn = _UnixHTTPConnection(self._socket_path, timeout=self._timeout)
|
||||||
|
request_path = f"{self._base_path}{path}"
|
||||||
|
else:
|
||||||
|
if self._mode == "https":
|
||||||
|
conn = http.client.HTTPSConnection(self._host, self._port, timeout=self._timeout)
|
||||||
|
else:
|
||||||
|
conn = http.client.HTTPConnection(self._host, self._port, timeout=self._timeout)
|
||||||
|
request_path = f"{self._base_path}{path}"
|
||||||
|
|
||||||
|
try:
|
||||||
|
conn.request("GET", request_path)
|
||||||
|
response = conn.getresponse()
|
||||||
|
payload = response.read().decode("utf-8", errors="replace")
|
||||||
|
except OSError as exc:
|
||||||
|
raise DockerApiError(f"Docker API request failed for {path}: {exc}") from exc
|
||||||
|
finally:
|
||||||
|
conn.close()
|
||||||
|
|
||||||
|
if response.status < 200 or response.status >= 300:
|
||||||
|
raise DockerApiError(
|
||||||
|
f"Docker API error for {path}: HTTP {response.status} {response.reason}; body={payload[:400]}"
|
||||||
|
)
|
||||||
|
|
||||||
|
return payload
|
||||||
|
|
||||||
|
|
||||||
|
class DockerUsageResolver:
|
||||||
|
def __init__(self, docker_client: DockerApiClient):
|
||||||
|
self._docker = docker_client
|
||||||
|
|
||||||
|
def resolve_ip_usage(self, ips: set[str]) -> dict[str, set[str]]:
|
||||||
|
usage: dict[str, set[str]] = {ip: set() for ip in ips}
|
||||||
|
if not ips:
|
||||||
|
return usage
|
||||||
|
|
||||||
|
containers = self._docker.list_containers(include_stopped=True)
|
||||||
|
for container in containers:
|
||||||
|
container_id = container.get("Id")
|
||||||
|
if not container_id:
|
||||||
|
continue
|
||||||
|
inspect_payload = self._docker.inspect_container(container_id)
|
||||||
|
container_name = _display_name(container, inspect_payload)
|
||||||
|
ports = ((inspect_payload.get("NetworkSettings") or {}).get("Ports") or {})
|
||||||
|
if not isinstance(ports, dict):
|
||||||
|
continue
|
||||||
|
|
||||||
|
for bindings in ports.values():
|
||||||
|
if not bindings:
|
||||||
|
continue
|
||||||
|
for binding in bindings:
|
||||||
|
if not isinstance(binding, dict):
|
||||||
|
continue
|
||||||
|
host_ip = binding.get("HostIp")
|
||||||
|
if host_ip in usage:
|
||||||
|
usage[host_ip].add(container_name)
|
||||||
|
|
||||||
|
return usage
|
||||||
|
|
||||||
|
|
||||||
|
def _display_name(container_summary: dict, container_inspect: dict) -> str:
|
||||||
|
inspect_name = container_inspect.get("Name")
|
||||||
|
if isinstance(inspect_name, str) and inspect_name.strip("/"):
|
||||||
|
return inspect_name.strip("/")
|
||||||
|
|
||||||
|
names = container_summary.get("Names")
|
||||||
|
if isinstance(names, list) and names:
|
||||||
|
first_name = str(names[0]).strip("/")
|
||||||
|
if first_name:
|
||||||
|
return first_name
|
||||||
|
|
||||||
|
container_id = container_summary.get("Id", "")
|
||||||
|
return str(container_id)[:12] or "unknown"
|
||||||
@@ -0,0 +1,28 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import subprocess
|
||||||
|
|
||||||
|
|
||||||
|
def list_host_interfaces() -> list[str]:
|
||||||
|
result = subprocess.run(
|
||||||
|
["ip", "-o", "link", "show"],
|
||||||
|
capture_output=True,
|
||||||
|
text=True,
|
||||||
|
check=False,
|
||||||
|
)
|
||||||
|
if result.returncode != 0:
|
||||||
|
raise RuntimeError(f"Failed to list interfaces: {result.stderr.strip()}")
|
||||||
|
|
||||||
|
interfaces: list[str] = []
|
||||||
|
seen = set()
|
||||||
|
for line in result.stdout.splitlines():
|
||||||
|
parts = line.split(":", maxsplit=2)
|
||||||
|
if len(parts) < 2:
|
||||||
|
continue
|
||||||
|
name = parts[1].strip().split("@", maxsplit=1)[0]
|
||||||
|
if not name or name in seen:
|
||||||
|
continue
|
||||||
|
seen.add(name)
|
||||||
|
interfaces.append(name)
|
||||||
|
|
||||||
|
return interfaces
|
||||||
@@ -0,0 +1,48 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import subprocess
|
||||||
|
from typing import Callable
|
||||||
|
|
||||||
|
|
||||||
|
class CommandError(RuntimeError):
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
Runner = Callable[[list[str]], subprocess.CompletedProcess[str]]
|
||||||
|
|
||||||
|
|
||||||
|
def _default_runner(args: list[str]) -> subprocess.CompletedProcess[str]:
|
||||||
|
return subprocess.run(args, capture_output=True, text=True, check=False)
|
||||||
|
|
||||||
|
|
||||||
|
class IpAddressManager:
|
||||||
|
def __init__(self, runner: Runner | None = None):
|
||||||
|
self._runner = runner or _default_runner
|
||||||
|
|
||||||
|
def is_present(self, ip: str, cidr: int, device: str) -> bool:
|
||||||
|
target = f"{ip}/{cidr}"
|
||||||
|
result = self._runner(["ip", "-4", "-o", "addr", "show", "dev", device])
|
||||||
|
if result.returncode != 0:
|
||||||
|
raise CommandError(f"Failed to query addresses on {device}: {result.stderr.strip()}")
|
||||||
|
|
||||||
|
for line in result.stdout.splitlines():
|
||||||
|
tokens = line.split()
|
||||||
|
if target in tokens:
|
||||||
|
return True
|
||||||
|
return False
|
||||||
|
|
||||||
|
def ensure_present(self, ip: str, cidr: int, device: str) -> None:
|
||||||
|
if self.is_present(ip, cidr, device):
|
||||||
|
return
|
||||||
|
|
||||||
|
result = self._runner(["ip", "addr", "add", f"{ip}/{cidr}", "dev", device])
|
||||||
|
if result.returncode != 0 and "File exists" not in result.stderr:
|
||||||
|
raise CommandError(f"Failed to add address {ip}/{cidr} on {device}: {result.stderr.strip()}")
|
||||||
|
|
||||||
|
def ensure_absent(self, ip: str, cidr: int, device: str) -> None:
|
||||||
|
if not self.is_present(ip, cidr, device):
|
||||||
|
return
|
||||||
|
|
||||||
|
result = self._runner(["ip", "addr", "del", f"{ip}/{cidr}", "dev", device])
|
||||||
|
if result.returncode != 0 and "Cannot assign requested address" not in result.stderr:
|
||||||
|
raise CommandError(f"Failed to remove address {ip}/{cidr} on {device}: {result.stderr.strip()}")
|
||||||
@@ -0,0 +1,204 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from pathlib import Path
|
||||||
|
import threading
|
||||||
|
|
||||||
|
from fastapi import FastAPI, HTTPException
|
||||||
|
from fastapi.responses import FileResponse, JSONResponse
|
||||||
|
from fastapi.staticfiles import StaticFiles
|
||||||
|
|
||||||
|
from app.config import get_settings
|
||||||
|
from app.docker_api import DockerApiClient, DockerApiError, DockerUsageResolver
|
||||||
|
from app.dns_sync import DnsSyncError, build_dns_provider
|
||||||
|
from app.ip_commands import CommandError, IpAddressManager
|
||||||
|
from app.service import (
|
||||||
|
ConflictError,
|
||||||
|
DependencyError,
|
||||||
|
EntryService,
|
||||||
|
NotFoundError,
|
||||||
|
ValidationError,
|
||||||
|
)
|
||||||
|
from app.storage import EntryStorage
|
||||||
|
|
||||||
|
|
||||||
|
def build_service() -> EntryService:
|
||||||
|
settings = get_settings()
|
||||||
|
storage = EntryStorage(settings.state_file)
|
||||||
|
docker_client = DockerApiClient(settings.docker_api_url, timeout_seconds=settings.docker_timeout_seconds)
|
||||||
|
usage_resolver = DockerUsageResolver(docker_client)
|
||||||
|
ip_manager = IpAddressManager()
|
||||||
|
dns_provider = build_dns_provider(
|
||||||
|
settings.dns_provider,
|
||||||
|
adguard_url=settings.adguard_url,
|
||||||
|
adguard_username=settings.adguard_username,
|
||||||
|
adguard_password=settings.adguard_password,
|
||||||
|
rfc2136_server=settings.rfc2136_server,
|
||||||
|
rfc2136_zone=settings.rfc2136_zone,
|
||||||
|
rfc2136_port=settings.rfc2136_port,
|
||||||
|
rfc2136_tsig_key_name=settings.rfc2136_tsig_key_name,
|
||||||
|
rfc2136_tsig_secret=settings.rfc2136_tsig_secret,
|
||||||
|
rfc2136_tsig_algorithm=settings.rfc2136_tsig_algorithm,
|
||||||
|
timeout_seconds=settings.docker_timeout_seconds,
|
||||||
|
)
|
||||||
|
return EntryService(
|
||||||
|
storage=storage,
|
||||||
|
usage_resolver=usage_resolver,
|
||||||
|
ip_manager=ip_manager,
|
||||||
|
dns_provider=dns_provider,
|
||||||
|
dns_base_domain=settings.dns_base_domain,
|
||||||
|
dns_ttl_seconds=settings.dns_ttl_seconds,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
service = build_service()
|
||||||
|
app = FastAPI(title="Docker IP Addr Manager", version="0.1.0")
|
||||||
|
settings = get_settings()
|
||||||
|
stop_event = threading.Event()
|
||||||
|
background_thread: threading.Thread | None = None
|
||||||
|
|
||||||
|
static_dir = Path(__file__).parent / "static"
|
||||||
|
app.mount("/static", StaticFiles(directory=static_dir), name="static")
|
||||||
|
|
||||||
|
|
||||||
|
@app.on_event("startup")
|
||||||
|
def startup_reconcile() -> None:
|
||||||
|
errors = service.reconcile_enabled_entries()
|
||||||
|
if errors:
|
||||||
|
for error in errors:
|
||||||
|
print(f"[startup-reconcile] {error}")
|
||||||
|
dns_errors = service.reconcile_dns_records()
|
||||||
|
if dns_errors:
|
||||||
|
for error in dns_errors:
|
||||||
|
print(f"[dns-reconcile-startup] {error}")
|
||||||
|
_start_dns_background_loop()
|
||||||
|
|
||||||
|
|
||||||
|
@app.on_event("shutdown")
|
||||||
|
def shutdown_reconcile() -> None:
|
||||||
|
stop_event.set()
|
||||||
|
if background_thread and background_thread.is_alive():
|
||||||
|
background_thread.join(timeout=2.0)
|
||||||
|
|
||||||
|
|
||||||
|
def _dns_background_worker(interval_seconds: float) -> None:
|
||||||
|
while not stop_event.wait(interval_seconds):
|
||||||
|
errors = service.reconcile_dns_records()
|
||||||
|
for error in errors:
|
||||||
|
print(f"[dns-reconcile] {error}")
|
||||||
|
|
||||||
|
|
||||||
|
def _start_dns_background_loop() -> None:
|
||||||
|
global background_thread
|
||||||
|
if settings.dns_provider.strip().lower() in {"", "none"}:
|
||||||
|
return
|
||||||
|
if background_thread and background_thread.is_alive():
|
||||||
|
return
|
||||||
|
background_thread = threading.Thread(
|
||||||
|
target=_dns_background_worker,
|
||||||
|
args=(max(settings.dns_sync_interval_seconds, 1.0),),
|
||||||
|
daemon=True,
|
||||||
|
)
|
||||||
|
background_thread.start()
|
||||||
|
|
||||||
|
|
||||||
|
@app.get("/")
|
||||||
|
def index() -> FileResponse:
|
||||||
|
return FileResponse(static_dir / "index.html")
|
||||||
|
|
||||||
|
|
||||||
|
@app.get("/healthz")
|
||||||
|
def healthz() -> dict:
|
||||||
|
return {"ok": True}
|
||||||
|
|
||||||
|
|
||||||
|
@app.get("/api/interfaces")
|
||||||
|
def list_interfaces() -> dict:
|
||||||
|
try:
|
||||||
|
return {"items": service.list_interfaces()}
|
||||||
|
except DependencyError as exc:
|
||||||
|
raise HTTPException(status_code=503, detail=str(exc)) from exc
|
||||||
|
|
||||||
|
|
||||||
|
@app.get("/api/entries")
|
||||||
|
def list_entries() -> dict:
|
||||||
|
response = service.list_entries()
|
||||||
|
return response.to_dict()
|
||||||
|
|
||||||
|
|
||||||
|
@app.post("/api/refresh")
|
||||||
|
def refresh_entries() -> dict:
|
||||||
|
response = service.list_entries()
|
||||||
|
return response.to_dict()
|
||||||
|
|
||||||
|
|
||||||
|
@app.post("/api/entries")
|
||||||
|
def create_entry(payload: dict) -> dict:
|
||||||
|
try:
|
||||||
|
entry = service.create_entry(payload)
|
||||||
|
return entry.to_dict()
|
||||||
|
except ValidationError as exc:
|
||||||
|
raise HTTPException(status_code=422, detail=str(exc)) from exc
|
||||||
|
except ConflictError as exc:
|
||||||
|
raise HTTPException(status_code=409, detail=str(exc)) from exc
|
||||||
|
|
||||||
|
|
||||||
|
@app.put("/api/entries/{entry_id}")
|
||||||
|
def update_entry(entry_id: str, payload: dict) -> dict:
|
||||||
|
try:
|
||||||
|
entry = service.update_entry(entry_id, payload)
|
||||||
|
return entry.to_dict()
|
||||||
|
except ValidationError as exc:
|
||||||
|
raise HTTPException(status_code=422, detail=str(exc)) from exc
|
||||||
|
except ConflictError as exc:
|
||||||
|
raise HTTPException(status_code=409, detail=str(exc)) from exc
|
||||||
|
except NotFoundError as exc:
|
||||||
|
raise HTTPException(status_code=404, detail=str(exc)) from exc
|
||||||
|
|
||||||
|
|
||||||
|
@app.post("/api/entries/{entry_id}/enable")
|
||||||
|
def enable_entry(entry_id: str) -> dict:
|
||||||
|
try:
|
||||||
|
entry = service.set_enabled(entry_id, enabled=True)
|
||||||
|
return entry.to_dict()
|
||||||
|
except NotFoundError as exc:
|
||||||
|
raise HTTPException(status_code=404, detail=str(exc)) from exc
|
||||||
|
except (ConflictError, CommandError) as exc:
|
||||||
|
raise HTTPException(status_code=409, detail=str(exc)) from exc
|
||||||
|
except DependencyError as exc:
|
||||||
|
raise HTTPException(status_code=503, detail=str(exc)) from exc
|
||||||
|
|
||||||
|
|
||||||
|
@app.post("/api/entries/{entry_id}/disable")
|
||||||
|
def disable_entry(entry_id: str) -> dict:
|
||||||
|
try:
|
||||||
|
entry = service.set_enabled(entry_id, enabled=False)
|
||||||
|
return entry.to_dict()
|
||||||
|
except NotFoundError as exc:
|
||||||
|
raise HTTPException(status_code=404, detail=str(exc)) from exc
|
||||||
|
except (ConflictError, CommandError) as exc:
|
||||||
|
raise HTTPException(status_code=409, detail=str(exc)) from exc
|
||||||
|
except DependencyError as exc:
|
||||||
|
raise HTTPException(status_code=503, detail=str(exc)) from exc
|
||||||
|
|
||||||
|
|
||||||
|
@app.delete("/api/entries/{entry_id}")
|
||||||
|
def delete_entry(entry_id: str) -> dict:
|
||||||
|
try:
|
||||||
|
service.delete_entry(entry_id)
|
||||||
|
return {"deleted": True}
|
||||||
|
except NotFoundError as exc:
|
||||||
|
raise HTTPException(status_code=404, detail=str(exc)) from exc
|
||||||
|
except ConflictError as exc:
|
||||||
|
raise HTTPException(status_code=409, detail=str(exc)) from exc
|
||||||
|
except DependencyError as exc:
|
||||||
|
raise HTTPException(status_code=503, detail=str(exc)) from exc
|
||||||
|
|
||||||
|
|
||||||
|
@app.exception_handler(DockerApiError)
|
||||||
|
async def docker_error_handler(_, exc: DockerApiError):
|
||||||
|
return JSONResponse(status_code=503, content={"detail": str(exc)})
|
||||||
|
|
||||||
|
|
||||||
|
@app.exception_handler(DnsSyncError)
|
||||||
|
async def dns_error_handler(_, exc: DnsSyncError):
|
||||||
|
return JSONResponse(status_code=503, content={"detail": str(exc)})
|
||||||
@@ -0,0 +1,66 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from dataclasses import dataclass
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class IpEntry:
|
||||||
|
id: str
|
||||||
|
name: str
|
||||||
|
ip: str
|
||||||
|
cidr: int
|
||||||
|
device: str
|
||||||
|
enabled: bool
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def from_dict(cls, data: dict) -> "IpEntry":
|
||||||
|
return cls(
|
||||||
|
id=str(data["id"]),
|
||||||
|
name=str(data["name"]),
|
||||||
|
ip=str(data["ip"]),
|
||||||
|
cidr=int(data["cidr"]),
|
||||||
|
device=str(data["device"]),
|
||||||
|
enabled=bool(data.get("enabled", False)),
|
||||||
|
)
|
||||||
|
|
||||||
|
def to_dict(self) -> dict:
|
||||||
|
return {
|
||||||
|
"id": self.id,
|
||||||
|
"name": self.name,
|
||||||
|
"ip": self.ip,
|
||||||
|
"cidr": self.cidr,
|
||||||
|
"device": self.device,
|
||||||
|
"enabled": self.enabled,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class EntryView:
|
||||||
|
id: str
|
||||||
|
name: str
|
||||||
|
ip: str
|
||||||
|
cidr: int
|
||||||
|
device: str
|
||||||
|
enabled: bool
|
||||||
|
effective_enabled: bool
|
||||||
|
used: bool
|
||||||
|
containers: list[str]
|
||||||
|
usage_known: bool
|
||||||
|
dns_desired: bool = False
|
||||||
|
dns_last_error: str | None = None
|
||||||
|
|
||||||
|
def to_dict(self) -> dict:
|
||||||
|
return {
|
||||||
|
"id": self.id,
|
||||||
|
"name": self.name,
|
||||||
|
"ip": self.ip,
|
||||||
|
"cidr": self.cidr,
|
||||||
|
"device": self.device,
|
||||||
|
"enabled": self.enabled,
|
||||||
|
"effective_enabled": self.effective_enabled,
|
||||||
|
"used": self.used,
|
||||||
|
"containers": self.containers,
|
||||||
|
"usage_known": self.usage_known,
|
||||||
|
"dns_desired": self.dns_desired,
|
||||||
|
"dns_last_error": self.dns_last_error,
|
||||||
|
}
|
||||||
@@ -0,0 +1,383 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from dataclasses import dataclass
|
||||||
|
from ipaddress import IPv4Address
|
||||||
|
import threading
|
||||||
|
from typing import Callable
|
||||||
|
from uuid import uuid4
|
||||||
|
|
||||||
|
from app.docker_api import DockerApiError, DockerUsageResolver
|
||||||
|
from app.dns_sync import DnsProvider, DnsSyncError, to_fqdn
|
||||||
|
from app.interfaces import list_host_interfaces
|
||||||
|
from app.ip_commands import CommandError, IpAddressManager
|
||||||
|
from app.models import EntryView, IpEntry
|
||||||
|
from app.storage import EntryStorage
|
||||||
|
|
||||||
|
|
||||||
|
class ValidationError(RuntimeError):
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
class NotFoundError(RuntimeError):
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
class ConflictError(RuntimeError):
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
class DependencyError(RuntimeError):
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class EntryListResponse:
|
||||||
|
items: list[EntryView]
|
||||||
|
usage_known: bool
|
||||||
|
usage_error: str | None
|
||||||
|
|
||||||
|
def to_dict(self) -> dict:
|
||||||
|
return {
|
||||||
|
"items": [item.to_dict() for item in self.items],
|
||||||
|
"usage_known": self.usage_known,
|
||||||
|
"usage_error": self.usage_error,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
class EntryService:
|
||||||
|
def __init__(
|
||||||
|
self,
|
||||||
|
storage: EntryStorage,
|
||||||
|
usage_resolver: DockerUsageResolver,
|
||||||
|
ip_manager: IpAddressManager,
|
||||||
|
interface_provider: Callable[[], list[str]] = list_host_interfaces,
|
||||||
|
dns_provider: DnsProvider | None = None,
|
||||||
|
dns_base_domain: str = "",
|
||||||
|
dns_ttl_seconds: int = 120,
|
||||||
|
):
|
||||||
|
self._storage = storage
|
||||||
|
self._usage_resolver = usage_resolver
|
||||||
|
self._ip_manager = ip_manager
|
||||||
|
self._interface_provider = interface_provider
|
||||||
|
self._dns_provider = dns_provider
|
||||||
|
self._dns_base_domain = dns_base_domain
|
||||||
|
self._dns_ttl_seconds = dns_ttl_seconds
|
||||||
|
self._lock = threading.Lock()
|
||||||
|
self._dns_errors_by_id: dict[str, str] = {}
|
||||||
|
|
||||||
|
def list_interfaces(self) -> list[str]:
|
||||||
|
interfaces = self._interface_provider()
|
||||||
|
if not interfaces:
|
||||||
|
raise DependencyError("No network interfaces discovered on host")
|
||||||
|
return interfaces
|
||||||
|
|
||||||
|
def list_entries(self) -> EntryListResponse:
|
||||||
|
entries = self._storage.list_entries()
|
||||||
|
usage_map, usage_known, usage_error = self._resolve_usage(entries)
|
||||||
|
|
||||||
|
views: list[EntryView] = []
|
||||||
|
for entry in entries:
|
||||||
|
containers = sorted(usage_map.get(entry.ip, set()))
|
||||||
|
used = bool(containers) if usage_known else False
|
||||||
|
effective_enabled = False
|
||||||
|
try:
|
||||||
|
effective_enabled = self._ip_manager.is_present(entry.ip, entry.cidr, entry.device)
|
||||||
|
except CommandError:
|
||||||
|
effective_enabled = False
|
||||||
|
|
||||||
|
views.append(
|
||||||
|
EntryView(
|
||||||
|
id=entry.id,
|
||||||
|
name=entry.name,
|
||||||
|
ip=entry.ip,
|
||||||
|
cidr=entry.cidr,
|
||||||
|
device=entry.device,
|
||||||
|
enabled=entry.enabled,
|
||||||
|
effective_enabled=effective_enabled,
|
||||||
|
used=used,
|
||||||
|
containers=containers,
|
||||||
|
usage_known=usage_known,
|
||||||
|
dns_desired=bool(self._dns_provider) and usage_known and used and entry.enabled,
|
||||||
|
dns_last_error=self._dns_errors_by_id.get(entry.id),
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
||||||
|
return EntryListResponse(items=views, usage_known=usage_known, usage_error=usage_error)
|
||||||
|
|
||||||
|
def create_entry(self, payload: dict) -> IpEntry:
|
||||||
|
parsed = _parse_payload(payload)
|
||||||
|
with self._lock:
|
||||||
|
entries = self._storage.list_entries()
|
||||||
|
self._assert_device_exists(parsed["device"])
|
||||||
|
self._assert_unique_binding(entries, ip=parsed["ip"], cidr=parsed["cidr"], device=parsed["device"])
|
||||||
|
self._assert_unique_name(entries, name=parsed["name"])
|
||||||
|
|
||||||
|
created = IpEntry(
|
||||||
|
id=uuid4().hex,
|
||||||
|
name=parsed["name"],
|
||||||
|
ip=parsed["ip"],
|
||||||
|
cidr=parsed["cidr"],
|
||||||
|
device=parsed["device"],
|
||||||
|
enabled=False,
|
||||||
|
)
|
||||||
|
entries.append(created)
|
||||||
|
self._storage.save_entries(entries)
|
||||||
|
return created
|
||||||
|
|
||||||
|
def update_entry(self, entry_id: str, payload: dict) -> IpEntry:
|
||||||
|
parsed = _parse_payload(payload)
|
||||||
|
with self._lock:
|
||||||
|
entries = self._storage.list_entries()
|
||||||
|
index, current = _find_entry(entries, entry_id)
|
||||||
|
if current.enabled:
|
||||||
|
raise ConflictError("Disable entry before updating IP/cidr/device")
|
||||||
|
|
||||||
|
self._assert_device_exists(parsed["device"])
|
||||||
|
self._assert_unique_binding(
|
||||||
|
entries,
|
||||||
|
ip=parsed["ip"],
|
||||||
|
cidr=parsed["cidr"],
|
||||||
|
device=parsed["device"],
|
||||||
|
ignore_entry_id=entry_id,
|
||||||
|
)
|
||||||
|
self._assert_unique_name(entries, name=parsed["name"], ignore_entry_id=entry_id)
|
||||||
|
updated = IpEntry(
|
||||||
|
id=current.id,
|
||||||
|
name=parsed["name"],
|
||||||
|
ip=parsed["ip"],
|
||||||
|
cidr=parsed["cidr"],
|
||||||
|
device=parsed["device"],
|
||||||
|
enabled=False,
|
||||||
|
)
|
||||||
|
entries[index] = updated
|
||||||
|
self._storage.save_entries(entries)
|
||||||
|
return updated
|
||||||
|
|
||||||
|
def set_enabled(self, entry_id: str, enabled: bool) -> IpEntry:
|
||||||
|
with self._lock:
|
||||||
|
entries = self._storage.list_entries()
|
||||||
|
index, entry = _find_entry(entries, entry_id)
|
||||||
|
previous_enabled = entry.enabled
|
||||||
|
|
||||||
|
if enabled:
|
||||||
|
self._ip_manager.ensure_present(entry.ip, entry.cidr, entry.device)
|
||||||
|
entry.enabled = True
|
||||||
|
else:
|
||||||
|
self._assert_not_used(entry)
|
||||||
|
self._ip_manager.ensure_absent(entry.ip, entry.cidr, entry.device)
|
||||||
|
entry.enabled = False
|
||||||
|
|
||||||
|
try:
|
||||||
|
self._sync_dns_for_entry_locked(entry, strict=True)
|
||||||
|
except Exception: # noqa: BLE001
|
||||||
|
self._rollback_enable_change(entry, previous_enabled)
|
||||||
|
raise
|
||||||
|
|
||||||
|
entries[index] = entry
|
||||||
|
self._storage.save_entries(entries)
|
||||||
|
return entry
|
||||||
|
|
||||||
|
def delete_entry(self, entry_id: str) -> None:
|
||||||
|
with self._lock:
|
||||||
|
entries = self._storage.list_entries()
|
||||||
|
_, entry = _find_entry(entries, entry_id)
|
||||||
|
if entry.enabled:
|
||||||
|
raise ConflictError("Disable entry before deleting")
|
||||||
|
|
||||||
|
self._assert_not_used(entry)
|
||||||
|
self._delete_dns_for_entry_locked(entry, strict=True)
|
||||||
|
remaining = [candidate for candidate in entries if candidate.id != entry_id]
|
||||||
|
self._storage.save_entries(remaining)
|
||||||
|
self._dns_errors_by_id.pop(entry.id, None)
|
||||||
|
|
||||||
|
def reconcile_enabled_entries(self) -> list[str]:
|
||||||
|
errors: list[str] = []
|
||||||
|
with self._lock:
|
||||||
|
entries = self._storage.list_entries()
|
||||||
|
changed = False
|
||||||
|
for entry in entries:
|
||||||
|
if not entry.enabled:
|
||||||
|
continue
|
||||||
|
try:
|
||||||
|
self._ip_manager.ensure_present(entry.ip, entry.cidr, entry.device)
|
||||||
|
changed = True
|
||||||
|
except CommandError as exc:
|
||||||
|
errors.append(f"{entry.name} ({entry.ip}/{entry.cidr} {entry.device}): {exc}")
|
||||||
|
if changed:
|
||||||
|
self._storage.save_entries(entries)
|
||||||
|
return errors
|
||||||
|
|
||||||
|
def reconcile_dns_records(self) -> list[str]:
|
||||||
|
if not self._dns_provider:
|
||||||
|
return []
|
||||||
|
|
||||||
|
errors: list[str] = []
|
||||||
|
with self._lock:
|
||||||
|
entries = self._storage.list_entries()
|
||||||
|
usage_map, usage_known, usage_error = self._resolve_usage(entries)
|
||||||
|
if not usage_known:
|
||||||
|
msg = f"Docker usage check failed for DNS reconcile: {usage_error or 'unknown error'}"
|
||||||
|
for entry in entries:
|
||||||
|
self._dns_errors_by_id[entry.id] = msg
|
||||||
|
return [msg]
|
||||||
|
|
||||||
|
for entry in entries:
|
||||||
|
used = bool(usage_map.get(entry.ip, set()))
|
||||||
|
desired = entry.enabled and used
|
||||||
|
try:
|
||||||
|
self._apply_dns_state_locked(entry, desired)
|
||||||
|
self._dns_errors_by_id.pop(entry.id, None)
|
||||||
|
except (DnsSyncError, DependencyError, ConflictError) as exc:
|
||||||
|
self._dns_errors_by_id[entry.id] = str(exc)
|
||||||
|
errors.append(f"{entry.name}: {exc}")
|
||||||
|
return errors
|
||||||
|
|
||||||
|
def _rollback_enable_change(self, entry: IpEntry, previous_enabled: bool) -> None:
|
||||||
|
try:
|
||||||
|
if previous_enabled:
|
||||||
|
self._ip_manager.ensure_present(entry.ip, entry.cidr, entry.device)
|
||||||
|
entry.enabled = True
|
||||||
|
else:
|
||||||
|
self._ip_manager.ensure_absent(entry.ip, entry.cidr, entry.device)
|
||||||
|
entry.enabled = False
|
||||||
|
except CommandError:
|
||||||
|
pass
|
||||||
|
|
||||||
|
def _sync_dns_for_entry_locked(self, entry: IpEntry, strict: bool) -> None:
|
||||||
|
if not self._dns_provider:
|
||||||
|
return
|
||||||
|
|
||||||
|
usage_map, usage_known, usage_error = self._resolve_usage([entry])
|
||||||
|
if not usage_known:
|
||||||
|
msg = f"Docker usage check failed: {usage_error or 'unknown error'}"
|
||||||
|
self._dns_errors_by_id[entry.id] = msg
|
||||||
|
if strict:
|
||||||
|
raise DependencyError(msg)
|
||||||
|
return
|
||||||
|
|
||||||
|
desired = entry.enabled and bool(usage_map.get(entry.ip, set()))
|
||||||
|
self._apply_dns_state_locked(entry, desired)
|
||||||
|
self._dns_errors_by_id.pop(entry.id, None)
|
||||||
|
|
||||||
|
def _delete_dns_for_entry_locked(self, entry: IpEntry, strict: bool) -> None:
|
||||||
|
if not self._dns_provider:
|
||||||
|
return
|
||||||
|
|
||||||
|
try:
|
||||||
|
fqdn = to_fqdn(entry.name, self._dns_base_domain)
|
||||||
|
self._dns_provider.delete_a_record(fqdn)
|
||||||
|
self._dns_errors_by_id.pop(entry.id, None)
|
||||||
|
except DnsSyncError as exc:
|
||||||
|
self._dns_errors_by_id[entry.id] = str(exc)
|
||||||
|
if strict:
|
||||||
|
raise ConflictError(f"DNS delete failed for {entry.name}: {exc}") from exc
|
||||||
|
|
||||||
|
def _apply_dns_state_locked(self, entry: IpEntry, desired: bool) -> None:
|
||||||
|
if not self._dns_provider:
|
||||||
|
return
|
||||||
|
|
||||||
|
try:
|
||||||
|
fqdn = to_fqdn(entry.name, self._dns_base_domain)
|
||||||
|
if desired:
|
||||||
|
self._dns_provider.upsert_a_record(fqdn, entry.ip, self._dns_ttl_seconds)
|
||||||
|
else:
|
||||||
|
self._dns_provider.delete_a_record(fqdn)
|
||||||
|
except DnsSyncError as exc:
|
||||||
|
raise ConflictError(f"DNS sync failed for {entry.name}: {exc}") from exc
|
||||||
|
|
||||||
|
def _assert_not_used(self, entry: IpEntry) -> None:
|
||||||
|
try:
|
||||||
|
usage = self._usage_resolver.resolve_ip_usage({entry.ip})
|
||||||
|
except DockerApiError as exc:
|
||||||
|
raise DependencyError(f"Docker usage check failed: {exc}") from exc
|
||||||
|
|
||||||
|
containers = sorted(usage.get(entry.ip, set()))
|
||||||
|
if containers:
|
||||||
|
raise ConflictError(
|
||||||
|
"Entry is currently used by container port bindings: " + ", ".join(containers)
|
||||||
|
)
|
||||||
|
|
||||||
|
def _resolve_usage(self, entries: list[IpEntry]) -> tuple[dict[str, set[str]], bool, str | None]:
|
||||||
|
ips = {entry.ip for entry in entries}
|
||||||
|
if not ips:
|
||||||
|
return {}, True, None
|
||||||
|
|
||||||
|
try:
|
||||||
|
usage = self._usage_resolver.resolve_ip_usage(ips)
|
||||||
|
return usage, True, None
|
||||||
|
except DockerApiError as exc:
|
||||||
|
return {ip: set() for ip in ips}, False, str(exc)
|
||||||
|
|
||||||
|
def _assert_unique_binding(
|
||||||
|
self,
|
||||||
|
entries: list[IpEntry],
|
||||||
|
ip: str,
|
||||||
|
cidr: int,
|
||||||
|
device: str,
|
||||||
|
ignore_entry_id: str | None = None,
|
||||||
|
) -> None:
|
||||||
|
for entry in entries:
|
||||||
|
if ignore_entry_id and entry.id == ignore_entry_id:
|
||||||
|
continue
|
||||||
|
if entry.ip == ip and entry.cidr == cidr and entry.device == device:
|
||||||
|
raise ConflictError("Entry with same ip/cidr/device already exists")
|
||||||
|
|
||||||
|
def _assert_unique_name(self, entries: list[IpEntry], name: str, ignore_entry_id: str | None = None) -> None:
|
||||||
|
target = name.strip().lower()
|
||||||
|
for entry in entries:
|
||||||
|
if ignore_entry_id and entry.id == ignore_entry_id:
|
||||||
|
continue
|
||||||
|
if entry.name.strip().lower() == target:
|
||||||
|
raise ConflictError("Entry name must be unique")
|
||||||
|
|
||||||
|
def _assert_device_exists(self, device: str) -> None:
|
||||||
|
interfaces = self.list_interfaces()
|
||||||
|
if device not in interfaces:
|
||||||
|
raise ValidationError(f"Unknown network device: {device}")
|
||||||
|
|
||||||
|
|
||||||
|
def _find_entry(entries: list[IpEntry], entry_id: str) -> tuple[int, IpEntry]:
|
||||||
|
for index, entry in enumerate(entries):
|
||||||
|
if entry.id == entry_id:
|
||||||
|
return index, entry
|
||||||
|
raise NotFoundError(f"Entry not found: {entry_id}")
|
||||||
|
|
||||||
|
|
||||||
|
def _parse_payload(payload: dict) -> dict:
|
||||||
|
name = str(payload.get("name", "")).strip()
|
||||||
|
ip = str(payload.get("ip", "")).strip()
|
||||||
|
device = str(payload.get("device", "")).strip()
|
||||||
|
|
||||||
|
if not name:
|
||||||
|
raise ValidationError("Field 'name' is required")
|
||||||
|
if len(name) > 96:
|
||||||
|
raise ValidationError("Field 'name' is too long (max 96)")
|
||||||
|
|
||||||
|
if not ip:
|
||||||
|
raise ValidationError("Field 'ip' is required")
|
||||||
|
try:
|
||||||
|
ip_obj = IPv4Address(ip)
|
||||||
|
except ValueError as exc:
|
||||||
|
raise ValidationError(f"Invalid IPv4 address: {ip}") from exc
|
||||||
|
|
||||||
|
if not device:
|
||||||
|
raise ValidationError("Field 'device' is required")
|
||||||
|
if any(ch.isspace() for ch in device):
|
||||||
|
raise ValidationError("Field 'device' cannot contain whitespace")
|
||||||
|
|
||||||
|
cidr_raw = payload.get("cidr")
|
||||||
|
if cidr_raw is None:
|
||||||
|
raise ValidationError("Field 'cidr' is required")
|
||||||
|
try:
|
||||||
|
cidr = int(cidr_raw)
|
||||||
|
except (TypeError, ValueError) as exc:
|
||||||
|
raise ValidationError("Field 'cidr' must be an integer") from exc
|
||||||
|
if cidr < 0 or cidr > 32:
|
||||||
|
raise ValidationError("Field 'cidr' must be between 0 and 32")
|
||||||
|
|
||||||
|
return {
|
||||||
|
"name": name,
|
||||||
|
"ip": str(ip_obj),
|
||||||
|
"cidr": cidr,
|
||||||
|
"device": device,
|
||||||
|
}
|
||||||
@@ -0,0 +1,216 @@
|
|||||||
|
const { createApp } = Vue;
|
||||||
|
|
||||||
|
function mapEntry(entry) {
|
||||||
|
return {
|
||||||
|
...entry,
|
||||||
|
draft: {
|
||||||
|
name: entry.name,
|
||||||
|
ip: entry.ip,
|
||||||
|
cidr: entry.cidr,
|
||||||
|
device: entry.device,
|
||||||
|
},
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
createApp({
|
||||||
|
data() {
|
||||||
|
return {
|
||||||
|
interfaces: [],
|
||||||
|
entries: [],
|
||||||
|
usageKnown: true,
|
||||||
|
usageError: "",
|
||||||
|
loading: false,
|
||||||
|
saving: false,
|
||||||
|
busyById: {},
|
||||||
|
errorMessage: "",
|
||||||
|
form: {
|
||||||
|
name: "",
|
||||||
|
ip: "",
|
||||||
|
cidr: 16,
|
||||||
|
device: "",
|
||||||
|
},
|
||||||
|
sort: {
|
||||||
|
key: "name",
|
||||||
|
direction: "asc",
|
||||||
|
},
|
||||||
|
};
|
||||||
|
},
|
||||||
|
computed: {
|
||||||
|
sortedEntries() {
|
||||||
|
const list = [...this.entries];
|
||||||
|
const key = this.sort.key;
|
||||||
|
const multiplier = this.sort.direction === "asc" ? 1 : -1;
|
||||||
|
|
||||||
|
const extractors = {
|
||||||
|
name: (entry) => entry.name.toLowerCase(),
|
||||||
|
ip: (entry) => entry.ip,
|
||||||
|
cidr: (entry) => entry.cidr,
|
||||||
|
used: (entry) => (entry.usage_known ? (entry.used ? 1 : 0) : -1),
|
||||||
|
containers: (entry) => entry.containers.join(",").toLowerCase(),
|
||||||
|
device: (entry) => entry.device.toLowerCase(),
|
||||||
|
enabled: (entry) => (entry.enabled ? 1 : 0),
|
||||||
|
};
|
||||||
|
|
||||||
|
const extract = extractors[key] || extractors.name;
|
||||||
|
list.sort((a, b) => {
|
||||||
|
const left = extract(a);
|
||||||
|
const right = extract(b);
|
||||||
|
if (left < right) {
|
||||||
|
return -1 * multiplier;
|
||||||
|
}
|
||||||
|
if (left > right) {
|
||||||
|
return 1 * multiplier;
|
||||||
|
}
|
||||||
|
return a.name.localeCompare(b.name) * multiplier;
|
||||||
|
});
|
||||||
|
|
||||||
|
return list;
|
||||||
|
},
|
||||||
|
},
|
||||||
|
methods: {
|
||||||
|
async api(path, options = {}) {
|
||||||
|
const response = await fetch(path, {
|
||||||
|
headers: {
|
||||||
|
"Content-Type": "application/json",
|
||||||
|
},
|
||||||
|
...options,
|
||||||
|
});
|
||||||
|
|
||||||
|
let payload = {};
|
||||||
|
try {
|
||||||
|
payload = await response.json();
|
||||||
|
} catch (err) {
|
||||||
|
payload = {};
|
||||||
|
}
|
||||||
|
|
||||||
|
if (!response.ok) {
|
||||||
|
const message = payload.detail || `Request failed: ${response.status}`;
|
||||||
|
throw new Error(message);
|
||||||
|
}
|
||||||
|
|
||||||
|
return payload;
|
||||||
|
},
|
||||||
|
async loadInterfaces() {
|
||||||
|
const data = await this.api("/api/interfaces");
|
||||||
|
this.interfaces = data.items || [];
|
||||||
|
if (!this.form.device && this.interfaces.length > 0) {
|
||||||
|
this.form.device = this.interfaces[0];
|
||||||
|
}
|
||||||
|
},
|
||||||
|
async refreshEntries() {
|
||||||
|
this.loading = true;
|
||||||
|
this.errorMessage = "";
|
||||||
|
try {
|
||||||
|
const data = await this.api("/api/refresh", { method: "POST" });
|
||||||
|
this.entries = (data.items || []).map(mapEntry);
|
||||||
|
this.usageKnown = Boolean(data.usage_known);
|
||||||
|
this.usageError = data.usage_error || "";
|
||||||
|
} catch (err) {
|
||||||
|
this.errorMessage = err.message;
|
||||||
|
} finally {
|
||||||
|
this.loading = false;
|
||||||
|
}
|
||||||
|
},
|
||||||
|
async createEntry() {
|
||||||
|
this.saving = true;
|
||||||
|
this.errorMessage = "";
|
||||||
|
try {
|
||||||
|
await this.api("/api/entries", {
|
||||||
|
method: "POST",
|
||||||
|
body: JSON.stringify({
|
||||||
|
name: this.form.name,
|
||||||
|
ip: this.form.ip,
|
||||||
|
cidr: this.form.cidr,
|
||||||
|
device: this.form.device,
|
||||||
|
}),
|
||||||
|
});
|
||||||
|
this.form.name = "";
|
||||||
|
this.form.ip = "";
|
||||||
|
await this.refreshEntries();
|
||||||
|
} catch (err) {
|
||||||
|
this.errorMessage = err.message;
|
||||||
|
} finally {
|
||||||
|
this.saving = false;
|
||||||
|
}
|
||||||
|
},
|
||||||
|
async saveEntry(entry) {
|
||||||
|
this.errorMessage = "";
|
||||||
|
this.busyById = { ...this.busyById, [entry.id]: true };
|
||||||
|
try {
|
||||||
|
await this.api(`/api/entries/${entry.id}`, {
|
||||||
|
method: "PUT",
|
||||||
|
body: JSON.stringify({
|
||||||
|
name: entry.draft.name,
|
||||||
|
ip: entry.draft.ip,
|
||||||
|
cidr: entry.draft.cidr,
|
||||||
|
device: entry.draft.device,
|
||||||
|
}),
|
||||||
|
});
|
||||||
|
await this.refreshEntries();
|
||||||
|
} catch (err) {
|
||||||
|
this.errorMessage = err.message;
|
||||||
|
} finally {
|
||||||
|
this.busyById = { ...this.busyById, [entry.id]: false };
|
||||||
|
}
|
||||||
|
},
|
||||||
|
async toggleEntry(entry) {
|
||||||
|
this.errorMessage = "";
|
||||||
|
this.busyById = { ...this.busyById, [entry.id]: true };
|
||||||
|
try {
|
||||||
|
const action = entry.enabled ? "disable" : "enable";
|
||||||
|
await this.api(`/api/entries/${entry.id}/${action}`, { method: "POST" });
|
||||||
|
await this.refreshEntries();
|
||||||
|
} catch (err) {
|
||||||
|
this.errorMessage = err.message;
|
||||||
|
} finally {
|
||||||
|
this.busyById = { ...this.busyById, [entry.id]: false };
|
||||||
|
}
|
||||||
|
},
|
||||||
|
async deleteEntry(entry) {
|
||||||
|
this.errorMessage = "";
|
||||||
|
this.busyById = { ...this.busyById, [entry.id]: true };
|
||||||
|
try {
|
||||||
|
await this.api(`/api/entries/${entry.id}`, { method: "DELETE" });
|
||||||
|
await this.refreshEntries();
|
||||||
|
} catch (err) {
|
||||||
|
this.errorMessage = err.message;
|
||||||
|
} finally {
|
||||||
|
this.busyById = { ...this.busyById, [entry.id]: false };
|
||||||
|
}
|
||||||
|
},
|
||||||
|
containerLabel(entry) {
|
||||||
|
if (!entry.containers || entry.containers.length === 0) {
|
||||||
|
return "-";
|
||||||
|
}
|
||||||
|
if (entry.containers.length === 1) {
|
||||||
|
return entry.containers[0];
|
||||||
|
}
|
||||||
|
return `${entry.containers.join(", ")} (${entry.containers.length})`;
|
||||||
|
},
|
||||||
|
setSort(key) {
|
||||||
|
if (this.sort.key === key) {
|
||||||
|
this.sort.direction = this.sort.direction === "asc" ? "desc" : "asc";
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
this.sort.key = key;
|
||||||
|
this.sort.direction = "asc";
|
||||||
|
},
|
||||||
|
sortIndicator(key) {
|
||||||
|
if (this.sort.key !== key) {
|
||||||
|
return "";
|
||||||
|
}
|
||||||
|
return this.sort.direction === "asc" ? "▲" : "▼";
|
||||||
|
},
|
||||||
|
isBusy(entryId) {
|
||||||
|
return Boolean(this.busyById[entryId]);
|
||||||
|
},
|
||||||
|
},
|
||||||
|
async mounted() {
|
||||||
|
try {
|
||||||
|
await this.loadInterfaces();
|
||||||
|
await this.refreshEntries();
|
||||||
|
} catch (err) {
|
||||||
|
this.errorMessage = err.message;
|
||||||
|
}
|
||||||
|
},
|
||||||
|
}).mount("#app");
|
||||||
@@ -0,0 +1,126 @@
|
|||||||
|
<!doctype html>
|
||||||
|
<html lang="en">
|
||||||
|
<head>
|
||||||
|
<meta charset="utf-8" />
|
||||||
|
<meta name="viewport" content="width=device-width, initial-scale=1" />
|
||||||
|
<title>Docker IP Addr Manager</title>
|
||||||
|
<link rel="stylesheet" href="/static/styles.css" />
|
||||||
|
</head>
|
||||||
|
<body>
|
||||||
|
<main id="app" class="app-shell" v-cloak>
|
||||||
|
<header class="topbar">
|
||||||
|
<div>
|
||||||
|
<h1>Docker IP Addr Manager</h1>
|
||||||
|
<p>Manage host IP addresses used for ZimaOS port bindings.</p>
|
||||||
|
</div>
|
||||||
|
<button type="button" class="btn btn-secondary" @click="refreshEntries" :disabled="loading">
|
||||||
|
{{ loading ? 'Refreshing...' : 'Refresh' }}
|
||||||
|
</button>
|
||||||
|
</header>
|
||||||
|
|
||||||
|
<section class="panel">
|
||||||
|
<h2>Add IP Entry</h2>
|
||||||
|
<form class="entry-form" @submit.prevent="createEntry">
|
||||||
|
<label>
|
||||||
|
Name
|
||||||
|
<input v-model.trim="form.name" type="text" required maxlength="96" />
|
||||||
|
</label>
|
||||||
|
|
||||||
|
<label>
|
||||||
|
IP Address
|
||||||
|
<input v-model.trim="form.ip" type="text" placeholder="10.0.4.2" required />
|
||||||
|
</label>
|
||||||
|
|
||||||
|
<label>
|
||||||
|
CIDR
|
||||||
|
<input v-model.number="form.cidr" type="number" min="0" max="32" required />
|
||||||
|
</label>
|
||||||
|
|
||||||
|
<label>
|
||||||
|
Device
|
||||||
|
<select v-model="form.device" required>
|
||||||
|
<option disabled value="">Choose device</option>
|
||||||
|
<option v-for="iface in interfaces" :key="iface" :value="iface">{{ iface }}</option>
|
||||||
|
</select>
|
||||||
|
</label>
|
||||||
|
|
||||||
|
<button class="btn btn-primary" type="submit" :disabled="saving">Add</button>
|
||||||
|
</form>
|
||||||
|
</section>
|
||||||
|
|
||||||
|
<section class="panel">
|
||||||
|
<div class="status" v-if="errorMessage">{{ errorMessage }}</div>
|
||||||
|
<div class="status warning" v-if="!usageKnown">
|
||||||
|
Docker usage status is unknown. Disable/Delete operations are fail-closed until refresh succeeds.
|
||||||
|
<span v-if="usageError">Error: {{ usageError }}</span>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div class="table-wrap">
|
||||||
|
<table>
|
||||||
|
<thead>
|
||||||
|
<tr>
|
||||||
|
<th @click="setSort('name')">Name <span>{{ sortIndicator('name') }}</span></th>
|
||||||
|
<th @click="setSort('ip')">IP Address <span>{{ sortIndicator('ip') }}</span></th>
|
||||||
|
<th @click="setSort('cidr')">CIDR <span>{{ sortIndicator('cidr') }}</span></th>
|
||||||
|
<th @click="setSort('used')">Used <span>{{ sortIndicator('used') }}</span></th>
|
||||||
|
<th @click="setSort('containers')">Container(s) <span>{{ sortIndicator('containers') }}</span></th>
|
||||||
|
<th @click="setSort('device')">Device <span>{{ sortIndicator('device') }}</span></th>
|
||||||
|
<th @click="setSort('enabled')">Enabled <span>{{ sortIndicator('enabled') }}</span></th>
|
||||||
|
<th>Actions</th>
|
||||||
|
</tr>
|
||||||
|
</thead>
|
||||||
|
<tbody>
|
||||||
|
<tr v-for="entry in sortedEntries" :key="entry.id">
|
||||||
|
<td><input v-model.trim="entry.draft.name" type="text" maxlength="96" :disabled="entry.enabled" /></td>
|
||||||
|
<td><input v-model.trim="entry.draft.ip" type="text" :disabled="entry.enabled" /></td>
|
||||||
|
<td><input v-model.number="entry.draft.cidr" type="number" min="0" max="32" :disabled="entry.enabled" /></td>
|
||||||
|
<td class="status-cell">
|
||||||
|
<span v-if="!entry.usage_known" class="chip unknown">Unknown</span>
|
||||||
|
<span v-else-if="entry.used" class="chip used">Yes</span>
|
||||||
|
<span v-else class="chip free">No</span>
|
||||||
|
</td>
|
||||||
|
<td class="containers">{{ containerLabel(entry) }}</td>
|
||||||
|
<td>
|
||||||
|
<select v-model="entry.draft.device" :disabled="entry.enabled">
|
||||||
|
<option v-for="iface in interfaces" :key="`${entry.id}-${iface}`" :value="iface">{{ iface }}</option>
|
||||||
|
</select>
|
||||||
|
</td>
|
||||||
|
<td class="toggle-cell">
|
||||||
|
<button
|
||||||
|
type="button"
|
||||||
|
class="btn"
|
||||||
|
:class="entry.enabled ? 'btn-danger' : 'btn-primary'"
|
||||||
|
@click="toggleEntry(entry)"
|
||||||
|
:disabled="isBusy(entry.id)"
|
||||||
|
>
|
||||||
|
{{ entry.enabled ? 'Disable' : 'Enable' }}
|
||||||
|
</button>
|
||||||
|
</td>
|
||||||
|
<td class="actions">
|
||||||
|
<button
|
||||||
|
type="button"
|
||||||
|
class="btn btn-secondary"
|
||||||
|
@click="saveEntry(entry)"
|
||||||
|
:disabled="entry.enabled || isBusy(entry.id)"
|
||||||
|
>Save</button>
|
||||||
|
<button
|
||||||
|
type="button"
|
||||||
|
class="btn btn-danger"
|
||||||
|
@click="deleteEntry(entry)"
|
||||||
|
:disabled="entry.enabled || isBusy(entry.id)"
|
||||||
|
>Delete</button>
|
||||||
|
</td>
|
||||||
|
</tr>
|
||||||
|
<tr v-if="sortedEntries.length === 0">
|
||||||
|
<td colspan="8" class="empty">No entries configured.</td>
|
||||||
|
</tr>
|
||||||
|
</tbody>
|
||||||
|
</table>
|
||||||
|
</div>
|
||||||
|
</section>
|
||||||
|
</main>
|
||||||
|
|
||||||
|
<script src="https://unpkg.com/vue@3/dist/vue.global.prod.js"></script>
|
||||||
|
<script src="/static/app.js"></script>
|
||||||
|
</body>
|
||||||
|
</html>
|
||||||
@@ -0,0 +1,227 @@
|
|||||||
|
:root {
|
||||||
|
--bg: #f2f5f8;
|
||||||
|
--panel: #ffffff;
|
||||||
|
--border: #d7dee7;
|
||||||
|
--text: #1c2733;
|
||||||
|
--muted: #5f6b77;
|
||||||
|
--primary: #2274e5;
|
||||||
|
--danger: #c63434;
|
||||||
|
--warning: #8a6b00;
|
||||||
|
--chip-used: #ffe8e8;
|
||||||
|
--chip-free: #e7f7ec;
|
||||||
|
--chip-unknown: #fff6d8;
|
||||||
|
}
|
||||||
|
|
||||||
|
* {
|
||||||
|
box-sizing: border-box;
|
||||||
|
}
|
||||||
|
|
||||||
|
body {
|
||||||
|
margin: 0;
|
||||||
|
font-family: "Noto Sans", "Segoe UI", sans-serif;
|
||||||
|
background: linear-gradient(180deg, #f4f7fb 0%, #ecf1f7 100%);
|
||||||
|
color: var(--text);
|
||||||
|
}
|
||||||
|
|
||||||
|
[v-cloak] {
|
||||||
|
display: none;
|
||||||
|
}
|
||||||
|
|
||||||
|
.app-shell {
|
||||||
|
max-width: 1200px;
|
||||||
|
margin: 0 auto;
|
||||||
|
padding: 1.5rem;
|
||||||
|
display: grid;
|
||||||
|
gap: 1rem;
|
||||||
|
}
|
||||||
|
|
||||||
|
.topbar {
|
||||||
|
display: flex;
|
||||||
|
justify-content: space-between;
|
||||||
|
gap: 1rem;
|
||||||
|
align-items: center;
|
||||||
|
}
|
||||||
|
|
||||||
|
.topbar h1 {
|
||||||
|
margin: 0;
|
||||||
|
font-size: 1.45rem;
|
||||||
|
}
|
||||||
|
|
||||||
|
.topbar p {
|
||||||
|
margin: 0.2rem 0 0;
|
||||||
|
color: var(--muted);
|
||||||
|
}
|
||||||
|
|
||||||
|
.panel {
|
||||||
|
background: var(--panel);
|
||||||
|
border: 1px solid var(--border);
|
||||||
|
border-radius: 12px;
|
||||||
|
padding: 1rem;
|
||||||
|
}
|
||||||
|
|
||||||
|
.panel h2 {
|
||||||
|
margin: 0 0 0.8rem;
|
||||||
|
font-size: 1.05rem;
|
||||||
|
}
|
||||||
|
|
||||||
|
.entry-form {
|
||||||
|
display: grid;
|
||||||
|
grid-template-columns: repeat(auto-fit, minmax(140px, 1fr));
|
||||||
|
gap: 0.75rem;
|
||||||
|
align-items: end;
|
||||||
|
}
|
||||||
|
|
||||||
|
.entry-form label {
|
||||||
|
display: grid;
|
||||||
|
gap: 0.35rem;
|
||||||
|
font-size: 0.9rem;
|
||||||
|
color: var(--muted);
|
||||||
|
}
|
||||||
|
|
||||||
|
input,
|
||||||
|
select,
|
||||||
|
button {
|
||||||
|
font: inherit;
|
||||||
|
}
|
||||||
|
|
||||||
|
input,
|
||||||
|
select {
|
||||||
|
border: 1px solid var(--border);
|
||||||
|
border-radius: 8px;
|
||||||
|
padding: 0.5rem 0.55rem;
|
||||||
|
min-height: 2.2rem;
|
||||||
|
background: #fff;
|
||||||
|
color: var(--text);
|
||||||
|
}
|
||||||
|
|
||||||
|
.btn {
|
||||||
|
border: 0;
|
||||||
|
border-radius: 8px;
|
||||||
|
padding: 0.55rem 0.85rem;
|
||||||
|
cursor: pointer;
|
||||||
|
min-height: 2.2rem;
|
||||||
|
}
|
||||||
|
|
||||||
|
.btn:disabled {
|
||||||
|
opacity: 0.55;
|
||||||
|
cursor: not-allowed;
|
||||||
|
}
|
||||||
|
|
||||||
|
.btn-primary {
|
||||||
|
background: var(--primary);
|
||||||
|
color: #fff;
|
||||||
|
}
|
||||||
|
|
||||||
|
.btn-secondary {
|
||||||
|
background: #e8edf5;
|
||||||
|
color: var(--text);
|
||||||
|
}
|
||||||
|
|
||||||
|
.btn-danger {
|
||||||
|
background: var(--danger);
|
||||||
|
color: #fff;
|
||||||
|
}
|
||||||
|
|
||||||
|
.status {
|
||||||
|
margin-bottom: 0.8rem;
|
||||||
|
padding: 0.65rem 0.8rem;
|
||||||
|
border-radius: 8px;
|
||||||
|
background: #e8f0fc;
|
||||||
|
border: 1px solid #c7daf8;
|
||||||
|
color: #2a4b7d;
|
||||||
|
}
|
||||||
|
|
||||||
|
.status.warning {
|
||||||
|
background: #fff8df;
|
||||||
|
border-color: #f0db97;
|
||||||
|
color: var(--warning);
|
||||||
|
}
|
||||||
|
|
||||||
|
.table-wrap {
|
||||||
|
overflow-x: auto;
|
||||||
|
}
|
||||||
|
|
||||||
|
table {
|
||||||
|
width: 100%;
|
||||||
|
border-collapse: collapse;
|
||||||
|
min-width: 900px;
|
||||||
|
}
|
||||||
|
|
||||||
|
th,
|
||||||
|
td {
|
||||||
|
border-bottom: 1px solid var(--border);
|
||||||
|
padding: 0.55rem;
|
||||||
|
text-align: left;
|
||||||
|
vertical-align: middle;
|
||||||
|
}
|
||||||
|
|
||||||
|
th {
|
||||||
|
color: var(--muted);
|
||||||
|
font-weight: 600;
|
||||||
|
font-size: 0.88rem;
|
||||||
|
user-select: none;
|
||||||
|
cursor: pointer;
|
||||||
|
white-space: nowrap;
|
||||||
|
}
|
||||||
|
|
||||||
|
th:last-child {
|
||||||
|
cursor: default;
|
||||||
|
}
|
||||||
|
|
||||||
|
.status-cell,
|
||||||
|
.toggle-cell,
|
||||||
|
.actions {
|
||||||
|
white-space: nowrap;
|
||||||
|
}
|
||||||
|
|
||||||
|
.actions {
|
||||||
|
display: flex;
|
||||||
|
gap: 0.45rem;
|
||||||
|
}
|
||||||
|
|
||||||
|
.containers {
|
||||||
|
color: var(--muted);
|
||||||
|
max-width: 320px;
|
||||||
|
overflow: hidden;
|
||||||
|
text-overflow: ellipsis;
|
||||||
|
}
|
||||||
|
|
||||||
|
.chip {
|
||||||
|
display: inline-block;
|
||||||
|
border-radius: 999px;
|
||||||
|
padding: 0.2rem 0.55rem;
|
||||||
|
font-size: 0.78rem;
|
||||||
|
font-weight: 600;
|
||||||
|
}
|
||||||
|
|
||||||
|
.chip.used {
|
||||||
|
background: var(--chip-used);
|
||||||
|
color: #8a1f1f;
|
||||||
|
}
|
||||||
|
|
||||||
|
.chip.free {
|
||||||
|
background: var(--chip-free);
|
||||||
|
color: #145a2c;
|
||||||
|
}
|
||||||
|
|
||||||
|
.chip.unknown {
|
||||||
|
background: var(--chip-unknown);
|
||||||
|
color: #7d5b04;
|
||||||
|
}
|
||||||
|
|
||||||
|
.empty {
|
||||||
|
text-align: center;
|
||||||
|
color: var(--muted);
|
||||||
|
padding: 1rem;
|
||||||
|
}
|
||||||
|
|
||||||
|
@media (max-width: 760px) {
|
||||||
|
.app-shell {
|
||||||
|
padding: 1rem;
|
||||||
|
}
|
||||||
|
|
||||||
|
.topbar {
|
||||||
|
flex-direction: column;
|
||||||
|
align-items: stretch;
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,40 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
from pathlib import Path
|
||||||
|
import threading
|
||||||
|
from typing import Iterable
|
||||||
|
|
||||||
|
from app.models import IpEntry
|
||||||
|
|
||||||
|
|
||||||
|
class EntryStorage:
|
||||||
|
def __init__(self, file_path: str):
|
||||||
|
self._path = Path(file_path)
|
||||||
|
self._lock = threading.Lock()
|
||||||
|
|
||||||
|
def list_entries(self) -> list[IpEntry]:
|
||||||
|
with self._lock:
|
||||||
|
return self._read_entries_unlocked()
|
||||||
|
|
||||||
|
def save_entries(self, entries: Iterable[IpEntry]) -> None:
|
||||||
|
serializable = [entry.to_dict() for entry in entries]
|
||||||
|
with self._lock:
|
||||||
|
self._path.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
tmp_path = self._path.with_suffix(self._path.suffix + ".tmp")
|
||||||
|
tmp_path.write_text(json.dumps(serializable, indent=2, sort_keys=True), encoding="utf-8")
|
||||||
|
tmp_path.replace(self._path)
|
||||||
|
|
||||||
|
def _read_entries_unlocked(self) -> list[IpEntry]:
|
||||||
|
if not self._path.exists():
|
||||||
|
return []
|
||||||
|
|
||||||
|
raw = self._path.read_text(encoding="utf-8").strip()
|
||||||
|
if not raw:
|
||||||
|
return []
|
||||||
|
|
||||||
|
parsed = json.loads(raw)
|
||||||
|
if not isinstance(parsed, list):
|
||||||
|
raise ValueError(f"State file must be a list: {self._path}")
|
||||||
|
|
||||||
|
return [IpEntry.from_dict(item) for item in parsed]
|
||||||
@@ -0,0 +1,3 @@
|
|||||||
|
fastapi==0.116.1
|
||||||
|
uvicorn==0.35.0
|
||||||
|
dnspython==2.7.0
|
||||||
@@ -0,0 +1,149 @@
|
|||||||
|
name: docker-ip-addr-manager
|
||||||
|
|
||||||
|
services:
|
||||||
|
app:
|
||||||
|
image: joafri/docker-ip-addr-manager-app:main
|
||||||
|
build:
|
||||||
|
context: ./backend
|
||||||
|
dockerfile: Dockerfile
|
||||||
|
container_name: docker-ip-addr-manager
|
||||||
|
restart: unless-stopped
|
||||||
|
network_mode: host
|
||||||
|
cap_add:
|
||||||
|
- NET_ADMIN
|
||||||
|
security_opt:
|
||||||
|
- no-new-privileges:true
|
||||||
|
environment:
|
||||||
|
TZ: Europe/Stockholm
|
||||||
|
APP_PORT: "31810"
|
||||||
|
STATE_FILE: /data/entries.json
|
||||||
|
DOCKER_API_URL: unix:///var/run/docker.sock
|
||||||
|
DOCKER_TIMEOUT_SECONDS: "3"
|
||||||
|
DNS_PROVIDER: none
|
||||||
|
DNS_BASE_DOMAIN: home.arpa
|
||||||
|
DNS_TTL_SECONDS: "120"
|
||||||
|
DNS_SYNC_INTERVAL_SECONDS: "15"
|
||||||
|
ADGUARD_URL: http://127.0.0.1:3000
|
||||||
|
ADGUARD_USERNAME: ""
|
||||||
|
ADGUARD_PASSWORD: ""
|
||||||
|
ADGUARD_API_TOKEN: ""
|
||||||
|
RFC2136_SERVER: ""
|
||||||
|
RFC2136_ZONE: ""
|
||||||
|
RFC2136_PORT: "53"
|
||||||
|
RFC2136_TSIG_KEY_NAME: ""
|
||||||
|
RFC2136_TSIG_SECRET: ""
|
||||||
|
RFC2136_TSIG_ALGORITHM: hmac-sha256
|
||||||
|
volumes:
|
||||||
|
- type: bind
|
||||||
|
source: /DATA/AppData/$AppID/data
|
||||||
|
target: /data
|
||||||
|
- type: bind
|
||||||
|
source: /var/run/docker.sock
|
||||||
|
target: /var/run/docker.sock
|
||||||
|
read_only: true
|
||||||
|
x-casaos:
|
||||||
|
envs:
|
||||||
|
- container: APP_PORT
|
||||||
|
description:
|
||||||
|
en_us: HTTP port for the management UI
|
||||||
|
- container: DOCKER_API_URL
|
||||||
|
description:
|
||||||
|
en_us: Docker endpoint (unix socket default, optional tcp endpoint)
|
||||||
|
- container: DOCKER_TIMEOUT_SECONDS
|
||||||
|
description:
|
||||||
|
en_us: Timeout in seconds for Docker API requests
|
||||||
|
- container: DNS_PROVIDER
|
||||||
|
description:
|
||||||
|
en_us: DNS backend (none, adguard, rfc2136)
|
||||||
|
- container: DNS_BASE_DOMAIN
|
||||||
|
description:
|
||||||
|
en_us: Base domain for generated hostnames like <name>.<domain>
|
||||||
|
- container: DNS_TTL_SECONDS
|
||||||
|
description:
|
||||||
|
en_us: TTL in seconds for DNS A records
|
||||||
|
- container: DNS_SYNC_INTERVAL_SECONDS
|
||||||
|
description:
|
||||||
|
en_us: Background DNS reconcile interval in seconds
|
||||||
|
- container: ADGUARD_URL
|
||||||
|
description:
|
||||||
|
en_us: AdGuard Home URL for DNS_PROVIDER=adguard
|
||||||
|
- container: ADGUARD_USERNAME
|
||||||
|
description:
|
||||||
|
en_us: AdGuard Home username for DNS_PROVIDER=adguard
|
||||||
|
- container: ADGUARD_PASSWORD
|
||||||
|
description:
|
||||||
|
en_us: AdGuard Home password for DNS_PROVIDER=adguard
|
||||||
|
- container: RFC2136_SERVER
|
||||||
|
description:
|
||||||
|
en_us: RFC2136 nameserver host or IP
|
||||||
|
- container: RFC2136_ZONE
|
||||||
|
description:
|
||||||
|
en_us: RFC2136 zone name (for example home.arpa)
|
||||||
|
volumes:
|
||||||
|
- container: /data
|
||||||
|
description:
|
||||||
|
en_us: Persistent IP entry state
|
||||||
|
|
||||||
|
socket-proxy:
|
||||||
|
image: lscr.io/linuxserver/socket-proxy:version-3.0.9-r0
|
||||||
|
container_name: docker-ip-addr-manager-proxy
|
||||||
|
restart: unless-stopped
|
||||||
|
environment:
|
||||||
|
TZ: Europe/Stockholm
|
||||||
|
CONTAINERS: 1
|
||||||
|
INFO: 1
|
||||||
|
PING: 1
|
||||||
|
POST: 0
|
||||||
|
VERSION: 1
|
||||||
|
read_only: true
|
||||||
|
tmpfs:
|
||||||
|
- /run
|
||||||
|
ports:
|
||||||
|
- target: 2375
|
||||||
|
published: "2375"
|
||||||
|
host_ip: 127.0.0.1
|
||||||
|
protocol: tcp
|
||||||
|
volumes:
|
||||||
|
- type: bind
|
||||||
|
source: /var/run/docker.sock
|
||||||
|
target: /var/run/docker.sock
|
||||||
|
read_only: true
|
||||||
|
security_opt:
|
||||||
|
- no-new-privileges:true
|
||||||
|
cap_drop:
|
||||||
|
- ALL
|
||||||
|
|
||||||
|
x-casaos:
|
||||||
|
architectures:
|
||||||
|
- amd64
|
||||||
|
- arm64
|
||||||
|
- arm
|
||||||
|
main: app
|
||||||
|
category: phirna
|
||||||
|
author: Joachim Friberg
|
||||||
|
developer: Joachim Friberg
|
||||||
|
icon: https://cdn.simpleicons.org/docker
|
||||||
|
tagline:
|
||||||
|
en_us: Manage host LAN IP aliases for container port bindings
|
||||||
|
description:
|
||||||
|
en_us: >-
|
||||||
|
Adds/removes host interface IP aliases and shows whether an IP is used by Docker
|
||||||
|
container port bindings. Includes fail-closed disable/delete checks when usage cannot
|
||||||
|
be validated.
|
||||||
|
Start by adding a new IP Entry in this app and connect it to the appropriate Device.
|
||||||
|
Then install, or update, a zima app and choose network: bridge.
|
||||||
|
* Enter a name for the app, and the app can create DNS records as <name>.<DNS_BASE_DOMAIN> when DNS sync is enabled
|
||||||
|
* Click add, and a new row appears under.
|
||||||
|
* Click "Enable" to have this app setup the host to listen to this IP Address
|
||||||
|
* To to the ZimaOS App Store, choose an app, and do a "Custom Install"
|
||||||
|
* Add your chosen ip to all fields related to a host port
|
||||||
|
Such as the ip to the WebUI, and under "Port", theres a "Host" column.
|
||||||
|
!!! DON'T add the ip-address to the Container column. !!!
|
||||||
|
|
||||||
|
Note:
|
||||||
|
Due to a builtin "convenience" checker in ZimaOS, even if each container gets their own IP, the UI might still complain if two applications use the same port
|
||||||
|
title:
|
||||||
|
en_us: Docker IP Addr Manager
|
||||||
|
index: /
|
||||||
|
port_map: "31810"
|
||||||
|
scheme: http
|
||||||
@@ -0,0 +1,290 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from pathlib import Path
|
||||||
|
import sys
|
||||||
|
import tempfile
|
||||||
|
|
||||||
|
|
||||||
|
ROOT_DIR = Path(__file__).resolve().parents[1]
|
||||||
|
BACKEND_DIR = ROOT_DIR / "backend"
|
||||||
|
sys.path.insert(0, str(BACKEND_DIR))
|
||||||
|
|
||||||
|
from app.docker_api import DockerApiError, DockerUsageResolver
|
||||||
|
from app.dns_sync import DnsSyncError
|
||||||
|
from app.models import IpEntry
|
||||||
|
from app.service import ConflictError, DependencyError, EntryService
|
||||||
|
from app.storage import EntryStorage
|
||||||
|
|
||||||
|
|
||||||
|
class FakeDockerClient:
|
||||||
|
def list_containers(self, include_stopped: bool = True):
|
||||||
|
assert include_stopped is True
|
||||||
|
return [
|
||||||
|
{"Id": "a1", "Names": ["/alpha"]},
|
||||||
|
{"Id": "b2", "Names": ["/beta"]},
|
||||||
|
]
|
||||||
|
|
||||||
|
def inspect_container(self, container_id: str):
|
||||||
|
if container_id == "a1":
|
||||||
|
return {
|
||||||
|
"Name": "/alpha",
|
||||||
|
"NetworkSettings": {
|
||||||
|
"Ports": {
|
||||||
|
"8080/tcp": [{"HostIp": "10.0.4.2", "HostPort": "8080"}],
|
||||||
|
"8443/tcp": [{"HostIp": "0.0.0.0", "HostPort": "8443"}],
|
||||||
|
}
|
||||||
|
},
|
||||||
|
}
|
||||||
|
if container_id == "b2":
|
||||||
|
return {
|
||||||
|
"Name": "/beta",
|
||||||
|
"NetworkSettings": {
|
||||||
|
"Ports": {
|
||||||
|
"9000/tcp": [{"HostIp": "10.0.4.3", "HostPort": "9000"}],
|
||||||
|
}
|
||||||
|
},
|
||||||
|
}
|
||||||
|
raise AssertionError(f"unexpected container_id: {container_id}")
|
||||||
|
|
||||||
|
|
||||||
|
class FakeUsageResolver:
|
||||||
|
def __init__(self, mapping=None, should_fail=False):
|
||||||
|
self._mapping = mapping or {}
|
||||||
|
self._should_fail = should_fail
|
||||||
|
|
||||||
|
def resolve_ip_usage(self, ips: set[str]):
|
||||||
|
if self._should_fail:
|
||||||
|
raise DockerApiError("docker unreachable")
|
||||||
|
response = {ip: set() for ip in ips}
|
||||||
|
for ip in ips:
|
||||||
|
response[ip] = set(self._mapping.get(ip, set()))
|
||||||
|
return response
|
||||||
|
|
||||||
|
|
||||||
|
class FakeIpManager:
|
||||||
|
def __init__(self):
|
||||||
|
self.present = set()
|
||||||
|
|
||||||
|
def is_present(self, ip: str, cidr: int, device: str):
|
||||||
|
return (ip, cidr, device) in self.present
|
||||||
|
|
||||||
|
def ensure_present(self, ip: str, cidr: int, device: str):
|
||||||
|
self.present.add((ip, cidr, device))
|
||||||
|
|
||||||
|
def ensure_absent(self, ip: str, cidr: int, device: str):
|
||||||
|
self.present.discard((ip, cidr, device))
|
||||||
|
|
||||||
|
|
||||||
|
class FakeDnsProvider:
|
||||||
|
def __init__(self, fail_upsert=False, fail_delete=False):
|
||||||
|
self.records = {}
|
||||||
|
self.upserts = []
|
||||||
|
self.deletes = []
|
||||||
|
self.fail_upsert = fail_upsert
|
||||||
|
self.fail_delete = fail_delete
|
||||||
|
|
||||||
|
def upsert_a_record(self, fqdn: str, ip: str, ttl: int):
|
||||||
|
if self.fail_upsert:
|
||||||
|
raise DnsSyncError("upsert failed")
|
||||||
|
self.records[fqdn] = ip
|
||||||
|
self.upserts.append((fqdn, ip, ttl))
|
||||||
|
|
||||||
|
def delete_a_record(self, fqdn: str):
|
||||||
|
if self.fail_delete:
|
||||||
|
raise DnsSyncError("delete failed")
|
||||||
|
self.records.pop(fqdn, None)
|
||||||
|
self.deletes.append(fqdn)
|
||||||
|
|
||||||
|
|
||||||
|
def assert_true(condition, message):
|
||||||
|
if not condition:
|
||||||
|
raise AssertionError(message)
|
||||||
|
|
||||||
|
|
||||||
|
def build_service(
|
||||||
|
tmp_path: Path,
|
||||||
|
entries=None,
|
||||||
|
usage_resolver=None,
|
||||||
|
ip_manager=None,
|
||||||
|
dns_provider=None,
|
||||||
|
dns_base_domain="home.arpa",
|
||||||
|
):
|
||||||
|
storage = EntryStorage(str(tmp_path / "entries.json"))
|
||||||
|
if entries:
|
||||||
|
storage.save_entries(entries)
|
||||||
|
|
||||||
|
resolver = usage_resolver or FakeUsageResolver()
|
||||||
|
ipm = ip_manager or FakeIpManager()
|
||||||
|
|
||||||
|
return EntryService(
|
||||||
|
storage=storage,
|
||||||
|
usage_resolver=resolver,
|
||||||
|
ip_manager=ipm,
|
||||||
|
interface_provider=lambda: ["eth0", "eth1"],
|
||||||
|
dns_provider=dns_provider,
|
||||||
|
dns_base_domain=dns_base_domain,
|
||||||
|
dns_ttl_seconds=120,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def test_exact_hostip_match_only():
|
||||||
|
resolver = DockerUsageResolver(FakeDockerClient())
|
||||||
|
usage = resolver.resolve_ip_usage({"10.0.4.2", "10.0.4.3"})
|
||||||
|
|
||||||
|
assert_true(usage["10.0.4.2"] == {"alpha"}, "10.0.4.2 should match alpha")
|
||||||
|
assert_true(usage["10.0.4.3"] == {"beta"}, "10.0.4.3 should match beta")
|
||||||
|
|
||||||
|
|
||||||
|
def test_disable_blocked_when_used(tmp_path: Path):
|
||||||
|
entry = IpEntry(id="one", name="lan-a", ip="10.0.4.2", cidr=16, device="eth0", enabled=True)
|
||||||
|
resolver = FakeUsageResolver(mapping={"10.0.4.2": {"alpha"}})
|
||||||
|
ip_manager = FakeIpManager()
|
||||||
|
ip_manager.present.add(("10.0.4.2", 16, "eth0"))
|
||||||
|
|
||||||
|
service = build_service(tmp_path, entries=[entry], usage_resolver=resolver, ip_manager=ip_manager)
|
||||||
|
|
||||||
|
blocked = False
|
||||||
|
try:
|
||||||
|
service.set_enabled("one", enabled=False)
|
||||||
|
except ConflictError:
|
||||||
|
blocked = True
|
||||||
|
|
||||||
|
assert_true(blocked, "disable must be blocked when entry is used")
|
||||||
|
assert_true(("10.0.4.2", 16, "eth0") in ip_manager.present, "IP must stay present when disable is blocked")
|
||||||
|
|
||||||
|
|
||||||
|
def test_disable_blocked_when_docker_check_fails(tmp_path: Path):
|
||||||
|
entry = IpEntry(id="two", name="lan-b", ip="10.0.4.8", cidr=16, device="eth0", enabled=True)
|
||||||
|
resolver = FakeUsageResolver(should_fail=True)
|
||||||
|
ip_manager = FakeIpManager()
|
||||||
|
ip_manager.present.add(("10.0.4.8", 16, "eth0"))
|
||||||
|
|
||||||
|
service = build_service(tmp_path, entries=[entry], usage_resolver=resolver, ip_manager=ip_manager)
|
||||||
|
|
||||||
|
blocked = False
|
||||||
|
try:
|
||||||
|
service.set_enabled("two", enabled=False)
|
||||||
|
except DependencyError:
|
||||||
|
blocked = True
|
||||||
|
|
||||||
|
assert_true(blocked, "disable must fail-closed when Docker usage check fails")
|
||||||
|
|
||||||
|
|
||||||
|
def test_delete_blocked_when_enabled(tmp_path: Path):
|
||||||
|
entry = IpEntry(id="three", name="lan-c", ip="10.0.4.9", cidr=16, device="eth0", enabled=True)
|
||||||
|
service = build_service(tmp_path, entries=[entry])
|
||||||
|
|
||||||
|
blocked = False
|
||||||
|
try:
|
||||||
|
service.delete_entry("three")
|
||||||
|
except ConflictError:
|
||||||
|
blocked = True
|
||||||
|
|
||||||
|
assert_true(blocked, "delete must be blocked while entry is enabled")
|
||||||
|
|
||||||
|
|
||||||
|
def test_reconcile_reapplies_enabled(tmp_path: Path):
|
||||||
|
entry = IpEntry(id="four", name="lan-d", ip="10.0.4.10", cidr=16, device="eth0", enabled=True)
|
||||||
|
ip_manager = FakeIpManager()
|
||||||
|
service = build_service(tmp_path, entries=[entry], ip_manager=ip_manager)
|
||||||
|
|
||||||
|
errors = service.reconcile_enabled_entries()
|
||||||
|
|
||||||
|
assert_true(errors == [], "reconcile should succeed without errors")
|
||||||
|
assert_true(("10.0.4.10", 16, "eth0") in ip_manager.present, "enabled IP must be re-applied on startup reconcile")
|
||||||
|
|
||||||
|
|
||||||
|
def test_dns_upsert_on_enable_when_used(tmp_path: Path):
|
||||||
|
entry = IpEntry(id="dns1", name="Lan App", ip="10.0.4.20", cidr=16, device="eth0", enabled=False)
|
||||||
|
resolver = FakeUsageResolver(mapping={"10.0.4.20": {"nginx"}})
|
||||||
|
ip_manager = FakeIpManager()
|
||||||
|
dns = FakeDnsProvider()
|
||||||
|
service = build_service(tmp_path, entries=[entry], usage_resolver=resolver, ip_manager=ip_manager, dns_provider=dns)
|
||||||
|
|
||||||
|
service.set_enabled("dns1", enabled=True)
|
||||||
|
|
||||||
|
assert_true(dns.records.get("lan-app.home.arpa") == "10.0.4.20", "DNS record should be created on enable+used")
|
||||||
|
|
||||||
|
|
||||||
|
def test_dns_no_upsert_on_enable_when_unused(tmp_path: Path):
|
||||||
|
entry = IpEntry(id="dns2", name="Lan App 2", ip="10.0.4.21", cidr=16, device="eth0", enabled=False)
|
||||||
|
resolver = FakeUsageResolver(mapping={})
|
||||||
|
ip_manager = FakeIpManager()
|
||||||
|
dns = FakeDnsProvider()
|
||||||
|
service = build_service(tmp_path, entries=[entry], usage_resolver=resolver, ip_manager=ip_manager, dns_provider=dns)
|
||||||
|
|
||||||
|
service.set_enabled("dns2", enabled=True)
|
||||||
|
|
||||||
|
assert_true("lan-app-2.home.arpa" not in dns.records, "Unused entries must not create DNS record")
|
||||||
|
|
||||||
|
|
||||||
|
def test_dns_reconcile_deletes_when_no_longer_used(tmp_path: Path):
|
||||||
|
entry = IpEntry(id="dns3", name="Lan App 3", ip="10.0.4.22", cidr=16, device="eth0", enabled=True)
|
||||||
|
resolver = FakeUsageResolver(mapping={"10.0.4.22": {"nginx"}})
|
||||||
|
dns = FakeDnsProvider()
|
||||||
|
service = build_service(tmp_path, entries=[entry], usage_resolver=resolver, dns_provider=dns)
|
||||||
|
|
||||||
|
service.reconcile_dns_records()
|
||||||
|
assert_true(dns.records.get("lan-app-3.home.arpa") == "10.0.4.22", "record should exist after used reconcile")
|
||||||
|
|
||||||
|
resolver._mapping = {}
|
||||||
|
service.reconcile_dns_records()
|
||||||
|
assert_true("lan-app-3.home.arpa" not in dns.records, "record should be removed when usage disappears")
|
||||||
|
|
||||||
|
|
||||||
|
def test_dns_fail_closed_rolls_back_enable(tmp_path: Path):
|
||||||
|
entry = IpEntry(id="dns4", name="Lan App 4", ip="10.0.4.23", cidr=16, device="eth0", enabled=False)
|
||||||
|
resolver = FakeUsageResolver(mapping={"10.0.4.23": {"nginx"}})
|
||||||
|
ip_manager = FakeIpManager()
|
||||||
|
dns = FakeDnsProvider(fail_upsert=True)
|
||||||
|
service = build_service(tmp_path, entries=[entry], usage_resolver=resolver, ip_manager=ip_manager, dns_provider=dns)
|
||||||
|
|
||||||
|
failed = False
|
||||||
|
try:
|
||||||
|
service.set_enabled("dns4", enabled=True)
|
||||||
|
except ConflictError:
|
||||||
|
failed = True
|
||||||
|
|
||||||
|
assert_true(failed, "enable must fail-closed when DNS upsert fails")
|
||||||
|
current = service.list_entries().items[0]
|
||||||
|
assert_true(not current.enabled, "entry must roll back to disabled on DNS failure")
|
||||||
|
assert_true(not ip_manager.is_present("10.0.4.23", 16, "eth0"), "IP presence must roll back on DNS failure")
|
||||||
|
|
||||||
|
|
||||||
|
def test_dns_fail_closed_blocks_delete(tmp_path: Path):
|
||||||
|
entry = IpEntry(id="dns5", name="Lan App 5", ip="10.0.4.24", cidr=16, device="eth0", enabled=False)
|
||||||
|
resolver = FakeUsageResolver(mapping={})
|
||||||
|
dns = FakeDnsProvider(fail_delete=True)
|
||||||
|
service = build_service(tmp_path, entries=[entry], usage_resolver=resolver, dns_provider=dns)
|
||||||
|
|
||||||
|
failed = False
|
||||||
|
try:
|
||||||
|
service.delete_entry("dns5")
|
||||||
|
except ConflictError:
|
||||||
|
failed = True
|
||||||
|
|
||||||
|
assert_true(failed, "delete must fail-closed when DNS cleanup fails")
|
||||||
|
assert_true(len(service.list_entries().items) == 1, "entry must remain when delete fails")
|
||||||
|
|
||||||
|
|
||||||
|
def main():
|
||||||
|
test_exact_hostip_match_only()
|
||||||
|
|
||||||
|
with tempfile.TemporaryDirectory() as tmp:
|
||||||
|
tmp_path = Path(tmp)
|
||||||
|
test_disable_blocked_when_used(tmp_path)
|
||||||
|
test_disable_blocked_when_docker_check_fails(tmp_path)
|
||||||
|
test_delete_blocked_when_enabled(tmp_path)
|
||||||
|
test_reconcile_reapplies_enabled(tmp_path)
|
||||||
|
test_dns_upsert_on_enable_when_used(tmp_path)
|
||||||
|
test_dns_no_upsert_on_enable_when_unused(tmp_path)
|
||||||
|
test_dns_reconcile_deletes_when_no_longer_used(tmp_path)
|
||||||
|
test_dns_fail_closed_rolls_back_enable(tmp_path)
|
||||||
|
test_dns_fail_closed_blocks_delete(tmp_path)
|
||||||
|
|
||||||
|
print("Integration tests passed")
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
main()
|
||||||
@@ -0,0 +1,5 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
ROOT_DIR="$(cd "$(dirname "$0")/.." && pwd)"
|
||||||
|
PYTHONDONTWRITEBYTECODE=1 python3 "$ROOT_DIR/tests/integration_tests.py"
|
||||||
@@ -0,0 +1,88 @@
|
|||||||
|
# Llama Server
|
||||||
|
|
||||||
|
Local LLM inference server using llama.cpp. Serves GGUF models via OpenAI-compatible REST API.
|
||||||
|
|
||||||
|
**Image**: `ghcr.io/ggml-org/llama.cpp:server-b8840` (CPU-only, AVX2/AVX512)
|
||||||
|
|
||||||
|
## Purpose
|
||||||
|
|
||||||
|
- **Port**: 8080 (TCP)
|
||||||
|
- **Memory**: 8G reservation (7B Q4 models fit in ~6-7GB RAM)
|
||||||
|
- **Category**: AI / LLM inference
|
||||||
|
|
||||||
|
CPU-only inference with AVX2/AVX512 auto-detection. No GPU needed.
|
||||||
|
|
||||||
|
## Model Setup
|
||||||
|
|
||||||
|
llama-server does not bundle models. You must download GGUF files manually.
|
||||||
|
|
||||||
|
SSH into your ZimaOS device and run:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Create models directory
|
||||||
|
mkdir -p /DATA/AppData/llama-server/models
|
||||||
|
|
||||||
|
# Example: Download Llama 3.2 3B Q4_K_M (~1.8GB)
|
||||||
|
curl -L -o /DATA/AppData/llama-server/models/llama-3.2-3b-q4_k_m.gguf \
|
||||||
|
"https://huggingface.co/QuantFactory/Llama-3.2-3B-Instruct-GGUF/resolve/main/Llama-3.2-3B-Instruct.Q4_K_M.gguf"
|
||||||
|
```
|
||||||
|
|
||||||
|
## Recommended Models for 16GB RAM
|
||||||
|
|
||||||
|
| Model | Size | Quant | RAM Needed | Speed (est.) |
|
||||||
|
|-------|------|-------|------------|--------------|
|
||||||
|
| Llama 3.2 3B | 1.8GB | Q4_K_M | ~4GB | ~15-20 tok/s |
|
||||||
|
| Phi-3.5 Mini 3B | 1.8GB | Q4_K_M | ~4GB | ~15-20 tok/s |
|
||||||
|
| Mistral 7B | 4.1GB | Q4_K_M | ~6-7GB | ~8-12 tok/s |
|
||||||
|
| Qwen 2.5 7B | 4.4GB | Q4_K_M | ~6-7GB | ~8-12 tok/s |
|
||||||
|
|
||||||
|
For 7B models, close other apps to free RAM. 8G reservation leaves headroom.
|
||||||
|
|
||||||
|
## Environment Variables
|
||||||
|
|
||||||
|
| Variable | Default | Description |
|
||||||
|
|----------|---------|-------------|
|
||||||
|
| `MODEL` | `llama-3.2-3b-q4_k_m.gguf` | Model filename in `/models` |
|
||||||
|
| `CTX_SIZE` | `2048` | Context window size (tokens) |
|
||||||
|
| `N_THREADS` | `0` | CPU threads (0 = auto) |
|
||||||
|
| `HOST` | `0.0.0.0` | Listen address |
|
||||||
|
| `PORT` | `8080` | API port |
|
||||||
|
| `MAX_TOKENS` | `512` | Max tokens per response |
|
||||||
|
|
||||||
|
Change `MODEL` to match your downloaded file. Restart container after changing.
|
||||||
|
|
||||||
|
## API Testing
|
||||||
|
|
||||||
|
Once running, test the API:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Check server info
|
||||||
|
curl http://localhost:8080/v1/models
|
||||||
|
|
||||||
|
# Chat completions (OpenAI-compatible)
|
||||||
|
curl http://localhost:8080/v1/chat/completions \
|
||||||
|
-H "Content-Type: application/json" \
|
||||||
|
-d '{
|
||||||
|
"model": "llama-3.2-3b-q4_k_m.gguf",
|
||||||
|
"messages": [{"role": "user", "content": "Hello, who are you?"}],
|
||||||
|
"max_tokens": 128
|
||||||
|
}'
|
||||||
|
```
|
||||||
|
|
||||||
|
## Volumes
|
||||||
|
|
||||||
|
| Path | Description |
|
||||||
|
|------|-------------|
|
||||||
|
| `/models` | GGUF model files |
|
||||||
|
| `/logs` | Server log output |
|
||||||
|
|
||||||
|
## Architecture
|
||||||
|
|
||||||
|
- `amd64` (Intel/AMD x86_64)
|
||||||
|
- `arm64` (Apple Silicon, ARM servers)
|
||||||
|
|
||||||
|
## Security
|
||||||
|
|
||||||
|
- `security_opt: no-new-privileges:true`
|
||||||
|
- `cap_drop: ALL`
|
||||||
|
- CPU-only, no privileged access needed
|
||||||
@@ -0,0 +1,82 @@
|
|||||||
|
name: llama-server
|
||||||
|
|
||||||
|
services:
|
||||||
|
llama-server:
|
||||||
|
image: ghcr.io/ggml-org/llama.cpp:server-b8840@sha256:99d2554c4c8d5339649dde530056cf10771823d7cd983dbd0441da9c419976b1
|
||||||
|
container_name: llama-server
|
||||||
|
restart: unless-stopped
|
||||||
|
environment:
|
||||||
|
TZ: Europe/Stockholm
|
||||||
|
MODEL: llama-3.2-3b-q4_k_m.gguf
|
||||||
|
CTX_SIZE: "2048"
|
||||||
|
N_THREADS: "0"
|
||||||
|
HOST: 0.0.0.0
|
||||||
|
PORT: "8080"
|
||||||
|
MAX_TOKENS: "512"
|
||||||
|
ports:
|
||||||
|
- target: 8080
|
||||||
|
published: "8080"
|
||||||
|
protocol: tcp
|
||||||
|
volumes:
|
||||||
|
- type: bind
|
||||||
|
source: /DATA/AppData/$AppID/models
|
||||||
|
target: /models
|
||||||
|
- type: bind
|
||||||
|
source: /DATA/AppData/$AppID/logs
|
||||||
|
target: /logs
|
||||||
|
deploy:
|
||||||
|
resources:
|
||||||
|
reservations:
|
||||||
|
memory: 8G
|
||||||
|
security_opt:
|
||||||
|
- no-new-privileges:true
|
||||||
|
cap_drop:
|
||||||
|
- ALL
|
||||||
|
x-casaos:
|
||||||
|
envs:
|
||||||
|
- container: MODEL
|
||||||
|
description:
|
||||||
|
en_us: Model filename inside /models (e.g. llama-3.2-3b-q4_k_m.gguf). Download GGUF files manually into /models.
|
||||||
|
- container: CTX_SIZE
|
||||||
|
description:
|
||||||
|
en_us: Context window size in tokens
|
||||||
|
- container: N_THREADS
|
||||||
|
description:
|
||||||
|
en_us: CPU threads (0 = auto-detect all cores)
|
||||||
|
- container: MAX_TOKENS
|
||||||
|
description:
|
||||||
|
en_us: Maximum tokens to generate per response
|
||||||
|
- container: TZ
|
||||||
|
description:
|
||||||
|
en_us: Timezone, for example Europe/Stockholm
|
||||||
|
ports:
|
||||||
|
- container: "8080"
|
||||||
|
description:
|
||||||
|
en_us: llama.cpp REST API port
|
||||||
|
volumes:
|
||||||
|
- container: /models
|
||||||
|
description:
|
||||||
|
en_us: Model GGUF files directory
|
||||||
|
- container: /logs
|
||||||
|
description:
|
||||||
|
en_us: Server log output
|
||||||
|
|
||||||
|
x-casaos:
|
||||||
|
architectures:
|
||||||
|
- amd64
|
||||||
|
- arm64
|
||||||
|
main: llama-server
|
||||||
|
category: phirna
|
||||||
|
author: Joachim Friberg
|
||||||
|
developer: Joachim Friberg
|
||||||
|
icon: https://cdn.simpleicons.org/llama
|
||||||
|
tagline:
|
||||||
|
en_us: CPU-only LLM inference server with REST API
|
||||||
|
description:
|
||||||
|
en_us: >
|
||||||
|
Local LLM inference server using llama.cpp. Serves GGUF models via OpenAI-compatible REST API.
|
||||||
|
CPU-only with AVX2/AVX512 optimization. Requires manual model download.
|
||||||
|
title:
|
||||||
|
en_us: Llama Server
|
||||||
|
index: /
|
||||||
|
port_map: "8080"
|
||||||
@@ -0,0 +1,73 @@
|
|||||||
|
# OpenWebUI
|
||||||
|
|
||||||
|
Modern chat web interface for local LLMs. Connects to llama-server via Docker internal networking.
|
||||||
|
|
||||||
|
## Purpose
|
||||||
|
|
||||||
|
- **Port**: 3000 (TCP)
|
||||||
|
- **Memory**: 2G reservation
|
||||||
|
- **Category**: AI / LLM UI
|
||||||
|
|
||||||
|
Requires the **llama-server** app to be running first. Connects to `http://llama-server:8080` internally.
|
||||||
|
|
||||||
|
## Prerequisites
|
||||||
|
|
||||||
|
1. Deploy and start **llama-server** app first
|
||||||
|
2. Download a GGUF model into llama-server's `/models` directory
|
||||||
|
3. Ensure llama-server container is healthy
|
||||||
|
|
||||||
|
## Access
|
||||||
|
|
||||||
|
Open in browser:
|
||||||
|
|
||||||
|
```
|
||||||
|
http://<your-zimaos-host>:3000
|
||||||
|
```
|
||||||
|
|
||||||
|
First run may take a moment to initialize.
|
||||||
|
|
||||||
|
## Environment Variables
|
||||||
|
|
||||||
|
| Variable | Default | Description |
|
||||||
|
|----------|---------|-------------|
|
||||||
|
| `OLLAMA_BASE_URL` | `http://llama-server:8080` | Internal URL to llama-server API |
|
||||||
|
| `WEBUI_PORT` | `3000` | Container listen port |
|
||||||
|
| `TZ` | `Europe/Stockholm` | Timezone |
|
||||||
|
|
||||||
|
## If Connection Fails
|
||||||
|
|
||||||
|
1. Verify llama-server is running: `docker ps | grep llama-server`
|
||||||
|
2. Check llama-server logs: `docker logs llama-server`
|
||||||
|
3. Ensure llama-server MODEL env matches your downloaded file
|
||||||
|
4. From ZimaOS shell, test connectivity:
|
||||||
|
```bash
|
||||||
|
curl http://llama-server:8080/v1/models
|
||||||
|
```
|
||||||
|
|
||||||
|
## Volumes
|
||||||
|
|
||||||
|
| Path | Description |
|
||||||
|
|------|-------------|
|
||||||
|
| `/app/backend/data` | OpenWebUI persistent data (chat history, settings) |
|
||||||
|
|
||||||
|
## Architecture
|
||||||
|
|
||||||
|
- `amd64` (Intel/AMD x86_64)
|
||||||
|
- `arm64` (Apple Silicon, ARM servers)
|
||||||
|
|
||||||
|
## Security
|
||||||
|
|
||||||
|
- `security_opt: no-new-privileges:true`
|
||||||
|
- `cap_drop: ALL`
|
||||||
|
|
||||||
|
## Troubleshooting
|
||||||
|
|
||||||
|
**"Cannot connect to LLM" error in UI**
|
||||||
|
- Verify llama-server is running before open-webui
|
||||||
|
- Check that `OLLAMA_BASE_URL` is set to `http://llama-server:8080`
|
||||||
|
- Verify model file exists in `/DATA/AppData/llama-server/models/`
|
||||||
|
|
||||||
|
**Slow responses**
|
||||||
|
- 7B models on CPU are limited by single-thread performance
|
||||||
|
- 3B models recommended for interactive speeds (~15+ tok/s)
|
||||||
|
- Close other apps to free RAM
|
||||||
@@ -0,0 +1,68 @@
|
|||||||
|
name: open-webui
|
||||||
|
|
||||||
|
services:
|
||||||
|
open-webui:
|
||||||
|
image: ghcr.io/open-webui/open-webui:main
|
||||||
|
container_name: open-webui
|
||||||
|
restart: unless-stopped
|
||||||
|
environment:
|
||||||
|
TZ: Europe/Stockholm
|
||||||
|
OLLAMA_BASE_URL: http://llama-server:8080
|
||||||
|
WEBUI_PORT: "3000"
|
||||||
|
ports:
|
||||||
|
- target: 3000
|
||||||
|
published: "3000"
|
||||||
|
protocol: tcp
|
||||||
|
volumes:
|
||||||
|
- type: bind
|
||||||
|
source: /DATA/AppData/$AppID/data
|
||||||
|
target: /app/backend/data
|
||||||
|
deploy:
|
||||||
|
resources:
|
||||||
|
reservations:
|
||||||
|
memory: 2G
|
||||||
|
depends_on:
|
||||||
|
- llama-server
|
||||||
|
security_opt:
|
||||||
|
- no-new-privileges:true
|
||||||
|
cap_drop:
|
||||||
|
- ALL
|
||||||
|
x-casaos:
|
||||||
|
envs:
|
||||||
|
- container: OLLAMA_BASE_URL
|
||||||
|
description:
|
||||||
|
en_us: Internal URL to llama-server API (http://llama-server:8080)
|
||||||
|
- container: WEBUI_PORT
|
||||||
|
description:
|
||||||
|
en_us: Web UI listen port inside container
|
||||||
|
- container: TZ
|
||||||
|
description:
|
||||||
|
en_us: Timezone, for example Europe/Stockholm
|
||||||
|
ports:
|
||||||
|
- container: "3000"
|
||||||
|
description:
|
||||||
|
en_us: OpenWebUI web interface port
|
||||||
|
volumes:
|
||||||
|
- container: /app/backend/data
|
||||||
|
description:
|
||||||
|
en_us: OpenWebUI persistent data (chat history, settings)
|
||||||
|
|
||||||
|
x-casaos:
|
||||||
|
architectures:
|
||||||
|
- amd64
|
||||||
|
- arm64
|
||||||
|
main: open-webui
|
||||||
|
category: phirna
|
||||||
|
author: Joachim Friberg
|
||||||
|
developer: Joachim Friberg
|
||||||
|
icon: https://cdn.simpleicons.org/webui
|
||||||
|
tagline:
|
||||||
|
en_us: Modern chat UI for local LLMs
|
||||||
|
description:
|
||||||
|
en_us: >
|
||||||
|
OpenWebUI provides a modern, feature-rich web interface for interacting with local LLMs.
|
||||||
|
Connect to llama-server or any OpenAI-compatible API. Requires llama-server app to be running first.
|
||||||
|
title:
|
||||||
|
en_us: OpenWebUI
|
||||||
|
index: /
|
||||||
|
port_map: "3000"
|
||||||
@@ -0,0 +1,143 @@
|
|||||||
|
# Snacks
|
||||||
|
|
||||||
|
Automated video library encoder with hardware acceleration (NVENC, QSV, VAAPI, AMF).
|
||||||
|
|
||||||
|
## Purpose
|
||||||
|
|
||||||
|
Snacks batch-transcodes video libraries using FFmpeg with hardware acceleration.
|
||||||
|
It monitors directories, skips already-encoded files, retries with fallbacks, and supports distributed cluster encoding across multiple ZimaOS nodes.
|
||||||
|
|
||||||
|
## Port
|
||||||
|
|
||||||
|
- `6767/tcp` — Web UI at `http://localhost:6767`
|
||||||
|
|
||||||
|
## Volumes
|
||||||
|
|
||||||
|
| Host path | Container path | Description |
|
||||||
|
|---|---|---|
|
||||||
|
| `/DATA/AppData/$AppID/media` | `/app/work/uploads` | Media library — source files to encode |
|
||||||
|
| `/DATA/AppData/$AppID/logs` | `/app/work/logs` | Transcoding logs |
|
||||||
|
| `/DATA/AppData/$AppID/config` | `/app/work/config` | Settings and SQLite database |
|
||||||
|
|
||||||
|
## Hardware Acceleration
|
||||||
|
|
||||||
|
Snacks uses GPU encoding via `/dev/dri`:
|
||||||
|
|
||||||
|
| Driver | Codecs | Devices |
|
||||||
|
|---|---|---|
|
||||||
|
| VAAPI (Linux) | H.265, H.264 | Intel iHD/i965, AMD VAAPI |
|
||||||
|
| QSV (Intel) | H.265, H.264 | Intel Quick Sync Video |
|
||||||
|
| NVENC (NVIDIA) | H.265, H.264 | NVIDIA GPUs via CUDA |
|
||||||
|
| AMF (AMD) | H.265, H.264 | AMD GPUs |
|
||||||
|
|
||||||
|
Auto-detection runs on first encode and picks the best available encoder.
|
||||||
|
|
||||||
|
## Cluster Mode
|
||||||
|
|
||||||
|
Snacks supports distributed encoding across multiple ZimaOS nodes.
|
||||||
|
|
||||||
|
- Nodes discover each other via UDP broadcast on the LAN
|
||||||
|
- One instance acts as coordinator; others are workers
|
||||||
|
- Jobs are assigned automatically; failed nodes are re-assigned
|
||||||
|
- A shared secret authenticates intra-cluster communication
|
||||||
|
|
||||||
|
**UDP broadcast requirement**: Cluster mode requires `network_mode: host` — bridge mode blocks LAN broadcast discovery, making nodes invisible to each other.
|
||||||
|
|
||||||
|
## Health Check
|
||||||
|
|
||||||
|
`http://localhost:6767/Home/Health` — returns HTTP 200 when the backend is ready.
|
||||||
|
|
||||||
|
## Privilegier och säkerhet
|
||||||
|
|
||||||
|
Aktiva säkerhetsinställningar i denna app:
|
||||||
|
|
||||||
|
- `security_opt: ["no-new-privileges:true"]`
|
||||||
|
- `cap_drop: ["ALL"]`
|
||||||
|
- `privileged: true`
|
||||||
|
- `network_mode: host`
|
||||||
|
- Device mount: `/dev/dri:/dev/dri`
|
||||||
|
|
||||||
|
Motivering:
|
||||||
|
|
||||||
|
- `no-new-privileges:true` och `cap_drop: ["ALL"]` kompenserar med lägsta möjliga capability-yta.
|
||||||
|
- Isolerad data-path under `/DATA/AppData/$AppID/...`.
|
||||||
|
|
||||||
|
## Säkerhetsavvikelser
|
||||||
|
|
||||||
|
### 1. `network_mode: host`
|
||||||
|
|
||||||
|
**Varför det behövs:**
|
||||||
|
|
||||||
|
- Snacks cluster nodes discover each other via UDP broadcast on the local network.
|
||||||
|
- Bridge mode only forwards unicast traffic; broadcast packets never reach other nodes.
|
||||||
|
- Without host networking, cluster mode is non-functional.
|
||||||
|
|
||||||
|
**Alternativ som utvärderats:**
|
||||||
|
|
||||||
|
- Bridge mode with port exposure: broadcasts are not forwarded by the Docker bridge.
|
||||||
|
- Static IP configuration: requires manual node addressing and is error-prone.
|
||||||
|
- Multicast DNS (mDNS): not supported by Docker bridge in all deployments.
|
||||||
|
|
||||||
|
**Risker:**
|
||||||
|
|
||||||
|
- Container has full access to all host ports.
|
||||||
|
- No network isolation between Snacks and other services on the host.
|
||||||
|
- If the container is compromised, the attacker has host network access.
|
||||||
|
|
||||||
|
**Riskreducering:**
|
||||||
|
|
||||||
|
- `cap_drop: ["ALL"]` minimizes syscall surface.
|
||||||
|
- `no-new-privileges:true` prevents privilege escalation.
|
||||||
|
- No sensitive host directories are mounted beyond the app-specific volumes.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### 2. `privileged: true`
|
||||||
|
|
||||||
|
**Varför det behävs:**
|
||||||
|
|
||||||
|
- `/dev/dri` (Direct Rendering Infrastructure) is required for VAAPI/QSV hardware acceleration.
|
||||||
|
- On standard Linux, this device is accessible without privileged mode if the user is in the `video` or `render` group.
|
||||||
|
- ZimaOS does not reliably provide these groups in the container runtime context, making `privileged: true` the only reliable way to grant device access.
|
||||||
|
|
||||||
|
**Alternativ som utvärderats:**
|
||||||
|
|
||||||
|
- `security_opt: ["apparmor:..."]` with specific `/dev/dri` access: not reliably portable across ZimaOS kernel configurations.
|
||||||
|
- Pre-create device nodes with specific permissions: does not work dynamically when the device appears.
|
||||||
|
- Skip hardware acceleration (software encoding only): defeats the primary purpose of the app.
|
||||||
|
|
||||||
|
**Risker:**
|
||||||
|
|
||||||
|
- Container has full root capabilities on the host.
|
||||||
|
- If container is compromised, attacker has theoretical access to all host resources.
|
||||||
|
- Hardware acceleration devices can be accessed directly.
|
||||||
|
|
||||||
|
**Riskreducering:**
|
||||||
|
|
||||||
|
- `cap_drop: ["ALL"]` drops all capabilities even when privileged.
|
||||||
|
- Only the specific `/dev/dri` device is mounted; no other host devices.
|
||||||
|
- Data volumes are scoped to `/DATA/AppData/$AppID/...`.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### 3. Device mount: `/dev/dri:/dev/dri`
|
||||||
|
|
||||||
|
**Varför det behövs:**
|
||||||
|
|
||||||
|
- VAAPI and QSV hardware encoding require direct access to the GPU render nodes in `/dev/dri`.
|
||||||
|
- Without this mount, FFmpeg falls back to software encoding which is 10–50x slower on 4K content.
|
||||||
|
|
||||||
|
**Alternativ som utvärderats:**
|
||||||
|
|
||||||
|
- Specific device nodes (e.g., `/dev/dri/renderD128`): device names can vary by driver version and host kernel.
|
||||||
|
- No hardware acceleration: software fallback is too slow for practical use.
|
||||||
|
|
||||||
|
**Risker:**
|
||||||
|
|
||||||
|
- The container can enumerate and use all graphics devices on the host.
|
||||||
|
- On multi-user systems, other users' GPU resources may be accessible.
|
||||||
|
|
||||||
|
**Riskreducering:**
|
||||||
|
|
||||||
|
- `privileged: true` combined with `cap_drop: ["ALL"]` ensures the container cannot load additional kernel modules or escalate privileges.
|
||||||
|
- Only the render nodes are exposed; no other host devices are passed through.
|
||||||
@@ -0,0 +1,103 @@
|
|||||||
|
name: snacks
|
||||||
|
|
||||||
|
services:
|
||||||
|
snacks:
|
||||||
|
image: derekshreds/snacks-docker:2.3.1
|
||||||
|
container_name: snacks
|
||||||
|
restart: unless-stopped
|
||||||
|
deploy:
|
||||||
|
resources:
|
||||||
|
reservations:
|
||||||
|
memory: 1G
|
||||||
|
|
||||||
|
environment:
|
||||||
|
- TZ=Europe/Stockholm
|
||||||
|
- PUID=1000
|
||||||
|
- PGID=1000
|
||||||
|
- ASPNETCORE_ENVIRONMENT=Production
|
||||||
|
- SNACKS_WORK_DIR=/app/work
|
||||||
|
- FFMPEG_PATH=/usr/lib/jellyfin-ffmpeg/ffmpeg
|
||||||
|
- FFPROBE_PATH=/usr/lib/jellyfin-ffmpeg/ffprobe
|
||||||
|
|
||||||
|
network_mode: host
|
||||||
|
|
||||||
|
volumes:
|
||||||
|
- type: bind
|
||||||
|
source: /DATA/AppData/$AppID/media
|
||||||
|
target: /app/work/uploads
|
||||||
|
- type: bind
|
||||||
|
source: /DATA/AppData/$AppID/logs
|
||||||
|
target: /app/work/logs
|
||||||
|
- type: bind
|
||||||
|
source: /DATA/AppData/$AppID/config
|
||||||
|
target: /app/work/config
|
||||||
|
|
||||||
|
devices:
|
||||||
|
- /dev/dri:/dev/dri
|
||||||
|
|
||||||
|
privileged: true
|
||||||
|
|
||||||
|
security_opt:
|
||||||
|
- no-new-privileges:true
|
||||||
|
|
||||||
|
cap_drop:
|
||||||
|
- ALL
|
||||||
|
|
||||||
|
healthcheck:
|
||||||
|
test: ["CMD", "curl", "-f", "http://localhost:6767/Home/Health"]
|
||||||
|
interval: 30s
|
||||||
|
timeout: 10s
|
||||||
|
retries: 3
|
||||||
|
start_period: 40s
|
||||||
|
|
||||||
|
x-casaos:
|
||||||
|
envs:
|
||||||
|
- container: TZ
|
||||||
|
description:
|
||||||
|
en_US: Timezone, for example Europe/Stockholm
|
||||||
|
- container: PUID
|
||||||
|
description:
|
||||||
|
en_US: User ID for filesystem permissions
|
||||||
|
- container: PGID
|
||||||
|
description:
|
||||||
|
en_US: Group ID for filesystem permissions
|
||||||
|
- container: FFMPEG_PATH
|
||||||
|
description:
|
||||||
|
en_US: "FFmpeg binary path (default: /usr/lib/jellyfin-ffmpeg/ffmpeg). Use /usr/bin/ffmpeg on systems without jellyfin-ffmpeg."
|
||||||
|
- container: FFPROBE_PATH
|
||||||
|
description:
|
||||||
|
en_US: "FFprobe binary path (default: /usr/lib/jellyfin-ffmpeg/ffprobe). Use /usr/bin/ffprobe on systems without jellyfin-ffmpeg."
|
||||||
|
ports:
|
||||||
|
- container: "6767"
|
||||||
|
description:
|
||||||
|
en_US: Web UI port
|
||||||
|
volumes:
|
||||||
|
- container: /app/work/uploads
|
||||||
|
description:
|
||||||
|
en_US: Media library — source files to be encoded
|
||||||
|
- container: /app/work/logs
|
||||||
|
description:
|
||||||
|
en_US: Transcoding logs directory
|
||||||
|
- container: /app/work/config
|
||||||
|
description:
|
||||||
|
en_US: Application configuration and SQLite database
|
||||||
|
|
||||||
|
x-casaos:
|
||||||
|
architectures:
|
||||||
|
- amd64
|
||||||
|
main: snacks
|
||||||
|
category: phirna
|
||||||
|
author: Joachim Friberg
|
||||||
|
developer: Joachim Friberg
|
||||||
|
icon: https://cdn.simpleicons.org/snacks
|
||||||
|
tagline:
|
||||||
|
en_US: Automated video library encoder with hardware acceleration
|
||||||
|
description:
|
||||||
|
en_US: >-
|
||||||
|
Batch transcode your video library with hardware acceleration (NVENC, QSV, VAAPI, AMF).
|
||||||
|
Monitors directories, skips already-encoded files, and supports distributed cluster encoding.
|
||||||
|
Web UI at http://localhost:6767
|
||||||
|
title:
|
||||||
|
en_US: Snacks
|
||||||
|
index: /
|
||||||
|
port_map: "6767"
|
||||||
@@ -0,0 +1,78 @@
|
|||||||
|
# Steam Headless
|
||||||
|
|
||||||
|
Steam Headless kör en webbaserad Linux-desktop med Steam i container, baserat på LinuxServer image `lscr.io/linuxserver/steam`.
|
||||||
|
|
||||||
|
Imagepinning i denna app:
|
||||||
|
|
||||||
|
- Compose använder immutable referens: `lscr.io/linuxserver/steam:version-f4f48542@sha256:d7b9fbf302e05ae79248d1171fe9751b354f8397eafa1e13a3df0aa6a75de0b4`
|
||||||
|
- `latest` finns i registryt men används inte i repo enligt policy.
|
||||||
|
- Vid verifieringstillfället pekade `latest` på samma release (`version-f4f48542`).
|
||||||
|
|
||||||
|
## Syfte
|
||||||
|
|
||||||
|
- Ge enkel Steam-access via webbläsare i ZimaOS.
|
||||||
|
- Hålla v1 med minsta möjliga privilegier.
|
||||||
|
- Förbereda en separat senare fas för Moonlight-fokuserad streaming.
|
||||||
|
|
||||||
|
## Portar
|
||||||
|
|
||||||
|
- `3000/tcp` (HTTP desktop): `${STEAM_HTTP_PORT:-3000}`
|
||||||
|
- `3001/tcp` (HTTPS desktop): `${STEAM_HTTPS_PORT:-3001}`
|
||||||
|
|
||||||
|
## Volymer
|
||||||
|
|
||||||
|
- `/DATA/AppData/$AppID/config -> /config`
|
||||||
|
|
||||||
|
All Steam-data (profil, cache, installerade spel) lagras under appens egna AppData-sökväg.
|
||||||
|
|
||||||
|
## Privilegier och säkerhet
|
||||||
|
|
||||||
|
Aktiva säkerhetsinställningar i denna app:
|
||||||
|
|
||||||
|
- `security_opt: ["seccomp:unconfined", "no-new-privileges:true"]`
|
||||||
|
- `cap_drop: ["ALL"]`
|
||||||
|
- Ingen `privileged: true`
|
||||||
|
- Ingen `network_mode: host`
|
||||||
|
- Ingen mount av `/var/run/docker.sock`
|
||||||
|
|
||||||
|
Motivering:
|
||||||
|
|
||||||
|
- LinuxServer Steam använder sandbox/bubblewrap-mönster som normalt kräver `seccomp:unconfined` för att spel/launcher ska fungera stabilt.
|
||||||
|
- `no-new-privileges:true` och `cap_drop: ["ALL"]` används för att kompensera med lägsta möjliga capability-yta i övrigt.
|
||||||
|
|
||||||
|
Kända tradeoffs:
|
||||||
|
|
||||||
|
- På vissa Debian/Ubuntu-hostar kan även `apparmor:unconfined` behövas. Detta är inte default här av least-privilege-skäl.
|
||||||
|
- Browser-vägen (KasmVNC) är enkel men ger inte samma latens/gamepad-egenskaper som Moonlight.
|
||||||
|
|
||||||
|
## Säkerhetsavvikelser
|
||||||
|
|
||||||
|
Denna app använder en avvikelse från strikt seccomp-default:
|
||||||
|
|
||||||
|
- `seccomp:unconfined`
|
||||||
|
|
||||||
|
Varför det behövs:
|
||||||
|
|
||||||
|
- För kompatibilitet med LinuxServer Steam runtime och dess sandboxade processer.
|
||||||
|
|
||||||
|
Alternativ som utvärderats:
|
||||||
|
|
||||||
|
- Standard seccomp-profil: blockar delar av förväntad processmodell för Steam/spel.
|
||||||
|
- Full `privileged: true`: avvisat på grund av större attackyta.
|
||||||
|
|
||||||
|
Risker:
|
||||||
|
|
||||||
|
- Minskad syscall-filtrering jämfört med default seccomp-profil.
|
||||||
|
- Om container komprometteras finns större möjlighet att anropa kernel-funktioner än med strikt seccomp.
|
||||||
|
|
||||||
|
Riskreducering:
|
||||||
|
|
||||||
|
- Inga host-network eller docker-socket mounts.
|
||||||
|
- Capability-surface minimerad med `cap_drop: ["ALL"]`.
|
||||||
|
- Isolerad data-path under `/DATA/AppData/$AppID/...`.
|
||||||
|
|
||||||
|
## Driftnoteringar
|
||||||
|
|
||||||
|
- För GPU-acceleration kan extra device-mounts krävas beroende på host och drivrutiner.
|
||||||
|
- Om HTTPS används på `3001` kan webbläsaren visa certifikatvarning vid första anslutning.
|
||||||
|
- Rekommenderad nästa fas: separat Moonlight/Sunshine-spår som opt-in, med egen riskprofil.
|
||||||
@@ -0,0 +1,81 @@
|
|||||||
|
name: steam-headless
|
||||||
|
|
||||||
|
services:
|
||||||
|
steam:
|
||||||
|
image: lscr.io/linuxserver/steam:version-f4f48542
|
||||||
|
container_name: steam-headless
|
||||||
|
restart: unless-stopped
|
||||||
|
shm_size: "1gb"
|
||||||
|
|
||||||
|
environment:
|
||||||
|
TZ: Europe/Stockholm
|
||||||
|
PUID: "1000"
|
||||||
|
PGID: "1000"
|
||||||
|
STEAM_HTTP_PORT: "3000"
|
||||||
|
STEAM_HTTPS_PORT: "3001"
|
||||||
|
|
||||||
|
ports:
|
||||||
|
- target: 3000
|
||||||
|
published: "3000"
|
||||||
|
protocol: tcp
|
||||||
|
- target: 3001
|
||||||
|
published: "3001"
|
||||||
|
protocol: tcp
|
||||||
|
|
||||||
|
volumes:
|
||||||
|
- type: bind
|
||||||
|
source: /DATA/AppData/$AppID/config
|
||||||
|
target: /config
|
||||||
|
|
||||||
|
# Required by LinuxServer Steam for bubblewrap/game namespaces.
|
||||||
|
security_opt:
|
||||||
|
- seccomp:unconfined
|
||||||
|
- no-new-privileges:true
|
||||||
|
|
||||||
|
# Keep capability surface minimal unless a specific game requires otherwise.
|
||||||
|
cap_drop:
|
||||||
|
- ALL
|
||||||
|
|
||||||
|
x-casaos:
|
||||||
|
envs:
|
||||||
|
- container: TZ
|
||||||
|
description:
|
||||||
|
en_us: Timezone, for example Europe/Stockholm
|
||||||
|
- container: PUID
|
||||||
|
description:
|
||||||
|
en_us: User ID for filesystem permissions
|
||||||
|
- container: PGID
|
||||||
|
description:
|
||||||
|
en_us: Group ID for filesystem permissions
|
||||||
|
ports:
|
||||||
|
- container: "3000"
|
||||||
|
description:
|
||||||
|
en_us: Steam desktop GUI over HTTP
|
||||||
|
- container: "3001"
|
||||||
|
description:
|
||||||
|
en_us: Steam desktop GUI over HTTPS
|
||||||
|
volumes:
|
||||||
|
- container: /config
|
||||||
|
description:
|
||||||
|
en_us: Steam home, configuration, and game files
|
||||||
|
|
||||||
|
x-casaos:
|
||||||
|
architectures:
|
||||||
|
- amd64
|
||||||
|
main: steam
|
||||||
|
category: phirna
|
||||||
|
author: Joachim Friberg
|
||||||
|
developer: Joachim Friberg
|
||||||
|
icon: https://cdn.simpleicons.org/steam
|
||||||
|
tagline:
|
||||||
|
en_us: Browser-based Steam desktop container for ZimaOS
|
||||||
|
description:
|
||||||
|
en_us: >-
|
||||||
|
Runs LinuxServer Steam as a web-accessible desktop session.
|
||||||
|
Optimized for amd64 and least-privilege defaults, with optional future
|
||||||
|
Moonlight-focused expansion in a later phase.
|
||||||
|
title:
|
||||||
|
en_us: Steam Headless
|
||||||
|
index: /
|
||||||
|
port_map: "3001"
|
||||||
|
scheme: https
|
||||||
@@ -0,0 +1,51 @@
|
|||||||
|
# Moonlight Runtime Checklist
|
||||||
|
|
||||||
|
Strikt checklista för att aktivera `steam-moonlight`-profilen säkert.
|
||||||
|
|
||||||
|
## Preflight (måste vara grönt)
|
||||||
|
|
||||||
|
1. Bekräfta att du medvetet accepterar högriskprofil (`network_mode: host`, extra capabilities, device passthrough).
|
||||||
|
2. Sätt starkt lösenord i `SUNSHINE_PASS` (inte `change-me`, inte återanvänt).
|
||||||
|
3. Verifiera GPU-device mapping:
|
||||||
|
- `GPU_CARD_DEVICE` pekar på korrekt `/dev/dri/card*`
|
||||||
|
- `GPU_RENDER_DEVICE` pekar på korrekt `/dev/dri/renderD*`
|
||||||
|
4. Verifiera att hosten har:
|
||||||
|
- `/dev/fuse`
|
||||||
|
- `/dev/uinput`
|
||||||
|
- korrekt `/dev/dri/*`
|
||||||
|
5. Bekräfta att Sunshine-portar inte exponeras publikt mot internet.
|
||||||
|
6. Verifiera att volymerna är under `/DATA/AppData/$AppID/...`.
|
||||||
|
7. Verifiera compose-rendering:
|
||||||
|
- `docker compose -f Apps/steam-moonlight/docker-compose.yaml --profile moonlight config`
|
||||||
|
|
||||||
|
## Startsekvens
|
||||||
|
|
||||||
|
1. Starta endast Moonlight-profilen:
|
||||||
|
- `docker compose -f Apps/steam-moonlight/docker-compose.yaml --profile moonlight up -d steam-moonlight`
|
||||||
|
2. Kontrollera containerstatus:
|
||||||
|
- `docker compose -f Apps/steam-moonlight/docker-compose.yaml ps`
|
||||||
|
3. Kontrollera initiala logs:
|
||||||
|
- `docker logs --tail=200 steam-moonlight-profile`
|
||||||
|
|
||||||
|
## Post-start verifiering
|
||||||
|
|
||||||
|
1. Verifiera att Sunshine kräver autentisering.
|
||||||
|
2. Verifiera att streaming fungerar från avsedd klient (LAN/VPN).
|
||||||
|
3. Verifiera controller-input utan att aktivera fler capabilities än definierat.
|
||||||
|
4. Verifiera att inga oväntade portar exponeras.
|
||||||
|
5. Verifiera att defaultprofil fortfarande kan köras separat vid rollback.
|
||||||
|
|
||||||
|
## Driftregler
|
||||||
|
|
||||||
|
1. Kör inte Moonlight-profilen permanent om den inte används.
|
||||||
|
2. Roterar `SUNSHINE_PASS` regelbundet och alltid efter incident.
|
||||||
|
3. Uppdatera image-pin via digest i kontrollerade change windows.
|
||||||
|
4. Undvik ad-hoc ändringar av capabilities eller device mounts.
|
||||||
|
|
||||||
|
## Snabb rollback
|
||||||
|
|
||||||
|
1. Stoppa Moonlight-profil:
|
||||||
|
- `docker compose -f Apps/steam-moonlight/docker-compose.yaml --profile moonlight stop steam-moonlight`
|
||||||
|
2. Kör defaultprofil:
|
||||||
|
- `docker compose -f Apps/steam-moonlight/docker-compose.yaml up -d steam`
|
||||||
|
3. Bekräfta återställd låg-risk drift via logs och funktionstest.
|
||||||
@@ -0,0 +1,102 @@
|
|||||||
|
# Steam Moonlight (Scaffold)
|
||||||
|
|
||||||
|
Detta är en scaffold för hybridspåret (browser + Moonlight) baserat på `josh5/steam-headless`.
|
||||||
|
|
||||||
|
Kompletterande säkerhetsdokument:
|
||||||
|
|
||||||
|
- `SECURITY.md`
|
||||||
|
- `MOONLIGHT-RUNTIME-CHECKLIST.md`
|
||||||
|
|
||||||
|
## Syfte
|
||||||
|
|
||||||
|
- Ge en låg-risk default för browserbaserad Steam-desktop.
|
||||||
|
- Ge en separat opt-in-profil för Moonlight/Sunshine.
|
||||||
|
- Isolera högriskinställningar till en explicit profil (`moonlight`).
|
||||||
|
|
||||||
|
## Imagepinning
|
||||||
|
|
||||||
|
- Compose använder immutable referens:
|
||||||
|
- `josh5/steam-headless:debian-0.2.0@sha256:540366bee31297c5679a5006a84dbca039ca62aaab695852b51b5f62dffd2c14`
|
||||||
|
- Repon kräver att `latest` inte används i compose.
|
||||||
|
|
||||||
|
## Profiler
|
||||||
|
|
||||||
|
- `steam` (default): browser-first, lägre risk, ingen host network.
|
||||||
|
- `steam-moonlight` (`profiles: ["moonlight"]`): aktiverar Sunshine + controller/GPU passthrough med högre privilegier.
|
||||||
|
|
||||||
|
## Körning
|
||||||
|
|
||||||
|
- Default (rekommenderad start):
|
||||||
|
- `docker compose -f Apps/steam-moonlight/docker-compose.yaml up -d steam`
|
||||||
|
- Moonlight (opt-in):
|
||||||
|
- `docker compose -f Apps/steam-moonlight/docker-compose.yaml --profile moonlight up -d steam-moonlight`
|
||||||
|
- Innan Moonlight aktiveras:
|
||||||
|
- byt `SUNSHINE_PASS` till ett starkt lösenord,
|
||||||
|
- verifiera `GPU_CARD_DEVICE` och `GPU_RENDER_DEVICE` för rätt GPU.
|
||||||
|
|
||||||
|
## Portar
|
||||||
|
|
||||||
|
- Defaultprofil (`steam`):
|
||||||
|
- `${STEAM_WEB_PORT:-8083}/tcp` för webdesktop.
|
||||||
|
|
||||||
|
Moonlightprofilen använder `network_mode: host` och tar därför nätverk direkt från host.
|
||||||
|
|
||||||
|
## Volymer
|
||||||
|
|
||||||
|
- Defaultprofil:
|
||||||
|
- `/DATA/AppData/$AppID/home -> /home/default`
|
||||||
|
- `/DATA/AppData/$AppID/games -> /mnt/games`
|
||||||
|
- Moonlightprofil:
|
||||||
|
- `/DATA/AppData/$AppID/moonlight-home -> /home/default`
|
||||||
|
- `/DATA/AppData/$AppID/moonlight-games -> /mnt/games`
|
||||||
|
|
||||||
|
## Privilegier och säkerhet
|
||||||
|
|
||||||
|
Gemensamt:
|
||||||
|
|
||||||
|
- `no-new-privileges:true`
|
||||||
|
|
||||||
|
Defaultprofil (`steam`, lägre risk):
|
||||||
|
|
||||||
|
- `cap_drop: ["ALL"]`
|
||||||
|
- Ingen `network_mode: host`
|
||||||
|
- Inga device mounts
|
||||||
|
- Ingen Sunshine-aktivering som default
|
||||||
|
|
||||||
|
Moonlightprofil (högrisk):
|
||||||
|
|
||||||
|
- `ipc: host`
|
||||||
|
- `security_opt: ["seccomp:unconfined", "apparmor:unconfined", "no-new-privileges:true"]`
|
||||||
|
- `cap_drop: ["ALL"]` + `cap_add: [NET_ADMIN, SYS_ADMIN, SYS_NICE]`
|
||||||
|
- `network_mode: host`
|
||||||
|
- Device mounts: `/dev/fuse`, `/dev/uinput`, `/dev/dri/*`
|
||||||
|
- `device_cgroup_rules: ['c 13:* rmw']`
|
||||||
|
|
||||||
|
## Säkerhetsavvikelser
|
||||||
|
|
||||||
|
Denna app innehåller avsiktliga avvikelser, primärt i `moonlight`-profilen.
|
||||||
|
|
||||||
|
Varför det behövs:
|
||||||
|
|
||||||
|
- Moonlight/Sunshine och fysisk controller-input kräver i praktiken host-nära åtkomst i denna containerfamilj.
|
||||||
|
|
||||||
|
Alternativ som utvärderats:
|
||||||
|
|
||||||
|
- Browser-only (`steam` default): lägre risk men sämre latens/gamepad.
|
||||||
|
- Full `privileged: true`: avvisat i scaffolden för att begränsa attackytan.
|
||||||
|
|
||||||
|
Risker:
|
||||||
|
|
||||||
|
- Host network minskar nätverksisolering.
|
||||||
|
- Extra capabilities och device-passthrough ökar konsekvensen vid containerkompromettering.
|
||||||
|
|
||||||
|
Riskreducering:
|
||||||
|
|
||||||
|
- Högrisk är opt-in via profil, inte default.
|
||||||
|
- Ingen docker socket-mount används.
|
||||||
|
- Persistent data är begränsad till `/DATA/AppData/$AppID/...`.
|
||||||
|
- Defaultprofilen droppar alla Linux capabilities.
|
||||||
|
|
||||||
|
## Status
|
||||||
|
|
||||||
|
Scaffolden är avsedd för kontrollerad vidareutveckling och verifiering, inte som slutlig hardened release.
|
||||||
@@ -0,0 +1,50 @@
|
|||||||
|
# Roadmap
|
||||||
|
|
||||||
|
## Current State (2026-03-18)
|
||||||
|
|
||||||
|
- Browser-first service (`steam`) är testbar med låg-risk baseline.
|
||||||
|
- Moonlight service (`steam-moonlight`) finns som opt-in scaffold med dokumenterade risker.
|
||||||
|
- Operativa säkerhetskrav finns i `SECURITY.md` och `MOONLIGHT-RUNTIME-CHECKLIST.md`.
|
||||||
|
|
||||||
|
## Next Milestone: Moonlight Test-Ready (Fail-Closed)
|
||||||
|
|
||||||
|
Mål: Moonlight-profilen ska vägra starta vid osäker konfiguration.
|
||||||
|
|
||||||
|
1. Enforce credentials
|
||||||
|
- Blockera start om `SUNSHINE_PASS` är tomt eller default (`change-me`).
|
||||||
|
- Blockera start om `SUNSHINE_USER` är tomt.
|
||||||
|
|
||||||
|
2. Enforce GPU/device prerequisites
|
||||||
|
- Blockera start om `GPU_CARD_DEVICE` eller `GPU_RENDER_DEVICE` saknas på host.
|
||||||
|
- Blockera start om `/dev/fuse` eller `/dev/uinput` saknas.
|
||||||
|
|
||||||
|
3. Enforce network safety baseline
|
||||||
|
- Tydlig varning + explicit opt-in env för internet-exponering.
|
||||||
|
- Defaultantagande: LAN/VPN-only drift.
|
||||||
|
|
||||||
|
4. Preflight command and runbook
|
||||||
|
- Lägg till en verifierbar preflight-sekvens som måste passera före `up`.
|
||||||
|
- Uppdatera checklistan med pass/fail-kriterier.
|
||||||
|
|
||||||
|
## Hardening Milestone
|
||||||
|
|
||||||
|
1. Secrets hygiene
|
||||||
|
- Dokumentera rotationsintervall för Sunshine-credentials.
|
||||||
|
- Säkerställ att exempel aldrig innehåller riktiga credentials.
|
||||||
|
|
||||||
|
2. Capability minimization
|
||||||
|
- Verifiera om någon `cap_add` kan tas bort utan funktionsförlust.
|
||||||
|
- Dokumentera minsta uppsättning capabilities per use-case.
|
||||||
|
|
||||||
|
3. Upgrade discipline
|
||||||
|
- Definiera rutin för image-pin uppdatering (`tag + digest`) och rollback.
|
||||||
|
|
||||||
|
## Release Criteria (Moonlight)
|
||||||
|
|
||||||
|
För att kalla Moonlight-profilen releaseklar ska följande vara uppfyllt:
|
||||||
|
|
||||||
|
1. Fail-closed enforcement implementerat och verifierat.
|
||||||
|
2. Positiv test av Moonlight streaming med korrekt auth.
|
||||||
|
3. Negativ test: start blockeras vid osäkra defaults/missing devices.
|
||||||
|
4. `./scripts/validate-appstore.sh --enforce-risk-docs` passerar.
|
||||||
|
5. README + security docs uppdaterade med slutlig driftmodell.
|
||||||
@@ -0,0 +1,70 @@
|
|||||||
|
# Security Model
|
||||||
|
|
||||||
|
Detta dokument beskriver säkerhetsmodellen för `steam-moonlight` och hur risker begränsas mellan default- och Moonlight-profil.
|
||||||
|
|
||||||
|
## Mål
|
||||||
|
|
||||||
|
- Hålla default-profilen (`steam`) så nära least-privilege som möjligt.
|
||||||
|
- Göra Moonlight-profil (`steam-moonlight`) explicit opt-in med tydlig riskaccept.
|
||||||
|
- Hålla state/data begränsad till `/DATA/AppData/$AppID/...`.
|
||||||
|
|
||||||
|
## Tillitsgränser
|
||||||
|
|
||||||
|
- Host OS och Docker daemon är högsta trust-zon.
|
||||||
|
- Containern är lägre trust-zon.
|
||||||
|
- Moonlight-klienter och LAN-trafik är extern yta.
|
||||||
|
|
||||||
|
## Threat Model
|
||||||
|
|
||||||
|
Primära hot:
|
||||||
|
|
||||||
|
- Kontoövertagande via svagt `SUNSHINE_PASS`.
|
||||||
|
- Lateral movement via `network_mode: host` (Moonlight-profil).
|
||||||
|
- Ökad impact vid containerkompromiss p.g.a. extra capabilities och device passthrough.
|
||||||
|
- Dataförlust vid felaktig mount-path.
|
||||||
|
|
||||||
|
## Säkerhetskontroller
|
||||||
|
|
||||||
|
Gemensamt:
|
||||||
|
|
||||||
|
- Immutable image pin (`tag + digest`).
|
||||||
|
- Ingen docker socket mount.
|
||||||
|
- Ingen `privileged: true`.
|
||||||
|
- Data paths begränsade till `/DATA/AppData/$AppID/...`.
|
||||||
|
|
||||||
|
Defaultprofil (`steam`):
|
||||||
|
|
||||||
|
- `cap_drop: ["ALL"]`
|
||||||
|
- `no-new-privileges:true`
|
||||||
|
- Ingen `host` network
|
||||||
|
- Inga device mounts
|
||||||
|
- Sunshine avstängd som default (`ENABLE_SUNSHINE=false`)
|
||||||
|
|
||||||
|
Moonlightprofil (`steam-moonlight`):
|
||||||
|
|
||||||
|
- Högriskkontroller isolerade till `profiles: ["moonlight"]`
|
||||||
|
- `cap_drop: ["ALL"]` + minsta kända `cap_add`
|
||||||
|
- Explicit device lista (`/dev/fuse`, `/dev/uinput`, `/dev/dri/*`)
|
||||||
|
- `seccomp:unconfined` och `apparmor:unconfined` endast i Moonlight-profilen
|
||||||
|
|
||||||
|
## Fail-Closed Regler
|
||||||
|
|
||||||
|
- Moonlight ska inte startas om `SUNSHINE_PASS` är default eller tomt.
|
||||||
|
- Moonlight ska inte startas om GPU-device mapping saknas eller är fel.
|
||||||
|
- Vid osäkerhet, kör endast defaultprofilen (`steam`).
|
||||||
|
|
||||||
|
## Operativa krav
|
||||||
|
|
||||||
|
- Exponera inte Sunshine admin/UI mot internet.
|
||||||
|
- Begränsa åtkomst till LAN/VPN och pålitliga klienter.
|
||||||
|
- Rota image-pins kontrollerat och verifiera digest före uppdatering.
|
||||||
|
- Följ checklistan i `MOONLIGHT-RUNTIME-CHECKLIST.md` före varje aktivering.
|
||||||
|
|
||||||
|
## Incidentrespons (minimum)
|
||||||
|
|
||||||
|
Vid misstänkt kompromiss:
|
||||||
|
|
||||||
|
1. Stoppa Moonlight-profilen direkt.
|
||||||
|
2. Roterar `SUNSHINE_PASS` och övriga credentials.
|
||||||
|
3. Granska hostens nätverksexponering och Docker logs.
|
||||||
|
4. Byt image till känd god pin och starta om endast defaultprofil.
|
||||||
@@ -0,0 +1,131 @@
|
|||||||
|
name: steam-moonlight
|
||||||
|
|
||||||
|
x-steam-common: &steam-common
|
||||||
|
image: josh5/steam-headless:debian-0.2.0
|
||||||
|
restart: unless-stopped
|
||||||
|
shm_size: "2G"
|
||||||
|
environment:
|
||||||
|
TZ: Europe/Stockholm
|
||||||
|
PUID: "1000"
|
||||||
|
PGID: "1000"
|
||||||
|
UMASK: "000"
|
||||||
|
USER_PASSWORD: change-me
|
||||||
|
MODE: primary
|
||||||
|
WEB_UI_MODE: vnc
|
||||||
|
PORT_NOVNC_WEB: "8083"
|
||||||
|
ENABLE_STEAM: "true"
|
||||||
|
STEAM_ARGS: -silent
|
||||||
|
ENABLE_SUNSHINE: "false"
|
||||||
|
SUNSHINE_USER: admin
|
||||||
|
SUNSHINE_PASS: change-me
|
||||||
|
|
||||||
|
services:
|
||||||
|
steam:
|
||||||
|
<<: *steam-common
|
||||||
|
container_name: steam-moonlight
|
||||||
|
security_opt:
|
||||||
|
- no-new-privileges:true
|
||||||
|
cap_drop:
|
||||||
|
- ALL
|
||||||
|
ports:
|
||||||
|
- target: 8083
|
||||||
|
published: "8083"
|
||||||
|
protocol: tcp
|
||||||
|
volumes:
|
||||||
|
- type: bind
|
||||||
|
source: /DATA/AppData/$AppID/home
|
||||||
|
target: /home/default
|
||||||
|
- type: bind
|
||||||
|
source: /DATA/AppData/$AppID/games
|
||||||
|
target: /mnt/games
|
||||||
|
x-casaos:
|
||||||
|
envs:
|
||||||
|
- container: TZ
|
||||||
|
description:
|
||||||
|
en_us: Timezone, for example Europe/Stockholm
|
||||||
|
- container: PUID
|
||||||
|
description:
|
||||||
|
en_us: User ID for filesystem permissions
|
||||||
|
- container: PGID
|
||||||
|
description:
|
||||||
|
en_us: Group ID for filesystem permissions
|
||||||
|
- container: STEAM_WEB_PORT
|
||||||
|
description:
|
||||||
|
en_us: Browser desktop port
|
||||||
|
ports:
|
||||||
|
- container: "8083"
|
||||||
|
description:
|
||||||
|
en_us: Steam desktop over web browser
|
||||||
|
volumes:
|
||||||
|
- container: /home/default
|
||||||
|
description:
|
||||||
|
en_us: Persistent user home and runtime state
|
||||||
|
- container: /mnt/games
|
||||||
|
description:
|
||||||
|
en_us: Persistent Steam game library
|
||||||
|
|
||||||
|
steam-moonlight:
|
||||||
|
<<: *steam-common
|
||||||
|
container_name: steam-moonlight-profile
|
||||||
|
profiles: ["moonlight"]
|
||||||
|
network_mode: host
|
||||||
|
ipc: host
|
||||||
|
security_opt:
|
||||||
|
- seccomp:unconfined
|
||||||
|
- apparmor:unconfined
|
||||||
|
- no-new-privileges:true
|
||||||
|
cap_drop:
|
||||||
|
- ALL
|
||||||
|
cap_add:
|
||||||
|
- NET_ADMIN
|
||||||
|
- SYS_ADMIN
|
||||||
|
- SYS_NICE
|
||||||
|
devices:
|
||||||
|
- /dev/fuse
|
||||||
|
- /dev/uinput
|
||||||
|
- /dev/dri/card0
|
||||||
|
- /dev/dri/renderD128
|
||||||
|
device_cgroup_rules:
|
||||||
|
- 'c 13:* rmw'
|
||||||
|
environment:
|
||||||
|
TZ: Europe/Stockholm
|
||||||
|
PUID: "1000"
|
||||||
|
PGID: "1000"
|
||||||
|
UMASK: "000"
|
||||||
|
USER_PASSWORD: change-me
|
||||||
|
MODE: primary
|
||||||
|
WEB_UI_MODE: vnc
|
||||||
|
PORT_NOVNC_WEB: "8083"
|
||||||
|
ENABLE_STEAM: "true"
|
||||||
|
STEAM_ARGS: -silent
|
||||||
|
ENABLE_SUNSHINE: "true"
|
||||||
|
SUNSHINE_USER: admin
|
||||||
|
SUNSHINE_PASS: change-me
|
||||||
|
volumes:
|
||||||
|
- type: bind
|
||||||
|
source: /DATA/AppData/$AppID/moonlight-home
|
||||||
|
target: /home/default
|
||||||
|
- type: bind
|
||||||
|
source: /DATA/AppData/$AppID/moonlight-games
|
||||||
|
target: /mnt/games
|
||||||
|
|
||||||
|
x-casaos:
|
||||||
|
architectures:
|
||||||
|
- amd64
|
||||||
|
main: steam
|
||||||
|
category: phirna
|
||||||
|
author: Joachim Friberg
|
||||||
|
developer: Joachim Friberg
|
||||||
|
icon: https://moonlight-stream.org/images/moonlight.svg
|
||||||
|
tagline:
|
||||||
|
en_us: Steam web desktop with optional Moonlight profile
|
||||||
|
description:
|
||||||
|
en_us: >-
|
||||||
|
Browser-first Steam container with an explicit moonlight profile for higher
|
||||||
|
compatibility and controller support. The moonlight profile is opt-in and
|
||||||
|
carries additional security risk.
|
||||||
|
title:
|
||||||
|
en_us: Steam Moonlight (Scaffold)
|
||||||
|
index: /
|
||||||
|
port_map: "8083"
|
||||||
|
scheme: http
|
||||||
@@ -0,0 +1,107 @@
|
|||||||
|
# How To Verify
|
||||||
|
|
||||||
|
Detta dokument verifierar båda Steam-apparna i repo:
|
||||||
|
|
||||||
|
- `Apps/steam-headless`
|
||||||
|
- `Apps/steam-moonlight`
|
||||||
|
|
||||||
|
## 1) Repo-validering
|
||||||
|
|
||||||
|
Kör från repo-roten:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
./scripts/validate-appstore.sh --enforce-risk-docs
|
||||||
|
```
|
||||||
|
|
||||||
|
Förväntat: `Validation OK` eller `Validation OK with ... warning(s)`.
|
||||||
|
|
||||||
|
## 2) Verifiera steam-headless (browser-first)
|
||||||
|
|
||||||
|
Rendera compose:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker compose -f Apps/steam-headless/docker-compose.yaml config
|
||||||
|
```
|
||||||
|
|
||||||
|
Starta:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker compose -f Apps/steam-headless/docker-compose.yaml up -d steam
|
||||||
|
```
|
||||||
|
|
||||||
|
Kontroller:
|
||||||
|
|
||||||
|
1. `docker compose -f Apps/steam-headless/docker-compose.yaml ps` visar `steam` som running.
|
||||||
|
2. Web UI nås på `${STEAM_HTTP_PORT:-3000}` eller `${STEAM_HTTPS_PORT:-3001}`.
|
||||||
|
3. Inga extra högriskflaggor används (`privileged`, `host network`, `docker.sock`).
|
||||||
|
|
||||||
|
Stoppa:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker compose -f Apps/steam-headless/docker-compose.yaml down
|
||||||
|
```
|
||||||
|
|
||||||
|
## 3) Verifiera steam-moonlight defaultprofil
|
||||||
|
|
||||||
|
Rendera compose (default):
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker compose -f Apps/steam-moonlight/docker-compose.yaml config
|
||||||
|
```
|
||||||
|
|
||||||
|
Starta default service:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker compose -f Apps/steam-moonlight/docker-compose.yaml up -d steam
|
||||||
|
```
|
||||||
|
|
||||||
|
Kontroller:
|
||||||
|
|
||||||
|
1. `steam` är running.
|
||||||
|
2. Webdesktop nås på `${STEAM_WEB_PORT:-8083}`.
|
||||||
|
3. Defaultprofilen kör med låg-risk baseline (`cap_drop: ALL`, ingen `host network`).
|
||||||
|
|
||||||
|
Stoppa:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker compose -f Apps/steam-moonlight/docker-compose.yaml down
|
||||||
|
```
|
||||||
|
|
||||||
|
## 4) Verifiera steam-moonlight moonlight-profil (opt-in)
|
||||||
|
|
||||||
|
Preflight:
|
||||||
|
|
||||||
|
1. Sätt starkt `SUNSHINE_PASS`.
|
||||||
|
2. Verifiera GPU devices (`GPU_CARD_DEVICE`, `GPU_RENDER_DEVICE`).
|
||||||
|
3. Verifiera `/dev/fuse` och `/dev/uinput` på host.
|
||||||
|
|
||||||
|
Rendera moonlight-profil:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker compose -f Apps/steam-moonlight/docker-compose.yaml --profile moonlight config
|
||||||
|
```
|
||||||
|
|
||||||
|
Starta:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker compose -f Apps/steam-moonlight/docker-compose.yaml --profile moonlight up -d steam-moonlight
|
||||||
|
```
|
||||||
|
|
||||||
|
Kontroller:
|
||||||
|
|
||||||
|
1. `steam-moonlight` är running.
|
||||||
|
2. Sunshine kräver autentisering.
|
||||||
|
3. Moonlight-klient kan ansluta från LAN/VPN.
|
||||||
|
4. Ingen oavsiktlig internetexponering av Sunshine-portar.
|
||||||
|
|
||||||
|
Stoppa/rollback:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker compose -f Apps/steam-moonlight/docker-compose.yaml --profile moonlight down
|
||||||
|
```
|
||||||
|
|
||||||
|
Vid problem, återgå till defaultprofil:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker compose -f Apps/steam-moonlight/docker-compose.yaml up -d steam
|
||||||
|
```
|
||||||
@@ -20,6 +20,10 @@ Skelett för att bygga och underhålla ZimaOS/CasaOS-appar i ett eget appstore-r
|
|||||||
├── featured-apps.json
|
├── featured-apps.json
|
||||||
├── recommend-list.json
|
├── recommend-list.json
|
||||||
└── scripts/
|
└── scripts/
|
||||||
|
├── build-appstore-zip.sh
|
||||||
|
├── build-and-push-image.sh
|
||||||
|
├── discover-zima-repos-and-apps.sh
|
||||||
|
├── map-unraid-images-to-zima-apps.sh
|
||||||
└── validate-appstore.sh
|
└── validate-appstore.sh
|
||||||
```
|
```
|
||||||
|
|
||||||
@@ -46,6 +50,134 @@ Inför release/publicering, kör strikt validering för högrisk-inställningar:
|
|||||||
./scripts/validate-appstore.sh --enforce-risk-docs
|
./scripts/validate-appstore.sh --enforce-risk-docs
|
||||||
```
|
```
|
||||||
|
|
||||||
|
Bygg och publicera app-specifika custom images (Docker Hub namespace `joafri` som default):
|
||||||
|
|
||||||
|
```bash
|
||||||
|
./scripts/build-and-push-image.sh --app-id caddy-autogen --tag 0.1.0
|
||||||
|
```
|
||||||
|
|
||||||
|
Scriptet läser `Apps/<app-id>/docker-compose.yaml` och bygger alla services som har `build:` om `--component` inte anges.
|
||||||
|
`--component` kan användas för en enskild service (service-namn eller context-path).
|
||||||
|
Arkitekturer hämtas från `x-casaos.architectures`; builden kör fail-open per arkitektur och visar varningar i slutet för misslyckade arch-builds.
|
||||||
|
|
||||||
|
## Scripts
|
||||||
|
|
||||||
|
### `validate-appstore.sh`
|
||||||
|
|
||||||
|
Basvalidering:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
./scripts/validate-appstore.sh
|
||||||
|
```
|
||||||
|
|
||||||
|
Strikt riskdokumentationskontroll:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
./scripts/validate-appstore.sh --enforce-risk-docs
|
||||||
|
```
|
||||||
|
|
||||||
|
### `build-and-push-image.sh`
|
||||||
|
|
||||||
|
Bygger och pushar appens custom images till Docker Hub (`joafri` som default).
|
||||||
|
|
||||||
|
```bash
|
||||||
|
./scripts/build-and-push-image.sh --app-id caddy-autogen --tag 0.2.0
|
||||||
|
```
|
||||||
|
|
||||||
|
Viktiga flaggor:
|
||||||
|
|
||||||
|
- `--component <value>`: bygg endast en service (service-namn eller context-path).
|
||||||
|
- `--repo <namespace>`: byt namespace (default `joafri`).
|
||||||
|
- `--no-push`: bygg utan push.
|
||||||
|
|
||||||
|
Beteende:
|
||||||
|
|
||||||
|
- Läser `Apps/<app-id>/docker-compose.yaml`.
|
||||||
|
- Bygger alla services med `build:` om `--component` inte anges.
|
||||||
|
- Läser arkitekturer från `x-casaos.architectures`.
|
||||||
|
- Bygger per arkitektur och skapar manifest-tag.
|
||||||
|
- Fail-open per arkitektur (varning i slutet om någon arch misslyckas).
|
||||||
|
|
||||||
|
### `build-appstore-zip.sh`
|
||||||
|
|
||||||
|
Bygger appstore-zip (inklusive `.yaml` -> `.yml` export i staging).
|
||||||
|
|
||||||
|
```bash
|
||||||
|
./scripts/build-appstore-zip.sh
|
||||||
|
```
|
||||||
|
|
||||||
|
Bygg och publicera `dist/` till `main`:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
./scripts/build-appstore-zip.sh --push
|
||||||
|
```
|
||||||
|
|
||||||
|
Viktiga flaggor:
|
||||||
|
|
||||||
|
- `--push`: byter till `main`, `git add dist/`, commit `Updated appstore`, push `origin main`.
|
||||||
|
- `--strict-images`: fail-closed om någon image i apparna saknas online.
|
||||||
|
|
||||||
|
Image-verifiering:
|
||||||
|
|
||||||
|
- Scriptet verifierar alla `image:` i `Apps/*/docker-compose.yaml` (skippar `_template`) via registry-manifest.
|
||||||
|
- Standardläge: fail-open (varningar skrivs ut, zip byggs ändå).
|
||||||
|
- CI/Gitea-runner (`CI=true` eller `GITEA_ACTIONS=true`): scriptet blir automatiskt strikt och returnerar felkod om någon image saknas.
|
||||||
|
|
||||||
|
### `discover-zima-repos-and-apps.sh`
|
||||||
|
|
||||||
|
Hämtar appstore-repon från en ZimaOS-host via SSH (auto scan av kända CasaOS/ZimaOS-paths), validerar repo-zippar och skriver en inventerings-YAML.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
./scripts/discover-zima-repos-and-apps.sh \
|
||||||
|
--host 10.0.1.10 \
|
||||||
|
--user root \
|
||||||
|
--out artifacts/zima-repo-app-inventory.yaml
|
||||||
|
```
|
||||||
|
|
||||||
|
Med explicit extra repo-URL (om host-scan missar något):
|
||||||
|
|
||||||
|
```bash
|
||||||
|
./scripts/discover-zima-repos-and-apps.sh \
|
||||||
|
--host 10.0.1.10 \
|
||||||
|
--repo-url https://example.org/my-appstore.zip \
|
||||||
|
--out artifacts/zima-repo-app-inventory.yaml
|
||||||
|
```
|
||||||
|
|
||||||
|
Endast manuella repo-URL:er (utan SSH-scan):
|
||||||
|
|
||||||
|
```bash
|
||||||
|
./scripts/discover-zima-repos-and-apps.sh \
|
||||||
|
--host 10.0.1.10 \
|
||||||
|
--skip-host-scan \
|
||||||
|
--insecure-tls \
|
||||||
|
--repo-url https://example.org/my-appstore.zip \
|
||||||
|
--out artifacts/zima-repo-app-inventory.yaml
|
||||||
|
```
|
||||||
|
|
||||||
|
`--insecure-tls` ska bara användas när CA-kedja saknas i den miljö där scriptet körs.
|
||||||
|
|
||||||
|
Output innehåller:
|
||||||
|
|
||||||
|
- `repositories[]`: repo-URL, källa (host-scan/manual), status och felorsak.
|
||||||
|
- `apps[]`: `app_id`, `title`, `repo_url` och normaliserade `images`.
|
||||||
|
|
||||||
|
### `map-unraid-images-to-zima-apps.sh`
|
||||||
|
|
||||||
|
Mappar en lista av Unraid-image-referenser mot inventerings-YAML från discovery-scriptet.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
./scripts/map-unraid-images-to-zima-apps.sh \
|
||||||
|
--unraid-images /tmp/unraid-images.txt \
|
||||||
|
--inventory artifacts/zima-repo-app-inventory.yaml \
|
||||||
|
--out artifacts/unraid-to-zima-map.yaml
|
||||||
|
```
|
||||||
|
|
||||||
|
Output innehåller:
|
||||||
|
|
||||||
|
- `mapping[]` per image med `match_status` = `found`, `missing` eller `ambiguous`.
|
||||||
|
- `matched_apps[]` med träfforsaker (`image_exact`, `image_basename`, `name_alias`).
|
||||||
|
- `summary` för total/found/missing/ambiguous.
|
||||||
|
|
||||||
## Säkerhetsriktlinjer
|
## Säkerhetsriktlinjer
|
||||||
|
|
||||||
- Undvik privilegierad container, host network och `docker.sock` om det inte är absolut nödvändigt.
|
- Undvik privilegierad container, host network och `docker.sock` om det inte är absolut nödvändigt.
|
||||||
|
|||||||
@@ -0,0 +1,15 @@
|
|||||||
|
## Backlog
|
||||||
|
|
||||||
|
| # | Done | Name | Source | What | Agent instructions |
|
||||||
|
|---|---|---|---|---|---|
|
||||||
|
| 1 | [x] | Snacks | https://github.com/derekshreds/snacks | Automated video library encoder | Branch `snacks/initial/add-video-encoder`; implemented in `Apps/snacks/` |
|
||||||
|
|
||||||
|
## Adding a new app
|
||||||
|
|
||||||
|
1. Copy `Apps/_template/` → `Apps/<app-id>/`
|
||||||
|
2. Set `name` in compose (lowercase + hyphen only)
|
||||||
|
3. Pin image to explicit version/tag (no `:latest`); verify tag exists in registry
|
||||||
|
4. Add `x-casaos` metadata (title, description, icon, category, author, port_map)
|
||||||
|
5. Write `README.md` with purpose, ports, volumes, envs, and risk justifications
|
||||||
|
6. Validate: `./scripts/validate-appstore.sh`
|
||||||
|
7. Run final validation before release: `./scripts/validate-appstore.sh --enforce-risk-docs`
|
||||||
+35
-6
@@ -1,8 +1,37 @@
|
|||||||
[
|
[
|
||||||
"Utilities",
|
{
|
||||||
"Developer",
|
"name": "phirna",
|
||||||
"Media",
|
"font": "apps",
|
||||||
"Productivity",
|
"description": "Phirna App Store"
|
||||||
"Network",
|
},
|
||||||
"System"
|
{
|
||||||
|
"name": "Utilities",
|
||||||
|
"font": "tools",
|
||||||
|
"description": ""
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "Developer",
|
||||||
|
"font": "terminal",
|
||||||
|
"description": ""
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "Media",
|
||||||
|
"font": "video",
|
||||||
|
"description": ""
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "Productivity",
|
||||||
|
"font": "note",
|
||||||
|
"description": ""
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "Network",
|
||||||
|
"font": "globe",
|
||||||
|
"description": ""
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "System",
|
||||||
|
"font": "settings",
|
||||||
|
"description": ""
|
||||||
|
}
|
||||||
]
|
]
|
||||||
|
|||||||
Vendored
BIN
Binary file not shown.
Executable
+370
@@ -0,0 +1,370 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
repo_root="$(cd "$(dirname "$0")/.." && pwd)"
|
||||||
|
default_repo="joafri"
|
||||||
|
|
||||||
|
usage() {
|
||||||
|
cat <<'USAGE'
|
||||||
|
Usage:
|
||||||
|
./scripts/build-and-push-image.sh --app-id <app-id> --tag <tag> [options]
|
||||||
|
|
||||||
|
Required:
|
||||||
|
--app-id <app-id> App folder under Apps/, for example: caddy-autogen
|
||||||
|
--tag <tag> Docker tag base (must not be latest)
|
||||||
|
|
||||||
|
Options:
|
||||||
|
--component <value> Optional. Service name OR build context path under app.
|
||||||
|
If omitted, build all services with build contexts.
|
||||||
|
--repo <namespace> Docker namespace/org (default: joafri)
|
||||||
|
--skip-non-buildable Exit 0 (with warning) when app has no services with build context
|
||||||
|
--no-push Build only, no push or manifest creation
|
||||||
|
-h, --help Show this help
|
||||||
|
|
||||||
|
Behavior:
|
||||||
|
- Reads buildable services from Apps/<app-id>/docker-compose.yaml.
|
||||||
|
- Reads architectures from x-casaos.architectures in the same compose file.
|
||||||
|
- Builds each architecture separately and tags as <tag>-<arch>.
|
||||||
|
- When pushing, creates a manifest tag <tag> from successful arch tags.
|
||||||
|
- Fail-open per architecture: failed arch builds are reported as warnings.
|
||||||
|
|
||||||
|
Examples:
|
||||||
|
./scripts/build-and-push-image.sh --app-id caddy-autogen --tag 0.2.0
|
||||||
|
./scripts/build-and-push-image.sh --app-id caddy-autogen --component agent --tag 0.2.0
|
||||||
|
USAGE
|
||||||
|
}
|
||||||
|
|
||||||
|
app_id=""
|
||||||
|
component=""
|
||||||
|
tag=""
|
||||||
|
repo_ns="$default_repo"
|
||||||
|
push_image=1
|
||||||
|
skip_non_buildable=0
|
||||||
|
|
||||||
|
while [[ $# -gt 0 ]]; do
|
||||||
|
case "$1" in
|
||||||
|
--app-id)
|
||||||
|
app_id="${2:-}"
|
||||||
|
shift 2
|
||||||
|
;;
|
||||||
|
--component)
|
||||||
|
component="${2:-}"
|
||||||
|
shift 2
|
||||||
|
;;
|
||||||
|
--tag)
|
||||||
|
tag="${2:-}"
|
||||||
|
shift 2
|
||||||
|
;;
|
||||||
|
--repo)
|
||||||
|
repo_ns="${2:-}"
|
||||||
|
shift 2
|
||||||
|
;;
|
||||||
|
--no-push)
|
||||||
|
push_image=0
|
||||||
|
shift
|
||||||
|
;;
|
||||||
|
--skip-non-buildable)
|
||||||
|
skip_non_buildable=1
|
||||||
|
shift
|
||||||
|
;;
|
||||||
|
-h|--help)
|
||||||
|
usage
|
||||||
|
exit 0
|
||||||
|
;;
|
||||||
|
*)
|
||||||
|
echo "ERROR: unknown argument: $1" >&2
|
||||||
|
usage
|
||||||
|
exit 2
|
||||||
|
;;
|
||||||
|
esac
|
||||||
|
done
|
||||||
|
|
||||||
|
if [[ -z "$app_id" || -z "$tag" ]]; then
|
||||||
|
echo "ERROR: --app-id and --tag are required" >&2
|
||||||
|
usage
|
||||||
|
exit 2
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [[ "$tag" == "latest" ]]; then
|
||||||
|
echo "ERROR: tag 'latest' is not allowed by repo policy" >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [[ "$app_id" =~ [^a-z0-9-] ]]; then
|
||||||
|
echo "ERROR: app-id must match [a-z0-9-]+" >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [[ "$repo_ns" =~ [^a-zA-Z0-9._-] ]]; then
|
||||||
|
echo "ERROR: repo namespace contains invalid characters" >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
component="${component#./}"
|
||||||
|
if [[ -n "$component" && ( "$component" == /* || "$component" == *".."* ) ]]; then
|
||||||
|
echo "ERROR: --component must be service name or relative subpath under Apps/$app_id" >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
app_dir="$repo_root/Apps/$app_id"
|
||||||
|
compose_file="$app_dir/docker-compose.yaml"
|
||||||
|
|
||||||
|
if [[ ! -d "$app_dir" ]]; then
|
||||||
|
echo "ERROR: app directory not found: $app_dir" >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [[ ! -f "$compose_file" ]]; then
|
||||||
|
echo "ERROR: missing compose file: $compose_file" >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
if ! command -v docker >/dev/null 2>&1; then
|
||||||
|
echo "ERROR: docker is required" >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
if ! docker buildx version >/dev/null 2>&1; then
|
||||||
|
echo "ERROR: docker buildx is required" >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
if ! command -v jq >/dev/null 2>&1; then
|
||||||
|
echo "ERROR: jq is required" >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
map_arch_to_platform() {
|
||||||
|
case "$1" in
|
||||||
|
amd64) echo "linux/amd64" ;;
|
||||||
|
arm64) echo "linux/arm64" ;;
|
||||||
|
arm) echo "linux/arm/v7" ;;
|
||||||
|
*) return 1 ;;
|
||||||
|
esac
|
||||||
|
}
|
||||||
|
|
||||||
|
read_compose_architectures() {
|
||||||
|
awk '
|
||||||
|
/^x-casaos:[[:space:]]*$/ { in_casa=1; next }
|
||||||
|
in_casa && /^[^[:space:]]/ { in_casa=0 }
|
||||||
|
in_casa && /^[[:space:]]+architectures:[[:space:]]*$/ { in_arch=1; next }
|
||||||
|
in_arch {
|
||||||
|
if ($0 ~ /^[[:space:]]*-[[:space:]]*[A-Za-z0-9_-]+[[:space:]]*$/) {
|
||||||
|
line=$0
|
||||||
|
sub(/^[[:space:]]*-[[:space:]]*/, "", line)
|
||||||
|
sub(/[[:space:]]*$/, "", line)
|
||||||
|
print line
|
||||||
|
next
|
||||||
|
}
|
||||||
|
if ($0 ~ /^[[:space:]]*$/) next
|
||||||
|
in_arch=0
|
||||||
|
}
|
||||||
|
' "$compose_file"
|
||||||
|
}
|
||||||
|
|
||||||
|
compose_json="$(mktemp)"
|
||||||
|
cleanup() {
|
||||||
|
rm -f "$compose_json"
|
||||||
|
}
|
||||||
|
trap cleanup EXIT
|
||||||
|
|
||||||
|
if [[ -z "${AppID:-}" ]]; then
|
||||||
|
export AppID="$app_id"
|
||||||
|
fi
|
||||||
|
|
||||||
|
docker compose -f "$compose_file" config --format json > "$compose_json"
|
||||||
|
|
||||||
|
mapfile -t all_services < <(
|
||||||
|
jq -r '.services | to_entries[] | select(.value.build != null) | .key' "$compose_json"
|
||||||
|
)
|
||||||
|
|
||||||
|
if [[ ${#all_services[@]} -eq 0 ]]; then
|
||||||
|
if [[ "$skip_non_buildable" -eq 1 ]]; then
|
||||||
|
echo "WARN: no buildable services found in $compose_file (skipped)"
|
||||||
|
exit 0
|
||||||
|
fi
|
||||||
|
echo "ERROR: no buildable services found in $compose_file" >&2
|
||||||
|
echo "Hint: pass --skip-non-buildable for batch loops over mixed app types." >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
mapfile -t compose_arches < <(read_compose_architectures)
|
||||||
|
if [[ ${#compose_arches[@]} -eq 0 ]]; then
|
||||||
|
compose_arches=(amd64)
|
||||||
|
echo "WARN: no x-casaos.architectures found, defaulting to amd64"
|
||||||
|
fi
|
||||||
|
|
||||||
|
selected_services=()
|
||||||
|
if [[ -n "$component" ]]; then
|
||||||
|
for svc in "${all_services[@]}"; do
|
||||||
|
context_abs="$(jq -r --arg svc "$svc" '.services[$svc].build.context // empty' "$compose_json")"
|
||||||
|
if [[ -z "$context_abs" ]]; then
|
||||||
|
continue
|
||||||
|
fi
|
||||||
|
|
||||||
|
context_rel="$context_abs"
|
||||||
|
if [[ "$context_abs" == "$app_dir" ]]; then
|
||||||
|
context_rel="."
|
||||||
|
elif [[ "$context_abs" == "$app_dir"/* ]]; then
|
||||||
|
context_rel="${context_abs#"$app_dir"/}"
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [[ "$svc" == "$component" || "$context_rel" == "$component" ]]; then
|
||||||
|
selected_services+=("$svc")
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
|
||||||
|
if [[ ${#selected_services[@]} -eq 0 ]]; then
|
||||||
|
echo "ERROR: --component '$component' did not match any build service name or context path" >&2
|
||||||
|
echo "Build services in this app:" >&2
|
||||||
|
for svc in "${all_services[@]}"; do
|
||||||
|
ctx="$(jq -r --arg svc "$svc" '.services[$svc].build.context // empty' "$compose_json")"
|
||||||
|
if [[ "$ctx" == "$app_dir" ]]; then
|
||||||
|
ctx_rel="."
|
||||||
|
elif [[ "$ctx" == "$app_dir"/* ]]; then
|
||||||
|
ctx_rel="${ctx#"$app_dir"/}"
|
||||||
|
else
|
||||||
|
ctx_rel="$ctx"
|
||||||
|
fi
|
||||||
|
echo " - $svc (context: $ctx_rel)" >&2
|
||||||
|
done
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
else
|
||||||
|
selected_services=("${all_services[@]}")
|
||||||
|
fi
|
||||||
|
|
||||||
|
git_sha="unknown"
|
||||||
|
if git -C "$repo_root" rev-parse --is-inside-work-tree >/dev/null 2>&1; then
|
||||||
|
git_sha="$(git -C "$repo_root" rev-parse --short HEAD)"
|
||||||
|
fi
|
||||||
|
|
||||||
|
source_url="unknown"
|
||||||
|
if git -C "$repo_root" remote get-url origin >/dev/null 2>&1; then
|
||||||
|
source_url="$(git -C "$repo_root" remote get-url origin)"
|
||||||
|
fi
|
||||||
|
|
||||||
|
echo "App: $app_id"
|
||||||
|
echo "Compose: $compose_file"
|
||||||
|
echo "Services: ${selected_services[*]}"
|
||||||
|
echo "Architectures: ${compose_arches[*]}"
|
||||||
|
if [[ "$push_image" -eq 1 ]]; then
|
||||||
|
echo "Mode: build + push + manifest"
|
||||||
|
else
|
||||||
|
echo "Mode: build only (--no-push)"
|
||||||
|
fi
|
||||||
|
|
||||||
|
warn_messages=()
|
||||||
|
total_success=0
|
||||||
|
|
||||||
|
for svc in "${selected_services[@]}"; do
|
||||||
|
context_abs="$(jq -r --arg svc "$svc" '.services[$svc].build.context // empty' "$compose_json")"
|
||||||
|
dockerfile_rel="$(jq -r --arg svc "$svc" '.services[$svc].build.dockerfile // "Dockerfile"' "$compose_json")"
|
||||||
|
|
||||||
|
if [[ -z "$context_abs" ]]; then
|
||||||
|
warn_messages+=("$svc: missing build context in compose config, skipped")
|
||||||
|
continue
|
||||||
|
fi
|
||||||
|
|
||||||
|
case "$context_abs" in
|
||||||
|
"$app_dir"|"$app_dir"/*) ;;
|
||||||
|
*)
|
||||||
|
warn_messages+=("$svc: build context outside app dir ($context_abs), skipped")
|
||||||
|
continue
|
||||||
|
;;
|
||||||
|
esac
|
||||||
|
|
||||||
|
dockerfile_abs="$context_abs/$dockerfile_rel"
|
||||||
|
if [[ ! -f "$dockerfile_abs" ]]; then
|
||||||
|
warn_messages+=("$svc: Dockerfile missing at $dockerfile_abs, skipped")
|
||||||
|
continue
|
||||||
|
fi
|
||||||
|
|
||||||
|
image_repo="${repo_ns}/${app_id}-${svc}"
|
||||||
|
success_arch_refs=()
|
||||||
|
|
||||||
|
echo
|
||||||
|
echo "=== Service: $svc ==="
|
||||||
|
echo "Context: $context_abs"
|
||||||
|
echo "Dockerfile: $dockerfile_abs"
|
||||||
|
echo "Image repo: $image_repo"
|
||||||
|
|
||||||
|
for arch in "${compose_arches[@]}"; do
|
||||||
|
if ! platform="$(map_arch_to_platform "$arch")"; then
|
||||||
|
warn_messages+=("$svc: unsupported architecture '$arch' in x-casaos.architectures")
|
||||||
|
continue
|
||||||
|
fi
|
||||||
|
|
||||||
|
arch_ref="${image_repo}:${tag}-${arch}"
|
||||||
|
echo "Building $svc for $arch ($platform) -> $arch_ref"
|
||||||
|
|
||||||
|
build_cmd=(
|
||||||
|
docker buildx build
|
||||||
|
--platform "$platform"
|
||||||
|
--file "$dockerfile_abs"
|
||||||
|
--tag "$arch_ref"
|
||||||
|
--label "org.opencontainers.image.source=$source_url"
|
||||||
|
--label "org.opencontainers.image.revision=$git_sha"
|
||||||
|
)
|
||||||
|
|
||||||
|
if [[ "$push_image" -eq 1 ]]; then
|
||||||
|
build_cmd+=(--push)
|
||||||
|
else
|
||||||
|
build_cmd+=(--load)
|
||||||
|
fi
|
||||||
|
|
||||||
|
build_cmd+=("$context_abs")
|
||||||
|
|
||||||
|
if "${build_cmd[@]}"; then
|
||||||
|
success_arch_refs+=("$arch_ref")
|
||||||
|
total_success=$((total_success + 1))
|
||||||
|
else
|
||||||
|
warn_messages+=("$svc: build failed for arch '$arch' ($platform)")
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
|
||||||
|
if [[ ${#success_arch_refs[@]} -eq 0 ]]; then
|
||||||
|
warn_messages+=("$svc: no successful architecture builds")
|
||||||
|
continue
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [[ "$push_image" -eq 1 ]]; then
|
||||||
|
manifest_ref="${image_repo}:${tag}"
|
||||||
|
echo "Creating manifest tag: $manifest_ref"
|
||||||
|
if docker buildx imagetools create -t "$manifest_ref" "${success_arch_refs[@]}"; then
|
||||||
|
echo "Manifest created: $manifest_ref"
|
||||||
|
else
|
||||||
|
warn_messages+=("$svc: failed to create manifest tag ${manifest_ref}")
|
||||||
|
fi
|
||||||
|
else
|
||||||
|
first_ref="${success_arch_refs[0]}"
|
||||||
|
local_ref="${image_repo}:${tag}"
|
||||||
|
if docker tag "$first_ref" "$local_ref"; then
|
||||||
|
echo "Tagged local alias: $local_ref -> $first_ref"
|
||||||
|
else
|
||||||
|
warn_messages+=("$svc: failed to tag local alias ${local_ref}")
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
|
||||||
|
echo
|
||||||
|
if [[ "$total_success" -eq 0 ]]; then
|
||||||
|
echo "ERROR: no successful builds were produced" >&2
|
||||||
|
if [[ ${#warn_messages[@]} -gt 0 ]]; then
|
||||||
|
echo "Warnings:" >&2
|
||||||
|
for w in "${warn_messages[@]}"; do
|
||||||
|
echo " - $w" >&2
|
||||||
|
done
|
||||||
|
fi
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
echo "Build completed with $total_success successful arch build(s)."
|
||||||
|
|
||||||
|
if [[ ${#warn_messages[@]} -gt 0 ]]; then
|
||||||
|
echo
|
||||||
|
echo "Warnings (fail-open):"
|
||||||
|
for w in "${warn_messages[@]}"; do
|
||||||
|
echo " - $w"
|
||||||
|
done
|
||||||
|
fi
|
||||||
Executable
+251
@@ -0,0 +1,251 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
repo_root="$(cd "$(dirname "$0")/.." && pwd)"
|
||||||
|
push_mode=0
|
||||||
|
strict_image_check=0
|
||||||
|
positional=()
|
||||||
|
|
||||||
|
while [[ $# -gt 0 ]]; do
|
||||||
|
case "$1" in
|
||||||
|
--push)
|
||||||
|
push_mode=1
|
||||||
|
shift
|
||||||
|
;;
|
||||||
|
--strict-images)
|
||||||
|
strict_image_check=1
|
||||||
|
shift
|
||||||
|
;;
|
||||||
|
-h|--help)
|
||||||
|
# handled below via usage
|
||||||
|
positional+=("$1")
|
||||||
|
shift
|
||||||
|
;;
|
||||||
|
*)
|
||||||
|
positional+=("$1")
|
||||||
|
shift
|
||||||
|
;;
|
||||||
|
esac
|
||||||
|
done
|
||||||
|
|
||||||
|
if [[ ${#positional[@]} -gt 2 ]]; then
|
||||||
|
echo "ERROR: too many arguments"
|
||||||
|
exit 2
|
||||||
|
fi
|
||||||
|
|
||||||
|
output_dir="${positional[0]:-$repo_root/dist}"
|
||||||
|
#zip_name="${positional[1]:-zima-appstore-$(date +%Y%m%d-%H%M%S).zip}"
|
||||||
|
zip_name="${positional[1]:-phirna-appstore.zip}"
|
||||||
|
|
||||||
|
usage() {
|
||||||
|
cat <<USAGE
|
||||||
|
Usage: $0 [output_dir] [zip_name]
|
||||||
|
$0 --push [output_dir] [zip_name]
|
||||||
|
$0 --strict-images [output_dir] [zip_name]
|
||||||
|
$0 --push --strict-images [output_dir] [zip_name]
|
||||||
|
|
||||||
|
Examples:
|
||||||
|
$0
|
||||||
|
$0 ./dist zima-apps-main.zip
|
||||||
|
$0 --push
|
||||||
|
$0 --strict-images
|
||||||
|
USAGE
|
||||||
|
}
|
||||||
|
|
||||||
|
if [[ "${positional[0]:-}" == "-h" || "${positional[0]:-}" == "--help" ]]; then
|
||||||
|
usage
|
||||||
|
exit 0
|
||||||
|
fi
|
||||||
|
|
||||||
|
if ! command -v zip >/dev/null 2>&1; then
|
||||||
|
echo "ERROR: 'zip' command not found"
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [[ "${CI:-}" == "true" || "${GITEA_ACTIONS:-}" == "true" ]]; then
|
||||||
|
strict_image_check=1
|
||||||
|
fi
|
||||||
|
|
||||||
|
required_root_files=(
|
||||||
|
"category-list.json"
|
||||||
|
"recommend-list.json"
|
||||||
|
"featured-apps.json"
|
||||||
|
)
|
||||||
|
|
||||||
|
for file in "${required_root_files[@]}"; do
|
||||||
|
if [[ ! -f "$repo_root/$file" ]]; then
|
||||||
|
echo "ERROR: missing required file: $file"
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
|
||||||
|
if [[ ! -d "$repo_root/Apps" ]]; then
|
||||||
|
echo "ERROR: missing required directory: Apps"
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
extract_images_from_compose() {
|
||||||
|
local compose_file="$1"
|
||||||
|
awk '
|
||||||
|
/^[[:space:]]*image:[[:space:]]*/ {
|
||||||
|
line=$0
|
||||||
|
sub(/^[[:space:]]*image:[[:space:]]*/, "", line)
|
||||||
|
sub(/[[:space:]]+#.*/, "", line)
|
||||||
|
gsub(/^["'"'"']|["'"'"']$/, "", line)
|
||||||
|
if (line != "") print line
|
||||||
|
}
|
||||||
|
' "$compose_file"
|
||||||
|
}
|
||||||
|
|
||||||
|
verify_images_online() {
|
||||||
|
local -a images=()
|
||||||
|
local -a missing=()
|
||||||
|
local compose_file app_id
|
||||||
|
|
||||||
|
while IFS= read -r compose_file; do
|
||||||
|
app_id="$(basename "$(dirname "$compose_file")")"
|
||||||
|
if [[ "$app_id" == "_template" ]]; then
|
||||||
|
continue
|
||||||
|
fi
|
||||||
|
|
||||||
|
while IFS= read -r image_ref; do
|
||||||
|
[[ -z "$image_ref" ]] && continue
|
||||||
|
images+=("$image_ref")
|
||||||
|
done < <(extract_images_from_compose "$compose_file")
|
||||||
|
done < <(find "$repo_root/Apps" -type f -name 'docker-compose.yaml' | sort)
|
||||||
|
|
||||||
|
if [[ ${#images[@]} -eq 0 ]]; then
|
||||||
|
echo "WARN: no images found to verify"
|
||||||
|
return 0
|
||||||
|
fi
|
||||||
|
|
||||||
|
mapfile -t images < <(printf '%s\n' "${images[@]}" | sort -u)
|
||||||
|
|
||||||
|
if ! command -v docker >/dev/null 2>&1; then
|
||||||
|
echo "WARN: docker is not installed; skipping online image verification"
|
||||||
|
if [[ "$strict_image_check" -eq 1 ]]; then
|
||||||
|
echo "ERROR: strict image verification is enabled (CI/Gitea or --strict-images)"
|
||||||
|
return 1
|
||||||
|
fi
|
||||||
|
return 0
|
||||||
|
fi
|
||||||
|
|
||||||
|
echo "Verifying ${#images[@]} image(s) online..."
|
||||||
|
for image_ref in "${images[@]}"; do
|
||||||
|
if docker manifest inspect "$image_ref" >/dev/null 2>&1; then
|
||||||
|
echo "OK: $image_ref"
|
||||||
|
else
|
||||||
|
echo "WARN: image not found or inaccessible: $image_ref"
|
||||||
|
missing+=("$image_ref")
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
|
||||||
|
if [[ ${#missing[@]} -gt 0 ]]; then
|
||||||
|
echo
|
||||||
|
echo "Image verification warnings:"
|
||||||
|
for image_ref in "${missing[@]}"; do
|
||||||
|
echo " - $image_ref"
|
||||||
|
done
|
||||||
|
|
||||||
|
if [[ "$strict_image_check" -eq 1 ]]; then
|
||||||
|
echo
|
||||||
|
echo "ERROR: strict image verification is enabled and one or more images are missing."
|
||||||
|
return 1
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
}
|
||||||
|
|
||||||
|
if ! verify_images_online; then
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
is_git_repo=0
|
||||||
|
if git -C "$repo_root" rev-parse --is-inside-work-tree >/dev/null 2>&1; then
|
||||||
|
is_git_repo=1
|
||||||
|
fi
|
||||||
|
|
||||||
|
tmp_dir="$(mktemp -d)"
|
||||||
|
staging_dir="$tmp_dir/appstore"
|
||||||
|
cleanup() {
|
||||||
|
rm -rf "$tmp_dir"
|
||||||
|
}
|
||||||
|
trap cleanup EXIT
|
||||||
|
|
||||||
|
mkdir -p "$staging_dir/Apps"
|
||||||
|
|
||||||
|
copy_file() {
|
||||||
|
local rel="$1"
|
||||||
|
mkdir -p "$staging_dir/$(dirname "$rel")"
|
||||||
|
cp "$repo_root/$rel" "$staging_dir/$rel"
|
||||||
|
}
|
||||||
|
|
||||||
|
if [[ "$is_git_repo" -eq 1 ]]; then
|
||||||
|
# Build from tracked files only so accidental local files are excluded.
|
||||||
|
while IFS= read -r rel; do
|
||||||
|
[[ -z "$rel" ]] && continue
|
||||||
|
|
||||||
|
case "$rel" in
|
||||||
|
Apps/_template/*)
|
||||||
|
;;
|
||||||
|
Apps/*|category-list.json|recommend-list.json|featured-apps.json)
|
||||||
|
if [[ -f "$repo_root/$rel" ]]; then
|
||||||
|
copy_file "$rel"
|
||||||
|
fi
|
||||||
|
;;
|
||||||
|
esac
|
||||||
|
done < <(git -C "$repo_root" ls-files)
|
||||||
|
else
|
||||||
|
cp -R "$repo_root/Apps" "$staging_dir/"
|
||||||
|
rm -rf "$staging_dir/Apps/_template"
|
||||||
|
for file in "${required_root_files[@]}"; do
|
||||||
|
copy_file "$file"
|
||||||
|
done
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Export compatibility: source uses .yaml, AppStore tooling often expects .yml
|
||||||
|
while IFS= read -r compose_yaml; do
|
||||||
|
compose_yml="${compose_yaml%.yaml}.yml"
|
||||||
|
cp "$compose_yaml" "$compose_yml"
|
||||||
|
rm -f "$compose_yaml"
|
||||||
|
done < <(find "$staging_dir/Apps" -type f -name 'docker-compose.yaml' | sort)
|
||||||
|
|
||||||
|
if ! find "$staging_dir/Apps" -type f -name 'docker-compose.yml' | grep -q .; then
|
||||||
|
echo "ERROR: no docker-compose.yml files found in exported Apps"
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
mkdir -p "$output_dir"
|
||||||
|
zip_path="$(cd "$output_dir" && pwd)/$zip_name"
|
||||||
|
rm -f "$zip_path"
|
||||||
|
|
||||||
|
(
|
||||||
|
cd "$staging_dir"
|
||||||
|
zip -rq "$zip_path" . -x "*.DS_Store" "__MACOSX/*"
|
||||||
|
)
|
||||||
|
|
||||||
|
if command -v shasum >/dev/null 2>&1; then
|
||||||
|
checksum="$(shasum -a 256 "$zip_path" | awk '{print $1}')"
|
||||||
|
echo "ZIP created: $zip_path"
|
||||||
|
echo "SHA256: $checksum"
|
||||||
|
else
|
||||||
|
echo "ZIP created: $zip_path"
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [[ "$push_mode" -eq 1 ]]; then
|
||||||
|
if [[ "$is_git_repo" -ne 1 ]]; then
|
||||||
|
echo "ERROR: --push requires a git repository"
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
echo "Push mode enabled: switching to main and publishing dist/"
|
||||||
|
git -C "$repo_root" checkout main
|
||||||
|
git -C "$repo_root" add dist/
|
||||||
|
|
||||||
|
if git -C "$repo_root" diff --cached --quiet; then
|
||||||
|
echo "No staged changes in dist/. Nothing to commit."
|
||||||
|
exit 0
|
||||||
|
fi
|
||||||
|
|
||||||
|
git -C "$repo_root" commit -m "Updated appstore"
|
||||||
|
git -C "$repo_root" push origin main
|
||||||
|
fi
|
||||||
Executable
+467
@@ -0,0 +1,467 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
HOST=""
|
||||||
|
USER_NAME="root"
|
||||||
|
PORT="22"
|
||||||
|
IDENTITY=""
|
||||||
|
OUT_PATH="artifacts/zima-repo-app-inventory.yaml"
|
||||||
|
SSH_TIMEOUT="12"
|
||||||
|
HTTP_TIMEOUT="20"
|
||||||
|
MAX_ZIP_BYTES="$((512 * 1024 * 1024))"
|
||||||
|
EXTRA_REPO_URLS=()
|
||||||
|
SKIP_HOST_SCAN=0
|
||||||
|
INSECURE_TLS=0
|
||||||
|
|
||||||
|
usage() {
|
||||||
|
cat <<USAGE
|
||||||
|
Usage: $0 --host <host> [options]
|
||||||
|
|
||||||
|
Options:
|
||||||
|
--host <host> ZimaOS host/IP (required)
|
||||||
|
--user <user> SSH user (default: root)
|
||||||
|
--port <port> SSH port (default: 22)
|
||||||
|
--identity <path> SSH private key path
|
||||||
|
--out <path> Output YAML (default: artifacts/zima-repo-app-inventory.yaml)
|
||||||
|
--ssh-timeout <seconds> SSH connect timeout (default: 12)
|
||||||
|
--http-timeout <seconds> HTTP timeout for repo ZIP fetch (default: 20)
|
||||||
|
--max-zip-bytes <bytes> Max ZIP size to download per repo (default: 536870912)
|
||||||
|
--repo-url <url> Additional appstore ZIP URL (repeatable)
|
||||||
|
--skip-host-scan Skip SSH host scan and only use --repo-url values
|
||||||
|
--insecure-tls Disable TLS cert verification for ZIP downloads (use only if needed)
|
||||||
|
-h, --help Show this help
|
||||||
|
|
||||||
|
Behavior:
|
||||||
|
1. SSH to host and scan likely CasaOS/ZimaOS config paths for appstore ZIP URLs.
|
||||||
|
2. Validate each URL fail-closed (reachable ZIP + parseable app structure).
|
||||||
|
3. Write repository and app inventory to YAML.
|
||||||
|
USAGE
|
||||||
|
}
|
||||||
|
|
||||||
|
while [[ $# -gt 0 ]]; do
|
||||||
|
case "$1" in
|
||||||
|
--host)
|
||||||
|
HOST="${2:-}"
|
||||||
|
shift 2
|
||||||
|
;;
|
||||||
|
--user)
|
||||||
|
USER_NAME="${2:-}"
|
||||||
|
shift 2
|
||||||
|
;;
|
||||||
|
--port)
|
||||||
|
PORT="${2:-}"
|
||||||
|
shift 2
|
||||||
|
;;
|
||||||
|
--identity)
|
||||||
|
IDENTITY="${2:-}"
|
||||||
|
shift 2
|
||||||
|
;;
|
||||||
|
--out)
|
||||||
|
OUT_PATH="${2:-}"
|
||||||
|
shift 2
|
||||||
|
;;
|
||||||
|
--ssh-timeout)
|
||||||
|
SSH_TIMEOUT="${2:-}"
|
||||||
|
shift 2
|
||||||
|
;;
|
||||||
|
--http-timeout)
|
||||||
|
HTTP_TIMEOUT="${2:-}"
|
||||||
|
shift 2
|
||||||
|
;;
|
||||||
|
--max-zip-bytes)
|
||||||
|
MAX_ZIP_BYTES="${2:-}"
|
||||||
|
shift 2
|
||||||
|
;;
|
||||||
|
--repo-url)
|
||||||
|
EXTRA_REPO_URLS+=("${2:-}")
|
||||||
|
shift 2
|
||||||
|
;;
|
||||||
|
--skip-host-scan)
|
||||||
|
SKIP_HOST_SCAN=1
|
||||||
|
shift
|
||||||
|
;;
|
||||||
|
--insecure-tls)
|
||||||
|
INSECURE_TLS=1
|
||||||
|
shift
|
||||||
|
;;
|
||||||
|
-h|--help)
|
||||||
|
usage
|
||||||
|
exit 0
|
||||||
|
;;
|
||||||
|
*)
|
||||||
|
echo "ERROR: Unknown argument: $1" >&2
|
||||||
|
usage
|
||||||
|
exit 2
|
||||||
|
;;
|
||||||
|
esac
|
||||||
|
done
|
||||||
|
|
||||||
|
if [[ -z "$HOST" ]]; then
|
||||||
|
echo "ERROR: --host is required" >&2
|
||||||
|
usage
|
||||||
|
exit 2
|
||||||
|
fi
|
||||||
|
|
||||||
|
if ! [[ "$PORT" =~ ^[0-9]+$ ]]; then
|
||||||
|
echo "ERROR: --port must be numeric" >&2
|
||||||
|
exit 2
|
||||||
|
fi
|
||||||
|
|
||||||
|
if ! [[ "$SSH_TIMEOUT" =~ ^[0-9]+$ && "$HTTP_TIMEOUT" =~ ^[0-9]+$ && "$MAX_ZIP_BYTES" =~ ^[0-9]+$ ]]; then
|
||||||
|
echo "ERROR: timeout/size values must be numeric" >&2
|
||||||
|
exit 2
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [[ -n "$IDENTITY" && ! -f "$IDENTITY" ]]; then
|
||||||
|
echo "ERROR: identity file not found: $IDENTITY" >&2
|
||||||
|
exit 2
|
||||||
|
fi
|
||||||
|
|
||||||
|
tmp_dir="$(mktemp -d)"
|
||||||
|
cleanup() {
|
||||||
|
rm -rf "$tmp_dir"
|
||||||
|
}
|
||||||
|
trap cleanup EXIT
|
||||||
|
|
||||||
|
raw_scan_tsv="$tmp_dir/raw-scan.tsv"
|
||||||
|
candidate_tsv="$tmp_dir/candidates.tsv"
|
||||||
|
manual_tsv="$tmp_dir/manual.tsv"
|
||||||
|
|
||||||
|
run_remote_scan() {
|
||||||
|
local -a ssh_cmd
|
||||||
|
ssh_cmd=(ssh -o BatchMode=yes -o ConnectTimeout="$SSH_TIMEOUT" -p "$PORT")
|
||||||
|
if [[ -n "$IDENTITY" ]]; then
|
||||||
|
ssh_cmd+=(-i "$IDENTITY")
|
||||||
|
fi
|
||||||
|
ssh_cmd+=("${USER_NAME}@${HOST}" "bash -s")
|
||||||
|
|
||||||
|
"${ssh_cmd[@]}" <<'REMOTE'
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
roots=(
|
||||||
|
/etc/casaos
|
||||||
|
/var/lib/casaos
|
||||||
|
/usr/local/etc/casaos
|
||||||
|
/etc/zimaos
|
||||||
|
/var/lib/zimaos
|
||||||
|
/DATA/AppData
|
||||||
|
/DATA/.casaos
|
||||||
|
)
|
||||||
|
|
||||||
|
scan_roots=()
|
||||||
|
for path in "${roots[@]}"; do
|
||||||
|
if [[ -d "$path" ]]; then
|
||||||
|
scan_roots+=("$path")
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
|
||||||
|
if [[ "${#scan_roots[@]}" -eq 0 ]]; then
|
||||||
|
exit 0
|
||||||
|
fi
|
||||||
|
|
||||||
|
grep -ERHIno --binary-files=without-match \
|
||||||
|
--include='*.yaml' --include='*.yml' \
|
||||||
|
--include='*.json' --include='*.conf' --include='*.ini' --include='*.txt' \
|
||||||
|
'https?://[^"[:space:]]+\.zip(\?[^"[:space:]]*)?' "${scan_roots[@]}" 2>/dev/null \
|
||||||
|
| awk -F: '{
|
||||||
|
file=$1
|
||||||
|
line=$2
|
||||||
|
$1=""
|
||||||
|
$2=""
|
||||||
|
sub(/^::?/, "", $0)
|
||||||
|
sub(/^:/, "", $0)
|
||||||
|
printf "%s\t%s\t%s\n", file, line, $0
|
||||||
|
}'
|
||||||
|
REMOTE
|
||||||
|
}
|
||||||
|
|
||||||
|
echo "[1/3] Scanning ZimaOS host for configured appstore repositories..."
|
||||||
|
if [[ "$SKIP_HOST_SCAN" -eq 1 ]]; then
|
||||||
|
echo "INFO: --skip-host-scan enabled; skipping SSH scan." >&2
|
||||||
|
: > "$raw_scan_tsv"
|
||||||
|
else
|
||||||
|
if ! run_remote_scan > "$raw_scan_tsv"; then
|
||||||
|
if [[ "${#EXTRA_REPO_URLS[@]}" -gt 0 ]]; then
|
||||||
|
echo "WARN: SSH host scan failed; continuing with --repo-url entries only." >&2
|
||||||
|
: > "$raw_scan_tsv"
|
||||||
|
else
|
||||||
|
echo "ERROR: SSH host scan failed and no --repo-url fallback provided." >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [[ -s "$raw_scan_tsv" ]]; then
|
||||||
|
awk -F $'\t' 'NF>=3 && !seen[$3]++ { print $1 "\t" $2 "\t" $3 }' "$raw_scan_tsv" > "$candidate_tsv"
|
||||||
|
fi
|
||||||
|
|
||||||
|
: > "$manual_tsv"
|
||||||
|
if [[ "${#EXTRA_REPO_URLS[@]}" -gt 0 ]]; then
|
||||||
|
for url in "${EXTRA_REPO_URLS[@]}"; do
|
||||||
|
[[ -z "$url" ]] && continue
|
||||||
|
printf "%s\t%s\t%s\n" "manual-input" "0" "$url" >> "$manual_tsv"
|
||||||
|
done
|
||||||
|
fi
|
||||||
|
|
||||||
|
candidate_count=0
|
||||||
|
manual_count=0
|
||||||
|
if [[ -s "$candidate_tsv" ]]; then
|
||||||
|
candidate_count="$(wc -l < "$candidate_tsv" | tr -d ' ')"
|
||||||
|
fi
|
||||||
|
if [[ -s "$manual_tsv" ]]; then
|
||||||
|
manual_count="$(wc -l < "$manual_tsv" | tr -d ' ')"
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [[ "$candidate_count" -eq 0 && "$manual_count" -eq 0 ]]; then
|
||||||
|
echo "ERROR: No appstore ZIP URLs discovered on host and no --repo-url provided." >&2
|
||||||
|
echo "Hint: pass one or more --repo-url <https://...zip> if host scan misses your setup." >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [[ "$INSECURE_TLS" -eq 1 ]]; then
|
||||||
|
echo "WARN: --insecure-tls enabled; certificate verification is disabled for ZIP downloads." >&2
|
||||||
|
fi
|
||||||
|
|
||||||
|
echo "[2/3] Validating repositories and extracting app inventory..."
|
||||||
|
|
||||||
|
mkdir -p "$(dirname "$OUT_PATH")"
|
||||||
|
|
||||||
|
CANDIDATE_TSV="$candidate_tsv" \
|
||||||
|
MANUAL_TSV="$manual_tsv" \
|
||||||
|
OUT_PATH="$OUT_PATH" \
|
||||||
|
HOST="$HOST" \
|
||||||
|
HTTP_TIMEOUT="$HTTP_TIMEOUT" \
|
||||||
|
MAX_ZIP_BYTES="$MAX_ZIP_BYTES" \
|
||||||
|
INSECURE_TLS="$INSECURE_TLS" \
|
||||||
|
python3 - <<'PY'
|
||||||
|
import datetime as dt
|
||||||
|
import io
|
||||||
|
import os
|
||||||
|
import re
|
||||||
|
import ssl
|
||||||
|
import sys
|
||||||
|
import urllib.request
|
||||||
|
import zipfile
|
||||||
|
from pathlib import PurePosixPath
|
||||||
|
|
||||||
|
import yaml
|
||||||
|
|
||||||
|
candidate_tsv = os.environ["CANDIDATE_TSV"]
|
||||||
|
manual_tsv = os.environ["MANUAL_TSV"]
|
||||||
|
out_path = os.environ["OUT_PATH"]
|
||||||
|
host = os.environ["HOST"]
|
||||||
|
http_timeout = int(os.environ["HTTP_TIMEOUT"])
|
||||||
|
max_zip_bytes = int(os.environ["MAX_ZIP_BYTES"])
|
||||||
|
insecure_tls = os.environ["INSECURE_TLS"] == "1"
|
||||||
|
|
||||||
|
|
||||||
|
def normalize_image_ref(ref: str) -> str:
|
||||||
|
value = (ref or "").strip().strip('"\'')
|
||||||
|
if not value:
|
||||||
|
return ""
|
||||||
|
if "@" in value:
|
||||||
|
value = value.split("@", 1)[0]
|
||||||
|
if "/" in value:
|
||||||
|
prefix, tail = value.rsplit("/", 1)
|
||||||
|
if ":" in tail:
|
||||||
|
tail = tail.split(":", 1)[0]
|
||||||
|
value = f"{prefix}/{tail}"
|
||||||
|
else:
|
||||||
|
if ":" in value:
|
||||||
|
value = value.split(":", 1)[0]
|
||||||
|
return value.lower()
|
||||||
|
|
||||||
|
|
||||||
|
def read_sources(path: str, source_type: str):
|
||||||
|
results = []
|
||||||
|
if not os.path.exists(path):
|
||||||
|
return results
|
||||||
|
with open(path, "r", encoding="utf-8") as handle:
|
||||||
|
for line in handle:
|
||||||
|
line = line.rstrip("\n")
|
||||||
|
if not line:
|
||||||
|
continue
|
||||||
|
parts = line.split("\t")
|
||||||
|
if len(parts) < 3:
|
||||||
|
continue
|
||||||
|
evidence_file, evidence_line, url = parts[0], parts[1], parts[2]
|
||||||
|
url = url.strip()
|
||||||
|
if not url:
|
||||||
|
continue
|
||||||
|
results.append(
|
||||||
|
{
|
||||||
|
"url": url,
|
||||||
|
"source": {
|
||||||
|
"type": source_type,
|
||||||
|
"evidence_file": evidence_file,
|
||||||
|
"evidence_line": int(evidence_line) if evidence_line.isdigit() else evidence_line,
|
||||||
|
},
|
||||||
|
}
|
||||||
|
)
|
||||||
|
return results
|
||||||
|
|
||||||
|
|
||||||
|
source_items = read_sources(candidate_tsv, "host_scan") + read_sources(manual_tsv, "manual_input")
|
||||||
|
if not source_items:
|
||||||
|
print("ERROR: no source items found", file=sys.stderr)
|
||||||
|
sys.exit(1)
|
||||||
|
|
||||||
|
# Keep first evidence for each URL.
|
||||||
|
repo_sources = {}
|
||||||
|
for item in source_items:
|
||||||
|
repo_sources.setdefault(item["url"], item["source"])
|
||||||
|
|
||||||
|
|
||||||
|
def fetch_zip(url: str) -> bytes:
|
||||||
|
request = urllib.request.Request(
|
||||||
|
url,
|
||||||
|
headers={"User-Agent": "zima-repo-discovery/1.0", "Accept": "application/zip,application/octet-stream,*/*"},
|
||||||
|
)
|
||||||
|
context = ssl._create_unverified_context() if insecure_tls else None
|
||||||
|
with urllib.request.urlopen(request, timeout=http_timeout, context=context) as response:
|
||||||
|
status = getattr(response, "status", 200)
|
||||||
|
if status >= 400:
|
||||||
|
raise RuntimeError(f"HTTP {status}")
|
||||||
|
payload = response.read(max_zip_bytes + 1)
|
||||||
|
if len(payload) > max_zip_bytes:
|
||||||
|
raise RuntimeError(f"ZIP larger than allowed limit ({max_zip_bytes} bytes)")
|
||||||
|
return payload
|
||||||
|
|
||||||
|
|
||||||
|
def extract_title(compose_data):
|
||||||
|
if not isinstance(compose_data, dict):
|
||||||
|
return ""
|
||||||
|
x_casaos = compose_data.get("x-casaos")
|
||||||
|
if not isinstance(x_casaos, dict):
|
||||||
|
return ""
|
||||||
|
title = x_casaos.get("title")
|
||||||
|
if isinstance(title, str):
|
||||||
|
return title.strip()
|
||||||
|
if isinstance(title, dict):
|
||||||
|
for key in ("en_US", "en_us", "en", "sv_SE", "sv_se"):
|
||||||
|
value = title.get(key)
|
||||||
|
if isinstance(value, str) and value.strip():
|
||||||
|
return value.strip()
|
||||||
|
for value in title.values():
|
||||||
|
if isinstance(value, str) and value.strip():
|
||||||
|
return value.strip()
|
||||||
|
return ""
|
||||||
|
|
||||||
|
|
||||||
|
def extract_images(compose_data):
|
||||||
|
images = []
|
||||||
|
if not isinstance(compose_data, dict):
|
||||||
|
return images
|
||||||
|
services = compose_data.get("services")
|
||||||
|
if not isinstance(services, dict):
|
||||||
|
return images
|
||||||
|
for svc in services.values():
|
||||||
|
if not isinstance(svc, dict):
|
||||||
|
continue
|
||||||
|
image = svc.get("image")
|
||||||
|
if isinstance(image, str) and image.strip():
|
||||||
|
images.append(image.strip())
|
||||||
|
dedup = []
|
||||||
|
seen = set()
|
||||||
|
for image in images:
|
||||||
|
norm = normalize_image_ref(image)
|
||||||
|
if norm and norm not in seen:
|
||||||
|
seen.add(norm)
|
||||||
|
dedup.append(norm)
|
||||||
|
return dedup
|
||||||
|
|
||||||
|
|
||||||
|
all_repo_entries = []
|
||||||
|
app_index = {}
|
||||||
|
|
||||||
|
for repo_url, source in sorted(repo_sources.items(), key=lambda t: t[0].lower()):
|
||||||
|
entry = {
|
||||||
|
"url": repo_url,
|
||||||
|
"source": source,
|
||||||
|
"status": "error",
|
||||||
|
"error": "",
|
||||||
|
"app_count": 0,
|
||||||
|
"app_ids": [],
|
||||||
|
}
|
||||||
|
|
||||||
|
try:
|
||||||
|
payload = fetch_zip(repo_url)
|
||||||
|
with zipfile.ZipFile(io.BytesIO(payload)) as archive:
|
||||||
|
app_compose = {}
|
||||||
|
for member in archive.namelist():
|
||||||
|
member_lower = member.lower()
|
||||||
|
if not (member_lower.endswith("docker-compose.yml") or member_lower.endswith("docker-compose.yaml")):
|
||||||
|
continue
|
||||||
|
|
||||||
|
path = PurePosixPath(member)
|
||||||
|
parts = path.parts
|
||||||
|
if "Apps" not in parts:
|
||||||
|
continue
|
||||||
|
idx = parts.index("Apps")
|
||||||
|
if idx + 2 >= len(parts):
|
||||||
|
continue
|
||||||
|
app_id = parts[idx + 1]
|
||||||
|
if not app_id or app_id == "_template":
|
||||||
|
continue
|
||||||
|
app_compose.setdefault(app_id, member)
|
||||||
|
|
||||||
|
if not app_compose:
|
||||||
|
raise RuntimeError("ZIP contains no Apps/*/docker-compose.yml|yaml")
|
||||||
|
|
||||||
|
app_ids = sorted(app_compose.keys())
|
||||||
|
entry["status"] = "ok"
|
||||||
|
entry["error"] = ""
|
||||||
|
entry["app_count"] = len(app_ids)
|
||||||
|
entry["app_ids"] = app_ids
|
||||||
|
|
||||||
|
for app_id, compose_member in sorted(app_compose.items(), key=lambda t: t[0].lower()):
|
||||||
|
title = ""
|
||||||
|
images = []
|
||||||
|
try:
|
||||||
|
raw = archive.read(compose_member)
|
||||||
|
parsed = yaml.safe_load(raw.decode("utf-8", errors="replace"))
|
||||||
|
title = extract_title(parsed)
|
||||||
|
images = extract_images(parsed)
|
||||||
|
except Exception:
|
||||||
|
# Keep inventory resilient; compose parse failures should not drop app id.
|
||||||
|
title = ""
|
||||||
|
images = []
|
||||||
|
|
||||||
|
app_key = (repo_url, app_id)
|
||||||
|
app_index[app_key] = {
|
||||||
|
"app_id": app_id,
|
||||||
|
"title": title,
|
||||||
|
"repo_url": repo_url,
|
||||||
|
"images": images,
|
||||||
|
}
|
||||||
|
|
||||||
|
except Exception as exc:
|
||||||
|
entry["status"] = "error"
|
||||||
|
entry["error"] = str(exc)
|
||||||
|
|
||||||
|
all_repo_entries.append(entry)
|
||||||
|
|
||||||
|
apps = [app_index[key] for key in sorted(app_index.keys(), key=lambda t: (t[0].lower(), t[1].lower()))]
|
||||||
|
|
||||||
|
inventory = {
|
||||||
|
"schema_version": 1,
|
||||||
|
"generated_at": dt.datetime.now(dt.timezone.utc).isoformat(),
|
||||||
|
"host": host,
|
||||||
|
"repositories": all_repo_entries,
|
||||||
|
"apps": apps,
|
||||||
|
}
|
||||||
|
|
||||||
|
with open(out_path, "w", encoding="utf-8") as handle:
|
||||||
|
yaml.safe_dump(inventory, handle, sort_keys=False, allow_unicode=False)
|
||||||
|
|
||||||
|
ok_count = sum(1 for repo in all_repo_entries if repo["status"] == "ok")
|
||||||
|
err_count = len(all_repo_entries) - ok_count
|
||||||
|
print(f"Wrote {out_path}")
|
||||||
|
print(f"Repositories: total={len(all_repo_entries)} ok={ok_count} error={err_count}")
|
||||||
|
print(f"Apps discovered: {len(apps)}")
|
||||||
|
|
||||||
|
if ok_count == 0:
|
||||||
|
sys.exit(1)
|
||||||
|
PY
|
||||||
|
|
||||||
|
echo "[3/3] Done. Inventory written to: $OUT_PATH"
|
||||||
Executable
+331
@@ -0,0 +1,331 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
UNRAID_IMAGES=""
|
||||||
|
INVENTORY_PATH=""
|
||||||
|
OUT_PATH="artifacts/unraid-to-zima-map.yaml"
|
||||||
|
|
||||||
|
usage() {
|
||||||
|
cat <<USAGE
|
||||||
|
Usage: $0 --unraid-images <file> --inventory <yaml> [--out <yaml>]
|
||||||
|
|
||||||
|
Options:
|
||||||
|
--unraid-images <file> Text file with one image reference per line (required)
|
||||||
|
--inventory <yaml> Inventory YAML from discover-zima-repos-and-apps.sh (required)
|
||||||
|
--out <yaml> Output YAML (default: artifacts/unraid-to-zima-map.yaml)
|
||||||
|
-h, --help Show this help
|
||||||
|
USAGE
|
||||||
|
}
|
||||||
|
|
||||||
|
while [[ $# -gt 0 ]]; do
|
||||||
|
case "$1" in
|
||||||
|
--unraid-images)
|
||||||
|
UNRAID_IMAGES="${2:-}"
|
||||||
|
shift 2
|
||||||
|
;;
|
||||||
|
--inventory)
|
||||||
|
INVENTORY_PATH="${2:-}"
|
||||||
|
shift 2
|
||||||
|
;;
|
||||||
|
--out)
|
||||||
|
OUT_PATH="${2:-}"
|
||||||
|
shift 2
|
||||||
|
;;
|
||||||
|
-h|--help)
|
||||||
|
usage
|
||||||
|
exit 0
|
||||||
|
;;
|
||||||
|
*)
|
||||||
|
echo "ERROR: Unknown argument: $1" >&2
|
||||||
|
usage
|
||||||
|
exit 2
|
||||||
|
;;
|
||||||
|
esac
|
||||||
|
done
|
||||||
|
|
||||||
|
if [[ -z "$UNRAID_IMAGES" || -z "$INVENTORY_PATH" ]]; then
|
||||||
|
echo "ERROR: --unraid-images and --inventory are required" >&2
|
||||||
|
usage
|
||||||
|
exit 2
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [[ ! -f "$UNRAID_IMAGES" ]]; then
|
||||||
|
echo "ERROR: unraid image file not found: $UNRAID_IMAGES" >&2
|
||||||
|
exit 2
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [[ ! -f "$INVENTORY_PATH" ]]; then
|
||||||
|
echo "ERROR: inventory file not found: $INVENTORY_PATH" >&2
|
||||||
|
exit 2
|
||||||
|
fi
|
||||||
|
|
||||||
|
mkdir -p "$(dirname "$OUT_PATH")"
|
||||||
|
|
||||||
|
UNRAID_IMAGES="$UNRAID_IMAGES" \
|
||||||
|
INVENTORY_PATH="$INVENTORY_PATH" \
|
||||||
|
OUT_PATH="$OUT_PATH" \
|
||||||
|
python3 - <<'PY'
|
||||||
|
import datetime as dt
|
||||||
|
import os
|
||||||
|
import re
|
||||||
|
import sys
|
||||||
|
|
||||||
|
import yaml
|
||||||
|
|
||||||
|
unraid_images_path = os.environ["UNRAID_IMAGES"]
|
||||||
|
inventory_path = os.environ["INVENTORY_PATH"]
|
||||||
|
out_path = os.environ["OUT_PATH"]
|
||||||
|
|
||||||
|
ALIASES = {
|
||||||
|
"arch-sonarr": "sonarr",
|
||||||
|
"arch-radarr": "radarr",
|
||||||
|
"arch-prowlarr": "prowlarr",
|
||||||
|
"arch-overseerr": "overseerr",
|
||||||
|
"arch-flaresolverr": "flaresolverr",
|
||||||
|
"arch-qbittorrentvpn": "qbittorrent",
|
||||||
|
"arch-plexpass": "plex",
|
||||||
|
"open-webui": "openwebui",
|
||||||
|
"open_webui": "openwebui",
|
||||||
|
"act_runner": "act-runner",
|
||||||
|
"socket-proxy": "socketproxy",
|
||||||
|
"postgres": "postgresql",
|
||||||
|
"postgresql": "postgres",
|
||||||
|
}
|
||||||
|
|
||||||
|
GENERIC_BASE_IMAGES = {
|
||||||
|
"redis",
|
||||||
|
"postgres",
|
||||||
|
"postgresql",
|
||||||
|
"mariadb",
|
||||||
|
"mysql",
|
||||||
|
"mongo",
|
||||||
|
"mongodb",
|
||||||
|
"nginx",
|
||||||
|
"memcached",
|
||||||
|
"rabbitmq",
|
||||||
|
"valkey",
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def normalize_text(value: str) -> str:
|
||||||
|
value = (value or "").strip().lower()
|
||||||
|
return re.sub(r"[^a-z0-9]+", "", value)
|
||||||
|
|
||||||
|
|
||||||
|
def normalize_image_ref(ref: str) -> str:
|
||||||
|
value = (ref or "").strip().strip('"\'')
|
||||||
|
if not value:
|
||||||
|
return ""
|
||||||
|
if "@" in value:
|
||||||
|
value = value.split("@", 1)[0]
|
||||||
|
|
||||||
|
if "/" in value:
|
||||||
|
prefix, tail = value.rsplit("/", 1)
|
||||||
|
if ":" in tail:
|
||||||
|
tail = tail.split(":", 1)[0]
|
||||||
|
value = f"{prefix}/{tail}"
|
||||||
|
else:
|
||||||
|
if ":" in value:
|
||||||
|
value = value.split(":", 1)[0]
|
||||||
|
|
||||||
|
return value.lower()
|
||||||
|
|
||||||
|
|
||||||
|
def basename_image(normalized_ref: str) -> str:
|
||||||
|
if not normalized_ref:
|
||||||
|
return ""
|
||||||
|
return normalized_ref.rsplit("/", 1)[-1]
|
||||||
|
|
||||||
|
|
||||||
|
def candidate_names_from_unraid_image(image_ref: str):
|
||||||
|
normalized = normalize_image_ref(image_ref)
|
||||||
|
base = basename_image(normalized)
|
||||||
|
candidates = set()
|
||||||
|
if base:
|
||||||
|
candidates.add(base)
|
||||||
|
candidates.add(base.replace("_", "-"))
|
||||||
|
if base.startswith("arch-"):
|
||||||
|
candidates.add(base[5:])
|
||||||
|
if base in ALIASES:
|
||||||
|
candidates.add(ALIASES[base])
|
||||||
|
replaced = base.replace("_", "-")
|
||||||
|
if replaced in ALIASES:
|
||||||
|
candidates.add(ALIASES[replaced])
|
||||||
|
return {c for c in candidates if c}
|
||||||
|
|
||||||
|
|
||||||
|
with open(inventory_path, "r", encoding="utf-8") as handle:
|
||||||
|
inventory = yaml.safe_load(handle) or {}
|
||||||
|
|
||||||
|
apps = inventory.get("apps") or []
|
||||||
|
if not isinstance(apps, list):
|
||||||
|
print("ERROR: inventory apps must be a list", file=sys.stderr)
|
||||||
|
sys.exit(1)
|
||||||
|
|
||||||
|
app_records = []
|
||||||
|
for app in apps:
|
||||||
|
if not isinstance(app, dict):
|
||||||
|
continue
|
||||||
|
app_id = str(app.get("app_id") or "").strip()
|
||||||
|
if not app_id:
|
||||||
|
continue
|
||||||
|
title = str(app.get("title") or "").strip()
|
||||||
|
repo_url = str(app.get("repo_url") or "").strip()
|
||||||
|
images = app.get("images") if isinstance(app.get("images"), list) else []
|
||||||
|
normalized_images = []
|
||||||
|
seen_images = set()
|
||||||
|
for image in images:
|
||||||
|
if not isinstance(image, str):
|
||||||
|
continue
|
||||||
|
norm = normalize_image_ref(image)
|
||||||
|
if not norm or norm in seen_images:
|
||||||
|
continue
|
||||||
|
seen_images.add(norm)
|
||||||
|
normalized_images.append(norm)
|
||||||
|
|
||||||
|
app_records.append(
|
||||||
|
{
|
||||||
|
"app_id": app_id,
|
||||||
|
"title": title,
|
||||||
|
"repo_url": repo_url,
|
||||||
|
"images": normalized_images,
|
||||||
|
"id_key": normalize_text(app_id),
|
||||||
|
"title_key": normalize_text(title),
|
||||||
|
}
|
||||||
|
)
|
||||||
|
|
||||||
|
index_exact = {}
|
||||||
|
index_basename = {}
|
||||||
|
for app in app_records:
|
||||||
|
for image in app["images"]:
|
||||||
|
index_exact.setdefault(image, []).append(app)
|
||||||
|
bname = basename_image(image)
|
||||||
|
if bname:
|
||||||
|
index_basename.setdefault(bname, []).append(app)
|
||||||
|
|
||||||
|
def app_name_matches(app, candidate_base: str) -> bool:
|
||||||
|
key = normalize_text(candidate_base)
|
||||||
|
if not key:
|
||||||
|
return False
|
||||||
|
candidate_keys = {key}
|
||||||
|
alias = ALIASES.get(candidate_base)
|
||||||
|
if alias:
|
||||||
|
candidate_keys.add(normalize_text(alias))
|
||||||
|
for alias_source, alias_target in ALIASES.items():
|
||||||
|
if alias_target == candidate_base:
|
||||||
|
candidate_keys.add(normalize_text(alias_source))
|
||||||
|
return (app["id_key"] in candidate_keys) or (app["title_key"] and app["title_key"] in candidate_keys)
|
||||||
|
|
||||||
|
|
||||||
|
with open(unraid_images_path, "r", encoding="utf-8") as handle:
|
||||||
|
unraid_images = [line.strip() for line in handle if line.strip() and not line.strip().startswith("#")]
|
||||||
|
|
||||||
|
mapping = []
|
||||||
|
for raw_image in unraid_images:
|
||||||
|
normalized = normalize_image_ref(raw_image)
|
||||||
|
base = basename_image(normalized)
|
||||||
|
|
||||||
|
matched = {}
|
||||||
|
|
||||||
|
# Strong match: exact normalized image reference from app compose.
|
||||||
|
# For generic bases (redis/postgres/etc), only accept exact matches when app name matches.
|
||||||
|
for app in index_exact.get(normalized, []):
|
||||||
|
if base in GENERIC_BASE_IMAGES and not app_name_matches(app, base):
|
||||||
|
continue
|
||||||
|
key = (app["repo_url"], app["app_id"])
|
||||||
|
matched.setdefault(
|
||||||
|
key,
|
||||||
|
{
|
||||||
|
"app_id": app["app_id"],
|
||||||
|
"title": app["title"],
|
||||||
|
"repo_url": app["repo_url"],
|
||||||
|
"reasons": set(),
|
||||||
|
},
|
||||||
|
)["reasons"].add("image_exact")
|
||||||
|
|
||||||
|
# Medium match: basename image.
|
||||||
|
# Restrict generic/shared images to avoid false positives from sidecar dependencies.
|
||||||
|
if base:
|
||||||
|
for app in index_basename.get(base, []):
|
||||||
|
if base in GENERIC_BASE_IMAGES and not app_name_matches(app, base):
|
||||||
|
continue
|
||||||
|
if base not in GENERIC_BASE_IMAGES and not app_name_matches(app, base):
|
||||||
|
continue
|
||||||
|
key = (app["repo_url"], app["app_id"])
|
||||||
|
matched.setdefault(
|
||||||
|
key,
|
||||||
|
{
|
||||||
|
"app_id": app["app_id"],
|
||||||
|
"title": app["title"],
|
||||||
|
"repo_url": app["repo_url"],
|
||||||
|
"reasons": set(),
|
||||||
|
},
|
||||||
|
)["reasons"].add("image_basename")
|
||||||
|
|
||||||
|
# Fallback match: inferred app name from image aliases.
|
||||||
|
for candidate_name in candidate_names_from_unraid_image(raw_image):
|
||||||
|
candidate_key = normalize_text(candidate_name)
|
||||||
|
if not candidate_key:
|
||||||
|
continue
|
||||||
|
for app in app_records:
|
||||||
|
if candidate_key == app["id_key"] or (app["title_key"] and candidate_key == app["title_key"]):
|
||||||
|
key = (app["repo_url"], app["app_id"])
|
||||||
|
matched.setdefault(
|
||||||
|
key,
|
||||||
|
{
|
||||||
|
"app_id": app["app_id"],
|
||||||
|
"title": app["title"],
|
||||||
|
"repo_url": app["repo_url"],
|
||||||
|
"reasons": set(),
|
||||||
|
},
|
||||||
|
)["reasons"].add("name_alias")
|
||||||
|
|
||||||
|
matched_apps = []
|
||||||
|
for key in sorted(matched.keys(), key=lambda item: (item[0].lower(), item[1].lower())):
|
||||||
|
info = matched[key]
|
||||||
|
matched_apps.append(
|
||||||
|
{
|
||||||
|
"app_id": info["app_id"],
|
||||||
|
"title": info["title"],
|
||||||
|
"repo_url": info["repo_url"],
|
||||||
|
"reasons": sorted(info["reasons"]),
|
||||||
|
}
|
||||||
|
)
|
||||||
|
|
||||||
|
if not matched_apps:
|
||||||
|
status = "missing"
|
||||||
|
elif len(matched_apps) == 1:
|
||||||
|
status = "found"
|
||||||
|
else:
|
||||||
|
status = "ambiguous"
|
||||||
|
|
||||||
|
mapping.append(
|
||||||
|
{
|
||||||
|
"unraid_image": raw_image,
|
||||||
|
"normalized_image": normalized,
|
||||||
|
"match_status": status,
|
||||||
|
"matched_apps": matched_apps,
|
||||||
|
}
|
||||||
|
)
|
||||||
|
|
||||||
|
summary = {
|
||||||
|
"total": len(mapping),
|
||||||
|
"found": sum(1 for item in mapping if item["match_status"] == "found"),
|
||||||
|
"missing": sum(1 for item in mapping if item["match_status"] == "missing"),
|
||||||
|
"ambiguous": sum(1 for item in mapping if item["match_status"] == "ambiguous"),
|
||||||
|
}
|
||||||
|
|
||||||
|
output = {
|
||||||
|
"schema_version": 1,
|
||||||
|
"generated_at": dt.datetime.now(dt.timezone.utc).isoformat(),
|
||||||
|
"inventory_source": inventory_path,
|
||||||
|
"summary": summary,
|
||||||
|
"mapping": mapping,
|
||||||
|
}
|
||||||
|
|
||||||
|
with open(out_path, "w", encoding="utf-8") as handle:
|
||||||
|
yaml.safe_dump(output, handle, sort_keys=False, allow_unicode=False)
|
||||||
|
|
||||||
|
print(f"Wrote {out_path}")
|
||||||
|
print(f"Summary: total={summary['total']} found={summary['found']} missing={summary['missing']} ambiguous={summary['ambiguous']}")
|
||||||
|
PY
|
||||||
Reference in New Issue
Block a user