Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
6e48f0d
Add submodules for OPResource + OPDB
clouths Mar 15, 2026
5af7381
Add container support using linux containers, add migration script fo…
clouths Mar 15, 2026
26512e6
Update mssql-tools to a more recent version (18) to fix migration, up…
clouths Mar 16, 2026
0689ed4
Fix comment in .env.local, remove preparePatch from the migration scr…
clouths Mar 16, 2026
1073ced
Move db persistent storage into a docker volume
clouths Mar 17, 2026
0fa79c6
Most fo the refactor from System.Drawing to SkiaSharp is done.
clouths Mar 21, 2026
decdcfd
Fix migration order of copy to ensure the generated perpetuum.ini is …
clouths Mar 21, 2026
67f8314
Add DefinitionConfig refactor
clouths Mar 21, 2026
b31e821
Update db submodule + migration script to use P34 and P35 full sql files
clouths Mar 21, 2026
1eba63b
Update port mapping to allow more ports for the server to fix black s…
clouths Mar 24, 2026
984ed0a
Fix port mapping, defined extra SERVER_PORTS for port range to forewa…
clouths Mar 24, 2026
0bf693c
Disable EnlistTransactaction
clouths Mar 25, 2026
c19dedf
Add documentation in the readme for how to setup the containers, make…
clouths Mar 28, 2026
17c38fa
Update db image to use mssql-2025
clouths Mar 28, 2026
afe07ea
Fix WithCanvas extension to preserve the Bitmap using SKCanvas instea…
clouths Mar 28, 2026
cba0e76
Fix font.size to use Size property
clouths Mar 28, 2026
8a40610
Fix pixel check to use Alpha instead of the Luminance
clouths Mar 29, 2026
005b8ce
Fix typo, replace usage of manual creation of Area with size.ToArea()…
clouths Mar 29, 2026
14a1e65
Fix PresenceConfiguration to use the Configuration size instead of th…
clouths Mar 29, 2026
c1a425d
Merge develop branch into linux+containerize
clouths Aug 30, 2026
a10dcd8
Merge branch 'develop' of github.com:OpenPerpetuum/PerpetuumServer2 i…
clouths Aug 30, 2026
3adb6bf
Try fix merge of FakeDb.cs causing ci build error
clouths Aug 30, 2026
d8c5168
Add make commands to run tests (unit+integration) using the docker co…
clouths Aug 30, 2026
0af068c
Add Live_36 to migration, all integration tests passes now
clouths Aug 31, 2026
8256cf6
Add unit and integration tests
clouths Aug 31, 2026
a75e424
Merge branch 'develop' of github.com:OpenPerpetuum/PerpetuumServer2 i…
clouths Sep 2, 2026
4953b3d
Update db submodule
clouths Sep 2, 2026
87cd9d8
Add CI to run tests with docker
clouths Sep 2, 2026
2d884d9
Revert "Add CI to run tests with docker" for now, I'll create a new P…
clouths Sep 2, 2026
c9d5ec0
perf(migration): automate patch discovery and add snapshot caching
clouths Sep 4, 2026
da38e09
perf(server): add SIMD acceleration, terrain metadata, and idle throt…
clouths Sep 4, 2026
96d4826
fix(data): share connection within TransactionScope to prevent MSDTC …
clouths Sep 4, 2026
bb0d416
Update db submodule
clouths Sep 4, 2026
b8911a5
Fix an issue with SQL server when waking up from sleep using 1-core@1…
clouths Sep 5, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
52 changes: 52 additions & 0 deletions .env.local
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
# Environment for local development.
# Compatible with podman/docker with linux containers.

# Database settings
DB_PASSWORD="l3<Nv2-QIat4YK*ydt#u-^+VAAVVdytL"

# Uncomment to force a reset of the DB + run migration
# FORCE_MIGRATION="true"

# Port mapping setting
# Here you can modify the default values
# Only the SERVER and the ASSET ports are required
# The SERVER port is used to allow access to the server when you connect your client
# The SERVER ports is a range that needs to start from the SERVER_PORT + ~300
# Note Without proper SERVER_PORTS, the client will have a black screen when entering a zone
# The ASSET port is used by the client to fetch assets when connecting to the server
# The DB port is optional, used for debugging
SERVER_PORT=17700
SERVER_PORTS=17700-17900
ASSET_PORT=16999
DB_PORT=16998

ASSET_URL="http://localhost:${ASSET_PORT}"

GAME_ROOT="/data"

# Original data directory from Stream: Perpetuum Dedicated Server installer
PERPETUUM_DATA="./perpetuum-data"

# User-provided additional map layers assets
# TODO: Move to a guide/readme
# Required if you want to load the server with Gamma Islands
# Download URL: https://drive.google.com/file/d/1qDjPHbTSdal_aKN6SvYC_EoGW-COkwhJ/view?usp=sharing
# sha256sum:
# f4d42d5dbf6a61d2d7730b2058f5d2bb80a63d0994bbc84b906f19441bb8e425 GAMMA_LAYERS_2021_10_17_P26.rar
CUSTOM_LAYERS="./custom-layers"

# Connection string: Specific changes made to work on linux:
# - Added "sa" user, password to use SQL server authentication;
# - Added TrustServerCertificate=True to trust the self-signed certificate
# - Changed Server to the name of the database container: "db"
# - Removed due to incompatibility on linux: "Connection Reset=True;"
# - Removed Trusted_Connection since we don't want to use Windows Authentication (SPPI)
CONNECTION_STRING="Server=db;Database=perpetuumsa;User Id=sa;Password=${DB_PASSWORD};TrustServerCertificate=True;Pooling=True;Connection Timeout=30;Connection Lifetime=260;Min Pool Size=20;Max Pool Size=60;"


RUNTIME_IDENTIFIER="linux-x64" # TODO env for windows as well (windows-x64)
DISTRIBUTED_TRANSACTIONS=false # .NET on linux does not support Distributed Transactions

# Database memory limit config.
# Set to 4096 for local dev to prevent post-sleep spin loops; 8192+ for production
DB_MEMORY_LIMIT_MB=4096
9 changes: 9 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,15 @@ src/Perpetuum.ServerService2/data/layers/
src/Perpetuum.ServerService2/data/logs/
bin/
Releases/

custom-assets/

# User-provided additional map layers assets
custom-layers/

# Original directory folder from Stream: Perpetuum Dedicated Server installer
perpetuum-data/

.claude/settings.local.json
.planning/
*.log
Expand Down
6 changes: 6 additions & 0 deletions .gitmodules
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
[submodule "db"]
path = db
url = https://github.com/OpenPerpetuum/OPDB.git
[submodule "asset"]
path = asset
url = https://github.com/OpenPerpetuum/OPResource.git
46 changes: 46 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
help: ## Show this help message
@grep -E '^[a-zA-Z0-9_-]+:.*?## .*$$' $(MAKEFILE_LIST) | awk 'BEGIN {FS = ":.*?## "}; {printf "%-20s %s\n", $$1, $$2}'

up: ## Create and start the containers
./script/compose.sh up -d --build --remove-orphans --wait

start: ## Start the containers
./script/compose.sh start

stop: ## Stop the containers
./script/compose.sh stop

down: ## Stop and delete the containers
./script/compose.sh down

delete: ## Stop and delete the containers, also delete the volumes (openperpetuum-data, openperpetuum-db)
./script/compose.sh down -v

restart: down up ## Stop, delete and start the containers

reset: ## Stop, delete and start the containers with forced migration
FORCE_MIGRATION=true $(MAKE) restart

clean-cache: ## Delete migration snapshot cache to force full re-migration on next start
rm -f ./perpetuum-data/database/perpetuumsa_migrated.bak ./perpetuum-data/database/perpetuumsa_migrated.hash 2>/dev/null || true

log-asset: ## Follow asset logs
./script/compose.sh logs asset -f

log-db: ## Follow db logs
./script/compose.sh logs db -f

log-server: ## Follow server logs
./script/compose.sh logs server -f

test-unit: ## Run the unit test tier (2) in the test container, no database required
./script/compose.sh --profile test run --no-deps --build --rm test dotnet test src/Perpetuum.Tests/Perpetuum.Tests.csproj -c Release -p:Platform=x64 --no-build

test-integration: ## Run the integration test tier (3) in the test container, against the live database, bringing up db + migration first (migration is idempotent and exits when already done)
./script/compose.sh up -d db --wait
./script/compose.sh up migration
./script/compose.sh --profile test run --build --rm test dotnet test src/Perpetuum.Tests.Integration/Perpetuum.Tests.Integration.csproj -c Release -p:Platform=x64 --no-build

.PHONY: help up start stop down delete restart reset clean-cache log-asset log-db log-server test-unit test-integration


134 changes: 134 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,9 @@

# The Open Perpetuum Server 2

## Native Windows host

`Perpetuum.Server` is annotated `[SupportedOSPlatform("windows")]` and the Admin Tool is WPF. You need the .NET 8 SDK, a SQL Server instance, and the `perpetuumsa` database — see [OPDB](https://github.com/OpenPerpetuum/OPDB) for restore and patches.
## Running a local server

Windows and x64 only. The bootstrapper is annotated `[SupportedOSPlatform("windows")]` and the Admin
Expand Down Expand Up @@ -79,3 +82,134 @@ dotnet run -- "C:\PerpetuumServer\data"
The server is up when the log reads `>>>> Perpetuum Server State : [Online]`. Ctrl+C shuts it down;
a clean shutdown ends at `State : [Off]`.

## Docker compose

Local development runs in **Linux containers** (Docker or Podman).

`compose.yml` defines the asset server, SQL Server, migration job, and game server. Configuration lives in `.env.local`.

Two named volumes persist between restarts:

- `openperpetuum-data` — original `PerpetuumServer/data`, custom layers, and a generated `perpetuum.ini`
- `openperpetuum-db` — SQL Server files

`make` wraps the compose commands; you can call `docker compose` / `podman compose` yourself if you prefer.

### Requirements

- Docker or Podman (Linux containers)
- (optional) `make`
- Steam: Perpetuum Dedicated Server installed
- Latest gamma island layers: https://drive.google.com/file/d/1Xp0T1K57Pv-vjgmpXMG8Iea_ec0bWYR4/view?usp=drive_link
- Latest asset resource: https://drive.google.com/file/d/18fh8aRqMP1J7ycGBNGraFyQ31mMXZaq1/view?usp=drive_link

### 1. Clone and submodules

```sh
git clone https://github.com/OpenPerpetuum/PerpetuumServer2.git
# or: git clone git@github.com:OpenPerpetuum/PerpetuumServer2.git
cd PerpetuumServer2
git submodule init && git submodule update
```

Submodules:

- `db` (OPDB) — database migration files per game update
- `asset` (OPResource) — client resources served when a client connects (definitions, translations, gfx, layers, audio, custom bot models)

### 2. Custom resources

Do this whenever the gamma layers or asset pack are updated.

- Uncompress the gamma layers and copy every `.bin` into both:
- `asset/lang0000/layers/GAMMA_LAYERS_NEW`
- a new `custom-layers` directory (same files)
- Unarchive the asset resource and copy `gfx`, `sfx`, and `textures` into `asset/lang0000`
- Create `perpetuum-data` and copy the Dedicated Server installer `data` folder into it (`database`, `layers`)

Paths for `perpetuum-data` and `custom-layers` can be changed in `.env.local`.

### 3. Configuration

Edit `.env.local` for ports, the database password, paths, and the SQL connection string.

The migration job writes `perpetuum.ini` from `template/perpetuum.ini.template`. Do not copy the installer `perpetuum.ini` into the data volume — that file was written for `System.Data.SqlClient` and this server uses `Microsoft.Data.SqlClient`. The template already uses a Linux-compatible string: SQL authentication (`sa`), `TrustServerCertificate=True`, no `Trusted_Connection`, and no keywords the driver refuses (`Connection Reset`, `Network Library`, `Context Connection`).

Linux does not support distributed transactions. `.env.local` sets `DISTRIBUTED_TRANSACTIONS=false` for that reason.

`SERVER_PORTS` must be a range of about 300 ports starting at `SERVER_PORT` (default `17700-17900`). A single mapped port is enough to log in; entering a zone then shows a black screen.

### 4. Run the server

```sh
make up
```

This builds and starts the containers and runs migrations. The command returns before the game host is fully up; wait a few minutes.

```sh
make log-server
```

The server is ready for a client when you see lines such as `Unit enter to zone` or `Planthandler STOP SIGNAL received`.

### 5. Point the client at this host

- Open the client → **Server list** → **ADD PRIVATE SERVER**
- Name: `local`
- Address: `127.0.0.1:17700` (use `SERVER_PORT` from `.env.local` if you changed it)
- Connect, then log in with user `test` / password `test`

The first connect can take several minutes while the asset server transfers files.

### 6. Stop and Cache Management

```sh
make down # stop and remove containers; keep data and db volumes
make delete # also delete the docker volumes
make reset # force re-run full migration from scratch and refresh cache
make clean-cache # delete migration snapshot backup and hash
```

### Database Migration & Snapshot Cache

The migration container seeds configuration files and applies all database patches from the `db` (OPDB) submodule to the SQL Server database.

To optimize local development startup time from ~90s down to ~2s, the migration job employs an **automatic snapshot caching mechanism** with SHA-256 change detection:

```mermaid
flowchart TD
Start(["Start migration container"]) --> CheckDone{"/data/done exists && !FORCE_MIGRATION?"}
CheckDone -- "Yes" --> Skip(["Skip migration (0s)"])

CheckDone -- "No" --> SyncFiles["Sync layer assets & perpetuum.ini to /data"]
SyncFiles --> ComputeHash["Compute SHA-256 hash of all SQL patches, base .bak & scripts"]

ComputeHash --> CheckCache{"perpetuumsa_migrated.bak exists && Hash matches?"}

subgraph FastPath["Fast Path (Cache Hit: ~2s)"]
CheckCache -- "Yes (Cache Hit)" --> RestoreSnapshot["RESTORE DATABASE from snapshot (perpetuumsa_migrated.bak)"]
end

subgraph SlowPath["Full Migration (First Run / Patch Changed: ~90s)"]
CheckCache -- "No (Miss / Changed / Force)" --> CreateDB["Ensure perpetuumsa DB exists"]
CreateDB --> RestoreBase["RESTORE DATABASE from Steam base perpetuumsa.bak"]
RestoreBase --> DiscoverPatches["Auto-discover patches in numerical order (Pre_Alpha_* -> Live_*)"]
DiscoverPatches --> ApplyPatches["Apply SQL scripts (live_patch_*.sql, Raw_SQL, or *.sql)"]
ApplyPatches --> AddTestAccount["Add test account (TOOL_test_account.sql)"]
AddTestAccount --> BackupSnapshot["BACKUP DATABASE to perpetuumsa_migrated.bak WITH COMPRESSION"]
BackupSnapshot --> SaveHash["Save SHA-256 hash to perpetuumsa_migrated.hash"]
end

RestoreSnapshot --> MarkDone["touch /data/done"]
SaveHash --> MarkDone
MarkDone --> End(["Migration complete -> Game server starts"])
```

#### Key Features

- **Automated Patch Discovery**: Automatically iterates through `Pre_Alpha_*` and `Live_*` patch folders in version order. It runs consolidated patch files (`live_patch_*.sql` / `prealpha_patch_*.sql`), `Raw_SQL/*.sql`, or loose `*.sql` files, and copies any `Server/data` assets automatically.
- **Instant Restore on Volume Wipe**: When recreating containers with `make delete && make up`, the database snapshot (`perpetuum-data/database/perpetuumsa_migrated.bak`) is preserved on host disk and restored in ~2 seconds.
- **Zero-touch Invalidation**: If you modify, add, or delete any SQL patch in the `db/` submodule, the SHA-256 hash mismatch is detected automatically, triggering a full re-migration and snapshot update.
- **Clean / Force Options**: Use `make clean-cache` to delete the snapshot, or `make reset` (`FORCE_MIGRATION=true`) to force a fresh re-migration from the raw Steam base backup.

1 change: 1 addition & 0 deletions asset
Submodule asset added at e00fe9
Loading
Loading