Can You Break the Model?

We built a scoring system for infrastructure autonomy — how much you actually control your data. Not quality. Not security. Not usability. Just one thing: can you stop it, leave with your data, and roll back? Every rating is based on testable, yes/no criteria — not opinions. We think two people following the same checklist will arrive at the same score. Prove us wrong.

4 pilot assessments · 10 structural tests · 5 diagnostic checks · all open source

The reproducibility test

Pick any technology. Evaluate it using our Assessment Criteria v1.1. Compare your result with ours.

If your score matches — the model works.
If it doesn't — tell us where the criteria are ambiguous. That's more valuable than agreement.

We went first — 4 pilot assessments

Syncthing

S3/3 — D5/5 — A3/T2

P2P file sync · no server, no account, no cloud

clean case

Show assessment details

S1: Pause — Yes

1a. Stop process? → Yes — docker stop works cleanly

1b. Data on disk? → Yes — ordinary files on filesystem, unchanged when stopped

1c. Restart continues? → Yes — resumes sync, no re-registration

S2: Exit — Yes

2a. Export mechanism? → Yes — files ARE your data, no export needed

2b. All data included? → Yes — all files always on disk

2c. Non-proprietary format? → Yes — original file formats preserved

2d. Another tool can consume? → Yes — any tool reads the files

S3: Recoverability — Yes

3a. Backup possible? → Yes — regular files, any backup tool works

3b. Restore works? → Yes — copy files back, reinstall, add folder

3c. Same state? → Yes — all data restored

Transparency — T2

Source: github.com/syncthing/syncthing · License: MPL-2.0 (OSI-approved)

Counterarguments

A3 challenge: relies on discovery servers → Resolution: self-hostable, LAN works without them

D5 challenge: usage reporting exists → Resolution: opt-in, anonymous, disabled by default in Docker

Full assessment →

KeePass

S3/3 — D5/5 — A3/T2

Local password vault · one encrypted file, zero cloud

clean case

Show assessment details

S1: Pause — Yes

1a. Stop process? → Yes — close application

1b. Data on disk? → Yes — .kdbx file on disk, encrypted

1c. Restart continues? → Yes — open app, open file, all entries present

S2: Exit — Yes

2a. Export mechanism? → Yes — CSV, XML, or copy .kdbx file

2b. All data included? → Yes — all entries, groups, attachments, history

2c. Non-proprietary format? → Yes — .kdbx is open format with published spec

2d. Another tool can consume? → Yes — KeePassXC, KeePassDX, KeeWeb, Bitwarden all read .kdbx

S3: Recoverability — Yes

3a. Backup possible? → Yes — single file, any backup tool. Auto .kdbx.bak on save

3b. Restore works? → Yes — copy file back, open in KeePass

3c. Same state? → Yes — file is self-contained

Transparency — T2

Source: sourceforge.net/projects/keepass · License: GPL-2.0 (OSI-approved)

Counterarguments

A3 challenge: no built-in sync → Resolution: by design, sync delegated to Syncthing/USB

D5 challenge: single maintainer (since 2003) → Resolution: KDBX format is open, 10+ independent implementations

Full assessment →

Nextcloud

S3/3 — D4/5 — A3/T2

Self-hosted cloud platform · push notification server is the catch

moderate

Show assessment details

S1: Pause — Yes

1a. Stop process? → Yes — docker stop nextcloud

1b. Data on disk? → Yes — files in volume, database in PostgreSQL volume

1c. Restart continues? → Yes — all files, calendar, contacts intact

S2: Exit — Yes

2a. Export mechanism? → Yes — files: copy volume. Calendar: CalDAV (.ics). Contacts: CardDAV (.vcf). DB: pg_dump

2b. All data included? → Yes — files, calendar, contacts, notes, bookmarks

2c. Non-proprietary format? → Yes — iCalendar, vCard, standard files

2d. Another tool can consume? → Yes — any CalDAV/CardDAV client

S3: Recoverability — Yes

3a. Backup possible? → Yes — file volume + database dump, documented

3b. Restore works? → Yes — restore volume + import DB → working instance

3c. Same state? → Yes — all data restored, sessions regenerated (expected)

Transparency — T2

Source: github.com/nextcloud/server · License: AGPL-3.0 (OSI-approved)

Diagnostic: D4/5 — one concern

D3 Hidden cost ⚠️: Push notification server (push-notifications.nextcloud.com) routes through Nextcloud GmbH by default. Self-hosting the push proxy is possible but underdocumented.

Counterarguments

A3 challenge: depends on Docker + PostgreSQL → Resolution: both are A3/T2 open-source infrastructure

A3 challenge: push notifications route through Nextcloud GmbH → Resolution: all features work without push, only mobile notifications delayed. Captured as D3.

T2 challenge: enterprise apps have CLA → Resolution: core server is AGPL-3.0, community apps don't require CLA

Full assessment →

Signal

S2/3 — D4/5 — A2/T2

E2E messaging · proprietary backup format caps Exit at Partial → A2

borderline

Show assessment details — this is the interesting one

S1: Pause — Yes

1a. Stop process? → Yes — close app, force-stop in Android

1b. Data on disk? → Yes — local encrypted SQLite database

1c. Restart continues? → Yes — all messages, contacts, settings intact

S2: Exit — Partial-E1 (proprietary format)

2a. Export mechanism? → Yes — Settings → Chats → Chat backups

2b. All data included? → Yes — messages, media, group info

2c. Non-proprietary format? → No — encrypted proprietary format, only Signal can read it

2d. Another tool can consume? → No — no third-party parser exists

S3: Recoverability — Yes

3a. Backup possible? → Yes — built-in encrypted backup to local storage

3b. Restore works? → Yes — new device, install Signal, restore from backup

3c. Same state? → Yes — messages and media restored

Aggregation (mechanical, no overrides)

S1=Yes, S2=Partial-E1, S3=Yes → table lookup → A2

Under v1.0 we rated this A3 with a judgment note. v1.1 removed overrides: Partial-E1 = A2, no exceptions. The model corrected itself.

Transparency — T2

Source: github.com/signalapp (client + server) · License: AGPL-3.0

Diagnostic: D4/5 — one concern

D3 Hidden cost ⚠️: Requires phone number for registration. Links identity to telecom infrastructure.

Counterarguments

A2 challenge: "but you CAN migrate all data to a new device" → Yes, but only to another Signal instance. Partial-E1 is clear: proprietary format.

A2 challenge: most messaging apps are A0, isn't A2 unfair? → A2 is not a penalty. It accurately describes: you can pause and recover, but cannot exit to a non-Signal tool.

What would make Signal A3? Document the backup format publicly + one third-party parser, OR adopt open export format (JSON message archive).

Full assessment →

Signal went from A3 to A2 when we tightened the criteria in v1.1. That's the model working as intended — no sacred cows.

How to participate

Read the Assessment Criteria v1.1 — 10 structural tests, 5 diagnostic checks, all yes/no.

Pick a technology — from our catalog or any tool you use daily.

Fill in the assessment template — one table per criterion, evidence for each.

Submit as a GitHub Issue or PR. We publish everything.

What counts as success

✅

A-level and T-level match — criteria are clear enough to be reproducible. The model works.

⚠️

D-score within ±1 — acceptable. Diagnostic questions involve more judgment than structural ones.

❌

A or T differ — the most valuable outcome. It means the criteria have an ambiguity we need to fix. Your assessment shows us exactly where.

Every disagreement makes the model better.

We will credit every independent assessment in the catalog — whether it agrees with ours or not.
The first external validations carry particular weight.

Submit your assessment →