diff options
Diffstat (limited to 'news/atom.xml')
| -rw-r--r-- | news/atom.xml | 1991 |
1 files changed, 1991 insertions, 0 deletions
diff --git a/news/atom.xml b/news/atom.xml new file mode 100644 index 0000000..660ecac --- /dev/null +++ b/news/atom.xml @@ -0,0 +1,1991 @@ +<?xml version="1.0" encoding="UTF-8"?> +<feed xmlns="http://www.w3.org/2005/Atom" xml:lang="en"> + <title>Tesseras - News</title> + <subtitle>P2P network for preserving human memories across millennia</subtitle> + <link rel="self" type="application/atom+xml" href="https://tesseras.net/news/atom.xml"/> + <link rel="alternate" type="text/html" href="https://tesseras.net/news/"/> + <generator uri="https://www.getzola.org/">Zola</generator> + <updated>2026-02-16T10:00:00+00:00</updated> + <id>https://tesseras.net/news/atom.xml</id> + <entry xml:lang="en"> + <title>Packaging Tesseras for Debian</title> + <published>2026-02-16T10:00:00+00:00</published> + <updated>2026-02-16T10:00:00+00:00</updated> + + <author> + <name> + + Unknown + + </name> + </author> + + <link rel="alternate" type="text/html" href="https://tesseras.net/news/packaging-debian/"/> + <id>https://tesseras.net/news/packaging-debian/</id> + + <content type="html" xml:base="https://tesseras.net/news/packaging-debian/"><p>Tesseras now ships a <code>.deb</code> package for Debian and Ubuntu. This post walks +through building and installing the package from source using <code>cargo-deb</code>.</p> +<h2 id="prerequisites">Prerequisites</h2> +<p>You need a working Rust toolchain and the required system libraries:</p> +<pre><code data-lang="sh">sudo apt install build-essential pkg-config libsqlite3-dev +rustup toolchain install stable +cargo install cargo-deb +</code></pre> +<h2 id="building">Building</h2> +<p>Clone the repository and run the <code>just deb</code> recipe:</p> +<pre><code data-lang="sh">git clone https://git.sr.ht/~ijanc/tesseras +cd tesseras +just deb +</code></pre> +<p>This recipe does three things:</p> +<ol> +<li><strong>Compiles</strong> <code>tesd</code> (the daemon) and <code>tes</code> (the CLI) in release mode with +<code>cargo build --release</code></li> +<li><strong>Generates shell completions</strong> for bash, zsh, and fish from the <code>tes</code> binary</li> +<li><strong>Packages</strong> everything into a <code>.deb</code> file with +<code>cargo deb -p tesseras-daemon --no-build</code></li> +</ol> +<p>The result is a <code>.deb</code> file in <code>target/debian/</code>.</p> +<h2 id="installing">Installing</h2> +<pre><code data-lang="sh">sudo dpkg -i target/debian/tesseras-daemon_*.deb +</code></pre> +<p>If there are missing dependencies, fix them with:</p> +<pre><code data-lang="sh">sudo apt install -f +</code></pre> +<h2 id="post-install-setup">Post-install setup</h2> +<p>The <code>postinst</code> script automatically creates a <code>tesseras</code> system user and the +data directory <code>/var/lib/tesseras</code>. To use the CLI without sudo, add yourself to +the group:</p> +<pre><code data-lang="sh">sudo usermod -aG tesseras $USER +</code></pre> +<p>Log out and back in, then start the daemon:</p> +<pre><code data-lang="sh">sudo systemctl enable --now tesd +</code></pre> +<h2 id="what-the-package-includes">What the package includes</h2> +<table><thead><tr><th>Path</th><th>Description</th></tr></thead><tbody> +<tr><td><code>/usr/bin/tesd</code></td><td>Full node daemon</td></tr> +<tr><td><code>/usr/bin/tes</code></td><td>CLI client</td></tr> +<tr><td><code>/etc/tesseras/config.toml</code></td><td>Default configuration (marked as conffile)</td></tr> +<tr><td><code>/lib/systemd/system/tesd.service</code></td><td>Systemd unit with security hardening</td></tr> +<tr><td>Shell completions</td><td>bash, zsh, and fish</td></tr> +</tbody></table> +<h2 id="how-cargo-deb-works">How cargo-deb works</h2> +<p>The packaging metadata lives in <code>crates/tesseras-daemon/Cargo.toml</code> under +<code>[package.metadata.deb]</code>. This section defines:</p> +<ul> +<li><strong>depends</strong> — runtime dependencies: <code>libc6</code> and <code>libsqlite3-0</code></li> +<li><strong>assets</strong> — files to include in the package (binaries, config, systemd unit, +shell completions)</li> +<li><strong>conf-files</strong> — files treated as configuration (preserved on upgrade)</li> +<li><strong>maintainer-scripts</strong> — <code>postinst</code> and <code>postrm</code> scripts in +<code>packaging/debian/scripts/</code></li> +<li><strong>systemd-units</strong> — automatic systemd integration</li> +</ul> +<p>The <code>postinst</code> script creates the <code>tesseras</code> system user and data directory on +install. The <code>postrm</code> script cleans up the user, group, and data directory only +on <code>purge</code> (not on simple removal).</p> +<h2 id="systemd-hardening">Systemd hardening</h2> +<p>The <code>tesd.service</code> unit includes security hardening directives:</p> +<pre><code data-lang="ini">NoNewPrivileges=true +ProtectSystem=strict +ProtectHome=true +ReadWritePaths=/var/lib/tesseras +PrivateTmp=true +PrivateDevices=true +ProtectKernelTunables=true +ProtectControlGroups=true +RestrictSUIDSGID=true +MemoryDenyWriteExecute=true +</code></pre> +<p>The daemon runs as the unprivileged <code>tesseras</code> user and can only write to +<code>/var/lib/tesseras</code>.</p> +<h2 id="deploying-to-a-remote-server">Deploying to a remote server</h2> +<p>The justfile includes a <code>deploy</code> recipe for pushing the <code>.deb</code> to a remote host:</p> +<pre><code data-lang="sh">just deploy bootstrap1.tesseras.net +</code></pre> +<p>This builds the <code>.deb</code>, copies it via <code>scp</code>, installs it with <code>dpkg -i</code>, and +restarts the <code>tesd</code> service.</p> +<h2 id="updating">Updating</h2> +<p>After pulling new changes, simply run <code>just deb</code> again and reinstall:</p> +<pre><code data-lang="sh">git pull +just deb +sudo dpkg -i target/debian/tesseras-daemon_*.deb +</code></pre> +</content> + + </entry> + <entry xml:lang="en"> + <title>Packaging Tesseras for Arch Linux</title> + <published>2026-02-16T09:00:00+00:00</published> + <updated>2026-02-16T09:00:00+00:00</updated> + + <author> + <name> + + Unknown + + </name> + </author> + + <link rel="alternate" type="text/html" href="https://tesseras.net/news/packaging-archlinux/"/> + <id>https://tesseras.net/news/packaging-archlinux/</id> + + <content type="html" xml:base="https://tesseras.net/news/packaging-archlinux/"><p>Tesseras now ships a PKGBUILD for Arch Linux. This post walks through building +and installing the package from source.</p> +<h2 id="prerequisites">Prerequisites</h2> +<p>You need a working Rust toolchain and the base-devel group:</p> +<pre><code data-lang="sh">sudo pacman -S --needed base-devel sqlite +rustup toolchain install stable +</code></pre> +<h2 id="building">Building</h2> +<p>Clone the repository and run the <code>just arch</code> recipe:</p> +<pre><code data-lang="sh">git clone https://git.sr.ht/~ijanc/tesseras +cd tesseras +just arch +</code></pre> +<p>This runs <code>makepkg -sf</code> inside <code>packaging/archlinux/</code>, which:</p> +<ol> +<li><strong>prepare</strong> — fetches Cargo dependencies with <code>cargo fetch --locked</code></li> +<li><strong>build</strong> — compiles <code>tesd</code> and <code>tes</code> (the CLI) in release mode</li> +<li><strong>package</strong> — installs binaries, systemd service, sysusers/tmpfiles configs, +shell completions (bash, zsh, fish), and a default config file</li> +</ol> +<p>The result is a <code>.pkg.tar.zst</code> file in <code>packaging/archlinux/</code>.</p> +<h2 id="installing">Installing</h2> +<pre><code data-lang="sh">sudo pacman -U packaging/archlinux/tesseras-*.pkg.tar.zst +</code></pre> +<h2 id="post-install-setup">Post-install setup</h2> +<p>The package creates a <code>tesseras</code> system user and group automatically via +systemd-sysusers. To use the CLI without sudo, add yourself to the group:</p> +<pre><code data-lang="sh">sudo usermod -aG tesseras $USER +</code></pre> +<p>Log out and back in, then start the daemon:</p> +<pre><code data-lang="sh">sudo systemctl enable --now tesd +</code></pre> +<h2 id="what-the-package-includes">What the package includes</h2> +<table><thead><tr><th>Path</th><th>Description</th></tr></thead><tbody> +<tr><td><code>/usr/bin/tesd</code></td><td>Full node daemon</td></tr> +<tr><td><code>/usr/bin/tes</code></td><td>CLI client</td></tr> +<tr><td><code>/etc/tesseras/config.toml</code></td><td>Default configuration (marked as backup)</td></tr> +<tr><td><code>/usr/lib/systemd/system/tesd.service</code></td><td>Systemd unit with security hardening</td></tr> +<tr><td><code>/usr/lib/sysusers.d/tesseras.conf</code></td><td>System user definition</td></tr> +<tr><td><code>/usr/lib/tmpfiles.d/tesseras.conf</code></td><td>Data directory <code>/var/lib/tesseras</code></td></tr> +<tr><td>Shell completions</td><td>bash, zsh, and fish</td></tr> +</tbody></table> +<h2 id="pkgbuild-details">PKGBUILD details</h2> +<p>The PKGBUILD builds directly from the local git checkout rather than downloading +a source tarball. The <code>TESSERAS_ROOT</code> environment variable points makepkg to the +workspace root. Cargo's target directory is set to <code>$srcdir/target</code> to keep +build artifacts inside the makepkg sandbox.</p> +<p>The package depends only on <code>sqlite</code> at runtime and <code>cargo</code> at build time.</p> +<h2 id="updating">Updating</h2> +<p>After pulling new changes, simply run <code>just arch</code> again and reinstall:</p> +<pre><code data-lang="sh">git pull +just arch +sudo pacman -U packaging/archlinux/tesseras-*.pkg.tar.zst +</code></pre> +</content> + + </entry> + <entry xml:lang="en"> + <title>Phase 4: Storage Deduplication</title> + <published>2026-02-15T23:00:00+00:00</published> + <updated>2026-02-15T23:00:00+00:00</updated> + + <author> + <name> + + Unknown + + </name> + </author> + + <link rel="alternate" type="text/html" href="https://tesseras.net/news/phase4-storage-deduplication/"/> + <id>https://tesseras.net/news/phase4-storage-deduplication/</id> + + <content type="html" xml:base="https://tesseras.net/news/phase4-storage-deduplication/"><p>When multiple tesseras share the same photo, the same audio clip, or the same +fragment data, the old storage layer kept separate copies of each. On a node +storing thousands of tesseras for the network, this duplication adds up fast. +Phase 4 continues with storage deduplication: a content-addressable store (CAS) +that ensures every unique piece of data is stored exactly once on disk, +regardless of how many tesseras reference it.</p> +<p>The design is simple and proven: hash the content with BLAKE3, use the hash as +the filename, and maintain a reference count in SQLite. When two tesseras +include the same 5 MB photo, one file exists on disk with a refcount of 2. When +one tessera is deleted, the refcount drops to 1 and the file stays. When the +last reference is released, a periodic sweep cleans up the orphan.</p> +<h2 id="what-was-built">What was built</h2> +<p><strong>CAS schema migration</strong> (<code>tesseras-storage/migrations/004_dedup.sql</code>) — Three +new tables:</p> +<ul> +<li><code>cas_objects</code> — tracks every object in the store: BLAKE3 hash (primary key), +byte size, reference count, and creation timestamp</li> +<li><code>blob_refs</code> — maps logical blob identifiers (tessera hash + memory hash + +filename) to CAS hashes, replacing the old filesystem path convention</li> +<li><code>fragment_refs</code> — maps logical fragment identifiers (tessera hash + fragment +index) to CAS hashes, replacing the old <code>fragments/</code> directory layout</li> +</ul> +<p>Indexes on the hash columns ensure O(1) lookups during reads and reference +counting.</p> +<p><strong>CasStore</strong> (<code>tesseras-storage/src/cas.rs</code>) — The core content-addressable +storage engine. Files are stored under a two-level prefix directory: +<code>&lt;root&gt;/&lt;2-char-hex-prefix&gt;/&lt;full-hash&gt;.blob</code>. The store provides five +operations:</p> +<ul> +<li><code>put(hash, data)</code> — writes data to disk if not already present, increments +refcount. Returns whether a dedup hit occurred.</li> +<li><code>get(hash)</code> — reads data from disk by hash</li> +<li><code>release(hash)</code> — decrements refcount. If it reaches zero, the on-disk file is +deleted immediately.</li> +<li><code>contains(hash)</code> — checks existence without reading</li> +<li><code>ref_count(hash)</code> — returns the current reference count</li> +</ul> +<p>All operations are atomic within a single SQLite transaction. The refcount is +the source of truth — if the refcount says the object exists, the file must be +on disk.</p> +<p><strong>CAS-backed FsBlobStore</strong> (<code>tesseras-storage/src/blob.rs</code>) — Rewritten to +delegate all storage to the CAS. When a blob is written, its BLAKE3 hash is +computed and passed to <code>cas.put()</code>. A row in <code>blob_refs</code> maps the logical path +(tessera + memory + filename) to the CAS hash. Reads look up the CAS hash via +<code>blob_refs</code> and fetch from <code>cas.get()</code>. Deleting a tessera releases all its blob +references in a single transaction.</p> +<p><strong>CAS-backed FsFragmentStore</strong> (<code>tesseras-storage/src/fragment.rs</code>) — Same +pattern for erasure-coded fragments. Each fragment's BLAKE3 checksum is already +computed during Reed-Solomon encoding, so it's used directly as the CAS key. +Fragment verification now checks the CAS hash instead of recomputing from +scratch — if the CAS says the data is intact, it is.</p> +<p><strong>Sweep garbage collector</strong> (<code>cas.rs:sweep()</code>) — A periodic GC pass that handles +three edge cases the normal refcount path can't:</p> +<ol> +<li><strong>Orphan files</strong> — files on disk with no corresponding row in <code>cas_objects</code>. +Can happen after a crash mid-write. Files younger than 1 hour are skipped +(grace period for in-flight writes); older orphans are deleted.</li> +<li><strong>Leaked refcounts</strong> — rows in <code>cas_objects</code> with refcount zero that weren't +cleaned up (e.g., if the process died between decrementing and deleting). +These rows are removed.</li> +<li><strong>Idempotent</strong> — running sweep twice produces the same result.</li> +</ol> +<p>The sweep is wired into the existing repair loop in <code>tesseras-replication</code>, so +it runs automatically every 24 hours alongside fragment health checks.</p> +<p><strong>Migration from old layout</strong> (<code>tesseras-storage/src/migration.rs</code>) — A +copy-first migration strategy that moves data from the old directory-based +layout (<code>blobs/&lt;tessera&gt;/&lt;memory&gt;/&lt;file&gt;</code> and +<code>fragments/&lt;tessera&gt;/&lt;index&gt;.shard</code>) into the CAS. The migration:</p> +<ol> +<li>Checks the storage version in <code>storage_meta</code> (version 1 = old layout, version +2 = CAS)</li> +<li>Walks the old <code>blobs/</code> and <code>fragments/</code> directories</li> +<li>Computes BLAKE3 hashes and inserts into CAS via <code>put()</code> — duplicates are +automatically deduplicated</li> +<li>Creates corresponding <code>blob_refs</code> / <code>fragment_refs</code> entries</li> +<li>Removes old directories only after all data is safely in CAS</li> +<li>Updates the storage version to 2</li> +</ol> +<p>The migration runs on daemon startup, is idempotent (safe to re-run), and +reports statistics: files migrated, duplicates found, bytes saved.</p> +<p><strong>Prometheus metrics</strong> (<code>tesseras-storage/src/metrics.rs</code>) — Ten new metrics for +observability:</p> +<table><thead><tr><th>Metric</th><th>Description</th></tr></thead><tbody> +<tr><td><code>cas_objects_total</code></td><td>Total unique objects in the CAS</td></tr> +<tr><td><code>cas_bytes_total</code></td><td>Total bytes stored</td></tr> +<tr><td><code>cas_dedup_hits_total</code></td><td>Number of writes that found an existing object</td></tr> +<tr><td><code>cas_bytes_saved_total</code></td><td>Bytes saved by deduplication</td></tr> +<tr><td><code>cas_gc_refcount_deletions_total</code></td><td>Objects deleted when refcount reached zero</td></tr> +<tr><td><code>cas_gc_sweep_orphans_cleaned_total</code></td><td>Orphan files removed by sweep</td></tr> +<tr><td><code>cas_gc_sweep_leaked_refs_cleaned_total</code></td><td>Leaked refcount rows cleaned</td></tr> +<tr><td><code>cas_gc_sweep_skipped_young_total</code></td><td>Young orphans skipped (grace period)</td></tr> +<tr><td><code>cas_gc_sweep_duration_seconds</code></td><td>Time spent in sweep GC</td></tr> +</tbody></table> +<p><strong>Property-based tests</strong> — Two proptest tests verify CAS invariants under random +inputs:</p> +<ul> +<li><code>refcount_matches_actual_refs</code> — after N random put/release operations, the +refcount always matches the actual number of outstanding references</li> +<li><code>cas_path_is_deterministic</code> — the same hash always produces the same +filesystem path</li> +</ul> +<p><strong>Integration test updates</strong> — All integration tests across <code>tesseras-core</code>, +<code>tesseras-replication</code>, <code>tesseras-embedded</code>, and <code>tesseras-cli</code> updated for the +new CAS-backed constructors. Tamper-detection tests updated to work with the CAS +directory layout.</p> +<p>347 tests pass across the workspace. Clippy clean with <code>-D warnings</code>.</p> +<h2 id="architecture-decisions">Architecture decisions</h2> +<ul> +<li><strong>BLAKE3 as CAS key</strong>: the content hash we already compute for integrity +verification doubles as the deduplication key. No additional hashing step — +the hash computed during <code>create</code> or <code>replicate</code> is reused as the CAS address.</li> +<li><strong>SQLite refcount over filesystem reflinks</strong>: we considered using +filesystem-level copy-on-write (reflinks on btrfs/XFS), but that would tie +Tesseras to specific filesystems. SQLite refcounting works on any filesystem, +including FAT32 on cheap USB drives and ext4 on Raspberry Pis.</li> +<li><strong>Two-level hex prefix directories</strong>: storing all CAS objects in a flat +directory would slow down filesystems with millions of entries. The +<code>&lt;2-char prefix&gt;/</code> split limits any single directory to ~65k entries before a +second prefix level is needed. This matches the approach used by Git's object +store.</li> +<li><strong>Grace period for orphan files</strong>: the sweep GC skips files younger than 1 +hour to avoid deleting objects that are being written by a concurrent +operation. This is a pragmatic choice — it trades a small window of potential +orphans for crash safety without requiring fsync or two-phase commit.</li> +<li><strong>Copy-first migration</strong>: the migration copies data to CAS before removing old +directories. If the process is interrupted, the old data is still intact and +migration can be re-run. This is slower than moving files but guarantees no +data loss.</li> +<li><strong>Sweep in repair loop</strong>: rather than adding a separate GC timer, the CAS +sweep piggybacks on the existing 24-hour repair loop. This keeps the daemon +simple — one background maintenance cycle handles both fragment health and +storage cleanup.</li> +</ul> +<h2 id="what-comes-next">What comes next</h2> +<ul> +<li><strong>Phase 4 continued</strong> — security audits, OS packaging (Alpine, Arch, Debian, +OpenBSD, FreeBSD)</li> +<li><strong>Phase 5: Exploration and Culture</strong> — public tessera browser by +era/location/theme/language, institutional curation, genealogy integration +(FamilySearch, Ancestry), physical media export (M-DISC, microfilm, acid-free +paper with QR), AI-assisted context</li> +</ul> +<p>Storage deduplication completes the storage efficiency story for Tesseras. A +node that stores fragments for thousands of users — common for institutional +nodes and always-on full nodes — now pays the disk cost of unique data only. +Combined with Reed-Solomon erasure coding (which already minimizes redundancy at +the network level), the system achieves efficient storage at both the local and +distributed layers.</p> +</content> + + </entry> + <entry xml:lang="en"> + <title>Phase 4: Institutional Node Onboarding</title> + <published>2026-02-15T22:00:00+00:00</published> + <updated>2026-02-15T22:00:00+00:00</updated> + + <author> + <name> + + Unknown + + </name> + </author> + + <link rel="alternate" type="text/html" href="https://tesseras.net/news/phase4-institutional-onboarding/"/> + <id>https://tesseras.net/news/phase4-institutional-onboarding/</id> + + <content type="html" xml:base="https://tesseras.net/news/phase4-institutional-onboarding/"><p>A P2P network of individuals is fragile. Hard drives die, phones get lost, +people lose interest. The long-term survival of humanity's memories depends on +institutions — libraries, archives, museums, universities — that measure their +lifetimes in centuries. Phase 4 continues with institutional node onboarding: +verified organizations can now pledge storage, run searchable indexes, and +participate in the network with a distinct identity.</p> +<p>The design follows a principle of trust but verify: institutions identify +themselves via DNS TXT records (the same mechanism used by SPF, DKIM, and DMARC +for email), pledge a storage budget, and receive reciprocity exemptions so they +can store fragments for others without expecting anything in return. In +exchange, the network treats their fragments as higher-quality replicas and +limits over-reliance on any single institution through diversity constraints.</p> +<h2 id="what-was-built">What was built</h2> +<p><strong>Capability bits</strong> (<code>tesseras-core/src/network.rs</code>) — Two new flags added to +the <code>Capabilities</code> bitfield: <code>INSTITUTIONAL</code> (bit 7) and <code>SEARCH_INDEX</code> (bit 8). +A new <code>institutional_default()</code> constructor returns the full Phase 2 capability +set plus these two bits and <code>RELAY</code>. Normal nodes advertise <code>phase2_default()</code> +which lacks institutional flags. Serialization roundtrip tests verify the new +bits survive MessagePack encoding.</p> +<p><strong>Search types</strong> (<code>tesseras-core/src/search.rs</code>) — Three new domain types for +the search subsystem:</p> +<ul> +<li><code>SearchFilters</code> — query parameters: <code>memory_type</code>, <code>visibility</code>, <code>language</code>, +<code>date_range</code>, <code>geo</code> (bounding box), <code>page</code>, <code>page_size</code></li> +<li><code>SearchHit</code> — a single result: content hash plus a <code>MetadataExcerpt</code> (title, +description, memory type, creation date, visibility, language, tags)</li> +<li><code>GeoFilter</code> — bounding box with <code>min_lat</code>, <code>max_lat</code>, <code>min_lon</code>, <code>max_lon</code> for +spatial queries</li> +</ul> +<p>All types derive <code>Serialize</code>/<code>Deserialize</code> for wire transport and +<code>Clone</code>/<code>Debug</code> for diagnostics.</p> +<p><strong>Institutional daemon config</strong> (<code>tesd/src/config.rs</code>) — A new <code>[institutional]</code> +TOML section with <code>domain</code> (the DNS domain to verify), <code>pledge_bytes</code> (storage +commitment in bytes), and <code>search_enabled</code> (toggle for the FTS5 index). The +<code>to_dht_config()</code> method now sets <code>Capabilities::institutional_default()</code> when +institutional config is present, so institutional nodes advertise the right +capability bits in Pong responses.</p> +<p><strong>DNS TXT verification</strong> (<code>tesd/src/institutional.rs</code>) — Async DNS resolution +using <code>hickory-resolver</code> to verify institutional identity. The daemon looks up +<code>_tesseras.&lt;domain&gt;</code> TXT records and parses key-value fields: <code>v</code> (version), +<code>node</code> (hex-encoded node ID), and <code>pledge</code> (storage pledge in bytes). +Verification checks:</p> +<ol> +<li>A TXT record exists at <code>_tesseras.&lt;domain&gt;</code></li> +<li>The <code>node</code> field matches the daemon's own node ID</li> +<li>The <code>pledge</code> field is present and valid</li> +</ol> +<p>On startup, the daemon attempts DNS verification. If it succeeds, the node runs +with institutional capabilities. If it fails, the node logs a warning and +downgrades to a normal full node — no crash, no manual intervention.</p> +<p><strong>CLI setup command</strong> (<code>tesseras-cli/src/institutional.rs</code>) — A new +<code>institutional setup</code> subcommand that guides operators through onboarding:</p> +<ol> +<li>Reads the node's identity from the data directory</li> +<li>Prompts for domain name and pledge size</li> +<li>Generates the exact DNS TXT record to add: +<code>v=tesseras1 node=&lt;hex&gt; pledge=&lt;bytes&gt;</code></li> +<li>Writes the institutional section to the daemon's config file</li> +<li>Prints next steps: add the TXT record, restart the daemon</li> +</ol> +<p><strong>SQLite search index</strong> (<code>tesseras-storage</code>) — A migration +(<code>003_institutional.sql</code>) that creates three structures:</p> +<ul> +<li><code>search_content</code> — an FTS5 virtual table for full-text search over tessera +metadata (title, description, creator, tags, language)</li> +<li><code>geo_index</code> — an R-tree virtual table for spatial bounding-box queries over +latitude/longitude</li> +<li><code>geo_map</code> — a mapping table linking R-tree row IDs to content hashes</li> +</ul> +<p>The <code>SqliteSearchIndex</code> adapter implements the <code>SearchIndex</code> port trait with +<code>index_tessera()</code> (insert/update) and <code>search()</code> (query with filters). FTS5 +queries support natural language search; geo queries use R-tree <code>INTERSECT</code> for +bounding box lookups. Results are ranked by FTS5 relevance score.</p> +<p>The migration also adds an <code>is_institutional</code> column to the <code>reciprocity</code> table, +handled idempotently via <code>pragma_table_info</code> checks (SQLite's +<code>ALTER TABLE ADD COLUMN</code> lacks <code>IF NOT EXISTS</code>).</p> +<p><strong>Reciprocity bypass</strong> (<code>tesseras-replication/src/service.rs</code>) — Institutional +nodes are exempt from reciprocity checks. When <code>receive_fragment()</code> is called, +if the sender's node ID is marked as institutional in the reciprocity ledger, +the balance check is skipped entirely. This means institutions can store +fragments for the entire network without needing to "earn" credits first — their +DNS-verified identity and storage pledge serve as their credential.</p> +<p><strong>Node-type diversity constraint</strong> (<code>tesseras-replication/src/distributor.rs</code>) — +A new <code>apply_institutional_diversity()</code> function limits how many replicas of a +single tessera can land on institutional nodes. The cap is +<code>ceil(replication_factor / 3.5)</code> — with the default <code>r=7</code>, at most 2 of 7 +replicas go to institutions. This prevents the network from becoming dependent +on a small number of large institutions: if a university's servers go down, at +least 5 replicas remain on independent nodes.</p> +<p><strong>DHT message extensions</strong> (<code>tesseras-dht/src/message.rs</code>) — Two new message +variants:</p> +<table><thead><tr><th>Message</th><th>Purpose</th></tr></thead><tbody> +<tr><td><code>Search</code></td><td>Client sends query string, filters, and page number</td></tr> +<tr><td><code>SearchResult</code></td><td>Institutional node responds with hits and total count</td></tr> +</tbody></table> +<p>The <code>encode()</code> function was switched from positional to named MessagePack +serialization (<code>rmp_serde::to_vec_named</code>) to handle <code>SearchFilters</code>' optional +fields correctly — positional encoding breaks when <code>skip_serializing_if</code> omits +fields.</p> +<p><strong>Prometheus metrics</strong> (<code>tesd/src/metrics.rs</code>) — Eight institutional-specific +metrics:</p> +<ul> +<li><code>tesseras_institutional_pledge_bytes</code> — configured storage pledge</li> +<li><code>tesseras_institutional_stored_bytes</code> — actual bytes stored</li> +<li><code>tesseras_institutional_pledge_utilization_ratio</code> — stored/pledged ratio</li> +<li><code>tesseras_institutional_peers_served</code> — unique peers served fragments</li> +<li><code>tesseras_institutional_search_index_total</code> — tesseras in the search index</li> +<li><code>tesseras_institutional_search_queries_total</code> — search queries received</li> +<li><code>tesseras_institutional_dns_verification_status</code> — 1 if DNS verified, 0 +otherwise</li> +<li><code>tesseras_institutional_dns_verification_last</code> — Unix timestamp of last +verification</li> +</ul> +<p><strong>Integration tests</strong> — Two tests in +<code>tesseras-replication/tests/integration.rs</code>:</p> +<ul> +<li><code>institutional_peer_bypasses_reciprocity</code> — verifies that an institutional +peer with a massive deficit (-999,999 balance) is still allowed to store +fragments, while a non-institutional peer with the same deficit is rejected</li> +<li><code>institutional_node_accepts_fragment_despite_deficit</code> — full async test using +<code>ReplicationService</code> with mocked DHT, fragment store, reciprocity ledger, and +blob store: sends a fragment from an institutional sender and verifies it's +accepted</li> +</ul> +<p>322 tests pass across the workspace. Clippy clean with <code>-D warnings</code>.</p> +<h2 id="architecture-decisions">Architecture decisions</h2> +<ul> +<li><strong>DNS TXT over PKI or blockchain</strong>: DNS is universally deployed, universally +understood, and already used for domain verification (SPF, DKIM, Let's +Encrypt). Institutions already manage DNS. No certificate authority, no token, +no on-chain transaction — just a TXT record. If an institution loses control +of their domain, the verification naturally fails on the next check.</li> +<li><strong>Graceful degradation on DNS failure</strong>: if DNS verification fails at startup, +the daemon downgrades to a normal full node instead of refusing to start. This +prevents operational incidents — a DNS misconfiguration shouldn't take a node +offline.</li> +<li><strong>Diversity cap at <code>ceil(r / 3.5)</code></strong>: with <code>r=7</code>, at most 2 replicas go to +institutions. This is conservative — it ensures the network never depends on +institutions for majority quorum, while still benefiting from their storage +capacity and uptime.</li> +<li><strong>Named MessagePack encoding</strong>: switching from positional to named encoding +adds ~15% overhead per message but eliminates a class of serialization bugs +when optional fields are present. The DHT is not bandwidth-constrained at the +message level, so the tradeoff is worth it.</li> +<li><strong>Reciprocity exemption over credit grants</strong>: rather than giving institutions +a large initial credit balance (which is arbitrary and needs tuning), we +exempt them entirely. Their DNS-verified identity and public storage pledge +replace the bilateral reciprocity mechanism.</li> +<li><strong>FTS5 + R-tree in SQLite</strong>: full-text search and spatial indexing are built +into SQLite as loadable extensions. No external search engine (Elasticsearch, +Meilisearch) needed. This keeps the deployment a single binary with a single +database file — critical for institutional operators who may not have a DevOps +team.</li> +</ul> +<h2 id="what-comes-next">What comes next</h2> +<ul> +<li><strong>Phase 4 continued</strong> — storage deduplication (content-addressable store with +BLAKE3 keying), security audits, OS packaging (Alpine, Arch, Debian, OpenBSD, +FreeBSD)</li> +<li><strong>Phase 5: Exploration and Culture</strong> — public tessera browser by +era/location/theme/language, institutional curation, genealogy integration +(FamilySearch, Ancestry), physical media export (M-DISC, microfilm, acid-free +paper with QR), AI-assisted context</li> +</ul> +<p>Institutional onboarding closes a critical gap in Tesseras' preservation model. +Individual nodes provide grassroots resilience — thousands of devices across the +globe, each storing a few fragments. Institutional nodes provide anchoring — +organizations with professional infrastructure, redundant storage, and +multi-decade operational horizons. Together, they form a network where memories +can outlast both individual devices and individual institutions.</p> +</content> + + </entry> + <entry xml:lang="en"> + <title>Phase 4: Performance Tuning</title> + <published>2026-02-15T20:00:00+00:00</published> + <updated>2026-02-15T20:00:00+00:00</updated> + + <author> + <name> + + Unknown + + </name> + </author> + + <link rel="alternate" type="text/html" href="https://tesseras.net/news/phase4-performance-tuning/"/> + <id>https://tesseras.net/news/phase4-performance-tuning/</id> + + <content type="html" xml:base="https://tesseras.net/news/phase4-performance-tuning/"><p>A P2P network that can traverse NATs but chokes on its own I/O is not much use. +Phase 4 continues with performance tuning: centralizing database configuration, +caching fragment blobs in memory, managing QUIC connection lifecycles, and +eliminating unnecessary disk reads from the attestation hot path.</p> +<p>The guiding principle was the same as the rest of Tesseras: do the simplest +thing that actually works. No custom allocators, no lock-free data structures, +no premature complexity. A centralized <code>StorageConfig</code>, an LRU cache, a +connection reaper, and a targeted fix to avoid re-reading blobs that were +already checksummed.</p> +<h2 id="what-was-built">What was built</h2> +<p><strong>Centralized SQLite configuration</strong> (<code>tesseras-storage/src/database.rs</code>) — A +new <code>StorageConfig</code> struct and <code>open_database()</code> / <code>open_in_memory()</code> functions +that apply all SQLite pragmas in one place: WAL journal mode, foreign keys, +synchronous mode (NORMAL by default, FULL for unstable hardware like RPi + SD +card), busy timeout, page cache size, and WAL autocheckpoint interval. +Previously, each call site opened a connection and applied pragmas ad hoc. Now +the daemon, CLI, and tests all go through the same path. 7 tests covering +foreign keys, busy timeout, journal mode, migrations, synchronous modes, and +on-disk WAL file creation.</p> +<p><strong>LRU fragment cache</strong> (<code>tesseras-storage/src/cache.rs</code>) — A +<code>CachedFragmentStore</code> that wraps any <code>FragmentStore</code> with a byte-aware LRU +cache. Fragment blobs are cached on read and invalidated on write or delete. +When the cache exceeds its configured byte limit, the least recently used +entries are evicted. The cache is transparent: it implements <code>FragmentStore</code> +itself, so the rest of the stack doesn't know it's there. Optional Prometheus +metrics track hits, misses, and current byte usage. 3 tests: cache hit avoids +inner read, store invalidates cache, eviction when over max bytes.</p> +<p><strong>Prometheus storage metrics</strong> (<code>tesseras-storage/src/metrics.rs</code>) — A +<code>StorageMetrics</code> struct with three counters/gauges: <code>fragment_cache_hits</code>, +<code>fragment_cache_misses</code>, and <code>fragment_cache_bytes</code>. Registered with the +Prometheus registry and wired into the fragment cache via <code>with_metrics()</code>.</p> +<p><strong>Attestation hot path fix</strong> (<code>tesseras-replication/src/service.rs</code>) — The +attestation flow previously read every fragment blob from disk and recomputed +its BLAKE3 checksum. Since <code>list_fragments()</code> already returns <code>FragmentId</code> with +a stored checksum, the fix is trivial: use <code>frag.checksum</code> instead of +<code>blake3::hash(&amp;data)</code>. This eliminates one disk read per fragment during +attestation — for a tessera with 100 fragments, that's 100 fewer reads. A test +with <code>expect_read_fragment().never()</code> verifies no blob reads happen during +attestation.</p> +<p><strong>QUIC connection pool lifecycle</strong> (<code>tesseras-net/src/quinn_transport.rs</code>) — A +<code>PoolConfig</code> struct controlling max connections, idle timeout, and reaper +interval. <code>PooledConnection</code> wraps each <code>quinn::Connection</code> with a <code>last_used</code> +timestamp. When the pool reaches capacity, the oldest idle connection is evicted +before opening a new one. A background reaper task (Tokio spawn) periodically +closes connections that have been idle beyond the timeout. 4 new pool metrics: +<code>tesseras_conn_pool_size</code>, <code>pool_hits_total</code>, <code>pool_misses_total</code>, +<code>pool_evictions_total</code>.</p> +<p><strong>Daemon integration</strong> (<code>tesd/src/config.rs</code>, <code>main.rs</code>) — A new <code>[performance]</code> +section in the TOML config with fields for SQLite cache size, synchronous mode, +busy timeout, fragment cache size, max connections, idle timeout, and reaper +interval. The daemon's <code>main()</code> now calls <code>open_database()</code> with the configured +<code>StorageConfig</code>, wraps <code>FsFragmentStore</code> with <code>CachedFragmentStore</code>, and binds +QUIC with the configured <code>PoolConfig</code>. The direct <code>rusqlite</code> dependency was +removed from the daemon crate.</p> +<p><strong>CLI migration</strong> (<code>tesseras-cli/src/commands/init.rs</code>, <code>create.rs</code>) — Both +<code>init</code> and <code>create</code> commands now use <code>tesseras_storage::open_database()</code> with +the default <code>StorageConfig</code> instead of opening raw <code>rusqlite</code> connections. The +<code>rusqlite</code> dependency was removed from the CLI crate.</p> +<h2 id="architecture-decisions">Architecture decisions</h2> +<ul> +<li><strong>Decorator pattern for caching</strong>: <code>CachedFragmentStore</code> wraps +<code>Box&lt;dyn FragmentStore&gt;</code> and implements <code>FragmentStore</code> itself. This means +caching is opt-in, composable, and invisible to consumers. The daemon enables +it; tests can skip it.</li> +<li><strong>Byte-aware eviction</strong>: the LRU cache tracks total bytes, not entry count. +Fragment blobs vary wildly in size (a 4KB text fragment vs a 2MB photo shard), +so counting entries would give a misleading picture of memory usage.</li> +<li><strong>No connection pool crate</strong>: instead of pulling in a generic pool library, +the connection pool is a thin wrapper around +<code>DashMap&lt;SocketAddr, PooledConnection&gt;</code> with a Tokio reaper. QUIC connections +are multiplexed, so the "pool" is really about lifecycle management (idle +cleanup, max connections) rather than borrowing/returning.</li> +<li><strong>Stored checksums over re-reads</strong>: the attestation fix is intentionally +minimal — one line changed, one disk read removed per fragment. The checksums +were already stored in SQLite by <code>store_fragment()</code>, they just weren't being +used.</li> +<li><strong>Centralized pragma configuration</strong>: a single <code>StorageConfig</code> struct replaces +scattered <code>PRAGMA</code> calls. The <code>sqlite_synchronous_full</code> flag exists +specifically for Raspberry Pi deployments where the kernel can crash and lose +un-checkpointed WAL transactions.</li> +</ul> +<h2 id="what-comes-next">What comes next</h2> +<ul> +<li><strong>Phase 4 continued</strong> — Shamir's Secret Sharing for heirs, sealed tesseras +(time-lock encryption), security audits, institutional node onboarding, +storage deduplication, OS packaging</li> +<li><strong>Phase 5: Exploration and Culture</strong> — public tessera browser by +era/location/theme/language, institutional curation, genealogy integration, +physical media export (M-DISC, microfilm, acid-free paper with QR)</li> +</ul> +<p>With performance tuning in place, Tesseras handles the common case efficiently: +fragment reads hit the LRU cache, attestation skips disk I/O, idle QUIC +connections are reaped automatically, and SQLite is configured consistently +across the entire stack. The next steps focus on cryptographic features (Shamir, +time-lock) and hardening for production deployment.</p> +</content> + + </entry> + <entry xml:lang="en"> + <title>Phase 4: Verify Without Installing Anything</title> + <published>2026-02-15T20:00:00+00:00</published> + <updated>2026-02-15T20:00:00+00:00</updated> + + <author> + <name> + + Unknown + + </name> + </author> + + <link rel="alternate" type="text/html" href="https://tesseras.net/news/phase4-wasm-browser-verification/"/> + <id>https://tesseras.net/news/phase4-wasm-browser-verification/</id> + + <content type="html" xml:base="https://tesseras.net/news/phase4-wasm-browser-verification/"><p>Trust shouldn't require installing software. If someone sends you a tessera — a +bundle of preserved memories — you should be able to verify it's genuine and +unmodified without downloading an app, creating an account, or trusting a +server. That's what <code>tesseras-wasm</code> delivers: drag a tessera archive into a web +page, and cryptographic verification happens entirely in your browser.</p> +<h2 id="what-was-built">What was built</h2> +<p><strong>tesseras-wasm</strong> — A Rust crate that compiles to WebAssembly via wasm-pack, +exposing four stateless functions to JavaScript. The crate depends on +<code>tesseras-core</code> for manifest parsing and calls cryptographic primitives directly +(blake3, ed25519-dalek) rather than depending on <code>tesseras-crypto</code>, which pulls +in C-based post-quantum libraries that don't compile to +<code>wasm32-unknown-unknown</code>.</p> +<p><code>parse_manifest</code> takes raw MANIFEST bytes (UTF-8 plain text, not MessagePack), +delegates to <code>tesseras_core::manifest::Manifest::parse()</code>, and returns a JSON +string with the creator's Ed25519 public key, signature file paths, and a list +of files with their expected BLAKE3 hashes, sizes, and MIME types. Internal +structs (<code>ManifestJson</code>, <code>CreatorPubkey</code>, <code>SignatureFiles</code>, <code>FileEntry</code>) are +serialized with serde_json. The ML-DSA public key and signature file fields are +present in the JSON contract but set to <code>null</code> — ready for when post-quantum +signing is implemented on the native side.</p> +<p><code>hash_blake3</code> computes a BLAKE3 hash of arbitrary bytes and returns a +64-character hex string. It's called once per file in the tessera to verify +integrity against the MANIFEST.</p> +<p><code>verify_ed25519</code> takes a message, a 64-byte signature, and a 32-byte public key, +constructs an <code>ed25519_dalek::VerifyingKey</code>, and returns whether the signature +is valid. Length validation returns descriptive errors ("Ed25519 public key must +be 32 bytes") rather than panicking.</p> +<p><code>verify_ml_dsa</code> is a stub that returns an error explaining ML-DSA verification +is not yet available. This is deliberate: the <code>ml-dsa</code> crate on crates.io is +v0.1.0-rc.7 (pre-release), and <code>tesseras-crypto</code> uses <code>pqcrypto-dilithium</code> +(C-based CRYSTALS-Dilithium) which is byte-incompatible with FIPS 204 ML-DSA. +Both sides need to use the same pure Rust implementation before +cross-verification works. Ed25519 verification is sufficient — every tessera is +Ed25519-signed.</p> +<p>All four functions use a two-layer pattern for testability: inner functions +return <code>Result&lt;T, String&gt;</code> and are tested natively, while thin <code>#[wasm_bindgen]</code> +wrappers convert errors to <code>JsError</code>. This avoids <code>JsError::new()</code> panicking on +non-WASM targets during testing.</p> +<p>The compiled WASM binary is 109 KB raw and 44 KB gzipped — well under the 200 KB +budget. wasm-opt applies <code>-Oz</code> optimization after wasm-pack builds with +<code>opt-level = "z"</code>, LTO, and single codegen unit.</p> +<p><strong>@tesseras/verify</strong> — A TypeScript npm package (<code>crates/tesseras-wasm/js/</code>) +that orchestrates browser-side verification. The public API is a single +function:</p> +<pre><code data-lang="typescript">async function verifyTessera( + archive: Uint8Array, + onProgress?: (current: number, total: number, file: string) =&gt; void +): Promise&lt;VerificationResult&gt; +</code></pre> +<p>The <code>VerificationResult</code> type provides everything a UI needs: overall validity, +tessera hash, creator public keys, signature status (valid/invalid/missing for +both Ed25519 and ML-DSA), per-file integrity results with expected and actual +hashes, a list of unexpected files not in the MANIFEST, and an errors array.</p> +<p>Archive unpacking (<code>unpack.ts</code>) handles three formats: gzip-compressed tar +(detected by <code>\x1f\x8b</code> magic bytes, decompressed with fflate then parsed as +tar), ZIP (<code>PK\x03\x04</code> magic, unpacked with fflate's <code>unzipSync</code>), and raw tar +(<code>ustar</code> at offset 257). A <code>normalizePath</code> function strips the leading +<code>tessera-&lt;hash&gt;/</code> prefix so internal paths match MANIFEST entries.</p> +<p>Verification runs in a Web Worker (<code>worker.ts</code>) to keep the UI thread +responsive. The worker initializes the WASM module, unpacks the archive, parses +the MANIFEST, verifies the Ed25519 signature against the creator's public key, +then hashes each file with BLAKE3 and compares against expected values. Progress +messages stream back to the main thread after each file. If any signature is +invalid, verification stops early without hashing files — failing fast on the +most critical check.</p> +<p>The archive is transferred to the worker with zero-copy +(<code>worker.postMessage({ type: "verify", archive }, [archive.buffer])</code>) to avoid +duplicating potentially large tessera files in memory.</p> +<p><strong>Build pipeline</strong> — Three new justfile targets: <code>wasm-build</code> runs wasm-pack +with <code>--target web --release</code> and optimizes with wasm-opt; <code>wasm-size</code> reports +raw and gzipped binary size; <code>test-wasm</code> runs the native test suite.</p> +<p><strong>Tests</strong> — 9 native unit tests cover BLAKE3 hashing (empty input, known value), +Ed25519 verification (valid signature, invalid signature, wrong key, bad key +length), and MANIFEST parsing (valid manifest, invalid UTF-8, garbage input). 3 +WASM integration tests run in headless Chrome via +<code>wasm-pack test --headless --chrome</code>, verifying that <code>hash_blake3</code>, +<code>verify_ed25519</code>, and <code>parse_manifest</code> work correctly when compiled to +<code>wasm32-unknown-unknown</code>.</p> +<h2 id="architecture-decisions">Architecture decisions</h2> +<ul> +<li><strong>No tesseras-crypto dependency</strong>: the WASM crate calls blake3 and +ed25519-dalek directly. <code>tesseras-crypto</code> depends on <code>pqcrypto-kyber</code> (C-based +ML-KEM via pqcrypto-traits) which requires a C compiler toolchain and doesn't +target wasm32. By depending only on pure Rust crates, the WASM build has zero +C dependencies and compiles cleanly to WebAssembly.</li> +<li><strong>ML-DSA deferred, not faked</strong>: rather than silently skipping post-quantum +verification, the stub returns an explicit error. This ensures that if a +tessera contains an ML-DSA signature, the verification result will report +<code>ml_dsa: "missing"</code> rather than pretending it was checked. The JS orchestrator +handles this gracefully — a tessera is valid if Ed25519 passes and ML-DSA is +missing (not yet implemented on either side).</li> +<li><strong>Inner function pattern</strong>: <code>JsError</code> cannot be constructed on non-WASM +targets (it panics). Splitting each function into +<code>foo_inner() -&gt; Result&lt;T, String&gt;</code> and <code>foo() -&gt; Result&lt;T, JsError&gt;</code> lets the +native test suite exercise all logic without touching JavaScript types. The +WASM integration tests in headless Chrome test the full <code>#[wasm_bindgen]</code> +surface.</li> +<li><strong>Web Worker isolation</strong>: cryptographic operations (especially BLAKE3 over +large media files) can take hundreds of milliseconds. Running in a Worker +prevents UI jank. The streaming progress protocol +(<code>{ type: "progress", current, total, file }</code>) lets the UI show a progress bar +during verification of tesseras with many files.</li> +<li><strong>Zero-copy transfer</strong>: <code>archive.buffer</code> is transferred to the Worker, not +copied. For a 50 MB tessera archive, this avoids doubling memory usage during +verification.</li> +<li><strong>Plain text MANIFEST, not MessagePack</strong>: the WASM crate parses the same +plain-text MANIFEST format as the CLI. This is by design — the MANIFEST is the +tessera's Rosetta Stone, readable by anyone with a text editor. The +<code>rmp-serde</code> dependency in the Cargo.toml is not used and will be removed.</li> +</ul> +<h2 id="what-comes-next">What comes next</h2> +<ul> +<li><strong>Phase 4: Resilience and Scale</strong> — OS packaging (Alpine, Arch, Debian, +FreeBSD, OpenBSD), CI on SourceHut and GitHub Actions, security audits, +browser-based tessera explorer at tesseras.net using @tesseras/verify</li> +<li><strong>Phase 5: Exploration and Culture</strong> — Public tessera browser by +era/location/theme/language, institutional curation, genealogy integration, +physical media export (M-DISC, microfilm, acid-free paper with QR)</li> +</ul> +<p>Verification no longer requires trust in software. A tessera archive dropped +into a browser is verified with the same cryptographic rigor as the CLI — same +BLAKE3 hashes, same Ed25519 signatures, same MANIFEST parser. The difference is +that now anyone can do it.</p> +</content> + + </entry> + <entry xml:lang="en"> + <title>Phase 4: Punching Through NATs</title> + <published>2026-02-15T18:00:00+00:00</published> + <updated>2026-02-15T18:00:00+00:00</updated> + + <author> + <name> + + Unknown + + </name> + </author> + + <link rel="alternate" type="text/html" href="https://tesseras.net/news/phase4-nat-traversal/"/> + <id>https://tesseras.net/news/phase4-nat-traversal/</id> + + <content type="html" xml:base="https://tesseras.net/news/phase4-nat-traversal/"><p>Most people's devices sit behind a NAT — a network address translator that lets +them reach the internet but prevents incoming connections. For a P2P network, +this is an existential problem: if two nodes behind NATs can't talk to each +other, the network fragments. Phase 4 continues with a full NAT traversal stack: +STUN-based discovery, coordinated hole punching, and relay fallback.</p> +<p>The approach follows the same pattern as most battle-tested P2P systems (WebRTC, +BitTorrent, IPFS): try the cheapest option first, escalate only when necessary. +Direct connectivity costs nothing. Hole punching costs a few coordinated +packets. Relaying costs sustained bandwidth from a third party. Tesseras tries +them in that order.</p> +<h2 id="what-was-built">What was built</h2> +<p><strong>NatType classification</strong> (<code>tesseras-core/src/network.rs</code>) — A new <code>NatType</code> +enum (Public, Cone, Symmetric, Unknown) added to the core domain layer. This +type is shared across the entire stack: the STUN client writes it, the DHT +advertises it in Pong messages, and the punch coordinator reads it to decide +whether hole punching is even worth attempting (Cone-to-Cone works ~80% of the +time; Symmetric-to-Symmetric almost never works).</p> +<p><strong>STUN client</strong> (<code>tesseras-net/src/stun.rs</code>) — A minimal STUN implementation +(RFC 5389 Binding Request/Response) that discovers a node's external address. +The codec encodes 20-byte binding requests with a random transaction ID and +decodes XOR-MAPPED-ADDRESS responses. The <code>discover_nat()</code> function queries +multiple STUN servers in parallel (Google, Cloudflare by default), compares the +mapped addresses, and classifies the NAT type:</p> +<ul> +<li>Same IP and port from all servers → <strong>Public</strong> (no NAT)</li> +<li>Same mapped address from all servers → <strong>Cone</strong> (hole punching works)</li> +<li>Different mapped addresses → <strong>Symmetric</strong> (hole punching unreliable)</li> +<li>No responses → <strong>Unknown</strong></li> +</ul> +<p>Retries with exponential backoff and configurable timeouts. 12 tests covering +codec roundtrips, all classification paths, and async loopback queries.</p> +<p><strong>Signed punch coordination</strong> (<code>tesseras-net/src/punch.rs</code>) — Ed25519 signing +and verification for <code>PunchIntro</code>, <code>RelayRequest</code>, and <code>RelayMigrate</code> messages. +Every introduction is signed by the initiator with a 30-second timestamp window, +preventing reflection attacks (where an attacker replays an old introduction to +redirect traffic). The payload format is <code>target || external_addr || timestamp</code> +— changing any field invalidates the signature. 6 unit tests plus 3 +property-based tests with proptest (arbitrary node IDs, ports, and session +tokens).</p> +<p><strong>Relay session manager</strong> (<code>tesseras-net/src/relay.rs</code>) — Manages transparent +UDP relay sessions between NATed peers. Each session has a random 16-byte token; +peers prefix their packets with the token, the relay strips it and forwards. +Features:</p> +<ul> +<li>Bidirectional forwarding (A→R→B and B→R→A)</li> +<li>Rate limiting: 256 KB/s for reciprocal peers, 64 KB/s for non-reciprocal</li> +<li>10-minute maximum duration for bootstrap (non-reciprocal) sessions</li> +<li>Address migration: when a peer's IP changes (Wi-Fi to cellular), a signed +<code>RelayMigrate</code> updates the session without tearing it down</li> +<li>Idle cleanup with configurable timeout</li> +<li>8 unit tests plus 2 property-based tests</li> +</ul> +<p><strong>DHT message extensions</strong> (<code>tesseras-dht/src/message.rs</code>) — Seven new message +variants added to the DHT protocol:</p> +<table><thead><tr><th>Message</th><th>Purpose</th></tr></thead><tbody> +<tr><td><code>PunchIntro</code></td><td>"I want to connect to node X, here's my signed external address"</td></tr> +<tr><td><code>PunchRequest</code></td><td>Introducer forwards the request to the target</td></tr> +<tr><td><code>PunchReady</code></td><td>Target confirms readiness, sends its external address</td></tr> +<tr><td><code>RelayRequest</code></td><td>"Create a relay session to node X"</td></tr> +<tr><td><code>RelayOffer</code></td><td>Relay responds with its address and session token</td></tr> +<tr><td><code>RelayClose</code></td><td>Tear down a relay session</td></tr> +<tr><td><code>RelayMigrate</code></td><td>Update session after network change</td></tr> +</tbody></table> +<p>The <code>Pong</code> message was extended with NAT metadata: <code>nat_type</code>, +<code>relay_slots_available</code>, and <code>relay_bandwidth_used_kbps</code>. All new fields use +<code>#[serde(default)]</code> for backward compatibility — old nodes ignore what they +don't recognize, new nodes fall back to defaults. 9 new serialization roundtrip +tests.</p> +<p><strong>NatHandler trait and dispatch</strong> (<code>tesseras-dht/src/engine.rs</code>) — A new +<code>NatHandler</code> async trait (5 methods) injected into the DHT engine, following the +same dependency injection pattern as the existing <code>ReplicationHandler</code>. The +engine's message dispatch loop now routes all punch/relay messages to the +handler. This keeps the DHT engine protocol-agnostic while allowing the NAT +traversal logic to live in <code>tesseras-net</code>.</p> +<p><strong>Mobile reconnection types</strong> (<code>tesseras-embedded/src/reconnect.rs</code>) — A +three-phase reconnection state machine for mobile devices:</p> +<ol> +<li><strong>QuicMigration</strong> (0-2s) — try QUIC connection migration for all active peers</li> +<li><strong>ReStun</strong> (2-5s) — re-discover external address via STUN</li> +<li><strong>ReEstablish</strong> (5-10s) — reconnect peers that migration couldn't save</li> +</ol> +<p>Peers are reconnected in priority order: bootstrap nodes first, then nodes +holding our fragments, then nodes whose fragments we hold, then general DHT +neighbors. A new <code>NetworkChanged</code> event variant was added to the FFI event +stream so the Flutter app can show reconnection progress.</p> +<p><strong>Daemon NAT configuration</strong> (<code>tesd/src/config.rs</code>) — A new <code>[nat]</code> section in +the TOML config with STUN server list, relay toggle, max relay sessions, +bandwidth limits (reciprocal vs bootstrap), and idle timeout. All fields have +sensible defaults; relay is disabled by default.</p> +<p><strong>Prometheus metrics</strong> (<code>tesseras-net/src/metrics.rs</code>) — 16 metrics across four +subsystems:</p> +<ul> +<li><strong>STUN</strong>: requests, failures, latency histogram</li> +<li><strong>Punch</strong>: attempts/successes/failures (by NAT type pair), latency histogram</li> +<li><strong>Relay</strong>: active sessions, total sessions, bytes forwarded, idle timeouts, +rate limit hits</li> +<li><strong>Reconnect</strong>: network changes, attempts/successes by phase, duration +histogram</li> +</ul> +<p>6 tests verifying registration, increment, label cardinality, and +double-registration detection.</p> +<p><strong>Integration tests</strong> — Two end-to-end tests using <code>MemTransport</code> (in-memory +simulated network):</p> +<ul> +<li><code>punch_integration.rs</code> — Full 3-node hole-punch flow: A sends signed +<code>PunchIntro</code> to introducer I, I verifies and forwards <code>PunchRequest</code> to B, B +verifies the original signature and sends <code>PunchReady</code> back, A and B exchange +messages directly. Also tests that a bad signature is correctly rejected.</li> +<li><code>relay_integration.rs</code> — Full 3-node relay flow: A requests relay from R, R +creates session and sends <code>RelayOffer</code> to both peers, A and B exchange +token-prefixed packets through R, A migrates to a new address mid-session, A +closes the session, and the test verifies the session is torn down and further +forwarding fails.</li> +</ul> +<p><strong>Property tests</strong> — 7 proptest-based tests covering: signature round-trips for +all three signed message types (arbitrary node IDs, ports, tokens), NAT +classification determinism (same inputs always produce same output), STUN +binding request validity, session token uniqueness, and relay rejection of +too-short packets.</p> +<p><strong>Justfile targets</strong> — <code>just test-nat</code> runs all NAT traversal tests across +<code>tesseras-net</code> and <code>tesseras-dht</code>. <code>just test-chaos</code> is a placeholder for future +Docker Compose chaos tests with <code>tc netem</code>.</p> +<h2 id="architecture-decisions">Architecture decisions</h2> +<ul> +<li><strong>STUN over TURN</strong>: we implement STUN (discovery) and custom relay rather than +full TURN. TURN requires authenticated allocation and is designed for media +relay; our relay is simpler — token-prefixed UDP forwarding with rate limits. +This keeps the protocol minimal and avoids depending on external TURN servers.</li> +<li><strong>Signatures on introductions</strong>: every <code>PunchIntro</code> is signed by the +initiator. Without this, an attacker could send forged introductions to +redirect a node's hole-punch attempts to an attacker-controlled address (a +reflection attack). The 30-second timestamp window limits replay.</li> +<li><strong>Reciprocal bandwidth tiers</strong>: relay nodes give 4x more bandwidth (256 vs 64 +KB/s) to peers with good reciprocity scores. This incentivizes nodes to store +fragments for others — if you contribute, you get better relay service when +you need it.</li> +<li><strong>Backward-compatible Pong extension</strong>: new NAT fields in <code>Pong</code> use +<code>#[serde(default)]</code> and <code>Option&lt;T&gt;</code>. Old nodes that don't understand these +fields simply skip them during deserialization. No protocol version bump +needed.</li> +<li><strong>NatHandler as async trait</strong>: the NAT traversal logic is injected into the +DHT engine via a trait, just like <code>ReplicationHandler</code>. This keeps the DHT +engine focused on routing and peer management, and allows the NAT +implementation to be swapped or disabled without touching core DHT code.</li> +</ul> +<h2 id="what-comes-next">What comes next</h2> +<ul> +<li><strong>Phase 4 continued</strong> — performance tuning (connection pooling, fragment +caching, SQLite WAL), security audits, institutional node onboarding, OS +packaging</li> +<li><strong>Phase 5: Exploration and Culture</strong> — public tessera browser by +era/location/theme/language, institutional curation, genealogy integration, +physical media export (M-DISC, microfilm, acid-free paper with QR)</li> +</ul> +<p>With NAT traversal, Tesseras can connect nodes regardless of their network +topology. Public nodes talk directly. Cone-NATed nodes punch through with an +introducer's help. Symmetric-NATed or firewalled nodes relay through willing +peers. The network adapts to the real world, where most devices are behind a NAT +and network conditions change constantly.</p> +</content> + + </entry> + <entry xml:lang="en"> + <title>CLI Meets Network: Publish, Fetch, and Status Commands</title> + <published>2026-02-15T00:00:00+00:00</published> + <updated>2026-02-15T00:00:00+00:00</updated> + + <author> + <name> + + Unknown + + </name> + </author> + + <link rel="alternate" type="text/html" href="https://tesseras.net/news/cli-daemon-rpc/"/> + <id>https://tesseras.net/news/cli-daemon-rpc/</id> + + <content type="html" xml:base="https://tesseras.net/news/cli-daemon-rpc/"><p>Until now the CLI operated in isolation: create a tessera, verify it, export it, +list what you have. Everything stayed on your machine. With this release, <code>tes</code> +gains three commands that bridge the gap between local storage and the P2P +network — <code>publish</code>, <code>fetch</code>, and <code>status</code> — by talking to a running <code>tesd</code> over +a Unix socket.</p> +<h2 id="what-was-built">What was built</h2> +<p><strong><code>tesseras-rpc</code> crate</strong> — A new shared crate that both the CLI and daemon +depend on. It defines the RPC protocol using MessagePack serialization with +length-prefixed framing (4-byte big-endian size header, 64 MiB max). Three +request types (<code>Publish</code>, <code>Fetch</code>, <code>Status</code>) and their corresponding responses. +A sync <code>DaemonClient</code> handles the Unix socket connection with configurable +timeouts. The protocol is deliberately simple — one request, one response, +connection closed — to keep the implementation auditable.</p> +<p><strong><code>tes publish &lt;hash&gt;</code></strong> — Publishes a tessera to the network. Accepts full +hashes or short prefixes (e.g., <code>tes publish a1b2</code>), which are resolved against +the local database. The daemon reads all tessera files from storage, packs them +into a single MessagePack buffer, and hands them to the replication engine. +Small tesseras (&lt; 4 MB) are replicated as a single fragment; larger ones go +through Reed-Solomon erasure coding. Output shows the short hash and fragment +count:</p> +<pre><code>Published tessera 9f2c4a1b (24 fragments created) +Distribution in progress — use `tes status 9f2c4a1b` to track. +</code></pre> +<p><strong><code>tes fetch &lt;hash&gt;</code></strong> — Retrieves a tessera from the network using its full +content hash. The daemon collects locally available fragments, reconstructs the +original data via erasure decoding if needed, unpacks the files, and stores them +in the content-addressable store. Returns the number of memories and total size +fetched.</p> +<p><strong><code>tes status &lt;hash&gt;</code></strong> — Displays the replication health of a tessera. The +output maps directly to the replication engine's internal health model:</p> +<table><thead><tr><th>State</th><th>Meaning</th></tr></thead><tbody> +<tr><td>Local</td><td>Not yet published — exists only on your machine</td></tr> +<tr><td>Publishing</td><td>Fragments being distributed, critical redundancy</td></tr> +<tr><td>Replicated</td><td>Distributed but below target redundancy</td></tr> +<tr><td>Healthy</td><td>Full redundancy achieved</td></tr> +</tbody></table> +<p><strong>Daemon RPC listener</strong> — The daemon now binds a Unix socket (default: +<code>$XDG_RUNTIME_DIR/tesseras/daemon.sock</code>) with proper directory permissions +(0700), stale socket cleanup, and graceful shutdown. Each connection is handled +in a Tokio task — the listener converts the async stream to sync I/O for the +framing layer, dispatches to the RPC handler, and writes the response back.</p> +<p><strong>Pack/unpack in <code>tesseras-core</code></strong> — A small module that serializes a list of +file entries (path + data) into a single MessagePack buffer and back. This is +the bridge between the tessera's directory structure and the replication +engine's opaque byte blobs.</p> +<h2 id="architecture-decisions">Architecture decisions</h2> +<ul> +<li><strong>Unix socket over TCP</strong>: RPC between CLI and daemon happens on the same +machine. Unix sockets are faster, don't need port allocation, and filesystem +permissions provide access control without TLS.</li> +<li><strong>MessagePack over JSON</strong>: the same wire format used everywhere else in +Tesseras. Compact, schema-less, and already a workspace dependency. A typical +publish request/response round-trip is under 200 bytes.</li> +<li><strong>Sync client, async daemon</strong>: the <code>DaemonClient</code> uses blocking I/O because +the CLI doesn't need concurrency — it sends one request and waits. The daemon +listener is async (Tokio) to handle multiple connections. The framing layer +works with any <code>Read</code>/<code>Write</code> impl, bridging both worlds.</li> +<li><strong>Hash prefix resolution on the client side</strong>: <code>publish</code> and <code>status</code> resolve +short prefixes locally before sending the full hash to the daemon. This keeps +the daemon stateless — it doesn't need access to the CLI's database.</li> +<li><strong>Default data directory alignment</strong>: the CLI default changed from +<code>~/.tesseras</code> to <code>~/.local/share/tesseras</code> (via <code>dirs::data_dir()</code>) to match +the daemon. A migration hint is printed when legacy data is detected.</li> +</ul> +<h2 id="what-comes-next">What comes next</h2> +<ul> +<li><strong>DHT peer count</strong>: the <code>status</code> command currently reports 0 peers — wiring +the actual peer count from the DHT is the next step</li> +<li><strong><code>tes show</code></strong>: display the contents of a tessera (memories, metadata) without +exporting</li> +<li><strong>Streaming fetch</strong>: for large tesseras, stream fragments as they arrive +rather than waiting for all of them</li> +</ul> +</content> + + </entry> + <entry xml:lang="en"> + <title>Phase 4: Heir Key Recovery with Shamir's Secret Sharing</title> + <published>2026-02-15T00:00:00+00:00</published> + <updated>2026-02-15T00:00:00+00:00</updated> + + <author> + <name> + + Unknown + + </name> + </author> + + <link rel="alternate" type="text/html" href="https://tesseras.net/news/phase4-shamir-heir-recovery/"/> + <id>https://tesseras.net/news/phase4-shamir-heir-recovery/</id> + + <content type="html" xml:base="https://tesseras.net/news/phase4-shamir-heir-recovery/"><p>What happens to your memories when you die? Until now, Tesseras could preserve +content across millennia — but the private and sealed keys died with their +owner. Phase 4 continues with a solution: Shamir's Secret Sharing, a +cryptographic scheme that lets you split your identity into shares and +distribute them to the people you trust most.</p> +<p>The math is elegant: you choose a threshold T and a total N. Any T shares +reconstruct the full secret; T-1 shares reveal absolutely nothing. This is not +"almost nothing" — it is information-theoretically secure. An attacker with one +fewer share than the threshold has exactly zero bits of information about the +secret, no matter how much computing power they have.</p> +<h2 id="what-was-built">What was built</h2> +<p><strong>GF(256) finite field arithmetic</strong> (<code>tesseras-crypto/src/shamir/gf256.rs</code>) — +Shamir's Secret Sharing requires arithmetic in a finite field. We implement +GF(256) using the same irreducible polynomial as AES (x^8 + x^4 + x^3 + x + 1), +with compile-time lookup tables for logarithm and exponentiation. All operations +are constant-time via table lookups — no branches on secret data. The module +includes Horner's method for polynomial evaluation and Lagrange interpolation at +x=0 for secret recovery. 233 lines, exhaustively tested: all 256 elements for +identity/inverse properties, commutativity, and associativity.</p> +<p><strong>ShamirSplitter</strong> (<code>tesseras-crypto/src/shamir/mod.rs</code>) — The core +split/reconstruct API. <code>split()</code> takes a secret byte slice, a configuration +(threshold T, total N), and the owner's Ed25519 public key. For each byte of the +secret, it constructs a random polynomial of degree T-1 over GF(256) with the +secret byte as the constant term, then evaluates it at N distinct points. +<code>reconstruct()</code> takes T or more shares and recovers the secret via Lagrange +interpolation. Both operations include extensive validation: threshold bounds, +session consistency, owner fingerprint matching, and BLAKE3 checksum +verification.</p> +<p><strong>HeirShare format</strong> — Each share is a self-contained, serializable artifact +with:</p> +<ul> +<li>Format version (v1) for forward compatibility</li> +<li>Share index (1..N) and threshold/total metadata</li> +<li>Session ID (random 8 bytes) — prevents mixing shares from different split +sessions</li> +<li>Owner fingerprint (first 8 bytes of BLAKE3 hash of the Ed25519 public key)</li> +<li>Share data (the Shamir y-values, same length as the secret)</li> +<li>BLAKE3 checksum over all preceding fields</li> +</ul> +<p>Shares are serialized in two formats: <strong>MessagePack</strong> (compact binary, for +programmatic use) and <strong>base64 text</strong> (human-readable, for printing and physical +storage). The text format includes a header with metadata and delimiters:</p> +<pre><code>--- TESSERAS HEIR SHARE --- +Format: v1 +Owner: a1b2c3d4e5f6a7b8 (fingerprint) +Share: 1 of 3 (threshold: 2) +Session: 9f8e7d6c5b4a3210 +Created: 2026-02-15 + +&lt;base64-encoded MessagePack data&gt; +--- END HEIR SHARE --- +</code></pre> +<p>This format is designed to be printed on paper, stored in a safe deposit box, or +engraved on metal. The header is informational — only the base64 payload is +parsed during reconstruction.</p> +<p><strong>CLI integration</strong> (<code>tesseras-cli/src/commands/heir.rs</code>) — Three new +subcommands:</p> +<ul> +<li><code>tes heir create</code> — splits your Ed25519 identity into heir shares. Prompts for +confirmation (your full identity is at stake), generates both <code>.bin</code> and +<code>.txt</code> files for each share, and writes <code>heir_meta.json</code> to your identity +directory.</li> +<li><code>tes heir reconstruct</code> — loads share files (auto-detects binary vs text +format), validates consistency, reconstructs the secret, derives the Ed25519 +keypair, and optionally installs it to <code>~/.tesseras/identity/</code> (with automatic +backup of the existing identity).</li> +<li><code>tes heir info</code> — displays share metadata and verifies the checksum without +exposing any secret material.</li> +</ul> +<p><strong>Secret blob format</strong> — Identity keys are serialized into a versioned blob +before splitting: a version byte (0x01), a flags byte (0x00 for Ed25519-only), +followed by the 32-byte Ed25519 secret key. This leaves room for future +expansion when X25519 and ML-KEM-768 private keys are integrated into the heir +share system.</p> +<p><strong>Testing</strong> — 20 unit tests for ShamirSplitter (roundtrip, all share +combinations, insufficient shares, wrong owner, wrong session, threshold-1 +boundary, large secrets up to ML-KEM-768 key size). 7 unit tests for GF(256) +arithmetic (exhaustive field properties). 3 property-based tests with proptest +(arbitrary secrets up to 5000 bytes, arbitrary T-of-N configurations, +information-theoretic security verification). Serialization roundtrip tests for +both MessagePack and base64 text formats. 2 integration tests covering the +complete heir lifecycle: generate identity, split into shares, serialize, +deserialize, reconstruct, verify keypair, and sign/verify with reconstructed +keys.</p> +<h2 id="architecture-decisions">Architecture decisions</h2> +<ul> +<li><strong>GF(256) over GF(prime)</strong>: we use GF(256) rather than a prime field because +it maps naturally to bytes — each element is a single byte, each share is the +same length as the secret. No big-integer arithmetic, no modular reduction, no +padding. This is the same approach used by most real-world Shamir +implementations including SSSS and Hashicorp Vault.</li> +<li><strong>Compile-time lookup tables</strong>: the LOG and EXP tables for GF(256) are +computed at compile time using <code>const fn</code>. This means zero runtime +initialization cost and constant-time operations via table lookups rather than +loops.</li> +<li><strong>Session ID prevents cross-session mixing</strong>: each call to <code>split()</code> generates +a fresh random session ID. If an heir accidentally uses shares from two +different split sessions (e.g., before and after a key rotation), +reconstruction fails cleanly with a validation error rather than producing +garbage output.</li> +<li><strong>BLAKE3 checksums detect corruption</strong>: each share includes a BLAKE3 checksum +over its contents. This catches bit rot, transmission errors, and accidental +truncation before any reconstruction attempt. A share printed on paper and +scanned back via OCR will fail the checksum if a single character is wrong.</li> +<li><strong>Owner fingerprint for identification</strong>: shares include the first 8 bytes of +BLAKE3(Ed25519 public key) as a fingerprint. This lets heirs verify which +identity a share belongs to without revealing the full public key. During +reconstruction, the fingerprint is cross-checked against the recovered key.</li> +<li><strong>Dual format for resilience</strong>: both binary (MessagePack) and text (base64) +formats are generated because physical media has different failure modes than +digital storage. A USB drive might fail; paper survives. A QR code might be +unreadable; base64 text can be manually typed.</li> +<li><strong>Blob versioning</strong>: the secret is wrapped in a versioned blob (version + +flags + key material) so future versions can include additional keys (X25519, +ML-KEM-768) without breaking backward compatibility with existing shares.</li> +</ul> +<h2 id="what-comes-next">What comes next</h2> +<ul> +<li><strong>Phase 4 continued: Resilience and Scale</strong> — advanced NAT traversal +(STUN/TURN), performance tuning (connection pooling, fragment caching, SQLite +WAL), security audits, institutional node onboarding, OS packaging</li> +<li><strong>Phase 5: Exploration and Culture</strong> — public tessera browser by +era/location/theme/language, institutional curation, genealogy integration, +physical media export (M-DISC, microfilm, acid-free paper with QR)</li> +</ul> +<p>With Shamir's Secret Sharing, Tesseras closes the last critical gap in long-term +preservation. Your memories survive infrastructure failures through erasure +coding. Your privacy survives quantum computers through hybrid encryption. And +now, your identity survives you — passed on to the people you chose, requiring +their cooperation to unlock what you left behind.</p> +</content> + + </entry> + <entry xml:lang="en"> + <title>Phase 4: Encryption and Sealed Tesseras</title> + <published>2026-02-14T16:00:00+00:00</published> + <updated>2026-02-14T16:00:00+00:00</updated> + + <author> + <name> + + Unknown + + </name> + </author> + + <link rel="alternate" type="text/html" href="https://tesseras.net/news/phase4-encryption-sealed/"/> + <id>https://tesseras.net/news/phase4-encryption-sealed/</id> + + <content type="html" xml:base="https://tesseras.net/news/phase4-encryption-sealed/"><p>Some memories are not meant for everyone. A private journal, a letter to be +opened in 2050, a family secret sealed until the grandchildren are old enough. +Until now, every tessera on the network was open. Phase 4 changes that: Tesseras +now encrypts private and sealed content with a hybrid cryptographic scheme +designed to resist both classical and quantum attacks.</p> +<p>The principle remains the same — encrypt as little as possible. Public memories +need availability, not secrecy. But when someone creates a private or sealed +tessera, the content is now locked behind AES-256-GCM encryption with keys +protected by a hybrid key encapsulation mechanism combining X25519 and +ML-KEM-768. Both algorithms must be broken to access the content.</p> +<h2 id="what-was-built">What was built</h2> +<p><strong>AES-256-GCM encryptor</strong> (<code>tesseras-crypto/src/encryption.rs</code>) — Symmetric +content encryption with random 12-byte nonces and authenticated associated data +(AAD). The AAD binds ciphertext to its context: for private tesseras, the +content hash is included; for sealed tesseras, both the content hash and the +<code>open_after</code> timestamp are bound into the AAD. This means moving ciphertext +between tesseras with different open dates causes decryption failure — you +cannot trick the system into opening a sealed memory early by swapping its +ciphertext into a tessera with an earlier seal date.</p> +<p><strong>Hybrid Key Encapsulation Mechanism</strong> (<code>tesseras-crypto/src/kem.rs</code>) — Key +exchange using X25519 (classical elliptic curve Diffie-Hellman) combined with +ML-KEM-768 (the NIST-standardized post-quantum lattice-based KEM, formerly +Kyber). Both shared secrets are combined via <code>blake3::derive_key</code> with a fixed +context string ("tesseras hybrid kem v1") to produce a single 256-bit content +encryption key. This follows the same "dual from day one" philosophy as the +project's dual signing (Ed25519 + ML-DSA): if either algorithm is broken in the +future, the other still protects the content.</p> +<p><strong>Sealed Key Envelope</strong> (<code>tesseras-crypto/src/sealed.rs</code>) — Wraps a content +encryption key using the hybrid KEM, so only the tessera owner can recover it. +The KEM produces a transport key, which is XORed with the content key to produce +a wrapped key stored alongside the KEM ciphertext. On unsealing, the owner +decapsulates the KEM ciphertext to recover the transport key, then XORs again to +recover the content key.</p> +<p><strong>Key Publication</strong> (<code>tesseras-crypto/src/sealed.rs</code>) — A standalone signed +artifact for publishing a sealed tessera's content key after its <code>open_after</code> +date has passed. The owner signs the content key, tessera hash, and publication +timestamp with their dual keys (Ed25519, with ML-DSA placeholder). The manifest +stays immutable — the key publication is a separate document. Other nodes verify +the signature against the owner's public key before using the published key to +decrypt the content.</p> +<p><strong>EncryptionContext</strong> (<code>tesseras-core/src/enums.rs</code>) — A domain type that +represents the AAD context for encryption. It lives in tesseras-core rather than +tesseras-crypto because it's a domain concept (not a crypto implementation +detail). The <code>to_aad_bytes()</code> method produces deterministic serialization: a tag +byte (0x00 for Private, 0x01 for Sealed), followed by the content hash, and for +Sealed, the <code>open_after</code> timestamp as little-endian i64.</p> +<p><strong>Domain validation</strong> (<code>tesseras-core/src/service.rs</code>) — +<code>TesseraService::create()</code> now rejects Sealed and Private tesseras that don't +provide encryption keys. This is a domain-level validation: the service layer +enforces that you cannot create a sealed memory without the cryptographic +machinery to protect it. The error message is clear: "missing encryption keys +for visibility sealed until 2050-01-01."</p> +<p><strong>Core type updates</strong> — <code>TesseraIdentity</code> now includes an optional +<code>encryption_public: Option&lt;HybridEncryptionPublic&gt;</code> field containing both the +X25519 and ML-KEM-768 public keys. <code>KeyAlgorithm</code> gained <code>X25519</code> and <code>MlKem768</code> +variants. The identity filesystem layout now supports <code>node.x25519.key</code>/<code>.pub</code> +and <code>node.mlkem768.key</code>/<code>.pub</code>.</p> +<p><strong>Testing</strong> — 8 unit tests for AES-256-GCM (roundtrip, wrong key, tampered +ciphertext, wrong AAD, cross-context decryption failure, unique nonces, plus 2 +property-based tests for arbitrary payloads and nonce uniqueness). 5 unit tests +for HybridKem (roundtrip, wrong keypair, tampered X25519, KDF determinism, plus +1 property-based test). 4 unit tests for SealedKeyEnvelope and KeyPublication. 2 +integration tests covering the complete sealed and private tessera lifecycle: +generate keys, create content key, encrypt, seal, unseal, decrypt, publish key, +and verify — the full cycle.</p> +<h2 id="architecture-decisions">Architecture decisions</h2> +<ul> +<li><strong>Hybrid KEM from day one</strong>: X25519 + ML-KEM-768 follows the same philosophy +as dual signing. We don't know which cryptographic assumptions will hold over +millennia, so we combine classical and post-quantum algorithms. The cost is +~1.2 KB of additional key material per identity — trivial compared to the +photos and videos in a tessera.</li> +<li><strong>BLAKE3 for KDF</strong>: rather than adding <code>hkdf</code> + <code>sha2</code> as new dependencies, we +use <code>blake3::derive_key</code> with a fixed context string. BLAKE3's key derivation +mode is specifically designed for this use case, and the project already +depends on BLAKE3 for content hashing.</li> +<li><strong>Immutable manifests</strong>: when a sealed tessera's <code>open_after</code> date passes, the +content key is published as a separate signed artifact (<code>KeyPublication</code>), not +by modifying the manifest. This preserves the append-only, content-addressed +nature of tesseras. The manifest was signed at creation time and never +changes.</li> +<li><strong>AAD binding prevents ciphertext swapping</strong>: the <code>EncryptionContext</code> binds +both the content hash and (for sealed tesseras) the <code>open_after</code> timestamp +into the AES-GCM authenticated data. An attacker who copies encrypted content +from a "sealed until 2050" tessera into a "sealed until 2025" tessera will +find that decryption fails — the AAD no longer matches.</li> +<li><strong>XOR key wrapping</strong>: the sealed key envelope uses a simple XOR of the content +key with the KEM-derived transport key, rather than an additional layer of +AES-GCM. Since the transport key is a fresh random value from the KEM and is +used exactly once, XOR is information-theoretically secure for this specific +use case and avoids unnecessary complexity.</li> +<li><strong>Domain validation, not storage validation</strong>: the "missing encryption keys" +check lives in <code>TesseraService::create()</code>, not in the storage layer. This +follows the hexagonal architecture pattern: domain rules are enforced at the +service boundary, not scattered across adapters.</li> +</ul> +<h2 id="what-comes-next">What comes next</h2> +<ul> +<li><strong>Phase 4 continued: Resilience and Scale</strong> — Shamir's Secret Sharing for heir +key distribution, advanced NAT traversal (STUN/TURN), performance tuning, +security audits, OS packaging</li> +<li><strong>Phase 5: Exploration and Culture</strong> — Public tessera browser by +era/location/theme/language, institutional curation, genealogy integration, +physical media export (M-DISC, microfilm, acid-free paper with QR)</li> +</ul> +<p>Sealed tesseras make Tesseras a true time capsule. A father can now record a +message for his unborn grandchild, seal it until 2060, and know that the +cryptographic envelope will hold — even if the quantum computers of the future +try to break it open early.</p> +</content> + + </entry> + <entry xml:lang="en"> + <title>Phase 3: Memories in Your Hands</title> + <published>2026-02-14T14:00:00+00:00</published> + <updated>2026-02-14T14:00:00+00:00</updated> + + <author> + <name> + + Unknown + + </name> + </author> + + <link rel="alternate" type="text/html" href="https://tesseras.net/news/phase3-api-and-apps/"/> + <id>https://tesseras.net/news/phase3-api-and-apps/</id> + + <content type="html" xml:base="https://tesseras.net/news/phase3-api-and-apps/"><p>People can now hold their memories in their hands. Phase 3 delivers what the +previous phases built toward: a mobile app where someone downloads Tesseras, +creates an identity, takes a photo, and that memory enters the preservation +network. No cloud accounts, no subscriptions, no company between you and your +memories.</p> +<h2 id="what-was-built">What was built</h2> +<p><strong>tesseras-embedded</strong> — A full P2P node that runs inside a mobile app. The +<code>EmbeddedNode</code> struct owns a Tokio runtime, SQLite database, QUIC transport, +Kademlia DHT engine, replication service, and tessera service — the same stack +as the desktop daemon, compiled into a shared library. A global singleton +pattern (<code>Mutex&lt;Option&lt;EmbeddedNode&gt;&gt;</code>) ensures one node per app lifecycle. On +start, it opens the database, runs migrations, loads or generates an Ed25519 +identity with proof-of-work node ID, binds QUIC on an ephemeral port, wires up +DHT and replication, and spawns the repair loop. On stop, it sends a shutdown +signal and drains gracefully.</p> +<p>Eleven FFI functions are exposed to Dart via flutter_rust_bridge: lifecycle +(<code>node_start</code>, <code>node_stop</code>, <code>node_is_running</code>), identity (<code>create_identity</code>, +<code>get_identity</code>), memories (<code>create_memory</code>, <code>get_timeline</code>, <code>get_memory</code>), and +network status (<code>get_network_stats</code>, <code>get_replication_status</code>). All types +crossing the FFI boundary are flat structs with only <code>String</code>, <code>Option&lt;String&gt;</code>, +<code>Vec&lt;String&gt;</code>, and primitives — no trait objects, no generics, no lifetimes.</p> +<p>Four adapter modules bridge core ports to concrete implementations: +<code>Blake3HasherAdapter</code>, <code>Ed25519SignerAdapter</code>/<code>Ed25519VerifierAdapter</code> for +cryptography, <code>DhtPortAdapter</code> for DHT operations, and +<code>ReplicationHandlerAdapter</code> for incoming fragment and attestation RPCs.</p> +<p>The <code>bundled-sqlite</code> feature flag compiles SQLite from source, required for +Android and iOS where the system library may not be available. Cargokit +configuration passes this flag automatically in both debug and release builds.</p> +<p><strong>Flutter app</strong> — A Material Design 3 application with Riverpod state +management, targeting Android, iOS, Linux, macOS, and Windows from a single +codebase.</p> +<p>The <em>onboarding flow</em> is three screens: a welcome screen explaining the project +in one sentence ("Preserve your memories across millennia. No cloud. No +company."), an identity creation screen that triggers Ed25519 keypair generation +in Rust, and a confirmation screen showing the user's name and cryptographic +identity.</p> +<p>The <em>timeline screen</em> displays memories in reverse chronological order with +image previews, context text, and chips for memory type and visibility. +Pull-to-refresh reloads from the Rust node. A floating action button opens the +<em>memory creation screen</em>, which supports photo selection from gallery or camera +via <code>image_picker</code>, optional context text, memory type and visibility dropdowns, +and comma-separated tags. Creating a memory calls the Rust FFI synchronously, +then returns to the timeline.</p> +<p>The <em>network screen</em> shows two cards: node status (peer count, DHT size, +bootstrap state, uptime) and replication health (total fragments, healthy +fragments, repairing fragments, replication factor). The <em>settings screen</em> +displays the user's identity — name, truncated node ID, truncated public key, +and creation date.</p> +<p>Three Riverpod providers manage state: <code>nodeProvider</code> starts the embedded node +on app launch using the app documents directory and stops it on dispose; +<code>identityProvider</code> loads the existing profile or creates a new one; +<code>timelineProvider</code> fetches the memory list with pagination.</p> +<p><strong>Testing</strong> — 9 Rust unit tests in tesseras-embedded covering node lifecycle +(start/stop without panic), identity persistence across restarts, restart cycles +without SQLite corruption, network event streaming, stats retrieval, memory +creation and timeline retrieval, and single memory lookup by hash. 2 Flutter +tests: an integration test verifying Rust initialization and app startup, and a +widget smoke test.</p> +<h2 id="architecture-decisions">Architecture decisions</h2> +<ul> +<li><strong>Embedded node, not client-server</strong>: the phone runs the full P2P stack, not a +thin client talking to a remote daemon. This means memories are preserved even +without internet. Users with a Raspberry Pi or VPS can optionally connect the +app to their daemon via GraphQL for higher availability, but it's not +required.</li> +<li><strong>Synchronous FFI</strong>: all flutter_rust_bridge functions are marked +<code>#[frb(sync)]</code> and block on the internal Tokio runtime. This simplifies the +Dart side (no async bridge complexity) while the Rust side handles concurrency +internally. Flutter's UI thread stays responsive because Riverpod wraps calls +in async providers.</li> +<li><strong>Global singleton</strong>: a <code>Mutex&lt;Option&lt;EmbeddedNode&gt;&gt;</code> global ensures the node +lifecycle is predictable — one start, one stop, no races. Mobile platforms +kill processes aggressively, so simplicity in lifecycle management is a +feature.</li> +<li><strong>Flat FFI types</strong>: no Rust abstractions leak across the FFI boundary. Every +type is a plain struct with strings and numbers. This makes the auto-generated +Dart bindings reliable and easy to debug.</li> +<li><strong>Three-screen onboarding</strong>: identity creation is the only required step. No +email, no password, no server registration. The app generates a cryptographic +identity locally and is ready to use.</li> +</ul> +<h2 id="what-comes-next">What comes next</h2> +<ul> +<li><strong>Phase 4: Resilience and Scale</strong> — Advanced NAT traversal (STUN/TURN), +Shamir's Secret Sharing for heirs, sealed tesseras with time-lock encryption, +performance tuning, security audits, OS packaging for +Alpine/Arch/Debian/FreeBSD/OpenBSD</li> +<li><strong>Phase 5: Exploration and Culture</strong> — Public tessera browser by +era/location/theme/language, institutional curation, genealogy integration, +physical media export (M-DISC, microfilm, acid-free paper with QR)</li> +</ul> +<p>The infrastructure is complete. The network exists, replication works, and now +anyone with a phone can participate. What remains is hardening what we have and +opening it to the world.</p> +</content> + + </entry> + <entry xml:lang="en"> + <title>Reed-Solomon: How Tesseras Survives Data Loss</title> + <published>2026-02-14T14:00:00+00:00</published> + <updated>2026-02-14T14:00:00+00:00</updated> + + <author> + <name> + + Unknown + + </name> + </author> + + <link rel="alternate" type="text/html" href="https://tesseras.net/news/reed-solomon/"/> + <id>https://tesseras.net/news/reed-solomon/</id> + + <content type="html" xml:base="https://tesseras.net/news/reed-solomon/"><p>Your hard drive will die. Your cloud provider will pivot. The RAID array in your +closet will outlive its controller but not its owner. If a memory is stored in +exactly one place, it has exactly one way to be lost forever.</p> +<p>Tesseras is a network that keeps human memories alive through mutual aid. The +core survival mechanism is <strong>Reed-Solomon erasure coding</strong> — a technique +borrowed from deep-space communication that lets us reconstruct data even when +pieces go missing.</p> +<h2 id="what-is-reed-solomon">What is Reed-Solomon?</h2> +<p>Reed-Solomon is a family of error-correcting codes invented by Irving Reed and +Gustave Solomon in 1960. The original use case was correcting errors in data +transmitted over noisy channels — think Voyager sending photos from Jupiter, or +a CD playing despite scratches.</p> +<p>The key insight: if you add carefully computed redundancy to your data <em>before</em> +something goes wrong, you can recover the original even after losing some +pieces.</p> +<p>Here's the intuition. Suppose you have a polynomial of degree 2 — a parabola. +You need 3 points to define it uniquely. But if you evaluate it at 5 points, you +can lose any 2 of those 5 and still reconstruct the polynomial from the +remaining 3. Reed-Solomon generalizes this idea to work over finite fields +(Galois fields), where the "polynomial" is your data and the "evaluation points" +are your fragments.</p> +<p>In concrete terms:</p> +<ol> +<li><strong>Split</strong> your data into <em>k</em> data shards</li> +<li><strong>Compute</strong> <em>m</em> parity shards from the data shards</li> +<li><strong>Distribute</strong> all <em>k + m</em> shards across different locations</li> +<li><strong>Reconstruct</strong> the original data from any <em>k</em> of the <em>k + m</em> shards</li> +</ol> +<p>You can lose up to <em>m</em> shards — any <em>m</em>, data or parity, in any combination — +and still recover everything.</p> +<h2 id="why-not-just-make-copies">Why not just make copies?</h2> +<p>The naive approach to redundancy is replication: make 3 copies, store them in 3 +places. This gives you tolerance for 2 failures at the cost of 3x your storage.</p> +<p>Reed-Solomon is dramatically more efficient:</p> +<table><thead><tr><th>Strategy</th><th style="text-align: right">Storage overhead</th><th style="text-align: right">Failures tolerated</th></tr></thead><tbody> +<tr><td>3x replication</td><td style="text-align: right">200%</td><td style="text-align: right">2 out of 3</td></tr> +<tr><td>Reed-Solomon (16,8)</td><td style="text-align: right">50%</td><td style="text-align: right">8 out of 24</td></tr> +<tr><td>Reed-Solomon (48,24)</td><td style="text-align: right">50%</td><td style="text-align: right">24 out of 72</td></tr> +</tbody></table> +<p>With 16 data shards and 8 parity shards, you use 50% extra storage but can +survive losing a third of all fragments. To achieve the same fault tolerance +with replication alone, you'd need 3x the storage.</p> +<p>For a network that aims to preserve memories across decades and centuries, this +efficiency isn't a nice-to-have — it's the difference between a viable system +and one that drowns in its own overhead.</p> +<h2 id="how-tesseras-uses-reed-solomon">How Tesseras uses Reed-Solomon</h2> +<p>Not all data deserves the same treatment. A 500-byte text memory and a 100 MB +video have very different redundancy needs. Tesseras uses a three-tier +fragmentation strategy:</p> +<p><strong>Small (&lt; 4 MB)</strong> — Whole-file replication to 7 peers. For small tesseras, the +overhead of erasure coding (encoding time, fragment management, reconstruction +logic) outweighs its benefits. Simple copies are faster and simpler.</p> +<p><strong>Medium (4–256 MB)</strong> — 16 data shards + 8 parity shards = 24 total fragments. +Each fragment is roughly 1/16th of the original size. Any 16 of the 24 fragments +reconstruct the original. Distributed across 7 peers.</p> +<p><strong>Large (≥ 256 MB)</strong> — 48 data shards + 24 parity shards = 72 total fragments. +Higher shard count means smaller individual fragments (easier to transfer and +store) and higher absolute fault tolerance. Also distributed across 7 peers.</p> +<p>The implementation uses the <code>reed-solomon-erasure</code> crate operating over GF(2⁸) — +the same Galois field used in QR codes and CDs. Each fragment carries a BLAKE3 +checksum so corruption is detected immediately, not silently propagated.</p> +<pre><code>Tessera (120 MB photo album) + ↓ encode +16 data shards (7.5 MB each) + 8 parity shards (7.5 MB each) + ↓ distribute +24 fragments across 7 peers (subnet-diverse) + ↓ any 16 fragments +Original tessera recovered +</code></pre> +<h2 id="the-challenges">The challenges</h2> +<p>Reed-Solomon solves the mathematical problem of redundancy. The engineering +challenges are everything around it.</p> +<h3 id="fragment-tracking">Fragment tracking</h3> +<p>Every fragment needs to be findable. Tesseras uses a Kademlia DHT for peer +discovery and fragment-to-peer mapping. When a node goes offline, its fragments +need to be re-created and distributed to new peers. This means tracking which +fragments exist, where they are, and whether they're still intact — across a +network with no central authority.</p> +<h3 id="silent-corruption">Silent corruption</h3> +<p>A fragment that returns wrong data is worse than one that's missing — at least a +missing fragment is honestly absent. Tesseras addresses this with +attestation-based health checks: the repair loop periodically asks fragment +holders to prove possession by returning BLAKE3 checksums. If a checksum doesn't +match, the fragment is treated as lost.</p> +<h3 id="correlated-failures">Correlated failures</h3> +<p>If all 24 fragments of a tessera land on machines in the same datacenter, a +single power outage kills them all. Reed-Solomon's math assumes independent +failures. Tesseras enforces <strong>subnet diversity</strong> during distribution: no more +than 2 fragments per /24 IPv4 subnet (or /48 IPv6 prefix). This spreads +fragments across different physical infrastructure.</p> +<h3 id="repair-speed-vs-network-load">Repair speed vs. network load</h3> +<p>When a peer goes offline, the clock starts ticking. Lost fragments need to be +re-created before more failures accumulate. But aggressive repair floods the +network. Tesseras balances this with a configurable repair loop (default: every +24 hours with 2-hour jitter) and concurrent transfer limits (default: 4 +simultaneous transfers). The jitter prevents repair storms where every node +checks its fragments at the same moment.</p> +<h3 id="long-term-key-management">Long-term key management</h3> +<p>Reed-Solomon protects against data loss, not against losing access. If a tessera +is encrypted (private or sealed visibility), you need the decryption key to make +the recovered data useful. Tesseras separates these concerns: erasure coding +handles availability, while Shamir's Secret Sharing (a future phase) will handle +key distribution among heirs. The project's design philosophy — encrypt as +little as possible — keeps the key management problem small.</p> +<h3 id="galois-field-limitations">Galois field limitations</h3> +<p>The GF(2⁸) field limits the total number of shards to 255 (data + parity +combined). For Tesseras, this is not a practical constraint — even the Large +tier uses only 72 shards. But it does mean that extremely large files with +thousands of fragments would require either a different field or a layered +encoding scheme.</p> +<h3 id="evolving-codec-compatibility">Evolving codec compatibility</h3> +<p>A tessera encoded today must be decodable in 50 years. Reed-Solomon over GF(2⁸) +is one of the most widely implemented algorithms in computing — it's in every CD +player, every QR code scanner, every deep-space probe. This ubiquity is itself a +survival strategy. The algorithm won't be forgotten because half the world's +infrastructure depends on it.</p> +<h2 id="the-bigger-picture">The bigger picture</h2> +<p>Reed-Solomon is a piece of a larger puzzle. It works in concert with:</p> +<ul> +<li><strong>Kademlia DHT</strong> for finding peers and routing fragments</li> +<li><strong>BLAKE3 checksums</strong> for integrity verification</li> +<li><strong>Bilateral reciprocity</strong> for fair storage exchange (no blockchain needed)</li> +<li><strong>Subnet diversity</strong> for failure independence</li> +<li><strong>Automatic repair</strong> for maintaining redundancy over time</li> +</ul> +<p>No single technique makes memories survive. Reed-Solomon ensures that data <em>can</em> +be recovered. The DHT ensures fragments <em>can be found</em>. Reciprocity ensures +peers <em>want to help</em>. Repair ensures none of this degrades over time.</p> +<p>A tessera is a bet that the sum of these mechanisms, running across many +independent machines operated by many independent people, is more durable than +any single institution. Reed-Solomon is the mathematical foundation of that bet.</p> +</content> + + </entry> + <entry xml:lang="en"> + <title>Phase 2: Memories Survive</title> + <published>2026-02-14T12:00:00+00:00</published> + <updated>2026-02-14T12:00:00+00:00</updated> + + <author> + <name> + + Unknown + + </name> + </author> + + <link rel="alternate" type="text/html" href="https://tesseras.net/news/phase2-replication/"/> + <id>https://tesseras.net/news/phase2-replication/</id> + + <content type="html" xml:base="https://tesseras.net/news/phase2-replication/"><p>A tessera is no longer tied to a single machine. Phase 2 delivers the +replication layer: data is split into erasure-coded fragments, distributed +across multiple peers, and automatically repaired when nodes go offline. A +bilateral reciprocity ledger ensures fair storage exchange — no blockchain, no +tokens.</p> +<h2 id="what-was-built">What was built</h2> +<p><strong>tesseras-core</strong> (updated) — New replication domain types: <code>FragmentPlan</code> +(selects fragmentation tier based on tessera size), <code>FragmentId</code> (tessera hash + +index + shard count + checksum), <code>FragmentEnvelope</code> (fragment with its metadata +for wire transport), <code>FragmentationTier</code> (Small/Medium/Large), <code>Attestation</code> +(proof that a node holds a fragment at a given time), and <code>ReplicateAck</code> +(acknowledgement of fragment receipt). Three new port traits define the +hexagonal boundaries: <code>DhtPort</code> (find peers, replicate fragments, request +attestations, ping), <code>FragmentStore</code> (store/read/delete/list/verify fragments), +and <code>ReciprocityLedger</code> (record storage exchanges, query balances, find best +peers). Maximum tessera size is 1 GB.</p> +<p><strong>tesseras-crypto</strong> (updated) — The existing <code>ReedSolomonCoder</code> now powers +fragment encoding. Data is split into shards, parity shards are computed, and +any combination of data shards can reconstruct the original — as long as the +number of missing shards does not exceed the parity count.</p> +<p><strong>tesseras-storage</strong> (updated) — Two new adapters:</p> +<ul> +<li><code>FsFragmentStore</code> — stores fragment data as files on disk +(<code>{root}/{tessera_hash}/{index:03}.shard</code>) with a SQLite metadata index +tracking tessera hash, shard index, shard count, checksum, and byte size. +Verification recomputes the BLAKE3 hash and compares it to the stored +checksum.</li> +<li><code>SqliteReciprocityLedger</code> — bilateral storage accounting in SQLite. Each peer +has a row tracking bytes stored for them and bytes they store for us. The +<code>balance</code> column is a generated column +(<code>bytes_they_store_for_us - bytes_stored_for_them</code>). UPSERT ensures atomic +increment of counters.</li> +</ul> +<p>New migration (<code>002_replication.sql</code>) adds tables for fragments, fragment plans, +holders, holder-fragment mappings, and reciprocity balances.</p> +<p><strong>tesseras-dht</strong> (updated) — Four new message variants: <code>Replicate</code> (send a +fragment envelope), <code>ReplicateAck</code> (confirm receipt), <code>AttestRequest</code> (ask a +node to prove it holds a tessera's fragments), and <code>AttestResponse</code> (return +attestation with checksums and timestamp). The engine handles these in its +message dispatch loop.</p> +<p><strong>tesseras-replication</strong> — The new crate, with five modules:</p> +<ul> +<li> +<p><em>Fragment encoding</em> (<code>fragment.rs</code>): <code>encode_tessera()</code> selects the +fragmentation tier based on size, then calls Reed-Solomon encoding for Medium +and Large tiers. Three tiers:</p> +<ul> +<li><strong>Small</strong> (&lt; 4 MB): whole-file replication to r=7 peers, no erasure coding</li> +<li><strong>Medium</strong> (4–256 MB): 16 data + 8 parity shards, distributed across r=7 +peers</li> +<li><strong>Large</strong> (≥ 256 MB): 48 data + 24 parity shards, distributed across r=7 +peers</li> +</ul> +</li> +<li> +<p><em>Distribution</em> (<code>distributor.rs</code>): subnet diversity filtering limits peers per +/24 IPv4 subnet (or /48 IPv6 prefix) to avoid correlated failures. If all your +fragments land on the same rack, a single power outage kills them all.</p> +</li> +<li> +<p><em>Service</em> (<code>service.rs</code>): <code>ReplicationService</code> is the orchestrator. +<code>replicate_tessera()</code> encodes the data, finds the closest peers via DHT, +applies subnet diversity, and distributes fragments round-robin. +<code>receive_fragment()</code> validates the BLAKE3 checksum, checks reciprocity balance +(rejects if the sender's deficit exceeds the configured threshold), stores the +fragment, and updates the ledger. <code>handle_attestation_request()</code> lists local +fragments and computes their checksums as proof of possession.</p> +</li> +<li> +<p><em>Repair</em> (<code>repair.rs</code>): <code>check_tessera_health()</code> requests attestations from +known holders, falls back to ping for unresponsive nodes, verifies local +fragment integrity, and returns one of three actions: <code>Healthy</code>, +<code>NeedsReplication { deficit }</code>, or <code>CorruptLocal { fragment_index }</code>. The +repair loop runs every 24 hours (with 2-hour jitter) via <code>tokio::select!</code> with +shutdown integration.</p> +</li> +<li> +<p><em>Configuration</em> (<code>config.rs</code>): <code>ReplicationConfig</code> with defaults for repair +interval (24h), jitter (2h), concurrent transfers (4), minimum free space (1 +GB), deficit allowance (256 MB), and per-peer storage limit (1 GB).</p> +</li> +</ul> +<p><strong>tesd</strong> (updated) — The daemon now opens a SQLite database (<code>db/tesseras.db</code>), +runs migrations, creates <code>FsFragmentStore</code>, <code>SqliteReciprocityLedger</code>, and +<code>FsBlobStore</code> instances, wraps the DHT engine in a <code>DhtPortAdapter</code>, builds a +<code>ReplicationService</code>, and spawns the repair loop as a background task with +graceful shutdown.</p> +<p><strong>Testing</strong> — 193 tests across the workspace:</p> +<ul> +<li>15 unit tests in tesseras-replication (fragment encoding tiers, checksum +validation, subnet diversity, repair health checks, service receive/replicate +flows)</li> +<li>3 integration tests with real storage (full encode→distribute→receive cycle +for medium tessera, small whole-file replication, tampered fragment rejection)</li> +<li>Tests use in-memory SQLite + tempdir fragments with mockall mocks for DHT and +BlobStore</li> +<li>Zero clippy warnings, clean formatting</li> +</ul> +<h2 id="architecture-decisions">Architecture decisions</h2> +<ul> +<li><strong>Three-tier fragmentation</strong>: small files don't need erasure coding — the +overhead isn't worth it. Medium and large files get progressively more parity +shards. This avoids wasting storage on small tesseras while providing strong +redundancy for large ones.</li> +<li><strong>Owner-push distribution</strong>: the tessera owner encodes fragments and pushes +them to peers, rather than peers pulling. This simplifies the protocol (no +negotiation phase) and ensures fragments are distributed immediately.</li> +<li><strong>Bilateral reciprocity without consensus</strong>: each node tracks its own balance +with each peer locally. No global ledger, no token, no blockchain. If peer A +stores 500 MB for peer B, peer B should store roughly 500 MB for peer A. Free +riders lose redundancy gradually — their fragments are deprioritized for +repair, but never deleted.</li> +<li><strong>Subnet diversity</strong>: fragments are spread across different network subnets to +survive correlated failures. A datacenter outage shouldn't take out all copies +of a tessera.</li> +<li><strong>Attestation-first health checks</strong>: the repair loop asks holders to prove +possession (attestation with checksums) before declaring a tessera degraded. +Only when attestation fails does it fall back to a simple ping. This catches +silent data corruption, not just node departure.</li> +</ul> +<h2 id="what-comes-next">What comes next</h2> +<ul> +<li><strong>Phase 3: API and Apps</strong> — Flutter mobile/desktop app via +flutter_rust_bridge, GraphQL API (async-graphql), WASM browser node</li> +<li><strong>Phase 4: Resilience and Scale</strong> — ML-DSA post-quantum signatures, advanced +NAT traversal, Shamir's Secret Sharing for heirs, packaging for +Alpine/Arch/Debian/FreeBSD/OpenBSD, CI on SourceHut</li> +<li><strong>Phase 5: Exploration and Culture</strong> — public tessera browser, institutional +curation, genealogy integration, physical media export</li> +</ul> +<p>Nodes can find each other and keep each other's memories alive. Next, we give +people a way to hold their memories in their hands.</p> +</content> + + </entry> + <entry xml:lang="en"> + <title>Phase 1: Nodes Find Each Other</title> + <published>2026-02-14T11:00:00+00:00</published> + <updated>2026-02-14T11:00:00+00:00</updated> + + <author> + <name> + + Unknown + + </name> + </author> + + <link rel="alternate" type="text/html" href="https://tesseras.net/news/phase1-basic-network/"/> + <id>https://tesseras.net/news/phase1-basic-network/</id> + + <content type="html" xml:base="https://tesseras.net/news/phase1-basic-network/"><p>Tesseras is no longer a local-only tool. Phase 1 delivers the networking layer: +nodes discover each other through a Kademlia DHT, communicate over QUIC, and +publish tessera pointers that any peer on the network can find. A tessera +created on node A is now findable from node C.</p> +<h2 id="what-was-built">What was built</h2> +<p><strong>tesseras-core</strong> (updated) — New network domain types: <code>TesseraPointer</code> +(lightweight reference to a tessera's holders and fragment locations), +<code>NodeIdentity</code> (node ID + public key + proof-of-work nonce), <code>NodeInfo</code> +(identity + address + capabilities), and <code>Capabilities</code> (bitflags for what a +node supports: DHT, storage, relay, replication).</p> +<p><strong>tesseras-net</strong> — The transport layer, built on QUIC via quinn. The <code>Transport</code> +trait defines the port: <code>send</code>, <code>recv</code>, <code>disconnect</code>, <code>local_addr</code>. Two adapters +implement it:</p> +<ul> +<li><code>QuinnTransport</code> — real QUIC with self-signed TLS, ALPN negotiation +(<code>tesseras/1</code>), connection pooling via DashMap, and a background accept loop +that handles incoming streams.</li> +<li><code>MemTransport</code> + <code>SimNetwork</code> — in-memory channels for deterministic testing +without network I/O. Every integration test in the DHT crate runs against +this.</li> +</ul> +<p>The wire protocol uses length-prefixed MessagePack: a 4-byte big-endian length +header followed by an rmp-serde payload. <code>WireMessage</code> carries a version byte, +request ID, and a body that can be a request, response, or protocol-level error. +Maximum message size is 64 KiB.</p> +<p><strong>tesseras-dht</strong> — A complete Kademlia implementation:</p> +<ul> +<li><em>Routing table</em>: 160 k-buckets with k=20. Least-recently-seen eviction, +move-to-back on update, ping-check before replacing a full bucket's oldest +entry.</li> +<li><em>XOR distance</em>: 160-bit XOR metric with bucket indexing by highest differing +bit.</li> +<li><em>Proof-of-work</em>: nodes grind a nonce until <code>BLAKE3(pubkey || nonce)[..20]</code> has +8 leading zero bits (~256 hash attempts on average). Cheap enough for any +device, expensive enough to make Sybil attacks impractical at scale.</li> +<li><em>Protocol messages</em>: Ping/Pong, FindNode/FindNodeResponse, +FindValue/FindValueResult, Store — all serialized with MessagePack via serde.</li> +<li><em>Pointer store</em>: bounded in-memory store with configurable TTL (24 hours +default) and max entries (10,000 default). When full, evicts pointers furthest +from the local node ID, following Kademlia's distance-based responsibility +model.</li> +<li><em>DhtEngine</em>: the main orchestrator. Handles incoming RPCs, runs iterative +lookups (alpha=3 parallelism), bootstrap, publish, and find. The <code>run()</code> +method drives a <code>tokio::select!</code> loop with maintenance timers: routing table +refresh every 60 seconds, pointer expiry every 5 minutes.</li> +</ul> +<p><strong>tesd</strong> — A full-node binary. Parses CLI args (bind address, bootstrap peers, +data directory), generates a PoW-valid node identity, binds a QUIC endpoint, +bootstraps into the network, and runs the DHT engine. Graceful shutdown on +Ctrl+C via tokio signal handling.</p> +<p><strong>Infrastructure</strong> — OpenTofu configuration for two Hetzner Cloud bootstrap +nodes (cx22 instances in Falkenstein, Germany and Helsinki, Finland). Cloud-init +provisioning script creates a dedicated <code>tesseras</code> user, writes a config file, +and sets up a systemd service. Firewall rules open UDP 4433 (QUIC) and restrict +metrics to internal access.</p> +<p><strong>Testing</strong> — 139 tests across the workspace:</p> +<ul> +<li>47 unit tests in tesseras-dht (routing table, distance, PoW, pointer store, +message serialization, engine RPCs)</li> +<li>5 multi-node integration tests (3-node bootstrap, 10-node lookup convergence, +publish-and-find, node departure detection, PoW rejection)</li> +<li>14 tests in tesseras-net (codec roundtrips, transport send/recv, backpressure, +disconnect)</li> +<li>Docker Compose smoke tests with 3 containerized nodes communicating over real +QUIC</li> +<li>Zero clippy warnings, clean formatting</li> +</ul> +<h2 id="architecture-decisions">Architecture decisions</h2> +<ul> +<li><strong>Transport as a port</strong>: the <code>Transport</code> trait is the only interface between +the DHT engine and the network. Swapping QUIC for any other protocol means +implementing four methods. All DHT tests use the in-memory adapter, making +them fast and deterministic.</li> +<li><strong>One stream per RPC</strong>: each DHT request-response pair uses a fresh +bidirectional QUIC stream. No multiplexing complexity, no head-of-line +blocking between independent operations. QUIC handles the multiplexing at the +connection level.</li> +<li><strong>MessagePack over Protobuf</strong>: compact binary encoding without code generation +or schema files. Serde integration means adding a field to a message is a +one-line change. Trade-off: no built-in schema evolution guarantees, but at +this stage velocity matters more.</li> +<li><strong>PoW instead of stake or reputation</strong>: a node identity costs ~256 BLAKE3 +hashes. This runs in under a second on any hardware, including a Raspberry Pi, +but generating thousands of identities for a Sybil attack becomes expensive. +No tokens, no blockchain, no external dependencies.</li> +<li><strong>Iterative lookup with routing table updates</strong>: discovered nodes are added to +the routing table as they're encountered during iterative lookups, following +standard Kademlia behavior. This ensures the routing table improves +organically as nodes interact.</li> +</ul> +<h2 id="what-comes-next">What comes next</h2> +<ul> +<li><strong>Phase 2: Replication</strong> — Reed-Solomon erasure coding over the network, +fragment distribution, automatic repair loops, bilateral reciprocity ledger +(no blockchain, no tokens)</li> +<li><strong>Phase 3: API and Apps</strong> — Flutter mobile/desktop app via +flutter_rust_bridge, GraphQL API (async-graphql), WASM browser node</li> +<li><strong>Phase 4: Resilience and Scale</strong> — ML-DSA post-quantum signatures, advanced +NAT traversal, Shamir's Secret Sharing for heirs, packaging for +Alpine/Arch/Debian/FreeBSD/OpenBSD, CI on SourceHut</li> +<li><strong>Phase 5: Exploration and Culture</strong> — public tessera browser, institutional +curation, genealogy integration, physical media export</li> +</ul> +<p>Nodes can find each other. Next, they learn to keep each other's memories alive.</p> +</content> + + </entry> + <entry xml:lang="en"> + <title>Phase 0: Foundation Laid</title> + <published>2026-02-14T10:00:00+00:00</published> + <updated>2026-02-14T10:00:00+00:00</updated> + + <author> + <name> + + Unknown + + </name> + </author> + + <link rel="alternate" type="text/html" href="https://tesseras.net/news/phase0-foundation/"/> + <id>https://tesseras.net/news/phase0-foundation/</id> + + <content type="html" xml:base="https://tesseras.net/news/phase0-foundation/"><p>The first milestone of the Tesseras project is complete. Phase 0 establishes the +foundation that every future component will build on: domain types, +cryptography, storage, and a usable command-line interface.</p> +<h2 id="what-was-built">What was built</h2> +<p><strong>tesseras-core</strong> — The domain layer defines the tessera format: <code>ContentHash</code> +(BLAKE3, 32 bytes), <code>NodeId</code> (Kademlia, 20 bytes), memory types (Moment, +Reflection, Daily, Relation, Object), visibility modes (Private, Circle, Public, +PublicAfterDeath, Sealed), and a plain-text manifest format that can be parsed +by any programming language for the next thousand years. The application service +layer (<code>TesseraService</code>) handles create, verify, export, and list operations +through port traits, following hexagonal architecture.</p> +<p><strong>tesseras-crypto</strong> — Ed25519 key generation, signing, and verification. A +dual-signature framework (Ed25519 + ML-DSA placeholder) ready for post-quantum +migration. BLAKE3 content hashing. Reed-Solomon erasure coding behind a feature +flag for future replication.</p> +<p><strong>tesseras-storage</strong> — SQLite index via rusqlite with plain-SQL migrations. +Filesystem blob store with content-addressable layout +(<code>blobs/&lt;tessera_hash&gt;/&lt;memory_hash&gt;/&lt;filename&gt;</code>). Identity key persistence on +disk.</p> +<p><strong>tesseras-cli</strong> — A working <code>tesseras</code> binary with five commands:</p> +<ul> +<li><code>init</code> — generates Ed25519 identity, creates SQLite database</li> +<li><code>create &lt;dir&gt;</code> — scans a directory for media files, creates a signed tessera</li> +<li><code>verify &lt;hash&gt;</code> — checks signature and file integrity</li> +<li><code>export &lt;hash&gt; &lt;dest&gt;</code> — writes a self-contained tessera directory</li> +<li><code>list</code> — shows a table of stored tesseras</li> +</ul> +<p><strong>Testing</strong> — 67+ tests across the workspace: unit tests in every module, +property-based tests (proptest) for hex roundtrips and manifest serialization, +integration tests covering the full create-verify-export cycle including +tampered file and invalid signature detection. Zero clippy warnings.</p> +<h2 id="architecture-decisions">Architecture decisions</h2> +<ul> +<li><strong>Hexagonal architecture</strong>: crypto operations are injected via trait objects +(<code>Box&lt;dyn Hasher&gt;</code>, <code>Box&lt;dyn ManifestSigner&gt;</code>, <code>Box&lt;dyn ManifestVerifier&gt;</code>), +keeping the core crate free of concrete crypto dependencies.</li> +<li><strong>Feature flags</strong>: the <code>service</code> feature on tesseras-core gates the async +application layer. The <code>classical</code> and <code>erasure</code> features on tesseras-crypto +control which algorithms are compiled in.</li> +<li><strong>Plain-text manifest</strong>: parseable without any binary format library, with +explicit <code>blake3:</code> hash prefixes and human-readable layout.</li> +</ul> +<h2 id="what-comes-next">What comes next</h2> +<p>Phase 0 is the local-only foundation. The road ahead:</p> +<ul> +<li><strong>Phase 1: Networking</strong> — QUIC transport (quinn), Kademlia DHT for peer +discovery, NAT traversal</li> +<li><strong>Phase 2: Replication</strong> — Reed-Solomon erasure coding over the network, +repair loops, bilateral reciprocity (no blockchain, no tokens)</li> +<li><strong>Phase 3: Clients</strong> — Flutter mobile/desktop app via flutter_rust_bridge, +GraphQL API, WASM browser node</li> +<li><strong>Phase 4: Hardening</strong> — ML-DSA post-quantum signatures, packaging for +Alpine/Arch/Debian/FreeBSD/OpenBSD, CI on SourceHut</li> +</ul> +<p>The tessera format is stable. Everything built from here connects to and extends +what exists today.</p> +</content> + + </entry> + <entry xml:lang="en"> + <title>Hello, World</title> + <published>2026-02-13T00:00:00+00:00</published> + <updated>2026-02-13T00:00:00+00:00</updated> + + <author> + <name> + + Unknown + + </name> + </author> + + <link rel="alternate" type="text/html" href="https://tesseras.net/news/hello-world/"/> + <id>https://tesseras.net/news/hello-world/</id> + + <content type="html" xml:base="https://tesseras.net/news/hello-world/"><p>Today we're announcing the Tesseras project: a peer-to-peer network for +preserving human memories across millennia.</p> +<p>Tesseras is built on a simple idea — your photos, recordings, and writings +deserve to outlast any company, platform, or file format. Each person creates a +tessera, a self-contained time capsule that the network keeps alive through +mutual aid and redundancy.</p> +<p>The project is in its earliest stage. We're building the foundation: tools to +create, verify, and export tesseras offline. The network layer, replication, and +apps will follow.</p> +<p>If this mission resonates with you, <a href="/subscriptions/">join the mailing list</a> or +browse the <a rel="external" href="https://git.sr.ht/~ijanc/tesseras">source code</a>.</p> +</content> + + </entry> +</feed> |