diff options
| author | murilo ijanc | 2026-03-24 21:45:05 -0300 |
|---|---|---|
| committer | murilo ijanc | 2026-03-24 21:45:05 -0300 |
| commit | 01c17c68277ff88fab812920732d9bbe9e6bb571 (patch) | |
| tree | 035398ae34263b981b621c6275835d2cc6847d57 /news/phase4-institutional-onboarding | |
| parent | f186b71ca51e83837db60de13322394bb5e6d348 (diff) | |
| download | website-01c17c68277ff88fab812920732d9bbe9e6bb571.tar.gz | |
Remove old Zola-generated content, keep only the essential
landing page with about, contact, and license sections.
Diffstat (limited to 'news/phase4-institutional-onboarding')
| -rw-r--r-- | news/phase4-institutional-onboarding/index.html | 239 | ||||
| -rw-r--r-- | news/phase4-institutional-onboarding/index.html.gz | bin | 5539 -> 0 bytes |
2 files changed, 0 insertions, 239 deletions
diff --git a/news/phase4-institutional-onboarding/index.html b/news/phase4-institutional-onboarding/index.html deleted file mode 100644 index cbd9ac3..0000000 --- a/news/phase4-institutional-onboarding/index.html +++ /dev/null @@ -1,239 +0,0 @@ -<!DOCTYPE html> -<html lang="en"> -<head> - <meta charset="utf-8"> - <meta name="viewport" content="width=device-width, initial-scale=1"> - <title>Phase 4: Institutional Node Onboarding — Tesseras</title> - <meta name="description" content="Libraries, archives, and museums can now join the Tesseras network as verified institutional nodes with DNS-based identity, full-text search indexes, and configurable storage pledges."> - <!-- Open Graph --> - <meta property="og:type" content="article"> - <meta property="og:title" content="Phase 4: Institutional Node Onboarding"> - <meta property="og:description" content="Libraries, archives, and museums can now join the Tesseras network as verified institutional nodes with DNS-based identity, full-text search indexes, and configurable storage pledges."> - <meta property="og:image" content="https://tesseras.net/images/social.jpg"> - <meta property="og:image:width" content="1200"> - <meta property="og:image:height" content="630"> - <meta property="og:site_name" content="Tesseras"> - <!-- Twitter Card --> - <meta name="twitter:card" content="summary_large_image"> - <meta name="twitter:title" content="Phase 4: Institutional Node Onboarding"> - <meta name="twitter:description" content="Libraries, archives, and museums can now join the Tesseras network as verified institutional nodes with DNS-based identity, full-text search indexes, and configurable storage pledges."> - <meta name="twitter:image" content="https://tesseras.net/images/social.jpg"> - <link rel="stylesheet" href="https://tesseras.net/style.css?h=21f0f32121928ee5c690"> - - - <link rel="alternate" type="application/atom+xml" title="Tesseras" href="https://tesseras.net/atom.xml"> - - - <link rel="icon" type="image/png" sizes="32x32" href="https://tesseras.net/images/favicon.png?h=be4e123a23393b1a027d"> - -</head> -<body> - <header> - <h1> - <a href="https://tesseras.net/"> - <img src="https://tesseras.net/images/logo-64.png?h=c1b8d0c4c5f93b49d40b" alt="Tesseras" width="40" height="40" class="logo"> - Tesseras - </a> - </h1> - <nav> - - <a href="https://tesseras.net/about/">About</a> - <a href="https://tesseras.net/news/">News</a> - <a href="https://tesseras.net/releases/">Releases</a> - <a href="https://tesseras.net/faq/">FAQ</a> - <a href="https://tesseras.net/subscriptions/">Subscriptions</a> - <a href="https://tesseras.net/contact/">Contact</a> - - </nav> - <nav class="lang-switch"> - - <strong>English</strong> | <a href="/pt-br/news/phase4-institutional-onboarding/">Português</a> - - </nav> - </header> - - <main> - -<article> - <h2>Phase 4: Institutional Node Onboarding</h2> - <p class="news-date">2026-02-15</p> - <p>A P2P network of individuals is fragile. Hard drives die, phones get lost, -people lose interest. The long-term survival of humanity's memories depends on -institutions — libraries, archives, museums, universities — that measure their -lifetimes in centuries. Phase 4 continues with institutional node onboarding: -verified organizations can now pledge storage, run searchable indexes, and -participate in the network with a distinct identity.</p> -<p>The design follows a principle of trust but verify: institutions identify -themselves via DNS TXT records (the same mechanism used by SPF, DKIM, and DMARC -for email), pledge a storage budget, and receive reciprocity exemptions so they -can store fragments for others without expecting anything in return. In -exchange, the network treats their fragments as higher-quality replicas and -limits over-reliance on any single institution through diversity constraints.</p> -<h2 id="what-was-built">What was built</h2> -<p><strong>Capability bits</strong> (<code>tesseras-core/src/network.rs</code>) — Two new flags added to -the <code>Capabilities</code> bitfield: <code>INSTITUTIONAL</code> (bit 7) and <code>SEARCH_INDEX</code> (bit 8). -A new <code>institutional_default()</code> constructor returns the full Phase 2 capability -set plus these two bits and <code>RELAY</code>. Normal nodes advertise <code>phase2_default()</code> -which lacks institutional flags. Serialization roundtrip tests verify the new -bits survive MessagePack encoding.</p> -<p><strong>Search types</strong> (<code>tesseras-core/src/search.rs</code>) — Three new domain types for -the search subsystem:</p> -<ul> -<li><code>SearchFilters</code> — query parameters: <code>memory_type</code>, <code>visibility</code>, <code>language</code>, -<code>date_range</code>, <code>geo</code> (bounding box), <code>page</code>, <code>page_size</code></li> -<li><code>SearchHit</code> — a single result: content hash plus a <code>MetadataExcerpt</code> (title, -description, memory type, creation date, visibility, language, tags)</li> -<li><code>GeoFilter</code> — bounding box with <code>min_lat</code>, <code>max_lat</code>, <code>min_lon</code>, <code>max_lon</code> for -spatial queries</li> -</ul> -<p>All types derive <code>Serialize</code>/<code>Deserialize</code> for wire transport and -<code>Clone</code>/<code>Debug</code> for diagnostics.</p> -<p><strong>Institutional daemon config</strong> (<code>tesd/src/config.rs</code>) — A new <code>[institutional]</code> -TOML section with <code>domain</code> (the DNS domain to verify), <code>pledge_bytes</code> (storage -commitment in bytes), and <code>search_enabled</code> (toggle for the FTS5 index). The -<code>to_dht_config()</code> method now sets <code>Capabilities::institutional_default()</code> when -institutional config is present, so institutional nodes advertise the right -capability bits in Pong responses.</p> -<p><strong>DNS TXT verification</strong> (<code>tesd/src/institutional.rs</code>) — Async DNS resolution -using <code>hickory-resolver</code> to verify institutional identity. The daemon looks up -<code>_tesseras.<domain></code> TXT records and parses key-value fields: <code>v</code> (version), -<code>node</code> (hex-encoded node ID), and <code>pledge</code> (storage pledge in bytes). -Verification checks:</p> -<ol> -<li>A TXT record exists at <code>_tesseras.<domain></code></li> -<li>The <code>node</code> field matches the daemon's own node ID</li> -<li>The <code>pledge</code> field is present and valid</li> -</ol> -<p>On startup, the daemon attempts DNS verification. If it succeeds, the node runs -with institutional capabilities. If it fails, the node logs a warning and -downgrades to a normal full node — no crash, no manual intervention.</p> -<p><strong>CLI setup command</strong> (<code>tesseras-cli/src/institutional.rs</code>) — A new -<code>institutional setup</code> subcommand that guides operators through onboarding:</p> -<ol> -<li>Reads the node's identity from the data directory</li> -<li>Prompts for domain name and pledge size</li> -<li>Generates the exact DNS TXT record to add: -<code>v=tesseras1 node=<hex> pledge=<bytes></code></li> -<li>Writes the institutional section to the daemon's config file</li> -<li>Prints next steps: add the TXT record, restart the daemon</li> -</ol> -<p><strong>SQLite search index</strong> (<code>tesseras-storage</code>) — A migration -(<code>003_institutional.sql</code>) that creates three structures:</p> -<ul> -<li><code>search_content</code> — an FTS5 virtual table for full-text search over tessera -metadata (title, description, creator, tags, language)</li> -<li><code>geo_index</code> — an R-tree virtual table for spatial bounding-box queries over -latitude/longitude</li> -<li><code>geo_map</code> — a mapping table linking R-tree row IDs to content hashes</li> -</ul> -<p>The <code>SqliteSearchIndex</code> adapter implements the <code>SearchIndex</code> port trait with -<code>index_tessera()</code> (insert/update) and <code>search()</code> (query with filters). FTS5 -queries support natural language search; geo queries use R-tree <code>INTERSECT</code> for -bounding box lookups. Results are ranked by FTS5 relevance score.</p> -<p>The migration also adds an <code>is_institutional</code> column to the <code>reciprocity</code> table, -handled idempotently via <code>pragma_table_info</code> checks (SQLite's -<code>ALTER TABLE ADD COLUMN</code> lacks <code>IF NOT EXISTS</code>).</p> -<p><strong>Reciprocity bypass</strong> (<code>tesseras-replication/src/service.rs</code>) — Institutional -nodes are exempt from reciprocity checks. When <code>receive_fragment()</code> is called, -if the sender's node ID is marked as institutional in the reciprocity ledger, -the balance check is skipped entirely. This means institutions can store -fragments for the entire network without needing to "earn" credits first — their -DNS-verified identity and storage pledge serve as their credential.</p> -<p><strong>Node-type diversity constraint</strong> (<code>tesseras-replication/src/distributor.rs</code>) — -A new <code>apply_institutional_diversity()</code> function limits how many replicas of a -single tessera can land on institutional nodes. The cap is -<code>ceil(replication_factor / 3.5)</code> — with the default <code>r=7</code>, at most 2 of 7 -replicas go to institutions. This prevents the network from becoming dependent -on a small number of large institutions: if a university's servers go down, at -least 5 replicas remain on independent nodes.</p> -<p><strong>DHT message extensions</strong> (<code>tesseras-dht/src/message.rs</code>) — Two new message -variants:</p> -<table><thead><tr><th>Message</th><th>Purpose</th></tr></thead><tbody> -<tr><td><code>Search</code></td><td>Client sends query string, filters, and page number</td></tr> -<tr><td><code>SearchResult</code></td><td>Institutional node responds with hits and total count</td></tr> -</tbody></table> -<p>The <code>encode()</code> function was switched from positional to named MessagePack -serialization (<code>rmp_serde::to_vec_named</code>) to handle <code>SearchFilters</code>' optional -fields correctly — positional encoding breaks when <code>skip_serializing_if</code> omits -fields.</p> -<p><strong>Prometheus metrics</strong> (<code>tesd/src/metrics.rs</code>) — Eight institutional-specific -metrics:</p> -<ul> -<li><code>tesseras_institutional_pledge_bytes</code> — configured storage pledge</li> -<li><code>tesseras_institutional_stored_bytes</code> — actual bytes stored</li> -<li><code>tesseras_institutional_pledge_utilization_ratio</code> — stored/pledged ratio</li> -<li><code>tesseras_institutional_peers_served</code> — unique peers served fragments</li> -<li><code>tesseras_institutional_search_index_total</code> — tesseras in the search index</li> -<li><code>tesseras_institutional_search_queries_total</code> — search queries received</li> -<li><code>tesseras_institutional_dns_verification_status</code> — 1 if DNS verified, 0 -otherwise</li> -<li><code>tesseras_institutional_dns_verification_last</code> — Unix timestamp of last -verification</li> -</ul> -<p><strong>Integration tests</strong> — Two tests in -<code>tesseras-replication/tests/integration.rs</code>:</p> -<ul> -<li><code>institutional_peer_bypasses_reciprocity</code> — verifies that an institutional -peer with a massive deficit (-999,999 balance) is still allowed to store -fragments, while a non-institutional peer with the same deficit is rejected</li> -<li><code>institutional_node_accepts_fragment_despite_deficit</code> — full async test using -<code>ReplicationService</code> with mocked DHT, fragment store, reciprocity ledger, and -blob store: sends a fragment from an institutional sender and verifies it's -accepted</li> -</ul> -<p>322 tests pass across the workspace. Clippy clean with <code>-D warnings</code>.</p> -<h2 id="architecture-decisions">Architecture decisions</h2> -<ul> -<li><strong>DNS TXT over PKI or blockchain</strong>: DNS is universally deployed, universally -understood, and already used for domain verification (SPF, DKIM, Let's -Encrypt). Institutions already manage DNS. No certificate authority, no token, -no on-chain transaction — just a TXT record. If an institution loses control -of their domain, the verification naturally fails on the next check.</li> -<li><strong>Graceful degradation on DNS failure</strong>: if DNS verification fails at startup, -the daemon downgrades to a normal full node instead of refusing to start. This -prevents operational incidents — a DNS misconfiguration shouldn't take a node -offline.</li> -<li><strong>Diversity cap at <code>ceil(r / 3.5)</code></strong>: with <code>r=7</code>, at most 2 replicas go to -institutions. This is conservative — it ensures the network never depends on -institutions for majority quorum, while still benefiting from their storage -capacity and uptime.</li> -<li><strong>Named MessagePack encoding</strong>: switching from positional to named encoding -adds ~15% overhead per message but eliminates a class of serialization bugs -when optional fields are present. The DHT is not bandwidth-constrained at the -message level, so the tradeoff is worth it.</li> -<li><strong>Reciprocity exemption over credit grants</strong>: rather than giving institutions -a large initial credit balance (which is arbitrary and needs tuning), we -exempt them entirely. Their DNS-verified identity and public storage pledge -replace the bilateral reciprocity mechanism.</li> -<li><strong>FTS5 + R-tree in SQLite</strong>: full-text search and spatial indexing are built -into SQLite as loadable extensions. No external search engine (Elasticsearch, -Meilisearch) needed. This keeps the deployment a single binary with a single -database file — critical for institutional operators who may not have a DevOps -team.</li> -</ul> -<h2 id="what-comes-next">What comes next</h2> -<ul> -<li><strong>Phase 4 continued</strong> — storage deduplication (content-addressable store with -BLAKE3 keying), security audits, OS packaging (Alpine, Arch, Debian, OpenBSD, -FreeBSD)</li> -<li><strong>Phase 5: Exploration and Culture</strong> — public tessera browser by -era/location/theme/language, institutional curation, genealogy integration -(FamilySearch, Ancestry), physical media export (M-DISC, microfilm, acid-free -paper with QR), AI-assisted context</li> -</ul> -<p>Institutional onboarding closes a critical gap in Tesseras' preservation model. -Individual nodes provide grassroots resilience — thousands of devices across the -globe, each storing a few fragments. Institutional nodes provide anchoring — -organizations with professional infrastructure, redundant storage, and -multi-decade operational horizons. Together, they form a network where memories -can outlast both individual devices and individual institutions.</p> - -</article> - - </main> - - <footer> - <p>© 2026 Tesseras Project. <a href="/atom.xml">News Feed</a> · <a href="https://git.sr.ht/~ijanc/tesseras">Source</a></p> - </footer> -</body> -</html> diff --git a/news/phase4-institutional-onboarding/index.html.gz b/news/phase4-institutional-onboarding/index.html.gz Binary files differdeleted file mode 100644 index 92beb89..0000000 --- a/news/phase4-institutional-onboarding/index.html.gz +++ /dev/null |