Code architecture

The runtime architecture page follows one request through the process. This page is for the developer changing that process: how the source splits into crates, which way the dependencies point, and what you implement to add a packaging format. It assumes no prior Rust or packaging knowledge and links each term the first time it appears.

peryx is written in Rust and organized as a Cargo workspace: one repository holding many crates, where a crate is Rust's unit of compilation and dependency (a library or a binary). Splitting a program into crates is how Rust enforces boundaries. A crate can only call what another crate makes public, and the dependency graph between crates must form no cycles, so the layering below is not a convention the compiler could let you break.

One rule shapes the layout. A shared crate defines an abstraction and the functionality every ecosystem reuses; an ecosystem crate owns the full implementation of its own format and nothing else. Here an ecosystem is a packaging format peryx serves: PyPI (the Python Package Index, where Python libraries are published) and OCI (the Open Container Initiative, the standard behind the container images Docker and Podman push and pull) today. Each speaks its own wire protocol, the on-the-wire request and response format a client and server agree on. Adding OCI beside PyPI meant writing a new peryx-ecosystem-oci crate against a trait, not editing the server that hosts it. A third ecosystem is the same shape again.

The crate map

Dependencies point down: a crate at the tail of an arrow uses the crate at its head. Both ecosystems and both hosts (the router and the web UI) depend on peryx-driver, the seam in blue. Neither ecosystem depends on the router. You can prove that with cargo tree, the command that prints a crate's dependency graph: cargo tree -p peryx-ecosystem-pypi --edges normal -i peryx-http prints nothing, because no normal (non-test) dependency path leads from the PyPI crate back to the router. The ecosystems reference peryx-http only as a dev-dependency (a dependency compiled for tests, never for the shipped binary), so their integration tests can spin up a real router without the normal build ever pointing an ecosystem back at its host. The binary at the top is the one place that names pypi and oci and wires them together.

peryx-web points at no ecosystem. Server rendering calls the driver directly, and the browser fetches neutral view-model JSON the server produced by calling the driver, so the client draws a PyPI project or an OCI manifest through the same neutral models (see the block protocol below) without ever parsing a format itself.

The layers

Foundation. The crates a driver reads through, none of which knows a wire protocol. peryx-core holds the neutral domain: the closed set of packaging formats and the closed set of index roles (each is a Rust enum, a type whose value is exactly one of a fixed set of variants), the per-ecosystem vocabulary, the neutral view models, and URL path safety. peryx-index is the role engine, covering the index model, route resolution, virtual-layer shadowing, and the serving caches. peryx-storage is the two neutral stores, neither knowing any format's schema: artifacts as content-addressed blobs on disk (a file's key is the SHA-256 hash of its bytes, so identical bytes are stored once and every reference is tamper-evident), and a redb (an embedded key-value database, the Rust counterpart to SQLite or LMDB) database holding the serial counter, the webhook queue, and the shared key-value store each ecosystem lays its own metadata into.

The rest of the foundation is the cross-cutting machinery every ecosystem reuses. peryx-search is the package index, built on Tantivy (a full-text search library, the Rust counterpart to Lucene). peryx-events carries Prometheus-format metrics, security events, and webhooks (an outbound HTTP callback fired when something changes); peryx-policy the neutral allow/deny engine; peryx-upstream the neutral upstream transport — conditional GET, retry, and range and streaming fetch — that each ecosystem's protocol layer builds on to reach the real upstream index peryx proxies (such as pypi.org or Docker Hub); peryx-identity the upload-token check.

The seam. peryx-driver defines what an ecosystem plugs into. The word seam here is the software-design sense: a place where you can change behavior by substituting a component rather than editing in place. The formal name for this shape is a Service Provider Interface: the host defines an interface, and providers (the ecosystems) implement it. That interface is the EcosystemDriver trait (a trait is Rust's version of an interface: a set of methods a type promises to provide). A mount declaration says where a driver's protocol lives in the URL space; the process state carries the stores and caches a driver serves from; a standalone driver registry lets the binary's build and admin paths reach a driver without spinning up the full serving state. Everything a driver needs to serve a request lives here, and nothing about which ecosystems are installed.

The ecosystems. peryx-ecosystem-pypi and peryx-ecosystem-oci each implement one EcosystemDriver. They read the foundation crates through the seam and hold every format-specific decision: PyPI's Simple repository API and OCI's distribution spec, wheel (Python's built-package format, standardized in a PEP, a Python Enhancement Proposal) and manifest (an image's list of layers) parsing, the artifact rules each format layers on top of neutral policy.

The hosts. peryx-http is the HTTP server, built on axum (a web framework) and tokio (the async runtime that drives concurrent I/O without a thread per request). It resolves a request to a configured index and hands it to that index's driver; it names no ecosystem. peryx-web is the web UI, built on Leptos, a Rust UI framework that runs the same components in two places. On the server it does SSR (server-side rendering): it produces finished HTML so the first page load shows content without waiting on the browser. That HTML then needs hydration (the step where client-side code attaches event handlers to the already-rendered HTML so it becomes interactive), which runs as WebAssembly (Wasm, a portable binary instruction format browsers execute at near-native speed) compiled from the same Rust. peryx-web renders the neutral view models a driver produces.

The composition root. The peryx binary depends on everything, names the two ecosystems, and wires them in at startup. Composition root is the one place in a program that assembles the concrete pieces; keeping it single means the rest of the code names no ecosystem. This is the only crate that gets to know both formats at once.

Core concepts

Ecosystem. A closed set in peryx-core, one entry per packaging format. Each maps to a fixed slot, so a driver registry is a fixed-size array and dispatch is a static match rather than a runtime lookup. Static dispatch means the compiler resolves the call at build time; the alternative, dynamic dispatch through a trait object, resolves it at run time through a pointer. An ecosystem a request does not touch costs it nothing.

Role. How an index (a package repository) behaves: a cached role proxies an upstream, a hosted role accepts uploads, a virtual role merges other indexes under one route. Every ecosystem gets all three roles from peryx-index for free. The product (role × ecosystem) is the real unit of behavior: a cached PyPI index and a cached OCI index share the role engine and differ in wire protocol.

Index and resolution. An index pairs a route with its kind and compiled policy. The router resolves a request path to an index by longest-prefix match (the same rule an IP router uses: the most specific configured route that the path starts with wins). A virtual index walks its layers in configured order and merges their answers first-match, so an artifact in an earlier layer shadows a later one.

The driver interface and its mount. One trait carries the metadata every ecosystem declares — which ecosystem it serves, where it mounts, how to classify a route for rate limiting, how to compile its policy — and the request-serving behavior, split by mount. An indexed driver like PyPI serves the reads, uploads, and deletes the router routes to it after resolving the index. An absolute driver like OCI owns a fixed top-level prefix (the root the distribution spec mandates) and dispatches the whole request itself. Each driver implements only the half its mount uses.

The two-part process state. The state splits in two. The serving state holds the stores, caches, indexes, and background handles a driver needs; a driver receives it as a shared, atomically reference-counted pointer (the way Rust hands one heap value to many tasks safely). The application state wraps the serving state and adds the driver registry the router and rate limiter reach. Because a driver never receives the registry, it cannot reach another ecosystem's driver or enumerate them, and the compiler enforces that rather than a convention. Handlers still read the serving state directly through the application state.

The driver registry. A standalone set of drivers keyed by ecosystem, which the composition root builds once. The binary's config-build and admin commands never construct the full serving state, so they dispatch through this registry to compile an index's policy or run a per-ecosystem admin scan without naming a format.

Lexicon. Each ecosystem's user-facing vocabulary, which its driver registers at install time. A surface localizes a label from an index's ecosystem without the neutral core naming any format's words. PyPI calls a stored unit a project; OCI calls it a repository; the lexicon holds that mapping so shared code stays neutral.

The block protocol. peryx-web renders a page from a list of neutral presentation blocks, an open set of primitives keyed by shape (key-value, chips, links, groups) rather than by format. This is the same idea as a server-driven UI: the server decides what blocks to show, the client knows how to draw each block type. A driver turns its metadata into these blocks, so the UI gains an ecosystem's page without a web-crate change.

The serving path

The router resolves a request to an index and hands it to that index's driver, which reads through three neutral layers: the store that holds artifacts and metadata, the cache that serves a warm copy from memory, and the upstream client that fetches whatever is missing. The sections below take them in that order.

An absolute-mount ecosystem skips the index resolution: the router mounts a catch-all under each prefix the driver declares and hands it the whole request, which the driver resolves against the configured indexes itself.

The storage layer

peryx-storage is the only crate that touches disk. It keeps two stores side by side under one data directory, so a restart loses nothing and a backup is a directory copy. This layer and the two after it are each read the same way: first the neutral abstraction every ecosystem shares, then the extension each format builds on top of it.

Abstraction. Two primitives are genuinely format-neutral. The blob store holds artifacts as ordinary files under a content-addressed tree: each file is named for the SHA-256 of its bytes and sharded two levels deep (sha256/ab/cd/abcd…) so no directory holds millions of entries. A write streams to a temporary file and atomically renames it into place once the hash is known, so a reader never sees a half-written blob and two clients fetching the same artifact converge on one file. Because the name is the hash, identical bytes are stored once across every ecosystem, and a truncated or tampered file is detectable. The second neutral primitive is the shared key-value store, one redb space of opaque bytes the store never interprets: a driver owns its keys end to end, reads and writes values the store treats as blobs, and enumerates them by ordered prefix scan. redb is an embedded, transactional (ACID) key-value store with one writer and MVCC readers that never block it, so a fetch reads consistent state while a publish commits, and each write is one transaction.

A driver rarely writes one entry at a time. The store applies a batch of opaque entries in a single transaction — the atomicity a cached page or a publish needs — and a journaled variant allocates the store's monotonic serial and records an opaque changelog entry in the same commit, so a publish's entries, its serial, and its journal entry land together or not at all. Nothing here is format-specific: the store holds content-addressed blobs, the shared key-value store, the serial counter, and the durable webhook queue, and knows no ecosystem's schema.

Extension. Both ecosystems build their whole model on the shared key-value store, each under its own private namespace, so the store never grows a table per format. PyPI keeps cached Simple API pages, observed projects and their PEP 700 yank/hide status, hosted uploads and their overrides, PEP 658 metadata siblings, and the upstream URLs a cold blob refetches from — committing a freshly fetched page in one batch, and a publish's entries, serial, and replication journal entry atomically. OCI keeps manifests (byte-for-byte, so their digest stays stable), tags, cached tag-list pages, tag freshness, and the referrers graph. Both keep only artifact bytes in the shared blob store, where an OCI layer and a Python wheel dedupe against the same content-addressed tree. The value encodings and key layout are the driver's own; the store never reads them, so a third ecosystem adds its model without a storage change.

The blob store and the shared key-value store in blue are neutral; the green drivers own the entries they lay into the key-value store. Both drivers reach the blob store, and each keeps its own model in its own namespace of the key-value store without the store ever reading it.

The caching layer

peryx-index owns the coordination that lets a cold cache serve at upstream wire speed and a warm one from memory.

Abstraction. One ServingCache, shared by every ecosystem, carries five mechanisms:

• Single-flight. A per-key coalescing map: when many clients request the same uncached artifact at once, one fetch runs upstream and the rest await it instead of each starting its own download. The name comes from Go's singleflight package; the effect is protection against a thundering herd.
• Stale-on-error. A bounded staleness window that lets a proxy serve the last good page when the upstream is unreachable, following RFC 5861's stale-if-error (a close cousin of stale-while-revalidate). The bound is an operator's explicit choice, so a lasting outage surfaces as an error rather than as quietly ancient data.
• The hot page cache. A parsed, rewritten index page kept in memory in a moka cache bounded by a byte budget, so a warm request skips re-parsing. Every entry is re-derivable from the stored raw page, so evicting one costs hit rate, never correctness.
• The negative cache. Known-absent keys with a short expiry, so a flood of requests for a package that does not exist does not become a flood of upstream 404s.
• The mutation epoch. An atomic counter advanced on every write. Derived state (the search index, hot page entries) records the epoch it was built at and rebuilds when the counter moves, so a publish becomes visible without invalidating each cache by hand.

Extension. The two ecosystems use different slices of that one cache. PyPI uses all five: it keys the hot page cache by route, project, rendering, and epoch together, so the PEP 691 JSON, PEP 503 HTML, and legacy-JSON renderings of a page share a raw fetch yet cache separately, and one epoch advance on a yank or upload retires every rendering at once. It reads the negative cache on a miss and coordinates each upstream fetch through single-flight. OCI uses only the two primitives that carry no assumption about an in-memory page: single-flight (keyed by blob digest or manifest reference) and the stale bound. It caches nothing in the hot page or negative caches and never advances the epoch; its proxy cache lives in the shared key-value store as cached tag pages and their freshness, and it gates freshness by comparing its own stored fetch time through the same shared staleness window.

Blue marks the two primitives both ecosystems share; green marks the in-memory caches only PyPI uses; orange marks the persistent key-value store OCI reaches for instead.

The upstream layer

peryx-upstream is how a cached role reaches the real index it proxies.

Abstraction. The upstream client is a neutral HTTP client that names no format. It fetches a resource conditionally: it sends the stored ETag as If-None-Match and returns the raw status, validator, serial, and freshness lifetime to the caller rather than acting on them, so a 304 Not Modified or a 404 is a value the caching layer interprets, not a branch buried in the client. A transient failure retries up to twice (three attempts total) on a 5xx, 408, or 429, and on a timeout or connection error, with exponential backoff and jitter (100 ms base, 2 s cap). Bounding how many fetches run at once lives one layer up, in the driver's upstream limiter: an optional per-index semaphore, off by default, that caps simultaneous upstream requests and applies back-pressure. A waiter blocks up to 30 s, then gets a retryable rate-limit error.

Extension. Both ecosystems fetch through the same client and the same limiter; the format-specific part is only what they request and how they read the reply. PyPI fetches Simple-index pages (conditional on the index's etag and serial) and wheels; OCI fetches manifests, blobs, and tag lists. Neither the client nor the limiter names a format.

Cross-cutting concerns

Three more layers run alongside the serving path rather than on it: full-text search across every index, the policy engine that gates what may be served or published, and the events that make the server observable. Each stays neutral, and each ecosystem extends it through the same seam.

The search layer

peryx-search answers a substring query across every index in the process.

Abstraction. It is full-text search on Tantivy (a Rust search engine, the counterpart to Lucene) over a fixed neutral schema — display name, normalized name, route, ecosystem, summary, and a free-text field — tokenized with an n-gram tokenizer so a fragment matches. The index records the mutation epoch it was built at; when a write moves the epoch, it does a full wipe-and-rebuild (delete every document, re-add, commit, reload) rather than an incremental update, which keeps the indexing seam a single pure function of the current records. A query intersects the n-gram terms (or runs a regex for a re: or very short query) and returns results ordered alphabetically by a composite sort key — not by relevance score, a deliberate choice for a package index where the exact name is what a user wants.

Extension. Each ecosystem registers an indexer that maps its records into the neutral search document; a composite indexer concatenates every ecosystem's documents into the one index. PyPI turns its projects into documents, OCI its repositories. The schema and the query path never name a format; a driver owns only the record-to-document mapping.

The policy layer

peryx-policy decides whether an artifact may be served, cached, or uploaded. It is the cleanest example of the abstraction-plus-extension shape: a neutral engine that an ecosystem feeds typed rules.

Abstraction. A compiled policy carries format-agnostic controls (allow and block project lists with PEP 503-normalized keys, a maximum file size, a maximum project size) plus an ordered list of artifact rules (trait objects) the ecosystem supplies. Evaluation is first-match: it checks the project lists (allow-list before block-list), then file size, then each rule in order, and the first denial wins. A denial carries the action, the offending project, stable rule and field identifiers, and a human reason, which maps to 403 Forbidden with the serialized denial as the JSON body.

Extension. Policy compilation is the seam. The binary splits an index's policy table: neutral keys go to the neutral engine, and whatever remains goes to the driver. The default driver accepts no extra keys, so an unknown field is an error rather than a silent no-op. PyPI compiles its leftover keys into artifact rules: a version-specifier rule, a package-type (sdist versus wheel) rule, and Python-tag and platform-tag wheel rules. OCI adds none today and runs on the neutral controls alone. A new format contributes rules without the engine ever learning its vocabulary.

The events layer

peryx-events carries three streams, all format-neutral.

Abstraction. Metrics: request-path code emits an event over an mpsc channel to a dedicated aggregator thread, off the hot path, rendered at /metrics as Prometheus counters and gauges (no histograms). Security events: structured tracing records tagged as security events (an auth failure, a rate-limit denial) for an audit sink. Webhooks: durable, signed outbound callbacks. A change persists a delivery record to redb before any network call, so a crash never drops one (at-least-once delivery); a single background worker drains due deliveries in batches, POSTs with an HMAC-SHA256 signature plus event, delivery, and timestamp headers, and on failure reschedules with exponential backoff (5 s, tripling each attempt, capped at 300 s) up to five attempts before marking the delivery failed.

Extension. This layer stays neutral by construction: an ecosystem emits an event through the shared API rather than defining its own stream. A publish or a yank from either driver becomes the same metric increment and the same webhook payload shape; the event names a project or a repository as data, never as a type the events crate knows.

Adding an ecosystem

The seam turns a new format into a bounded checklist rather than a server change.

1. Create a peryx-ecosystem-<name> crate that depends on peryx-driver and the foundation crates it needs.
2. Add the format to the set of packaging formats in peryx-core, which sizes the driver registries.
3. Implement the driver interface: declare where it mounts, classify routes for rate limiting, and serve the half your mount uses. Turn cached metadata into presentation blocks for the web UI and compile artifact rules from the index's policy table.
4. Expose an install entry point that registers the driver, its search indexer, and its vocabulary.
5. Implement the admin operations your format needs (blob-reference scanning, fsck as a filesystem-check-style consistency scan of the stored records, import, purge), which the binary's maintenance commands dispatch through the driver.
6. Wire the crate into the peryx binary: install it at startup and add the driver to the driver registry.

Nothing in peryx-http, peryx-web, or the other ecosystem changes.