Skip to main content

Gentian OS — Platform Architecture

An overview of the Gentian OS architecture. Each section says what exists and links to the document that holds the detail.


1. What Gentian OS Is​

Gentian OS is a cloud-native operating system for open-source business applications. It runs on Kubernetes and exposes the same "install / uninstall / use" experience to organizations that a desktop OS exposes to a single user — except the "user" is a tenant (organization) and the "apps" are full multi-user products like Nextcloud, OpenProject, Element, or XWiki.

The design optimizes for two things:

  1. Onboarding a new application is one catalog entry: a ComponentProfile and the few objects that travel with it.
  2. Onboarding a new tenant is a single declarative resource (Tenant) that triggers the entire provisioning pipeline.

For the rationale — why this gap exists in the open-source landscape and why a kernel-style abstraction is the right answer — see why-a-cloud-os.md.


2. The OS Analogy​

Gentian OS is structured like a traditional operating system, with direct analogues for every layer:

Traditional OSGentian OS
Syscall API (open, socket, fork)CRDs: Tenant, ComponentProfile, Component, IntegrationBinding
libc — friendly call → raw syscallsCrossplane Compositions
Syscall dispatcher / VFSCrossplane Composition engine
Loadable kernel modules / device driversCrossplane providers (provider-helm, provider-vault, provider-kubernetes, provider-keycloak)
Hardware (disks, NICs)External operators & APIs (Keycloak, CloudNativePG, SeaweedFS, OpenBao, cloud APIs)
File descriptor / process handleManaged Resource (MR) status
Kernel scheduler / writebackCrossplane reconcile loop
init / systemdArgo CD
Default mounts (C:, /, ~/)Default-install kernel components (Keycloak, OpenFGA, OpenBao, PostgreSQL, SeaweedFS, Envoy Gateway, the desktop, …)

The reasoning behind each mapping is in kernel.md.


3. Architecture at a Glance​

Seen as layers, from what people use down to what runs it. gentian.org draws the same five with plain labels; here each carries its components.

Git says what a cluster is. Two processes of the platform stand on either side of it: the director is the only one that writes to the deployments repository, and the operator is the one that writes to the cluster. Argo CD applies what git holds; Crossplane composes what the operator asks for.

PartRoleBoundary
Argo CD (kernel-gitops)Pulls git and applies it: the kernel's charts from gentian-os, and from the deployments repository the cluster's claims, its materialized catalog and its tenants. Syncs only commits signed by the director or the break-glass key.Provisions nothing itself.
Director (kernel-control)The API people's requests go to. It verifies the caller's token, asks OpenFGA whether that person may make the change, and commits it to the deployments repository as that person.Holds the git push credential and no cluster credential.
Operator (kernel-control)Reconciles Tenant, Component and the other kinds of §4: seeds secrets in OpenBao, writes the tenant's composite, renders and applies what each app's chart is installed with, routes, policies, and status. Projects tenants, installs and group memberships into OpenFGA.Writes to the cluster only; holds no git credential.
Crossplane (kernel-provisioning)Composes XCluster and XTenant into managed resources (namespaces, vault policies, Jobs, ExternalSecrets), and an App claim for a profile that brings its own Composition. Its providers reconcile the managed resources the operator renders for an app, Helm Releases among them.Owns what can be written down before it happens (below).
OpenBao + ESO (kernel-secrets)The one secret store, synced into Kubernetes Secrets that charts read.Secrets never touch git or a resource's spec.
Usher, custodian, registrar (kernel-control)The other three services people's requests reach: the usher answers reads (what is here, what may I open), the custodian sets credentials in the vault, the registrar manages people and groups at Keycloak. Each asks OpenFGA first and holds one credential only.None of them writes git.
Arbiter (system-authorization, on demand)The one way into the app instance of OpenFGA, which holds the components' authorization stores. The operator deploys it when a component declares a store.Serves the app stores only; the kernel's rights store is separate.
Bouncer (kernel-edge, beside the Gateways)Asks OpenFGA, per request, whether the signed-in person may reach the host.Decides nothing else (routing.md §4.1).

3.0 Who does what (tenant install)​

OwnerDoesDoes not do
DirectorDecide whether the caller may; commit the tenant's manifest, an app's entry in it, a materialized profileTouch the cluster
Argo CDApply the kernel's charts, the claims, the catalog directory and each tenant's directoryRun provisioning logic; create a Helm release per tenant app
OperatorSeed secrets; write XTenant; create a Component per installed app; render and apply each Component's Helm release and what it reads (an App claim instead for a profile that brings its own Composition); routes, network policies, statusCreate what a Composition creates
CrossplaneReconcile XTenant, the managed resources the operator renders and any App claim into Jobs, Objects, ESO, provider-helm ReleasesSync git

How soon Argo CD sees a commit. Argo CD reads the deployments repository every few minutes. So that a change does not wait for that, the director tells the operator after each push, and the operator asks Argo CD to read the repository now: it marks the ApplicationSet that finds the tenants' directories and every Application that reads the repository, with Argo CD's own refresh annotations. This is a refresh and not a sync. Each Application still syncs under its own policy, and Argo CD still checks the commit's signature. If the operator cannot be told, the commit is in git all the same and Argo CD finds it at its next read.

Where the imperative/declarative line falls. The matrix above says who does what. The rule behind it is one question:

Can the answer be written down before it happens?

If it can, it is a statement about what should exist and Crossplane owns it — namespace shell and policy via provider-kubernetes, realms, clients, groups, identity providers and authentication flows via provider-keycloak, policies via provider-vault, charts via provider-helm. An object already existing is not a reason to keep it imperative: Crossplane adopts by crossplane.io/external-name.

If it cannot, the operator owns it, and only for four reasons:

  1. Discovery — enumerating external state and acting per item found. Keycloak's current users are in no spec, so a credential minted per user cannot be rendered from one.
  2. Computation — producing a value rather than restating one (rsa.GenerateKey, hmac.New, argon2.IDKey). Compositions template; they do not compute.
  3. Adoption gaps — where a provider cannot safely take over an object that already exists. provider-vault's jwt AuthBackend is the standing example; see scripts/steps/B-09-vault-oidc-mount.sh.
  4. Change-triggered action — "restart when this changes" is a moment, not a thing.

Observing Crossplane's work and aggregating it into Tenant.status is not a fifth reason; it is the operator being a controller.

This is a boundary, not a description of how far a migration got. Where the two disagree, the boundary is right and the code has not caught up — a Job that survives is only correct if one of the four reasons above names it. One place where the code is behind is stated in the code itself: the operator still seeds an app's secrets and its model-gateway key before the app's release can resolve them (internal/controller/app_reconciler.go).

An app's managed resources are written down too, by the operator rather than by a Composition: it renders them from the profile (internal/apprender) and applies them (internal/controller/component_render.go), and the providers reconcile them like any other.

The same rule applies to installer steps. scripts/steps/* is the same question asked at bootstrap time instead of tenant-onboarding time: a step that only applies a manifest belongs in an ApplicationSet (git is where the answer lives), a step that calls a running service's admin API is the operator's four reasons above, and a step that only guards or validates stays a step.

Why two tools, not one: Argo CD's drift detection, UI, and rollback work for every Kubernetes resource, not just managed resources. Crossplane's reconcile loop handles the slow, eventually-consistent external APIs that Argo CD cannot reason about.

3.1 How provisioning works on the cluster​

A fresh install leaves the platform tenant (Tenant/platform, whose realm is the kernel realm) and, on a single-tenancy cluster, the one user tenant user. Every other tenant is created afterwards, through the director.

What Argo CD syncs

FromWhatBy
gentian-osthe bootstrap Applications (Reloader, CNPG, Kyverno, Headlamp, OpenBao and its seal, the kernel Postgres) and the platform's own chart, charts/gentian-os (operator, director, usher, custodian, registrar, bouncer)kernel/bootstrap/chart, applied by the installer
gentian-osthe system tier's data plane, identity (Keycloak, OpenFGA), and — where the claim asks — mail and the model gatewaythe ApplicationSets of kernel/appsets/raw/
deployments repositoryclusters/<cluster>/kernel/claims/ — the cluster's Crossplane claimsgentian-claims
deployments repositoryclusters/<cluster>/catalogue/ — the profiles tenants have installedgentian-catalogue
deployments repositoryclusters/<cluster>/tenants/<tenant>/ — one directory per tenantgentian-tenants

There is no Argo CD Application per tenant app.

Tenant lifecycle. Once a Tenant is applied the operator runs its stages (internal/controller/tenant_reconcile_stages.go):

  1. Bootstrap — seed the tenant's credentials in OpenBao; write the provisioning manifests (tenant-<name>-provisioning-jobs) and the XTenant composite, which Crossplane's tenant-default turns into the namespaces tenant-<name> and tenant-<name>-dmz, the vault policy and the provisioning Jobs; network policies, registry credential, CA trust.
  2. Data plane — the tenant's Keycloak realm, then databases, object storage and cache for the apps that declare them.
  3. Apps and edge — one Component per entry of spec.apps and per component the platform places on every tenant (the desktop, the administration console, the App Store app where a store is offered); privileges, drop-ins, model-gateway keys; routes and their policies.
  4. Integrations — IntegrationBindings and app grants.
  5. Shared services — mail registration and the model gateway's team, where the cluster runs them.
  6. Status — per-step conditions on the Tenant; Ready needs both the operator's steps and the composite.

App install flow. A person asks the director (kubectl gentian apps install, the App Store app). The director fetches the profile's bundle from a catalog — an https address serving an index and one bundle per app — checks it against the digest the install names, and commits two things: the bundle under clusters/<cluster>/catalogue/ and the app's entry in the tenant's spec.apps. Argo CD applies both; the operator creates the Component, and the component reconciler renders and applies the app's provider-helm Release with the ExternalSecret its values read, its client at the tenant's realm and its post-install Job. Nothing copies a catalog into a cluster ahead of an install. See custom-catalogs.md and store-contract.md.

Identity. Realms, clients, groups and brokering are described in iam.md §1.8–1.9.

A profile whose app needs more than the operator renders brings a Composition of its own in its bundle and names it in spec.package.composition; the Component then writes an App claim that Composition answers. Only a profile of trustTier: platform may.

Placeholder semantics (${TENANT_DOMAIN} vs ${KERNEL_DOMAIN}) are documented in gentian-apps/docs/app-profile-guide.md §4.3.

3.2 Diffing: server-side​

Argo CD runs with server-side diff (controller.diff.server.side, set by installer step A-06-argocd). It asks the API server what a manifest would become — a dry-run apply — and compares that against the live object, rather than comparing the YAML in Git against the live object directly.

The question it answers is therefore "would syncing change anything", not "does the file match the object". Fields the platform never wrote are not differences: CRD defaults, and the mutations Kyverno applies, appear on both sides and cancel.

This is not a preference. Without it a CRD's own defaults read as drift — an ExternalSecret declaring a key comes back with five more fields set, a CNPG Cluster declaring seven comes back with forty-three — and applications sit permanently OutOfSync while entirely healthy. The alternative, listing the defaulted paths per CRD, covers less after each upstream release without saying so.

Two consequences worth knowing:

  • A permanently OutOfSync application is a real finding. That is the point of removing the false ones.
  • A mutating webhook rewriting a field is invisible here, by design, because the dry-run applies the same webhook. If that ever needs auditing, it is a question for the admission side, not for the diff.

4. The Kinds​

Four kinds carry the model. The rest are listed at the end of the section.

4.1 ComponentProfile (cluster-scoped) — the catalog entry​

Declares what a component is: whether it is an app, a shared app or a service (classes), how a person opens it (launch), its trust tier, how it is delivered (package: a chart, a Composition, an API integration or an addon, with a typed valueMapping that says where the chart takes platform-provided values), what it needs from the platform (requires.services: identity, database, storage, cache, mail, models; requires.privileges), what it offers and consumes (provides, integrations), and where it answers (expose, each entry with a surface and a mandatory authMode, and optionally a tile for the desktop). Nothing in a profile grants anything: every permissive statement is a request that a named person answers. A profile reaches a cluster only when a tenant installs it (§3.1). The types are api/v1alpha1/componentprofile_types.go and profile_parts.go.

4.2 Tenant (cluster-scoped) — the customer​

Declares who uses the platform: isolation, quotas, mail mode, deletion policy, the catalogs only this tenant sees, and spec.apps — the profiles installed for it, each optionally pinned to a digest, with its addons. It also records what was approved for the tenant: public addresses (spec.exposures) and privileges (spec.privileges). The manifest lives in the deployments repository and the director writes it. Creating a Tenant provisions its namespaces, vault policy, DNS and TLS, and Keycloak realm. A custom domain is a TenantDomain beside the tenant, not a Tenant field.

4.3 Component (namespace-scoped) — one installed instance​

One per installed app and per component the platform places, in the tenant's namespace, created by the operator. It names its profile (and the digest the install pinned), and it is what installs the app: the operator renders its Helm release and what the release reads, or, for a profile that brings its own Composition, writes a Crossplane App claim that Composition answers. Its conditions say whether the app is ready and what it is waiting for (an unapproved privilege, a reserved address, a cluster without a model gateway). The App claim is an implementation detail of delivery; nobody writes one by hand.

4.4 IntegrationBinding (namespace-scoped) — the cross-app contract​

Where two apps installed in one tenant declare the two sides of a contract (provides / integrations), the platform creates an IntegrationBinding: the operator derives it and the tenant's composite carries it. It is a request: the tenant's administrator grants it for the consumer through the director, and only then are the two network policies written that let the consumer's pods reach the provider's. The granted capabilities are recorded and not enforced by the platform, and the provider is not told which app is calling (app-customization.md §2.3).

Other kinds. Cluster (the Crossplane claim that says what a cluster is: domain, tenancy mode, mail, models, catalogs), Customization (app-customization.md §5), AppPackage, AppGrant, Branding, ResourcePlan, TenantDomain, BackupPolicy, TenantExport, TenantRestore, MailboxRemoval, CredentialRequirement, OIDCPackCatalog, PlatformSecurityPolicy, BundleSource, ProfileBuild. Profile fields, tiers and contracts are in app-catalog.md.


5. The Kernel and the Default Install​

Like a desktop OS, Gentian OS ships with a default install — components that must exist before any tenant app can run. Namespaces are named by tier and function; the list is kernel/namespaces.yaml.

FunctionComponentNamespace
GitOpsArgo CDkernel-gitops
ProvisioningCrossplane, its providers and functionskernel-provisioning
SecretsOpenBao, External Secrets Operator, Reloader; the transit seal apartkernel-secrets, kernel-seal
AuthenticationKeycloakkernel-authentication
AuthorizationOpenFGAkernel-authorization
Kernel dataCloudNativePG operator and the kernel's own Postgreskernel-data
Controloperator, director, usher, custodian, registrarkernel-control
EdgeEnvoy Gateway and the two Gateways, the bouncer, cert-manager, external-dnskernel-edge
AdmissionKyverno and the baseline policieskernel-admission
Cluster viewHeadlampkernel-observability
Relational data for tenantsPostgreSQL (CloudNativePG), MariaDBsystem-postgresql, system-mariadb
CacheRedissystem-cache
Object storageSeaweedFSsystem-s3
Mail (where the claim asks)Postfix and Dovecot; the mail edgesystem-mail, system-mail-dmz
Models (where the claim asks)LiteLLM and vLLMsystem-llm

Kernel namespaces are created by the installer; system namespaces are composed from the Cluster claim; each tenant gets tenant-<t> and tenant-<t>-dmz.

The user interfaces are components, not kernel services. The desktop (desktop.<tenant domain>), the administration console (admin.), the App Store app (store.) and the sign-in page (the concierge) are built in gentian-ui, described by ComponentProfiles this repository's chart ships, and installed into tenant namespaces by the operator like any other component. The platform is itself a tenant: its desktop answers at platform.<kernel domain> and shows platform administrators the kernel's consoles.

Catalog apps (Nextcloud, Element, …) are installed per tenant from a catalog (§3.1). See kernel.md for the kernel functions and the kernel-versus-catalog split.


6. Multi-Tenancy, Domains and Security​

Multiple tenants share one cluster:

  • Isolation is one Kubernetes namespace per tenant (tenant-<name>) and a second for what it publishes without sign-in (tenant-<name>-dmz), with NetworkPolicies, ResourceQuotas and LimitRanges. Identity, data and mail are separated by a Keycloak realm per tenant, per-app database users, bucket policies, and per-domain DKIM keys.
  • Domains. The cluster's wildcard (*.<kernelDomain>) covers the kernel's own hosts. Each tenant has its own zone: <tenant>.<kernelDomain> under tenancyMode: multi; under single the one user tenant, user, answers directly on <kernelDomain>. The platform tenant is platform.<kernelDomain> under either mode. A custom domain is a TenantDomain. See routing.md §3 and multi-tenancy.md §3.
  • The edge is the only session authority. One sign-in client per tenant zone at the gateway; behind it every request passes the bouncer, which asks OpenFGA whether this person may use this app. Apps receive identity headers, not the edge's token (routing.md §4).
  • Publishing without sign-in is a surface: perimeter entry, served by a proxy in the tenant's DMZ namespace only after the tenant's perimeter approver approved it (app-customization.md §2.10).
  • Database isolation: each app within each tenant gets its own database user with grants limited to its own database.

The nine rules everything above follows are in security/principles.md; the detail is in multi-tenancy.md and security/design.md.

6.1 TLS certificate provisioning​

For each tenant the operator ensures:

  1. One cert-manager Certificate for *.<effectiveDomain> (DNS-01), stored as tenant-<name>-wildcard-tls in the tenant's namespace. The bare domain is deliberately not in it (routing.md §3).
  2. One Gateway API HTTPRoute per host an app's expose entries answer on (<subDomain>.<effectiveDomain>), attached to the kernel's Gateways in kernel-edge, whose listeners read the tenant's certificate across namespaces under a ReferenceGrant.

effectiveDomain is the custom domain a TenantDomain binds, when there is one; otherwise it follows the tenancy mode (§6). The issuer is configured cluster-wide via TENANT_DNS01_CLUSTER_ISSUER (Helm: tenantDNS01ClusterIssuer). A profile has no field that selects an issuer.

The kernel wildcard (*.<kernelDomain>, DNS-01 at install) covers the platform's own hostnames (platform, id, argocd, headlamp, …).

When traffic is proxied through Cloudflare, an optional operator adapter maintains the tenant's wildcard DNS record so the provider can issue edge certificates for multi-level hostnames. See multi-tenancy.md §3.

6.2 CORS and iframe embedding​

The desktop opens apps in frames, so the operator sets the frame policy on every route it creates, at the edge and not in the app: it removes upstream X-Frame-Options and sets Content-Security-Policy: frame-ancestors 'self' plus, by name, the desktop of the component's own tenant and the component's own other hosts. No wildcard is used, and the platform's desktop frames no tenant's app. Keycloak's endpoints on id.<kernelDomain> carry a separate policy naming the hosts whose pages embed its session frames. A frame only works where desktop and app are on the same site, because the session cookie is SameSite=Lax. The rules and the table of who may frame what are in routing.md §4.3.


7. Secrets and Credentials​

All secrets live in OpenBao and are synced into Kubernetes Secrets by External Secrets Operator (ESO). Helm charts consume them via existingSecret references; for charts that lack existingSecret support, provider-helm injects values via valuesFrom. Either way, secrets never appear in git or in a resource's spec.

Three properties:

  1. Generated secrets are derived by default. With the Cluster claim's secretMode: derived the kernel's and each app's are computed from the master password (HKDF-SHA256 in internal/kernel/secrets for an app's), so the same password and salt reproduce them; random makes them independent of it and leaves OpenBao holding the only copy. An app's own secrets follow the mode too, and travel in the tenant's bundle with the data that was written with them.
  2. No person's password is derived. An administrator account has no password until its holder sets one through a single-use activation link.
  3. Seeding is write-once. The operator writes a credential where the path is empty and does not overwrite a live one.

A credential a person supplies (a DNS token, a registry login) is set through the custodian, which can write a secret and cannot read one.

7.1 Two Secret Delivery Patterns​

PatternMechanismWhen to use
AESO syncs OpenBao → Secret; the chart references it via existingSecretCharts with native existingSecret support, which covers the kernel's services.
BESO syncs OpenBao → Secret; provider-helm valuesFrom maps individual keys to Helm value pathsCharts that accept secrets only as plain values. The profile's package.valueMapping says which path takes which key.

7.2 OpenBao Bootstrap​

OpenBao must be initialized before ESO or Crossplane can authenticate to it. The installer does this once, in its B phase: it initializes the transit seal and the vault (B-02, B-03), creates the KV mount and Crossplane's policy and token (B-04), and seeds the kernel's paths (B-07, B-08). Everything after that — Kubernetes auth, roles, the kernel's ESO ClusterSecretStore, and for each tenant a policy, a role and a store of its own — is composed from the Cluster claim and the tenant composites through provider-vault.

The path layout, the derivation, rotation and the recovery kit are in security/design.md §4–6 and reference/install.md §4.


8. Repository Structure​

gentian-os/ # The OS itself
├── api/ # The resource types (a Go module of its own)
├── cmd/, internal/ # Operator, director, usher, custodian, registrar, bouncer, arbiter
├── charts/gentian-os/ # The chart that installs them, with the CRDs
├── crossplane/ # XRDs, Compositions, providers
├── kernel/ # What Argo CD applies that is not composed
├── scripts/, install.sh # The installer and kubectl-gentian
└── docs/

gentian-apps/ # A catalog: profiles, charts, app sources
gentian-ui/ # The desktop, the consoles, the sign-in page

<deployments repository> # Per-cluster state; the only repository specific to a cluster
├── profiles/ # Values shared by the clusters of a stage
└── clusters/<cluster>/
├── kernel/ # claims/, values.yaml, signing/
├── catalogue/ # Profiles installed on this cluster, as the director committed them
└── tenants/<tenant>/tenant.yaml

The layout of this repository is in folder-structure.md; the deployments repository and how a cluster follows a release are in deployment.md.


9. The Mail Kernel Extension​

Mail is optional and decided twice.

Per cluster, the Cluster claim's mail.serviceMode says whether the cluster runs its own mail stack. With system, Postfix and Dovecot run in system-mail with no load balancer, and a proxy in system-mail-dmz takes ports 25, 587 and 993 from the internet and holds no mail, no user and no key. Otherwise the cluster relays through a provider and neither namespace exists. A cluster behind a tunnel cannot run its own stack.

Per tenant, Tenant.spec.mail.mode says what the operator registers:

  • selfhosted — a domain and mailboxes on the cluster's shared stack.
  • external — the tenant's own provider, with its credential.
  • transport-only — outbound through the cluster's relay, no mailboxes.
  • disabled — no mail.

Configuration, isolation and DNS records are in mail.md.


9b. Collabora (catalog app)​

Collaborative document editing is a catalog app, not a kernel service: it is installed with the file store that uses it, as part of that app's profile or as an addon of it. See app-catalog.md.


10. Backup, DR and Observability​

A tenant is backed up as a whole. A TenantExport captures one tenant's data — its apps' databases, buckets, volumes and vault paths, and its realm — into an encrypted bundle; a BackupPolicy names object storage other than the platform's own to write them to and the keys they are encrypted to; backups on a schedule are an add-on's, which creates the exports; a TenantRestore or kubectl gentian tenants import brings one back, on the same cluster or another. A profile's spec.backup says how its app is quiesced and what of it is captured. The cluster itself is rebuilt from git and its recovery kit.

State is read through the Kubernetes API: kubectl get tenants, kubectl get components -A, kubectl get integrationbindings -A, and crossplane beta trace on a composite show what was provisioned and what is waiting.

See data-lifecycle.md, operations.md, audit-events.md, tenant-backup.md and recovery.md.


11. Kernel and App Image Updates​

11.1 The platform's own image​

Operator, director, usher, custodian, registrar, bouncer and arbiter are one image and one chart, delivered by the gentian-os Argo CD Application (kernel/bootstrap/chart/templates/gentian-os.yaml). The installer pins the image to the build of the commit it is installing from (<branch>-<short-sha>, or the version for a release tag), so the manifests Argo CD syncs and the binary the kubelet pulls come from one commit.

Nothing in the cluster advances that pin. A cluster following a branch takes a newer build when the installer is run from a newer checkout (./install.sh --only B-01 for that alone); a cluster pins its own tag in clusters/<cluster>/kernel/values.yaml. argocd-image-updater is installed with Argo CD, and the platform ships no ImageUpdater resource: a cluster that wants an image to roll by itself writes one. See reference/install.md §4, "Image tags".

11.1.1 The user interfaces​

The desktop, the administration console, the App Store app and the concierge are charts published by gentian-ui. Their ComponentProfiles ship in this repository's chart, and the installer resolves the chart version each one names (PORTAL_IMAGE_TAG, step D-03). They are installed per tenant by the operator, not by an Argo CD Application of their own.

11.2 Install-time bootstrap​

install.sh drives the steps of scripts/steps/ in five phases: A the control plane (namespaces, cert-manager, ESO, Crossplane, Envoy Gateway, Argo CD), B secrets (the bootstrap Applications, the vault, Crossplane's providers and definitions, seeded secrets, the signing keys), C the platform (the Cluster claim, the ApplicationSets, the wildcard certificate), D the applications (operator and director, the kernel realm, the platform tenant and its desktop), E the handover (recovery kit, the bootstrap token revoked, the user tenant of a single-tenancy cluster). Step 0 writes the cluster's definition into the deployments repository first. See reference/install.md §1 and deployment.md §3.

11.3 App images​

An app's images are what its chart, at the version its profile names, pulls. An install is pinned to the digest of the profile's bundle, so an app moves to a newer build when it is installed again at that build (custom-catalogs.md).


12. The AI Layer​

A cluster may serve language models. Where the Cluster claim enables it, LiteLLM runs in system-llm as the model gateway, in front of the vLLM instances the claim lists. An app that declares requires.services.llm is given a key of its own at the gateway, the gateway's address and a network path to it; no other app can reach it. The gateway has no public route; its console is a claim setting, for platform administrators, behind the kernel sign-in.

See llms.md, agentic-ai.md and security/design.md §2.9.


13. Operational Roles​

A role is membership of a Keycloak group; what a role may do is a relation in OpenFGA, asked by the director, the custodian, the registrar, the usher and the bouncer before they act. Nobody is given Kubernetes RBAC on the platform's kinds to do their work.

RoleScopeDoes it throughCannot do
Platform administratorThe cluster and the platform tenantkubectl gentian, the platform desktop's consoles; creates tenants, declares catalogs, sets plansOpen a tenant's apps
Tenant administratorOne tenantThe administration console and the App Store app: people, groups, installs, grantsUse the tenant's apps as a member; touch another tenant or the kernel
Perimeter approverOne tenantApproves what the tenant publishes without sign-inAnything else
MemberOne tenantThe desktop: the apps they were givenInstall, administer

Administrator and member are separate accounts by design. Roles, groups and the relations behind them are in iam.md §1.2–1.3 and multi-tenancy.md.


14. Why This Architecture Scales​

  • Adding an app to a catalog = one profile bundle. No code and no Composition change for a typical app; the operator renders it from the profile.
  • Installing an app for a tenant = one request to the director, which becomes one entry in Tenant.spec.apps. Tenant administrators do this themselves.
  • Adding a tenant = one Tenant. The operator and Crossplane reconcile kernel and app resources in parallel where dependencies allow.
  • Adding a cluster = one directory in a deployments repository and one run of the installer. The same Compositions serve every cluster; differences are the claim and a values file.
  • Adding a kernel capability = one provider.
  • Queryable. The platform's state is in git and in the Kubernetes API.

15. Document Map​

Every document, by section and audience, is listed in index.md.