Case Reference: my-diet

A real-world example of a full App Platform deployment — from AppPlatformRegistration to four namespaces with SOPS secrets, a home page, and preview environments.

Application Overview

Property	Value
App	my-diet (meal planning application)
Repo	labrats-work/apps.my-diet
Stack	Next.js 16, MongoDB, Redis
Namespaces	my-diet-platform, my-diet-splash, my-diet-dev, my-diet-prod
Secrets	SOPS-encrypted in platform/ directory of platform repo
Image	ghcr.io/labrats-work/apps.my-diet:<version>

Architecture

AppPlatformRegistration/my-diet (cluster-scoped)
  │
  ├─ creates GitHub repo: my-diet-platform
  ├─ generates deploy key + AGE encryption key
  ├─ commits Flux config to flux repo (apps/my-diet-platform/)
  ├─ commits .sops.yaml to my-diet-platform repo
  └─ enforces RBAC in labeled namespaces
     │
     Flux syncs apps/my-diet-platform/ from flux repo:
       ├─ Secret: my-diet-deploy-key    (decrypted into my-diet-platform)
       ├─ Secret: my-diet-platform-age-key (decrypted into my-diet-platform)
       └─ GitRepository: my-diet-platform  (points to my-diet-platform repo)
            │
            Flux syncs my-diet-platform repo (Kustomization manifests):
              ├─ Kustomization/my-diet-platform → my-diet-platform ns (SOPS secrets)
              ├─ Kustomization/my-diet-home     → my-diet-splash ns   (home page)
              ├─ Kustomization/my-diet-dev      → my-diet-dev ns      (full app)
              └─ Kustomization/my-diet-prod     → my-diet-prod ns     (full app)

The platform repo contains Flux Kustomization manifests for each environment and platform secrets. The AppPlatformRegistration operator creates namespaces, RBAC, OIDC client, and Flux plumbing — the app team controls what gets deployed and where.

Every environment — including the home page — gets its own namespace. The my-diet-platform namespace holds platform secrets and RBAC. No workloads run there.

Namespace Layout

Namespace	Role Label	Purpose	K8s Path	URL
my-diet-platform	platform	Platform secrets, RBAC	./platform	—
my-diet-splash	app	Home page (nginx)	./k8s/home	my-diet.hcl.labrats.work
my-diet-dev	app	Development environment	./k8s/overlays/dev	app.my-diet-dev.hcl.labrats.work
my-diet-prod	app	Production environment	./k8s/overlays/production	app.my-diet.hcl.labrats.work

All app namespaces carry the labels labrats.work/app: my-diet and labrats.work/namespace-role: app. The platform namespace has labrats.work/namespace-role: platform — it is hidden from the dashboard and protected from deletion.

CRD Manifests

AppPlatformRegistration

apiVersion: labrats.work/v1alpha1
kind: AppPlatformRegistration
metadata:
  name: my-diet
spec:
  displayName: "MyDiet"
  access:
    - group: mydiet-ops
      role: admin
    - group: mydiet-developer
      role: developer
    - group: mydiet-viewer
      role: reader

OIDC Client

OIDC clients are managed by the OidcClientReconciler, which watches labeled Secrets (labrats.work/oidc-client: <client-id>) in the <app>-platform namespace. For each Secret the operator:

• Generates a random clientSecret and writes it back into the Secret
• Registers the OIDC client in Authelia's config via the flux repo
• Cleans up on Secret deletion (finalizer-driven)

Platform Repo: my-diet-platform

The platform repo is created automatically by AppPlatformRegistration. It contains Flux Kustomization manifests and platform secrets:

my-diet-platform/
├── .sops.yaml                         # Auto-committed: per-platform AGE public key
├── kustomization.yaml                 # Kustomize resource list
├── my-diet-platform.yaml              # Flux Kustomization → my-diet-platform ns (secrets)
├── my-diet-home.yaml                  # Flux Kustomization → my-diet-splash ns
├── my-diet-dev.yaml                   # Flux Kustomization → my-diet-dev ns
├── my-diet-prod.yaml                  # Flux Kustomization → my-diet-prod ns
└── platform/                          # SOPS-encrypted secrets + RBAC
    ├── kustomization.yaml
    ├── rbac.yaml
    ├── *.sops.yaml                    # Encrypted with per-platform AGE key

Kustomization manifests

All Flux Kustomizations live in the my-diet-platform namespace (not flux-system):

# my-diet-platform/my-diet-platform.yaml
apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
  name: my-diet-platform
  namespace: my-diet-platform
spec:
  interval: 5m
  path: ./platform
  targetNamespace: my-diet-platform
  prune: true
  sourceRef:
    kind: GitRepository
    name: my-diet-platform
  decryption:
    provider: sops
    secretRef:
      name: my-diet-platform-age-key   # per-platform AGE key (auto-provisioned)

# my-diet-platform/my-diet-home.yaml
apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
  name: my-diet-home
  namespace: my-diet-platform
spec:
  interval: 5m
  path: ./k8s/home
  targetNamespace: my-diet-splash
  prune: true
  wait: true
  sourceRef:
    kind: GitRepository
    name: my-diet-app
  postBuild:
    substitute:
      HOSTNAME: my-diet.hcl.labrats.work

# my-diet-platform/my-diet-dev.yaml
apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
  name: my-diet-dev
  namespace: my-diet-platform
spec:
  interval: 5m
  path: ./k8s/overlays/dev
  targetNamespace: my-diet-dev
  prune: true
  wait: true
  sourceRef:
    kind: GitRepository
    name: my-diet-app
  postBuild:
    substitute:
      HOSTNAME: app.my-diet-dev.hcl.labrats.work
      LOG_LEVEL: debug
      NODE_ENV: development
      OTEL_EXPORTER_OTLP_ENDPOINT: ""

Key points:

• Kustomizations are in my-diet-platform namespace — not flux-system
• platform uses sourceRef: my-diet-platform (the platform repo itself) with SOPS decryption using the per-platform AGE key
• home deploys the landing page into its own my-diet-splash namespace
• dev and prod use sourceRef: my-diet-app (the app source repo) with environment-specific variable substitutions
• The .sops.yaml at the repo root (auto-committed by the controller) enables sops --encrypt to work directly

Repository Structure

There are two repos: the app source repo (apps.my-diet) and the platform repo (my-diet-platform).

App Source Repo (apps.my-diet)

Contains the application code and K8s manifests:

apps.my-diet/
├── k8s/
│   ├── base/                          # Shared app manifests
│   │   ├── kustomization.yaml
│   │   ├── deployment.yaml
│   │   ├── service.yaml
│   │   ├── ingress.yaml               # Uses ${HOSTNAME}
│   │   ├── configmap.yaml
│   │   ├── secret-store.yaml          # SA + SecretStore → reads from my-diet-platform ns
│   │   ├── external-secret.yaml       # ghcr-pull-secret (ClusterSecretStore)
│   │   ├── external-secret-oidc.yaml  # App-specific (SecretStore)
│   │   └── ...
│   ├── overlays/
│   │   ├── dev/
│   │   │   └── kustomization.yaml     # namespace: my-diet-dev
│   │   ├── production/
│   │   │   └── kustomization.yaml     # namespace: my-diet-prod
│   │   └── preview/
│   │       └── kustomization.yaml     # namespace: my-diet-pr-N
│   └── home/                          # Home/landing page (nginx)
│       ├── kustomization.yaml
│       ├── deployment.yaml
│       ├── service.yaml
│       ├── ingress.yaml               # Uses ${HOSTNAME}
│       └── external-secret.yaml       # ghcr-pull-secret (ClusterSecretStore)

Home Page Pattern

The k8s/home/ directory is independent of the base/overlays structure. It deploys a separate nginx container serving a static home page at the app's root domain.

Key differences from the main app overlays:

• Does not reference ../../base — it has its own deployment, service, and ingress
• Uses a separate image — ghcr.io/labrats-work/apps.my-diet/home:<tag>
• Needs its own ghcr-pull-secret — the base ExternalSecret is not inherited, so k8s/home/ includes its own ExternalSecret referencing the ClusterSecretStore
• Deploys to its own namespace — my-diet-splash, like every other environment

Home kustomization.yaml

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

resources:
  - deployment.yaml
  - service.yaml
  - ingress.yaml
  - external-secret.yaml

labels:
  - pairs:
      app.kubernetes.io/part-of: my-diet
      app.kubernetes.io/managed-by: flux
    includeSelectors: false

images:
  - name: ghcr.io/labrats-work/apps.my-diet/home
    newTag: "0.2.0"

Secret Management

my-diet uses the platform secrets pattern with per-platform AGE encryption:

1. Platform secrets are SOPS-encrypted in platform/ of the platform repo using the per-platform AGE key (not the cluster key) and decrypted into the my-diet-platform namespace by the platform Flux Kustomization
2. RBAC in platform/rbac.yaml grants secret-reader access to dev, prod, and splash service accounts
3. SecretStore in k8s/base/secret-store.yaml reads from my-diet-platform namespace
4. ExternalSecrets in k8s/base/ pull individual secrets from the SecretStore into each environment namespace

ghcr-pull-secret

The ghcr-pull-secret is a shared cluster credential managed via ClusterSecretStore, not the app-specific SecretStore. Every kustomization path that uses private images needs its own ExternalSecret for it:

• k8s/base/external-secret.yaml — used by dev/prod overlays
• k8s/home/external-secret.yaml — used by the home page

This is easy to miss: if a kustomization path doesn't inherit from base/, it won't get the pull secret automatically.

Per-Platform AGE Key

The platform Kustomization uses a per-platform AGE key for SOPS decryption — not the cluster-wide AGE key. This key is automatically provisioned by the AppPlatformRegistration controller:

1. The controller generates an AGE X25519 keypair and stores it as a Secret in the operator namespace
2. A SOPS-encrypted copy is committed to the Flux repo as apps/my-diet-platform/age-secret.sops.yaml
3. When Flux decrypts it (using the cluster AGE key), the secret is created in my-diet-platform
4. The platform Kustomization references this secret for SOPS decryption

This means secrets in platform/*.sops.yaml must be encrypted with the platform's AGE public key (found in status.agePublicKey or in the .sops.yaml at the platform repo root), not the cluster key. The .sops.yaml is auto-committed, so sops --encrypt works out of the box inside the my-diet-platform repo.

Variable Substitution

Flux postBuild.substitute injects environment-specific values. my-diet uses:

Variable	Home	Dev	Prod
HOSTNAME	my-diet.hcl.labrats.work	app.my-diet-dev.hcl.labrats.work	app.my-diet.hcl.labrats.work
LOG_LEVEL	—	debug	info
NODE_ENV	—	development	production

These are set on each Flux Kustomization's postBuild.substitute in the platform repo.

Preview Environments

my-diet supports PR preview environments via a GitHub Action workflow:

1. Add deploy-preview label to a PR
2. The workflow creates a namespace my-diet-pr-N and a Flux Kustomization for it
3. The preview is accessible at pr-N.my-diet.hcl.labrats.work
4. When the PR is closed or the label is removed, the preview namespace is deleted

The preview overlay at k8s/overlays/preview/ uses reduced resource limits.

Verification Commands

# AppPlatformRegistration status (includes agePublicKey)
kubectl get appplatformregistration my-diet -o wide
kubectl get appplatformregistration my-diet -o jsonpath='{.status.agePublicKey}'

# Platform repo Flux resources
kubectl get gitrepository my-diet-platform -n my-diet-platform

# Kustomizations (in my-diet-platform namespace)
kubectl get kustomization -n my-diet-platform

# All environments healthy
curl -sI https://my-diet.hcl.labrats.work                       # home
curl -sI https://app.my-diet.hcl.labrats.work/api/health        # prod
curl -sI https://app.my-diet-dev.hcl.labrats.work/api/health    # dev

# Namespace labels
kubectl get namespaces -l labrats.work/app=my-diet -o custom-columns=NAME:.metadata.name,ROLE:.metadata.labels.'labrats\.work/namespace-role'

# Platform secrets
kubectl get secrets -n my-diet-platform
kubectl get externalsecret -n my-diet-prod
kubectl get externalsecret -n my-diet-dev

# RBAC
kubectl get roles,rolebindings -n my-diet-dev
kubectl get roles,rolebindings -n my-diet-prod

Lessons Learned

Namespace ownership matters

If a namespace is created by an external Flux Kustomization (e.g., a legacy entry in the cluster flux repo) with prune: true, removing that Kustomization will delete the namespace and everything in it. The AppPlatformRegistration (cluster-scoped) survives, but all namespaced resources (secrets, RBAC) are lost.

Fix: Ensure namespaces are declared in spec.namespaces on the AppPlatformRegistration so the operator creates them with correct labels. When migrating from legacy Flux resources, remove them carefully and verify namespace ownership before deletion.

Non-base kustomization paths need their own pull secrets

The k8s/home/ directory doesn't inherit from base/, so it doesn't get ghcr-pull-secret automatically. Any standalone kustomization path that uses private container images must include its own ExternalSecret for the pull secret.

Symptom: ImagePullBackOff on pods in the home namespace while dev/prod work fine.

Platform namespace is hidden from the dashboard

Namespaces with labrats.work/namespace-role: platform are filtered out of the dashboard namespace list and cannot be deleted via the API. This protects infrastructure namespaces from accidental deletion.

Platform Kustomization requires the per-platform AGE key

The SOPS decryption secret referenced by the platform Kustomization must be the per-platform AGE key (my-diet-platform-age-key), not the cluster-wide sops-age secret. The AppPlatformRegistration controller provisions this key automatically — it's stored as a SOPS-encrypted secret in the Flux repo and decrypted into my-diet-platform by the cluster AGE key. Secrets in platform/ must be encrypted with the platform's AGE public key (available in .sops.yaml at the platform repo root).