Python on ferkakta.dev

The missing layer in compliance RAG: why your search results need a judge

Thu, 19 Mar 2026 12:00:00 -0500

If you’re building search over a knowledge base with an LLM — the pattern everyone calls RAG — you’ve seen the standard pipeline: embed the user’s question, find the closest chunks in a vector store, hand them to the LLM, get an answer. For documentation search or internal wikis, this works. The LLM is good at ignoring irrelevant context when the relevant stuff is also in the window.

I’m building a CMMC compliance platform, and I wanted a way to dogfood our own product against our own development process. Every commit we make to the platform touches some aspect of NIST 800-171 — access control, audit logging, encryption, configuration management. I wanted our pull requests to show which compliance controls each change addresses. Not as a compliance artifact (though it could become one), but as a consciousness-raising tool: every engineer on the team sees the compliance implications of their own code, every reviewer sees which controls are being strengthened. It’s ambient education that turns into culture.

Making a Kopf operator idempotent: three-layer existence checks and the redisReady race

Fri, 20 Feb 2026 12:00:00 -0500

Our tenant operator provisions databases, cache users, and credentials for each tenant in a multi-tenant SaaS platform. PostgreSQL roles on shared RDS, ElastiCache RBAC users, SSM parameters with generated passwords. It worked exactly once per tenant. The second time it ran, it regenerated every password and overwrote every SSM parameter. Running services holding the old credentials immediately lost their database and cache connections.

This was the blocker for auto-deploy.

Every deploy was a coordinated outage

The orchestrator runs terraform apply for each tenant on every deploy. Terraform reconciles the Tenant CRD, which fires Kopf’s on_tenant_create handler. The handler doesn’t distinguish between “new tenant” and “existing tenant whose CRD was re-applied.” It generates fresh passwords, creates new PostgreSQL roles (which fail because the role exists, or worse, succeed and orphan the old one), and overwrites SSM parameters with credentials that no running pod knows about.