Platformengineering on ferkakta.dev

I answered 114 AWS Well-Architected Review questions from my terminal

Fri, 03 Apr 2026 23:00:00 -0500

I was fourteen questions into the AWS Well-Architected Review when my wrists told me to stop. Each question is a page: read the description, check the boxes, type notes into a 2084-character text field, click Next. The Container Build Lens alone has 28 questions. I had two more lenses queued — the main Well-Architected Framework (57 questions) and the Generative AI Lens (29). That’s 114 questions total, and the console wants me to click through every one.

I replaced the AWS CLI completer with a datalake

Thu, 02 Apr 2026 08:00:00 -0500

I needed to tell someone in Italy my availability in their timezone, typed TZ= and hit tab, and discovered a completer that’s apparently been sitting in zsh since the Pleistocene. That made me finally look at how completion actually works: #compdef, the dispatch table, _files, the whole vocabulary kit I’d been leaning on for years without really seeing. And in the middle of that I remembered the thing that had made me write off tab completion in the first place: aws_completer, the Python-spawning hog that claims every argument position and still makes a mockery of my left pinky finger when it innocently asks for a filename, interrupting to say: but wait, are you sure you don’t want to marry one of my 428 eligible daughters first?

FinOps portfolio: 71 tickets over 5 years

Wed, 01 Apr 2026 15:30:00 -0500

My first finops ticket was called “Optimize the AWS infrastcuture.” The typo is still there. That was 2021 — a one-person infrastructure team at a startup that didn’t have the word finops in its vocabulary and didn’t know it needed one.

Five years later I went looking for every cost-related ticket I’d ever created. I expected maybe thirty. I found 71, spread across 8 Jira projects, touching every layer of the stack from EBS volumes to LLM inference spend. Nobody asked me to create a finops practice. I just kept looking at the bill and refusing to pay for things that didn’t earn their keep.

I was wrong about shell completions for 15 years

Wed, 01 Apr 2026 15:00:00 -0500

I thought shell completions were tab-operated Jinja — fill in the blank, template-style. Type half a filename, press tab, get the rest. That’s all I thought they did.

And the AWS CLI completer kept confirming that impression in the worst way. When you’re typing aws s3 cp somefile.txt s3://bucket/, the completer hijacks tab on the local path argument and sits there forever trying to guess AWS-side completions that will never come. You’re done with the AWS part of the command. You just want to tab-complete a filename. It won’t let you. AWS hogs the empty space where your possibilities want to go, because it thinks it’s important to block your progress in case you wanted to speak to any of its 867 pet names. As John Roderick and Merlin Mann put it on Roderick on the Line: keep moving and get out of the way.

Three holes in the partition wall

Tue, 31 Mar 2026 20:00:00 -0500

I assumed GovCloud was AWS with a different region code. I wrote a whole post about how wrong that was. The partition wall between commercial AWS and GovCloud is real — no shared IAM, no cross-partition role assumption, no federated identity, no common STS endpoints. An arn:aws: principal cannot see an arn:aws-us-gov: resource. They are separate universes connected by a billing relationship and nothing else.

Except that’s not quite true either. There are three holes in the wall, and I found them one at a time over the course of a month.

Bananas Acquisition: a CMMC CRM playbook

Mon, 30 Mar 2026 20:00:00 -0500

I spent a Monday getting the same document from two cloud service providers. AWS took five minutes and a command-line PDF extraction tool. Google took eight hours, two simultaneous support chats, an LLM-drafted support ticket, an escalation sherpa, and a tripartite NDA structure whose existence is unknown to Google’s own frontline support.

Both vendors publish a CMMC Customer Responsibility Matrix — the spreadsheet that maps NIST 800-171 controls to inherited, shared, or customer responsibility. Both are legally required to provide it. The experience of obtaining them could not be more different.

Every AI session starts from zero. Mine doesn't.

Sat, 28 Mar 2026 01:00:00 -0500

The first thing I do in every Claude Code session is the same thing I’d do if I woke up with amnesia in my own office. Where am I? What branch? What did I ship yesterday? What tickets moved overnight? Did someone create a new workflow I haven’t seen? Is there a live patch on the cluster that never made it into terraform?

The model doesn’t know any of this. Every session starts from zero. The context window is empty. The handoff docs exist but nobody reads them unless you make it a rule. The Jira board changed while you were asleep. The teammate who was blocked yesterday merged a PR at 3am and now the branch you were working on has conflicts.

Do not fake smallness

Fri, 27 Mar 2026 15:00:00 -0500

I found an old gist from my BAM consulting days. It shows csvq — a one-line alias that turns any CSV stream into a queryable in-memory SQLite database with JSON output:

alias csvq="sqlite3 :memory: -cmd '.mode csv' -cmd '.import /dev/stdin s3' '.mode json'"

Pipe a CSV into it, write SQL, get JSON. I wrote it for a colleague I was initiating into my ways. The gist is polite. It shows the output being piped into jq '.[]', which is perfectly respectable Unix.

One module block per service per tenant

Fri, 27 Mar 2026 00:00:00 -0500

Every tenant on my platform gets three services: an API server, an auth service, and a frontend. Each one is a single module block in Terraform that creates a Kubernetes deployment, a ClusterIP service, an ALB ingress, IRSA for AWS access, ESO-synced secrets from SSM, and a feature flag discovery mechanism. The module is the same for all three services. The variables are different.

I extracted it into an open source module because I kept explaining the design decisions to people who asked “how do you deploy services to EKS?” and the answer was always “let me show you the module.” The module is the answer.

Every tool I've ever used is a CloudFormation frontend

Thu, 26 Mar 2026 18:00:00 -0500

I was reading a job description that wanted CloudFormation experience, and I had the thought that derails the actual task: I’ve spent my entire career using tools that compile down to CloudFormation and don’t mention it until something breaks. I’ve just never framed it that way.

My career is a parade of progressively nicer frontends for the same underlying control plane — but one at a time.

The first one was the AWS console. Click, wait, refresh, click. Then CloudFormation itself, which was an improvement in the way that a paper map is an improvement over asking for directions — technically correct, nearly unusable in practice. Then Serverless Framework, which promised to abstract the whole stack into a YAML file and a deploy command. Then Terraform, which promised cloud-agnostic infrastructure as code with a state model that actually worked.

Newspapers aren't dead. You read one every morning.

Thu, 26 Mar 2026 16:00:00 -0500

Every morning I open Slack, scan three channels, check Jira for overnight transitions, read Google Meet transcripts from the India-side scrum, glance at AWS support cases, and pull up the git log to see what merged while I slept. By the time standup starts I’ve assembled a mental model of where everything is. I do this every day. I’ve never called it what it is.

I’m reading the morning paper.

from feature_flags import *

Wed, 25 Mar 2026 21:00:00 -0500

A colleague needed a feature flag enabled on one tenant. FEATURE_FLAG_ENABLE_AGENTS=True — one environment variable, one pod. I added it to the K8s secret manually, restarted the pod, and he was unblocked in two minutes.

Then I realized: the next terraform apply would overwrite that secret without the flag. The ExternalSecret syncs from SSM, and the flag wasn’t in SSM through any path terraform knew about. My manual fix had a shelf life of one deploy.

The $233 Day, Part 2: The Inference Iceberg

Fri, 20 Mar 2026 17:00:00 -0500

I posted the part 1 findings to the team thread — model switch, cache invalidation, 20× call volume, $173 training run. Case closed. The numbers were clean, the explanation was satisfying, and the model got reverted within the hour.

Except $173 was wrong. Not wrong in the analysis — the training run did cost that much. Wrong in scope. I’d found the visible part of the spend and stopped looking.

The $173 Training Run

Fri, 20 Mar 2026 15:00:00 -0500

The Slack message landed at 3pm on a Wednesday: “model training successful, previously 20min, now 1h30m.” I had finished an EKS 1.32-to-1.33 upgrade on the ramparts cluster that morning. My upgrade, my timeline, my problem.

The first theory wrote itself. New cluster version, fresh nodes, cold image caches. I’d fixed a broken cluster autoscaler earlier that day — the old autoscaler deployment was pinned to a node selector that no longer matched after the upgrade, so pods were stacking up in Pending until I caught it. First-run penalties after a major version bump are real. Everyone on the call nodded. I almost typed up that explanation and moved on.