Cloudwatch on ferkakta.dev

Per-Tenant CloudWatch Log Isolation on EKS, or: Why I Stopped Using aws-for-fluent-bit

Mon, 02 Mar 2026 00:00:00 +0000

The starting assumption

I’m building ramparts, a multi-tenant compliance platform running on EKS. Each tenant gets a Kubernetes namespace – tenant-acme, tenant-globex, whatever – and the compliance controls require that their application logs land in isolated storage with 365-day retention. CMMC maps this to AU-2 (audit events), AU-3 (audit content), AU-11 (retention), and AC-4 (information flow isolation). A tenant cannot read another tenant’s container output.

The obvious first move was aws-for-fluent-bit, AWS’s own Helm chart and container image for shipping logs to CloudWatch. AWS service, AWS chart, AWS logging destination. The blessed path.

Why we removed aws-for-fluent-bit from EKS

Mon, 02 Mar 2026 00:00:00 +0000

We deployed aws-for-fluent-bit because AWS recommends it.

If you follow the EKS logging documentation, that’s the default path. It assumes you use AWS’s distribution of Fluent Bit rather than the upstream Helm chart.

We did.

Two days later, we ripped it out.

The AWS chart and the upstream chart are not the same thing. The differences aren’t cosmetic. They affect how quickly you receive security patches, how transparently your configuration maps to the underlying plugin, and how many boundaries sit between your logs and the CloudWatch API.

Stop copying AWS managed policies — deny what you don't want instead

Fri, 27 Feb 2026 00:00:00 +0000

I needed to give a developer full CloudWatch read access — metrics, alarms, dashboards, log groups — but deny access to three categories of log groups containing security-sensitive data: WorkSpaces OS event logs, VPC flow logs, and WAF request logs.

The reflex is to copy CloudWatchReadOnlyAccess into a custom policy and delete the parts you don’t want. I’ve seen this in every organization I’ve worked in. It produces a policy with 50+ actions that you now own. Every time AWS ships a new CloudWatch feature, your policy is stale. You won’t update it. It’ll rot.

Amazon WorkSpaces are invisible to SSM and CloudWatch (and how to fix it)

Thu, 12 Feb 2026 10:00:00 -0600

I spent an afternoon arguing with Windows about whether I was allowed to be root on a machine I created. Six hours and six layers of undocumented workarounds later, I got CMMC-compliant audit logging on a desktop that doesn’t know it exists.

The problem

WorkSpaces don’t show up in AWS Systems Manager. They’re not EC2 instances — no instance profile, no metadata endpoint, no identity. SSM Agent is pre-installed but thinks it’s nobody. CloudWatch Agent has no credentials and doesn’t know what region it’s in.