Two fields join the Kubernetes API to what ran inside the pod, and one turns up a container escape your process events never recorded.
If you already ship Kubernetes (K8s) audit logs and Defend for Containers (D4C) into Elastic, you still have to join them by hand, and neither source is complete on its own. In our lab, a compromised workload service account ran discovery, read secrets, minted a token, created a privileged pod, and execed into it to attempt a container escape. The escape wrappers, nsenter
and chroot
, never appeared in the runtime process events. Kubernetes audit logs recorded them in the decoded requestURI
.
Real incidents cross both planes, which is why our write-up of the Hugging Face intrusion shows Kubernetes and runtime abuse in the same campaign. The two data sources answer different questions:
Data source | Typical data streams | What it answers |
|---|---|---|
Kubernetes audit |
| Who called the API and which objects changed |
D4C |
| What ran or changed inside the container |
This post makes that connection practical in Elastic. We cover the fields that join the two data sources and two ways to join across them, depending on how the API call is attributed. We also cover the one-sided context fields that help you read the activity. Plus, we walk through each of those with queries and result views from our Amazon Elastic Kubernetes Service (EKS) lab.
This continues the control-plane correlation thread from the TeamPCP container attack scenario. For D4C setup and policy basics, see Getting started with Defend for Containers.
How Kubernetes audit logs and container runtime data connect
Two joins cover most cases. Which one you use depends on how Kubernetes attributed the API call.
| Pod caller identity | Service account and objectRef |
When to use it | The audit event carries the pod-name extra, which happens when a request uses the pod's bound service account identity from inside the pod | The call is attributed only as a service account in |
Kubernetes audit field |
|
|
Container runtime field |
|
|
What you get | Direct attribution; the pod that made the API call is the pod you look at in runtime | The pods a suspicious service account created or execed into, as a set to investigate |
Limits | Only works when the pod-name extra is present, so it misses calls attributed as a bare service account | If more than one service account touched the same pod in your window, runtime gives shared context rather than clean attribution |
Join keys shared by Kubernetes audit logs and container runtime
These fields carry the same meaning on both data sources. Pair them with a time window around the activity you care about (often 15 to 60 minutes).
Join idea | D4C field | Kubernetes audit field | How it connects |
|---|---|---|---|
Namespace |
|
| Scopes both data sources to the same workload space |
Cluster |
|
| Keeps multi-cluster environments from mixing |
Time |
|
| Bounds the join |
Pod (create → use) |
|
| Links a control-plane change to runtime inside the new or targeted pod |
Pod (caller identity) |
|
| Links runtime in a pod to API calls Kubernetes attributes to that same calling pod |
One of the most useful pod joins for tracking threat behavior is create and then use. K8s audit records the create (or an exec) and names the pod. We then use that same pod name on the D4C side to see what ran inside it a few minutes later. Once you know the namespace and pod name, look for matching orchestrator.namespace
and orchestrator.resource.name
, and then scan process.name
and process.args
for what ran in that pod.
Joining on pod caller identity
Some Elastic prebuilt rules correlate D4C and K8s audit in one detection using Event Query Language (EQL) sequences across both data sources. Service Account Token or Certificate Access Followed by Kubernetes API Request is a useful example of the pod (caller identity) join from the table above. Within 60 seconds, it joins a service account token or digital certificate file open on orchestrator.resource.name
to a Kubernetes audit API event on kubernetes.audit.user.extra.authentication.kubernetes.io/pod-name
.
sequence with maxspan=60s
[file where host.os.type == "linux" and event.type == "change" and event.action == "open" and
file.path in (
"/var/run/secrets/kubernetes.io/serviceaccount/token",
"/var/run/secrets/kubernetes.io/serviceaccount/ca.crt"
) and
process.interactive == true and container.id like "*"] by orchestrator.resource.name
[any where data_stream.dataset == "kubernetes.audit_logs" and
kubernetes.audit.stage in ("ResponseComplete", "ResponseStarted")]
by `kubernetes.audit.user.extra.authentication.kubernetes.io/pod-name`
That pod-name extra on a Kubernetes audit event is often present when a request uses the pod’s bound service account (SA) identity, including controllers and agents that call the API from inside the pod. When it’s present, you can line it up with the D4C field orchestrator.resource.name
, including in sequence rules like the one above. Some API traffic is attributed only as a service account in user.name
, without the pod-name extra. That requires a different join; start from the SA, collect pod names from objectRef
, and carry those into D4C with the namespace and a shared time window. That path is shown next.
Joining on service account and objectRef pod names
Caller identity joins on the pod that made the API call. This section joins on the service account principal and the pods it touched. Both are valid ways to move between Kubernetes audit and D4C; which one fits depends on how the audit event is attributed.
This join assumes a compromised or suspicious service account is driving API activity and that the pods it creates or execs into are where we should look next in D4C.
The join runs in three steps:
Timeline the service account in Kubernetes audit logs. Filter on
user.name
matchingsystem:serviceaccount::
to see every API call that principal made and which objects it touched.Collect pod names from
kubernetes.audit.objectRef.name
. Keep the rows whereobjectRef.resource
ispods
, which covers both creates and execs, and note the namespace fromorchestrator.namespace
.Query Defend for Containers with those pod names and namespace. Match
orchestrator.resource.name
against the pod names andorchestrator.namespace
against the namespace, using the same time window as the audit activity.
Each step is expanded below with the queries we ran.
Kubernetes audit carries the API principal in user.name
(and kubernetes.audit.user.username
), for example system:serviceaccount::
. On a D4C process event, that K8s principal isn’t present as user.name
, and user.id
is the Linux user ID inside the container (for example, 0
when the process runs as root). There are different mappings from different integrations, so even when both documents expose a field by the same name, the value reflects that data source’s schema, which isn’t always a 1:1 correlation.
A good approach is to timeline the suspicious service account in audit. This shows what that principal did and which objects it touched:
user.name: "system:serviceaccount::"
Or as Elasticsearch Query Language (ES|QL), keeping the fields that matter for the next hop:
FROM logs-kubernetes.audit_logs-*
| WHERE @timestamp > NOW() - 6 hours
AND user.name == "system:serviceaccount::"
| KEEP @timestamp,
kubernetes.audit.verb,
kubernetes.audit.objectRef.resource,
kubernetes.audit.objectRef.subresource,
orchestrator.namespace,
kubernetes.audit.objectRef.name,
user_agent.original,
`kubernetes.audit.annotations.authorization_k8s_io/decision`,
event.outcome
| SORT @timestamp ASC
In the service account timeline results, look at kubernetes.audit.verb
, objectRef.resource
, objectRef.subresource
, orchestrator.namespace
, and objectRef.name
for what was touched.
Collect pod names from the objects that SA touched. For pod creates and execs, the pod sits in kubernetes.audit.objectRef.name
, the namespace in orchestrator.namespace
, and the verb in kubernetes.audit.verb
, and exec shows up as kubernetes.audit.objectRef.subresource: exec (with objectRef.resource: pods
)
. Filter the same SA timeline to create and exec activity when you only want pod targets:
data_stream.dataset: "kubernetes.audit_logs" and
user.name: "system:serviceaccount::" and
kubernetes.audit.objectRef.resource: pods and
(
kubernetes.audit.verb: create or
kubernetes.audit.objectRef.subresource: exec
)
Query D4C with those pod names and the shared namespace. Use the namespace and pod names you collected (and the same time window as the audit activity):
data_stream.dataset: "cloud_defend.*" and
orchestrator.namespace: "" and
orchestrator.resource.name: ("" or "")
If more than one service account creates or execs into the same pod in that window, D4C on that pod name shows shared runtime context, not a clean attribution back to a single SA.
Namespace and pod name work in the other direction too. Starting from a Defend for Containers process event, take orchestrator.namespace
and orchestrator.resource.name
and search Kubernetes audit logs for that pod in objectRef
to recover create, exec, and related API activity.
Context fields that make the joined events readable
Join keys get you onto the right documents. Context fields usually exist on only one data source and add the detail you need to read those documents. We use several of them in the lab that follows.
Context fields in Kubernetes audit logs
Field | Why it matters |
|---|---|
| Identifies the API principal (human user, service account, or system component) |
| Identifies group membership for the principal (for example service account groups) |
| Shows which client made the request; unusual clients on a workload identity are often worth a closer look |
| Describes what was attempted against which Kubernetes object type |
| Adds request detail beyond verb and objectRef; for exec, the decoded URI includes the command list |
| Exposes the submitted object spec (security context, volumes, and similar) when you need to judge how sensitive a create or update was |
| Shows where the API call came from (node, pod network, or external client) |
| Distinguishes successful/allowed versus failed/forbidden API calls via native K8s authorization |
| Describes the reason for a failed API request |
Context fields in Defend for Containers
Field | Why it matters |
|---|---|
| Shows what’s executed inside the container and with which arguments; minimal images often use wrappers which put the executed process details in |
| Flags interactive sessions; often useful next to audit |
| On file events, shows which path was touched (for example, a projected service account token or certificate authority [CA] file) |
| Shows whether the container ran privileged; useful to cross-check a sensitive audit pod create |
| Tie the runtime event to a specific pod and namespace |
| Identifies which node the workload ran on |
| Narrow activity to a specific container when a pod has more than one |
| Point you at the right activity type (process versus file) and what operation occurred |
Lab scenario: Tracing a compromised service account from audit logs to runtime
In our EKS lab, we ran a compromised workload service account through discovery, secret access, a privileged breakout create, exec, and cloud metadata probes. That traffic is attributed as an SA without the caller pod-name extra, so the following walkthrough uses the service-account / objectRef join and the context fields from above to move between Kubernetes audit and D4C.
Finding a suspicious service account in Kubernetes audit logs
We started on the K8s audit side, looking for workload identities whose clients didn’t look like normal controllers or app traffic. Most legitimate SA requests come from in-cluster components with familiar user agents. When a system:serviceaccount:…
principal shows up with something like curl
or python-requests
, that can mean someone is driving the API with a stolen or projected token from a shell or a short script, rather than through the application’s usual client. That pattern doesn’t prove compromise on its own, but it gives us an identity and namespace to expand on. The same hunt works for other HTTP clients you may see on a stolen token (axios
, undici
, and similar
). Treat user_agent.original
as a lead, not proof. It’s easy to spoof. A curl session can send a legitimate-looking kubectl
user agent, so a “normal” user agent doesn’t rule the identity out.
Using the audit context fields user.name
and user_agent.original
, we looked for SA principals (system:serviceaccount:…
) paired with clients like python-requests/…
or curl/…
.
user.name: system\:serviceaccount\:* and
(user_agent.original: python-requests* or user_agent.original: curl/*)
We kept that service account and namespace for the rest of the example.
Timelining a suspicious service account's API activity
We timelined the SA and kept verb, object, user agent, authorization decision, and outcome in view:
FROM logs-kubernetes.audit_logs-*
| WHERE @timestamp > NOW() - 6 hours
AND user.name == "system:serviceaccount:emu-satoken-0297d6:worker-sa"
| KEEP @timestamp,
kubernetes.audit.verb,
kubernetes.audit.objectRef.resource,
kubernetes.audit.objectRef.subresource,
kubernetes.audit.objectRef.namespace,
kubernetes.audit.objectRef.name,
user_agent.original,
`kubernetes.audit.annotations.authorization_k8s_io/decision`,
event.outcome
| SORT @timestamp ASC
Context fields did most of the work here. user_agent.original
separated python-requests discovery from curl harvest. event.outcome
separated forbidden recon from allowed secret gets. objectRef.resource
/ subresource
showed TokenRequest (serviceaccounts
+ token
) and later pods
/ exec
.
From those same rows, we also collected pod names where objectRef.resource
was pods. One name was the original workload pod already running with that SA (the foothold). A later create
event named a second pod, which is the breakout that we explore next.
Spotting the privileged pod create behind a container escape
In the same timeline, a kubernetes.audit.verb: create on objectRef.resource: pods
named a new breakout pod. The creation alone is interesting; the audit-only fields on requestObject.spec
are what showed how sensitive it was. That document had hostPID
, hostIPC
, a privileged security context, and a hostPath
mount to /
, still attributed to the same workload SA and curl user agent that had been reading secrets.
That same document also carried the pod create → use join into D4C. objectRef.name
and objectRef.namespace
are the pod and namespace we query on the runtime side next.
Querying container runtime data with pod names from the audit timeline
The service account timeline gave us two pod names: the foothold pod and the breakout pod. We queried Defend for Containers for both:
data_stream.dataset: "cloud_defend.process" and
orchestrator.namespace: "emu-satoken-0297d6" and
orchestrator.resource.name: ("worker-0297d6" or "breakout-0297d6")
On the foothold pod, process context showed a projected-token read:
On the breakout pod, process context showed cloud instance-metadata clients were queried:
Why nsenter and chroot never reach container runtime events
On the breakout pod, we also ran a lab-safe host breakout attempt through pods/exec: nsenter -t1 -m -u -n -i true
and chroot /host true
. The true
processes are no-ops, so the session doesn’t do anything destructive. In a real breakout, the same wrappers usually target a shell or follow-on tool, for example nsenter … /bin/bash
or chroot /host /bin/sh
.
We went back to the D4C process events for that breakout pod looking for nsenter
and chroot
next to the metadata activity. On runtime events, we read process.name
and process.args
together. Those wrapper names weren’t there. D4C showed process.name: true
and process.args: true
instead, matching the true
targets in our lab commands. nsenter
and chroot
only set up the breakout and then hand off to that target. After the handoff, runtime records that target command, which here was true
. Even with both fields, the wrappers never made it into D4C, so we turn to Kubernetes audit next.
K8s audit and D4C answer different questions on the same breakout. The same SA’s pods/exec
events carry the full command list in kubernetes.audit.requestURI
. Decoding that URI showed nsenter
and chroot
against the breakout pod (the API-level breakout tools), while D4C showed what actually ran in the container after those wrappers and what that session did next.
FROM logs-kubernetes.audit_logs-*
| WHERE @timestamp > NOW() - 6 hours
AND user.name == "system:serviceaccount:emu-satoken-0297d6:worker-sa"
AND kubernetes.audit.objectRef.subresource == "exec"
AND kubernetes.audit.objectRef.name == "breakout-0297d6"
| EVAL uri = URL_DECODE(kubernetes.audit.requestURI)
| WHERE uri LIKE "*nsenter*" OR uri LIKE "*chroot*"
| WHERE kubernetes.audit.stage == "ResponseComplete"
| KEEP @timestamp, kubernetes.audit.objectRef.name, uri
| SORT @timestamp ASC
Conclusion
We’ve covered the core of unified container visibility: bridging Kubernetes audit logs with Defend for Containers telemetry. Plus, we’ve explored the two primary ways to join these datasets; that is, using pod-caller identity for direct attribution or service account and objectRef
mapping for broader investigations. And we’ve demonstrated how context fields like user_agent.original
or decoded requestURI
parameters turn those joined events into clear, actionable attack timelines.
Kubernetes audit and D4C remain two halves of the same picture: the API on one side, what ran inside the container on the other. Know which fields actually join them and which one-sided context fields still change how you read an investigation. Two joins cover most cases: caller identity when the pod-name extra is on the audit event, and service account plus objectRef
pod names when the call is attributed only as a service account. Fields such as requestObject.spec
, a decoded exec requestURI
, and user_agent.original
won’t join the two data sources by themselves, but they’re often what makes the joined documents actionable.
What next?
Now that you have the mapping foundation, the best way to operationalize these techniques is to test them against your own environment's telemetry. Start by implementing the queries we walked through in our lab scenario to identify suspicious service account activity, and then pivot to D4C to verify the actual runtime impact. For a shortcut, explore our prebuilt rules tagged Data Source: Elastic Defend for Containers
and Data Source: Kubernetes
, which serve as ready-to-use examples of these join patterns in action.
You can browse the same detection content in the detection-rules repository or our rules explorer page.