What a short, intense rotation taught me about uncertainty, evidence, and working alongside AI in the SOC.
Let me get one thing out of the way: I am not a career SOC analyst.
I did not walk into Black Hat with years of incident-response experience or an encyclopedic knowledge of threat actors. I was there for a concentrated SOC rotation, stepping into an unfamiliar role for a limited time and trying to absorb as much as I could.
The first time I walked into the Black Hat NOC, where the SOC team was operating, was a few days before the event began. I was surrounded by people with years—sometimes decades—of experience, many of whom had supported Black Hat for years. I looked around and thought, What the hell am I doing here?
Having never worked in a SOC, I spent those first few days asking myself basic questions:
- What does a SOC analyst actually do after an alert appears?
- Which parts of the screen matter?
- How do you know whether something is malicious?
- What happens when two tools seem to tell different stories?
- And perhaps most importantly, how do you make a decision when you know you do not have perfect visibility?
I had also never worked with that many security tools at once. The number of unfamiliar dashboards, data sources, and workflows was intimidating. More than anything, I was afraid I would miss something real—that a genuine incident would be buried in the noise, and I would not recognize it.
I did not yet know the answers. By the end of the rotation, I learned something more useful than a collection of answers: a repeatable way to ask better questions—and a clearer sense of how AI agents could help me ask them sooner.
The dashboards looked more certain than I felt
Security tools are very good at presenting conclusions. They give an incident a title, a severity, a score, a collection of tactics, and sometimes an automated explanation. To a first-timer, all of that can feel authoritative.
But an alert is not a verdict. It is a claim.
One of my earliest lessons was to stop asking, “What does the alert say happened?” and start asking, “What evidence caused the alert to say that?”
That shift sounds small. It changed everything.
A dramatic title might come from an informational detection. A suspicious domain might have no established reputation at all. A large number of network records might represent unanswered probes rather than successful sessions. Several real events might be grouped into one incident even though they came from different devices and had nothing to do with one another.
Dashboards, scores, summaries, reputation results, and prior incidents were useful for orientation. They helped generate hypotheses. Even when an AI assistant produced a confident explanation, the conclusion still had to be earned from the evidence.
I learned to investigate one question at a time
At first, the temptation was to open every tool and search for everything. That felt thorough. In practice, it produced more data than understanding.
The better approach was slower at first and much faster later: identify the uncertainty that could actually change the decision, then test that uncertainty with the best available evidence.
Instead of “Let’s check the SIEM,” the question became something like:
- Did the connection succeed, or was it only attempted?
- Which device held this address at the time of the event?
- Does the application-layer behavior match the story in the alert?
- Are these findings parts of one sequence, or independent events placed in the same container?
- What result would make me change my current hypothesis?
For each step, I wanted to know why I was running a query, what evidence I expected to find, what different results would mean, and what the data could not prove.
Afterward, I separated the result into four categories: verified facts, inferences, limitations, and unresolved questions. That structure prevented a plausible explanation from quietly turning into a “fact” just because it sounded right.
At first, I tried to answer too many questions at once. That moved the investigation forward, but it hid too much of the reasoning I was there to understand. I deliberately returned to one bounded stage at a time: define the question, explain the test, interpret the result, and decide what the evidence justified.
The goal was not to memorize clicks or borrow someone else’s conclusion. It was to develop judgment I could defend for myself.
The agents helped me work above my experience level
The biggest surprise was that the SOC did not feel like one analyst sitting in front of one product. It felt like a team that now included specialized AI agents and assistants. As a first-timer, I could ask for context, test an interpretation, and move from an alert to a sharper question without pretending I already knew every interface.
Cisco XDR Instant Attack Verification used multiple agents to correlate signals and present an attack story with a verdict and supporting reasoning. Splunk Attack Analyzer helped follow suspicious content through an attack chain and surface behavior that would have taken me much longer to unpack manually. I also used the new Splunk Enterprise Security Triage Agent, which conducted an analysis of every Splunk Attack Analyzer report that had a high threat score.
The XSIAM AI Assistant and Corelight AI Assistant helped translate platform-specific evidence into language and the next steps I could work with. Within the boundaries of what could safely be shared, Codex served as my personal AI assistant across the investigation: helping me turn uncertainty into bounded questions, separate facts from inferences, track limitations, and shape the final documentation.
The important thing was that these were not five votes on the same question. Each assistant had different visibility, different strengths, and different limits. A confident answer from one system could not fill a gap in another system’s telemetry.
When their outputs lined up, confidence increased. When they disagreed, the disagreement was useful because it exposed an assumption that needed testing. Instead of asking an assistant, “Is this malicious?” I learned to ask, “What evidence supports that conclusion, what are you assuming, and what result would change it?”
That interaction let me operate at a higher level sooner, but it did not make me senior—or remove the need for an analyst in general. The agents accelerated orientation and reduced the cost of asking basic questions. Human analysts still supplied context, challenged the reasoning, controlled sensitive data, and owned the decision.
AI could accelerate the investigation. It could not lower the standard of proof.
Context mattered—but it did not lower the standard of proof
Black Hat is not a typical corporate environment. Training, research, scanning, tunneling, testing tools, temporary infrastructure, and unusual traffic are all part of the landscape. Behavior that would be alarming somewhere else may have a legitimate explanation there.
That context was essential, but it was also dangerous if used carelessly.
“This looks like a training activity” is a reasonable hypothesis. It is not evidence that a particular device was authorized to perform a particular action at a particular time.
Network maps, schedules, prior cases, and familiar patterns helped me decide what to test next. I still needed incident-time evidence to support attribution and behavior. An address shown in a current asset view might have belonged to a different device during the event. A known application could still be used unsafely. A detection of a legitimate red-team tool could be technically correct even when the activity was authorized.
One of the most portable lessons from the week was this:
Context should change the hypothesis, not the evidence standard.
The real risk was not always the one named in the alert
In one sanitized, composite example drawn from the week, an alert appeared to point toward credential abuse. Investigation weakened the alert’s original theory: the application involved was legitimate, and the observed sequence made sense as a real authentication flow.
It would have been easy to stop there and call the alert harmless.
But the evidence supported a different concern. Sensitive authentication material appeared to have crossed an unsafe channel. The initial theory had weakened, yet a genuine exposure remained—one that required escalation because remediation and account actions were outside my authority.
That experience taught me not to grade an investigation solely on whether the alert title was “right.” The analyst’s job is to follow the evidence toward the most important, supported risk, even when it is not the risk the platform originally named.
It also sharpened my understanding of dispositions for the event. “False positive” is not a catch-all term for anything that does not become a confirmed incident. A detector can accurately identify a behavior that turns out to be benign, expected, authorized, or tolerated. A near miss is different from a detection error. An administrative merge is not a security judgment. Escalation is not a failure to finish—it can be the correct outcome when the evidence establishes a credible risk, and the required response belongs to someone else.
The label matters because the reasoning behind it matters.
An IP address is not an identity
Some of the most convincing stories fell apart when I checked the time.
A correlation system might group several behaviors around one address and present them as a clean attack chain. But addresses are reassigned. On a busy event network, the device using an address now may not be the device that used it earlier.
Reviewing lease timing, device identifiers, host information, and other source-native records could turn one apparent multi-stage incident into several unrelated behaviors. Even then, identifying a device did not automatically identify a person.
The lesson was simple and surprisingly easy to forget: ask who held the address at the time of the event, not who appears to own it now.
This was also a reminder that an incident is a container, not necessarily one story. Correlation is helpful because it gives the analyst somewhere to begin. It is still only a hypothesis about how the pieces fit together. Every contributing finding has to be accounted for before the container can be assessed as a whole.
I got faster by asking smaller questions
Early in the rotation, thoroughness felt like checking every available source. Later, I learned that investigation quality is not measured by the number of tools opened.
Speed came from finding the smallest question whose answer could change the outcome.
Sometimes one field mattered more than hundreds of events: a connection state showing that no session was established, a request path revealing automated reconnaissance, a response sequence showing a legitimate login, or a lease boundary proving that two behaviors came from different devices.
That did not mean skipping validation. It meant checking coverage first, examining the decisive exception second, and stopping when the remaining uncertainty would not change the disposition or response.
Knowing when to stop was a skill of its own. So was knowing when not to begin. When another analyst already owned a case, the correct move was to deconflict and step back—not add competing notes or create a second conclusion.
The worklog exposed whether I really understood the case
The final challenge was explaining the investigation clearly and compactly.
A query history is not an analysis. Neither is a one-line conclusion with no evidence behind it. A useful worklog must connect the trigger, the relevant evidence, attribution, assessment, limitations, and recommended disposition in a form another analyst can scan and defend.
Writing within a strict character limit forced me to distinguish what was important from what was merely available. It also exposed gaps. If I could not explain why the evidence supported the conclusion, the investigation might not be ready to close.
The notes also needed boundaries. Credentials, tokens, sensitive request bodies, packet contents, and other secrets did not belong in the worklog—or in my learning material. It was possible to acknowledge that sensitive evidence existed without copying it into another system.
AI assistance required the same discipline. I did not need to place every raw detail into a prompt to get useful help. The right pattern was to provide only the context needed for the question, keep sensitive evidence in its authorized system, and carry forward the conclusion without copying the secret.
Documentation was not paperwork after the investigation. It was part of the analysis.
What I carried out of the SOC
A short Black Hat rotation did not turn me into a veteran SOC analyst, and I do not want to present it that way. What it gave me was a close look at how skilled analysts reason under uncertainty—and how specialized agents can help a newcomer contribute without pretending machine output is proof.
The most valuable outcome was not memorizing where buttons lived in a particular security product or finding the perfect prompt. I may never work with that exact collection of tools again. What transfers is the method:
- Understand the claim.
- Inventory the evidence that produced it.
- Identify the uncertainty that matters to the decision.
- Ask one precise question.
- Use the right assistant to accelerate the next bounded step.
- Verify the result against the best available telemetry.
- Separate facts, inferences, limitations, and unresolved questions.
- Stop, document, or escalate when the evidence supports it.
That method is useful far beyond a SOC. It can apply anywhere people, and AI systems make decisions from imperfect data.
That, to me, is the promise of the Agentic SOC: not removing people from the work, but helping a new analyst ask better questions sooner, reach the right evidence faster, and learn from every step—without lowering the standard required to act.
My time in the SOC began with one question: What the hell am I doing here? It ended with a better one: What does the evidence actually prove?
Check out the other blogs from our team at Black Hat USA 2026.
- Building the Agentic SOC, One Live Event at a Time
- Safeguarding DNS With Secure Access at Black Hat
- Building a Risk-Based Secure Network Analytics Detection with Splunk Detection Editor (Alpha)
- Troubleshooting Wi-Fi at Black Hat USA 2026 with ThousandEyes
- Distributed Latency Monitoring at Black Hat
About Black Hat
Black Hat is the cybersecurity industry’s most established and in-depth security event series. Founded in 1997, these annual, multi-day events provide attendees with the latest in cybersecurity research, development, and trends. Driven by the needs of the community, Black Hat events showcase content directly from the community through Briefings presentations, Trainings courses, Summits, and more. As the event series where all career levels and academic disciplines convene to collaborate, network, and discuss the cybersecurity topics that matter most to them, attendees can find Black Hat events in the United States, Canada, Europe, Middle East and Africa, and Asia. For more information, please visit www.BlackHat.com.