AI is useful in investigations, but the collection phase is not one of those times. During collection, you may have one shot at getting the data you need. You need determinism, which AI doesn’t provide. But, we’ll talk about how AI can be useful when you are desperate and out of options.
The Collection Phase
The collection phase happens early in an investigation when you copy data that could be relevant to your investigation.
Examples:
- Retrieving telemetry or logs from an API
- Copying registry hives, event logs, and executables from a host
- Making a full disk or memory image of a host or VM
Collection happens after the planning phase, which is when you identify what you want to collect.
Your goal is to access the data and get a copy before it goes away. In some situations, you can go back and get more. In others, you may really have one chance.
Why AI Isn’t Needed for Collection
A key part of collecting is reliability. You want to know that if you need a file, that you will get it. And you want to avoid having to tell the client you missed some files. That’s why people often used tools like KAPE, UAC, and the Cyber Triage Collector.
Collection requires determinism and asking AI to collect data doesn’t give you that. It’s going to make you a disposable script that you need to spend time testing and validating. Because every host is different, testing is time consuming to ensure it works in all scenarios.
It’s also impractical to rely on an LLM when you are on an isolated host. If your collection depends on AI access, then your collection tool needs to embed an LLM or it needs Internet access. If you embed an LLM, that will then depend on what GPU resources the host has.
The basic takeaway is that collection doesn’t need creativity and it’s most time efficient for you to rely on validated, deterministic tools.
That being said, your tools can be smart and go beyond basic file copying. The Cyber Triage Collector is adaptive and parses files while it collects them, but that is done using deterministic methods that can be tested ahead of time.
New APIs: When AI is Useful
Sometimes there aren’t deterministic tools though and that’s where GenAI becomes interesting. For example, if you need to access logs from a cloud provider’s API and no tools exist for it.
You can ask GenAI to write you a deterministic script that you can test ahead of time. Then use it in your investigation.
This obviously works best if the API is documented. GenAI may be able to help with reversing engineering the API and bypassing controls, but make sure you are allowed to do that!
What If You Don’t Know What To Collect
I’m sure some of you are saying, “but, I don’t always know what to collect and I think AI can help me with that”. If so, you’re right, but to be a bit pedantic, that’s the planning phase. Planning is where you map your investigative questions (or goals) to what data you should collect.
We talked about that in our last post. AI is useful there.
Applying our Framework
We’ve been using a framework for the past few posts to evaluate the effectiveness of GenAI vs deterministic tools. Let’s go through those to support the conclusions we came up with above.
The more green boxes a column has, the better the solution it is.
Device and Data Are Supported
Lets first look at when there are deterministic tools that collect what you need.
| Validated, Deterministic Tool | Non-Validated, Deterministic Tool | Direct GenAI Output | |
| Feasibility | Yes | Yes | Sometimes. It can make you a script. |
| Time: Prepare | Low | Low | Low |
| Time: Perform | Avg | Avg | Avg |
| Time: Verify | Low | Med | High |
| Cost | Free/License | Free/License | Tokens, low |
| Creativity | Not Needed | Not Needed | Not Needed |
| Defensibility | High | Med | Low |
Validated, deterministic tools are the clear winner when they exist. They have the lowest verification time and highest defensibility.
Unsupported Devices or Services
If there are no existing deterministic tools, then AI can help to make you one.
| Validated, Deterministic Tool | Non-Validated, Deterministic Tool | Direct GenAI Output | |
| Feasibility | No. None Exists | No. None Exists | Maybe if it can reverse engineer it or find docs. |
| Time: Prepare | N/A | N/A | Low |
| Time: Perform | N/A | N/A | Avg |
| Time: Verify | N/A | N/A | High |
| Cost | N/A | N/A | Tokens, low |
| Creativity | N/A | N/A | High |
| Defensibility | N/A | N/A | Avg |
Basic takeaway is that GenAI may be able to write a script for you if you have credentials to an API that no one else has written a script for.
Validated, Deterministic First. Then AI.
As we’ve seen in this series so far and will continue to see, you’re better off to rely on validated, deterministic tools if they meet your requirements. GenAI can write you a program, but you’re going to need to spend time validating it.
If you are looking for a deterministic and validated collection tool that goes beyond a static set of rules, then check out the Cyber Triage Collector. It adapts to what is on the host and gets as many relevant files as possible, even if you don’t know where everything is going to be ahead of time. That data can then feed into Cyber Triage for you to get analyzed results of Bad and Suspicious files. You can get it within the 7-day evaluation.