TL;DR
Agent harnesses cut time-to-finding from days to hours, surfacing two information leaks totaling more than 12 million records. This post breaks down practical MCP tooling and prompting patterns across external, application, and cloud penetration testing. It also draws the line on where these tactics are fair game and where client scope and inference provider choice still demand a judgment call
AI is changing the execution of penetration tests. By combining agent harnesses with Model Context Protocol (MCP) servers and a persistent knowledge store, allows pen testers to expand testing coverage. The result is more thorough assessments without sacrificing human oversight or accountability.
Using inference with agent harnesses has demonstrated an increase in coverage when executing fieldwork at Bishop Fox. When the mission is how much can we secure in a time boxed, budget-approved, and business friendly manner, it would be irresponsible to not stack the deck in our favor.
Over the past few months of work, these tactics were used to identify two separate personal information leaks totaling over 12 million unique records. Would we have found these issues without the help of LLMs? Absolutely, but instead of a few days to find the issues, they were found within hours of beginning fieldwork. Both targets were client-commissioned work, and one target was identified as being in scope for multiple bug bounty programs for years. We were able to cover more ground using these technologies than we would have been able to otherwise.
Even though these letters are produced with organic phalanges banging out artisanal tokens for your cortex, what's the point if it's just yapping by tapping. Let's look at a few approaches across external, cloud, and application penetration testing for practical approaches to responsibly using MCP agents to our advantage.
External Penetration Testing
When attacking targets over the internet as an unauthenticated attacker, we can use a simple MCP server with the ability to read files, write files, edit files, and execute code to perform most tasks required for determining if an attacker is able to gain unauthorized access to an organization over the internet.
In its most basic form, we can provide a target or multiple targets and tell the LLM to attack this target. Depending on the model's alignment, we may need to persuade the model that we are ethical hackers and that the activity we are carrying out has been approved. In some cases, stating that your account is part of the security approved process may also bypass guardrails.
However, the interaction above originally worked two months before this was published and now no longer works due to strengthening of model guardrails. One of the more basic ways to interact with LLMs that doesn’t consider token usage or completeness of testing involves prompting the LLM to perform an external penetration test.
Perform an external penetration test against bishopfox.com and related subdomains.
We can level up our process by providing a test plan as part of the prompt. In most cases, it is best to dynamically retrieve test plans, so our agent harness can use a template with interpolated prompts and test plans.
Perform an external penetration test against bishopfox.com and related subdomains. Use the methodology here: [Pasted text #1 +46 lines]
We can take this one step further by including directives to keep the test plan updated as the agent orchestrator delegates individual test plan items to a sub-agent.
Perform an external penetration test against bishopfox.com and related subdomains. Use the methodology here and as steps are completed, document the specific commands ran, the result location, and add to the attack summary for the scope as you go: [Pasted text #1 +46 lines]
In all of these cases, we are leaving the execution of the test plan to the LLM instead of executing the test plan on its behalf. To be more efficient with our tokens and more sure of our process, we can instead expose a MCP server with limited tools to the LLM with a description. An example could be "Execute a port scan of all TCP ports," and scan a network the exact way that we want to and not worry if the LLM remembered to included -Pn
in our nmap
scan.
❯ /mcp ───────────────────────────────────────────────────────── Tools for kali 34 tools ↑ 25. masscan-all destructive, open-world 26. masscan-discovery destructive, open-world 27. nmap-100-udp destructive, open-world 28. nmap-1000-udp destructive, open-world ❯ 29. test-plan-status destructive, open-world ↑/↓ to navigate · Enter to select · Esc to back
If we wanted to take this logic a step further, there are a series of actions we should be taking against any discovered network, such as socket discovery, service fingerprinting, looking for misconfigurations, and determining service exploitability. An event-driven loop that recursively processes data types to generate an initial report for the LLM is a more effective approach when it comes to ensuring expected process execution, controlling token usage, and guiding the LLM towards additional tasks given the information collected. This is the penetration testing equivalent of feature-specific pull requests in software development—we are being specific with a small, scoped task. Also, I typed that em dash, don’t @ me!
For using LLMs in the context of external penetration testing, we looked at multiple ways to approach this. From using a basic prompt of "attack this system", to "attack this system with a test plan", to "attack this system with these tools and with this test plan", to "let's automate what we can without an LLM and then give the report to an LLM to iterate on the process".
Bottom line: The value of AI isn't simply asking it to "attack a target"; it's giving it structured workflows, deterministic tools, and enough context to execute repeatable penetration testing processes.
Let's move into application penetration testing now.
Application Penetration Testing
Again, starting with the lowest barrier of entry, we can tell an LLM to attack a web application:
Attack this target and look for OWASP Top 10 vulnerabilities: <a href="https://target.com/"&gt;https://target.com&lt;/a&gt
Now unless we are monitoring the log file of the system and reviewing every single command, we may not have visibility into what the LLM is doing. So let's improve our prompt and ensure our web application testing proxy is running:
Attack this target and look for OWASP Top 10 vulnerabilities: <a href="https://target.com/"&gt;https://target.com&lt;/a&gt
ALL HTTP REQUESTS MUST BE PROXIED TO <a href="http://127.0.0.1:8080/"&gt;HTTP://127.0.0.1:8080&lt;/a&gt
With the additional instruction, we will start seeing the requests show up that the agent orchestration harness makes within our testing proxy. In the first case noted where these tactics were effective, the web application had an open registration capability where users could create user accounts. Two user accounts were created and awareness of the credentials shared with the LLM by referencing a series of environment variables for the username, password, and issued authentication token. We can use this for authorization testing.
Attack this target and look for OWASP Top 10 vulnerabilities: <a href="https://target.com/"&gt;https://target.com&lt;/a&gt
ALL HTTP REQUESTS MUST BE PROXIED TO <a href="http://127.0.0.1:8080/"&gt;HTTP://127.0.0.1:8080&lt;/a&gt
For authorization testing, the first user account's information is stored in $USER1, $PASS1, and $TOKEN1 whereas the second user account is stored in $USER2, $PASS2, and $TOKEN2
For each request, also place an HTTP Header in the request for what is being tested
Do not issue destructive requests!!
We provided two final instructions there to help us understand what the LLM is doing with each of the requests and to not issue destructive requests. From my thorough research, two exclamation points appear to be the sweet spot. Jokes aside, this is an unreliable method for ensuring this doesn’t happen and is best combined with out-of-band rules in a security testing proxy. Along the way, various requests were selectively passed into the LLM to make it aware of functionality that I wanted a second set of eyes on. We can combine the approach above with what was discussed for external penetration testing and also pass in a test plan to keep updated as execution progresses.
Again, with this approach, millions of individuals’ personal information was able to be properly restricted from unauthorized attackers.
Cloud Penetration Testing
We can take a similar approach here by taking our read only cloud account and instructing the model to run tools such as scoutsuite
, pmapper
, aws
, cirro
, and cloudfox
for those pesky multi-provider cloud-native organizations. We can either gather this data manually in a quick manner or instruct the LLM to do so after configuring credentials. Once the data is gathered, this is a perfect use case for LLMs where we have read only access and an incredible amount of data to analyze for larger cloud environments.
Perform a cloud penetration test against the following AWS account: 012345678901
Utilize the cloud penetration testing methodology in the file cpt-test-plans.md and keep track of which test plans have been executed as work prorgresses. Utilize the profile `testers-read-only` that is authenticated.
Current data collected exists in the ./data directory for scoutsuite, pmapper, manual AWS CLI enumeration, and cloudfox
Determine the validity and applicability of the observations in the data initially gathered. Use the MCP server’s execute functionality with the `aws` CLI to be sure that no false positives exist in the suggested findings. It is common that these tools will report issues that are not relevant given the context of the environment.
There is an example prompt that communicates the scope, how to authenticate with our read only account, where current data is, and what the task is. Even with an account restricted to read only access, it's important to take ownership of every command executed. This approach is doubly important because we want to be learning from the system as much as we are using it, else our cognitive abilities atrophy.
Important for readers to note, while performing external penetration tests, mobile application assessments for publicly available applications, or application penetration assessments against publicly available applications, these are all systems available on the internet. This is not the case for authenticated access to internal client cloud accounts. When in doubt, check with the client, and they may provision access to their own inference provider for use by consultants doing fieldwork. They may be fine using the consulting company’s existing inference provider. Otherwise, local inference options get us most of the value that state-of-the-art models grant on a mid-tier system spec.
Closing Thoughts
Outside of using these technologies in the discussed ways, there are others that come to mind which bring certain attacks and threat models to bear in time-constrained penetration tests when they weren't possible or feasible before:
- Discovering original vulnerabilities via source code analysis or network and application fuzzing
- Developing custom tooling to aid technical efficiencies per engagement
- Executing test cases that may not be pursued due to time constraints
- Identify software locally or remotely, find a CVE without exploit code publicly available, see if the LLM can figure it out with network fuzzing or local debugging.
Hopefully this information gets ideas going on what's possible this this technology when it comes to responsibly using it for security assessments.
FAQs
What is an MCP?
Model Context Protocol (MCP) is an open protocol that standardizes how applications supply tools, data, and context to LLMs, using JSON-RPC 2.0 over a stdio or HTTP transport.
What is MCP in penetration testing?
Model Context Protocol (MCP) enables AI models to interact with tools, files, and systems during a penetration test. It allows AI to execute tasks deterministically, analyze results, and document findings while the tester maintains oversight and control.
How can AI improve penetration testing?
AI helps penetration testers automate repetitive work, analyze more data, and cover more ground during an engagement. Rather than replacing human expertise, it enables testers to spend more time validating findings and investigating complex attack paths.
Can AI perform penetration testing on its own?
Not yet. AI can accelerate many testing tasks, but it cannot replace human judgment. Experienced penetration testers are still responsible for defining scope, validating findings, supervising tool execution, and assessing business risk.
How is AI used during application security testing?
AI can help execute testing methodologies, analyze application behavior, inspect HTTP traffic, and identify potential vulnerabilities. When paired with security tools, it also helps document testing activities and maintain context throughout an assessment.
Can AI help with cloud penetration testing?
Yes. AI can quickly analyze cloud security data, identify potential misconfigurations, correlate findings across services, and help validate risks. Human oversight remains essential, particularly when working within authenticated customer environments.
Subscribe to our blog
Be first to learn about latest tools, advisories, and findings.
Thank You! You have been subscribed.
Recommended Posts