How to start the AI-accelerated defense

Early on in the AI adoption boom, I gained a reputation for just throwing everything at it to see what would stick. That wasn’t the most effective strategy, and my token usage was crazy high. There are a ton of talks and posts on all the cool ways you can use AI for detection engineering, but I didn’t see any that showed you where to begin.

So, this isn’t another blog about why you need to use AI in your defensive workflows. It seems most people understand why we need that. My focus is to show how our team got started and realized that providing AI with the necessary context is key to detection engineering successfully adopting AI.

This is not just about building detection logic, but that is one of the goals. This foundation helps create the AI Detection Engineering stack: logging pipelines, log onboarding, detection validation, threat modeling, and more.

An LLM does not know your stack, so out of the box it has limited value in a security review. In our experience, reliable results depend less on the fanciest model and more on the documentation and context around the workflow. If a human reads your log inventory and still has to ask three people what the ingestion method is, your agent does too.

Why cold prompting fails

When we first started using AI tooling, we realized prompts alone could get stuff done, but the output was inconsistent. Fields were missed, assumptions were made, and some detection logic was wrong. We saw it write queries that would not work in our SIEM. Usually these were around wildcards. The playbooks it wrote were generic, the tuning was poor, and some detections were just bad.

With enough re-prompting, the output would improve, but it always required some massaging. The effort invested in the agent inputs had a noticeable impact on the quality of the outputs. TL;DR: garbage in, garbage out.

Where our context came from

At the start, this was just internal documentation we built to make our own lives easier. It started with new-hire materials about where logs live, who owns each tool, and what needs protection. That is the bar. If you would hand it to a new hire on

day one, it is good enough to onboard the AI (I fear saying this a bit, knowing onboarding isn’t always the best). Just like an intern, if you neglect to set the context, your agent will guess and hand you something that looks right, but isn’t aligned to the team’s actual goals.

So every artifact below gets judged against that one test.

All of these documents are dual-use. Humans read them, and agents read them. Once again, think of this like onboarding documentation, then modify it to your agents’ needs. The better the data, the better the context, the better the outcome for your agents.

The good news is AI can help you make these with a read-only access key. The bad news is you are going to have to read and edit some slop.

I ordered these roughly by what to do first. If you have two hours this week, start at the top.

Context inventories and libraries

Log inventory — what, how, where, format

Start with this. A basic log inventory needs a name, category, priority, and owner. It gets more complicated with ingestion methods, vendor or internal contacts, and, to go one step further, detailed notes on what the logs provide.

We always include external resources. Blogs, vendor documentation, anything else that will help get someone up to speed on what is being provided. This part does more work than you would think. When an agent has the vendor documentation for a log source, it stops guessing at field names.

An agent with a real log inventory can tell you whether a detection idea is even possible before you write a line of logic. That alone is worth the afternoon.

Detection library — centralized, mapped, and tagged

Creating a single point for all your EDR, SIEM, and CNAPP detections can be hard, but it helps answer that age-old question. Can we detect that?

This will require some normalization because not all security tools save the same data. You'll want ATT&CK tagging, log source, and in some cases, origin if you're tracking where the detection ideas came from. Some fields you will need for metrics are “Date Created”, “Date Modified”, and “Dates Tested”. Believe it or not, not all companies track the “Date Modified” field.

The last big part is having a description and/or playbook for the detection. Depending on the tool, some limit the number of characters you can put in a description. Our team uses a modified version of the Alerting and Detection Strategies Framework. This gives us a clear understanding of every detection and the next steps for triage when it fires.

Having a structured output format defined is also why an agent can generate consistent, high-quality playbooks for us now.

Tool inventory — who owns what, contact info

These are the tools you'll use to investigate, but you'll also want the tools your counterpart teams use: EDR, SIEM, CNAPP, as well as ticketing, inventory, or internal knowledge sources. These will also include contact information and resources.

Without this, every playbook an agent writes would lead to a dead end. It knows what to do but has no idea where to go to do it.

Company/entity profile — crown jewels, VIPs, locations, and possible attack paths

This is how we tell our agents what is critical to us.

Crown jewels, VIP accounts, office locations, and the attack paths are what actually matter for our environment. This context turns a generic severity rating into a significant one. An agent triaging an alert on a production secrets service should not treat it the same as a hit on a test box. The only way it knows the difference is if you wrote it down.

Team charters — scopes, ownership boundaries, and the tools they use

This data helps us know who does what and with what. It is vital to building out playbooks and knowing who to escalate to. Scopes, ownership boundaries, and the tools each team uses help you or the AI make those decisions quickly.

We already had these because we needed them for humans. Turns out an agent building an incident response playbook needs the exact same thing.

Infra and log pipelines as code — parsing, normalization, schema

We currently have our security tooling and cloud infrastructure configured as infrastructure as code. This takes the longest but pays off the most. If your parsing, normalization, and schema live in code, it can become more context for your agent.

Once it can read how a log source is currently parsed and normalized, it can write the next one. It matches your existing patterns instead of inventing new ones; it uses your real field names, and the output goes through the same review as anything else in the repository. Our ingest pipeline work went from a multi-day task to about an hour.

If you do not have this as code yet, that is fine. Document the schema and the naming conventions in plain text and start there. Even a written schema beats nothing.

What we built on top of it

We store all our skills, agents, hooks, and MCP servers in a custom plug-in repository. This creates consistency across the team's work and enforces change control. We have three primary focus areas for our agents: logging, detection, and knowledge transfer.

Logging

On the logging side, we built skills to log knowledge and provide context not just on what we are logging, but how it's logged and enriched.

We have a skill that stands up a full log ingestion pipeline: an S3 bucket, a collector Lambda that polls the vendor API, a forwarder into the SIEM, CloudWatch monitoring, and a PR at the end. It writes the handler, the Terraform, the README, and the PR body, then wires the new pipeline into the existing monitoring stack. Logging is where the gap analysis lives too, comparing the log inventory against the tool inventory and our knowledge bases to find what we are blind to in logging and detections.

Detection

On the detection side, we have MCP access to our detection tooling. When a new log is onboarded, we run a threat modeling agent to verify and suggest new detections. This can be passed off to our detection engineering agent, which can then create a detection in the targeted tool. It verifies we have logging, checks for detection overlap, and looks for false positives. Once done, it creates a pull request designed to be easily verified. The pull request includes the logic it used, hyperlinks straight into the SIEM query, and a basic description of its work.

Knowledge transfer

Knowledge transfer is the one people skip. Documentation, normalization, and schema creation. It is the least fun of the three, but it's why the other two keep working. The easy one is building something to write your detection descriptions. We have several agents that look for future work around logging or configuration changes

Notice that none of these agents are doing anything crazy. Each is a thin wrapper around documentation we already had.

Security considerations

A few things to think about before you go feed your entire security program into a chat window.Not every platform is safe for internal data. Know what you are agreeing to, where the data lands, whether it is retained, and whether it trains publicly accessible models. Some of these documents map exactly what you protect and how you monitor it. Keep your company secrets safe by only using approved and appropriate AI tooling.

Remember to use least privilege. Prompt injection is real, and untrusted content can carry instructions; in our world, untrusted content is the whole job. Alert bodies, email samples, file names, ticket comments. If an agent reads attacker-controlled text and also has write access somewhere, you have a problem. Scope your MCP permissions like you would scope a service account, because that is what it is.

Do not trust the output. It can look completely right and be wrong. Validate before you ship. Every detection an agent writes still goes through the same review and testing as a human-written detection. The point is to speed up the boring parts, not skip the review.

What context do you already have?

Document what you have before you automate what you do not. The work is boring and front-loaded

You have two options here. You can be me in 2025, throwing everything at the model to see what sticks, burning tokens, and getting output that is okay. It works. It gets you there. It just costs you a lot of back-end massaging, and you never quite trust what comes out. Or you can spend a couple of weeks writing down what you already know about your own environment and get results you will actually ship, in a fraction of the time.

If you want a first move for this week, pick one log source and document it end to end. Name, owner, ingestion method, which fields it actually provides, which parsing it goes through, and one link to the vendor doc. Then ask your AI tool a real question about it and compare that answer to what you got before.

That difference in what you wrote and the AI wrote is the whole reason to do this.

Automate secret references with the 1Password CLI

The 1Password CLI can reference approved secrets from scripts and automation without pasting plaintext credentials into prompts or source code. Connect secure secret management to detection engineering workflows with our free developer tools.

Explore the 1Password CLI