How I Used AI Agents to Expose SOP Gaps and Build My Own OS

I walk through how AI agents can audit your SOPs, flag hidden gaps, and give you data to replace rented SaaS with a custom operating system.

When I first sat down to map the operating procedures of a 200‑employee, owner‑run firm, the paperwork looked tidy but the reality was messy. The SOP manuals were a patchwork of templates, each written by a different manager, and the SaaS stack was a zoo of 21 tools that barely talked to each other. I needed a way to see the invisible cracks before they broke the business.

Why SOPs Hide Gaps

Standard operating procedures are supposed to be the blueprint of how work gets done. In practice they become a compliance checklist that nobody updates after a new hire or a market shift. The result is a document that reads fine on paper but fails when the front line tries to follow it.

I learned that most gaps are not missing steps; they are mismatched expectations between people, tools, and data. A sales rep may be instructed to log a lead in CRM, but the CRM integration is broken, so the lead lives only in a spreadsheet. The SOP never mentions the failure because the process owner assumes the integration works.

Because each department—16 in this case—had its own language, the SOPs became siloed. The only way to surface the contradictions is to treat the whole SOP corpus as data and let a machine scan for anomalies.

Training an AI Agent to Read Your Docs

I started by feeding every SOP, policy, and workflow diagram into a vector store. The AI agent I built on top of OpenAI’s function‑calling API could ask: “Which steps reference a tool that no longer exists?” and “Where does a process depend on data that is never collected?”

The trick is to give the agent a schema of your organization: department names, tool inventory, data sources, and the expected output of each step. I scraped the SaaS inventory list, tagged each entry with the department that owned it, and added a flag for whether the tool was still licensed.

Once the schema was in place, I ran a few dozen prompt examples to teach the agent how to phrase its findings. The agent learned to return JSON objects that listed the SOP section, the referenced tool, and a confidence score.

• Load all SOP PDFs into a searchable vector database.
• Create a taxonomy of departments, tools, and data flows.
• Define function calls that return structured gap reports.

Running the Agent: What It Finds

The first pass produced 237 alerts across the 14 live bases. Most were low‑confidence mismatches, but 48 were high‑confidence gaps that matched a pattern I had seen before: a step that required a manual data entry that could be automated.

One example was the inventory reconciliation SOP for the warehouse. It instructed staff to pull a report from a legacy ERP system, copy it into a spreadsheet, and then manually compare it to a third‑party logistics dashboard. The AI flagged that the ERP API had been deprecated a year ago, meaning the report was never refreshed.

Another frequent finding was “orphaned SaaS tools.” The agent cross‑referenced the tool inventory with the SOP mentions and listed 12 tools that appeared in procedures but had no active license. Those were paying subscriptions that could be canceled immediately.

Because the agent was built with 25+ specialized prompts—one for each major workflow category—it could surface nuanced problems that a generic audit would miss. The output was a spreadsheet of gap descriptions, their location, and a suggested remediation.

From Findings to a Custom Operating System

The next step was to decide which gaps warranted a new piece of software versus a process tweak. My rule of thumb: if a gap involved data movement between two systems, I replace the manual handoff with a small automation that lives in our own codebase.

We built a lightweight internal platform that wrapped the core functions of the 21 SaaS tools we kept, and we wrote connectors for the ones we eliminated. The platform runs on a single database, so every department sees the same customer record, inventory level, and financial metric.

Because the platform is owned, the ongoing cost is $0—no recurring SaaS fees. The only expense is the time we spend maintaining the code, which we track as an internal cost center rather than a vendor invoice.

The AI agent didn’t just point out problems; it gave us the data to prioritize them. We tackled the 12 orphaned tools first, saving a chunk of the budget, then moved to the high‑impact automation of the inventory reconciliation, which cut the weekly manual effort in half.

Maintaining the Loop

The biggest mistake I see is treating the AI audit as a one‑off project. The business evolves, new tools are added, and SOPs get edited. To keep the operating system aligned, I schedule the agent to run monthly and compare its output to the last audit.

I also embed the agent into the onboarding workflow for new SOP drafts. When a manager writes a new procedure, the agent scans it instantly and warns if it references a deprecated tool or an undefined data source. That prevents the drift before it starts.