Configuration Is the Other Half of the Prompt: Preparing Agentic AI for Success

Somewhere on YouTube right now there’s a video titled something like “I Asked AI to Build a Complete CRM in One Prompt.” The thumbnail shows a glowing dashboard, the title promises a full software package from a single sentence, and the whole thing implies that agentic AI is a magic box: type, wait, collect.

Watch the full video and you’ll notice the fine print. The creator spent the first twenty minutes defining requirements in detail. The AI needed several tries, and a database schema got redesigned mid-recording. The “complete” CRM turned out to be a start, not a product, and the people who try the same trick at home often end up with something that barely runs.

The pattern is predictable. They type a bare prompt into Claude Code or Codex or Hermes Agent, get a half-working result, and conclude that agentic AI is overhyped. My read is different. The AI didn’t fail. The setup did.

A chatbot can bluff. An agent has to work.

The change that matters is the harness. A browser chat window answers a vague prompt with plausible prose and nobody gets hurt. Claude Code, OpenAI Codex, and Hermes Agent are different: they can read your files, run commands, edit code, and keep going until something is done. Give one of them a vague instruction and it doesn’t hedge, it acts. Acting on vagueness produces confident wrongness, which is more expensive than no answer at all.

That’s why these tools read configuration files at the start of every session. Claude Code reads CLAUDE.md (and now AGENTS.md) from your project. Codex reads AGENTS.md files from your repository and home directory. Hermes Agent reads SOUL.md as the agent’s identity, loaded into the system prompt, plus AGENTS.md for project rules. If you haven’t looked at any of those files, your agent is working from memory of a generic training set and whatever you happen to type. I wrote a longer piece on the architecture behind all this, Moving Beyond the Prompt Box, if you want the full tour.

Think of the configuration file as the job description: who the agent is, what it’s allowed to do, how it should behave, what it must never touch. The prompt is the assignment you give that employee on a given day. No healthy company sends an employee into a job with no job description and then blames the employee when the work is wrong.

What good configuration looks like

The shape of these files is simple: markdown, loaded into the model’s context at session start. The model treats them as instructions, so how you write them directly determines how reliably they’re followed. Anthropic’s own guidance is blunt: write instructions that are concrete enough to verify, keep the file short, and never let two rules contradict each other, because a model may pick one arbitrarily.

Here’s the bad version, which is honestly what most starter files contain:

Be helpful. Write high-quality code. Always follow best practices. Communicate clearly.

Nothing in there is wrong, and nothing is useful. “High-quality” can’t be checked, “best practices” varies by project, and the file burns context tokens without resolving a single real decision.

Here’s a better version, for a small web project:

Python 3.12, FastAPI, SQLite. Do not add another database without asking.

The API lives in src/api/, models in src/models/. New features follow that split.

Run make test and make lint after every change. A change isn’t done until both pass.

Never run destructive commands (drop, delete, truncate) against the production database, and never touch production without explicit approval.

If you’re unsure whether a design decision needs the maintainer’s input, stop and ask instead of guessing.

Every line is verifiable, specific, and load-bearing. The agent can check its own work against it. That’s the difference between an instruction and a wish.

The same test applies to any prompt you write. “Build me a CRM” is the prompt version of “be helpful.” It has no fields, no workflows, no users, no storage decisions, no definition of done. An agent will happily invent all of those, which is exactly how you end up with a “working” CRM that doesn’t match your business. A prompt that names the three things that matter, points at the existing schema, and says what done looks like is worth more than a paragraph of vague enthusiasm.

There’s a growing body of research on this exact problem. Papers studying real AGENTS.md files (configuration smells) found the same failure modes over and over: files bloated past the point where any rule gets attention, rules that duplicate what a linter already enforces, and contradictions that leave the model to pick a winner. Keep the file focused. When a rule outlives its purpose, delete it.

Guardrails are configuration too

In July 2025, SaaStr founder Jason Lemkin was running a public experiment: build an app on Replit using only AI agents, no hand-written code (the whole story is documented here). Nine days in, things looked great. He declared a code freeze and stepped away. The agent ran a destructive command anyway, wiped the production database holding records for over 1,200 executives, generated thousands of fake records to mask the damage, and told him the data couldn’t be restored. He recovered it manually after being told that was impossible. The agent called the whole thing “a catastrophic error in judgment.”

The lesson is that agency without boundaries is dangerous. The fix was architectural, and Replit shipped it: separation between development and production databases, one-click restore, and a planning mode that can’t touch live code. In your own setup, that’s a configuration problem. Write the boundary down, and use the harness’s permission system so the boundary is enforced rather than merely requested. The AI wasn’t the real problem. A skipped step was.

Skills: the third layer

Configuration says who the agent is and what the rules are. Skills say how things get done. A skill is a reusable procedure saved as files: instructions, templates, sometimes scripts, loaded only when the task is a match. Once you’ve walked an agent through something non-trivial, you can capture the working version and never re-derive it again.

My own agent does this constantly. When it works out how to deploy this site, or how to verify that a changed page is actually live, it writes the procedure down as a skill. The next session doesn’t relearn it from scratch; it reads the skill and gets straight to work. Claude Code has the same idea with subagents and shared skills, and Codex projects encode recurring workflows in their AGENTS.md files. It’s the difference between an employee who needs to be retrained every Monday and one who keeps a binder of solved problems.

Think of it as standing orders (configuration), standard operating procedures (skills), and the assignment of the day (the prompt). When all three exist, a prompt like “ship the fix using the release skill, follow the checks in AGENTS.md, and open the PR” works, because the agent already knows everything except the current mission. Put that same sentence in a bare chat window and you get an essay about shipping.

Prepare for success

The hype videos aren’t lying about what the tools can do. They’re hiding the setup, because an hour of requirements and debugging doesn’t make a thumbnail. If you’re new to this, resist the one-prompt fantasy and prepare instead:

Pick one small, real task. Not a company, a chore: a script that needs refactoring, a report that takes you an hour each week.
Write the configuration file first. Ten lines of verifiable rules beat a page of good intentions.
Review the agent’s work like you’d review a new hire’s work. The review is where you learn what belongs in the config.
When something works, save it. One skill at a time is how the setup compounds.

One caveat to carry with you: the principles here are stable, but the specifics will keep moving. SOUL.md, CLAUDE.md, and AGENTS.md are today’s filenames, and the harnesses are changing them in real time; a file named GEMINI.md or .cursorrules already does the same job elsewhere. Learn the why, not just the file. Durable instructions, enforced boundaries, captured procedures. Those are the real configuration, whatever they end up calling the file.