A smooth demonstration can answer one question: can this sequence look convincing under the conditions shown? An operator needs answers to several others. What triggers the sequence? Which systems can it change? What happens when information is missing? How does a person stop or repair the work? Those questions turn an entertaining presentation into an evaluation.
The point is not to make every demo longer. It is to choose evidence that matches the decision. A buyer considering a small pilot may need a clear task boundary and a safe test plan. A team approving wider use needs a more complete account of permissions, evaluation, and ongoing ownership. Start by stating which decision the demonstration is meant to support.
Name the job before naming the agents
Write the workflow as a sentence with an input and an observable output. An illustrative example is “Turn an approved set of meeting notes into a draft customer follow-up for review.” That sentence establishes what enters the workflow, what leaves it, and where a person still makes a decision.
Next, list the actual steps. The system may retrieve a record, extract relevant facts, prepare a draft, and present it to a reviewer. Whether those steps involve one model, several agents, or ordinary application code is an implementation question. The evaluator first needs to understand the work and its boundaries.
If the demo starts with an animated network of agents, ask to see one request move through the sequence. Keep the original input visible. A useful explanation should connect each stage to a reason it exists. A stage that cannot be explained in terms of the task may not belong in the first evaluation.
Ask what the system is allowed to do
A tool’s name does not tell you its permissions. A connection called “documents” might read one folder or modify an entire repository. Ask for the operations granted to the workflow and the identity under which those operations occur. The distinction between preparing a draft and sending it should be explicit.
OWASP’s excessive agency guidance describes problems that can arise from unnecessary functionality, permissions, and autonomy. Use it as a prompt to inspect the actual setup. A statement that a system follows best practices is less useful than a concrete list of what it can read and change.
For a first pilot, a narrower permission set may make the result easier to evaluate. If the task is draft preparation, ask why it needs sending authority. If the task is finding a record, ask why it needs deletion rights. The answers should follow from the task, not from what a connector happens to offer by default.
Use a small set of contrasting inputs
Prepare several examples that expose different behavior. Include an ordinary input, one with a required detail missing, one with contradictory information, and one outside the intended scope. Keep personal or confidential material out of the evaluation unless its use has been properly authorized and controlled.
The examples should be selected before the demo. Otherwise, it is easy to keep choosing inputs that happen to work. Record the expected behavior in plain language. For missing information, success may mean asking a clear question or stopping for review. It should not automatically mean producing a complete-looking answer.
Run the same examples after a material change. A new prompt, connector, or model configuration can affect behavior in ways a polished single example will not reveal. The OWASP AI Agent Security Cheat Sheet includes structured testing among its security practices. Testing still needs to be designed for the particular workflow and its risks.
Inspect the review step closely
A review button is only useful if the reviewer has enough information to make the decision. Show the proposed action, the relevant source, and the consequence of approval. If a reviewer must open several unrelated systems to reconstruct the context, the workflow may be moving effort rather than reducing it.
In the follow-up example, the reviewer should see which meeting notes informed the draft and which recipient is proposed. If the draft includes a date that does not appear in the notes, that discrepancy should be easy to identify. A generic confidence indicator cannot replace the source material needed to check a statement.
Also ask what happens after rejection. Can the reviewer edit the draft? Does the system preserve the rejected version? Is a new approval required after a material change? These details matter because a reviewer approves a particular action or artifact, not an unlimited future sequence.
Interrupt the happy path
Ask the presenter to show a failed connection or an unavailable source. Observe whether the workflow stops, retries, or continues with less information. None of those responses is universally correct. The question is whether the response is appropriate to the task and visible to the operator.
Then ask what happens when the same request arrives twice. If the workflow can create records or send messages, duplicate handling deserves attention. The evaluator does not need to dictate a technical mechanism, but should understand whether a repeat request can cause a repeated external action and how that situation is managed.
Finally, identify the stop control. Who can pause the workflow, and what does pausing mean for work already underway? An operator needs to know whether pending actions remain queued, are canceled, or require a separate decision. A clear stopping behavior is part of the product’s practical usefulness.
Measure the whole task
Speed is easy to demonstrate and easy to misunderstand. Measure the time spent reviewing, correcting, and resolving exceptions as well as the time spent generating an output. A draft produced quickly may still be costly if the reviewer must verify every detail from scratch.
Choose evaluation criteria that match the task. For a draft follow-up, those might include preservation of source facts, omission of unsupported commitments, correct recipient selection, and reviewer changes. State how each criterion will be assessed. Avoid combining everything into a single success percentage that hides different kinds of failure.
Compare the workflow with the current method using similar examples. The comparison may reveal that the new approach helps with one part of the job but complicates another. That is useful evidence for deciding where to pilot it. It is not a reason to convert an early observation into a general productivity claim.
Leave the room with an operator checklist
A short evaluation record should answer these questions:
- What exact request is in scope, and what output counts as complete?
- Which sources and tools are available, with which permissions?
- What must a person approve, and what information supports that approval?
- How do missing data, duplicate requests, and unavailable tools behave?
- Who owns the workflow, its evaluation examples, and its stop control?
- What evidence is still needed before the next level of use?
The record should include unresolved questions rather than smoothing them away. A team can make a sensible decision to run a limited pilot with known gaps. It cannot make a sensible decision if those gaps are hidden behind a successful demonstration.
Before the next pitch, remove one slide that merely dramatizes activity and replace it with one failed-input example. Show the original request, the point where work stopped, and the next action available to the operator. That small change will make the presentation more useful to the person who has to run the workflow after the meeting ends.