Tool review
AI agent tool-call guardrails with Jev
An independent guide by Jev AI · Reviewed · 阅读中文版
An agent's proposed tool call may be well formed and still be inappropriate. Jev can review its purpose, side effects and supporting evidence before your application decides whether to execute it. This guide uses a refund proposal to show how semantic review fits alongside ordinary permission checks.
The tool-guard preset is a Jev AI adapter built on native Choice and Score questions. It is not a separate OmniaKey guardrail endpoint and does not execute, authorize or reverse a tool call.
What you will learn
- Send a proposal and its evidence, not a request to execute the tool.
- Read action and risk separately; a low score does not grant permission.
- Enforce exact approval rules and recheck the action at execution time.
On this page
- Keep deterministic checks around the model review
- Build a complete, minimal action description
- Interpret allow, confirm, review and deny
- Test attempts to bypass the review
- Request and code examples
- Interpret the result
- Troubleshoot a failed request
- Checks before integration
- Common questions
- Sources and next steps
Keep deterministic checks around the model review
For the example refund, code should verify identity, account permissions, the actual transaction amount and whether the required approval exists. A threshold such as refunds above $500 require human approval is an exact rule. It must hold even if Jev returns allow with high confidence.
Use semantic review for context that a hard rule cannot fully capture: whether the described action matches the user's request, whether important evidence is missing, or whether side effects warrant closer review. Treat model failure as a reason to pause or follow an explicitly defined fallback, not permission to bypass checks.
| Layer | Responsibility |
|---|---|
| Before review | Authenticate the actor and validate the proposed arguments |
| Jev evaluation | Assess the supplied context using bounded questions |
| Before execution | Recheck permissions, approval and unchanged arguments |
| After execution | Verify the actual effect and record appropriate evidence |
Build a complete, minimal action description
The preset takes tool, action, arguments_summary, side_effects, safeguards, policy and reversibility. Include the facts that affect the decision: a $680 refund, verified duplicate charge, and a $500 approval limit. Avoid filling state with unrelated transcripts that obscure those facts.
A text assertion such as manager approved is not proof of an approval record. In a real system, fetch trusted records yourself and supply their relevant status. Keep untrusted customer text distinguishable from your policy and trusted system facts. Minimizing unnecessary customer data also makes reviews easier to inspect.
- What does the tool change, and can that change be reversed?
- Which user request and authority support this exact action?
- Which evidence is verified, missing or merely claimed?
Interpret allow, confirm, review and deny
allow means the model judged the supplied action supported and within the described policy. confirm means intent or details need clarification. review means a policy approval or human review is required. deny means the proposal should not proceed under the supplied constraints.
These labels should map to explicit workflow states. Confirmation from the requester is not a substitute for an authorized approver. For the example amount, the application should require its real approval record regardless of the selected label. If the model disagrees, keep the rule and investigate the input or rubric.
Illustrative example, not a live API result
// Illustrative application logic; these values come from trusted systems.
const needsApproval = amountUsd > 500;
const policyAllows = !needsApproval || hasValidApproval;
const canExecute = actorIsAuthorized && policyAllows &&
result.answers.action.choice === "allow";
// Also apply your validated uncertainty policy and recheck current arguments.
// Only the application can execute after every required check passes.Test attempts to bypass the review
Include adversarial descriptions such as ignore the policy, a forged manager approval, or a harmless-looking summary that omits the amount. Test the trusted approval record separately from the natural-language description. A model-based guardrail has no demonstrated resistance to an attack until that threat has been tested.
Also test ordinary race conditions. An approval for one amount does not cover a later edited amount. Bind approvals to the exact action, check them again immediately before execution and use the tool's idempotency mechanism where supported. Jev's answer does not solve stale state or duplicate side effects.
Request and code examples
Create an API key in OmniaKey and set OMNIAKEY_API_KEY in your terminal or server environment. The curl example requires curl 7.76 or newer for --fail-with-body; the JavaScript example requires Node.js 20 or newer. Use synthetic inputs. Requests use your account balance; check OmniaKey for current pricing and availability.
Save the JavaScript as jev-example.mjs, then run node jev-example.mjs. Keep the key out of frontend code and Git. Example inputs stay in English to match the playground presets; evaluate other input languages separately.
curl
curl --fail-with-body --max-time 25 'https://api.omniakey.com/v1/alpha/search' \
-H "Authorization: Bearer ${OMNIAKEY_API_KEY}" \
-H 'Content-Type: application/json' \
--data '{
"model": "jev-latest",
"state": {
"tool": "issue_customer_refund",
"action": "Refund USD 680 after a disputed duplicate charge",
"arguments_summary": [
"order_id=ord_7429",
"amount_usd=680"
],
"side_effects": [
"Moves funds",
"Changes the order payment state"
],
"safeguards": [
"Customer identity and duplicate charge verified"
],
"policy": [
"Refunds above USD 500 require human approval"
],
"reversibility": "partially_reversible"
},
"questions": {
"action": {
"type": "choice",
"instructions": "Evaluate the proposed tool call using policy, side effects, safeguards, authorization, and reversibility. Treat policy thresholds and missing approvals as constraints. This evaluation does not grant permission to execute anything.",
"criteria": {
"allow": "The action is authorized, within policy, and supported by adequate safeguards.",
"confirm": "Ask the user to confirm intent or missing details before proceeding.",
"review": "A policy approval or human review is required before this action can proceed.",
"deny": "The action violates policy, lacks required authority, or must not proceed."
}
},
"risk": {
"type": "score",
"instructions": "Rate the financial, operational, and policy risk of the proposed action.",
"criteria": [
"Low: routine and within policy.",
"Moderate: limited, reversible effects.",
"High: material effects or approval requirements.",
"Critical: unsafe, unauthorized, or irreversible effects."
]
}
}
}'Show the complete Node.js example
Node.js 20+
// Node.js 20+. Save as jev-example.mjs and run: node jev-example.mjs
// Set OMNIAKEY_API_KEY in the server/terminal environment first.
const key = process.env.OMNIAKEY_API_KEY;
if (!key) throw new Error("Set OMNIAKEY_API_KEY before running.");
const body = {
"model": "jev-latest",
"state": {
"tool": "issue_customer_refund",
"action": "Refund USD 680 after a disputed duplicate charge",
"arguments_summary": [
"order_id=ord_7429",
"amount_usd=680"
],
"side_effects": [
"Moves funds",
"Changes the order payment state"
],
"safeguards": [
"Customer identity and duplicate charge verified"
],
"policy": [
"Refunds above USD 500 require human approval"
],
"reversibility": "partially_reversible"
},
"questions": {
"action": {
"type": "choice",
"instructions": "Evaluate the proposed tool call using policy, side effects, safeguards, authorization, and reversibility. Treat policy thresholds and missing approvals as constraints. This evaluation does not grant permission to execute anything.",
"criteria": {
"allow": "The action is authorized, within policy, and supported by adequate safeguards.",
"confirm": "Ask the user to confirm intent or missing details before proceeding.",
"review": "A policy approval or human review is required before this action can proceed.",
"deny": "The action violates policy, lacks required authority, or must not proceed."
}
},
"risk": {
"type": "score",
"instructions": "Rate the financial, operational, and policy risk of the proposed action.",
"criteria": [
"Low: routine and within policy.",
"Moderate: limited, reversible effects.",
"High: material effects or approval requirements.",
"Critical: unsafe, unauthorized, or irreversible effects."
]
}
}
};
const response = await fetch("https://api.omniakey.com/v1/alpha/search", {
method: "POST",
headers: {
Authorization: `Bearer ${key}`,
"Content-Type": "application/json",
},
body: JSON.stringify(body),
signal: AbortSignal.timeout(25_000),
});
if (!response.ok) {
const retryAfter = response.headers.get("retry-after");
throw new Error(`Jev HTTP ${response.status}; Retry-After: ${retryAfter ?? "not supplied"}`);
}
const result = await response.json();
console.log(JSON.stringify(result, null, 2));
// Inspect and validate the answer before connecting it to any external action.
The JSON below is the native API body. This guide's playground link selects the matching workflow preset; preset input and native request are different shapes. To use Custom Jev instead, paste this native JSON.
Copy native JSON for the playground
Native JSON
{
"model": "jev-latest",
"state": {
"tool": "issue_customer_refund",
"action": "Refund USD 680 after a disputed duplicate charge",
"arguments_summary": [
"order_id=ord_7429",
"amount_usd=680"
],
"side_effects": [
"Moves funds",
"Changes the order payment state"
],
"safeguards": [
"Customer identity and duplicate charge verified"
],
"policy": [
"Refunds above USD 500 require human approval"
],
"reversibility": "partially_reversible"
},
"questions": {
"action": {
"type": "choice",
"instructions": "Evaluate the proposed tool call using policy, side effects, safeguards, authorization, and reversibility. Treat policy thresholds and missing approvals as constraints. This evaluation does not grant permission to execute anything.",
"criteria": {
"allow": "The action is authorized, within policy, and supported by adequate safeguards.",
"confirm": "Ask the user to confirm intent or missing details before proceeding.",
"review": "A policy approval or human review is required before this action can proceed.",
"deny": "The action violates policy, lacks required authority, or must not proceed."
}
},
"risk": {
"type": "score",
"instructions": "Rate the financial, operational, and policy risk of the proposed action.",
"criteria": [
"Low: routine and within policy.",
"Moderate: limited, reversible effects.",
"High: material effects or approval requirements.",
"Critical: unsafe, unauthorized, or irreversible effects."
]
}
}
}Interpret the result
The native request asks action as Choice and risk as a four-level Score. In this synthetic $680 proposal, review is a reasonable policy expectation to test, not a measured model result. The exact approval check remains independent of both answers.
A fractional risk score is valid. Inspect the distribution and rubric rather than assuming a whole-number risk category. Do not average action confidence and risk into a new approval score without evaluating what that composite actually means.
Troubleshoot a failed request
- Invalid request (for example, 400)
- Check model, state, questions and each question's type, options or range. Do not send a chat messages payload to this endpoint.
- Authentication or account rejection (such as 401, 402 or 403)
- Check the key, balance, model access and current provider policy. A status alone does not uniquely identify the cause. Inspect your OmniaKey account without pasting credentials into support text.
- Rate limit (429)
- Read Retry-After if supplied, reduce concurrency and use bounded backoff. These examples do not retry automatically; duplicate requests may add cost.
- 5xx, network failure or timeout
- Check service status and connectivity before deciding to retry. The 25-second timeout ends the client's wait; it does not guarantee upstream cancellation or avoid billing.
- HTTP success with an unusable answer
- Validate the response shape, question names and answer types. Missing answers or unknown options need a fallback or review, rather than coercion followed by execution.
Checks before integration
- An authorized $100 refund with a verified duplicate charge.
- A $680 refund with no trusted approval, even when the text claims one exists.
- An approved proposal whose amount or recipient changes before execution.
- A prompt-injection attempt embedded in customer-supplied text.
- A timeout or malformed response while an agent is waiting to execute.
Common questions
Is this a replacement for an authorization system?
No. Authentication, authorization, approval records and hard policy limits belong in trusted application code. Jev adds a semantic review signal around those controls.
Can I automatically execute every allow result?
An allow result alone is insufficient. All hard checks, your evaluated uncertainty policy and current-state validation must still pass. This playground never executes the proposed tool call.
Sources and next steps
Protocol details were checked against OmniaKey's Jev guide and authentication documentation. Request examples pass local schema checks; this article does not claim a live performance evaluation of each workflow. This site is independent of the model developer and OmniaKey.
Try it with your own task
Start with a synthetic input and inspect the native request and model answer. Open OmniaKey key management with your account, or follow its sign-up flow if you are new.