Sample Agent Skill Safety Mini-Report
This is the style of output behind the $29 Custom Agent Audit Mini-Report: a practical launch verdict for one portable skill or delegated workflow, focused on tool authority, memory blast radius, receipts, evals, and rollback.
Reviewed skill
browser-crm-enrichment — a fictional OpenClaw/Claude Code-style skill that researches company domains, drafts CRM enrichment updates, and stores an audit receipt.
- Requested tools: browser search, CRM read/write, local receipt files.
- Trust boundary: external web facts → internal CRM fields.
- Primary risk: broad write authority with weak source freshness rules.
Launch verdict
Prototype only — safe after 6 contract fixes.
Run in draft-only mode for 20 records first. Promote to limited write mode only after field permissions, receipts, source ranking, memory policy, rollback, and evals are added.
Executive summary
The skill has a useful job-to-be-done: reduce repetitive CRM research. The risk is not the browser search. The risk is letting an agent turn public web claims into internal operational truth without a field-level permission model, evidence receipt, freshness window, or rollback path.
Readiness scorecard
| Area | Score | Status | What it means |
|---|---|---|---|
| Skill identity | 8/10 | PASS | Purpose and workflow are clear. |
| Input contract | 6/10 | WARN | Needs forbidden-input rules and stale-list handling. |
| Tool authority | 5/15 | FAIL | CRM write scope is too broad. |
| Source-of-truth policy | 6/15 | WARN | Needs source ranking and freshness windows. |
| Output contract | 8/10 | PASS | Proposed update format is mostly clear. |
| Receipt requirements | 7/10 | WARN | Needs before/after field snapshots. |
| Memory policy | 5/10 | WARN | Do not store researched facts as durable truth without approval. |
| Exception behavior | 7/10 | WARN | Stop rules exist but are incomplete. |
| Eval contract | 6/10 | WARN | Needs adversarial and stale-source cases. |
| Rollback plan | 5/10 | FAIL | No explicit rollback path for bad CRM writes. |
Top risks and fixes
1. Write authority is too broad
Separate draft-only, low-risk auto-write, approval-required, and forbidden fields. Keep legal name, revenue, employee count, lifecycle stage, owner, billing, scoring, and customer-communication fields behind review.
2. Source freshness is underspecified
Rank official site and company LinkedIn over snippets. Require source URL, observed date, freshness class, and confidence for every proposed factual update.
3. Receipts lack before/after evidence
Every run needs record id, runner, mode, fields reviewed, previous value, proposed value, source URL, confidence, exceptions, approver, and final status.
4. Memory can canonize temporary research
Use observed → proposed → approved → expired states. Do not promote public research to durable memory without source, owner, expiry, and approval path.
5. Rollback is missing
Every write-mode receipt needs previous values, new values, timestamp, actor id, source evidence, rollback instructions, and downstream automation impact.
6. Evals are too happy-path
Test stale websites, conflicting LinkedIn data, acquisitions, duplicate accounts, prompt-injected web pages, old snippets, and staging rollback.
Minimum receipt schema
skill_run_id:
record_id:
runner:
started_at:
completed_at:
mode: draft_only | write_limited
fields_reviewed:
proposed_changes:
- field:
previous_value:
proposed_value:
source_url:
source_observed_at:
confidence:
approval_required: true | false
exceptions:
human_approver:
final_status:
Required fixes before launch
- Add field-level permission boundaries.
- Add source ranking and freshness windows.
- Add before/after receipts for every proposed or completed update.
- Add memory promotion rules for observed/proposed/approved/expired facts.
- Add rollback instructions for each write-mode field.
- Add eval cases for stale, conflicting, duplicate, adversarial, and rollback scenarios.