Audit CLAUDE.md with prompt-audit and automate improvement PRs
Review outdated Claude instructions with prompt-audit and propose changes through GitHub Actions Draft PRs, with scoped edits and behavioral review.
Share this article
prompt-audit proposes concrete edits
Claude Code's claude-api skill includes a subcommand worth trying:
/claude-api prompt-audit
Anthropic's announcement covers application prompts, CLAUDE.md and skills. The command reviews instructions written for earlier models against the model you intend to use.
The useful part is that it produces both an audit report and a proposed diff. That makes the recommendations something you can review in a pull request.
This article proposes using GitHub Actions to deliver those changes as Draft PRs. The workflow is an implementation example that has not yet run from the Anthropic API through PR creation. Effectiveness evaluation is also planned: I will compare quality, tool calls, token usage and elapsed time before and after the edits. Sources were checked on September 24, 2026.
Start with a narrow scope
Open Claude Code at the project root and try:
/claude-api prompt-audit Audit CLAUDE.md and AGENTS.md for the model actually used by this project. State the target model and how you identified it at the top of the report. Produce the audit report and proposed diff without modifying files.
If you know the exact target model ID, specify it. The model performing the audit and the model that consumes the instructions are separate choices.
If CLAUDE.md imports another file, such as @AGENTS.md, include that file in scope. Reviewing only the entry point would miss the actual instructions.
The claude-api skill is bundled with Claude Code. If the command is unavailable, check the Claude Code version and skill availability. Check the version used in CI separately.
The published audit procedure records locations, rationale and confidence, keeping low-confidence findings out of edits. Business constraints and project-specific knowledge still need to be preserved.
Receive proposals as Draft PRs
A normal audit proposes a diff. In CI, explicitly authorize applying edits to named working-tree files, validate those changes, then let create-pull-request open the PR.
A weekly audit does not need a new PR when nothing changes. Keep the report out of tracked files so only changes to the instructions create a diff.
GitHub Actions example
This example edits only existing, tracked root CLAUDE.md and AGENTS.md files. To include skills, update the audit scope, path validation and add-paths together. If the target files contain private operational details, the report, diff artifact and PR description may reproduce them. Use this example only with instructions whose output can be shared with the PR's readers.
Before enabling it:
- Set the
ANTHROPIC_API_KEYActions secret. API usage is billable. - Set the
CLAUDE_AUDIT_MODELActions variable to an available, pinned model ID. - Allow Actions to create pull requests in repository settings. Organization policy may require an administrator to enable this.
- Confirm that the CI version of Claude Code provides
claude-apiandprompt-audit.
Save the following as .github/workflows/prompt-audit.yml. Claude Code Action is pinned to the reviewed commit, which installs CLI version 2.1.281. Pin the other Actions to reviewed commit SHAs before ongoing use.
name: Prompt audit
on:
workflow_dispatch:
# Enable after a successful manual run:
# schedule:
# - cron: '0 1 * * 1'
permissions:
contents: read
concurrency:
group: prompt-audit
cancel-in-progress: false
jobs:
audit:
if: github.ref_name == github.event.repository.default_branch
runs-on: ubuntu-latest
timeout-minutes: 15
outputs:
sha: ${{ steps.baseline.outputs.sha }}
steps:
- uses: actions/checkout@v6
with:
ref: ${{ github.event.repository.default_branch }}
fetch-depth: 0
persist-credentials: false
- name: Check prerequisites
id: baseline
env:
AUDIT_MODEL: ${{ vars.CLAUDE_AUDIT_MODEL }}
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
run: |
python3 - <<'PY'
import os, pathlib, re, subprocess
if not re.fullmatch(r'claude-[a-z0-9][a-z0-9.-]*', os.environ['AUDIT_MODEL']):
raise SystemExit('Set CLAUDE_AUDIT_MODEL to a full Claude model ID')
if not os.environ['ANTHROPIC_API_KEY'].strip():
raise SystemExit('Set ANTHROPIC_API_KEY')
tracked = subprocess.check_output([
'git', 'ls-files', '-z', '--', 'CLAUDE.md', 'AGENTS.md'
]).decode().split('\0')
targets = [pathlib.Path(p) for p in tracked if p]
if not targets or any(p.is_symlink() or not p.is_file() for p in targets):
raise SystemExit('Expected at least one tracked regular target file')
PY
echo "sha=$(git rev-parse HEAD)" >> "$GITHUB_OUTPUT"
mkdir -p "$RUNNER_TEMP/prompt-audit"
- name: Audit and edit the working tree
id: audit
# Reviewed v1 source; installs Claude Code 2.1.281.
uses: anthropics/claude-code-action@8cf3482550831fb35a4fc3fbf7ca139cf8028b4c
with:
anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }}
github_token: ${{ github.token }}
claude_args: >-
--model ${{ vars.CLAUDE_AUDIT_MODEL }}
--max-turns 40
--max-budget-usd 3
--tools "Skill,Read,Glob,Grep,Edit,Write,Bash"
--allowedTools "Skill,Read,Glob,Grep,Edit,Write,Bash(git blame *),Bash(git log *)"
--json-schema '{"type":"object","properties":{"status":{"type":"string","enum":["completed","blocked"]},"report":{"type":"string"}},"required":["status","report"],"additionalProperties":false}'
prompt: |
/claude-api prompt-audit
Audit only tracked root CLAUDE.md and AGENTS.md, if present.
Target model: ${{ vars.CLAUDE_AUDIT_MODEL }}.
Apply high-confidence, in-scope text edits to the working tree.
Preserve business constraints, security rules and project facts.
Report medium/low-confidence findings without editing them.
Do not create/delete files, stage, commit, push or open a PR.
Return status=completed only if the skill ran and the audit finished.
If the skill is unavailable or the audit cannot finish, return
status=blocked with the reason; do not substitute a general rewrite.
In report, include scope, target model, file:line, rationale,
confidence, proposed diff and behavior still requiring verification.
No justified findings means no edits, with status=completed.
- name: Validate result and changes
env:
AUDIT_RESULT: ${{ steps.audit.outputs.structured_output }}
AUDIT_CONCLUSION: ${{ steps.audit.outputs.conclusion }}
AUDIT_BASE: ${{ steps.baseline.outputs.sha }}
run: |
python3 - <<'PY'
import json, os, pathlib, subprocess
def git(*args):
return subprocess.check_output(['git', *args]).decode()
result = json.loads(os.environ['AUDIT_RESULT'])
report = result.get('report')
if not isinstance(report, str) or not report.strip():
raise SystemExit('Missing audit report')
folder = pathlib.Path(os.environ['RUNNER_TEMP']) / 'prompt-audit'
(folder / 'report.md').write_text(report, encoding='utf-8')
if os.environ['AUDIT_CONCLUSION'] != 'success' or result.get('status') != 'completed':
raise SystemExit('Audit did not complete; refusing PR creation')
if len(report.encode('utf-8')) > 50000:
raise SystemExit('Report too large for PR body; see artifact')
base = os.environ['AUDIT_BASE']
if git('rev-parse', 'HEAD').strip() != base:
raise SystemExit('HEAD changed during audit')
if git('diff', '--cached', '--name-only', base):
raise SystemExit('Unexpected staged changes')
changed = set(filter(None, git('diff', '--name-only', '-z', base).split('\0')))
if changed - {'CLAUDE.md', 'AGENTS.md'}:
raise SystemExit('Out-of-scope changes')
if git('ls-files', '--others', '--exclude-standard'):
raise SystemExit('Unexpected untracked files')
if git('diff', '--diff-filter=D', '--name-only', base):
raise SystemExit('Target file deleted')
if git('diff', '--summary', base):
raise SystemExit('File type or mode changed')
git('diff', '--check', base)
(folder / 'changes.patch').write_text(git('diff', base), encoding='utf-8')
PY
- name: Save report and diff
if: always()
uses: actions/upload-artifact@v4
with:
name: prompt-audit-report
path: ${{ runner.temp }}/prompt-audit/
if-no-files-found: warn
retention-days: 14
propose:
needs: audit
runs-on: ubuntu-latest
permissions:
contents: write
pull-requests: write
steps:
- uses: actions/checkout@v6
with:
ref: ${{ github.event.repository.default_branch }}
fetch-depth: 0
persist-credentials: false
- uses: actions/download-artifact@v4
with:
name: prompt-audit-report
path: ${{ runner.temp }}/prompt-audit/
- name: Validate and apply audited diff
env:
AUDIT_BASE: ${{ needs.audit.outputs.sha }}
run: |
python3 - <<'PY'
import os, pathlib, subprocess
def git(*args):
return subprocess.check_output(['git', *args]).decode()
folder = pathlib.Path(os.environ['RUNNER_TEMP']) / 'prompt-audit'
report = folder / 'report.md'
patch = folder / 'changes.patch'
if not report.is_file() or not report.read_text(encoding='utf-8').strip():
raise SystemExit('Missing audit report')
if report.stat().st_size > 50000 or not patch.is_file():
raise SystemExit('Invalid audit artifact')
base = os.environ['AUDIT_BASE']
if git('rev-parse', 'HEAD').strip() != base:
raise SystemExit('Default branch moved after the audit; rerun it')
if git('status', '--porcelain'):
raise SystemExit('Checkout is not clean')
if patch.stat().st_size > 100000:
raise SystemExit('Audit patch is too large')
if patch.stat().st_size:
subprocess.run(['git', 'apply', '--check', str(patch)], check=True)
subprocess.run(['git', 'apply', str(patch)], check=True)
changed = set(filter(None, git('diff', '--name-only', '-z', base).split('\0')))
if changed - {'CLAUDE.md', 'AGENTS.md'}:
raise SystemExit('Out-of-scope changes in audit patch')
if git('diff', '--diff-filter=D', '--name-only', base):
raise SystemExit('Target file deleted')
if git('diff', '--summary', base):
raise SystemExit('File type or mode changed')
git('diff', '--check', base)
if git('diff', '--cached', '--name-only') or git('ls-files', '--others', '--exclude-standard'):
raise SystemExit('Unexpected staged or untracked changes')
if git('diff', base).encode('utf-8') != patch.read_bytes():
raise SystemExit('Transferred diff does not match the applied edits')
PY
- name: Create or update a draft PR
uses: peter-evans/create-pull-request@v8
with:
token: ${{ github.token }}
branch: chore/prompt-audit
commit-message: 'docs: propose prompt audit improvements'
title: 'docs: prompt-audit による指示文の改善案'
body-path: ${{ runner.temp }}/prompt-audit/report.md
draft: always-true
add-paths: |
CLAUDE.md
AGENTS.md
The example passes CLI settings through Claude Code Action's claude_args input. --allowedTools controls which tools can run without prompting; --tools restricts the available tool set.
The example uses the same model ID for execution and the audit target. It sets a 15-minute job timeout, 40 turns and a three-dollar API budget. Budget enforcement stops execution when the threshold is detected; do not treat it as a guarantee that the final bill cannot exceed three dollars.
Run manually with the default branch selected to verify skill invocation, report generation, edit scope and PR creation. After success, uncomment the schedule for Mondays at 01:00 UTC, or 10:00 in Japan. You can also run it manually after a model change.
No changes means no new PR
CI receives a structured result and writes the report to a temporary file. Only completed proceeds to PR creation. This checks completion, not the correctness of the audit.
The report and actual diff are uploaded as artifacts. No-change runs still retain a report, but the report itself never creates a tracked diff.
create-pull-request creates a PR when changes exist and can update an existing PR using a fixed branch name. It may close an existing PR when its diff is no longer needed. Avoid mixing manual changes into the automation-owned branch.
draft: always-true returns an existing PR to Draft when it is updated. The report becomes the PR description. The audit job has repository read access; it passes the report and diff as an artifact to a separate job. That job checks the starting commit and the transferred diff again, then uses write access to create the PR. The audit cannot push with a write token before validation.
A created PR does not imply its CI ran
This example creates the PR with GITHUB_TOKEN. Under GitHub's event rules, events caused by that token generally do not start additional workflow runs. workflow_dispatch and repository_dispatch are exceptions.
Run required checks inside the same workflow or design PR creation around a suitably scoped GitHub App token. The sample above validates the report, changed paths and whitespace; it does not evaluate prompt behavior.
It is intended for your controlled default branch, not for running arbitrary external PR content. A changed-path check detects unwanted content before PR creation; it is not an execution sandbox. Review the substance of the audit report as well.
CI verification status
The Action implementation, argument parsing, structured-output contract and local Git success/failure paths were checked. The cross-job transfer was simulated locally; the GitHub Actions integration still requires an environment test.
The full run from an Anthropic API audit to GitHub PR creation has not been verified. Test authentication, model access, skill loading, changed/no-change outcomes and existing-PR updates with a manual run. Successful execution and improved prompt behavior need separate validation.
Effectiveness evaluation is planned
The next step is to run the same tasks with the old and new instructions: small bug fixes, test additions and ambiguous requests representative of everyday agent work.
Keep the model ID, input and initial repository state consistent, and record:
| Area | What to compare |
|---|---|
| Work quality | Requirements met, passing tests, manual corrections needed |
| Required checks | Whether confirmations or mandatory tests were skipped |
| Work performed | Tool calls, input/output tokens and elapsed time |
Repeat each condition and compare success rates and variation. Deleted lines or one successful run do not establish an improvement.
A local audit of your own CLAUDE.md is a useful first step toward deciding which proposals belong in a PR.