Add post-generation disclaimer footer and hallucinated citation flagging #98

New issue

Closed

opened 2026-04-08 18:33:21 -04:00 by pook · 1 comment

pook commented

2026-04-08 18:33:21 -04:00

Owner

AI-generated compliance documents may contain hallucinated legal citations (fake statute numbers, non-existent regulation references). This is a critical liability risk for a compliance product.

Add a post-generation step that scans output for citation patterns (e.g., '§ \d+', 'Article \d+\.\d+', statute-like references) and flags those not in a known allowlist
Add a citations_flagged: number field to the generation response metadata
Append a standard disclaimer footer: 'This document was AI-generated and requires review by a qualified legal professional before use.'

Acceptance criteria:

All generated documents include the legal review disclaimer footer
Suspicious citations are counted in response metadata
Unit tests verify citation detection regex on known good/bad samples

Generated by CEO Planner (priority: 2)

AI-generated compliance documents may contain hallucinated legal citations (fake statute numbers, non-existent regulation references). This is a critical liability risk for a compliance product. 1. Add a post-generation step that scans output for citation patterns (e.g., '§ \\d+', 'Article \\d+\\.\\d+', statute-like references) and flags those not in a known allowlist 2. Add a `citations_flagged: number` field to the generation response metadata 3. Append a standard disclaimer footer: 'This document was AI-generated and requires review by a qualified legal professional before use.' Acceptance criteria: - All generated documents include the legal review disclaimer footer - Suspicious citations are counted in response metadata - Unit tests verify citation detection regex on known good/bad samples --- *Generated by CEO Planner (priority: 2)*

pook added the

agent-task

label

2026-04-08 18:33:21 -04:00

pook commented

2026-04-10 14:48:29 -04:00

Author

Owner

Bulk-closed 2026-04-10 during pipeline triage.

Context: CEO agent had created 100 open agent-task issues against compliancebot, largely duplicates of each other and of the 50 currently-open PRs. Root cause traced to a git-push race in agent-worker executor (dispatch jobs collided on branch agent/dispatch/* because jobId prefix truncated to literal "dispatch"). Fix deployed: runId is now threaded from Paperclip shim through /dispatch → TaskJob → executor, and branches are keyed on a 12-char unique run key.

What to do next:

Triage the 50 open PRs at https://192.168.183.110:3000/pook/compliancebot/pulls — many are ready to merge
CEO should halt new task creation until open PRs drop below 10
Surviving kept issues: #313, #314, #315, #341, #342, #350, #351, #352 (PR review/merge tasks)

This issue was superseded, not abandoned. Reopen if still relevant after PR triage.

Bulk-closed 2026-04-10 during pipeline triage. **Context:** CEO agent had created 100 open agent-task issues against compliancebot, largely duplicates of each other and of the 50 currently-open PRs. Root cause traced to a git-push race in agent-worker executor (dispatch jobs collided on branch `agent/dispatch/*` because jobId prefix truncated to literal "dispatch"). Fix deployed: runId is now threaded from Paperclip shim through /dispatch → TaskJob → executor, and branches are keyed on a 12-char unique run key. **What to do next:** 1. Triage the 50 open PRs at https://192.168.183.110:3000/pook/compliancebot/pulls — many are ready to merge 2. CEO should halt new task creation until open PRs drop below 10 3. Surviving kept issues: #313, #314, #315, #341, #342, #350, #351, #352 (PR review/merge tasks) This issue was superseded, not abandoned. Reopen if still relevant after PR triage.

pook closed this issue

2026-04-10 14:48:29 -04:00

No labels

agent-task

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

pook/compliancebot#98

No description provided.

Rows
Columns