AI-assisted redaction can help a team find sensitive information in scanned forms, claims, onboarding packets, and low-quality faxes. The safe starting point is a supervised workflow: the system proposes what to remove, creates a redacted candidate, and gives a trained person the evidence needed to approve it. A completed batch or a high confidence score is not permission to share the files.
The business decision is whether automation reduces the effort required to produce a correctly redacted, usable document—not whether it can draw black boxes quickly. Start with one recurring document type, one approved use, and a named owner. Keep detection quality, permanent removal, and the release decision separate.
What the September 16 AWS workflow demonstrates
AWS's September 16, 2026 example combines Amazon Bedrock Data Automation, S3, Step Functions, and Lambda. A custom blueprint identifies target fields and bounding boxes; page images are redacted and assembled into a new PDF. Custom and standard word-level output from one API call feed an additional token-matching check.
| Detection design | Precision | Recall |
|---|---|---|
| Blueprint only | 97.0% | 89.3% |
| Blueprint + token matching | 96.5% | 95.2% |
AWS's Table 3 reports higher recall with token matching, but slightly lower precision. This is evidence of a trade-off in a small vendor evaluation, not evidence that all identifiers were removed. Treat the numbers as a reason to test your own document mix. Neither successful processing nor an average confidence score establishes that a file is ready to leave your organization.
Define the fields before choosing the detector
Inventory the forms, their owners, languages, page counts, scan quality, handwriting, and receiving systems. List sensitive fields by person and purpose: applicant versus employee, claimant versus provider, account holder versus business contact. Include repeated mentions in narrative text, fax cover sheets, marginal notes, stamps, signatures, barcodes, and attachments where they carry protected information. Do not assume text recognition covers every image or encoded identifier.
Create a field policy with three explicit decisions: always remove, retain for this approved purpose, or refer for review. Define inclusions, exclusions, and allowed exceptions at document-type level. Each exception needs an owner, rationale, audience, expiry or review date, and test example. A supplier name or examination date is not universally safe merely because another organization's example retained it.
For example, an onboarding packet may require removing an employee's home address and government identifier while retaining a company contact name. That is an illustrative policy, not a legal determination. Privacy, legal, and business owners must decide what the recipient actually needs. Do not make a blanket allowlist for a common name: the same text can identify different people in different fields.
Where Presidio fits—and its current limitations
Microsoft-origin Presidio supports configurable PII recognition, including patterns and named-entity models. Its documentation explicitly warns that automated detection cannot guarantee finding all sensitive information. The image redactor combines OCR with PII detection; it offers Tesseract and an Azure Document Intelligence OCR option. The OCR engine, language configuration, recognizers, and input quality all belong in your evaluation.
Current project guidance describes Presidio's transition from Microsoft ownership to the community-governed Data Privacy Stack. Follow the current documentation and package publisher rather than assuming an old Microsoft container image is maintained. Pin the chosen release and record the recognizer and OCR versions. This article compares design considerations; it does not claim a tested BDA–Presidio integration.
Presidio can be evaluated as a primary detector or an additional review signal. A second detector is useful only if it catches errors the first misses without creating unacceptable over-redaction. If both depend on the same flawed text extraction, agreement can hide a shared blind spot. For a rare, small batch, careful manual redaction may be more practical than building and maintaining a pipeline.
Build a representative ground-truth test set
Use documents you are authorized to process. Synthetic records are useful for controlled edge cases, but they cannot by themselves establish performance on real handwriting and fax artifacts. Keep an approved, access-restricted sample that represents actual document types, languages, layouts, and degradation. Include skewed pages, faint ink, cropping, overlapping stamps, repeated names, mixed typed and handwritten text, and pages with no PII.
Have trained reviewers mark every in-scope occurrence and its full region, not merely the first occurrence of each distinct value. Record fields that should remain visible as well. Resolve disagreements with a second reviewer and the policy owner. Split development examples from a held-out acceptance set, keeping near-duplicate forms and pages from the same source document together to reduce misleading test results.
Define what counts as a complete hit. A rectangle that leaves the last digits of an identifier readable is not a successful full-field redaction. Score field-level misses and document-level escapes separately. Review the source image when OCR has omitted content; evaluating only against extracted text makes invisible-to-OCR errors invisible to the test too.
Set confidence and escalation rules using observed errors
Do not copy a universal threshold such as 0.90 into every workflow. A detector's confidence is not automatically the probability that the entire document is safe. Compare score bands with human-labeled results by field type and scan quality. Set conservative thresholds for proposing redactions and separate criteria for escalation. Lower thresholds may find more sensitive fields while also removing useful content.
Low confidence should send a suspected sensitive field to review, not silently leave it exposed. Unreadable regions, missing pages, unexpected layouts, unsupported languages, detector disagreement, and incomplete outputs also require escalation—even if the remaining fields score highly. A page with no detections can mean no PII, or it can mean detection failed. During the pilot, a human must approve every document before external sharing or downstream use.
Add token matching without widening exceptions accidentally
Use detected values as candidates for repeat-occurrence searches across the permitted document context. Normalize cautiously, retain location evidence, and check full phrases as well as relevant tokens. A known name appearing again in notes deserves attention; a common word inside that name may also occur in ordinary prose. Test false positives introduced by splitting names, stripping punctuation, or matching partial identifiers.
Token matching is a supplement, not an independent discovery guarantee. It can miss a value never detected initially, an OCR mistranscription, or a handwritten variation. Reapply person and field rules to each new match, deduplicate overlapping boxes, and verify coordinates after rotation or resizing. Keep authorized exceptions visible to the reviewer rather than automatically deleting every matching word.
Preserve originals and isolate each processing stage
1. Secure intake
Preserve the original; identify every document and page.
2. Detect and cross-check
Apply field rules, confidence review, and repeat-token checks.
3. Create a redacted candidate
Remove sensitive pixels or content; sanitize the output file.
4. Review the complete artifact
Compare every page, check hidden data, and resolve exceptions.
5. Approve and release
A named human approves this file, recipient, and purpose.
Failed page, unreadable field, or unresolved exception → hold the document for correction. No automatic release.
Keep an immutable or version-controlled original in restricted storage under the applicable retention and legal-hold rules. Process a working copy. Separate originals, temporary page images, detector outputs, redacted candidates, approved releases, and operational logs with appropriate identities and permissions. Different folder names alone do not enforce isolation. A detector result containing a name or address remains sensitive data.
Use encryption, least-privilege access, approved processing regions and provider terms, and explicit deletion schedules for temporary artifacts. Keep raw document contents out of broad application logs and error messages. Record document IDs, versions, page counts, timings, policy decisions, and protected evidence references instead. Treat unreviewed outputs as sensitive; they may still contain missed fields.
Reconcile every submitted document and page to its result. Set bounded retries, quarantine corrupt or incomplete documents, and make reruns idempotent so they cannot duplicate an external delivery. Release only the exact approved artifact version. A recovered batch must not replace a reviewed file with a newly generated, unreviewed one.
Remove content, not just its visible appearance
Drawing an annotation over text is not the same as removing it. A production redaction step must eliminate the targeted content in the released artifact, including underlying image pixels or text objects. Adobe's redaction guidance distinguishes marking content from applying redactions and sanitizing hidden information such as metadata, comments, and layers.
Inspect the final PDF—not only the preview. Test text extraction, search and copy/paste, hidden OCR layers, comments, form values, attachments, metadata, and recoverable original objects. For an image-based export, verify that reconstruction includes only the redacted page images, not an original attachment or unredacted text layer. If searchable text is regenerated, derive it from the redacted artifact and recheck it. Also validate page order, legibility, and accessibility for the intended recipient.
Presidio's anonymizer documentation includes multiple operators, including reversible encryption. Choosing an operator called anonymization does not automatically provide permanent removal. Partial masking, encrypted replacements, and retained re-identification mappings serve different purposes; do not present them as irreversible redaction.
Apply NIST governance to the release decision
NIST SP 800-188, published in September 2023, addresses government dataset de-identification. It emphasizes release goals, sharing models, disclosure review, measurable performance, and re-identification risk. It also warns that masking tools alone may be insufficient. These principles inform the business workflow here; the publication is not a certification of this design or a blanket compliance determination.
Removing obvious identifiers does not necessarily make a document anonymous. Combinations of dates, locations, job details, or unusual events may still identify someone. Assign a privacy owner to assess the residual content and intended audience, not just the redacted fields. Prefer restricted access or a narrower extract when the full document would expose unnecessary detail. Route healthcare, employment, financial, and legal questions to the responsible specialists.
Measure usable, approved documents—not boxes drawn
Track precision as correct in-scope detections divided by all detections proposed for redaction. Track recall as correctly detected in-scope occurrences divided by all ground-truth in-scope occurrences. Define matching rules before testing and separately measure whether the final redaction fully covers each field. Neither metric alone captures a dangerous missed identifier or an unreadable output.
Record false positives, missed fields, partial coverage, and policy exceptions by document type, field, language, scan quality, and version. Include the proportion of documents with any missed sensitive occurrence, review minutes per approved document, reviewer corrections, failure and retry rates, processing cost per page, and total cost per approved document. Count OCR, model calls, storage, orchestration, rework, and human review. Compare with the same manual baseline and report small sample sizes and uncertainty.
A practical pilot starts in shadow mode, with no automatic export. Expand only after the owner accepts the held-out results, reviewers can handle the workload, recovery is tested, and released files pass the whole-artifact checks. Revalidate after changing the model, blueprint, OCR engine, recognizers, matching logic, or PDF library. Our AI evaluation guide explains why end-to-end outcomes matter more than a component's score.
Keep human approval attached to the output hash or version, approved recipient, purpose, policy version, reviewer, and timestamp. If a missed field is discovered after release, stop further distribution, preserve restricted evidence, identify affected outputs and recipients, and involve security, privacy, and legal owners in containment and notification decisions. An accountable workflow automation pilot should make this correction path as explicit as the happy path.
