Case Study

AI-Powered KYC Address Verification

For Nigerian Fintechs — solving the problem of unreliable, manually-reviewed address verification at onboarding.

Architecture

A three-layer pipeline: Claude vision extracts identity fields from uploaded documents; a deterministic rules layer checks those fields against CBN KYC requirements; a RAG reasoning layer resolves ambiguous cases against regulatory source documents. Deterministic rules exist because compliance decisions need to be auditable and reproducible — an LLM alone can't guarantee that.

Eval Results

22 Nigerian-specific test cases, run against v1.0 and v1.1 of the pipeline.

$ eval-runner --cases 22
 
v1.0  Precision: 72.7%  Recall: 88.9%
      F1: 80.0%  Accuracy: 72.7%
 
v1.1  Precision: 90.0%  Recall: 100.0%
      F1: 94.7%  Accuracy: 95.5%
 
Δ  +22.8pp accuracy · 0 false approvals

Three Failure Patterns

Nigerian naming conventions

The system rejected customers named "Hajiya Zainab Danladi" because "Hajiya" — a common Nigerian honorific — wasn't in the title-stripping list. Fixed by building a Nigerian-specific honorific and title dictionary rather than relying on generic name-parsing.

Unvalidated field assumptions

Address fields extracted correctly but weren't cross-validated against state/LGA lookups, allowing internally inconsistent addresses to pass. Added deterministic cross-field validation after extraction.

Cross-city address matching

Utility bills and ID documents listing addresses in different but valid formats (abbreviated vs. full state names) were flagged as mismatches. Added a normalisation layer before comparison.

Key Product Decisions

Why Claude vision: document quality varies wildly across Nigerian fintech onboarding flows — phone photos, scans, low light. Vision extraction needed to be robust to that variance.
Why deterministic rules: compliance decisions need an audit trail. A rules layer makes every approval/rejection traceable to a specific check.
Why RAG: edge cases need to be resolved against actual CBN source text, not a model's memorised (and possibly outdated) understanding of regulation.