Improving document verification accuracy with a custom ML fraud detection system
Content
99% precision
1.2% false positive rate in production
~50% reduction
Manual escalation workload
10× better
Accuracy vs LLMs and baseline tools
Services
AI Deal Analysis, Workflow Automation, Investment Intelligence Platform
Industry
FinTech / Real Estate Technology
Client
Esusu (acquired Celeri)
Duration
3 months
The Challenge
Celeri’s verification team processed hundreds of rental applications daily, with a significant portion flagged for manual review. Reviewers were spending 8+ hours per day validating documents, and existing vendor tools produced high false-positive rates, forcing unnecessary escalations.
As application volume grew, the team needed a way to scale verification throughput without linearly increasing manual review effort or sacrificing fraud detection accuracy.
The Solution
Casper Studios designed and deployed a custom ML fraud detection system optimized for document verification accuracy, not generic text reasoning. After evaluating LLM based approaches, we identified traditional machine learning as significantly more reliable for this use case.
Key capabilities included:
Interpretable ML classifier trained on 65+ engineered document metadata features
Pass / Fail / Caution prediction architecture using calibrated one-vs-rest models
AI Assist layer providing plain-English explanations to support reviewer trust
Feedback-driven ETL pipeline capturing human decisions for continuous retraining
The system consistently outperformed both off-the-shelf LLMs and existing vendor tools in precision and false-positive reduction.
The Impact
The ML model dramatically reduced unnecessary escalations while preserving fraud detection accuracy. Reviewers now focus their time on genuinely high risk cases, cutting manual workload by roughly half and improving overall processing speed.
By prioritizing precision and interpretability, the solution increased operational confidence and enabled Celeri to scale application volume without expanding the verification team, while setting a new internal benchmark for AI performance across the platform.
