Self-Reported Data vs Smartphone Behaviour: Why Tala Rebuilt Its Credit Test
Short answer: Automate a credit decision only when the new signals improve prediction after fraud, missing data and economic shifts, while customers can understand, consent to and challenge their use. Run the automated score beside the manual decision for at least one full repayment cycle before letting it approve or decline alone. If you cannot explain why an applicant was rejected and correct bad data, you have automated risk rather than judgement.
Tala's founder Shivani Siroya began by interviewing borrowers and gathering self-reported information. Observing smartphone use in Kenya suggested another route: with permission, phone-derived signals could help assess people who lacked conventional credit files and make small decisions quickly.
The tempting story is speed. The hard story is governance. Behavioural data can expand access, but it can also encode income, gender, location or device inequality through proxies the customer never sees.
Use the Automation Accountability Gate
The Automation Accountability Gate requires five approvals before a model controls a live decision.
| Gate | Question | Evidence |
|---|---|---|
| Purpose | Is each input relevant to repayment or fraud? | Documented rationale |
| Permission | Is collection lawful, necessary and understood? | Consent or other lawful basis, clear notice |
| Performance | Does it improve outcomes outside training data? | Holdout and live shadow results |
| Fairness | Who is excluded or harmed? | Group and missing-data testing |
| Remedy | Can a person correct data and challenge a result? | Working review process |
Accuracy does not override the other gates. A predictive signal can still be unlawful, unfair or operationally dangerous.
What changed in Tala's early method
Siroya's early work involved in-person interviews and close observation of how people earned, spent and repaid. She initially used survey-style and reported information. In Kenya, the growing use of Android phones revealed a potential source of behavioural evidence for applicants without traditional bureau files.
In a founder interview with Flyover Labs, Siroya describes Tala's mission and the development of its data-led lending approach. A Business Daily Africa profile reports on the move towards using phone information in credit decisions.
Founder accounts have referred to roughly 10,000 data points per customer and decisions in about 20 seconds. Treat those as descriptions of Tala's system at a point in time, not a target for another lender. More data points do not automatically produce a better or fairer decision.
The important pivot was from labour-intensive evidence that borrowers supplied in interviews towards digital signals that could serve far more applicants. It changed unit economics and reach. It also transferred judgement into data selection, model design and monitoring.
My position is that a lender should prefer the smallest explainable dataset that delivers acceptable risk performance. Collecting everything on a phone because permission is technically available is poor operations and weak governance.
Define the manual judgement first
Write down what the human process is trying to infer: stable income, ability to repay, identity confidence or fraud risk. Record the evidence, decision, reason and later outcome for each case.
A manual process may contain useful tacit judgement and harmful inconsistency. Shadow it before automating. If two reviewers disagree on 40% of the same cases, the target label is unstable. If both agree but repayments still fail, consistency is not accuracy.
Remove inputs that are merely convenient proxies. Test missingness deliberately. People with older phones, shared devices or privacy settings may have less data, and automatic rejection can turn data poverty into credit exclusion.
Run the model in shadow mode
For one full loan cycle, let the model score applicants without controlling the outcome. Compare its proposed approvals and declines with the manual decision and actual repayment.
| Comparison | What it reveals | Required response |
|---|---|---|
| Both approve, customer repays | Shared correct decision | Monitor |
| Both approve, customer defaults | Missing risk signal or shock | Diagnose cohort |
| Model declines, customer repays | Potential exclusion error | Review variables and threshold |
| Model approves, customer defaults | Model risk and loss | Reprice or redesign |
Do not optimise only for loss. An algorithm that declines everyone has no credit loss and no lending business.
Lending, creditworthiness, automated decision-making, privacy, equality and consumer protection rules vary by country. In the UK, obtain current FCA, ICO and legal guidance plus qualified credit-risk and compliance advice. This case is not a recommendation to use any particular phone data.
Worked example: FairStep Working Capital
FairStep is a fictional UK lender serving sole traders. Its manual review costs £32 per application. Of 1,000 applications, 400 are approved for an average £600 loan. Net interest and fee revenue after funding cost is £78 per completed loan. Expected credit loss is £42 and servicing costs £14.
Manual cohort contribution is:
400 × (£78 - £42 - £14) - (1,000 × £32) = -£23,200
A proposed automated process costs £6 per application and approves 460. Shadow testing estimates £49 expected loss and £16 servicing per loan, including review of exceptions.
Automated expected contribution is:
460 × (£78 - £49 - £16) - (1,000 × £6) = -£20
The model nearly reaches break-even, but approves 60 more loans with higher loss. Fairness testing finds applicants with limited digital records are declined disproportionately. FairStep does not launch the model alone. It creates a manual-review path costing £18 for missing-data cases. If 120 cases require review, automation contribution falls by £2,160, but the company can test access without automatic exclusion.
The surprising result is that the economically weaker-looking human review may be necessary for a fair and resilient system.
Monitor decisions after launch
Set limits on total approvals, new-customer exposure and segments with little historical evidence. Review repayment, arrears, complaints, overrides and data-correction requests weekly at first.
Track model drift. A score trained during stable trading can fail after inflation, policy change or platform disruption. Stop automatic decisions when performance crosses a prewritten boundary.
Keep a human responsible for the system. “The model decided” is not an explanation to a customer, regulator or board. Document changes and test again before deploying them.
What to do over the next 45 days
During days one to five, document the manual decision, every data input and the outcome it is meant to predict. Remove data without a defensible purpose.
By day 15, establish lawful collection, customer explanation, fairness tests and a review route with qualified advisers. Run the model in shadow mode for the rest of a complete repayment cycle, even if that exceeds 45 days.
At day 45, review early disagreement and missing-data patterns, but do not automate from incomplete repayment. Launch only after performance, fairness, remedy and unit economics pass together. Cap the first live cohort and retain manual override with recorded reasons.
Related guides
Frequently asked questions
Is smartphone data more accurate than self-reported data?
Sometimes, for a particular outcome and population, but not inherently. Behavioural data can reduce recall or deliberate misstatement, while also reflecting device access, shared-phone use and circumstances unrelated to creditworthiness. Self-reported information can be verified and directly relevant. Test both against later outcomes using holdout data and fairness analysis. Do not use Tala's experience as proof for your market. The lawful and ethical question is not only predictive power, but whether the data is necessary, explainable and open to prompt correction.
How much data should a credit model collect?
Collect the minimum needed to perform a defined lawful purpose at acceptable performance. Start with a small set of explainable variables and add one only when it improves results enough to justify privacy, security and fairness cost. Thousands of available fields can increase overfitting and governance burden. Maintain a data inventory, retention schedule and access control. Requirements vary by jurisdiction, so obtain qualified data-protection, credit-risk and legal advice. “More data” is not a responsible product strategy or a substitute for sound underwriting.
Can customers meaningfully consent to phone-data access?
Consent is meaningful only when it is informed, specific, freely given where required and withdrawable without deceptive pressure. In credit, refusal may affect access, which can complicate whether consent is the appropriate lawful basis. Explain each category and purpose in plain language rather than requesting blanket access. Offer a reasonable alternative where possible. The correct legal basis depends on jurisdiction and processing, so use current regulator guidance and qualified advice. Interface permission does not settle the wider fairness question or justify unnecessary collection.
What is shadow mode?
Shadow mode lets an automated model produce recommendations while the existing authorised process still controls decisions. You compare the model, human judgement and later outcome without exposing customers to unproven automation. It should use production-like data controls and record disagreements. Shadow mode does not eliminate privacy obligations because real data is still processed. Nor does it prove live performance if staff behaviour or applicant mix will change after launch. Use it as one gate within a broader validation and governance process.
Should a human review every automated decline?
Review at least cases required by law, cases with missing or disputed data and a meaningful sample across outcomes. For high-impact lending decisions, a genuine challenge route and human authority to change the result are essential. A rubber-stamp review is not protection. The right coverage depends on regulation, model risk and volume. Monitor overturn rates and reasons. If humans frequently reverse one pattern, fix the model rather than making reviewers compensate indefinitely. Give reviewers enough time and evidence to decide independently.
How do I measure fairness in a credit model?
Compare approval, pricing, error and repayment outcomes across legally relevant groups, investigate proxy variables and test people with sparse data. No single fairness metric resolves every trade-off. Use domain experts and affected-customer insight alongside quantitative analysis. Document why a threshold is chosen and what harm is accepted or mitigated. Equality and credit rules differ, so qualified legal and statistical advice is necessary. Continue monitoring because applicant mix and economic conditions change after deployment and can alter error patterns materially over time.
When should I return to manual decisions?
Return when loss, complaint, data-error or fairness measures breach written limits, when external conditions move beyond the model's validated range, or when a material input becomes unavailable. Design that fallback before launch, including staffing and service-time consequences. A temporary manual queue is safer than continuing fast decisions you no longer understand. Investigate, correct and rerun shadow testing before restoring automation. Automation should be reversible even when the business depends on its efficiency for current pricing and response times in production.
Comments (0)