MODEL VALIDATION: FAIR LENDING MODEL VALIDATION

Fair lending analysis tries to estimate a causal effect between race, gender or age and credit decision or pricing based on consumer lending transaction data from mortgages, credit cards, and auto loans.

Crucial aspects of the analysis rest on assessing the strength of the causal effect measured by the odds ratio or marginal effect. The most neglected aspect of validation for these models is the impact of unmeasured confounder.

It is important to understand how much confounding it will take to explain away the causal effect. This is sensitivity to missing or unmeasured confounder. If the disparity metric is not sensitive to unmeasured confounder, then we have stronger evidence of fair lending risk. Other aspects relate to missing data in areas of race, gender or age. The missing data is not missing at random (MAR), and large bias is introduced if the analysis does not take this into account.

Another aspect is the misclassification error in the BISG (Bayes-improved surname geocoding) proxy for race and gender when they are not reported. Because the classification error is very large for African Americans, this tends to introduce bias in the disparity measure.