Let’s review some answers to your questions and get you back in the game with this small business lending Q & A.
Q: What is the best metric to use when validating small business models?
The main objective for any validation is to make sure that the model is rank ordering the population properly, the delinquent accounts are scoring below cut-off score and the current accounts are scoring above cut-off score. In addition there are some stability reports that compare historical and current applicant populations.
A: Stability Indices, Analysis, Tests
- Population Stability | Characteristic analysis
The purpose of this test is to see if there are any significant changes in the group of applicants for loans as compared to the development population. A major shift in total score here could indicate trouble down the road. If this test indicates that there is a 10 point shift in total score then the next report will tell you where the difference is coming from. For each characteristic in the model a comparison is made between the development sample and the current applicant population. A point difference is calculated for each and a determination is made as to what to do about the shift. For example a change in cutoff strategy may be indicated.
Once you have examined the population indices you can start to take a look at how different each vintage of booked applicants are performing over time.
When looking at performance in terms of discriminating between performance sets there are a number of alternative statistical tests you can use.
- Kolmogorov-Smirnov Statistic (K-S)
The K-S statistic expresses how many more percent of the “bad” distribution than the “good” distribution are under the most efficient cutoff score in the distribution. The higher the K-S is, the larger the spread in the distributions and the better the scorecard performance. K-S is especially useful with small sample sizes.
- Performance by score – Known Good/Bad results
Note: *Due to smaller volumes you may need to use larger time windows to get sufficient cell counts, but otherwise it is exactly the same process.These odds by score can be graphed and various time frames compared.This process is also known as the score to odds performance test and is one of the most common test in use.The Odds to Score Report is used to directly measure the scorecards ability to rank order risk. The good to bad odds are measured for each score range, and we expect to see that the odds increase as the score increases. That is, that the higher score ranges have a higher percent of good accounts than the lower score ranges.
- Trade off Curves, also known as ROC Curves or Lorentz Curves
The ROC curve should be as far as possible above the diagonal. ROC curves also proved useful for the evaluation of modeling techniques. The first application of ROC was in comparing and evaluating different classification algorithms It is a popular business measure of model performance for binary (Good/Bad) outcomes The area under the ROC curve is a summary measure for this success. A high value of ROC area indicates that the score has good overall classification properties over a large range of possible cutoff values.
Q: My Underwriters refuse to stick with my small business lending rules. What can I do?
A: Review, Discuss. Collaborate. Work together to revise the best solution for your Small Business Lending rules, however, in the end, underwriters need to be trained to stick to underwriting rules.
Having an automated, score and rule based strategy with a grey area is a start. Each organization’s credit risk policy should provide details around allowable high and low side over-rides. Over-ride reports should track performance by over-ride rule to ensure that the rules are, indeed, contributing to the decision making process. Never, ever use “other” as a rule. And lastly, over-ride reports should be produced by underwriter to ensure compliance, provide feedback and enhance training.
- Final Score Report
The final score report monitors the number of overrides and policy decisions by reason- it can be used to Identify trends in overrides and policies.You should also track overrides by reason. This is used to identify trends in override policies. This report is typically produced for the most recent quarter. The override reasons used to approve applicants should be examined to determine that the risk involved in accepting the applicants is warranted.
Q: How do I convince senior management that we need to monitor our small business portfolios?
The question here isn’t “why monitor” but “why isn’t it possible for the organization to monitor?”
It is just a matter of having the right data, access to the data, the right infrastructure to get to performance monitoring. You MUST comply with regulations – regulators tend to have their heads spin around if you don’t monitor!!! You need to verify compliance.
A: Control operations & Manage risk
Monitoring early can save losses as early indicators that the portfolio is in trouble can be spotted and targeted for aggressive mitigation.
Uncover opportunities & Stay Ahead of the Competition – Spotting early trends not only in losses but in attrition and non-use can impact the bottom line.
Q: I can’t grow my small business portfolio as fast as senior management wants. What should I do?
You need to do some market research, testing and tracking to the equation. This also means comparisons to existing vintages (i.e., looking at early loss curves).
A: Be Prepared – do some market research, testing and tracking
Test populations can be tracked and managed – With automation, cohorts of expansion test populations can be readily tracked and aggressively managed until the organization can make a determination that the expansion is viable. This also means comparisons to existing vintages (i.e., looking at early loss curves).
- Cultivate referral sources. The internet is your friend. Do some research & dive into social media.
- Increase community involvement. Connect with existing customers, connect with businesses banking elsewhere & get to know them.
Q: I can’t make my small business margins without pricing myself out of the market. What can I do?
A: Evaluate the profitability of the small business effort on a more holistic basis… you need market research to tell if your products are priced competitively. It’s not a general question but one that is pertinent to each segment of the population.
Use of risk based pricing allows banks to compensate for increased risk (and those most likely to respond, need and use credit) as well as be attractive to good risk customers (who have lots of choices). This balance ensures profitability.
Secondly, technology and automation can drive out a great deal of the cost to acquire and manage (after an initial investment!). The use of expensive, time consuming judgmental underwriting only serves to drive up cost to acquire and introduce inconsistencies into the process.
Q: Compliance is a headache for me in the consumer world, now I am asked to take on the small business world. How do I start?
A: You need to understand what, if any, regulations apply to your SME portfolio. That means looking at all cross-border regulations, where you operate, as well. Start with the basics. Some things will apply, i.e., Basel, privacy, data security, portfolio monitoring (safety and soundness). Other things may not or may not initially (i.e., Dodd-Frank).
Some regulations may not be attainable initially (i.e., model management, CCAR, stress testing for smaller or newer portfolios like SME portfolios often are). Whenever possible, we recommend that organizations treat the SME portfolio the same as they treat any other retail portfolio (this is required under Basel anyway). Regulations are going to get there at some point, so it’s best to be prepared and organizations have the experience and infrastructure to do compliance activities on the consumer side already.
View the slideshare here
Contact us with questions on small business Lending or better yet, explore our courses to hone your skills with simulation training!