CUSTOMER Q&A: Affirm, FICO, and 2nd Order Solutions

How credit leaders are rethinking model risk in the age of AI

As AI moves deeper into credit decisioning, risk management is no longer just about whether a model performs well. It’s about understanding the data going in, the decisions coming out, and the people affected by those decisions.

In this conversation from Plaid Effects, Brett Manning, Product Manager at Plaid, spoke with Ethan Dornhelm, VP of Scores and Predictive Analytics at FICO; Don Lemire, VP of Risk and Analytics at Affirm; and Syed Raza, Head of AI at 2nd Order Solutions, to discuss bias testing, model monitoring, explainability, and how new data and modeling techniques could help lenders make better credit decisions.

This conversation has been edited for clarity and length.

Brett Manning: To kick us off, when did you first realize risk management really mattered?

Ethan Dornhelm, FICO: For me, it was the Great Recession and the subprime mortgage crisis. That was the first time in my career that the stakes of not adhering to responsible risk management became really clear.

Don Lemire, Affirm: Fresh out of college, I built a sophisticated model to help determine how many loans a sales agent had to sell in a region. I thought, “Man, this thing rocks.” I hit send.

A few minutes later, a sales agent came up and said, “Don, what the F? There’s no way I can sell all these loans. I’ll get fired for this.” I remember thinking, “But the model said this was really cool.” That scared the living heck out of me. These things I do behind a computer are impacting hundreds of people. They’re not just numbers.

Syed Raza, 2nd Order Solutions: Banks usually call us when they have trouble, so I’ve seen my fair share. Recently, a bank saw losses rise among younger customers. They had built the model when student loan delinquency reporting was paused, so those borrowers looked low-risk. When models were blind, losses went up.

Manning: Don, Affirm makes a lot of credit decisions. Where can bias show up?

Lemire: I think about the features you put in the model. In credit analytics, you’re looking for behavior that impacts someone’s ability to repay. But if you start pulling in data and saying, “People from this region aren’t paying as they should, so I’ll price them higher or decline them,” you can run into trouble. If you’re not thinking through customer impact first and foremost—plus fair lending and bias—that’s where bias can show its head.

Manning: Ethan, FICO scores impact a huge number of people. How do you think about bias in input data?

Dornhelm: Speaking through the lens of the FICO Score, the bar for compliance, transparency, and explainability is very high. Before analysts even get eyes on the data to build the final model, we review it with product, legal, and compliance. Sometimes entire groups of features get pulled out. I remember one data source that identified people by the quality of the university they attended. There were interesting concepts there, but our product team said, “You’re not going to deliver reason codes based on that.”

Manning: Once a model is built, how should teams monitor it in the real world?

Raza: At a minimum, you need to monitor the data going into the model, the data coming out, and how the model performs across segments of interest. But people can get too focused on statistical thresholds. You also need to care about business impact, including fair lending risk.

AI can help with monitoring, too. I wouldn’t just give ChatGPT access to dashboards or all of the data; it will hallucinate. Where we see it work is agentic AI with deterministic tools, like Shapley values or drift analysis. Then it’s like having a really smart credit analyst who doesn’t tire and can run analyses around the clock.

Manning: Ethan, FICO recently announced a partnership with Plaid. How does cash flow data fit into the picture?

Dornhelm: In thin-file segments, a score based only on credit bureau data is simply not as powerful as in segments with broader credit history. That was a large part of the motivation to look beyond the credit file. We’re excited about the recent partnership with Plaid, where we’re getting cash flow data. If Plaid cash flow data can be used in combination with credit data, there could really be a win, especially for thin-file consumers. I think it was something like a 15% improvement in prediction, which is a big number for us.

Manning: Don, how do you think about monitoring at Affirm?

Lemire: I think about the human element. For credit monitoring, it’s culture, culture, culture: accountability, redundancy, and subject matter expertise. There’s an expectation that if you’re putting something out there, you know it forwards and backwards and you’re responsible for its performance. New tools can help us get better, but the people element, the culture, and subject matter expertise are still paramount to making quality risk decisions.

Manning: Ethan, how do you think about explainability?

Dornhelm: Our former CEO Larry Rosenberger used to say, “Complexity in development, but simplicity in implementation.” We force analysts to justify adding complexity. For the FICO Score, the framework is extremely straightforward. If you give me a credit report, I can calculate your FICO Score in five minutes. And if it’s that straightforward to calculate, it’s also straightforward to calculate the key drivers: the top reasons why you didn’t score higher.

We’re willing to give up one or two percent in predictive performance if, in exchange, we can be absolutely confident in how we explain scores back to consumers.

Manning: Don, how are transformer models changing credit decisions?

Lemire: It's no fun giving someone an adverse action notice. It’s less fun receiving one.

Transformer modeling can help us look again at a decline population. A traditional model may decline a user, but a transformer-based model can look at the same data and say, “Here’s a user where there’s a more complete picture of the story, and we feel good about extending an offer of credit.” Then we can avoid the negative outcome and deliver a positive experience.

Manning: Raza, what does AI change about explainability?

Raza: Explainability is the biggest bottleneck. Transformer models are clearly outperforming regular models, but the technology is not ready for a lot of adverse-actionable use cases.

Where transformers can be powerful is with unstructured data. Plaid’s cash flow data is a really good example. You get a much better understanding of the person’s finances, the sequence of transactions, the time-based signals, and all of those signals that a traditional model can’t capture. With transformers, you can capture those signals and make better credit decisions for the bank and for the customer.

Manning: What should teams keep in mind as they build more sophisticated models?

Lemire: If you’re working on these things, there’s an expectation that you understand the impact. It’s your responsibility to bring cross-functional partners in every step of the way.

Raza: I started my career thinking, “I just need to build the best model.” Now a lot of my time is spent building relationships and talking to various stakeholders such as legal and compliance. It’s not adversarial. It’s understanding the lens they’re coming from.

Dornhelm: Sometimes explaining a model is not just about explaining what’s in the model. It’s also about understanding the outside factors that may be affecting the results. The more insight you have into the broader context, the better you can explain the outcome.