UK financial regulators have warned banks that they can only deploy AI to approve loan applications only if they can prove it will not worsen discrimination against minorities, according to a report on the Financial Times.
At stake is potential bias against those who might already struggle to borrow, and who are then flagged by AI due to its reliance on historical data.
Good AI, bad AI
While there is strong agreement around the world that the use of AI needs to be regulated, this is probably one of the strongest and most specific stances to be publicly announced to date.
As noted by the FT report, banks are exploring ways to automate more of their internal processes and lending decisions and have turned to AI to make this decision for them.
Their argument is that using machine learning (ML) techniques to make lending decisions can reduce discrimination due to its ability to make objective decisions as opposed to humans who might make subjective ones against certain ethnic groups.
On the flip side, it is entirely possible that groups that are discriminated against share attributes that result in them being lumped together, noted the report. For instance, a certain minority could share a particular postal code. And while it has no bearing on their propensity to default on the loan, it might be picked up by AI to the detriment of certain borrowers.
What makes it worse is how organizations tend to be tight-lipped about the algorithms and training data they use. Mistakes in data preparation or bias might creep in and never be discovered.
In addition, ML models are veritable black boxes that can be difficult to assess. Moreover, the tendency is also high that organizations will keep using an ML model that has been trained and shown to work without further questioning.
Dangers of undetected bias are very real. As we reported in 2020, a popular AI training tool passed its bias to a generation of ML models for almost two decades.
In 2014, an experimental in-house tool for screening job applications for technology roles at Amazon also amplified existing gender imbalance by discriminating against women applying for technical roles.
Image credit: iStockphoto/Nuthawut Somsuk