The number of use cases for artificial intelligence (AI) is increasing rapidly, mainly due to larger data sets, faster compute and higher processing power. In a 2018 McKinsey report, there were 400 practical use cases for AI across 19 industries, a number that has most certainly grown since then.
With AI systems penetrating more and more facets of our daily lives – from deciding the adverts we see to determining if we’re eligible for a bank loan to having a hand in the success of our job applications – it is imperative that these models are making unbiased decisions. Currently, there is a lot of controversy around whether companies are accidentally using AI that is resulting in unethical outcomes.
For example, Amazon had been developing and trialling a hiring tool since 2014, which used AI to automatically rate and rank potential job candidates. However, for technical posts such as software development, the system had taught itself to show bias against female applicants, even though Amazon had edited the programs to make them neutral to gender terms. This unwelcome development had taken place because the computer models were trained to vet applicants by observing patterns in resumes submitted over the last ten years – and most tech candidates during this time had been male. Unable to get around this hurdle, the project was abandoned at the start of 2017.
Given that AI is driving more and more life-altering decisions, how can we be sure the outcomes of these decisions are fair for every person involved, every time?
Introducing bias through confounding variables
A common misconception is that AI models will be neutral if factors such as age, gender and race are excluded as data inputs. This is not true. To achieve a specific objective, machine learning models will always find ways to link data points together. When AI combines these points on its own, it becomes very difficult to interpret and decipher if bias has taken place.
When an AI model meets the inputs it has been programmed to avoid, it automatically forms proxies to take their place. These proxies are often a result of confounding variables – which are hidden factors that suggest the true relationship between input and output.
For instance, a variable that measures the daily consumption of dairy products could be seen, at the outset, as predictive of health levels. However, a deeper inspection could show that dairy consumption is actually a decent proxy for differentiating age in a population: given that many become more lactose intolerant with age, and older people tend to have more health issues than younger people, who may also consume larger quantities of dairy.
When we do not invest in understanding all the possible confounding variables, we may end up being unconsciously biased towards a certain class, say the elderly, like in the above example around dairy consumption. Without caring to fully understand the relationships machine learning models might find in the data they are being fed, models can inadvertently become biased.
Building better, more ethical AI models
These challenges have led to a new technology movement called Ethical AI, which is, as its name suggests, AI that is both transparent and designed and tested to make unbiased decisions. At FICO, we’re committed to helping businesses build AI models that are ethical.
In theory, this is achieved by constantly exposing what each machine learning model has learned, and how this drives the outcomes of the AI. Identifying if a model is biased allows its builders to take the necessary remediating actions, such as restricting the type of data being used in a model build, or prohibiting the offending relationships a model is allowed to explore and learn from.
Delivering truly Ethical AI requires close examination of each dataset separately, especially with respect to the relationships between data points that drive outcomes – known otherwise as latent features. These latent features need to be explainable, rather than the obscure and complex webs of interconnected variables they are now.
In contrast to conventional machine learning technology where these relationships are kept hidden, new machine learning algorithms simplify these latent features and make them transparent, which in turn allows them to be tested for bias.
Case study: Ethical AI in blockchain governance
To better examine breakdowns in various model governance processes, FICO designed a blockchain model governance system to automate and specify standards for the development of Ethical AI models.
By giving users a complete and immutable record of compliance with established standards, this system not only makes each AI model build process auditable, but it also enforces consistent practices regardless of the team or individual in charge.
Ethics were made a cornerstone of this system through prescribing algorithms, extracting derived relationships between model inputs, testing for bias and sensitivity, and ensuring a thorough review and approval process. All these measures work to ensure that any model development that falls through the cracks is swiftly identified.
This system leads to Ethical AI practices because it enables each bias test to be individually examined and validated with all the necessary approvals. If there is even one test that fails these checks, the model will be prevented from being released. By enforcing this standard for all AI builds, FICO is increasing quality, accountability and ultimately, ethical model development.
With governing bodies increasingly required to understand and explain the AI models powering the decisions of today and tomorrow, it is expected that Ethical AI as a regulatory requirement will accelerate dramatically in the coming years. Businesses will do well in starting to prepare for this eventuality now.