The Centre for Data Ethics and Innovation (CDEI) is an independent expert committee, led by a board of specialists, set up and tasked by the UK Government to investigate and advise on how the UK maximises the benefits of data driven technologies. In the October 2018 Budget, the Chancellor announced that the CDEI would investigate the potential bias in decisions made by algorithms. This review formed a key part of the CDEI’s 2019/2020 work programme, though completion was delayed by the onset of COVID-19 pandemic.

On 27 November 2020, the CDEI published its final report of its review which includes a set of formal recommendations to the UK Government. In relation to the financial services sector the CDEI review found:

• A mature sector that has long used data to support decision-making.
• Finance relies on making accurate predictions about peoples’ behaviours, for example how likely they are to repay debts. However, specific groups are historically underrepresented in the financial system, and there is a risk that these historic biases could be entrenched further through algorithmic systems.
• Financial service organisations ranged from being highly innovative to more risk averse in their use of new algorithmic approaches. They are keen to test their systems for bias, but there are mixed views and approaches regarding how this should be done. This was particularly evident around the collection and use of protected characteristic data, and therefore organisations’ ability to monitor outcomes.
• The CDEI’s main focus within financial services was on credit scoring decisions made about individuals by traditional banks. Its work found the key obstacles to further innovation in the sector included data availability, quality and how to source data ethically, available techniques with sufficient explainability, risk averse culture, in some parts, given the impacts of the financial crisis and difficulty in gauging consumer and wider public acceptance.
• The regulatory picture is clearer in financial services than in the other sectors the CDEI have looked at. The Financial Conduct Authority is the main regulator and is showing leadership in prioritising work to understand the impact and opportunities of innovative uses of data and AI in the sector. The use of data from non-traditional sources could enable population groups who have historically found it difficult to access credit, due to lower availability of data about them from traditional sources, to gain better access in future. At the same time, more data and more complex algorithms could increase the potential for the introduction of indirect bias via proxy as well as the ability to detect and mitigate it.

Key recommendations in the CDEI report include:

• Government should place a mandatory transparency obligation on all public sector organisations using algorithms that have an impact on significant decisions affecting individuals.
• Organisations should be actively using data to identify and mitigate bias. They should make sure that they understand the capabilities and limitations of algorithmic tools, and carefully consider how they will ensure fair treatment of individuals.
• Government should issue guidance that clarifies the application of the Equality Act to algorithmic decision-making. This should include guidance on the collection of data to measure bias, as well as the lawfulness of bias mitigation techniques (some of which risk introducing positive discrimination, which is illegal under the Equality Act).