Can Big Data be too much of a good thing? Professional data scientists themselves have been among the first to raise concerns about ethical data use. For instance, relying solely on machine learning to interpret data may lead you to start seeing trends that don’t really exist. Or, cause you to jump to prejudicial conclusions like “men don’t buy that so it must be fraud” or “people in this Zip Code can’t afford that.”
“Any organization working with Big Data runs the risk of removing the true, accurate human condition that we all want the data to tell us,” said Catherine Maloney, Vice President, Data Management and Analytics. “In setting up how we acquire the data, there needs to be a gut check – Does this feel right? This can only come from human beings looking at data gathering processes with empathy towards its impact on the lives of people.”
Maloney noted that processes that go without such considerations can lead to unintended biases related to race, sex, age and economic status. “We need to always be asking ourselves ‘How would this impact someone who is not like me?’” said Maloney.
These are some of the questions we asked ourselves at CO-OP. After all, we process millions of credit and debit transactions on behalf of our credit union clients, as well as the transactions that occur across the 30,000 ATMs and 5,700 branches in our network. Now, as we have begun applying machine learning algorithms across our network, it is critical that we take a leadership role in the responsible use of Big Data.
“By examining the inputs, outputs and decisioning models taking place, we are guarding against making any potential leaps so that unrelated events cannot be tied together,” said Maloney. “Additionally, we are looking at the sources of data to make sure they are sufficiently diverse to even make conclusions in the first place.”
Launching the CO-OP Ethical Data Use and Practices Council
Among CO-OP’s answers to this critical issue is the CO-OP Ethical Data Use and Practices Council, whose charter is “To provide independent review and guidance over how technology is used to automate decision-making that will have impact to end consumers.” First among the Council’s to-do’s: “Embrace the credit union mission of ‘People Helping People’ as CO-OP advances technical capabilities and products.” The Council itself is made up of 15 CO-OP employees, comprising a cross-functional team that represents all areas of the enterprise.
The Council meets once per month and hears from a guest employee technologist on what they are working on, allowing Council members to high-level review their data gathering methodology, inputs, outputs and any decisioning that takes place. The Council then engages in an open discussion on any possible or perceived unintended bias, guided by “Value Sensitive Design” (i.e., “design of technology that accounts for human values in a principled and comprehensive manner throughout the design process”).
No Substitute for Human Judgment
How can bias – or “digital red lining” – result from data? One example could be a prison and a cardholder’s residence sharing the same Zip Code, with transactions flagged as potentially fraudulent as a result.
It is these types of things that require human judgment – and there is no substitute. “Once you program a computer, it just goes, and that is actually what we like about it,” said Maloney. “But it is hard for machines to unlearn something. That’s why it is so important to pause and reflect on what we are doing. It’s a bit like doctors and nurses reconfirming with a patient several times just before surgery what exactly is being operated on. And, again, it calls for human empathy – it is not a scientific method, otherwise we would build it into the system.”
CO-OP founded the Council to help guide the company today, but perhaps even more critically in the future. “This is where the industry is going, with the increasing use of demographic information,” said Maloney. “We are neither late nor early. When we assembled the Council, the feedback we received from Cheryl Middleton Jones, our Chief People Officer, was that it is never the wrong time to do the right thing.”
De-Mystifying the Practice of Data Science
Ultimately, the Council’s work is aimed at helping credit unions maintain their trustworthiness with members, a key market differentiator with other financial services providers.
“At CO-OP, we want our products to be transparent,” said Maloney. “We want to be able to explain clearly to our clients how our algorithms and data models work. In this way, we can help de-mystify the data science practice and enable our credit unions to tell their members how their data is being used and assure them it is being used responsibly and to their benefit.”
COOPER Fraud Analyzer from CO-OP is coming up on its second anniversary in the fraud-fighting service of monitored in real-time credit union member transactions. Next year, CO-OP will make generally available COOPER Fraud Score, which uses machine learning to create risk-scoring models that determine the level of suspicion on card-based transactions.
With these existing and new, high-advanced fraud-fighting services, the Council’s work will only grow in importance.
“There is a lot we could do in terms of data gathering, the questions are should we be doing it and how should we go about it?” said Maloney. “It begins by making sure human empathy is part of it at every stage. This issue is a major one in the larger financial services industry, not just the credit union sphere. We want to be leading the charge as champions of the ethical use of data.”
For more information on COOPER Fraud Analyzer and COOPER Fraud Score, visit here.
The original article Use Responsibly: Imbibing Big Data Calls for Sober Human Judgment can be found on Insight Vault.
