AI & Prejudice: The Case for an Ethical Framework
“Artificial Intelligence (AI) will be either
the best or worst thing ever to happen to humanity.” I believe these words by Stephen Hawking are a reminder that AI, like any other
technology, is a reflection of its inputs. Molded by the right set of hands, AI
can bring about profound changes in our lives by improving social co-existence,
protecting the environment, and augmenting human capabilities and interactions.
At the same time, inherent human follies such as biases can significantly impact
the output, leading to disruption or even destruction.
Identified in
facial recognition systems, online search algorithms, and even hiring programs,
bias is all-pervasive in AI. Let me elaborate with an example. A few years ago,
Allegheny County in the U.S. launched the Allegheny Family Screening Tool (AFST). The tool uses predictive risk modeling
to rate incoming calls on general child protective issues. AFST uses upwards of
100 variables to generate a score from 1 to 20 – based on which calls may be
flagged for further investigation of families, where a child’s welfare could be
at risk. However, AFST has also received criticism for bias against low-income families who might be scrutinized more. This creates a
conundrum – does the county continue to use the tool with underlying concerns
or leave the task up to the interpretation of a sole unchecked human? The issue
of bias in AI is not straight-forward as we think, and requires a deeper
understanding of various factors at play.
According to
a WNS
DecisionPoint™ report[SN1] on the ethical use of AI, there are two primary
causes for prejudice, namely:
§
Dataset
Challenges
§
Targeted
Output Selection
Let me dive
into these two aspects a little more.
Dataset Challenges
When datasets
are not all-pervasive, the ability of AI models to accurately predict outcomes
is compromised. For example, a study of facial analysis techniques found that unrepresentative training data led
to higher error rates as far as minorities were concerned, especially women. Voice recognition tools such as Siri and Alexa, while trained on large
datasets, are known to understand the commands from white, upper-class
Americans more easily than others due to the over-representation of this segment
in the training data.
Natural
Language Processing (NLP) and Machine Learning (ML) technologies are designed
to help machines learn by observing patterns of human behavior. As a result, patterns
of human biases also creep in to negatively impact AI-driven, decision-making. Take
the example of a hiring tool built on AI that a leading company experimented with to review
resumes and shortlist candidates. The tool leveraged 10-year training data that
was skewed heavily in favor of men in the system. The AI tool, in extension,
showed the same bias and downgraded the resumes of women.
Targeted
Output Selection
Not assigning
the right goal or objective to an AI model can also result in unintended bias. Case
in point: grading essays through automated scoring that has its own biases. These tools consider
parameters such as the length of an essay and sophisticated words as key
criteria to assign high scores. They are unable to evaluate creativity or
detect gibberish. Designers will therefore need to reconsider the objectives
assigned to such tools to mitigate erroneous outputs.
The power of
AI is both exciting and worrisome. Therefore, detecting biases and building an
ethical AI framework is a moral obligation for everyone involved. It is a goal
we simply cannot afford to miss!
To know
more about creating ‘trust’ within an AI ecosystem, read the WNS
DecisionPointTM report[SP2]
Comments
Post a Comment