New York AI Bias Law Prompts Uncertainty

 New York AI Bias Law Prompts Uncertainty

The Wall Street Journal, Richard Vanderford

September 21, 2022

A law that goes into effect in New York City in January requires companies to audit their artificial intelligence (AI) hiring systems to assess whether they incorporate racial and gender biases. The law holds hiring companies liable for any biases and could impose fines for violations, but lacks clear guidelines for the AI audit process. While the city’s Department of Consumer and Worker Protection has not offered a timeline for when it will publish rules to implement the law, some companies already are taking steps to comply. Said Anthony Habayeb of AI governance software company Monitaur Inc., "Instead of waiting for someone to tell me what to do…I built controls around these applications because I know like with any software, things can and do go wrong."  ... ' 

Fixing Ad Bias

 Example of AI being used to address bias.

An AI Fix to Advertising Bias? Industry Giants Commit to IBM's Tech

WPP, Delta Air Lines, the 4A's tap computing giant's anti-bias toolkit

CANNES, France — A coalition of industry giants, including WPP, Delta Airlines, the 4A’s, Kellogg and Mindshare, have joined forces with IBM to combat bias in digital marketing.

Since the programmatic revolution of more than a decade ago, brands have used algorithms to determine which demographics to target and how to segment audiences in their campaigns. But they are far from perfect.

These algorithms have created a world where women are served ads for household goods while men are targeted by automakers. The bias is often an unintentional consequence of human assumptions, but the real danger has been technology’s ability to capture and scale that bias, alienating the audiences advertisers seek to connect with in the first place.

Today at Cannes Lions, IBM onboarded some of advertising’s major players to use its open-source toolkit to identify and mitigate ad tech bias.

Delta Air Lines, WPP, Mindshare, 4A’s, IAB and the Ad Council are among those committed to using up IBM’s Advertising Toolkit for AI Fairness 360, an open-source solution powered by Watson.

“While the risk of bias in advertising is well known, by making this commitment, these organizations are among the first in the industry to take action,” said Bob Lord, IBM senior vice-president of The Weather Company and Alliances, at a press event during Cannes Lions.

Here’s how it works: during media targeting, the toolkit deploys 75 “fairness” metrics and 13 state-of-the-art algorithms to help uncover bias in discrete data sets.

IBM claimed organizations that use the toolkit may gain a better understanding of the presence and impact of bias on their ad campaigns. The company is asking other advertising giants to make use of the software and pledge to break the bias in datasets.

“Together, we are agreeing to educate ourselves and our companies and ask other industry leaders to join us in helping to mitigate bias in advertising,” Lord added.

IBM has already tested the tool internally. Elsewhere, WPP-owned media agency Mindshare and the Ad Council have taken part in the research phase too.

The business case for removing bias

“We have to eliminate this bias, because it’s no good for society,” asserted WPP boss Mark Read. “There’s also a business case, if you’re biased you’re missing out on [commercial opportunities] and we don’t want to miss out on anything.”  ... ' 

Clearview and Facial Recognition.

 Remain intrigued by facial recognition. consider the similarity of its emergence to fingerprint techniques.  A comparison is useful. Can having enough data ultimately remove error? 

Why Facial Recognition Technology Has an Uncertain Future with Small Business.  Clearview AI's co-founder Hoan Ton-That recently defended his startup's use of controversial facial recognition software.

BY STEVEN I. WEISS, SENIOR STAFF WRITER

@STEVENIWEISS in Inc.

If you're skeptical about whether your company will ever use facial recognition technology as a business tool, you're not alone. Perhaps the most prominent facial recognition technology provider in the world, Clearview AI, has attracted significant criticism and raised ethical concerns even as it has been used by law enforcement.  

In a live interview with the Washington Post last week, New York-based Clearview AI's co-founder and CEO Hoan Ton-That addressed questions about the ethical and legal implications of his software, which became first known to many Americans when a billionaire used it to identify his daughter's dinner date, and for the involvement of far-right individuals in the creation of the company. Pressed on questions about the legal and ethical choices his firm has made while creating a searchable database of 20 billion facial images, Ton-That repeatedly brought up examples where the use cases of Clearview AI's technology would look better in the public eye, mentioning its use in helping catch criminals in child pornography and child abuse cases. Ton-That also pointed to the use of Clearview AI's technology by the Ukrainian government to identify dead Russian soldiers, for notifying their families of their passing.

While Clearview AI has some 20 billion facial images to feed its current product, the dataset is being used only by governments so far. "There's no non-governmental use of this dataset at this time," Ton-That said, adding that "we've developed as prototypes different versions of our technology for retail and banking."

Article continues after video. .... 

Epidemic Research

Simulation and Paradoxes are noted. 

Researchers Outline Bias in Epidemic Research, Offer Simulation Tool to Guide Future Work

New York University, March 31, 2022

Researchers at New York University (NYU) have identified biases in epidemic research and developed a simulation tool to improve research methodologies. The researchers examined the Grue Paradox, Simpson's Paradox, confirmation bias, and other paradoxes, fallacies, and bias in the context of epidemic research and the COVID-19 pandemic. They developed the open source Epidemic Simulation (Episimmer) platform to offer decision support. Episimmer undertakes "counterfactual" analyses, which involve measuring the impact to an ecosystem of such paradoxes, fallacies, and bias without interventions or policies to help users identify opportunities and optimizations that could be included in their COVID-19 strategies. Said NYU's Inavamsi Enaganti, "Faced with a rapidly evolving virus, inventors must experiment, iterate, and deploy both creative and effective solutions while avoiding pitfalls that plague clinical trials and related work."  .... ' 

Ethical AI

 Via DSC, below the intro, much more at the link:

The Ethical AI Application Pyramid

Posted by Bill Schmarzo  In the blog “How Can Your Organization Manage AI Model Biases?”, I wrote:

“In a world more and more driven by AI models, Data Scientists cannot effectively ascertain on their own the costs associated with the unintended consequences of False Positives and False Negatives. Mitigating unintended consequences requires the collaboration across a diverse set of stakeholders in order to identify the metrics against which the AI Utility Function will seek to optimize.”

I’ve been fortunate enough to have had some interesting conversations since publishing that blog, especially with an organization who is championing data ethics and “Responsible AI” (love that term). As was so well covered in Cathy O’Neil’s book “Weapons of Math Destruction”, the biases built into many of the AI models that are being used to approve loans and mortgages, hire job applicants, and accept university admissions are yielding unintended consequences that severely impact both individuals and society.

AI models only optimize against the metrics against which it has been programmed to optimize. If the AI model yields unintended consequences, that’s not the AI models fault. It’s the fault of the data science team and the operational stakeholders who are responsible for defining the AI Utility Function against which the AI model will judge model progress and success (see Figure 1).  ... " 

Getting Ideas from the Crowd

Only part of an article from Sloan on creative uses of crowdsourcing.   Also a favorite topic and means  of our data gathering. How much is the crowd (generally) biased, and how do we reasonably adjust for that?  Elements of reasonable and directional bias? Can we get a good indication of types of bias in play in a particular crowd? 

Get More Ideas From the Crowd

These five techniques for writing problem statements can improve results from crowdsourced challenges.

Claudia Kubowicz Malhotra, Arvind Malhotra, and Barry L. Bayus

The rise of crowdsourcing platforms as a potential source for innovative ideas presents a challenge: How do you attract contributors to work on your particular problem?1 Past research has demonstrated the importance of well-crafted problem statements as a means to attract more innovative solutions.2 But what really goes into a problem statement that engages the crowd? Do the statements that attract a large number of proposed ideas share common elements?

Our research sought to answer these questions by unpacking problem statements, word by word, to identify the characteristics that attract the most idea submissions. Our analysis points to guidelines for managers tapping crowdsourcing sites on do’s and don’ts when writing a problem statement meant to attract solutions from creative freelancers.

We used data from Eÿeka, an online crowdsourcing platform focused on creative projects. Our findings are based on an analysis of 362 unique problem statements posted by 85 companies between 2016 and 2018. The statements sought ideas for marketing and social media campaigns, solutions to complex issues, proposals for products, and recommendations for entering new markets. The average number of responses received for each statement was 88; the highest number received was 370 and the lowest was five. We found that those receiving an above-average number of submissions shared some common elements — and we also identified four approaches to avoid when presenting a challenge to the crowd.

Five Ways to Engage Creatives

Our research found that the problem statements that attracted an above-average number of proposals used one or more of the following five techniques to pique freelancers’ interest and engage them with the problem.

Personalize the problem. Statements that address freelancers as “you” and explicitly ask them to solve the problem as if they were the customer are very effective in attracting more ideas.

For example, a company wanted to invent a new coffee drink. “We are sure you’ve known these moments in the afternoons when you just need to take a break from sleepiness, stress, or anxiety,” the statement noted. It described the coffee shop environment and referred to the opportunity to “enjoy some sweet time all by yourself or maybe sharing this moment with your friends.” And it concluded with a direct appeal: “We’d like to invent a new range of cold and sweet, ready-to-drink, milk-based coffee that people will love.   ... ' 

Upcoming Talk: How Will the AI Genie Behave?

You are all cordially invited to the following 29th Arnoff-Schloss lecture, which will be on Zoom on Friday, September 18th, 2020, from 3:30PM to 4:30PM. Please register at https://us02web.zoom.us/webinar/register/WN_Dmhl-hxnTqyf9tkh01uwZQ 

The lecture is open to the public, so please feel free to distribute the lecture to any parties that might be interested.

Please visit https://business.uc.edu/faculty-and-research/departments/obais/research/arnoff-schloss-memorial-lectures.html    for more information about the Arnoff-Schloss Memorial Lectures. 

 -----------------

29th Annual E. Leonard Arnoff/Milton J. Schloss Memorial Lecture on the Practice of Business Analytics

How Will the AI Genie Behave

Arun Rai, Georgia State University

Regents' Professor of the University System of Georgia

J. Mack Robinson Chair of IT-Enabled Supply Chains and Process Innovation

We are witnessing the increasing ubiquity of AI, with accelerating deployment across task domains, entrepreneurial activity across industries, and embedding in core processes across digital platforms and firms. Alongside this increasing ubiquity of AI, we are encountering broad unintended consequences, both beneficial and harmful. These unintended consequences are arising from unexpected behaviors and downstream effects that emerge through interactions of AI agents with the environment and with other machine and human agents. To be able to control the actions of AI in ways that harvest its benefits while minimizing harm, we need to develop an integrated understanding of AI behavior in the environments in which it is deployed. I will offer an integrated perspective on AI behavior that considers how AI agents acquire their behavior, how the behavior is triggered and enacted, the interaction dynamics of AI agents with other agents, and the influence of policy decisions on how AI systems evolve. I will draw on this perspective to discuss the research, managerial, and policy implications related to AI behavior.

3:30-4:30 PM, Friday, September 18th, 2020

On Zoom:
Registration:https://us02web.zoom.us/webinar/register/WN_Dmhl-hxnTqyf9tkh01uwZQ  

What are Our Models Learning?

In some of my earliest work in statistics, we were informed of the "Clever Hans Effect" Which basically means your model is learning a pattern in the data which is unrelated to an the answer you seek.  It may work on the training and test sets, but the prediction evidence comes from some other  evidence hidden in the system.  This can happen when you have too much data,  or you are carrying along variables that are irrelevant.  The model might now work appropriately, but later drift to other results. Dangerous in autonomous systems.  In social data it can occur when humans hint at results  they want, which is why double-blind tests are constructed.   Overall rather important as we depend more on ML. ....

Unmasking Clever Hans predictors and assessing what machines really learn
Sebastian Lapuschkin, Stephan Wäldchen, Alexander Binder, Grégoire Montavon, Wojciech Samek & Klaus-Robert Müller 
Nature Communications volume 10, Article number: 1096 (2019)  

Abstract
Current learning machines have successfully solved hard application problems, reaching high accuracy and displaying seemingly intelligent behavior. Here we apply recent techniques for explaining decisions of state-of-the-art learning machines and analyze various tasks from computer vision and arcade games. This showcases a spectrum of problem-solving behaviors ranging from naive and short-sighted, to well-informed and strategic. We observe that standard performance evaluation metrics can be oblivious to distinguishing these diverse problem solving behaviors. 

Furthermore, we propose our semi-automated Spectral Relevance Analysis that provides a practically effective way of characterizing and validating the behavior of nonlinear learning machines. This helps to assess whether a learned model indeed delivers reliably for the problem that it was conceived for. Furthermore, our work intends to add a voice of caution to the ongoing excitement about machine intelligence and pledges to evaluate and judge some of these recent successes in a more nuanced manner.

Introduction
Artificial intelligence systems, based on machine learning (ML), are increasingly assisting our daily life. They enable industry and the sciences to convert a never ending stream of data—which per se is not informative—into information that may be helpful and actionable. ML has become a basis of many services and products that we use.

While it is broadly accepted that the nonlinear ML methods being used as predictors to maximize some prediction accuracy, are effectively (with few exceptions, such as shallow decision trees) black boxes; this intransparency regarding explanation and reasoning is preventing a wider usage of nonlinear prediction methods in the sciences (see Fig. 1a why understanding nonlinear learning machines is difficult). Due to this black-box character, a scientist may not be able to extract deep insights about what the nonlinear system has learned, despite the urge to unveil the underlying natural structures. In particular, the conclusion in many scientific fields has so far been to prefer linear models1,2,3,4 in order to rather gain insight (e.g. regression coefficients and correlations) even if this comes at the expense of predictivity.  ... " 

The article is also discussed here:
https://towardsdatascience.com/deep-learning-meet-clever-hans-3576144dc5a9

The Need for Loaded Dice

I remember having to explain to managers why dice had to be 'loaded' in some of the simulations we built.  Isn't that biased?  Yes, but being loaded is required in accurate models.   The article in Quanta Magazine shows why.  See also the link to the MIT work, which further explains the why.

How and Why Computers Roll Loaded Dice
Stephen Ornes, Contributing Writer   in QuantaMagazine

Researchers are one step closer to injecting probability into deterministic machines.
Here’s a deceptively simple exercise: Come up with a random phone number. Seven digits in a sequence, chosen so that every digit is equally likely, and so that your choice of one digit doesn’t affect the next. Odds are, you can’t. (But don’t take my word for it: Studies dating back to the 1950s reveal how mathematically nonrandom we are, even if we don’t recognize it.)

Don’t take it to heart. Computers don’t generate randomness well, either. They’re not supposed to: Computer software and hardware run on Boolean logic, not probability. “The culture of computing is centered on determinism,” said Vikash Mansinghka, who runs the Probabilistic Computing Project at the Massachusetts Institute of Technology, “and that shows up at pretty much every level.”

But computer scientists want programs that can handle randomness because sometimes that’s what a problem requires. Over the years, some have developed sleek new algorithms that, while they don’t themselves generate random numbers, offer clever and efficient ways to use and manipulate randomness. One of the most recent efforts comes from Mansinghka’s group at MIT, which will present an algorithm called Fast Loaded Dice Roller, or FLDR, at the online International Conference on Artificial Intelligence and Statistics this August. ... " 

How to Fight Bias in ML Based Experiences

Forrester piece with interesting interview and links:

How To Fight Bias In ML-Based Experiences  By Andrew Hogan in Forrester Blog

Recently, I interviewed Carol Smith about AI and ethics — she’s a Senior Research Scientist at Carnegie Mellon’s Software Engineering Institute, and she told me:

“You’re bringing yourself to the projects you do at work, and we’re all biased and flawed. We must accept that building a fancy system doesn’t change that. We’re going to make mistakes and there will be issues with what we make. The more imaginative we can be early on, the more prepared we can be for failure.”

I asked her about her paper “Designing Trustworthy AI: A Human-Machine Teaming Framework to Guide Development” because Forrester has written extensively about AI and ethics in reports and blog posts like this one: “The Ethics Of AI: Don’t Build Racist Models” and highlighted the importance of diverse teams in reports like this one Data-Fueled Products: How To Thrive On The Design And Data Science Collision. Here are the highlights that stood out to me from Carol’s paper and our conversation:  ... "

Emergence of the AI Risk Manager

In our day this was integrated with other kinds of analysis and management.   It may well make sense to focus this more generally. 

The emergence of the professional AI risk manager

By Kenn So  in Venturebeat

When the 1970s and 1980s were colored by banking crises, regulators from around the world banded together to set international standards on how to manage financial risk. Those standards, now known as the Basel standards, define a common framework and taxonomy on how risk should be measured and managed. This led to the rise of professional financial risk managers, which was my first job. The largest professional risk associations, GARP and PRMIA, now have over 250,000 certified members combined, and there are many more professional risk managers out there who haven’t gone through those particular certifications.

We are now beset by data breaches and data privacy scandals, and regulators around the world have responded with data regulations. GDPR is the current role model, but I expect a global group of regulators to expand the rules to cover AI more broadly and set the standard on how to manage it. The UK ICO just released a draft but detailed guide on auditing AI.  https://ico.org.uk/about-the-ico/ico-and-stakeholder-consultations/ico-consultation-on-the-draft-ai-auditing-framework-guidance-for-organisations/   The EU is developing one as well. Interestingly, their approach is very similar to that of the Basel standards: specific AI risks should be explicitly managed. This will lead to the emergence of professional AI risk managers.  ... "

Reducing Bias in AI by Forgetting

Intriguing thought.   When we built a learning system we included the notion of forgetting as an aspect of scheduling maintenance, re-running and retesting particular solutions.    But not in terms of changes in bias.  Which I would assume would be a re-test.

How to Reduce Bias in AI? Selective Amnesia.
USC Viterbi School of Engineering
Rishbha Bhagi
February 24, 2020

Artificial intelligence (AI) researchers at the University of Southern California (USC) Viterbi School of Engineering’s Information Sciences Institute have created a mechanism for inducing selective amnesia in computing models. This adversarial forgetting methodology could help reduce bias in AI by teaching deep learning models to ignore unwanted data factors. The mechanism is used to train a neural network to represent all underlying aspects of the data being analyzed, and then to forget specified biases, resulting in models that lack those biases when making decisions. Adversarial forgetting also could enhance content generation. USC's Greg Ver Steeg said, "For content generation to succeed, we need new ways to control and manipulate neural network representations and the forgetting mechanism could be a way of doing that."  ... "

Recidivism Prediction

An example of comparison between human and machine solution.

Algorithms are Better Than People in Predicting Recidivism, Study Says
Berkeley News
Edward Lempinen

A study by researchers at the University of California, Berkeley and Stanford University has found that algorithms can predict the likelihood that specific criminal defendants will be arrested for a new crime at some future point, with significantly greater accuracy than humans. In experiments, risk assessment algorithms were nearly 90% accurate in predicting which defendants might be arrested again, while human predictions were about 60% accurate. The researchers replicated previous research that evaluated recidivism likelihood based on a limited number of risk factors, but added other datasets to test the theory that real-world settings would make algorithmic assessment more effective than human evaluations. The results appear to support continued use and refinement of such algorithms. However, Stanford's Sharad Goel said, "Like any tools, risk assessment instruments must be coupled with sound policy and human oversight to support fair and effective criminal justice reform." ... 

Google Fairness Gym

A considerable effort reported on here to experiment with the broad idea of fairness in machine learning, via the notion of a 'gym' to exercise choices and results with varying data.    Article below has quite a bit of  detail on what this is trying to be.

ML-fairness-gym: A Tool for Exploring Long-Term Impacts of Machine Learning Systems
Wednesday, February 5, 2020
Posted by Hansa Srinivasan, Software Engineer, Google Research

Machine learning systems have been increasingly deployed to aid in high-impact decision-making, such as determining criminal sentencing, child welfare assessments, who receives medical attention and many other settings. Understanding whether such systems are fair is crucial, and requires an understanding of models’ short- and long-term effects. Common methods for assessing the fairness of machine learning systems involve evaluating disparities in error metrics on static datasets for various inputs to the system. Indeed, many existing ML fairness toolkits (e.g., AIF360, fairlearn, fairness-indicators, fairness-comparison) provide tools for performing such error-metric based analysis on existing datasets. While this sort of analysis may work for systems in simple environments, there are cases (e.g., systems with active data collection or significant feedback loops) where the context in which the algorithm operates is critical for understanding its impact. In these cases, the fairness of algorithmic decisions ideally would be analyzed with greater consideration for the environmental and temporal context than error metric-based techniques allow. ....  " 

Fairness Indicators for AI Machine Learning

Brought to my attention, dealing with a potential solution for a project at hand.   Good technical and non technical coverage.  The approach is now available in Beta. Examining.

Fairness Indicators: Scalable Infrastructure for Fair ML Systems
Wednesday, December 11, 2019
Posted by Catherina Xu and Tulsee Doshi, Product Managers, Google Research

While industry and academia continue to explore the benefits of using machine learning (ML) to make better products and tackle important problems, algorithms and the datasets on which they are trained also have the ability to reflect or reinforce unfair biases. For example, consistently flagging non-toxic text comments from certain groups as “spam” or “high toxicity” in a moderation system leads to exclusion of those groups from conversation.

In 2018, we shared how Google uses AI to make products more useful, highlighting AI principles that will guide our work moving forward. The second principle, “Avoid creating or reinforcing unfair bias,” outlines our commitment to reduce unjust biases and minimize their impacts on people.

As part of this commitment, at TensorFlow World, we recently released a beta version of Fairness Indicators, a suite of tools that enable regular computation and visualization of fairness metrics for binary and multi-class classification, helping teams take a first step towards identifying unjust impacts. Fairness Indicators can be used to generate metrics for transparency reporting, such as those used for model cards, to help developers make better decisions about how to deploy models responsibly. Because fairness concerns and evaluations differ case by case, we also include in this release an interactive case study with Jigsaw’s Unintended Bias in Toxicity dataset to illustrate how Fairness Indicators can be used to detect and remediate bias in a production machine learning (ML) model, depending on the context in which it is deployed. Fairness Indicators is now available in beta for you to try for your own use cases.  ... "

Avoiding Statistical Traps

 In KDNuggets.  One of the first things that practitioners in analytic spaces should study and understand.  I will also add that these should always be demonstrated at scale.    We  typically had examples with realistic scale and data types for the domain being considered.    Using general statements of the dangers, with small or out of domain data sets, usually did not make the dangers clear.

   Fallacies are what we call the results of faulty reasoning. By Matthew Mayo, KDnuggets. Statistical fallacies, a form of misuse of statistics, is poor statistical reasoning; you may have started off with sound data, but your use or interpretation of it, regardless of your possible purity of intent, has gone awry. Therefore, whatever decisions you base on these wrong moves will necessarily be incorrect.

There are infinite ways to incorrectly reason from data, some of which are much more obvious than others. Given that people have been making these mistakes for so long, many statistical fallacies have been identified and can be explained. The good thing is that once they are identified and studied, they can be avoided. Let's have a look at a few of these more common fallacies and see how we can avoid them.

Out of interest, when misuse of statistics is not intentional, the process bears a resemblance to cognitive biases, which Wikipedia defines as "tendencies to think in certain ways that can lead to systematic deviations from a standard of rationality or good judgment." The former builds incorrect reasoning on top of data and its explicit and active analysis, while the latter reaches a similar outcome much more implicitly and passively. That's not hard and fast, however, as there is definitely overlap between these 2 phenomena. The end results is the same, however: plain ol' wrong.

Here are five statistical fallacies — traps — which data scientists should be aware of and definitely avoid. The failure to do so will be catastrophic in terms of both data outcomes and a data scientist's credibility.  .... " 

Self-Consciousness

From the Edge.  What does consciousness mean, when is it necessary for useful intelligence?

News from Edge
To arrive at the edge of the world's knowledge, seek out the most complex and sophisticated minds, put them in a room together, and have them ask each other the questions they are asking themselves.

The Paradox of Self-Consciousness
A Conversation with Markus Gabriel

I have been trying, under the banner of "New Realism," to reconcile various philosophical and scientific traditions. I'm looking for a third way between various tensions. There's more to a human being than the fact that we are a bunch of cells that hang together in a certain way. Humans are not strictly identical to any material energetic system, even though I also think that humans cannot exist without being, in part, grounded in a material energetic system. So, I am rejecting both brutal materialism, according to which we are nothing but an arrangement of cells, and brutal idealism, according to which our minds are transcendent affairs that mysteriously peep into the universe. Both are false, so there has to be a third way.

Similarly, there must be a third way between postmodernism, which denies the objectivity of human knowledge claims and science altogether, and various trends in cognitive science, which also threaten objectivity without, of course, fully undermining it (for instance, research on cognitive biases better be immune to second-order biases). Similarly, I believe we urgently need to reconcile so-called continental philosophy—European traditions, broadly construed—and analytic philosophy, which means philosophy at its best when practiced in Anglophone context; there has to be something in between. That space in between is what I call New Realism. 

MARKUS GABRIEL, one of the founders of New Realism, holds the Chair for Epistemology, Modern and Contemporary Philosophy at the University of Bonn, where he is also Director of the International Center for Philosophy and the multidisciplinary Center for Science and Thought. He is the author of Why the World Does Not Exist.   .... "

Removing Bias from Predictive Modeling

Podcast of interest from Wharton re Bias, Podcast at the link.

Wharton's James Johndrow discusses his research on removing human bias from predictive modeling.

Predictive modeling is supposed to be neutral, a way to help remove personal prejudices from decision-making. But the algorithms are packed with the same biases that are built into the real-world data used to create them. Wharton statistics professor James Johndrow has developed a method to remove those biases. His latest research, “An Algorithm for Removing Sensitive Information: Application to Race-independent Recidivism Prediction,” focuses on removing information on race in data that predicts recidivism, but the method can be applied beyond the criminal justice system. He spoke to Knowledge@Wharton about his paper, which is co-authored with his wife, Kristian Lum, lead statistician with the Human Rights Data Analysis Group. (Listen to the podcast at the top of this page.)

An edited transcript of the conversation follows.

Knowledge@Wharton: Predictive modeling is becoming an increasingly popular way to assist human decision-makers, but it’s not perfect. What are some of the drawbacks?

James Johndrow: There has been a lot more attention lately about it, partly because things are being automated so much. There’s just more and more interest in having automatic scoring, automatic decision-making, or at least partly automatic decision-making. The area that I have been especially interested in — and this is a lot of work that I do with my wife — is criminal justice.  .... "

The Wisdom of Polarized Crowds

A topic that has long interested me.  How biased is the Wikipedia?

Wikipedia and the Wisdom of Polarized Crowds
A lesson in how to break out of filter bubbles.
By   Brian Gallagher

In 2013, James Evans, a University of Chicago sociologist and computational scientist, launched a study to see if science forged a bridge across the political divide. Did conservatives and liberals at least agree on biology and physics and economics? Short answer: No. “We found more polarization than we expected,” Evans told me recently. People were even more polarized over science than sports teams. At the outset, Evans said, “I was hoping to find that science was like a Switzerland. When we have problems, we can appeal to science as a neutral arbiter to produce a solution, or pathway to a solution. That wasn’t the case at all.”

Paper:  https://www.nature.com/articles/s41562-019-0541-6

Illusory Truth Effect

Brought back to my attention by a Wired article on Viral scares now  floating about the internet, and when it makes sense to respond to them.    Made me reexamine this kind of 'bias' again.  The bias towards believing the familiar via frequent exposure.

... The illusory truth effect (also known as the validity effect, truth effect or the reiteration effect) is the tendency to believe information to be correct after repeated exposure ...   From the Wikipedia.

How to Not Fall for Viral Scares  in Wired. .... This is the illusory truth effect—a glitch in human reasoning that makes things that are familiar feel true. It’s why sometimes even fact-checking a lie can ultimately lead to more people believing it, because it increases the lie’s exposure. ... "