Responsibility for Program Output

Not quite understanding what they mean by responsibility here, hard to know how it will produce unexpected results.  Which might also diminish use of analytics as well.    Certainly you could promote an awareness of the further implications of output.

Assessing Responsibility for Program Output  

Communications of the ACM, August 2018, Vol. 61 No. 8, Pages 12-13
Robin K. Hill, University of Wyoming

Remember the days when record-keeping trouble, such as an enormous and clearly erroneous bill for property taxes, was attributed to "computer error?" Our technological society fumbles the assignment of responsibility for program output. It can be seen easily in exaggerations like this, from a tech news digest: "Google's Artificial Intelligence (AI) has learned how to navigate like a human being." Oh, my. See the Nature article by the Google researchers2 for the accurate, cautious, description and assessment. The quote given cites an article in Fast Company, which states that "AI has spontaneously learned how to navigate to different places."4 Oh, dear.

But this is not the root of the problem. In the mass media, even on National Public Radio, I hear leads for stories about "machines that make biased decisions." Exaggeration has been overtaken by simple inaccuracy. We professionals in Tech often let this pass, apparently on the belief the public really understands machines and algorithms have no such capacity as is normally connoted by the term "decision"; we think the speakers are uttering our own trade shorthand. When we say "the COMPAS system decides that offender B is more likely to commit another crime than is offender D"1 (paraphrase mine), it is short for "the factors selected, quantified, and prioritized in advance by the staff of the software company Northpointe assign a higher numeric risk to offender B than to offender D." When the Motley Fool website6 says "computers have been responsible for a handful of 'flash crashes' in the stock market since 2010," it means that "reliance on programs that instantaneously implement someone's predetermined thresholds for stock sale and purchase has been responsible ... etc."

The trouble is that there is no handy way to say these things. The paraphrases here expose the human judgments that control the algorithms, but the paraphrases are unwieldy. For decades of software engineering, we have adopted slang that attributes volition and affect to programs. Observations can be found on Eric S. Raymond's page on anthropomorphization5. I doubt many hackers ascribe the intentional stance to programs; I suspect rather that programmers use these locutions for expedience, as the "convenient fictions that permit 'business as usual'."3 But the public misunderstanding is literal, and serious. .... "

Weapons of Math Destruction?

More on biases of algorithms.   Inevitable, and will require new kinds of transparency that require deep contextual expertise.  Only governments and big tech enabled companies will have these.    Which do you fear most? Personal or group or government biases can be added to an algorithm quite easily.  It won't be the market choosing.   More hidden than rogue.

‘Rogue Algorithms’ and the Dark Side of Big Data

In Knowledge@Wharton

 Most of us, unless we’re insurance actuaries or Wall Street quantitative analysts, have only a vague notion of algorithms and how they work. But they actually affect our daily lives by a considerable amount. Algorithms are a set of instructions followed by computers to solve problems. The hidden algorithms of Big Data might connect you with a great music suggestion on Pandora, a job lead on LinkedIn or the love of your life on Match.com.

These mathematical models are supposed to be neutral. But former Wall Street quant Cathy O’Neil, who had an insider’s view of algorithms for years, believes that they are quite the opposite. In her book, Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy, O’Neil says these WMDs are ticking time-bombs that are well-intended but ultimately reinforce harmful stereotypes, especially of the poor and minorities, and become “secret models wielding arbitrary punishments.” ... ' 

Outcome Bias and Generating Value

Thinking about outcome bias.   But sometimes its all we have.  Or can we always generate metadata to drive value?  In the HBR:

What We Miss When We Judge a Decision by the Outcome by Francesca Gino In the HBR

When people are judging the quality of leaders’ decisions, they tend to focus much more on outcomes than intentions. For example, they judge hiring decisions not on the basis of whether the decision was made thoughtfully or fairly but on whether the new employee performs well. They judge the quality of a product decision on whether the product was well received in the market, rather than the quality of the process that led to the decision in the first place.

As it turns out, this tendency affects virtually all human beings. When evaluating others’ actions, most people focus more on the outcome of decisions than on intentions, a phenomenon that psychologists call outcome bias. A decision — such as taking time to review an employee’s performance data and provide feedback carefully — is often judged to be lower in quality when it leads to a poor, rather than a good, outcome.

The outcome bias is costly to organizations. It causes employees and leaders to be blamed for negative outcomes even when they had good intentions and used a thoughtful decision-making process, considering all the information that should be taken into account.  ... "