Judging the Algorithm

Excellent piece.    I have been involved in several projects which look at the correctness of algorithmic methods.   One point to make where a distinction is necessary.   A Black Box model implies the details of how and/or why a model works are unknown or are being hidden.  An algorithm usually implies the how/why of the model is known.  But its precise parameters may or may not be revealed.    'Black Box' implies fewer details are being revealed, and usually makes me worry more about the stability of the method being used.

The topic explored in this article below will become more crucial.  Algorithms need to be judged correct and compared for specific contexts.

The great question of the 21st century: Whose black box do you trust?
Algorithms shape choice not just for consumers but for businesses.
By Tim O'Reilly September 15, 2016

The role of technology-fueled algorithms in shaping our society, and how to use them responsibly, is one of the topics I'll be exploring at the Next:Economy Summit in San Francisco, October 10-11. Join me there.

Some years ago, John Mattison, the chief medical information officer of Kaiser Permanente, the large integrated health provider, said to me, "The great question of the 21st century is going to be 'Whose black box do you trust?'" Mattison was talking about the growing importance of algorithms in medicine, but his point, more broadly, was that we increasingly place our trust in systems whose methods for making decisions we do not understand. (A black box, by definition, is a system whose inputs and outputs are known, but the system by which one is transformed to the other is unknown.)

A lot of attention has been paid to the role of algorithms in shaping the experience of consumers. Much less attention has been paid to the role of algorithms in shaping the incentives for business decision-making. .... "