Try Solving with Optimization

With all the talk about AI in the air, some of the other fundamental methods are being forgotten.   I spent most of my career using and teaching direct optimization methods in the enterprise.   Below a quick overview.   I am not particularly recommending this particular company, but they did a good job presenting the description.  Consider optimization .... it can be be better, easier to use and more direct than AI for the right applications.  It often uses less data.  It is often used for very complex models.   Every decision problem solver should have it in their capabilities.   As a manager I always ask, have we tried an optimization?   Have said it here many times, here it is again. 

What Must I Do to Use a Solver?

To use a solver, you must build a model of your decision problem that specifies:

The decisions to be made, called decision variables,

The measure to optimize, called the objective,

Any logical restrictions on potential solutions, called constraints.

The solver will find values for the decision variables that satisfy the constraints while optimizing (maximizing or minimizing) the objective. ...  " 

What will Quantum Computing Mean?

A largely non-technical view of what Quantum Computing will mean.  Our own minor investigations looked at how very complex combinatorial problems (problems with many, many solutions) that might then be solved with these methods more easily.   These problems also relate to things like cryptography.

You Won't See Quantum Internet Coming   By Ryan F. Mandelbaum   in Gizmodo.

 The quantum internet is coming sooner than you think—even sooner than quantum computing itself. When things change over, you might not even notice. But when they do, new rules will protect your data against attacks from computers that don’t even exist yet.

Despite the fancy name, the “quantum internet” won’t be some futuristic new way to navigate online. It won’t produce any mind-blowing new content, at least not for decades. The quantum internet will look more or less the same as the internet you’re using now, but scientists and cryptographers hope it could provide protection against not only theoretical threats but also those we haven’t dreamed up yet.

“The main contribution of a quantum internet is to allow encrypted communication in a perfectly secure fashion that can’t be broken in principle, even if in the future we develop a more fundamental theory of physics,” Ciarán Lee, a researcher at University College, London, explained to Gizmodo. In short, the quantum internet would hopefully protect us from planned new computers, along with every theoretical computer for the foreseeable future.

So what’s the quantum internet? It’s what happens when you apply the weird rules of quantum mechanics to the way computers communicate with one another.  ... " 

Statistical Engineering Framework for Solving Large, Complex, Unstructured Problems

I got this late, I think you can still attend, I plan to ...

 ... On November 13, there will be a Chapter Meeting of the American Statistical Association Cincinnati Chapter (you do not need to be a member to attend!) at P&G's Mason Business Center (8700 Mason Montgomery Rd, Mason, OH 45040) from 2pm to 4 pm. There will be a talk given on Statistical Engineering by Allison Jones-Farmer (Van Andel Professor of Business Analytics & Professor, Farmer School of Business, Miami University) and William A. Brenneman (Research Fellow and Global Statistics Discipline Leader, Data and Modeling Sciences, Procter & Gamble Company). The abstract is provided below.

If you plan to attend, please send Jeremy Christman (christman.jc@pg.com) a note so that you can be added to the visitor list at P&G by Novenber 9.  (Send a note ASAP)

The Statistical Engineering Framework for Solving Large, Complex, Unstructured Problems:

More below the fold: 

When to Hold Em, When to Compete

Nash Equilibrium's use in Competitive Situations is re-examined, with hope for its use in competitive behavior situations.  We looked at it for that and found no golden egg, but it doesn't mean others couldn't find it.   A reexamination.   Complexity technical.  Just because optimum solutions are known to be probably impossibly hard, very good solutions are probably better than what we are doing today.

When to Hold 'Em    By CACM Staff 
Communications of the ACM, Vol. 61 No. 6, Pages 6-7
10.1145/3210585

Neil Savage deserves praise for his informative overview of recent computational results related to Nash equilibrium in his news story "Always Out of Balance" (Apr. 2018). I fully agree that the notion of Nash equilibrium does not always reflect how competitors behave in competitive situations, and that the fact that Nash equilibrium is provably computationally intractable makes it less useful than John Nash himself might have envisioned when he developed it. However, Savage also overstated (somewhat) the effect of intractability by claiming the intractability of computing Nash equilibrium necessitates researchers abandon this notion in favor of other competition-related ideas.

While looking for Nash equilibrium yields additional computational complexity, the decision-making problem is, in general, already computationally intractable (NP-hard) for non-competitive situations (such as when a company makes internal planning decisions). In doing so, a company would be looking for an optimal solution (such as one that would aim to help produce maximum profit), but computational optimization is, in general, NP-hard. Such computational intractability does not mean researchers have to abandon the idea of optimization and look for other ideas. Many real-life problems are NP-hard (such as robotic movement) and what makes working on them such an intellectual and computational challenge.

Indeed, there is no general feasible algorithm (unless P = NP), so computer scientists need to be creative when designing algorithms for specific practical problems.  .... " 

Vladik Kreinovich, EL Paso, TX, USA

Geographic Optimization with Bayesian Networks

Had not seen this kind of optimization before with Bayesian Networks.  Webinar leads you through the process, largely non-technical.

 ... By Stefan Conrady, 
Managing Partner at Bayesia USA & Singapore: Bayesian Networks for Research, Analytics, and Reasoning .... 

Geographic Optimization with Bayesian Networks and BayesiaLab
You may not know that you can use BayesiaLab for geographic optimization. Today's webinar explained how you can find an optimal location for a distribution hub that needs to connect thousands of geographically dispersed suppliers and customers. Bayesian networks and BayesiaLab make this type of optimization remarkably quick and easy. ...  "

How to be a Systems Thinker

Podcast video interview.   Its a good idea to create deep understanding.  But shallower understanding sometimes leads to things that are useful,  there has been quite a long history of engineering to show that.  Thoughtful piece,

How To Be a Systems Thinker
A Conversation With Mary Catherine Bateson [4.17.18]

Until fairly recently, artificial intelligence didn’t learn. To create a machine that learns to think more efficiently was a big challenge. In the same sense, one of the things that I wonder about is how we'll be able to teach a machine to know what it doesn’t know that it might need to know in order to address a particular issue productively and insightfully. This is a huge problem for human beings. It takes a while for us to learn to solve problems, and then it takes even longer for us to realize what we don’t know that we would need to know to solve a particular problem.

The tragedy of the cybernetic revolution, which had two phases, the computer science side and the systems theory side, has been the neglect of the systems theory side of it. We chose marketable gadgets in preference to a deeper understanding of the world we live in.

MARY CATHERINE BATESON is a writer and cultural anthropologist. In 2004 she retired from her position as Clarence J. Robinson Professor in Anthropology and English at George Mason University, and is now Professor Emerita. Mary Catherine Bateson's Edge Bio  ... "

Google Deepmind for Navigation

Another kind of problem, which may well have applications in industry.

Google DeepMind's AI Learns Human Navigation Skills 
In The Guardian  By Ian Sample

Google's DeepMind unit has developed an algorithm that outperforms people in solving a virtual maze, after noting that it spontaneously generated electrical activity similar to that of "grid cells" governing human navigational skills. The scientists first built a deep neural network and taught it navigation fundamentals, inputting the types of signals that encode speed and direction in the brains of foraging rats. Feedback caused the network to improve its predictions of its location as it navigated a virtual environment. The team observed that 25 percent of the artificial neurons in one network layer had begun firing like organic grid cells. They then assembled a more refined network and applied it to the maze game. Tests revealed that the algorithm not only employed grid cells for position tracking, but also to formulate the direction and distance to its objective so it could follow the most direct pathway.   .. " 

(Updated) Optimization using Genetic Methods

In our earliest days,  addressing supply chain and blending type manufacturing problems, we were an optimization shop.  Using the math structure of difficult combinatorial problems to find best solutions based on known goals and constraints.    But if you couldn't glean enough low level structure, we tested genetic methods, described here.   In this era of faster machines and more contextual information even more useful to try today.  Also for certain kinds of structure, also consider Dynamic Programming.  Happen to be examining that again today.

In KDNuggets  By Ahmed Gad, KDnuggets Contributor 

This article gives a brief introduction about evolutionary algorithms (EAs) and describes genetic algorithm (GA) which is one of the simplest random-based EAs.

Selection of the optimal parameters values for machine learning tasks is challenging. Some results may be bad not because the data is noisy or the used learning algorithm is weak, but due to the bad selection of the parameters values. This article gives a brief introduction about evolutionary algorithms (EAs) and describes genetic algorithm (GA) which is one of the simplest random-based EAs.

Introduction

Suppose that a data scientist has an image dataset divided into a number of classes and an image classifier is to be created. After the data scientist investigated the dataset, the K-nearest neighbor (KNN) seems to be a good option. To use the KNN algorithm, there is an important parameter to use which is K. Suppose that an initial value of 3 is selected. The scientist starts the learning process of the KNN algorithm with the selected K=3. The trained model generated reached a classification accuracy of 85%. Is that percent acceptable? In another way, can we get a better classification accuracy than what we currently reached? We cannot say that 85% is the best accuracy to reach until conducting different experiments. But to do another experiment, we definitely must change something in the experiment such as changing the K value used in the KNN algorithm. We cannot definitely say 3 is the best value to use in this experiment unless trying to apply different values for K and noticing how the classification accuracy varies. The question is “how to find the best value for K that maximizes the classification performance?” This is what is called optimization.

In optimization, we start with some kind of initial values for the variables used in the experiment. Because these values may not be the best ones to use, we should change them until getting the best ones. In some cases, these values are generated by complex functions that we cannot solve manually easily. But it is very important to do optimization because a classifier may produce a bad classification accuracy not because, for example, the data is noisy or the used learning algorithm is weak but due to the bad selection of the learning parameters initial values. As a result, there are different optimization techniques suggested by operation research (OR) researchers to do such work of optimization. According to [1], optimization techniques are categorized into four main categories:  .... " 

  (Update) A comment I got made me add this.  'Optimization' in business practice implies you can get the provably, best possible solution to a problem.   But in reality it almost always means you only can get the best solution within some specific context.     A context can include structure, constraints and goals.    It may also vary over time.    It may be wrong because its too hard to completely understand the problem.  But its still often useful to get a better solution, even if not provably optimal, if its better than todays practice.     Further if you can calculate this 'theoretical' best solution, it can give you better understanding of a problem, and what to strive for.    - FAD 

Solving Problems

Nice general view of many techniques used to address problem solving ...  We used all of these from time to time.   Often one method is good to check another.

13 Methods to Forecast, Analyze, and Solve Problems
the-best-problem-solving-methods
Posted by Iryna Viter

Science of Brute Force

A technical point, but with important implications.   Analysts look for Algorithms, or simplified statements of what will be solutions in specific contexts.   These predict.   Algorithms can be a set of math or logical statements, or can be a universal proof of some truth we can apply.  It turns out that some algorithms are best addressed by brute force, or by exploring their context space, by iterating through many, many potential solutions.  This is called 'brute force', and is covered in the most recent issue of the Journal of the CACM, linked to below.   Also see this overview Vimeo presentation.   Technical.

The Science of Brute Force
Many relevant search problems, from artificial intelligence to combinatorics, explore large search spaces to determine the presence or absence of a certain object. These problems are hard due to combinatorial explosion, and have traditionally been called infeasible. The brute-force method, which at least implicitly explores all possibilities, is a general approach to systematically search through such spaces. ... " 

Brute force has long been regarded as suitable only for simple problems. This has changed in the last two decades, due to the progress in Satisfiability (SAT) solving, which by adding brute reason renders brute force into a powerful approach to deal with many problems easily and automatically. Search spaces with far more possibilities than the number of particles in the universe may be completely explored. ... " 

Machine Learning and Emergence

Complete article at the link in DSC:

The E-Dimension: Why Machine Learning Doesn’t Work Well for Some Problems?
by Shahab Sheikh-Bahaei, Ph.D.*
Principal Data Scientist,  Intertrust Technologies.

Introduction
Machine Learning (ML) is closely related to computational statistics which focuses on prediction-making through the use of computers. ML is a modern approach to an old problem:  predictive inference. It makes an inference from “feature” space to “outcome/target” space. In order to work properly, an ML algorithm has to discover and model hidden relationships between the feature space and the outcome space and create links between the two. Doing so requires overcoming barriers such as feature noise (randomness of features due to unexplained mechanisms).

In this article we argue that “Emergence” is also a barrier for predictive inference. Emergence is a phenomenon through which a completely new entity arises (emerges) from interactions among elementary entities such that the emerged entity exhibits properties the elementary entities do not exhibit. We present the idea that success of machine learning, and predictive inference in general, can be adversely affected by the phenomena of emergence. We argue that this phenomena might be partially responsible for unsuccessful use of current ML algorithms in some situations such as stock markets. .... " 

Real World Statistics

Its good to remember that statistics is the basis of all useful problem solving:

Making Statistics Work in the Real World

Wharton's Bhaswar Bhattacharya discusses his research on statistical methods.

The field of statistics is about more than just crunching numbers. Wharton statistics professor Bhaswar Bhattacharya is researching the best ways to apply statistical methods to solve problems in a range of fields, from health care to marketing to languages. Bhattacharya spoke with Knowledge@Wharton about shedding new light on one of the oldest mathematical disciplines.  

(Podcast) An edited transcript of the conversation follows. .... 

Knowledge@Wharton: Could you give us a brief summary of your research and what kind of question you were trying to answer?

Bhaswar Bhattacharya: My research interests are the intersection of statistics probability and combinatorics. Recently, numerous and very interesting combinatorial and graph theory-related problems have emerged in statistics, mainly because of the ubiquitous presence of network data and the increasing use of graph-based methods in modern-day analytics. As a consequence, many interesting connections have emerged between modern statistical methods and classical concepts in geometry and probability. You can use them to solve interesting problems in statistics. .... " 

Vertex Solution Science

Brought to my attention:  Vertex

" ... Every business faces challenges. There are some challenges that - if persist longer –turn into problems. So, you need solutions that can address these challenges before they turn into obstacles for your business.

We 'dub' ourselves 'Solution Scientists", and rightly so, as we craft innovative solutions with a holistic view that make your business smarter. We are not just technology consultants and problem solvers – we are your strategic partner, aligning our solutions with your business goals and objectives.

Over 15 years of experience in IT solutions & services  
More than 13 years of association with Fortune companies  
300+ consultants with knowledge of technologies across business verticals  
5 locations across U.S & India  
Process mature organization (CMMI , ISO, ITIL)  
High customer retention through our engagement maturity & value focus   ... " 

Blog.

Deep Learning Use and Best Practices

Very interesting introductory piece for the newcomer to using deep learning in a professional setting. Not quite for the novice, but if you have data manipulation, problem solving and coding experience with other systems, this could be very useful.  

I also always like best practice suggestions for incorporation in the enterprise.   Lots of time and effort to be saved and correctness to be achieved. Having used neural networks in the 90s,  then abandoned them, it is always remarkable to me how they have been re-born for great value.

Best Practices for Applying Deep Learning to Novel Applications

Leslie N. Smith,   US Naval Research Laboratory,   April 5, 2017

 .... This report is targeted to groups who are subject matter experts in their application but deep learning novices. It contains practical advice for those interested in testing the use of deep neural networks on applications that are novel for deep learning. We suggest making your project more manageable by dividing it into phases. For each phase this report contains numerous recommendations and insights to assist novice practitioners.   .... 

Subjects: Software Engineering (cs.SE); Artificial Intelligence (cs.AI); Neural and Evolutionary Computing (cs.NE)
Report number: NRL Technical Note 5510-052
Cite as: arXiv:1704.01568 [cs.SE]
  (or arXiv:1704.01568v1 [cs.SE] for this version)

   Full Report as a PDF

Acting Quickly

 In the HBR,     Its part of the risk model.

How to Act Quickly Without Sacrificing Critical Thinking  by Jesse Sostrin

An unbridled urgency can be counterproductive and costly. If you’re too quick to  react, you can end up with short-sighted decisions or superficial solutions, neglecting underlying causes and create collateral damage in the process.

But if you’re too deliberative and slow to respond, you can get caught flat-footed, potentially missing an opportunity or allowing an emergent challenge to consume you.

To balance these two extremes, you need reflective urgency — the ability to bring conscious, rapid reflection to the priorities of the moment — to align your best thinking with the swiftest course of action. In my work, coaching leaders at every level through a variety of management dilemmas, I’ve developed three strategies to practice reflective urgency: ... "

Structured Problem Solving

And taking it beyond to business process models?    Not commonly done well enough to define the structure.

In Sloan Review: 

Saving Money Through Structured Problem-Solving

Research Feature March 21, 2017  Reading Time: 6 min
Nelson P. Repenning, Don Kieffer, and Michael Morales

Closely observing how work really gets done in your organization can yield numerous opportunities for improvements.

Executives can’t lead effective organizational change just by sitting in their offices. As busy as they are, leaders need to find ways to observe fundamental work processes in their organizations. When they do, they usually discover that there are gaps between concept and reality in how work gets done. Michael Morales’ experience — in which identifying and addressing such gaps led to his company saving $50,000 in just 60 days — is a case in point. .... " 

Applying Design Thinking in the Organization

Good introduction to design thinking.  With pointers to lots of resources.

How to apply design thinking in your organization
Design thinking helps organizations grow, innovate, and improve financial performance.

By Jonathan Follett, Mary Treseler.   February 22, 2017

Design thinking is a process that uses design principles for solving complex problems. It helps organizations identify opportunities, unlock innovation, and improve their businesses.

Market leaders as varied as Apple, IBM, Intuit, Kaiser Permanente, and Nike have used design thinking to gain a competitive advantage, applying it to create innovative products and services. Within an organization, design thinking is a tool for unlocking cultural change. It makes companies more flexible, more responsive to their customers, and ultimately, more successful.

What are the elements of design thinking?

Although the name and number of its key principles may vary depending on how you apply them, the basic elements of design thinking always include some variation on the following: researching and defining the problem, ideating, and prototyping and iterating.  .... " 

Human Centered Design

Been a while since I have taken a look at the concept.  I like it, but I have seen accurate measurement ignored when design thinking was invoked.   Interesting view from O'Reilly.

 What is design thinking?

Human-centered design and the challenges of complex problem-solving.
By Jonathan Follett

" ... Design thinking is an abductive approach to complex problem solving that leverages the designer’s empathetic mindset in order to understand people’s unarticulated needs and identify opportunities for solutions. This is a human-centered innovation process that can be applied to a wide range of challenges: design thinking can be used to create everything from products and services to business models and processes. .... " 

Quest for a Topological Quantum Computer

In the CACM.  Pointing to a Nature article.  Was part of a group that looked at potential practical applications in complex supply chain applications.

' ... The race is on build a "universal" quantum computer. Such a device could be programmed to speedily solve problems that classical computers cannot crack, potentially revolutionizing fields from pharmaceuticals to cryptography. ... " 

Most Important Skill: Problem Solving

Not know the latest code or algorithms, just solve the problem.   In the simplest way possible.  If you can just test solution X easily, just test X!   So true. You have a toolbox full or tools, and very importantly also a book full of resources to call.   You can't know it all, so its good to have access to statisticians, data architects, process experts, decision wonks, domain historians,  coding regulation librarians and ....   Hope you will not need all of these.  And for sheer efficiency, you already have skills in a number of these areas as well.   Always,  Keep it simple as possible.   And when you care done, make sure the problem is really solved.

Nicely stated, and from the DSC:

The Only Skill you Should be Concerned With  ... Posted by Sean McClure
"The languages you learn, the technologies you use, and the way you frame your thoughts will be a byproduct of your attempts to solve the problem."

" ... When you are trying to solve challenges you don't use a language because it happens to be hyped. You don't use a technology stack because some expert of the day said this is how you do big data. You don't use a specific implementation of lean because you read an article by a millionaire who swears by it. When you are solving problems the only thing that matters is SOLVING THE PROBLEM. What is it the client actually needs? Put all the toys that the cool kids are playing with to the side and have an honest conversation about the problem that needs a solution. This is the only criterion that should govern what approach you take and...here's the key...your skills in the decided-upon tools of choice will grow as a result. .... "