Checklist for Debugging Networks

Not a bad high level view.     I would consider also frequently reviewing goals, data stability, Data biases and measure or sensor errors,  trending results.  Make sure to involve domain experts in every results review.  Take their input seriously.    The worst bug you can have is solving the wrong problem and not knowing it.

Checklist for debugging neural networks

Tangible steps you can take to identify and fix issues with training, generalization, and optimization for machine learning models    By Cecelia Shao

Machine learning code can be notoriously difficult to debug with bugs that are expensive to chase. Even for simple, feedforward neural networks, you often have to make several decisions around network architecture , weight initialization, and network optimization — all of which can lead to insidious bugs in your machine learning code.

As Chase Roberts wrote in an excellent piece on ‘How to unit test machine learning code’, his frustrations stemmed from common traps like:

The code never crashes, raises an exception, or even slows down.
The network still trains and the loss will still go down.
The values converge after a few hours, but to really poor results
So what is to be done about it?  ..... "