Model Structure and States

Interesting research, technical.   with potential linkages to AI. 

Technical Perspective: Model Structure Takes Guesswork Out of State Estimation  By Sayan Mitra

Communications of the ACM, February 2022, Vol. 65 No. 2, Page 110    10.1145/3505268

Communication can often be exchanged with computation in control systems. A car's computer needing to know the speed can either get the data from the speed sensor over the vehicle's communication network (bus); or it can calculate the speed from the initial speed, the history of throttle commands, using the laws of physics driving the car. In a fully deterministic world with powerful enough computers, communication may be redundant. In the real world, the degree of uncertainty in the physics can say something about the level of communication necessary. Quantifying this communication need can help principled design and allocation of network bandwidth and other resources in vehicles and other control systems.

Uncertainty or lack of information is usually measured by entropy of some flavor. Claude Shannon developed a definition of entropy in the context of engineering telephone networks. That definition uses probability distributions, not coincidentally, capturing noise in telephone channels. In contrast, topological entropy, used in studying evolution of worstcase uncertainty in safety-critical systems, does not use probabilities at all. Instead, it measures the rate of growth of uncertainty in a system's state with time. Topological entropy of a stable system like a pendulum will be smaller than that of an unstable system like an inverted pendulum.

Why do we care about topological entropy? First, as entropy describes the rate of growth of state uncertainty (without new measurements), it should also somehow relate to the rate of measurements necessary to accurately estimate the state. In the speed sensor-estimator example, the entropy of the system would give the minimal channel capacity necessary for connecting the two, so the computer can construct accurate speed estimates with worst-case error bounds. These lower bounds hold across all algorithms and codes, and therefore, can take the guesswork out of communication network design. As more devices feed into shared networks, entropy bounds can guide allocation of bandwidth to different processes.  .... '