Probabilistic Fault Tolerance of Neural Networks in the Continuous Limit

Regularizer for failing neurons

This work derives a closed-form probabilistic model of how neuron failures (permanent or transient) propagate through a deep neural network in the continuous-width limit. It uses this model to design a regulariser that, when added to standard training loss, encourages the network to learn weight distributions that are inherently robust to runtime neuron failures, without requiring fault injection during training.