Network Pruning Can Skew Deep Learning Models

Making Better Models

Network Pruning Can Skew Deep Learning Models

North Carolina State University News

Matt Shipman, November 2, 2022

Computer science researchers at North Carolina State (NC State), Syracuse, and Carnegie Mellon universities have shown that neural network pruning can undermine the performance of deep learning models at identifying certain groups. The researchers cited disparities in gradient norms across groups, and in Hessian norms linked to inaccuracies of a group's data, as factors impacting performance. This implies network pruning can compound existing accuracy deficiencies. NC State's Jung-Eun Kim said the team has demonstrated a remedial mathematical method "to equalize the groups that the deep learning model is using to categorize data samples." Tests of the mitigation technique found basically restored a deep learning model to pre-pruning levels of accuracy.  ... '

Full Article