Tessellation-Filtering ReLU Neural Networks

Moser, Bernhard A.;Lewandowski, Michal;Kargaran, Somayeh;Zellinger, Werner;Biggio, Battista;Koutschan, Christoph

2022-01-01

---

Abstract

We identify tessellation-filtering ReLU neural networks that, when composed with another ReLU network, keep its non-redundant tessellation unchanged or reduce it. The additional network complexity modifies the shape of the decision surface without increasing the number of linear regions. We provide a mathematical understanding of the related additional expressiveness by means of a novel measure of shape complexity by counting deviations from convexity which results in a Boolean algebraic characterization of this special class. A local representation theorem gives rise to novel approaches for pruning and decision surface analysis.