Neural Capacity Estimators: How Reliable Are They?

Recently, several methods have been proposed for estimating information-theoretic measures, such as Kullback-Leibler divergence and mutual information from sample data using neural networks. These methods have been shown to achieve state-of-the-art performance, especially for high-dimensional data. Inspired by these results, a few new methods have been proposed for estimating the channel capacity from sample data without knowing the underlying channel models. In this work, we explore different techniques that can be used to estimate the capacity from sample data using neural networks. We then focus on several well-known point-to-point and multiple access channels and evaluate how neural capacity estimation performs compared to known results and bounds. We also focus on the learned input distribution by the neural capacity estimator and compare it to known optimal input distributions. Our results suggest that while neural capacity estimation may not be precise, it can be computationally efficient compared to other known numerical methods and can learn input distributions that are capacity-approaching.