High-Dimensional Inference for Improved Machine Learning

Machine learning forms the backbone of many human-facing applications, from autonomous vehicles to computer-assisted medical diagnostics. These technologies often involve safety concerns that current theories—and the performance guarantees that these theories provide —cannot fully address. The gap is especially pronounced in systems that operate on real-world, high-dimensional data for which theoretical guarantees are most needed.

With this CAREER award, Ziv Goldfeld, Electrical and Computer Engineering, is developing smooth statistical distances, a novel framework for high-dimensional inference to support a scalable statistical analysis of modern machine-learning methods. Smooth distances are regularized discrepancy measures between probability distributions. They level out local irregularities in the measured distributions in a way that preserves inference capabilities but alleviates challenges associated with high dimensional data. This innovation will allow the coupling of empirical validation with principled performance-evaluation techniques and provable accuracy assurances.

This research has the potential to bridge central theoretical gaps and to provide increased reliability in machine-learning interfaces at scale. Ultimately, this project will promote the wide deployment of machine-learning technologies with invaluable societal benefits, from better health care to safer roads and improved crisis management. In conjunction, the educational component of this project will nurture the next generation of scientists in theoretical STEM disciplines while increasing the participation of women and girls, who remain underrepresented in these disciplines.

Cornell Researchers

Funding Received

$641 Thousand spanning 5 years