Learning to Estimate Without Bias

Tzvi Diskin*, Yonina C. Eldar, Ami Wiesel

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

1 Scopus citations

Abstract

The Gauss-Markov theorem states that the weighted least squares estimator is a linear minimum variance unbiased estimation (MVUE) in linear models. In this paper, we take a first step towards extending this result to non-linear settings via deep learning with bias constraints. The classical approach to designing non-linear MVUEs is through maximum likelihood estimation (MLE) which often involves computationally challenging optimizations. On the other hand, deep learning methods allow for non-linear estimators with fixed computational complexity. Learning based estimators perform optimally on average with respect to their training set but may suffer from significant bias in other parameters. To avoid this, we propose to add a simple bias constraint to the loss function, resulting in an estimator we refer to as Bias Constrained Estimator (BCE). We prove that this yields asymptotic MVUEs that behave similarly to the classical MLEs and asymptotically attain the Cramer Rao bound. We demonstrate the advantages of our approach in the context of signal to noise ratio estimation as well as covariance estimation. A second motivation to BCE is in applications where multiple estimates of the same unknown are averaged for improved performance. Examples include distributed sensor networks and data augmentation in test-time. In such applications, we show that BCE leads to asymptotically consistent estimators.

Original languageAmerican English
Pages (from-to)2162-2171
Number of pages10
JournalIEEE Transactions on Signal Processing
Volume71
DOIs
StatePublished - 2023

Bibliographical note

Publisher Copyright:
© 1991-2012 IEEE.

Keywords

  • Cramer Rao bounds
  • Machine learninig
  • estimation

Fingerprint

Dive into the research topics of 'Learning to Estimate Without Bias'. Together they form a unique fingerprint.

Cite this