Spectral algorithm for shared low-rank matrix regressions

Yotam Gigi, Sella Nevo, Gal Elidan, Avinatan Hassidim, Yossi Matias, Ami Wiesel

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

1 Scopus citations

Abstract

We consider multiple matrix regression tasks that share common weights in order to reduce sample complexity. For this purpose, we introduce the common mechanism regression model which assumes a shared right low-rank component across all tasks, but allows an individual per-task left low-rank component. We provide a closed form spectral algorithm for recovering the common component and derive a bound on its error as a function of the number of related tasks and the number of samples available for each of them. Both the algorithm and its analysis are natural extensions of known results in the context of phase retrieval and low rank reconstruction. We demonstrate the efficacy of our approach for the challenging task of remote river discharge estimation across multiple river sites, where data for each task is naturally scarce. In this scenario sharing a low-rank component between the tasks translates to a shared spectral reflection of the water, which is a true underlying physical model. We also show the benefit of the approach in the setting of image classification where the common component can be interpreted as the shared convolution filters.

Original languageAmerican English
Title of host publication2020 IEEE 11th Sensor Array and Multichannel Signal Processing Workshop, SAM 2020
PublisherIEEE Computer Society
ISBN (Electronic)9781728119465
DOIs
StatePublished - Jun 2020
Event11th IEEE Sensor Array and Multichannel Signal Processing Workshop, SAM 2020 - Hangzhou, China
Duration: 8 Jun 202011 Jun 2020

Publication series

NameProceedings of the IEEE Sensor Array and Multichannel Signal Processing Workshop
Volume2020-June
ISSN (Electronic)2151-870X

Conference

Conference11th IEEE Sensor Array and Multichannel Signal Processing Workshop, SAM 2020
Country/TerritoryChina
CityHangzhou
Period8/06/2011/06/20

Bibliographical note

Publisher Copyright:
© 2020 IEEE.

Keywords

  • Low rank optimization
  • Multitask Learning
  • Phase Retrieval

Fingerprint

Dive into the research topics of 'Spectral algorithm for shared low-rank matrix regressions'. Together they form a unique fingerprint.

Cite this