We consider multiple matrix regression tasks that share common weights in order to reduce sample complexity. For this purpose, we introduce the common mechanism regression model which assumes a shared right low-rank component across all tasks, but allows an individual per-task left low-rank component. We provide a closed form spectral algorithm for recovering the common component and derive a bound on its error as a function of the number of related tasks and the number of samples available for each of them. Both the algorithm and its analysis are natural extensions of known results in the context of phase retrieval and low rank reconstruction. We demonstrate the efficacy of our approach for the challenging task of remote river discharge estimation across multiple river sites, where data for each task is naturally scarce. In this scenario sharing a low-rank component between the tasks translates to a shared spectral reflection of the water, which is a true underlying physical model. We also show the benefit of the approach in the setting of image classification where the common component can be interpreted as the shared convolution filters.
|Original language||American English|
|Title of host publication||2020 IEEE 11th Sensor Array and Multichannel Signal Processing Workshop, SAM 2020|
|Publisher||IEEE Computer Society|
|State||Published - Jun 2020|
|Event||11th IEEE Sensor Array and Multichannel Signal Processing Workshop, SAM 2020 - Hangzhou, China|
Duration: 8 Jun 2020 → 11 Jun 2020
|Name||Proceedings of the IEEE Sensor Array and Multichannel Signal Processing Workshop|
|Conference||11th IEEE Sensor Array and Multichannel Signal Processing Workshop, SAM 2020|
|Period||8/06/20 → 11/06/20|
Bibliographical noteFunding Information:
This work was supported in part by the Israel Science Foundation (ISF) under Grant 1339/15.
© 2020 IEEE.
- Low rank optimization
- Multitask Learning
- Phase Retrieval