Joint Bi-layer Optimization for Single-Image Rain Streak Removal

Lei Zhu, Chi Wing Fu, Dani Lischinski, Pheng Ann Heng

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

221 Scopus citations


We present a novel method for removing rain streaks from a single input image by decomposing it into a rain-free background layer B and a rain-streak layer R. A joint optimization process is used that alternates between removing rain-streak details from B and removing non-streak details from R. The process is assisted by three novel image priors. Observing that rain streaks typically span a narrow range of directions, we first analyze the local gradient statistics in the rain image to identify image regions that are dominated by rain streaks. From these regions, we estimate the dominant rain streak direction and extract a collection of rain-dominated patches. Next, we define two priors on the background layer B, one based on a centralized sparse representation and another based on the estimated rain direction. A third prior is defined on the rain-streak layer R, based on similarity of patches to the extracted rain patches. Both visual and quantitative comparisons demonstrate that our method outperforms the state-of-the-art.

Original languageAmerican English
Title of host publicationProceedings - 2017 IEEE International Conference on Computer Vision, ICCV 2017
PublisherInstitute of Electrical and Electronics Engineers Inc.
Number of pages9
ISBN (Electronic)9781538610329
StatePublished - 22 Dec 2017
Event16th IEEE International Conference on Computer Vision, ICCV 2017 - Venice, Italy
Duration: 22 Oct 201729 Oct 2017

Publication series

NameProceedings of the IEEE International Conference on Computer Vision
ISSN (Print)1550-5499


Conference16th IEEE International Conference on Computer Vision, ICCV 2017

Bibliographical note

Funding Information:
We thank reviewers for the valuable comments. This work was supported in part by the 973 Program (2015CB351706), the Shenzhen Science and Technology Program (JCYJ20170413162617606 and JCYJ20170413162256793), the National Natural Science Foundation of China (61233012), the Israel Science Foundation (ISF grant No. 2366/16), and the ISF-NSFC joint research program (grant No. 2217/15).

Publisher Copyright:
© 2017 IEEE.


Dive into the research topics of 'Joint Bi-layer Optimization for Single-Image Rain Streak Removal'. Together they form a unique fingerprint.

Cite this