The application of deep reinforcement learning (DRL) to computer and networked systems has recently gained significant popularity. However, the obscurity of decisions by DRL policies renders it hard to ascertain that learning-augmented systems are safe to deploy, posing a significant obstacle to their real-world adoption. We observe that specific characteristics of recent applications of DRL to systems contexts give rise to an exciting opportunity: applying formal verification to establish that a given system provably satisfies designer/user-specified requirements, or to expose concrete counter-examples. We present whiRL, a platform for verifying DRL policies for systems, which combines recent advances in the verification of deep neural networks with scalable model checking techniques. To exemplify its usefulness, we employ whiRL to verify natural equirements from recently introduced learning-augmented systems for three real-world environments: Internet congestion control, adaptive video streaming, and job scheduling in compute clusters. Our evaluation shows that whiRL is capable of guaranteeing that natural requirements from these systems are satisfied, and of exposing specific scenarios in which other basic requirements are not.
|Original language||American English|
|Title of host publication||SIGCOMM 2021 - Proceedings of the ACM SIGCOMM 2021 Conference|
|Publisher||Association for Computing Machinery, Inc|
|Number of pages||14|
|State||Published - 9 Aug 2021|
|Event||2021 Annual Conference of the ACM Special Interest Group on Data Communication on the Applications, SIGCOMM 2021 - Virtual, Online, United States|
Duration: 23 Aug 2021 → 27 Aug 2021
|Name||SIGCOMM 2021 - Proceedings of the ACM SIGCOMM 2021 Conference|
|Conference||2021 Annual Conference of the ACM Special Interest Group on Data Communication on the Applications, SIGCOMM 2021|
|Period||23/08/21 → 27/08/21|
Bibliographical noteFunding Information:
We thank our shepherd, Prof. Mohammad Alizadeh, and the anonymous SIGCOMM reviewers, for their valuable comments and suggestions, which have significantly improved this work. We also thank Prof. Clark Barrett for his insightful comments on an earlier manuscript of this work. This research was partially supported by grants from the Binational Science Foundation (BSF grants 2017662 and 2019798), the Israel Science Foundation (ISF grant 683/18), and Facebook.
© 2021 ACM.
- adaptive bitrate algorithms
- congestion control
- deep learning
- deep reinforcement learning
- formal verification
- networked systems
- neural networks
- resource scheduling