Augmenting Deep Neural Networks with Scenario-Based Guard Rules

Guy Katz*

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

5 Scopus citations


Deep neural networks (DNNs) are becoming widespread, and can often outperform manually-created systems. However, these networks are typically opaque to humans, and may demonstrate undesirable behavior in corner cases that were not encountered previously. In order to mitigate this risk, one approach calls for augmenting DNNs with hand-crafted override rules. These override rules serve to prevent the DNN from making certain decisions, when certain criteria are met. Here, we build on this approach and propose to bring together DNNs and the well-studied scenario-based modeling paradigm, by encoding override rules as simple and intuitive scenarios. We demonstrate that the scenario-based paradigm can render override rules more comprehensible to humans, while keeping them sufficiently powerful and expressive to increase the overall safety of the model. We propose a method for applying scenario-based modeling to this new setting, and apply it to multiple DNN models. (This paper substantially extends the paper titled “Guarded Deep Learning using Scenario-Based Modeling”, published in Modelsward 2020 [47]. Most notably, it includes an additional case study, extends the approach to recurrent neural networks, and discusses various aspects of the proposed paradigm more thoroughly).

Original languageAmerican English
Title of host publicationModel-Driven Engineering and Software Development - 8th International Conference, MODELSWARD 2020, Revised Selected Papers
EditorsSlimane Hammoudi, Luís Ferreira Pires, Bran Selić
PublisherSpringer Science and Business Media Deutschland GmbH
Number of pages26
ISBN (Print)9783030674441
StatePublished - 2021
Event8th International Conference on Model-Driven Engineering and Software Development, MODELSWARD 2020 - Valletta, Malta
Duration: 25 Feb 202027 Feb 2020

Publication series

NameCommunications in Computer and Information Science
ISSN (Print)1865-0929
ISSN (Electronic)1865-0937


Conference8th International Conference on Model-Driven Engineering and Software Development, MODELSWARD 2020

Bibliographical note

Publisher Copyright:
© 2021, Springer Nature Switzerland AG.


  • Behavioral programming
  • Deep neural networks
  • Machine learning
  • Scenario-based modeling


Dive into the research topics of 'Augmenting Deep Neural Networks with Scenario-Based Guard Rules'. Together they form a unique fingerprint.

Cite this