Guarded deep learning using scenario-based modeling

Guy Katz*

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

5 Scopus citations

Abstract

Deep neural networks (DNNs) are becoming prevalent, often outperforming manually-created systems. Unfortunately, DNN models are opaque to humans, and may behave in unexpected ways when deployed. One approach for allowing safer deployment of DNN models calls for augmenting them with hand-crafted override rules, which serve to override decisions made by the DNN model when certain criteria are met. Here, we propose to bring together DNNs and the well-studied scenario-based modeling paradigm, by expressing these override rules as simple and intuitive scenarios. This approach can lead to override rules that are comprehensible to humans, but are also sufficiently expressive and powerful to increase the overall safety of the model. We describe how to extend and apply scenario-based modeling to this new setting, and demonstrate our proposed technique on multiple DNN models.

Original languageAmerican English
Title of host publicationMODELSWARD 2020 - Proceedings of the 8th International Conference on Model-Driven Engineering and Software Development
EditorsSlimane Hammoudi, Luis Ferreira Pires, Bran Selic
PublisherSciTePress
Pages126-136
Number of pages11
ISBN (Electronic)9789897584008
StatePublished - 2020
Event8th International Conference on Model-Driven Engineering and Software Development, MODELSWARD 2020 - Valletta, Malta
Duration: 25 Feb 202027 Feb 2020

Publication series

NameMODELSWARD 2020 - Proceedings of the 8th International Conference on Model-Driven Engineering and Software Development

Conference

Conference8th International Conference on Model-Driven Engineering and Software Development, MODELSWARD 2020
Country/TerritoryMalta
CityValletta
Period25/02/2027/02/20

Bibliographical note

Publisher Copyright:
Copyright © 2020 by SCITEPRESS - Science and Technology Publications, Lda. All rights reserved.

Keywords

  • Behavioral Programming
  • Deep Neural Networks
  • Machine Learning
  • Scenario-based Modeling

Fingerprint

Dive into the research topics of 'Guarded deep learning using scenario-based modeling'. Together they form a unique fingerprint.

Cite this