DOES NEURON COVERAGE MATTER FOR DEEP REINFORCEMENT LEARNING? A PRELIMINARY STUDY

M.TRUJILLO, M. LINARES-VÁSQUEZ, C. ESCOBAR-VELÁSQUEZ, I. DUSPARIC, N. CARDOZO

WORKSHOP ON TESTING FOR DEEP LEARNING AND DEEP LEARNING FOR TESTING AT ICSE 2020

Abstract

Deep Learning (DL) is powerful family of algorithms used for a wide variety of problems and systems, including safety critical systems. As a consequence, analyzing, understanding, and testing DL models is attracting more practitioners and researchers with the purpose of implementing DL systems that are robust, reliable, efficient, and accurate. First software testing approaches for DL systems have focused on black-box testing, white-box testing, and test cases generation, in particular for deep neural networks (CNNs and RNNs). However, Deep Reinforcement Learning (DRL), which is a branch of DL extending reinforcement learning, is still out of the scope of research providing testing techniques for DL systems. In this paper, we present a first step towards testing of DRL systems. In particular, we investigate whether neuron coverage (a widely used metric for white-box testing of DNNs) could be used also for DRL systems, by analyzing coverage evolutionary patterns, and the correlation with RL rewards. CCS CONCEPTS • Theory of computation → Reinforcement learning; • Software and its engineering → Software testing and debugging.

CONTACT

Addrs. Cra. 1 E No. 19A - 40. Mario Laserna Building - School of Engineering, Bogotá, Colombia, Zip 111711, Ph. +(571) 332 4327, 332 4328, 332 4329

Universidad de los Andes | Monitored by Mineducación
Recognition as University: Decree 1297 of May 30th, 1964.
Recognition as legal entity: Resolution 28 of February 23, 1949 Minjusticia.

© Universidad de los Andes. All rights reserved.