This focused section, edited by Wei Cai, Weinan E, Jan Hesthaven, Dongbin Xiu and Lexing Ying, presents several recent developments in improving deep neural networks (DNNs) for scientific computing and classification applications. Original research works on DNNs are included on the following topics: drop-activations as regularization, non-intrusive corrections for classifier, complexity of finding sparse representations for high dimensional functions, and discovery of phase field models for multiphase material studies.