Multi-task Neural Shared Structure Search: A Study Based on Text Mining

Abstract

Multi-task techniques are effective for handling the problem of small size of the datasets. They can leverage additional rich information from other tasks for improving the performance of the target task. One of the problems in the multi-task based methods is which resources are proper to be utilized as the auxiliary tasks and how to select the shared structures with an effective search mechanism. We propose a novel neural-based multi-task Shared Structure Encoding (SSE) to define the exploration space by which we can easily formulate the multi-task architecture search. For the search approaches, because these existing Network Architecture Search (NAS) techniques are not specially designed for the multi-task scenario, we propose two original search approaches, i.e., m-Sparse Search approach by Shared Structure encoding for neural-based Multi-Task models (m-S4MT) and Task-wise Greedy Generation Search approach by Shared Structure encoding for neural-based Multi-Task models (TGG-S3MT). The experiments based on the real text datasets with multiple text mining tasks show that SSE is effective for formulating the multi-task architecture search. Moreover, both m-S4MT and TGG-S3MT have better performance on the target aspects than the single-task method, multi-label method, naïve multi-task methods and the variant of the NAS approach from the existing works. Especially, 1-S4MT with a sparse assumption on the auxiliary tasks has good performance with very low computation cost.