Abstract
Recently, community has paid increasing attention on model scaling and contributed to developing a model family with a wide spectrum of scales. Current methods either simply resort to a one-shot NAS manner to construct a non-structural and non-scalable model family or rely on a manual yet fixed scaling strategy to scale an unnecessarily best base model. In this paper, we bridge both two components and propose ScaleNet to jointly search base model and scaling strategy so that the scaled large model can have more promising performance. Concretely, we design a super-supernet to embody models with different spectrum of sizes (e.g., FLOPs). Then, the scaling strategy can be learned interactively with the base model via a Markov chain-based evolution algorithm and generalized to develop even larger models. To obtain a decent super-supernet, we design a hierarchical sampling strategy to enhance its training sufficiency and alleviate the disturbance. Experimental results show our scaled networks enjoy significant performance superiority on various FLOPs, but with at least \(2.53\times \) reduction on search cost. Codes are available at https://github.com/luminolx/ScaleNet.
This work was supported in part by Beijing Natural Science Foundation Project No. Z200002 and in part by National Natural Science Foundation of China (NSFC) No. 61922015, U19B2036, 62225601.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Bishop, C.M.: Pattern Recognition and Machine Learning. Springer Science+Business Media LLC, Berlin (2006)
Bossard, L., Guillaumin, M., Van Gool, L.: Food-101 – mining discriminative components with random forests. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8694, pp. 446–461. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10599-4_29
Cai, H., Gan, C., Wang, T., Zhang, Z., Han, S.: Once-for-All: train one network and specialize it for efficient deployment. In: ICLR (2020)
Chang, D., Pang, K., Zheng, Y., Ma, Z., Song, Y.Z., Guo, J.: Your “flamingo” is my “bird”: fine-grained, or not. In: CVPR, pp. 11476–11485 (2021)
Cheng, Z., Su, X., Wang, X., You, S., Xu, C.: Sufficient vision transformer. In: KDD (2022)
Dai, X., et al.: FBNetV3: joint architecture-recipe search using predictor pretraining. In: CVPR, pp. 16276–16285 (2021)
Dollar, P., Singh, M., Girshick, R.: Fast and accurate model scaling. In: CVPR, pp. 924–932 (2021)
Du, R., Xie, J., Ma, Z., Chang, D., Song, Y.Z., Guo, J.: Progressive learning of category-consistent multi-granularity features for fine-grained visual classification. IEEE TPAMI (2021)
Guo, Z.: Single path one-shot neural architecture search with uniform sampling. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12361, pp. 544–560. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58517-4_32
Han, K., Wang, Y., Zhang, Q., Zhang, W., Xu, C., Zhang, T.: Model Rubik’s cube: twisting resolution, depth and width for TinyNets. In: NeurIPS (2020)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: CVPR (2016)
Howard, A., et al.: Searching for mobilenetV3. In: ICCV (2019)
Krause, J., Stark, M., Deng, J., Fei-Fei, L.: 3D object representations for fine-grained categorization. In: CVPRW, pp. 554–561 (2013)
Krizhevsky, A.: Learning multiple layers of features from tiny images. Technical Report, CIFAR (2009)
Li, S., et al.: Searching for fast model families on datacenter accelerators. In: CVPR, pp. 8085–8095 (2021)
Liu, C., et al.: Greedy network enlarging. ArXiv preprint, arXiv:2108.00177 (2021)
Lou, W., Xun, L., Sabet, A., Bi, J., Hare, J., Merrett, G.V.: Dynamic-OFA: Runtime DNN architecture switching for performance scaling on heterogeneous embedded platforms. In: CVPRW, pp. 3110–3118 (2021)
Maji, S., Rahtu, E., Kannala, J., Blaschko, M., Vedaldi, A.: Fine-grained visual classification of aircraft. ArXiv preprint, arXiv:1306.5151 (2013)
Radosavovic, I., Kosaraju, R.P., Girshick, R., He, K., Dollar, P.: Designing network design spaces. In: CVPR (2020)
Russakovsky, O., et al.: ImageNet large scale visual recognition challenge. IJCV 115(3), 211–252 (2015)
Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., Chen, L.C.: MobileNetV2: Inverted residuals and linear bottlenecks. In: CVPR (2018)
Su, X., et al.: Prioritized architecture sampling with Monto-Carlo tree search. In: CVPR, pp. 10968–10977 (2021)
Su, X., et al.: Locally free weight sharing for network width search. ArXiv preprint, arXiv:2102.05258 (2021)
Su, X., You, S., Wang, F., Qian, C., Zhang, C., Xu, C.: BCNet: searching for network width with bilaterally coupled network. In: CVPR, pp. 2175–2184 (2021)
Su, X., et al.: ViTAS: vision transformer architecture search. ArXiv preprint, arXiv:2106.13700 (2021)
Szegedy, C., Ioffe, S., Vanhoucke, V., Alemi, A.A.: Inception-v4, Inception-ResNet and the impact of residual connections on learning. In: AAAI (2017)
Tan, M., et al.: MnasNet: Platform-aware neural architecture search for mobile. In: CVPR (2019)
Tan, M., Le, Q.: EfficientNet: Rethinking model scaling for convolutional neural networks. In: ICML, pp. 6105–6114 (2019)
Tan, M., Le, Q.V.: EfficientNetV2: smaller models and faster training. ArXiv preprint, arXiv:2104.00298 (2021)
Tian, Y., Krishnan, D., Isola, P.: Contrastive multiview coding. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12356, pp. 776–794. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58621-8_45
Wan, A., et al.: FBNetV2: differentiable neural architecture search for spatial and channel dimensions. In: CVPR (2020)
Wang, C., Xu, C., Yao, X., Tao, D.: Evolutionary generative adversarial networks. IEEE Trans. Evol. Comput. 23(6), 921–934 (2019)
Wu, B., et al.: FBNet: hardware-aware efficient ConvNet design via differentiable neural architecture search. In: CVPR (2019)
Xie, J., Ma, Z., Chang, D., Zhang, G., Guo, J.: GPCA: a probabilistic framework for Gaussian process embedded channel attention. IEEE TPAMI (2021)
Xie, J., et al.: Advanced dropout: a model-free methodology for Bayesian dropout optimization. IEEE TPAMI (2021)
Xie, J., et al.: DS-UI: dual-supervised mixture of gaussian mixture models for uncertainty inference in image recognition. IEEE TIP 30, 9208–9219 (2021)
Xu, H., Su, X., Wang, Y., Cai, H., Cui, K., Chen, X.: Automatic bridge crack detection using a convolutional neural network. Appl. Sci. 9(14), 2867 (2019)
Yang, Z., Liu, D., Wang, C., Yang, J., Tao, D.: Modeling image composition for complex scene generation. ArXiv preprint, arXiv:2206.00923 (2022)
You, S., Huang, T., Yang, M., Wang, F., Qian, C., Zhang, C.: GreedyNAS: towards fast one-shot NAS with greedy supernet. In: CVPR (2020)
You, S., Xu, C., Xu, C., Tao, D.: Learning from multiple teacher networks. In: KDD, pp. 1285–1294 (2017)
Yu, J., et al.: BigNAS: scaling up neural architecture search with big single-stage models. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12352, pp. 702–717. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58571-6_41
Zagoruyko, S., Komodakis, N.: Wide residual networks. ArXiv preprint, arXiv:1605.07146 (2016)
Zhai, X., Kolesnikov, A., Houlsby, N., Beyer, L.: Scaling vision transformers. ArXiv preprint, arXiv:2106.04560 (2021)
Zoph, B., Vasudevan, V., Shlens, J., Le, Q.V.: Learning transferable architectures for scalable image recognition. In: CVPR (2018)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Xie, J., Su, X., You, S., Ma, Z., Wang, F., Qian, C. (2022). ScaleNet: Searching for the Model to Scale. In: Avidan, S., Brostow, G., Cissé, M., Farinella, G.M., Hassner, T. (eds) Computer Vision – ECCV 2022. ECCV 2022. Lecture Notes in Computer Science, vol 13681. Springer, Cham. https://doi.org/10.1007/978-3-031-19803-8_7
Download citation
DOI: https://doi.org/10.1007/978-3-031-19803-8_7
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-19802-1
Online ISBN: 978-3-031-19803-8
eBook Packages: Computer ScienceComputer Science (R0)