DA: towards distribution adaptive test-time adaptation in dynamic wild world

doi:10.1007/s44275-025-00035-2

Moore and More ›› 2026, Vol. 2 ›› Issue (1): 25-36.DOI: 10.1007/s44275-025-00035-2

• ORIGINAL ARTICLE • Previous Articles Next Articles

DA: towards distribution adaptive test-time adaptation in dynamic wild world

Zhendong Liu¹, Jiarong Liao¹, Chuyang Ye¹, Dongyan Wei¹, Tingting Zhang², Xianghua Fu¹^,^*(), Jingyan Jiang¹^,^*()

¹ The College of Bigdata and Internet, Shenzhen Technology University , Shenzhen 518118, Guangdong, China
² Department of Electrical and Computer Engineering, McGill University , Montreal H3A 0E9, Canada

Received:2024-12-19 Revised:2025-06-09 Accepted:2025-06-25 Published:2026-03-20 Online:2026-05-12
Contact: *Xianghua Fu (fuxianghua@sztu.edu.cn)
*Jingyan Jiang (jiangjingyan@sztu.edu.cn)
About author:Zhendong Liu is a candidate for a bachelor of engineering (B.Eng.) degree at Shenzhen Technology University (expected to graduate in July 2026). His major research interests include improving model deployment and enhancing robustness of deep learning models across diverse data distributions. He focuses on developing efficient algorithms to improve both accuracy and computational efficiency of machine learning models in real-world applications.
Jiarong Liao is a fourth-year undergraduate majoring in Data Science and Big Data Technology at Shenzhen Technology University. His research focuses on optimizing the underlying architecture of software systems. Specifically, he works on developing innovative methods to enhance the efficiency, scalability, and reliability of software frameworks, ensuring robust performance in handling large-scale data processing and complex computational tasks. By improving the foundational structure, his work aims to streamline software development and deployment processes while maximizing system resilience and adaptability in dynamic environments.
Chuyang Ye is currently a senior undergraduate student in the Data Science and Big Data Technology program at Shenzhen Technology University. His research centers on enhancing model inference and generalization, with a particular emphasis on creating optimized algorithms that expedite the deployment of deep learning models while ensuring their robustness across varied real-world data scenarios.
Dongyan Wei will obtain a B.E. degree in Data Science and Big Data Technology from Shenzhen Technology University, Shenzhen, China, in 2025. His academic journey has been marked by a deep interest in model inference optimization, a pivotal aspect of ensuring that machine learning models are not only accurate but also efficient in their application.
Tingting Zhang received her Ph.D. degree from the University of Alberta, Alberta, Canada, in 2024. She is currently a postdoctoral fellow at McGill University, Quebec, Canada. Her research interests include new computing architectures, approximate computing, Ising computing, combinatorial optimization, and nanoelectronic circuits and systems. She was a recipient of the Best Paper Award Candidate at the Design, Automation and Test in Europe Conference (DATE) 2022. She served as the session chair for the IEEE International Conference on Nanotechnology (IEEENANO) 2024 and a Technical Program Committee Member for the International Conference on Computer-Aided Design (ICCAD) 2025.
Xianghua Fu received a Ph.D. degree in Computer Science and Technology from Xi’an Jiaotong University, Shanxi, China. He is currently a professor of the College of Big Data and Internet, and vice dean at the College of Big Data and Internet. His research interests include natural language processing, information retrieval, machine learning, and data mining. He has presided over one item of the National Natural Science Foundation, several projects of Shenzhen Basic Research and the Natural Science Foundation of Guangdong Province, and participated in many projects, such as the National Natural Science Foundation Project and the National Support Program. He has published more than 80 articles in many important journals and international conferences at home and abroad. He has also completed two provincial-level quality courses and received the first prize for provincial teaching achievements.
Jingyan Jiang received her Ph.D. degree in 2020 from Jilin University, China, specializing in Computer Science. After completing her doctoral studies, she served as a postdoctoral research fellow at the Tsinghua University Shenzhen Graduate School from 2020 to 2022.
She is currently an assistant professor at the Shenzhen Technology University, Shenzhen, China. Her research interests mainly lie in the areas of edge intelligence and federated learning.

Abstract

Abstract:

Test-time adaptation (TTA) has demonstrated effectiveness in addressing distribution shifts between training and testing data by adjusting a given model on test samples. However, when faced with testing data that exhibit dynamic patterns, wherein a single test sample batch is drawn from various distribution, the traditional TTA methods, which typically follow a fixed pattern of estimating batch normalization (BN) statistics and then performing back-propagation, tend to experience performance degradation. The key reasons we observed are as follows: (i) different scenarios require different normalization approaches (such as instance normalization (IN) is optimal in mixture domains, but not for static domains) and (ii) back-propagation could potentially degrade the model and waste time. Based on these observations, in this paper, we introduce a novel one-size-fits-all approach, named distribution adaptive test-time adaptation (DA). DA is designed to adaptively select the appropriate batch normalization method and back-propagation approach. It utilizes an IN–based projection method to differentiate between various scenarios. Our method allows the model to achieve a more robust representation, enabling it to adapt effectively to both static and dynamic data patterns. Furthermore, our method avoids unnecessary or potentially harmful backward passes, paving the way for further enhancements. The results show that our method demonstrates robustness while maintaining good performance of the model. It can effectively respond to data stream patterns, and the selective back-propagation approach is more lightweight.

Key words: Test-time adaptation, Quality of experience, Test-time normalization, Domain generalization, Domain adaptation

Zhendong Liu, Jiarong Liao, Chuyang Ye, Dongyan Wei, Tingting Zhang, Xianghua Fu, Jingyan Jiang. DA: towards distribution adaptive test-time adaptation in dynamic wild world[J]. Moore and More, 2026, 2(1): 25-36.

References 21

[1]	Quionero-Candela J , Sugiyama M , Schwaighofer A , Lawrence ND (2009) Dataset shift in machine learning. https://doi.org/10.7551/mitpress/9780262170055.001.0001
[2]	Hendrycks D , Dietterich T (2019) Benchmarking neural network robustness to common corruptions and perturbations. https://doi.org/10.48550/arXiv.1903.12261
[3]	Recht B , Roelofs R , Schmidt L , Shankar V (2019) Do ImageNet classifiers generalize to ImageNet? In: Proceedings of the 36th International Conference on Machine Learning, PMLR. IEEE, New York, pp 5389-5400. https://doi.org/10.48550/arXiv.1902.10811
[4]	Wang D , Shelhamer E , Liu S , Olshausen B , Darrell T (2021) Tent: fully test-time adaptation by entropy minimization. https://doi.org/10.48550/arXiv.2006.10726
[5]	Wang Q , Fink O , Van Gool L , Dai D (2022) Continual test-time domain adaptation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, pp 7191-7201. https://doi.org/10.48550/arXiv.2203.13591
[6]	Gong T , Jeong J , Kim T , Kim Y , Shin J , Lee SJ (2023) NOTE: robust continual test-time adaptation against temporal correlation. In: Proceedings of the 36th International Conference on Neural Information Processing Systems (NIPS '22). Curran Associates Inc., Red Hook, NY, pp 27253-27266. https://doi.org/10.48550/arXiv.2208.05117
[7]	Yuan L , Xie B , Li S (2023) Robust test-time adaptation in dynamic scenarios. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, pp 15922-15932. https://doi.org/10.1109/CVPR52729.2023.01528
[8]	Ioffe S , Szegedy C (2015) Batch normalization: accelerating deep network training by reducing internal covariate shift. In: Proceedings of the 32nd International Conference on International Conference on Machine Learning-Volume 37 (ICML '15). JMLR.org, pp 448-456. https://doi.org/10.48550/arXiv.1502.03167
[9]	Wang W , Zhong Z , Wang W , Chen X , Ling C , Wang B et al (2023) Dynamically instance-guided adaptation: a backward-free approach for test-time domain adaptive semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, pp 24090-24099. https://doi.org/10.1109/CVPR52729.2023.02307
[10]	Iwasawa Y , Matsuo Y (2021) Test-time classifier adjustment module for model-agnostic domain generalization. In: Proceedings of the 35th International Conference on Neural Information Processing Systems (NIPS '21). Curran Associates Inc., Red Hook, NY, pp 2427-2440
[11]	You F , Li J , Zhao Z (2021) Test-time batch statistics calibration for covariate shift. https://doi.org/10.48550/arXiv.2110.04065
[12]	Yang H , Chen C , Jiang M , Liu Q , Cao J , Heng PA et al (2022) DLTTA: dynamic learning rate for test-time adaptation on cross-domain medical images. IEEE Trans Med Imaging 41(12): 3575-3586. https://doi.org/10.1109/TMI.2022.3191535
[13]	Niu S , Wu J , Zhang Y , Chen Y , Zheng S , Zhao P et al (2023) Efficient test-time model adaptation without forgetting. In: Proceedings of the 39th International Conference on Machine Learning. pp 16888-16905
[14]	Niu S , Wu J , Zhang Y , Wen Z , Chen Y , Zhao P et al (2023) Towards stable test-time adaptation in dynamic wild world. https://doi.org/10.48550/arXiv.2302.12400
[15]	Nado Z , Padhy S , Sculley D , D’Amour A , Lakshminarayanan B , Snoek J (2021) Evaluating prediction-time batch normalization for robustness under covariate shift. https://doi.org/10.48550/arXiv.2203.13591
[16]	Saikia T , Schmid C , Brox T (2021) Improving robustness against common corruptions with frequency biased models. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). IEEE, Montreal, pp 10191-10200. https://doi.org/10.1109/ICCV48922.2021.01005
[17]	Hendrycks D , Mu N , Cubuk ED , Zoph B , Gilmer J , Lakshminarayanan B (2020) AugMix: a simple data processing method to improve robustness and uncertainty. https://doi.org/10.48550/arXiv.1912.02781
[18]	Hu X , Uzunbas G , Chen S , Wang R , Shah A , Nevatia R et al (2021) MixNorm: test-time adaptation through online normalization estimation. https://doi.org/10.48550/arXiv.2110.11478
[19]	Lim H , Kim B , Choo J , Choi S (2023) TTN: a domain-shift aware batch normalization in test-time adaptation. https://doi.org/10.48550/arXiv.2302.05155
[20]	Zhao H , Liu Y , Alahi A , Lin T (2023) On pitfalls of test-time adaptation. In: Proceedings of the 40th International Conference on Machine Learning (ICML '23), Vol. 202. JMLR.org, pp 42058-42080. https://doi.org/10.48550/arXiv.2306.03536
[21]	He K , Zhang X , Ren S , Sun J (2015) Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE, pp 770-778. https://doi.org/10.48550/arXiv.1512.03385

DA: towards distribution adaptive test-time adaptation in dynamic wild world

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

References 21

Related Articles 0

Recommended Articles

Metrics