Classification of Internal Deterioration Depth in Concrete Using Decision-Level Fusion of Multimodal Nondestructive Testing Data

Hyeonsu Kim; Jungu Kang; Dongjun Kim; Jaehyun Jeong; Homin Song

doi:10.7734/COSEIK.2026.39.4.273

All Issue

2026 Vol.39, Issue 4 Preview Page

Classification of Internal Deterioration Depth in Concrete Using Decision-Level Fusion of Multimodal Nondestructive Testing Data 멀티모달 비파괴검사 데이터의 의사결정 수준 융합을 통한 콘크리트 내부 열화 깊이 분류

31 August 2026. pp. 273-281

PDF XML

Abstract

This paper proposes a multimodal deep learning framework based on decision-level fusion for classifying the depth of internal deteriorations in bridge decks using ground penetrating radar (GPR), phased array ultrasonic testing (PAUT), and impact-echo (IE) data. Three bridge deck specimens containing simulated deterioration were fabricated, and a multimodal dataset comprising 2,045 matched samples was constructed based on the deterioration-depth classes. Each single-modality classification model employed a ResNet-50 backbone pretrained on ImageNet-22k, and decision-level fusion was implemented using hard voting, soft voting, and weighted voting methods. The experimental results demonstrated that the multimodal fusion models outperformed the single-modality models. Among the fusion approaches, the soft voting method exhibited the best performance, achieving an accuracy of 90.46% and macro F1-score of 0.8907. In particular, the proposed framework improved the classification accuracy of deep deterioration classes that exhibited relatively low performance in the single-modality approaches. These results suggest that decision-level fusion of multimodal nondestructive testing data may contribute to improving deterioration-depth classification for bridge deck condition assessment.

Keywords

ground penetrating radar (GPR)

phased array ultrasonic testing (PAUT)

impact echo (IE)

multimodal NDT

decision-level fusion

본 연구에서는 지표투과레이더(Ground Penetrating Radar, GPR), 위상배열 초음파 탐상(Phased Array Ultrasonic Testing, PAUT), 충격-반향(Impact-Echo, IE) 데이터를 활용하여 교량 바닥판 내부 열화 깊이를 분류하기 위한 의사결정 수준 융합 기반 멀티모달 딥러닝 프레임워크를 제안하였다. 이를 위해 내부 열화를 모사한 3개의 교량 바닥판 시험체를 제작하였으며, 열화 깊이 조건에 따라 2,045개의 대응 샘플로 구성된 멀티모달 데이터세트를 구축하였다. 각 단일 모달 분류 모델은 ImageNet-22k로 사전학습된 ResNet-50 백본 네트워크를 기반으로 구성하였으며, 의사결정 수준 융합은 hard voting, soft voting 및 weighted voting 기법을 적용하여 수행하였다. 실험 결과, 멀티모달 융합 모델은 단일 모달 모델보다 우수한 분류 성능을 나타냈다. 융합 기법 중에서는 soft voting 방식이 가장 우수한 성능을 보였으며, 정확도 90.46%와 macro F1-score 0.8907을 달성하였다. 특히 제안된 프레임워크는 단일 모달 접근에서 상대적으로 낮은 성능을 보였던 심부 열화 클래스에 대한 분류 정확도를 향상시키는 것으로 나타났다. 이러한 결과는 다종 비파괴검사 데이터의 의사결정 수준 융합이 교량 바닥판 상태평가를 위한 열화 깊이 분류 성능 향상에 기여할 수 있음을 시사한다.

키워드

지표투과레이더

위상배열 초음파 탐상

충격-반향

멀티모달 비파괴검사

의사결정 수준 융합

References

Allen, J.B., Rabiner, L.R. (1977) A Unified approach to Short-time Fourier Analysis and Synthesis, Proc. IEEE, 65(11), pp.1558~1564.

10.1109/PROC.1977.10770

Construction & Economy Research Institute of Korea (CERIK) (2023) Current Status and Outlook of Aging Road Bridges, Seoul, Korea.

Coleman, Z.W., Schindler, A.K. (2025) Investigation of Ground-Penetrating Radar, Impact Echo, and Infrared Thermography Methods to Detect Defects in Concrete Bridge Decks, Transp. Res. Rec., 2679(1), pp.169~182.

10.1177/03611981221101027

Dikmen, M. (2022) Investigating Transfer Learning Performances of Deep Learning Models for Classification of GPR B-Scan Images, Traitement du Signal, 39(5), pp. 1761~1766.

10.18280/ts.390534

Dinh, K., Gucunski, N., Zayed, T. (2019) Automated Visualization of Concrete Bridge Deck Condition from GPR Data, Ndt & E Int., 102, pp.120~128.

10.1016/j.ndteint.2018.11.015

Gucunski, N., Pailes, B., Kim, J., Azari, H., Dinh, K. (2017) Bridge Deck Condition Assessment from Multiple NDE Technologies, J. Infrastruct. Syst., 23(1), B4016005.

10.1061/(ASCE)IS.1943-555X.0000321

He, K., Zhang, X., Ren, S., Sun, J. (2016) Deep Residual Learning for Image Recognition, Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp.770~778.

10.1109/CVPR.2016.90

Holmes, C., Drinkwater, B.W., Wilcox, P.D. (2005) Post-Processing of the Full Matrix of Ultrasonic Transmit-Receive Array Data for Non-Destructive Evaluation, Ndt & E Int., 38(8), pp.701~711.

10.1016/j.ndteint.2005.04.002

Jeong, Y., Lee, I., Min, G.-H., Kim, W.-S. (2024) Deterioration Mechanism and Repair Methods for Concrete Decks in Bridges, J. Korea Concr. Inst., 36(4), pp.347~355.

10.4334/JKCI.2024.36.4.347

Jol, H.M. (2008) Ground Penetrating Radar: Theory and Applications, Elsevier, p.544.

Jung, H.K., Rhee, I., Kim, J.-M. (2020) Ultrasonic Wave Propagation Analysis for Damage Detection in Heterogeneous Concrete Materials, J. Comput. Struct. Eng. Inst. Korea, 33(4), pp.225~235.

10.7734/COSEIK.2020.33.4.225

Kim, H., Kang, J., Jeong, J., Han, G.-P., Song, H. (2026) Application of Multi-type Nondestructive Testing Techniques for Evaluating the Internal Deterioration of Bridge Decks, J. Korea Concr. Inst., 38(1), pp.61~70.

10.4334/JKCI.2026.38.1.061

Kittler, J., Hatef, M., Duin, R.P.W., Matas, J. (1998) On Combining Classifiers, IEEE Trans. Pattern Analy. & Mach. Intell., 20(3), pp.226~239.

10.1109/34.667881

Korea Authority of Land and Infrastructure Safety (KALIS) (2024) Detailed Guidelines for Safety Inspection and Maintenance of Facilities (Bridge Section), Jinju, Korea.

Korea Expressway Corporation (KEC) (2011) Highway Design Practice Manual 2010, Gyeonggi-do, Korea: Korea Expressway Corporation.

Korea Expressway Corporation (KEC) (2023) Manual for Repairing and Rehabilitating Deteriorated Bridge Concrete Decks, Gimcheon, Korea: Korea Expressway Corporation.

Wu, Y., Fan, X., Zhang, J. (2024) Ensemble Learning Model for Concrete Delamination Depth Detection using Impact Echo, Ndt & E Int., 145, 103119.

10.1016/j.ndteint.2024.103119

Yehia, S., Abudayyeh, O., Nabulsi, S., Abdelqader, I. (2007) Detection of Common Defects in Concrete Bridge Decks using Nondestructive Evaluation Techniques, J. Bridge Eng., 12(2), pp.215~225.

10.1061/(ASCE)1084-0702(2007)12:2(215)

Zhou, Z.-H. (2012) Ensemble Methods: Foundations and Algorithms, CRC Press, p.240.

10.1201/b12207

Information

Publisher :Computational Structural Engineering Institute of Korea
Publisher(Ko) :한국전산구조공학회
Journal Title :Journal of the Computational Structural Engineering Institute of Korea
Journal Title(Ko) :한국전산구조공학회 논문집
Volume : 39
No :4
Pages :273-281
Received Date : 2026-06-02
Revised Date : 2026-06-03
Accepted Date : 2026-06-04
DOI :https://doi.org/10.7734/COSEIK.2026.39.4.273

Journal of the Computational Structural Engineering Institute of Korea ISSN:1229-3059(Print) 2287-2302(Online) 한국전산구조공학회 논문집

All Issue