PotholeSAM: Pothole Management Using Segment Anything Model 2

Wonjun Choi; Seunguk Na; Yonghun Lee; Cheekyeong Kim; Seokjae Heo

doi:10.7734/COSEIK.2025.38.3.139

All Issue

2025 Vol.38, Issue 3 Next Page

Research Paper

PotholeSAM: Pothole Management Using Segment Anything Model 2 PotholeSAM: Segment Anything Model 2를 이용한 포트홀 관리

30 June 2025. pp. 139-146

PDF XML

Abstract

Potholes are closely related to road safety and can cause vehicle damage and accidents. Thus, prompt detection and repair are required. However, existing pothole maintenance methods are inefficient owing to labor shortages. Attempts have been made to streamline management through the use of artificial intelligence for pothole recognition, but these approaches have limitations because of difficulties in data collection and insufficient generalization performance. To address these problems, this paper proposes the PotholeSAM framework, which combines the object detection model YOLOv8, the object segmentation model SAM2, and a large language model (LLM). This framework consists of three stages: first, YOLOv8 detects the locations of potholes within images; second, SAM2 precisely segments the detected pothole areas; third, the LLM analyzes the brightness information of the segmented regions to estimate the depths of the potholes. The proposed method automatically analyzes information on the location, area, and depth of potholes, efficiently acquiring the data necessary for road management. This can alleviate labor shortages in local governments and enable proactive responses by detecting dangerous potholes early for drivers.

Keywords

pothole detection

PotholeSAM

YOLOv8

SAM2

large language model

포트홀은 도로 안전과 밀접한 관련이 있으며 차량 손상 및 사고를 유발할 수 있어 신속한 발견과 보수가 필요하다. 그러나 기존의 포트홀 유지보수 방법은 인력 부족으로 인해 효율적이지 못하다. 관리를 간소화하기 위해 인공지능을 활용한 포트홀 객체 인식이 시도되었지만, 데이터 수집의 어려움과 부족한 일반화 성능으로 이러한 접근법에도 한계가 있다. 이를 해결하기 위해, 본 연구에서는 객체 검출 모델 YOLOv8, 객체 분할 모델 SAM2, 그리고 대형 언어 모델 LLM을 결합한 PotholeSAM 프레임워크를 제안한다. 이 프레임워크는 세 단계로 구성된다; 첫째, YOLOv8을 사용하여 이미지 내 포트홀의 위치를 검출한다; 둘째, SAM2를 활용하여 검출된 포트홀 영역을 정밀하게 분할한다; 셋째, LLM이 분할된 영역의 밝기 정보를 분석하여 포트홀의 깊이를 추정한다. 제안된 방법은 포트홀의 위치, 면적, 깊이에 대한 정보를 자동으로 분석하여 도로 관리에 필요한 데이터를 효율적으로 획득할 수 있다. 이를 통해 지방 정부의 인력 부족을 완화하고 운전자에게 위험한 포트홀을 조기에 발견하여 선제적으로 대응할 수 있다.

키워드

포트홀 인식

PotholeSAM

YOLO

SAM2

대형 언어 모델

References

Achiam, J., Steven, A., Sandhini, A., Lama, A., Ilge, A., Florencia, L.A., Diogo, A., Janko, A., Sam, A., Shyamal, Al., Red, A., Igor, B., Suchir, B., Valerie, B., Paul, B., Haiming, B., Mohammad, B., Jeff, B., Irwan, B., Jake, B., Gabriel, B., Christopher, B., Lenny, B. et al. (2023) GPT-4 Technical Report, arXiv preprint arXiv:2303.08774.

Anti-Corruption and Civil Rights Commission (2024) 2022-2024 Civil Complaint Analysis System Statistical Report, Anti -Corruption and Civil Rights Commission.

Chang, Y., Jung, C., Ke, P., Song, H., Hwang, J. (2018) Automatic Contrast-Limited Adaptive Histogram Equalization with Dual Gamma Correction, IEEE Access, 6, pp.11782~11792.

10.1109/ACCESS.2018.2797872

David, A., Yigitcanlar, T., Li, R.Y.M., Corchado, J.M., Cheong, P.H., Mossberger, K., Mehmood, R. (2023) Understanding Local Government Digital Technology Adoption Strategies: A Prisma Review, Sustain., 15(12), p.9645.

10.3390/su15129645

Jo, Y.T., Ryu, S.K. (2016) Pothole Detection Algorithm Based on Saliency Map for Improving Detection Performance, J. Korea Inst. Intell. Trans. Syst., 15(4), pp.104~114.

10.12815/kits.2016.15.4.104

Kersten, T.P., Lindstaedt, M. (2022) Geometric Accuracy Investigations of Terrestrial Laser Scanner Systems in the Laboratory and in the Field, Appl. Geomat., 14(2), pp.421~434.

10.1007/s12518-022-00442-2

Kim, S.U., Kim, E.S., Kim, C.K. (2024) Development of Web-based Construction Site Safety Management Platform Using Artificial Intelligence, J. Comput, Struct, Eng, Inst, Korea, 37(2), pp.77~84.

10.7734/COSEIK.2024.37.2.77

Ravi, N., Gabeur, V., Hu, Y.T., Hu, R., Ryali, C., Ma, T., Khedr, H., Rädle, R., Rolland, C., Gustafson, L., Mintun, E., Pan, J., Alwala, K.V., Carion, N., Wu, C.Y., Girshick, R., Dollár, P., Feichtenhofer, C. (2024) SAM 2: Segment Anything in Images and Videos, arXiv preprint arXiv:2408.00714.

Siew, E.F., Ireland-Hay, T., Stephens, G.T., Chen, J.J.J., Taylor, M.P. (2005) A Study of the Fundamentals of Pothole Formation, Light Met., pp.763~769.

Smolyanskiy, N., Kamenev, A., Birchfield, S. (2018) On the Importance of Stereo for Accurate Depth Estimation: An Efficient Semi-Supervised Deep Neural Network Approach, In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp.1007~1015.

10.1109/CVPRW.2018.00147

Thompson, E.M., Ranieri, A., Biasotti, S., Chicchon, M., Sipiran, I., Pham, M.K., Nguyen-Ho, T.L., Nguyen, H.D., Tran, M.T. (2022) SHREC 2022: Pothole and Crack Detection in the Road Pavement Using Images and RGB-D Data, Comput. & Graph., 107, pp.161~171.

10.1016/j.cag.2022.07.018

Varghese, R., Sambath, M. (2024) YOLOv8: A Novel Object Detection Algorithm with Enhanced Performance and Robustness, In 2024 International Conference on Advances in Data Engineering and Intelligent Computing Systems (ADICS), IEEE, pp.1~6.

10.1109/ADICS58448.2024.10533619PMC12148043

Xu, Y., Yang, X., Yu, Y., Jia, W., Chu, Z., Guo, Y. (2022) Depth Estimation by Combining Binocular Stereo and Monocular Structured-Light, In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.1746~1755.

10.1109/CVPR52688.2022.00179

Zhao, T., Xie, Y., Dring, M., Yang, L., Tomizuka, M., Wei, Y. (2024) A Road Surface Reconstruction Dataset for Autonomous Driving, Sci. Data, 11(1), p.459.

10.1038/s41597-024-03261-938710687PMC11074133

Information

Publisher :Computational Structural Engineering Institute of Korea
Publisher(Ko) :한국전산구조공학회
Journal Title :Journal of the Computational Structural Engineering Institute of Korea
Journal Title(Ko) :한국전산구조공학회 논문집
Volume : 38
No :3
Pages :139-146
Received Date : 2024-12-20
Revised Date : 2025-03-07
Accepted Date : 2025-03-10
DOI :https://doi.org/10.7734/COSEIK.2025.38.3.139

Journal of the Computational Structural Engineering Institute of Korea ISSN:1229-3059(Print) 2287-2302(Online) 한국전산구조공학회 논문집

All Issue