-
TLD算法是一种长时间跟踪算法,它在跟踪的基础上引入检测模块,从而具备跟踪失败后重新识别焊缝的能力[22]。TLD算法的核心思想是:跟踪模块利用中值流法跟踪特征点来获取目标的运动轨迹,由此计算目标在下一时刻的位置;检测模块通过滑动窗口扫描焊缝图像的目标及其周围区域,产生候选样本,以此来进行分类,根据分类结果可以定位目标的位置;最后将跟踪器与检测器所获得的结果进行融合,输出当前目标的最终位置。同时,学习模块通过正负(positive-negative,P-N)学习智能划分样本,并更新各个分类器的参量,以此提高跟踪的准确性。图 5是该算法流程图。
-
在焊接开始前,以焊缝特征点为中心,截取大小为W×H的图像区域,将其标记为跟踪目标。在目标区域周围选择若干个与其重叠度较大的候选区域,每个候选区域作一定范围内的仿射变换,由此来生成正样本集。再选出若干距离较远的候选区域作为负样本。这样能够解决训练样本不足的问题,令检测分类器初步具备识别目标的能力。跟踪过程中正负样本集为动态更新,即不会过度累积样本,生成焊缝图像正负样本的结果如图 6所示。
-
在跟踪模块中首先要确定有效特征点。在纵轴方向上激光条纹的光强近似服从高斯分布,而在跟踪过程中,同一列的激光条纹像素点主要沿纠偏方向移动,即沿焊缝图像的横坐标方向移动。因此,为了精准计算出焊缝偏差,特征点选择规则如下:在目标区域内,遍历图像中V型焊缝特征点的所在列,若像素点属于激光条纹区域(即灰度值大于背景阈值)则将其标记为有效特征点。图 7a中展示了第t帧图像检测到的特征点。
跟踪模块利用中值流法跟踪特征点,中值流法是一种改进的卢卡斯-卡纳德(Lucas-Kanade,LK)光流法。LK光流法能够计算相邻时刻特征点的运动矢量,光流方程为:
$ {I_x}u + {I_y}v + {I_t} = 0 $
(1) 式中, I代表激光条纹图像;Ix,Iy,It分别是t时刻激光条纹图像I的特征点灰度值对x,y,t方向求导;u,v分别为x,y方向的光流速率。
光流方程误差函数可以表示为[23]:
$ E = \sum\limits_{x, y \in w} {{{\left( {{I_x}u + {I_y}v + {I_t}} \right)}^2}} $
(2) 式中,在空间范围w内的光流为定值。函数E分别对u,v求导,当导数值为0时, 即可得到该点的光流矢量U:
$ \mathit{\boldsymbol{U}} = \left[ {\begin{array}{*{20}{l}} u\\ v \end{array}} \right] = {\left[ {\begin{array}{*{20}{c}} {\sum\limits_w {I_x^2} }&{\sum\limits_w {{I_x}} {I_y}}\\ {\sum\limits_w {{I_x}} {I_y}}&{\sum\limits_w {I_y^2} } \end{array}} \right]^{ - 1}}\left[ {\begin{array}{*{20}{c}} { - \sum\limits_w {{I_x}} {I_t}}\\ { - \sum\limits_w {{I_y}} {I_t}} \end{array}} \right] $
(3) 令S=(It, It+1, …, It+k)代表焊缝图像序列,在t时刻图像特征点的位置为xt,用光流法前向跟踪该特征点k步,得到前向轨迹Tf, k=(xt, xt+1, …, xt+k)。向后跟踪特征点xt+k到第t帧,产生验证轨迹Tb, k=$\left( {{{\hat x}_t}, {{\hat x}_{t + 1}}, \cdots , {{\hat x}_{t + k}}} \right)$,${{{\hat x}_t}}$为t时刻通过光流法后向跟踪得到的特征点位置,这两条轨迹的距离记为前向-后向误差f,通常使用欧氏距离进行计算:
$ f = \left\| {{x_t} - {{\hat x}_t}} \right\| $
(4) 令前向-后向误差作为指标,找到误差较小的特征点,通过统计这些特征点的位置变化规律,可以预测下一时刻矩形框的位置,图 7b为第t+1帧焊缝图像通过中值流法跟踪到的特征点。
-
检测模块利用一个多尺度窗口对当前帧的焊缝图像进行扫描并获得若干个目标候选样本,然后将它们输入级联分类器进行分类,其中包含目标的样本为正样本,反之为负样本。检测模块在ROI图中搜索目标,因此候选样本数量大幅减少,算法实时性能显著提升。
-
焊缝中心的最终位置是由综合模块输出得到的,在此之前综合模块接收了跟踪与检测的结果。若跟踪出现失效或漂移的现象,检测模块会找到焊缝特征点的位置并纠正跟踪模块,一般情况下跟踪和检测共同决策,输出焊缝特征点的位置。
-
学习模块的P-N学习算法负责对误分类的样本重新标记,这样有助于提高分类器的性能。从时域的角度分析,在时间序列上跟踪模块跟踪到的一系列焊缝特征点构成了一条较平滑的轨迹,该轨迹与焊枪纠偏轨迹相对一致。当跟踪器跟踪到的焊缝特征点所在区域被检测器误分类为负样本时,学习模块会重新将其标记为正样本。从空域的角度分析,在跟踪过程中,跟踪器跟踪到的焊缝特征点有可能出现“漂移”现象,从而导致与检测器的分类结果不一致。此时检测器找出焊缝特征点最可能出现的位置,并且将其它区域标记为负样本。
-
为了验证算法的有效性,制定了如下的试验方案:选择10mm厚V型坡口和5mm厚不带坡口的不锈钢板,分别用于V型焊缝和搭接焊缝跟踪,V型坡口宽度为8mm,传感器在路径上的运动速率为2mm/s,传感器的采集速率为5frame/s,在工作平面O-xy坐标系上,焊缝跟踪起点和终点坐标分别为(0, 0)和(100, 10),激光视觉传感器移动总路程为100mm,图 8中展示了焊缝与传感器路径。
对传感器采集到的图像运行改进TLD目标跟踪算法,进行两种焊缝的跟踪试验。图 9中展示了应用改进TLD算法跟踪后的不同情况。图中“×”点标示了焊缝特征点的位置,虚线框和实线框分别为跟踪模块和检测模块的输出。图 9a中出现了跟踪漂移现象,检测模块纠正了这种错误,并输出焊缝跟踪点的正确位置;图 9c中跟踪模块失效,不过检测模块找到目标,故选择检测模块输出作为最终结果;图 9b、图 9d中的结果为跟踪成功。综上各种情况,改进TLD算法均能成功识别焊缝。
一般V型焊缝和搭接焊缝以图 9所示的特征点作为焊缝中心,将跟踪到的特征点像素坐标值转换到激光测量坐标系中,并以此作为分析模型。
传统的线激光图像处理算法的工作流程通常包括平滑滤波、提取骨架、拟合直线、提取特定像素点。应用改进TLD算法和传统图像处理算法进行两种焊缝的跟踪试验,图 10中展示了不同算法的跟踪路径,表 1、表 2为数据统计情况。
Table 1. Data statistics of testing results for V-shaped weld tracking
algorithm mean absolute
error/mmmaximum absolute
error/mmmean square
error/mmtraditional algorithm 0.094 0.427 0.295 improved TLD algorithm 0.062 0.237 0.118 Table 2. Data statistics of testing results for lapped weld tracking
algorithm mean absolute
error/mmmaximum absolute
error/mmmean square
error/mmtraditional algorithm 0.069 0.349 0.157 improved TLD algorithm 0.052 0.235 0.087 由图 10和表 1、表 2可知,改进TLD算法相比传统图像处理算法检测精度更高,两种焊缝跟踪平均绝对误差分别为0.062mm和0.052mm。结果表明,改进TLD算法能精确定位焊缝特征点,单帧图像算法运行时间大约为60ms。
基于改进TLD算法的激光视觉传感型焊缝跟踪
A laser vision sensing method for seam tracking based on an improved TLD algorithm
-
摘要: 为了解决基于线激光视觉传感的焊缝中心位置定位精度不高的问题, 采用了一种基于改进跟踪-学习-检测(TLD)算法的焊缝跟踪方法。由激光视觉传感器实时获取焊缝图像, 采用将跟踪器与检测器结合的TLD算法实时跟踪焊缝特征点, 同时通过在线学习机制更新分类器参量。在此基础上对激光条纹图像截取感兴趣区域, 大幅减少检测器的搜索区域; 根据激光条纹光强分布特性, 结合纠偏方向选取跟踪器有效特征点, 以此提高算法效率, 对不锈钢板V型焊缝和搭接焊缝进行跟踪试验。结果表明, 跟踪与检测可实现共同定位焊缝中心位置, 其融合的焊缝跟踪方法能够准确地提取焊缝特征点, 两种焊缝跟踪平均绝对误差分别为0.062mm和0.052mm。此方法为提高焊缝跟踪精度提供了依据。
-
关键词:
- 图像处理 /
- 焊缝跟踪 /
- 跟踪-学习-检测算法 /
- 激光视觉
Abstract: In order to solve the problem of low positioning accuracy of the weld seam center based on line laser vision sensing, a seam tracking method based on an improved tracking-learning-detection (TLD) algorithm was adopted. The weld images were acquired in real time during the weld seam tracking. The TLD algorithm combining the tracker (tracking) and the detector (detection) was adopted to track weld feature points in real time and the online learning mechanism (learning) was adopted to update the classifier parameters, so as to improve the accuracy of seam tracking. On this basis, the region of interest (ROI) was intercepted from the laser stripe images, which greatly reduced the detector's search area. The effective feature points of the tracker were selected to improve the efficiency of the algorithm according to the characteristics of the light intensity distribution of the laser stripe in combination with the rectifying direction. The V-shaped weld and the lapped weld of the stainless steel plate were tracked. The results indicate that the location of the seam center can be achieved by tracking and detecting and the fusion weld tracking method can accurately extract weld feature points. The mean absolute tracking errors of both weld seams were 0.062mm and 0.052mm. This method provides the basis for improving the accuracy of weld seam tracking.-
Key words:
- image processing /
- seam tracking /
- TLD algorithm /
- laser vision
-
Table 1. Data statistics of testing results for V-shaped weld tracking
algorithm mean absolute
error/mmmaximum absolute
error/mmmean square
error/mmtraditional algorithm 0.094 0.427 0.295 improved TLD algorithm 0.062 0.237 0.118 Table 2. Data statistics of testing results for lapped weld tracking
algorithm mean absolute
error/mmmaximum absolute
error/mmmean square
error/mmtraditional algorithm 0.069 0.349 0.157 improved TLD algorithm 0.052 0.235 0.087 -
[1] GU J, ZHANG L L, WANG J Ch. Application of machine vision in automatic welding[J]. Applied Laser, 2018, 38(6): 1038-1043(in Chinese). [2] DING Y Y, HUANG W, KOVACEVIC R. An on-line shape-match-ing weld seam tracking system[J]. Robotics and Computer-Integrated Manufacturing, 2016, 42: 103-112. doi: 10.1016/j.rcim.2016.05.012 [3] ZOU Y B, CHEN X Zh, GONG G J, et al. A seam tracking system based on a laser vision sensor[J]. Measurement, 2018, 127: 489-500. doi: 10.1016/j.measurement.2018.06.020 [4] GAO X D, LIU Y Y, LAN Ch Zh, et al. Laser-induced infrared characteristic analysis for evaluating joint deviation during austenitic stainless steel laser welding[J]. International Journal of Advanced Manufacturing Technology, 2017, 88(5/8): 1877-1888. doi: 10.1007%2Fs00170-016-8892-x [5] SHEN H Y, LIN T, CHEN Sh B, et al. Real-time seam tracking technology of welding robot with visual sensing[J]. Journal of Intelligent and Robotic Systems, 2010, 59(3/4): 283-298. doi: 10.1007/s10846-010-9403-1 [6] GAO X D, CHEN Y Q, YOU D Y, et al. Detection of micro gap weld joint by using magneto-optical imaging and Kalman filtering compensated with RBF neural network[J]. Mechanical Systems and Signal Processing, 2017, 84: 570-583. doi: 10.1016/j.ymssp.2016.07.041 [7] MA N J, GAO X D, ZHOU X H, et al. Analysis of magneto-optical imaging characteristics of weld defects under magnetic field excitation[J]. Laser Technology, 2018, 42(4): 97-102(in Chinese). [8] LU X Q, GU D X, WANG Y D, et al. Feature extraction of welding seam image based on laser vision[J]. IEEE Sensors Journal, 2018, 18(11): 4715-4724. doi: 10.1109/JSEN.2018.2824660 [9] SHAO W J, HUANG Y, ZHANG Y. A novel weld seam detection method for space weld seam of narrow butt joint in laser welding[J]. Optics & Laser Technology, 2018, 99: 39-51. [10] FAN J F, JING F Sh, FANG Z J, et al. Automatic recognition system of welding seam type based on SVM method[J]. The International Journal of Advanced Manufacturing Technology, 2017, 92(1/4): 989-999. doi: 10.1007/s00170-017-0202-8 [11] LIN Sh D, GAO X D, LI Y J, et al. Weld deviation prediction algorithm based on neural network compensation Kalman filter[J]. A-pplied Laser, 2018, 38(6): 940-945(in Chinese). [12] DU R Q, XU Y L, HOU Zh, et al. Strong noise image processing for vision-based seam tracking in robotic gas metal arc welding[J]. International Journal of Advanced Manufacturing Technology, 2019, 101(5/8): 2135-2149. [13] ZOU Y B, ZHOU W L, WANG Y B. Laser vision seam automatic tracking based on probability continuous model[J]. Journal of Mechanical Engineering, 2017, 53(10): 70-78(in Chinese). doi: 10.3901/JME.2017.10.070 [14] ZOU Y B, CHEN T. Laser vision seam tracking system based on i-mage processing and continuous convolution operator tracker[J]. Optics and Lasers in Engineering, 2018, 105: 141-149. doi: 10.1016/j.optlaseng.2018.01.008 [15] MENG L, YANG X. A survey of object tracking algorithms[J]. Acta Automatica Sinica, 2019, 45(7): 1244-1260(in Chinese). [16] LIN Sh D, GAO X D, LI Y J, et al. Weld feature extraction and location based on corner detection and optical flow tracking[J]. Journal of Mechanical & Electrical Engineering, 2019, 36(4): 440-444(in Chinese). [17] MA B, LIN Sh D, ZHANG N F, et al. Weld seam tracking method based on slope characterization and Kalman filter[J]. Journal of Mechanical & Electrical Engineering, 2020, 37(2): 206-210(in Chin-ese). [18] ZOU Y B, WANG Y B, ZHOU W L. Research on line laser seam tracking method based on Guassian kernelized correlation filters[J]. Applied Laser, 2016, 36(5): 578-584(in Chinese). [19] ZOU Y B, LAN R, WEI X Zh, et al. Robust seam tracking via a deep learning framework combining tracking and detection[J]. A-pplied Optics, 2020, 59(14): 4321-4331. [20] KALAL Z, MIKOLAJCZYK K, MATAS J. Tracking-learning-detection[J]. IEEE Transactions on Software Engineering, 2011, 34(7): 1409-1422. [21] MA B, GAO X D, ZHANG N F, et al. Research on 3-D reconstruction method of multi-layer single-pass arc additive manufacture surface[J]. Laser Technology, 2020, 44(3): 321-325(in Chinese). [22] MAO X B, ZHOU X D, LIU Y H. Improved TLD object tracking algorithm based on fast feature points[J]. Journal of Zhengzhou University (Engineering Science Edition), 2018, 39(2): 1-5(in Chin-ese). [23] LI Ch M, BAI H Y, GUO H W, et al. Moving object detection and tracking based on improved optical flow method[J]. Chinese Journal of Scientific Instrument, 2018, 39(5): 249-256(in Chinese).