非可信智能驱动的可靠智造

研究概要

现有以深度学习为基础的人工智能技术本质上是非可信的，这限制了人工智能方法在以可靠性为基本要求的制造业中的应用，成为智能制造发展的根本性瓶颈所在。本项目团队围绕人工智能非可信性与智能制造可靠性之间的根本性矛盾，研究解决如下关键问题：智能制造环境下非可信智能的表征和边界判定；面向可靠智造的智能可信边界拓展方法框架和智能可信边界受限下的可靠智造方法框架。通过以上研究，系统性提出非可信智能下可靠智造的理论和方法框架，并依托联宝（合肥）电子科技有限公司笔记本生产全流程打造可靠智造示范样板，推动安徽省在智能制造领域的突破性发展，助力“中国制造2025”国家重大战略。

主要研究内容

• 智能制造环境下非可信智能的的表征和边界判定
• 面向可靠智造的智能可信边界拓展方法框架
• 智能可信边界受限下的可靠智造方法框架

基本研究框架

相关阅读

研究成果

Book Chapters

1. A Real-Time Detection Method for SMT Chip Component Defects Based on Adaptive Collaborative Feature Yunbo Zhao, Wangyou Gui, Lijun Zhao, Yu Kang, Kehao Shi, and Zhenyi Xu In Advances in Guidance, Navigation and Control 2025 [doi] [pdf]
2. SMT Component Defection Reassessment Based on Siamese Network Chengkai Yu, Yun-Bo Zhao , and Zhenyi Xu In Methods and Applications for Modeling and Simulation of Complex Systems 2022 [Abs] [doi] [pdf]

   In the SMT process, after component placement, checking the quality of component placement on the PCB board is a basic requirement for quality control of the motherboard. In this paper, we propose a deep learning-based classification method to identify the quality of component placement. This is a comparison method and the novelty is that the siamese network is trained to extract the features of the standard placement component map and the placement component map to be inspected and output the probability of similarity between the two to determine the goodness of the image to be inspected. Compared to traditional hand-crafted features, features extracted using convolutional neural networks are more abstract and robust. In addition, during training, the concatenated network pairs the sample images to expand the amount of training data, increasing the robustness of the network and reducing the risk of overfitting. The experimental results show that this method has better results than the general model for the classification of placement component images.

Journal Articles

1. Cross-Sensor Generative Self-Supervised Learning Network for Fault Detection Under Few Sample Huijuan Zhu, Yun-Bo Zhao , Xiaohui Yan, Yu Kang, and Binkun Liu J. Syst. Sci. Complex. 2025 [Abs] [pdf]

   In this paper, a cross-sensor generative self-supervised learning network is proposed for fault detection of multi-sensor. By modeling the sensor signals in multiple dimensions to achieve correlation information mining between channels to deal with the pretext task, the shared features between multisensor data can be captured, and the gap between channel data features will be reduced. Meanwhile, in order to model fault features in the downstream task, the salience module is developed to optimize cross-sensor data features based on a small amount of labeled data to make warning feature information prominent for improving the separator accuracy. Finally, experimental results on the public datasets FEMTO-ST dataset and the private datasets SMT shock absorber dataset(SMT-SA dataset) show that the proposed method performs favorably against other STATE-of-the-art methods.

2. Modelling and Optimizing Motherboard Functional Testing in Laptop Manufacturing Peng Bai, Yu Kang, Kangcheng Wang, Yunbo Zhao, and Shaojie Dong J Syst Sci Complex 2024 [Abs] [doi] [pdf]

   Functional testing is key to fulfill quality control in laptop manufacturing and is of great economic value. However, due to the unavailability of practical data, mathematical model and systematic perspective, it has barely been touched from the academic community to date. For the first time, this work provides technical understanding of the key principles of functional testing, mathematically models the general framework, elucidates existing testing strategy under the proposed framework and model, and finally proposes a specified optimization strategy which outperforms existing strategies. This work lays the model foundation for the further optimization of functional testing, and can be regarded as a good example of how a systematic approach can solve practical industrial challenges.

3. Deep Reinforcement Learning for Maintenance Optimization of Multi-Component Production Systems Considering Quality and Production Plan Ming Chen, Yu Kang, Kun Li, Pengfei Li, and Yun-Bo Zhao Quality Engineering 2024 [Abs] [doi] [pdf]

   In this article, the maintenance optimization of multi-component production systems is investigated by considering quality and production plan. On the one hand, the downtime determined by the production plan provides opportunities for reducing maintenance costs; on the other hand, the deterioration of product quality induced by poor health state leads to extra loss. The coupled relations between production plan, quality, and maintenance, as well as the dependence between multiple components, pose challenges for maintenance optimization. To overcome these challenges, a novel decision model and a deep reinforcement learning-based solving method are proposed. Specifically, in addition to the degradation states of all components, the remaining time of the current batch related to the production plan is also treated as the system state, and the quality loss related to the degradation states is added to the reward function. The deep Q-network algorithm is employed, solving the maintenance optimization problem that considers quality and production plan. The effectiveness of the proposed method is validated by a numerical experiment.

4. Multivariate Time-Series Modeling and Forecasting With Parallelized Convolution and Decomposed Sparse-Transformer Shusen Ma, Yun-Bo Zhao , Yu Kang, and Peng Bai IEEE Trans. Artif. Intell. 2024 [Abs] [doi] [pdf]

   Many real-world scenarios require accurate predictions of time series, especially in the case of long sequence timeseries forecasting (LSTF), such as predicting traffic flow and electricity consumption. However, existing time-series prediction models encounter certain limitations. First, they struggle with mapping the multidimensional information present in each time step to high dimensions, resulting in information coupling and increased prediction difficulty. Second, these models fail to effectively decompose the intertwined temporal patterns within the time series, which hinders their ability to learn more predictable features. To overcome these challenges, we propose a novel endto-end LSTF model with parallelized convolution and decomposed sparse-Transformer (PCDformer). PCDformer achieves the decoupling of input sequences by parallelizing the convolutional layers, enabling the simultaneous processing of different variables within the input sequence. To decompose distinct temporal patterns, PCDformer incorporates a temporal decomposition module within the encoder–decoder structure, effectively separating the input sequence into predictable seasonal and trend components. Additionally, to capture the correlation between variables and mitigate the impact of irrelevant information, PCDformer utilizes a sparse self-attention mechanism. Extensive experimentation conducted on five diverse datasets demonstrates the superior performance of PCDformer in LSTF tasks compared to existing approaches, particularly outperforming encoder–decoderbased models.

5. A Health Indicator Enabling Both First Predicting Time Detection and Remaining Useful Life Prediction: Application to Rotating Machinery Yun-Sheng Zhao, Pengfei Li, Yu Kang, and Yun-Bo Zhao Measurement 2024 [Abs] [doi] [pdf]

   Remaining Useful Life (RUL) prediction is vital for system functionality. Non-end-to-end approaches is an important type of RUL prediction approaches for their important application in industrial scenarios. In non-end-to-end approaches, Health Indicator (HI) construction is a critical aspect. However, existing HI construction approaches ignore First Predicting Time (FPT) detection, leading to increased domain knowledge demand and system health comprehension difficulty. To address this issue, this paper proposes a multi-objective-optimization-based HI construction approach enabling both FPT detection and RUL prediction. A novel metric called the monotonicity strength index is proposed to address the limitation of the conventional monotonicity. The constructed HI possesses the ability to indicate FPT by taking the detectability metric as an optimization objective. The optimization problem is solved by the combination of the multi-objective ant lion optimizer and the entropy weight method. The superiority of this HI is demonstrated through experiments on the widely used IMS bearing dataset and a gearbox dataset.

6. 非全时有效人类决策下的人机共享自主方法 游诗艺, 康宇, 赵云波, and 张倩倩 中国科学：信息科学 2022 [Abs] [doi] [pdf]

   在人机共享自主中, 人和智能机器以互补的能力共同完成实时控制任务, 以实现双方单独控制 无法达到的性能. 现有的许多人机共享自主方法倾向于假设人的决策始终“有效”, 即这些决策促进了 任务的完成, 且有效地反映了人类的真实意图. 然而, 在现实中, 由于疲劳、分心等多种原因, 人的决 策会在一定程度上“无效”, 不满足这些方法的基本假设, 导致方法失效, 进而导致任务失败. 在本文 中, 我们提出了一种新的基于深度强化学习的人机共享自主方法, 使系统能够在人类决策长期无效的情况下完成正确的目标. 具体来说, 我们使用深度强化学习训练从系统状态和人类决策到决策价值的 端到端映射, 以显式判断人类决策是否无效. 如果无效, 机器将接管系统以获得更好的性能. 我们将该 方法应用于实时控制任务中, 结果表明该方法能够及时、准确地判断人类决策的有效性, 分配相应的 控制权限, 并最终提高了系统性能.

Conference Articles

1. A Reliable Ensemble Model Based on Hierarchical Component Features for Repair Label Prediction of Soldering Defects Longxin Chen, Yunbo Zhao, Binkun Liu, Shaojie Dong, Huijuan Zhu, and Peng Bai In 2024 14th Asian Control Conf. ASCC 2024 [Abs] [pdf]

   Using solder paste inspection (SPI) and automated optical inspection (AOI) data, accurate prediction for repair labels of soldering defective printed circuit board (PCB) components can help reduce labor costs. Existing research tries to pick out both the false defect components (actually good) and impossible-to-repair components among defective PCB components, using SPI and AOI data. However, it is inappropriate to pick out the false defect components from screened components using defective information in AOI data. Therefore, the problem setting of existing research is inappropriate, resulting in the algorithm’s performance not meeting actual requirements. To address this problem, we only care about the reliable prediction of impossible-to-repair components. We propose a hierarchical component feature extraction method that can comprehensively characterize the degree of component defects from multiple levels, including pin level and component level. Then we apply the ensemble model based on XGBoost and TabNet and adjust the probability threshold of components judged as impossible-to-repair category, achieving the reliable prediction of impossible-to-repair components. Finally, we validated our method on real datasets and achieved better experimental results compared to baseline methods, which can meet actual requirements,

2. Prediction of Yield in Functional Testing of Motherboards in Laptop Manufacturing Yunbo Zhao, Shaojie Dong, Yu Kang, Kangcheng Wang, Longxin Chen, and Peng Bai In 2024 14th Asian Control Conf. ASCC 2024 [Abs] [pdf]

   Functional testing stands as a pivotal quality control step in the production process of laptop motherboards, aiming to validate the proper functioning of various components. However, due to the multitude of functional modules involved on the motherboard, testing all of them requires a significant amount of time and resources. As a result, production line engineers often rely on empirical selection of modules with low yield rates for testing. However, such empirical yield estimation is often inaccurate. To address this challenge, this study proposes a hybrid model based on XGBoost and Long Short-Term Memory (LSTM) networks to predict the yield of each functional module. By harnessing the feature learning capability of XGBoost and the sequential modeling power of LSTM, this model efficiently explores the intricate correlations among motherboard functional modules, thereby accurately forecasting their yields. We extensively train and validate the model using historical production data and successfully deploy it on real laptop motherboard production lines. Experimental results demonstrate that our hybrid model accurately predicts the yield of each functional module, providing crucial guidance for the functional testing process. Through in-depth analysis of the predicted yield results, engineers can systematically choose testing projects to save time and resources. This research offers a novel approach and pathway for enhancing motherboard production efficiency and quality.

3. Functional Test-Cost Reduction Based on Fault Tree Analysis and Binary Optimization Xiaojie Zuo, Kangcheng Wang, Yun-Bo Zhao , Yu Kang, and Peng Bai In 2024 43rd Chin. Control Conf. CCC 2024 [Abs] [doi] [pdf]

   With the rapid increase in the complexity of electronics, the cost of the functional testing process used to ensure product functionality continues to rise. Optimization modeling based on reliability analysis is an effective approach to reduce testing costs. However, the reliability calculations of existing methods often exhibit significant deviations, making it challenging to guarantee the effectiveness of the resulting testing strategies in practical applications. To address this issue, this article proposes an optimization modeling method that integrates statistical analysis and reliability analysis. The expression for the reliability of the system’s key stage is formulated by analyzing the system’s reliability. Statistical analysis is utilized to exploit the inherent reliability information in the enormous process data to determine the probability of the root causes of system failures. The reliability of the key stage is quantitatively calculated based on the structure of the fault tree tailored for the system. On this basis, a binary optimization model is established to obtain a testing strategy with strong generalization ability and reduce the cost of functional testing. The effectiveness of the proposed method is verified through a simulation case study.

4. Spectrally Normalized Adaptive Neural Identifier for Dynamic Modeling and Trajectory Tracking Control of Unmanned Aerial Vehicle Shaofeng Chen, Yu Kang, Yunbo Zhao, and Yang Cao In Adv. Guid. Navig. Control 2023 [Abs] [doi] [pdf]

   Accurate dynamic modeling is difficult for aerobatic unmanned aerial vehicles flying at their physical limit, due to the model uncertainty caused by unobservable hidden states like airflow and vibrations. Although some progresses have been made, these hidden states are still not properly characterized, rendering system identification problem for aerobatic unmanned aerial vehicle extremely challenging. To address this issue, a novel spectrally normalized adaptive neural identifier is proposed for the dynamic modeling of aerobatic unmanned aerial vehicles. Specifically, to characterize the model uncertainty, we propose a spectrally normalized adaptive neural network (SNANet) to extract deep features representing the hidden states of the system. Particularly, the proposed SNANet adopts a multi-model adaptive structure, quickly and dynamically updating the model online. Furthermore, the spectral normalization constraint is introduced into the training process to ensure the Lipschitz stability of the SNANet. Consequently, a trajectory tracking control scheme including the sliding mode controller and SNANet is presented. The modeling effectiveness of the proposed method is verified on a real flight dataset. The results demonstrate that our method has high modeling accuracy, short training time, and fast model response speed.

5. Board-Level Functional Test Selection Based on Fault Tree Analysis Yaoyao Li, Kangcheng Wang, Yu Kang, Yunbo Zhao, and Peng Bai In 2023 6th Int. Symp. Auton. Syst. ISAS 2023 [Abs] [doi] [pdf]

   With the increasing complexity of the circuit board, the cost of board-level functional test ensuring the board quality becomes dramatically high. Data-driven-based test selection methods have been widely studied for test-cost reduction. However, existing test selection methods tend to overfit due to overlooking the root causes of faulty boards. To address this issue, a test selection method based on reliability analysis is proposed. A fault tree oriented to the board-level functional test is established for analyzing the reliability of the board and test items. The reliability analysis result is then effectively utilized to formulate a test selection method. Three indices are introduced to evaluate the test efficiency and the test quality. Experimental results demonstrate the effectiveness of the proposed method.

6. A Feature Engineering-based Method for PCB Solder Paste Position Offset Prediction Binkun Liu, Yunbo Zhao, Yu Kang, Yang Cao, Peng Bai, and Zhenyi Xu In 2023 6th Int. Symp. Auton. Syst. ISAS 2023 [Abs] [doi] [pdf]

   Solder paste printing position offset is a common type of defective printed circuit board (PCB) printing, and accurate position offset prediction helps to avoid the production of defects, thus improving efficiency. The existing methods mainly use the powerful nonlinear fitting ability of deep learning to learn the variation pattern of solder paste printing quality to achieve a good prediction. However, factories also focus on the interpretability of the model, and existing methods are difficult to give the basis for decisions, so there are still limitations in the practical application. To solve this problem, we propose a Support vector machine (SVM) approach, in which we manually design 14 statistical features based on the original data, then the resampling reduces the effect of data imbalance and achieves PCB pad-level offset prediction. Finally we verified on about one week of real solder paste printing production data and achieved good experimental results.

7. Defect Detection of Laptop Appearance Based on Improved Multi-Scale Normalizing Flows Jie Zhang, Zerui Li, and Yunbo Zhao In 2023 38th Youth Acad. Annu. Conf. Chin. Assoc. Autom. YAC 2023 [Abs] [doi] [pdf]

   In the laptop production process, timely detection of appearance defects is essential to ensure product quality. At present, there are many shortcomings in the manual visual inspection-based method on the laptops production line. In addition, due to the wide variety of laptop appearance defects and extreme differences in defect scales, existing defect detection algorithms perform poorly in the field of laptop appearance inspection. In response to the above problems, this paper proposes a defect detection algorithm based on improved multi-scale normalizing flows. First, the multi-level features extracted from the backbone network are fused by using the pyramid feature fusion module to obtain multi-scale features with rich semantic and spatial information. Then, the effective density estimation of the multi-scale features is achieved by fusing the normalizing flows of attention mechanisms. Finally, the defects are detected and localized based on the output likelihood values. The experimental results demonstrate the effectiveness of the proposed method in detecting and locating appearance defects.

8. Shared Autonomy Based on Human-in-the-loop Reinforcement Learning with Policy Constraints Ming Li, Yu Kang, Yun-Bo Zhao , Jin Zhu, and Shiyi You In 2022 41st Chin. Control Conf. CCC 2022 [Abs] [doi] [pdf]

   In shared autonomous systems, humans and agents cooperate to complete tasks. Since reinforcement learning enables agents to train good policies through trial and error without knowing the dynamic model of the environment, it has been well applied in shared autonomous systems. After inferring the target from human inputs, agents trained by RL can accurately act accordingly. However, existing methods of this kind have big problems: the training of reinforcement learning algorithms require lots of exploration, which is time-consuming, lack of security guarantee and likely to cause great losses in the training process. Moreover, most of shared control methods are human-oriented, and do not consider the situation that humans may make wrong actions. In view of the above problems, this paper proposes human-in-the-loop reinforcement learning with policy constraints. In the training process, human prior knowledge is used to constrain the exploration of agents to achieve fast and efficient learning. In the process of testing we incorporate policy constraints in the arbitration to avoid serious consequences caused by human mistakes.

9. A Robustness Benchmark for Prognostics and Health Management Binkun Liu, Yun-Bo Zhao , Yang Cao, Yu Kang, and Zhenyi Xu In 2022 41st Chin. Control Conf. CCC 2022 [Abs] [doi] [pdf]

   With the rise of intelligent manufacturing, prognostics and health management(PHM) has developed rapidly as an important part of intelligent manufacturing.Existing deep learning-based PHM methods are data-dependent. However, sensor data often contains noise and is redundant and high-dimensional, making it difficult for the PHM methods to learn a stable set of model parameters, so the methods are likely to be wrong when disturbed. However, the factory hopes that the PHM methods are robust enough to adapt to various disturbances, so it is necessary to perform robustness evaluation on the existing methods in advance for easy deployment. Although the existing robust theoretical analysis methods for neural networks can obtain tight robust boundaries, they consume a lot of computing resources and are difficult to scale to large neural networks. To slove this problem, We design a benchmark for robustness analysis of large deep learning PHM models, in which we test the model robustness using a variety of perturbations to simulate the actual production environment of the factory. Specifically, Gaussian noise is used to test the robustness of the model to background noise; random mask is used to test the robustness of the model to data loss. We hope that our robustness benchmark can serve as a reference for designing PHM models to improve the robustness of factory PHM models.

10. Equipment Health Assessment Based on AHP-CRITIC Dynamic Weight Yunsheng Zhao, Pengfei Li, Tao Wang, Yu Kang, and Yun-Bo Zhao In 2022 41st Chin. Control Conf. CCC 2022 [Abs] [doi] [pdf]

    Prognostics Health and Management (PHM) has become a hot research problem with the improvement of different equipment. Besides, it is significant to assess the health status of equipment in PHM because an accurate health assessment can guide maintenance plans for engineers. To accurately reflect equipment health status by an index, an assessment method based on AHP-CRITIC dynamic weight is proposed in this paper. Analytic Hierarchy Process (AHP) is a subjective method used to evaluate the importance of different indicators. The criteria importance through inter-criteria correlation (CRITIC) method is used to calculate the contrast intensity of the same indicator and the conflict between indicators and obtain the objective weights. A set of more scientific weights is gained by combining the weights obtained from AHP and CRITIC, respectively. Moreover, to reflect each indicator’s real impact on overall health status, a dynamic weight adjustment mechanism is used. The case study of suction nozzles of a specific type of chip mounter shows that this method can reflect the health status accurately.

11. Anomaly Detection for Surface of Laptop Computer Based on Patchcore Gan Algorithm Huijuan Zhu, Yu Kang, Yun-Bo Zhao , Xiaohui Yan, and Junqiang Zhang In 2022 41st Chin. Control Conf. CCC 2022 [Abs] [doi] [pdf]

    Timely detection of notebook appearance defects is an important means to prevent products from being delivered to customers before leaving the factory.In industrial production, more emphasis is placed on fast and accurate detection methods, but the existing difficulties: 1. Defect samples are rare and difficult to obtain; 2. In high-resolution images, there are slight differences between abnormal samples and normal samples; 3. Slowly detection and insufficient accuracy.The existing methods mainly use a large amount of abnormal samples, so it is difficult to extend to the field of notebook appearance anomaly detection.To solve this problem, we designed a method that firstly uses unsupervised PatchCore which the algorithm was trained on normal samples and Defect GAN is used in test phase. To create a large number of verisimilitude abnormal samples and test these samples with PatchCore. On TKP-Surface datasets, the AUROC score of image-level anomaly detection achieves 96.1%, which meets the requirements of industrial applications.

Theses

1. 多部件系统的质量-维护-调度联合优化研究 陈明 中国科学技术大学, 合肥 2024 [Abs] [pdf]

   随着生产系统的复杂度不断提升，多部件系统逐渐成为现代生产制造过程 的核心，其生产管理研究引起了学术界和产业界的广泛关注。在此背景下，产品 质量、维护决策和生产调度是生产过程中的三个主要因素，且彼此间存在紧密的 相互作用。系统中任一部件故障会导致生产停机，影响生产计划的执行；在生产 调度过程中需要考虑维护时机，以便在计划的生产停机时间进行维护，减少维护 产生的停机成本；通过维护可以减少系统状态退化造成的质量损失，但频繁的维 护会增加维护成本。由于三者之间相互影响，同时优化质量、维护成本和交货期 这三个关键目标面临挑战。 针对以上问题，论文以多部件系统为研究对象，将产品质量退化过程纳入到 维护决策中，考虑维护与调度之间的相关性，在强化学习和遗传算法框架下，实 现质量、维护与调度的联合优化，主要研究内容包括： （1）针对固定生产计划下多部件系统的产品质量和维护决策联合优化问题， 将维护决策建模为马尔可夫决策过程，引入当前生产批次剩余时间的概念，在奖 励函数中加入质量损失以及与生产计划相关的停机成本，使用深度强化学习对 决策问题求解，实现降低维护成本的同时减少质量损失，并能够根据生产计划变 化动态调整维护策略。 （2）针对多部件系统的产品质量、维护决策和生产调度联合优化问题，设计 了一种两阶段联合优化方法。第一阶段以维护决策为主，在维护中考虑生产计划 以及产品质量，利用强化学习实现能够根据生产计划变化动态调整的维护策略。 第二阶段利用第一阶段得到的维护策略，计算生产过程中的维护时间、维护成本 和质量损失，使用遗传算法求解联合优化模型，实现减少延误成本的同时减少维 护成本和质量损失。 （3）针对生产计划扰动下并行多机系统的动态调度问题，设计了基于周期和 事件混合驱动策略，减少静态调度随时间累积的误差以及扰动事件对生产带来 的影响。并针对不同扰动事件分别设计了相应的调度修复策略，实现了处理生产 计划扰动的同时保持生产过程的稳定性。

2. SMT贴片元件自动光学检测关键技术研究 桂旺友 中国科学技术大学, 合肥 2024 [Abs] [pdf]

   近年来，表面贴装技术（Surface Mounted Technology, SMT）广泛地应用于 电子组装行业，高效的生产流程极大地促进了电子产品的智能化、精密化发展。 印刷电路板（Printed Circuit Board, PCB）上的布线和元件随之变得越来越密集， 这对其在生产过程中的质量检测问题也提出了更大的挑战。目前主流的方法是 采用自动光学检测（Automated Optical Inspection, AOI）技术对贴片元件经过回 流焊后的可能产生的缺陷情况进行检测。然而，当前的 AOI 技术大多还采用基 于模板匹配的传统视觉手段，在实际生产中仍然存在着较多的误检和漏检情况， 且整体工作效率不高，因此对其进一步地研究和改进有着重要的意义。 本文针对实际产线上的 AOI 系统的生产情况进行分析。研究发现，AOI 系 统的弊端主要在于使用的技术缺乏先进性，导致效率低下。一方面，由多张 PCB 局部图像拼接得到的整板图像质量不高，存在着明显的错位情况，这严重影响着 有缺陷区域的识别效果；另一方面，在缺陷检测时使用的模板匹配技术存在着较 大的弊端，该算法高度依赖于模板和拼接图像的质量，前者需要专业人员精心 设计建模，后者又取决于图像拼接的准确度。另外，匹配时的判定规则也比较单 一，难以应对复杂的缺陷特征，使得误检率达到了 80%，以至于需要大量人工复 检工作。针对以上问题，本文对这两个环节中所使用的关键技术进行研究，对图 像拼接算法进行了改进，并提出一种全新的缺陷检测算法，具体内容包括： 1. 针对 AOI 系统采集的若干张 PCB 局部图像拼接成 PCB 整板图像的问题， 将硬拼接和软拼接的方式相结合，提出一种基于 SURF 主要距离的主板图像拼 接算法，实现了 PCB 图像的实时高质量拼接。针对传统的 SURF 拼接算法进行 改进，提出基于划分区域的特征点提取方法和基于主要范围的匹配对细筛选方 法，在算法速度和准确度上都得到了较好的提升。以硬拼接流程为基础，为应 对由于震动导致的不同组相邻图像间偏移量一直在变化的情况，利用改进后的 SURF 算法提取出待拼接图像的特征点并匹配，将所有匹配对之间主要距离作为 偏移量，在每次拼接不同对象时都能做到实时调整。为了弥补光照不均造成的拼 接缝两边色差明显，采用基于加权像素的空域融合法处理该区域。以最先完成拼 接的第一列图像为准，通过偏移补偿方法保证所有其他列拼接图像尺寸与之一 致。在 PCB 整板图像拼接实验中，本文方法只需要 16s 左右的时间就可以在保 留图像高清晰度的前提下稳定地实现图像拼接过程，并且最终结果图的错位情 况得到明显改善。 2. 针对 SMT 产线中贴片元件经过回流焊后的缺陷检测问题，结合深度学习 技术，将目标检测应用于贴片元件的缺陷检测任务，提出一种基于自适应协同特征的 DETR 缺陷检测算法，实现了多种复杂缺陷元件的有效检测。为应对相关 数据稀少的情况，构建了 SMT 贴片元件缺陷数据集，包含多种存在不同缺陷情 况下的 PCB 图像。考虑到工业场景中对检测速度和精度的高要求，采用轻量化 的 RT-DETR 作为基准模型，在借助 transformer 强大的建模能力的同时，还具有 较好的实时性。针对贴片元件特殊的缺陷形态，设计出自适应协同模块替代部 分传统的卷积模块，能够更好地关注到元件缺陷的形变特征。提出一种全新的 损失函数，对 PCB 图像中的微小贴片元件更加友好，并且具备较快的收敛速度。 在自行收集贴片元件缺陷数据集的实验表明，本文方法在 mAP50 指标上达到了 88.3%，相对于基准方法提升了 8.4%，在 FPS 指标上达到了 90，在速度和精度 上达到了较好的平衡，优于目前所有的实时目标检测模型。 基于上述研究，本文实现了 SMT 贴片元件自动光学检测关键技术上的改进， 这些算法均将部署在课题组搭建的 AOI 设备上，预计建成后将有效提升制造商 的整体生产效率，并产生巨大经济效益。

3. 缺陷样本匮乏场景下的笔记本电脑表面缺陷检测方法研究 张杰 安徽大学, 合肥 2024 [Abs] [pdf]

   工业产品的表面缺陷检测是保证产品质量的一项重要步骤，在生产线上及时检测 并排除有缺陷的产品，可以确保制造出的产品符合质量标准和客户的期望，同时帮助企 业及时分析潜在的生产风险，提升产线的稳定性与生产效率。 在现有的笔记本电脑生产线上，仍然使用效率低、主观性强、成本高的人工目检方 式进行笔记本电脑的表面缺陷检测。随着计算机视觉、深度学习等人工智能技术的发 展，在缺陷样本数据充足的条件下，基于深度学习的缺陷检测算法能够有效地识别不同 类型的表面缺陷，具备较好的自适应性和准确性。然而，受限于笔记本电脑生产过程中 严格把控产品质量，产生缺陷的概率很小，导致难以收集相对充足的缺陷样本用于训练 深度学习算法。此外，基于有监督的深度学习检测算法往往难以检测训练集之外的未见 缺陷类型。为此，本文针对缺陷样本匮乏场景下的笔记本电脑表面缺陷检测问题，从无 监督学习、半监督学习的角度出发展开一系列研究，同时，通过合适的硬件选型构建图 像采集单元，并结合软件开发技术完成笔记本电脑外观缺陷检测系统的搭建。本文的主 要研究内容与工作如下：(1) 针对缺陷样本匮乏场景中难以获得足够的缺陷样本数据用于训练有监督缺陷检测 算法的问题，借助于无监督学习的思想，提出了一种基于多尺度标准化流的笔记 本电脑表面缺陷检测方法，在训练过程中仅利用正常样本。首先，通过多尺度特 征提取网络获取正常样本图像的多尺度特征，提高检测方法的尺度不变性。其次， 通过多尺度标准化流网络学习正常样本的特征分布，由于正常样本与缺陷样本之 间存在着明显的特征分布差异，通过在训练过程中对正常样本特征分布的拟合， 进而实现正常样本分布外的缺陷检测。 (2) 针对实际工业场景中，在获取充足的正常样本的同时，可以收集少量的缺陷样本 用于算法训练，借助于半监督、自监督学习的思想，提出了一种基于缺陷模拟的 记忆化笔记本电脑表面缺陷检测方法。在训练阶段，通过事先设计好的缺陷模拟 策略，在输入的正常样本的随机位置生成随机尺度大小的纹理缺陷，得到缺陷尺 度各异的模拟缺陷样本。然后，将生成的缺陷样本分别与正常样本特征记忆池和 缺陷样本特征记忆池进行最近匹配，利用与已知缺陷样本的共性和正常样本的差 异性辅助模型学习，最后通过解码器网络检测模拟缺陷样本的缺陷区域。在测试阶段，所提方法以端到端的方式对输入样本图像进行缺陷检测。 (3) 针对笔记本电脑生产过程中的表面缺陷检测流程，开发了一款笔记本电脑表面缺 陷检测系统。首先，基于工业相机、镜头、光源搭建实时图像采集单元。然后，基 于 PyQt5 配合图像采集单元进行软件开发。最后，通过集成训练好的缺陷检测算 法，该系统可以实时地对笔记本电脑进行表面缺陷检测，通过时耗分析，证明了 该系统的有效性与可应用性。

patent

1. 基于跨模态生成式学习的液压减震器设备异常检测方法 赵云波, 朱慧娟, 闫晓辉, and 康宇 2023 [pdf]
2. 锡膏印刷机参数调整数据处理软件V1.0 许镇义, 刘斌琨, 康宇, 曹洋, and 赵云波 2022 [pdf]
3. 锡膏印刷机在线故障预测软件V1.0 赵云波, 陈龙鑫, 朱慧娟, 康宇, and 许镇义 2022 [Abs] [pdf]

   本软件系统用于锡膏印刷机的在线故障预测，通过 url 接口实时获取锡膏印刷机 的状态数据，调用事先训练好的机器学习模型进行故障预测，并将故障预测结果 展示在前端可视化界面，以辅助工程师及时进行维护。

4. 锡膏印刷机离线故障预测软件V1.0 赵云波, 刘斌琨, 曹洋, 康宇, and 许镇义 2022 [pdf]
5. 基于特征工程的PCB板焊盘偏移预测方法及存储介质 曹洋, 刘斌琨, 赵云波, 康宇, and 许镇义 [Abs]

   本发明的一种基于特征工程的PCB板焊盘偏 移预测方法及存储介质，包括以下步骤，S1：获取 PCB板的历史偏移检测数据，进行数据预处理，并 使 用滑动窗 格法对数据进行 切分 ；S2 ：计算手工 设计的统计指标；S3：构建重加权支持向量机，输 出预测结果。本发明可根据手工设计的反应偏移 时间序列特点的统计特征对PCB板焊盘偏移进行 预测，通过工程师的适时介入从而避免生产不良 品 。同 时 可通过统 计特征对预 测结果进行解 释 ， 使得模型预测过程更容易被工程师理解。利用加 权的支持向量机对手工设计的特征进行核函数 的 非线性变换 ，提高 模型的 稳定性。对偏移不良 赋予更大的权重，克服了偏移不良数据极其稀少 的困难，从而实现偏移不良的时序预测。

6. 一种基于时间重构图卷积的PCB锡膏印刷质量预测方法 康宇, 刘斌琨, 赵云波, 曹洋, 许镇义, and 柏鹏 [Abs]

   本发明公开了一种基于时间重构图卷积的 PCB锡膏印刷质量预测方法，包括获取PCB板的历 史锡膏体积检测序列和对应的检测时间的历史 数据，并进行预处理；建立时间重构模块，根据对 应的检测时间的时间间隔对历史数据进行重构， 得到重构数据；建立时空图卷积模块，提取得到 的重构数据的锡膏体积之间的关联性和锡膏体 积自身的变化规律；根据锡膏体积之间的关联性 和锡膏体积自身的变化规律构建线性回归器，预 测得到锡膏体积变化规律。本发明根据焊盘上锡 膏体积的时间相关性构建稀疏图以提高相似演 变规律的协同同时避免不同演变规律的干扰然 后根据生产时刻的间隔构建时间注意力对数据 重构后预测PCB锡膏印刷质量。

7. 一种基于多传感器特征融合的轴承故障诊断方法 康宇, 刘斌琨, 赵云波, 许镇义, and 曹洋
8. 贴片机的健康度评估方法及系统 李鹏飞, 赵昀昇, 康宇, 赵云波, and 王涛 [Abs]

   本发明公开了一种贴片机的健康度评估方 法及系统，属于工业制造装备技术领域，包括：获 取贴片机当前运行状态数据；采用主成分分析法 对当前运行状态数 据进行特征提取 ，得到i个健 康监 测指标 ；采 用标准化欧氏 距离法 ，根据健康 监测指标在贴片机整体健康情况中的健康权重， 计算各健康监测指标的向量与理想健康向量之 间的距离；采用负向函数将距离换算成贴片机的 健康值。整个预测过程，简单易操作，可解释性很 强，不需要进行复杂地配置，贴合工业环境，无需 占 用过多的计算资源，即实现对贴片机的健康程 度评估。

9. 一种基于特征迁移的轴承剩余使用寿命预测方法 许镇义, 刘斌琨, 康宇, 赵云波, and 曹洋 [Abs]

   本发明涉及设备预测性维护技术领域，公开 了一种基于特征迁移的轴承剩余使用寿命预测 方法，包括：获取设备故障数据和设备全周期退 化数据，进行数据预处理；构建源网络，源网络包 括故障特征提取模块、故障分类器；设计故障分 类损失函数，并利用设备故障数据对源网络进行 训练 ；构建目标网络 ，目标网络包括退化特征提 取器和剩余使用寿命预测器；设计剩余使用寿命 预测损失函数，并利用设备全周期退化数据对目 标网络进行训练；设计故障特征迁移损失函数， 进行故障特征迁移。本发明通过对设备进行故障 诊断获取故障特征以及特征迁移，从而降低已有 方法对设备全周期退化数据的需求，并得到较好 的结果。

10. 一种基于多任务学习机制的PCB缺陷检测方法、系统 许镇义, 房明亮, 曹洋, 康宇, and 赵云波 [Abs]

    本发明涉及工业质检技术领域，公开了一种 基于多任务学习机制的PCB缺陷检测方法，将缺 陷图像输入至PCB缺陷检测模型，得到缺陷图像 对应的缺陷区域；PCB缺陷检测模型的训练方法， 包括以下步骤：获取缺陷图像后，进行数据预处 理 ，获得缺陷样本数据集 ；缺陷图像中缺陷的类 型包括偏移缺陷、缺件缺陷；构建基于U‑Net结构 的PCB缺陷检测模型：对偏移缺陷检测任务、缺件 缺陷检测任务、二分类缺陷检测任务分别设置损 失函数，并通过参数搜索方式为每个任务的损失 函数设置不同的权重；能够利用任务间的信息互 补性 ，降 低过拟合风险 ，充分利用多层次贴片元 件缺陷信息。

11. 印刷电路板的微小缺陷检测方法及存储介质 许镇义, 余程凯, 曹洋, 康宇, and 赵云波 [Abs]

    本发明的一种印刷电路板的微小缺陷检测方法及存储介质,包括以下步骤,S1、获取PCB缺陷样本数据并进行数据预处理；S2、在PCB训练集的边界框上使用k-means聚类来找到符合要求的anchor尺度；S3、采用多尺度特征金字塔结构提取特征,对主干卷积网络中得到自底向上的特征图进行上采样,得到自顶向下的特征图,然后将其与自底向上的特征图逐元素相加得到最终的特征图；S4、通过计算损失,训练网络参数。本发明通过数据增强技术提供深度学习所需要的充足训练数据,利用k-means聚类设计合理的锚点尺度,再将特征金字塔与Faster R-CNN网络相结合,加强了不同层次特征图之间的关系,从而实现对PCB微小缺陷的检测。本发明提高检测效率,并且能够适应多种缺陷检测,适应性强。

12. 用于PCB微小缺陷检测的单帧目标检测方法及存储介质 许镇义, 桂旺友, 曹洋, 康宇, and 赵云波 [Abs]

    本发明的一种用于PCB微小缺陷检测的单帧目标检测方法及存储介质,包括以下步骤：S1、获取PCB图像信息,并对图像进行数据预处理；S2、构建网络模型,将处理后的图像输入VGG-16特征提取网络,并对不同层次的特征进行融合,同时消除融合过程中所带来的负面影响；S3、对模型进行训练,并根据训练得到的结果评估性能。本发明利用注意机制来学习跨通道融合的特征之间的关系,并利用shuffle模块消除融合后的混叠效应。提出了非最大抑制方法,以减轻PCB图像的重叠效应。语义上升模块通过将不同层次的特征进行融合,不仅使低层次的特征具备丰富的语义信息,还能让高层次的特征具备更好的回归性,在目标分类与定位方面显著增强,能够更好地适应微小目标的检测。

13. 故障诊断模型的训练方法、装置、电子设备及存储介质 赵云波, 陈龙鑫, 刘斌琨, 朱慧娟, 许镇义, and 柏鹏 [Abs]

    本申请公开了一种故障诊断模型的训练方 法、装置、电子设备及存储介质，属于计算机技术 领域。所述方法包括：获取多个工况中各工况集 下的故障数据，分别作为源域数据，并获取目标 工况下的故障数据，作为目标域数据，所述工况 集包括所述多个工况中的至少一个工况，所述工 况集下的故障数据为已标记故障类别的数据，所 述目标工况下的故障数据为未标记故障类别的 数据；确定所述目标域数据与各所述源域数据之 间的目标分布差异；根据所述目标分布差异选取 源域数据作为训练数据；根据所述训练数据，对 所述目标工况的故障诊断模型进行训练。本申请 能够提高模型训练效果，进而提高对目标工况进 行故障诊断的准确率。

14. 产线设备故障预测方法、装置、电子设备及存储介质 赵云波, 董少杰, 刘斌琨, 朱慧娟, 许镇义, and 柏鹏 [Abs]

    本申请公开了一种产线设备故障预测方法、 装置、电子设备及存储介质，属于计算机技术领 域。所述方法包括：获取测试结果序列，测试结果 序列包括多个电路板的测试结果信息，测试结果 信息包括电路板中各功能模块的测试结果；根据 测试结果序列，获取各功能模块的良率序列；根 据各功能模块的良率序列，获取多个各功能模块 中的目标功能模块与其他功能模块的相关性序 列；根据各功能模块的良率序列和相关性序列， 获取目标功能模块的良率预测信息；根据相关性 序列和良率预测信息，预测目标功能模块对应的 产线设备是否存在故障。本申请能够准确预测产 线设备是否出现故障，以及时发现故障设备，避 免生产事故的发生。

15. 一种基于故障树和相关性分析的动态测项良率计算方法 赵云波, 马树森, 王康成, 康宇, and 柏鹏 [Abs]

    本发明涉及智能制造技术领域，公开了一种 基于故障树和相关性分析的动态测项良率计算 方法，包括：建立主板测项故障树，基于建立的测 项故障树构造测项的相关性矩阵，并利用该相关 性矩阵初步筛选出一个或多个与动态测项相关 性强的必测项；比较测项间的相关系数与人为设 定的阈值之间的大小，从而筛选出与动态测项关 联性最强的必测项；用必测项的良率作为动态测 项的良 率；本发明提供一种更加科学、合理的方 法，从而达到优化主板功能测试策略的目的，提 高测试效率。

16. 融合决策树和故障树分析的主板功能测试策略设计方法 赵云波, 李瑶瑶, 王康成, 康宇, and 柏鹏 [Abs]

    本发明涉及人工智能领域，具体涉及融合决 策树和故障树分析的主板功能测试策略设计方 法。本发明通过构建故障树分析主板故障，计算 测项故障概率和良率范围。在良率阈值设定范围 内，随机生成良率阈值，形成良率阈值矩阵，作为 决策树模型的输入，确定每个测项的测试策略。 进一步计算测试策略的经济效益，作为决策树模 型的输出。确定决策树模型中的目标路径，根据 目标路径上非叶节点包括的信息更新良率阈值 设定范围。设定准确的良 率阈值，与实际良 率比 较，得到最终测试策略。此方法为主板功能测试 环节设计出科学、有效、可解释的测试策略，解决 了行业内测试策略设计缺乏理论依据和可解释 性的问题，同时提升了整体经济效益，确保了主 板测试质量。

17. 基于多尺度标准化流的无监督笔记本外观缺陷检测方法 赵云波, 张杰, 李泽瑞, 康宇, and 吕文君 [Abs]

    本发明涉及工业缺陷检测技术领域，公开了 一种基于多尺度标准化流的无监督笔记本外观 缺陷检测方法，将采集得到的笔记本电脑外观图 像，依次输入到多尺度特征提取网络模型以及缺 陷检测模型，得到检测结果；训练方法包括：获取 笔记本电脑的原始外观图像后进行数据预处理， 得到训练数据集；构造基于ResNet50网络和特征 金字塔网络的多尺度特征提取网络模型，提取训 练数据集中外观图像的多尺度特征；构造基于多 尺度标准化流网络的缺陷检测模型，以多尺度特 征作为缺陷检测模型的输入，通过计算损失函数 对缺陷检测模型进行训练。本发明能很好地定位 不同尺度不同类型的缺陷，有着良好的检测效果 与缺陷定位效果。

18. 一种基于可靠性分析的主板功能测试策略方法及系统 赵云波, 李瑶瑶, 王康成, 康宇, and 柏鹏 [Abs]

    本发明公开了一种基于可靠性分析的主板 功能测试策略方法及系统，包括如下步骤：S1：分 析主板电路图 ，建立针对主板功能测试环节的故 障树；S2：对所述故障树进行可靠性分析，计算所 述故障树中的中间事件的发生概率，基于所述发 生概率设定可靠性指标阈值；S3：统计实际产线 上主板的测试数据，计算所述主板功能测项的实 际不良率，将所述实际不良率和所述可靠性指标 阈值进行比 较 ，得到主板功能测项的测试策略 ； 该主板功能测试策略方法及系统能够解决行业 内现行的主板功能测试策略设计方法缺乏理论 依据的问题，并且在保证行业良率要求的情况下 提升整体经济效益。

19. 一种细粒度自适应主板功能测项选择方法及系统 赵云波, 李瑶瑶, 王康成, 康宇, and 柏鹏 [Abs]

    本发明公开了一种细粒度自适应主板功能 测项选择方法及系统 ，包括如下步骤 ：S1 ：分析主 板电路图 ，建立针对主板功能测试的故障树 ；S2 ： 对所述故障树进行可靠性分析，获得所述故障树 中的顶事件和中间事件的发生概率表达式；S3： 搜集整理故障主板的维修数据并统计故障树中 基本事件的发生概率，设定主板功能测项的不良 率阈值；S4：当测试主板型号发生切换时，设定切 换后主板功能测试的不良率阈值；S5：计算主板 功能测项的实际不良率，将所述实际不良率和对 应主板型号的不良率阈值进行比较，得到主板功 能测项的测试策略 ；该测项选择方法为主板功能 测试的不同阶段设计出优于其他测项选择方法 的不良率阈值组合。

20. 一种基于金字塔增强网络的低光照下目标检测方法 赵云波, 尹相臣, 李泽瑞, 于桢达, and 康宇 [Abs]

    本发明涉及视觉感知技术领域，公开了一种 基于金字塔增强网络的低光照下目标检测方法， 构建一个金字塔增强网络，对图像进行增强并捕 获图像中的潜在信息。金字塔增强网络首先通过 拉普拉斯金字塔将图像分解成多个不同分辨率 的分量，在每个尺度的分量中，构建细节处理模 块和低频增强滤波器对分量进行增强。细节处理 模块由上下文分支和边缘分支组成，上下文分支 通过捕获长范围依赖来对分量进行全局增强，边 缘分支来对分量进行纹理的增强。低频增强滤波 器通过一个动态的低通滤波器获取低频的语义 信息并阻止高频的噪声，以丰富特征信息。金字 塔增强网络对低光图像进行增强后为后面的目 标检测模型提供更多的信息，提升检测器的性 能。

21. 减震器故障检测方法、装置、设备及存储介质 赵云波, 刘斌琨, 康宇, 曹洋, and 许镇义 [Abs]

    本发明涉及设备检测技术领域，公开了一种 减震器故障检测方法、装置、设备及存储介质，该 方法包括：基于待检测减震器的传感器数据获取 正对时频图像和负对时频图像；通过预设编码器 对正对时频图像和负对时频图像进行特征提取， 获得传感器特征集；对传感器特征集中各特征进 行更新，并对更新后的传感器特征集中的特征进 行特征融 合 ，获得融合特征集 ；通过预设线性分 类器对融合特征集进行分类检测，根据分类检测 结果判断待检测减震器是否存在故障。相比于现 有技术中通过花费高额成本来获取大量有标签 数据，本发明上述方法减少了现有技术中对有标 签数据的需求量过大导致成本过高的问题，从而 实现对减震器健康状态的预测性维护检测。

22. 笔记本生产线受损监控信息快速修复方法及存储介质 朱进, 黄蕾, and 赵云波 [Abs]

    本发明的一种笔记本生产线受损监控信息 快速修复方法及存储介质，其中方法包括粗略修 复阶段：先引入掩码使FlowNet2 .0能从参考帧中 提取受损的二维光流图 ，再建立线性预测模型， 根据运动在时间维度的连续性，来对受损帧进行 初步的 粗略 修复 ；精细修复阶 段 ：将粗略 修复的 结果充当精细修复阶段的参考信息，使用部分卷 积的帧修复网络PCFC‑Net来综合所有参考信息 并计算出精细的修复结果，挖掘深层的视频帧间 运 动信息 ，并 据此对缺 损帧 进行精细修复 。发明 实现了使用不完整的参考帧来对受损帧进行有 效的修复，同时显著减少了参考帧的采样窗口 ， 从而大大缩短监控视频修复模型在不稳定传输 场景下启动的等待时间，可有效提高笔记本生产 线上笔记本的精准快速定位。

项目人员

赵云波 何创创 余程凯 刘斌琨 刘朝虎 张天浩 张年坤 张杰 朱慧娟 李佳玉 李瑶瑶 桂旺友 王晓蓥 罗里恒 范冰 董少杰 谢飞 赵昀昇 陈明 陈龙鑫 青凡迪 马树森 齐振宇

项目合作