“Image-Text” Association Enhanced Multi-modal Swine Disease Knowledge Graph Fusion
Abstract
Traditional swine disease prevention primarily relies on human expertise, which risks missed diagnoses due to human error. To address this challenge, a multi-modal swine disease knowledge graph was developed to assist managers in better understanding the connections between pigs, providing a solid data foundation for identifying potential disease transmission paths and anomalies. Firstly, the swine disease data from various sources were collected, and then two preliminary multi-modal knowledge graphs were constructed through knowledge extraction and image matching. Secondly, a multi-modal knowledge graph fusion method based on “image-text” association was proposed, using a multi-head attention mechanism to reduce the impact of visual ambiguity and enhance swine disease entity representation. Finally, by calculating the similarity of entity representations in vector space, entities from the two multi-modal datasets were integrated into a more comprehensive knowledge graph. Experiments demonstrated that the proposed method improved alignment accuracy, as reflected by a 0.033 increase in Hits@1 compared with that of existing methods. Additional accuracy gains of 0.152, 0.236 and 0.180 were observed on the DBPZH-EN, DBPFR-EN, and DBPJA-EN datasets respectively, demonstrating its effectiveness in multi-modal knowledge graph fusion.
Keywords: swine disease, multi-modal knowledge graph, multi-modal fusion, entity alignment
Download Full Text:
PDFReferences
YAO Lii. Research on the influence of high-quality development of green finance enabling pig industry [J]. China Swine Industry, 2024, 19(5): 101 -109.
SHEN Mingxia, CHEN Jinxin, DING Qi'an, et al. Current situation and development trend of pig automated farming equipment application [J]. Transactions of the Chinese Society for Agricultural Machinery, 2022, 53(12); 1-19.
WU Qiong, LI Zhigang, CHEN Min. Knowledge map analysis of agricultural and rural informatization research at home and abroad [J]. Journal of Chinese Agricultural Mechanization, 2024, 45 ( 10) ; 215 -222.
ZHAO Chunjiang. Agricultural knowledge intelligent service technology: a review [J]. Smart Agriculture, 2023, 5(2) ; 126 - 148. (in Chinese)
XU Ruiqing, XU Duo, ZHANG Junmei, et al. A framework for constructing agricultural cognitive intelligence services and its application practice[J]. Journal of Nanjing University (Natural Science) , 2024, 60(4) ; 613 -624.
STEINER T, VERBORGH K, TRONCY H, et al. Adding realtime coverage to the Google knowledge graph [C] //Proceedings of the 11th International Semantic Web Conference (ISWC) , 2012; 65 -68.
NIU Peiyu, HOU Chen. Named entity recognition in Chinese rice breeding questions based on text data augmentation [J]. Transactions of the Chinese Society for Agricultural Machinery, 2024,55(8) ;333 -343. (in Chinese)
YUAN Licun, ZHOU Jun, GE Weixi, et al. Rice fertilization period discrimination method based on multi-modal knowledge graph [J]. Transactions of the Chinese Society for Agricultural Machinery, 2024, 55(9); 163 - 173.
CHEN Huaqiong. Technical analysis of prevention and treatment of African swine fever [J]. Anhui Agricultural Science Bulletin, 2024, 30(18):37 —41. (in Chinese)
LIU Guangtong, CAO Shunan, YANG Wanlin, et al. Development of feeding behavior analysis system for pigs in square cabin pig house based on multi-sensor [J]. Journal of Chinese Agricultural Mechanization, 2023, 44(9): 176 - 182,213.
YANG Liang, WANG Hui, CHEN Kuipeng, et al. Advances in research on pig-specific sensors [J]. Journal of Intelligent Agricultural Mechanization, 2023, 4(2) ;22 -34. (in Chinese)
LIU Zhenghao, ZENG Xi, ZHANG Zhijian. Research on construction and analysis of financial emergencies knowledge graph with event logical for emergency management [J ]. Journal of Information Resources Management, 2022, 12(3) : 137 — 151.
CHEN Lei, ZHOU Na, ZHU Pengxuan, et al. A dataset for constructing agricultural knowledge graph [J]. Journal of Agricultural Big Data, 2024, 6(1) :1 -8.
HOU Chen, NIU Peiyu. Review of research status and prospects of agricultural knowledge graphs [J] . Transactions of the Chinese Society for Agricultural Machinery, 2024, 55(6) ; 1-17. (in Chinese)
ZHANG Fu, YANG Linyan, LI Jianwei, et al. An overview of entity alignment methods [J]. Chinese Journal of Computers, 2022, 45(6) :1195 - 1225.
LIN Z, ZHANG Z, WANG M, et al. Multi-modal contrastive representation learning for entity alignment [С] / Proceedings of the 29th International Conference on Computational Linguistics (COLING) , 2022: 2572 -2584.
CHEN L, LI Z, WANG Y, et al. MMEA: entity alignment for multi-modal knowledge graph [C] // Knowledge Science, Engineering and Management; 13th International Conference (KSEM) , 2020: 134 - 147.
LIU F, CHEN M, ROTH D, et al. Visual pivoting for (unsupervised) entity alignment [C] // Proceedings of the AAAI Conference on Artificial Intelligence (AAAI), 2021; 4257 -4266.
GUO H, TANG J, ZENG W, et al. Multi-modal entity alignment in hyperlx>lic space[J] . Neurocomputing, 2021, 461; 598 -607.
CHEN Z, GUO L, FANG Y, et al. Rethinking uncertainly missing and ambiguous visual modality in multi-modal entity alignment [С] // International Semantic Web Conference. Springer (ISWC) , 2023: 121 - 139.
Refbacks
- There are currently no refbacks.