Multi-scenario Prescription Recommendations for Crop Diseases Based on Multi-modal Data of Electronic Medical Records

ZHANG Lingxian, DING Junqi, CHEN Feifei, LI Yibin, ZHANG Yiding

Abstract

Considering challenges such as diverse crop varieties, complex disease types, significant sample data imbalance, varied prescription categories, and multi-modal data, prescription recommendation methods tailored to diverse, extensible, and multi-modal application scenarios were explored by using multi-modal EMR data. To accommodate the varying prescription preferences of agricultural producers, a diversified crop disease prescription recommendation model based on CdsBERT-RCNN and diagnostic reasoning was developed, improving diagnostic accuracy and prescription diversity for 32 common diseases. For untrained rare diseases and newly added prescriptions, an extensible crop disease prescription recommendation model based on MC-SEM and semantic retrieval was developed, enhancing semantic matching accuracy and case library retrieval speed, and providing prescription recommendations for untrained diseases. For multimodal information collection and input, a multi-modal crop disease prescription recommendation model based on BATNet multi-layer feature fusion was developed, enhancing prescription recommendation performance for multimodal data inputs. Results demonstrated that CdsBERT-RCNN achieved an 85.65% diagnostic accuracy and an F1 score of 85.63% across the 32 common diseases. In tests with varying input completeness levels, the model achieved 81.19% accuracy with symptom information alone, and the inclusion of environmental and crop information improved accuracy by 1.65 percentage points and 3.61 percentage points, respectively. MC-SEM achieved a Pearson correlation coefficient of 86.34% and a Spearman correlation coefficient of 77.67% for EMR semantic matching tasks;and achieved accuracy of 88.20% and 82.04% in the closed-set and open-set prescription recommendation tests, respectively, demonstrating its capability to expand to untrained diseases. BATNet achieved an accuracy and F1 score of 98.88% and 98.83%, respectively, for multi-modal input prescription recommendation tasks. Application scenario analysis and testing validated the model’s generalization capability for incomplete modalities (pure text or pure image) and incomplete information input (crop, environment, symptoms). The research result would provide an idea for digitally enabled crop disease control decision-making.

 

Keywords: prescription recommendation for crop diseases, natural language processing, semantic retrieval, multi-modal fusion, electronic medical records

 

Download Full Text:

PDF


References


LI S, LI K, QIAO Y, et al. A multi-scale cucumber disease detection method in natural scenes based on Y0L0v5[J]. Computers and Electronics in Agriculture, 2022,202:107363.

ZHAO X, LI K, LI Y, et al. Identification method of vegetable diseases based on transfer learning and attention mechanism [J]. Computers and Electronics in Agriculture, 2022,193:106703.

LI К Y, ZHANG L X, LI B, et al. Attention-optimized DeepLab V3 + for automatic estimation of cucumber disease severity [J]. Plant Methods, 2022,18(1) : 1 - 16.

GAO J, WESTERGAAKD J C, SUNDMAKK E H R, et al. Automatic late blight lesion recognition and severity quantification based on field imagery of diverse potato genotypes by deep learning [J] . Knowledge-Based Systems, 2021 , 214; 106723.

WANG G, SUN Y, WANG J. Automatic image-based plant disease severity estimation using deep leaming[J] . Computational Intelligence and Neuroscience, 2017, 2017: 1 -8.

TAMBO J A, UZAYISENGA B, MUGAMBI I, et al. Do plant clinics improve household food security?

ZHANG Lingxian, HAN Mengyao, DING Junqi, et al. Research progress in intelligent diagnosis and prescription recommendation of crop diseases[J]. Transactions of the Chinese Society for Agricultural Machinery, 2023, 54(6) ; 1 —18.

BUSBY P E, RIDOUT M, NEWCOMBE G. Fungal endophytes: modifiers of plant disease J . Plant Molecular Biology, 2016, 90(6) :645 -655.

MALAKOUTI S, HAUSKKECHT M. Predicting patient’s diagnoses and diagnostic categories from clinical-events in EHR data [ С]//Conference on Artificial Intelligence in Medicine in Europe, 2019.

VENKATESH R, BALASUBRAMANIAN C, KALIAPPAN M. Development of big data predictive analytics model for disease prediction using machine learning technique [J]. Journal of Medical Systems, 2019,43(8) ; 272.

ACOSTA J N, DORR F, GOICOCHEA M T, et al. Acute headache diagnosis in the emergency department accuracy and safety of an artificial intelligence system. Neurology, 2019,92; 5 - 10.

LIANG D, KRISHNAN R, HOFFMAN M, et al. Variational autoencoders for collaborative filtering [С] //International World Wide Wei Conferences Steering Committee, 2018.

ALI F, ISLAM S M K, KWAK D, et al. Type-2 fuzzy ontology-aided recommendation systems for lolT-based healthcare [J]. Computer Communications, 2018,119; 138 - 155.

SHI Y, YANG W, THUNG K, et al. Learning-based computer-aided prescription model for Parkinson’s disease: a data-driven perspective[J] . IEEE Journal of Biomedical and Health Informatics, 2020,25(9) ;3258 -3269.

HE X, FOLKMAN L, BORGWARDT K. Kernelized rank learning for personalized drug recommendation [J]. Bioinformatics, 2018,34(16) :2808 -2816.

DONG X, ZHENG Y, SHU Z, et al. TCMPR; TCM prescription recommendation based on subnetwork term mapping and deep leaming[J]. Biomed Research International, 2022 (1) : 4845726.

RONG C, LI X, SUN X, et al. Chinese medicine prescription recommendation using generative adversarial network [J]. IEEE Access, 2022,10:12219 - 12228.

ZHAO W, LU W, LI Z, et al. TCM herbal prescription recommendation model based on multi-graph convolutional network [J]. Journal of Ethnopharmacology, 2022,297:115109.

SEGURA-BEDMAR I, COLON-RUfZ C, TEJEDOR-ALONSO M A, et al. Predicting of anaphylaxis in big data EMR by exploring machine learning approaches [J]. Journal of Biomedical Informatics, 2018,87 :50 -59.

ZHAO J, GU S, MCDERMA1D A. Predicting outcomes of chronic kidney disease from EMR data based on random forest regression [J]. Mathematical Biosciences, 2019,310:24 -30.

XU C, DING J, QIAO Y, et al. Tomato disease and pest diagnosis method based on the stacking of prescription data [J]. Computers and Electronics in Agriculture, 2022,197:106997.

ZHANG Lingxian, ZHAO Dantong, DING Junqi, et al. Recommendation method of crop disease prescription based on CDSSM [J]. Transactions of the Chinese Society for Agricultural Machinery, 2023, 54(3) :308 -317.

DEVLIN J, CHANG M W, LEE K, et al. BERT; pre-training of deep bidirectional transformers for language understanding [J]. NAACL HLT, 2019,1:4171 -4186.

HUANG X, PENG H, ZOU D, et al. CoSENT: consistent sentence embedding via similarity ranking [J]. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 2024, 32: 2800 -2813.

LIU J, WANG X. Plant diseases and pests detection based on deep learning: a review[J]. Plant Methods, 2021, 17(1) ; 22.


Refbacks

  • There are currently no refbacks.