Given specific data conditions, such as no more than k blood glucose measurements per day, we randomly discard blood glucose values within the trajectories to ensure that the remaining trajectories satisfy this criterion.

The traditional clinical methods of insulin dosage titration were used as the standard clinical methods for comparison, consisting of guidelines 42 and consensus formulas 43 , 44 for premixed insulin regimen, basal regimen and basal-bolus regimen.

The detailed adjustment was according to the following formula:. The insulin dosage titration rules of basal-bolus regimen were as follows. Retrospective study phase of the internal cohort. Forty eligible patients with T2D treated with insulin injection were randomly selected from the retrospective EHRs of one of the modeling development hospitals Qingpu Hospital from May to December Two treatment days were randomly selected for each patient, resulting in 80 cases with insulin points Extended Data Fig.

Three physician groups with different levels of clinical experience provided their dose recommendations, and the AI also generated insulin dose recommendations in silico for further evaluation. An expert consensus panel of three endocrinology specialists conducted blinded review and provided their own recommended insulin dosage.

This was used as a reference insulin dosage for each insulin point to assess the accuracy of AI-generated dosage versus the three physician groups. Retrospective study phase of the external cohort. The retrospective dataset was collected from a non-teaching hospital XuHui Hospital , which included 45 eligible consecutive patients with T2D from April to August Extended Data Fig.

The dataset contained insulin points from cases, and AI-generated dosage was compared to previously delivered insulin dosage by treating physicians human plan for accuracy evaluation.

Next, we randomly selected 40 cases from the dataset to evaluate the acceptability, effectiveness and safety of the AI plan and the previous human plan. The evaluations were blinded head-to-head comparisons of AI versus human plans by the expert consensus with three independent experts.

Prospective deployment study phase. In May , 40 consecutive AI-generated plans were tested for acceptance, effectiveness and safety by endocrinology physicians at the bedside Extended Data Fig.

After determining clinical adoption and ensuring adherence to standard clinical quality controls, the AI insulin regimen was used for patient treatment. The inclusion and exclusion criteria for patients were consistent across the three phases.

Inclusion criteria were patients with T2D treated with subcutaneous insulin injection for at least two consecutive days. Patients with acute complications of diabetes, such as ketoacidosis or hyperglycemic hyperosmolar state, or patients who were treated with glucocorticoids, were excluded.

Quantitative evaluation. We used the metrics of MAE and agreement percentage to quantitatively evaluate the accuracy performance of insulin regimens. MAE represents the errors between predicted values and consensus values. Effectiveness and safety. The effectiveness was scaled on a five-point Likert scale ranging from 1 very poor control of glycemia to 5 very good control of glycemia.

For safety evaluation, we used questionnaires 2 and 3 item 5 and questionnaire 4 item 6 , which asked the reviewers if the recommended insulin regimen was perceived to lead to an increased risk of hypoglycemia according to their judgment.

The safety was then scaled on a five-point Likert scale ranging from 1 very high risk to 5 very low risk Supplementary Information.

Superior plan. In the head-to-head comparison of the AI and human plans in the retrospective simulation study of the external cohort, the one AI or human selected as most clinically appropriate by the expert consensus review was considered as the superior plan Supplementary Information.

In the prospective deployment phase, the AI plans were reviewed by the endocrinology physicians at the bedside; the clinical adoption was determined; and the deemed AI insulin regimen was used for patient treatment following all standard clinical quality controls.

We conducted a proof-of-concept trial ClinicalTrials. gov: NCT to evaluate the feasibility and safety of AI in inpatients with T2D from 28 June to 6 October This trial was a patient-blinded and single-arm intervention, which was performed in the ward of the Department of Endocrinology and Metabolism, Zhongshan Hospital, in China.

The RL-DITR system was embedded in the insulin dosing interface of the health information system HIS , allowing real-time reading of patient clinical information and insulin dosage regimen recommendation Extended Data Fig.

An example of the AI recommendation report for the healthcare provider is presented in Extended Data Fig. Patients with T2D receiving subcutaneous insulin treatment were recruited and screened for the inclusion and exclusion criteria.

The pre-intervention initial insulin regimen served as reference for daily insulin regimen. Eligible patients received insulin dosage titration according to the AI model after the first cycle of insulin regimen, which was confirmed twice daily by the physician in charge.

The treating physician could reject the recommendation if deemed necessary. Throughout the trial, anti-hyperglycemic drugs remained unchanged; standard meals at usual mealtime were provided; and no physical activity was scheduled.

Capillary glucose concentration was measured at seven timepoints of fasting, after breakfast, before and after lunch, before and after dinner and before bedtime a day by a glucometer Glupad, Sinomedisite to estimate glucose control and to guide insulin regimen.

The goal was to achieve preprandial capillary blood glucose of 5. The CGM data were analyzed by physicians, and the treatment was not influenced by data gained by CGM.

CGM alarms were not activated during the feasibility clinical trial. The primary outcome was difference in glycemic control as measured by mean daily blood glucose concentration total, preprandial and post-prandial capillary blood glucose.

The secondary endpoints included glucose concentration in the target range TIR of 3. Glycemic variability was determined by the CV of glucose values. Safety was assessed as the number of hypoglycemic events.

Serious adverse events included severe hypoglycemia, defined as a capillary glucose level of less than 2. The sample size calculation was based on the primary outcome. PASS software version Clinical studies were analyzed using SAS 9. The matched t -test was used to compare the performance of RL-DITR and physicians.

The change from baseline measurements to the end of the trial was analyzed by two-sided paired t -test and a Wilcoxon signed-rank test for continuous measurements. The seven-point blood glucose profiles were analyzed using a generalized linear mixed model. The model used a Noisy-OR approach to aggregate WTR predicted probabilities of points for daily WTR prediction.

Further information on research design is available in the Nature Portfolio Reporting Summary linked to this article. IRB approval was obtained from institutions for EHR data collection. Individual-level patient records can be accessible with IRB consent and are not publicly available.

De-identified data can be requested by contacting the corresponding authors. All data access requests will be reviewed and if successful granted by the Data Access Committee. Data can be shared only for non-commercial academic purposes and will require a formal material transfer agreement.

Individual-level data of the clinical trial ClinicalTrials. gov: NCT reported in this study are not publicly shared. Data can be available to bona fide researchers for non-commercial academic purposes and necessitate a data user agreement.

Requests should be submitted by emailing the corresponding authors Y. at chen. ying4 zs-hospital. cn or guangyu. wang24 gmail. Requests will be processed within a 2-week timeframe. All data shared will be de-identified. The deep learning models were developed and deployed in Python version 3.

The following standard model libraries were used: scikit-learn 1. Sun, H. et al. IDF Diabetes Atlas: global, regional and country-level diabetes prevalence estimates for and projections for Diabetes Res. Article PubMed Google Scholar.

Stratton, I. Association of glycaemia with macrovascular and microvascular complications of type 2 diabetes UKPDS 35 : prospective observational study. BMJ , — Article CAS PubMed PubMed Central Google Scholar. Holman, R. Article CAS PubMed Google Scholar. ElSayed, N. Pharmacologic approaches to glycemic treatment: standards of care in diabetes— Diabetes Care 46 , S—S American Diabetes Association.

Glycemic targets: standards of medical care in diabetes— Diabetes Care 44 , S73—S84 Article Google Scholar. Martinez, M. Glycemic variability and cardiovascular disease in patients with type 2 diabetes.

BMJ Open Diabetes Res. Care 9 , e Article PubMed PubMed Central Google Scholar. Rodbard, D. Glycemic variability: measurement and utility in clinical medicine and research—one viewpoint. Diabetes Technol. Bi, W. Artificial intelligence in cancer imaging: clinical challenges and applications.

CA Cancer J. Esteva, A. Dermatologist-level classification of skin cancer with deep neural networks. Nature , — Kermany, D. Identifying medical diagnoses and treatable diseases by image-based deep learning.

Cell , — Wang, G. A deep-learning pipeline for the diagnosis and discrimination of viral, non-viral and COVID pneumonia from chest X-ray images. Zhang, K. Deep-learning models for the detection and incidence prediction of chronic kidney disease and type 2 diabetes from retinal fundus images.

Kaelbling, L. Reinforcement learning: a survey. Gottesman, O. Guidelines for reinforcement learning in healthcare. Komorowski, M. The Artificial Intelligence Clinician learns optimal treatment strategies for sepsis in intensive care.

Guo, H. Learning dynamic treatment strategies for coronary heart diseases by artificial intelligence: real-world data-driven study. BMC Med. Bothe, M. The use of reinforcement learning algorithms to meet the challenges of an artificial pancreas.

Expert Rev. Devices 10 , — Liu, Z. A deep reinforcement learning approach for type 2 diabetes mellitus treatment. Oh, S. Reinforcement learning-based expanded personalized diabetes treatment recommendation using South Korean electronic health records.

Expert Syst. Raheb, M. Subcutaneous insulin administration by deep reinforcement learning for blood glucose level control of type-2 diabetic patients. Thomas, M. Model-based reinforcement learning: a survey. Trends Mach. Learn 16 , 1— Google Scholar. Huang, Q. Model-based or model-free, a review of approaches in reinforcement learning.

Coronato, A. Reinforcement learning for intelligent healthcare applications: a survey. Nemati, S. Optimal medication dosing from suboptimal clinical examples: a deep reinforcement learning approach. IEEE Eng.

PubMed Google Scholar. Vasey, B. Reporting guideline for the early-stage clinical evaluation of decision support systems driven by artificial intelligence: DECIDE-AI.

Garg, S. Improved glycemic control in intensively treated adult subjects with type 1 diabetes using insulin guidance software. Farajtabar, M. More robust doubly robust off-policy evaluation. In Proc. of the 35th International Conference on Machine Learning , Vol.

Schrittwieser, J. Mastering Atari, Go, chess and shogi by planning with a learned model. Sun, C. Personalized vital signs control based on continuous action-space reinforcement learning with supervised experience.

Signal Process. Control 69 , McIntosh, C. Clinical integration of machine learning for curative-intent radiation treatment of patients with prostate cancer.

Ngassa Piotie, P. Designing an integrated, nurse-driven and home-based digital intervention to improve insulin management in under-resourced settings. Thomsen, C. Time for using machine learning for dose guidance in titration of people with type 2 diabetes? A systematic review of basal insulin dose guidance.

Diabetes Sci. Srivastava, N. Dropout: a simple way to prevent neural networks from overfitting. Sutton, R. Reinforcement Learning: An Introduction 2nd edn MIT Press, Fox, I.

Reinforcement learning for blood glucose control: challenges and opportunities. Wang, L. Supervised reinforcement learning with recurrent neural network for dynamic treatment recommendation. Zhang, H.

Generative planning for temporally coordinated exploration in reinforcement learning. In 10th International Conference on Learning Representations ICLR IEEE, Wang, X.

Beyond greedy search: tracking by multi-agent reinforcement learning-based beam search. IEEE Trans. Image Process. Song, H. Attend and diagnose: clinical time series analysis using attention models. of the Thirty-Second AAAI Conference on Artificial Intelligence Association for Computing Machinery, Kong, A.

Sequential imputations and Bayesian missing data problems. Martino, L. Effective sample size for importance sampling based on discrepancy measures. Moghissi, E. American Association of Clinical Endocrinologists and American Diabetes Association consensus statement on inpatient glycemic control.

Diabetes Care 32 , — Hirsch, I. A real-world approach to insulin therapy in primary care practice. Diabetes 23 , 78—86 Umpierrez, G. Randomized study of basal-bolus insulin therapy in the inpatient management of patients with type 2 diabetes RABBIT 2 trial.

Diabetes Care 30 , — Randomized study comparing a basal-bolus with a basal plus correction insulin regimen for the hospital management of medical and surgical patients with type 2 diabetes: basal plus trial. Diabetes Care 36 , — Comparison of inpatient insulin regimens with detemir plus aspart versus neutral protamine hagedorn plus regular in medical patients with type 2 diabetes.

Download references. This study was funded by the National Natural Science Foundation of China grants and to X. Li, grant to G. and grant to Y. is also supported by the New Cornerstone Science Foundation through the XPLORER PRIZE. is supported by the Wellcome Trust and National Institute for Health and Care Research Oxford Biomedical Research Centre.

The funders of the study had no role in study design, data collection, data analysis, data interpretation or writing of the report. The authors would like to acknowledge the Nanjing Institute of InforSuperBahn MLOps for providing the training and evaluation platform.

Ministry of Education Key Laboratory of Metabolism and Molecular Medicine, Department of Endocrinology and Metabolism, Zhongshan Hospital, Fudan University, Shanghai, China. State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications, Beijing, China.

Big Data and Artificial Intelligence Center, Zhongshan Hospital, Fudan University, Shanghai, China. Department of Endocrinology, XuHui Central Hospital of Shanghai, Shanghai, China. Department of Endocrinology and Metabolism, Qingpu Branch of Zhongshan Hospital affiliated to Fudan University, Shanghai, China.

Big Data and Biomedical AI Laboratory, College of Future Technology, Peking University, Beijing, China. Nuffield Department of Clinical Neurosciences, University of Oxford, Oxford, UK.

Shanghai Key Laboratory of Metabolic Remodeling and Health, Institute of Metabolism and Integrative Biology, Fudan University, Shanghai, China. You can also search for this author in PubMed Google Scholar. and X. Li collected and analyzed the data. Li conceived and supervised the project.

Li wrote the manuscript. All authors discussed the results and reviewed the manuscript. Correspondence to Guangyu Wang , Xiaoying Li or Ying Chen. Nature Medicine thanks the anonymous reviewers for their contribution to the peer review of this work.

Primary Handling Editors: Lorenzo Righetto and Saheli Sadanand, in collaboration with the Nature Medicine team. a , The sequential decision process with patient model and policy model in AI system. Given a trajectory, for the initial step, the representation function f R receives as input the past observations O 1: t from the trajectory.

The model is subsequently unrolled recurrently for K steps. The hidden states and actions recurrently update. b , The AI system training pipeline. Left, the patient model learning for patient tracking.

Right, the policy update for dynamic regimen with combined supervised learning and reinforcement learning. a , Visualization of the patient hidden states. Each node indicates a patient state. The state distribution showed association with diabetic outcome, colored by glucose level distribution.

The samples are patients from internal test dataset. PC: principal component. b , Illustration of reward function. It is a measurement of overall glucose variability that focus on the relationship between glucose variability and risks for hypo- and hyperglycemia.

c and d , Performance of the AI model on assessment of WTR shown as AUC curves. c , internal test set and d , external test set. ROC curves showing the pre-prandial time, the postprandial and overall performance. e , internal test set, and f , external test set.

a, c and e : the internal test set; b, d and f : external test set. g , Off-policy evaluation of RL-based model versus other SL-based and clinician methods in the internal test set in the AI development phase, measured by weighted importance sampling WIS score with standard deviation. a , Study design of internal cohort: 40 eligible T2D patients were included in the study.

b , Study design of external cohort: 45 T2D patients were collected, and a total of insulin points were included in the external validation analysis. An assessment with quantitative metrics was conducted to compare the performance between treating physicians and AI by expert panel.

After 2 weeks, a retest review was conducted. BMI, body mass index; A1c, glycated hemoglobin. Numerical variables were reported as mean±SD. Quantitative comparisons of insulin dosage given by human physicians and AI stratified by insulin catalogs in the external cohort.

Orange dashed line, average performance of AI; blue dashed line, average performance of treating physicians. Bar graphs indicate the mean±SEM. a , The user interface of AI deployment. a , Patient example during the proof-of-concept feasibility trial using the seven-point capillary blood glucose measurement.

b , Glucose control based on the sensor glucose measurements at the first 24 hours and the last 24 hours of the trial. GMI, glucose management indicator; CV, coefficient of variation. c , Post-intervention evaluation by physicians who used the AI during the feasibility trial.

Open Access This article is licensed under a Creative Commons Attribution 4. Reprints and permissions. Optimized glycemic control of type 2 diabetes with reinforcement learning: a proof-of-concept trial.

Nat Med 29 , — Download citation. Received : 31 January Accepted : 17 August Published : 14 September Issue Date : October Anyone you share the following link with will be able to read this content:. Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative. Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Skip to main content Thank you for visiting nature. nature nature medicine articles article. Download PDF. Subjects Computational biology and bioinformatics Therapeutics. Abstract The personalized titration and optimization of insulin regimens for treatment of type 2 diabetes T2D are resource-demanding healthcare tasks.

Main Type 2 diabetes T2D is one of the most prevalent chronic diseases and leads to a considerable rate of death and social burden worldwide 1. Full size image. Results Dataset characteristics and system overview A total of 12, inpatients with T2D with , treatment days were included in the AI model development phase analysis.

Performance of AI model to predict patient glycemic states To build a dynamic and individualized AI clinician for managing patients with T2D, we constructed the model-based RL framework. Discussion In this study, we developed an RL-based AI system, called RL-DITR, for personalized and dynamic insulin dosing for patients with T2D.

Methods Study design and participants To train and validate a computational clinical decision support model, we constructed a large multi-center dataset using EHRs of hospitalized patients with T2D who received insulin therapy from January to April in the Department of Endocrinology and Metabolism, Zhongshan Hospital and Qingpu Hospital, in Shanghai, China.

Development and validation of the model-based RL system Time-series data pre-process and NLP For time-series data representation, every patient in the dataset was represented as a temporal sequence of feature vectors. Building the computational model The process of patient trajectory and treatment decision-making could be formulated as a Markov decision process MDP.

Patient model for trajectory tracking For patient trajectory tracking, we trained a patient model. Data availability IRB approval was obtained from institutions for EHR data collection.

Code availability The deep learning models were developed and deployed in Python version 3. The simulated glucose and nitrate levels exhibited damped oscillations when using a PID controller, a common response for this controller type yellow lines in Fig.

In contrast, employing the GMPC controller eliminated the damping issues and enabled the glucose and nitrate level to more rapidly reach values near the target levels yellow lines in Fig. Meanwhile, the glucose supply and nitrate supply increased gradually in the GMPC controller red lines in Fig.

Instead, the amount added red lines in Fig. a Simulink model for model predictive control and PID control. Glucose control: b PID controller.

c GMPC controller. Nitrate control: d PID controller. e GMPC controller. The PID controller gains were tuned on Matlab TM to achieve optimal performance with proportional gain K p , integral gain K i and derivative gain K d equal to 1.

The glucose levels were measured every hour and the data was fed to both the PID controller and the closed-loop GMPC controller with a pure heterotrophic model since light and dark cycles were not presented. The feedback signal could compensate for the modeling errors and also help to reject the disturbance in the GMPC controller.

After the setpoint change, the glucose level gradually decreased and was stably controlled. Overall, both Simulink TM simulations and experimental results demonstrated that the GMPC approach provided more robust and precise control than traditional PID controllers.

While the model could anticipate the future behavior of the fermentation and take appropriate control action, the PID controller did not have this capability resulting in oscillations and overshoot behavior in both simulations and experiments.

Thus, our study demonstrates how GMPC systems can serve as a bridge between genome-scale metabolic modeling and control algorithms. Since the cultivation conditions can change and affect algal cellular metabolism, our system connected feedback measurements with genome-scale metabolic models and achieved more efficient nutrient utilization and higher product yields for dynamic algal cultivation conditions.

In this way, genome-scale metabolic models can be effectively utilized to improve biomanufacturing of microalgae and other industrially important microbial cell factories. Fed-batch cultivation and PID controllers have been widely used in bioprocess development.

Unfortunately, fed-batch cultivation often results in poor nutrient control and wasted nutrients and conventional PID control can lead to oscillating cell behaviors and poor performance under dynamic conditions. In this study, we have utilized the power of genome-scale metabolic models to predict and control glucose and nitrate supply for C.

vulgaris cultures under light and dark cycles and compared this approach to conventional autotrophic and heterotrophic processes. Our results first showed that utilizing genome scale models to track and limit glucose and nitrate feeding led to higher titers of biomass, FAs, and lutein than autotrophic conditions and more efficient glucose utilization and higher product yields than heterotrophic conditions.

Next, implementing these models into an open loop system modestly improved performance. Finally, both computational simulations and experimental results demonstrated that this genome-based MPC system exhibits superior controller performance compared to conventional PID methods.

Green microalgae C. vulgaris UTEX was obtained from the Culture Collection of Algae at the University of Texas at Austin and maintained on sterile agar plates 1.

Liquid cultures were inoculated with a single colony in For alternating light and dark cycles, autotrophic conditions were used for light sections and heterotrophic conditions were used for dark sections.

The lyophilized algal dry biomass was weighted gravimetrically using an analytical balance. The glucose concentration was measured using YSI biochemistry analyzer Yellow Springs, OH. FAME production followed the procedure provided by Dong et al. Helium was used as carrier gas.

Lutein extraction followed the procedure provided by Yuan et al. The solution was filtered before HPLC analysis. The mobile phases are eluent A dichloromethane: methanol: acetonitrile: water, 5.

The i CZ model, including six different biomass compositions for autotrophic conditions PAT1-PAT6 and five different biomass compositions for heterotrophic conditions HT1-HT5 , was obtained from Zuniga et al.

GSM simulations were performed using the Gurobi Optimizer Version 5. The experimental setup is shown in Supplementary Fig. The manipulated variables were glucose demand F G and nitrate demand on a per L basis F N for 8-h period.

Two pumps were used to control both variables automatically by Matlab TM through Arduino chip. All the control algorithms were run on Matlab TM and the codes are provided in Supplementary information.

The Simulink TM simulation is shown in Fig. The blue box in Fig. Four equations were built inside the blue box as shown in Supplementary Fig. The inputs were F G and F N. The outputs were biomass, nitrate level, glucose level and volume.

Only nitrate levels and glucose levels were fed into the PID and GMPC controller. For the proportional-integral-derivative PID controller, the proportional gain K p , integral gain K i and derivative gain K d equal to 1.

The PID controller and GMPC controller were used to control glucose supply and nitrate supply every hour in both simulation and experiment. Changes in the setpoint for glucose were introduced to see how both PID and GMPC responded to those changes. Initial biomass levels x 0 , glucose levels G 0 and nitrate levels N 0 were measured as described above and used as inputs into the open-loop system.

Three equations shown below were used to predict biomass growth, nitrate consumption rate, and glucose consumption rate in the open-loop system. The growth rates under light and dark cycles were determined based on previous experimental data. After that, the growth rates were constrained in the autotrophic and heterotrophic GSMs, respectively to determine nutrient exchange rates r N and r G under light and dark cycles.

The methods for using growth rate to estimate nutrient exchange rates have been described previously in Chen et al. We assumed a rapid switch to a new operational steady state following the transition between light and dark cycles. Initial biomass levels x 0 , glucose levels G 0 and nitrate levels N 0 were measured and used as inputs into the closed-loop system.

During the experiment, biomass levels x m , glucose levels G m and nitrate levels N m were msured and used as inputs into the closed-loop system. For the light cycle, two equations were built to describe and predict biomass accumulation rate and nitrate consumption rate.

Unlike the open loop system, the light shielding effect was considered and the growth rate would decrease as the biomass concentration increased as described in the equation below and shown in Fig. The GSM was used to predict nutrient exchange rate r N based on the measured growth rate.

For the dark cycles, three model equations were built to predict biomass accumulation rate, nitrate consumption rate and glucose consumption rate as listed below and shown in Fig.

In the biomass equation, we assumed a fraction of heterotrophic biomass, a , was derived from autotrophic metabolism and the simulated growth rate was μ A. Meanwhile, some biomass was derived through heterotrophic metabolism with the simulated growth rate, μ H.

The nutrient exchange rates r NA , r NH , r GH were determined by inputting simulated growth rates into the autotrophic and heterotrophic GSMs respectively. where μ A is simulation growth rate from autotrophic metabolism, μ H is the growth rate from heterotrophic metabolism, r NA is nitrate exchange rate from autotrophic metabolism, r NH is the nitrate exchange rate from heterotrophic metabolism, r GH is the glucose exchange rate from heterotrophic metabolism.

Next, we applied a fitting objective function J to minimize the difference between calculated values and simulated model values in order to estimate the optimal parameter values a , μ A , μ H , r NA , r NH , r GH for dictating the actual nitrate and glucose feeds to the bioreactor.

The actual bolus nitrate demand F N and the glucose demand F G were thus determined by using values obtained from this fitting objective function. The data that support the findings of this study are available from the corresponding author upon reasonable request.

Rosenberg, J. A green light for engineered algae: redirecting metabolism to fuel a biotechnology revolution. Article CAS PubMed Google Scholar. Shene, C. Metabolic modelling and simulation of the light and dark metabolism of Chlamydomonas reinhardtii.

Plant J. Kato, Y. et al. Biofuels 12 , 39 Article PubMed PubMed Central Google Scholar. Cheirsilp, B. Enhanced growth and lipid production of microalgae under mixotrophic culture condition: effect of light intensity, glucose concentration and fed-batch cultivation.

Zheng, Y. High-density fed-batch culture of a thermotolerant microalga Chlorella sorokiniana for biofuel production. Energy , — Article CAS Google Scholar.

Shi, X. High-yield production of lutein by the green microalga Chlorella protothecoides in heterotrophic fed-batch culture. Bordbar, A. Constraint-based models predict metabolic and associated cellular functions. Chang, R. Article Google Scholar.

Zuñiga, C. Genome-scale metabolic model for the green alga Chlorella vulgaris utex accurately predicts phenotypes under autotrophic, heterotrophic, and mixotrophic growth conditions. Plant Physiol. Loira, N. Reconstruction of the microalga Nannochloropsis salina genome-scale metabolic model with applications to lipid production.

BMC Syst. Chang, L. Nonlinear model predictive control of fed-batch fermentations using dynamic flux balance models. Process Control 42 , — Jabarivelisdeh, B. Model predictive control of a fed-batch bioreactor based on dynamic metabolic-genetic network models.

IFAC-PapersOnLine 51 , 34—37 Juneja, A. Model predictive control coupled with economic and environmental constraints for optimum algal production. Ogbonna, J. Zuniga, C. Predicting dynamic metabolic demands in the photosynthetic eukaryote Chlorella vulgaris. Li, C. Utilizing genome-scale models to optimize nutrient supply for sustained algal growth and lipid productivity.

NPJ Syst. Tebbani, S. Nonlinear predictive control for maximization of CO2 bio-fixation by microalgae in a photobioreactor. Bioprocess Biosyst. Hu, D. The design and optimization for light-algae bioreactor controller based on Artificial Neural Network-Model Predictive Control.

Acta Astronaut 63 , — White, R. Long-term cultivation of algae in open-raceway ponds: lessons from the field. Gu, C. Current status and applications of genome-scale metabolic models. Genome Biol.

Using genome-scale models to predict biological capabilities. Cell , — Colarusso, A. Computational modeling of metabolism in microbial communities on a genome-scale. Han, F. Enhancement of microalgal biomass and lipid productivities by a model of photoautotrophic culture with heterotrophic cells as seed.

Xiong, W. Double CO 2 fixation in photosynthesis—fermentation model enhances algal lipid synthesis for biodiesel production. Xiao, Y. Photosynthetic accumulation of lutein in Auxenochlorella protothecoides after heterotrophic growth.

Drugs 16 , Vidotti, A. Analysis of autotrophic, mixotrophic and heterotrophic phenotypes in the microalgae Chlorella vulgaris using time-resolved proteomics and transcriptomics approaches.

Algal Res. Park, J. The contribution ratio of autotrophic and heterotrophic metabolism during a mixotrophic culture of Chlorella sorokiniana. Public Health 18 , Huesemann, M. A validated model to predict microalgae growth in outdoor pond cultures subjected to fluctuating light intensities and water temperatures.

Mears, L. A review of control strategies for manipulating the feed rate in fed-batch fermentation processes. Sommeregger, W. Quality by control: Towards model predictive control of mammalian cell culture bioprocesses.

Chen, G. Viable cell density on-line auto-control in perfusion cell culture aided by in-situ Raman spectroscopy. Lee, H. In situ bioprocess monitoring of Escherichia coli bioreactions using Raman spectroscopy.

Xu, J. Application of metabolic controls for the maximization of lipid production in semicontinuous fermentation.

Natl Acad. USA , E—E Article CAS PubMed PubMed Central Google Scholar. Dong, T. Direct quantification of fatty acids in wet microalgal and yeast biomass via a rapid in situ fatty acid methyl ester derivatization approach.

Yuan, J. Carotenoid composition in the green microalga Chlorococcum. Food Chem. Chen, Y. An unconventional uptake rate objective function approach enhances applicability of genome-scale models for mammalian cells.

Download references. This work was supported by the U. National Science Foundation EFRI program Grant number: and CBET program Grant number: and the Department of Energy Grant number: DE-SC Department of Chemical and Biomolecular Engineering, Johns Hopkins University, North Charles Street, Baltimore, MD, , USA.

Department of Pediatrics, University of California, San Diego, Gilman Drive, La Jolla, CA, , USA. Department of Biology, San Diego State University, San Diego, USA. Department of Bioengineering, University of California, San Diego, Gilman Drive, La Jolla, CA, , USA.

Center for Microbiome Innovation, University of California, San Diego, Gilman Drive, La Jolla, CA, , USA. You can also search for this author in PubMed Google Scholar.

contributed to conception and design of the experiment. conducted the experiments. analyzed the data. drafted the paper. All authors read and approved the paper.

Correspondence to Michael J. Open Access This article is licensed under a Creative Commons Attribution 4. Reprints and permissions. Li, CT. Optimization of nutrient utilization efficiency and productivity for algal cultures under light and dark cycles using genome-scale model process control.

npj Syst Biol Appl 9 , 7 Download citation. Received : 27 September Accepted : 08 November Published : 15 March Anyone you share the following link with will be able to read this content:. Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative. Sign up for the Nature Briefing: Translational Research newsletter — top stories in biotechnology, drug discovery and pharma.

Skip to main content Thank you for visiting nature. nature npj systems biology and applications articles article. Download PDF. Subjects Computer modelling Metabolic engineering Plant sciences. Abstract Algal cultivations are strongly influenced by light and dark cycles.

Introduction Microalgae represent promising microorganisms for transforming renewable resources and inorganic carbon sources into biomass, biofuel precursors, and high-value products 1.

Results and discussion Advantages of using genome-scale model predictions on C. Full size image. a PID controller.

The application of a numerical filter to remove noise in the data resulted in values for regional cerebral glucose utilization that were stable with time and consistent with rates determined by the other established techniques. Based on the results of the current study, we expect that the spectral analysis technique will prove to be a highly flexible tool for kinetic analysis of other tracer compounds; it is capable of producing low-variance, time-stable estimates of physiological parameters when optimized for time interval of application, input spectrum of components, and processing of noise in the data.

Abstract A method for kinetic analysis of dynamic positron emission tomography PET data by linear programming that allows identification of the components of a measured PET signal without predefining a compartmental model has recently been proposed by Cunningham and co-workers. Publication types Research Support, Non-U.

Substances Fluorine Radioisotopes Fluorodeoxyglucose F18 Deoxyglucose Glucose. Figure 3c,d shows the dynamic treatment strategies generated by clinicians and model-based RL for two individual patients on different hospital days.

We further investigated whether the patient outcome WTR ratio varied with the difference of the dose actually administered and the dose suggested by the RL method by correlation analysis Fig. When the dose actually administered differed from the dose suggested by the AI algorithm, the average outcome got worse.

For the internal validation cohort, we compared the performance between our AI system and human physicians in giving insulin dosage recommendation using 40 patients with T2D with insulin data points Extended Data Fig.

RL-generated and physician-generated dosage titrations were evaluated by an expert panel, including quantitative metrics and qualitative metrics from clinical experience.

Taking the dosage recommended by the expert panel as references, the MAE of the AI system was 1. Evaluation was based on the expert panel review including effectiveness f , safety g and overall acceptability h. Orange dashed line represents the average performance of AI; blue dashed line represents the average performance of treating physicians.

G, group. These results suggest that our AI model is superior to junior physicians and similar to experienced physicians in the overall treatment regimen acceptability, hyperglycemia and hypoglycemia control.

Furthermore, we performed an external validation in 45 patients with T2D to compare the performance of AI plans and treating physician plans under a blinded review by an expert panel and by another blinded review for retesting at 2-week intervals at least Extended Data Fig.

The results demonstrated that the acceptability, effectiveness and safety of the AI plans were similar to the treating physicians who were board-certified endocrinologists, evaluated by subjective measurements made by an expert panel Fig. The percentage of selected superior AI plans was These results demonstrate consistently superior performance of the AI model compared to its physician counterparts.

We used adoption rate to evaluate the percentage of the AI regimens adopted by endocrinologists for patient treatment. Our proposed RL model demonstrated stable performance of effectiveness, safety and acceptability over time, even better in the retest review Fig. The score scale of effectiveness and safety is 1—5.

The adoption rate refers to the percentage of the AI regimens adopted by endocrinologists at the bedside for patient treatment. Intriguingly, a higher adoption rate of Although the adoption rate of the AI plan was relatively low at the initial test review, we found an increase of These results suggested a step-by-step increase of trust of the AI treatment regimen by physicians through human—machine interaction, and the AI system was gradually adopted by physicians into routine clinical practice.

A proof-of-concept feasibility trial was performed to investigate the clinical utility and safety of AI in hospitalized patients with T2D for glycemic control.

Sixteen inpatients with T2D were enrolled in the trial Extended Data Fig. Their mean HbA1c was 8. Over the trial, b , The capillary blood glucose of a patient with T2D during the treatment period. II Mean daily capillary blood glucose. III Mean preprandial capillary blood glucose.

IV Mean postprandial capillary blood glucose during the treatment period. The preprandial blood glucose target was 5. c , Average percentage of continuous glucose monitoring data within glycemic ranges throughout the treatment period. The satisfaction agreement was scored from a scale of 1—5.

IQR, interquartile range. of At the end of the trial, A patient example of the seven-point capillary blood glucose during the AI intervention is shown in Extended Data Fig. We also used continuous glucose monitoring CGM for the evaluation of the algorithm-directed glycemic control for the secondary outcomes.

The percentage of glucose concentration in time in range TIR 3. TIR 3. Time spent above Time spent below 3. In addition, glycemic variability was slightly decreased during the treatment period coefficient of variation CV of No episodes of severe hypoglycemia that is, requiring clinical intervention or hyperglycemia with ketosis occurred during the trial.

Most physicians stated that the AI interface is understandable 4. In this study, we developed an RL-based AI system, called RL-DITR, for personalized and dynamic insulin dosing for patients with T2D. We performed development phase validation and clinical validations, including internal validation, comparing AI to physicians using quantitative and qualitative metrics, external validation with test—retest, prospective deployment with test—retest and a proof-of-concept feasibility study with clinical trial.

Taken together, our findings demonstrate that our RL-DITR system has potential as a feasible approach for the optimized management of glycemic control in inpatients with T2D.

The management of blood glucose in diabetes remains challenging due to the complexity of human metabolism, which calls for the development of more adaptive and dynamic algorithms for blood glucose regulation. To address the challenge of personalized insulin titration algorithm for glycemic control, our RL-based architecture is tailored to achieve precise treatment for individual patients, with clinical supervision.

Our proposed patient model-based RL model can make multi-step planning to improve prescription consistency. In addition, because the multi-step plan can be interpreted as the intent of the model from now to a span of time period into the future, it offers a more informative and intuitive signal for interpretation Additionally, our RL-based system delivers continuous and real-time insulin dosage recommendation for patients with T2D who are receiving subcutaneous insulin injection, combining optimal policies for clinical decision-making and the mimicking of experienced physicians Another strength of our study is that we conducted a comprehensive early clinical validation of the AI-based clinical decision-making system across various clinical scenarios.

In clinical deployment, our AI framework offers potential benefits, including automated reading of a large number of inputs from the EHRs, integration of complex data and accessible insulin dosing interface. Although some algorithms have been developed to assist physicians in insulin titration, only a few have been validated in clinical trials 31 , We conducted a proof-of-concept feasibility trial demonstrating the viability of the RL-DITR system in inpatients with T2D.

Notably, the use of the RL-DITR system resulted in a considerable improvement in blood glucose control, meeting our pre-determined feasibility goal. The percentage of well-controlled blood glucose levels of TIR also demonstrated a substantial increase.

Managing hypoglycemia risk is a key consideration for real-world deployment of the AI system. While achieving improved control of blood glucose levels, the system did not increase the risk of hypoglycemia.

Additionally, physicians using the RL-DITR system have reported an increased level of satisfaction, including aspects such as efficiency in clinical practice and perceived effectiveness and safety in glycemic control.

These results suggest that our RL-DITR system has the potential to offer feasible insulin dosing to inpatients with T2D. A large and multi-center randomized controlled trial would help to determine the efficacy and benefits of this clinical AI solution.

Our RL-DITR system was designed as a closed-loop intelligent tool that could use real-time patient data to track blood glucose trajectories and modify treatment regimens accordingly. Furthermore, the RL-DITR system was developed using EHRs of inpatients with T2D, but its generalizability to other populations, such as outpatients, needs further investigation.

We conducted simulated experiments using Gaussian noise to mimic low data quality and dropout 33 to simulate missing data scenarios before deployment Supplementary Fig. Therefore, although the RL-DITR workflow was implemented and tested for inpatients with T2D, there exists the possibility to extend its application to a wider range of healthcare settings, such as outpatient management, given appropriate integration and continued development.

Although our RL-DITR system has achieved good performance in insulin dosage titration, some challenges remain. The generalization of the AI to other ethnicities needs to be further investigated. Second, the variety of diet during the hospitalized period was uniformly supplied in the EHRs to build our model.

For patients out of hospital, dietary variation and physical activity should be taken into account and explored by our RL model. We have opened an interface to accumulate dietary information for late updated model.

In conclusion, we developed an RL-based clinical decision-making system for dynamic recommendation of dosing that demonstrated feasibility for glycemic control in patients with T2D. The RL-DITR system is a model-based RL architecture that could enable multi-step planning for patients with long-term care.

With the integration of RL structure and supervised knowledge, the RL-DITR system could learn the optimal policy based on non-optimized data while retaining the safe states by balancing exploitation and exploration.

Furthermore, we performed a stepwise validation of the AI system from simulation to deployment and a proof-of-concept feasibility trial. These demonstrate the RL approaches as a potential tool to assist clinicians, especially junior physicians and non-endocrine specialists, with diabetes management in hospitalized patients with T2D.

To train and validate a computational clinical decision support model, we constructed a large multi-center dataset using EHRs of hospitalized patients with T2D who received insulin therapy from January to April in the Department of Endocrinology and Metabolism, Zhongshan Hospital and Qingpu Hospital, in Shanghai, China.

The demographics and clinical characteristics of patients are presented in Extended Data Table 1 , demonstrating a typical T2D population. We conducted stepwise studies to evaluate the performance of our RL-DITR model version 1. In addition, we performed a proof-of-concept feasibility trial of the RL-DITR system in clinical practice with inpatients with T2D who were admitted for optimization of glycemic control at Zhongshan Hospital ClinicalTrials.

gov: NCT details of proof-of-concept trial protocol provided in Supplementary Information. The retrospective study obtained the following institutional review board IRB approval: Zhongshan Hospital, Shanghai, China R ; XuHui Central Hospital, Shanghai, China and Qingpu Branch of Zhongshan Hospital, Shanghai, China Patient informed consent was waived by the Ethics Committee.

The prospective study and proof-of-concept feasibility trial were approved by the Ethics Committee of Zhongshan Hospital, Fudan University.

Each participant provided written informed consent for the prospective study and the proof-of-concept feasibility trial. For time-series data representation, every patient in the dataset was represented as a temporal sequence of feature vectors. Specifically, each day was broken into seven time periods, including pre-breakfast, post-breakfast, pre-lunch, post-lunch, pre-dinner, post-dinner and pre-bedtime.

All records that occurred within the same period were grouped together and formed a feature set to feed into the RL model as input detailed list of the input features provided in Supplementary Table 2. For structured data, we aligned and normalized them. For free-text notes, we applied a pre-trained language model, ClinicalBERT.

Specifically, we first trained the ClinicalBERT on a large corpus of EHR data. ClinicalBERT is a masked medical domain language model that predicts randomly masked words in a sequence and, hence, can be transformed into downstream tasks.

Then, the ClinicalBERT was fine-tuned for information extraction from free text. We further automatically extracted temporal features from patient clinical records, including clinical observations blood glucose records , a sequence of decision rules to determine the course of actions for example, treatment type and insulin dosage titration and clinical assessment of patients.

The numerical values were extracted from demographics, laboratory reports, blood glucose and medications and further translated with standard units according to the LOINC database. Then, each numerical value was normalized to a standard normal distribution.

In terms of discrete values, all the diagnoses of a patient were mapped onto the International Classification of Diseases-9 ICD-9 and used as discrete features, encoded as binary presence features. We constructed a large multi-center dataset with a large corpus of 1. ClinicalBERT was fine-tuned on a multi-label dataset to extract 40 symptom labels from medical notes.

Phenotype data were extracted from free-text notes of chief history of present illness and physical examination by ClinicalBERT. Validated on 1, annotated samples from the training set, the results showed that ClinicalBERT could accurately identify the symptom information with an average F1 score of Each extracted symptom label was encoded as a binary presence feature.

The process of patient trajectory and treatment decision-making could be formulated as a Markov decision process MDP. An MDP 34 is a tuple S , A , P , G , γ , where S and A are sets containing the states and actions, respectively; P is a transition function; G is a reward function; and γ is a discount factor.

The patient model was learned from historical trajectories, approximating the transition function P and the reward function G and providing support for policy model learning and planning. The policy model iteratively interacted with the patient model as an environment.

At each step, the patient model generated state transition, status prediction and reward estimation based on observed patient trajectories. The policy model, taking the state as input, generated an action that was fed to the patient model.

The patient model updated the states recurrently by an iterative process, enabling the policy model to plan for sequences of actions and find optimal solutions across generated trajectories. The hidden state would be used as input for patient model and policy model.

For patient trajectory tracking, we trained a patient model. When conducting correlation analysis with daily outcome, Magni risk values were summed for each day.

Both of the dynamics function f T and the prediction function f P shared the representation encoder f R when training and inference. f R was optimized together through backpropagation with the loss to capture meaningful patient representations and dynamics.

Each node indicates the states of a patient. The state distribution demonstrated a good cluster hierarchy that individuals in the same cluster are associated with their observable properties diabetes outcome, such as glucose level.

We combined the SL and RL to learn the policy model, with the expert supervision of safe actions to take into account. Specifically, we applied policy gradient optimization for training the policy model π to maximize the returned rewards while incorporating constrained supervision by expert experience.

For the SL part, we used the action made by the clinicians as supervision for policy update. For the RL part, we optimized the policy model π based on the patient model f T , f P as an interactive environment, where a given trajectory was updated recurrently by an iterative process.

The policy model π was trained by both historical and obtained trajectories. We applied a beam search for policy search The top B highest-value trajectories were stored at each timestep, where B was the beam size.

The training process involved two stages to optimize the models of our AI system Extended Data Fig. These functions were jointly optimized through the loss for state transitions and the loss for status prediction. The policy model was trained through a joint optimization process, minimizing both a policy gradient loss on trajectories and a supervised loss that constrains the difference between the recommended action from the policy model and the action taken by the clinician.

We used a transformer-based network with three layers as the representation function used to represent the observations of time-series data, as it has been shown to enable capturing the long dependence in the temporal information of patients The last hidden vector of the output hidden vectors was used for the initial state.

We also applied a transformer network with three layers for dynamics function. The hidden dimension was set to , and the number of multi-attention heads was set to 8.

We used three-layer multi-layer perceptrons MLPs for prediction function, policy function and value function. The hidden dimension was set to The beam size B was set to The models were implemented using PyTorch.

The importance sampling for policy evaluation was performed, which enables the evaluation of a target policy using data collected from a distinct policy To enhance the numerical stability of the calculations, we employed WIS along with effective sample size 40 , 41 , which normalizes the trajectories, thereby reducing variance Given specific data conditions, such as no more than k blood glucose measurements per day, we randomly discard blood glucose values within the trajectories to ensure that the remaining trajectories satisfy this criterion.

The traditional clinical methods of insulin dosage titration were used as the standard clinical methods for comparison, consisting of guidelines 42 and consensus formulas 43 , 44 for premixed insulin regimen, basal regimen and basal-bolus regimen.

The detailed adjustment was according to the following formula:. The insulin dosage titration rules of basal-bolus regimen were as follows. Retrospective study phase of the internal cohort.

Forty eligible patients with T2D treated with insulin injection were randomly selected from the retrospective EHRs of one of the modeling development hospitals Qingpu Hospital from May to December Two treatment days were randomly selected for each patient, resulting in 80 cases with insulin points Extended Data Fig.

Three physician groups with different levels of clinical experience provided their dose recommendations, and the AI also generated insulin dose recommendations in silico for further evaluation. An expert consensus panel of three endocrinology specialists conducted blinded review and provided their own recommended insulin dosage.

This was used as a reference insulin dosage for each insulin point to assess the accuracy of AI-generated dosage versus the three physician groups. Retrospective study phase of the external cohort. The retrospective dataset was collected from a non-teaching hospital XuHui Hospital , which included 45 eligible consecutive patients with T2D from April to August Extended Data Fig.

The dataset contained insulin points from cases, and AI-generated dosage was compared to previously delivered insulin dosage by treating physicians human plan for accuracy evaluation.

Next, we randomly selected 40 cases from the dataset to evaluate the acceptability, effectiveness and safety of the AI plan and the previous human plan. The evaluations were blinded head-to-head comparisons of AI versus human plans by the expert consensus with three independent experts.

Prospective deployment study phase. In May , 40 consecutive AI-generated plans were tested for acceptance, effectiveness and safety by endocrinology physicians at the bedside Extended Data Fig. After determining clinical adoption and ensuring adherence to standard clinical quality controls, the AI insulin regimen was used for patient treatment.

The inclusion and exclusion criteria for patients were consistent across the three phases. Inclusion criteria were patients with T2D treated with subcutaneous insulin injection for at least two consecutive days.

Patients with acute complications of diabetes, such as ketoacidosis or hyperglycemic hyperosmolar state, or patients who were treated with glucocorticoids, were excluded.

Quantitative evaluation. We used the metrics of MAE and agreement percentage to quantitatively evaluate the accuracy performance of insulin regimens. MAE represents the errors between predicted values and consensus values.

Effectiveness and safety. The effectiveness was scaled on a five-point Likert scale ranging from 1 very poor control of glycemia to 5 very good control of glycemia. For safety evaluation, we used questionnaires 2 and 3 item 5 and questionnaire 4 item 6 , which asked the reviewers if the recommended insulin regimen was perceived to lead to an increased risk of hypoglycemia according to their judgment.

The safety was then scaled on a five-point Likert scale ranging from 1 very high risk to 5 very low risk Supplementary Information. Superior plan. In the head-to-head comparison of the AI and human plans in the retrospective simulation study of the external cohort, the one AI or human selected as most clinically appropriate by the expert consensus review was considered as the superior plan Supplementary Information.

In the prospective deployment phase, the AI plans were reviewed by the endocrinology physicians at the bedside; the clinical adoption was determined; and the deemed AI insulin regimen was used for patient treatment following all standard clinical quality controls. We conducted a proof-of-concept trial ClinicalTrials.

gov: NCT to evaluate the feasibility and safety of AI in inpatients with T2D from 28 June to 6 October This trial was a patient-blinded and single-arm intervention, which was performed in the ward of the Department of Endocrinology and Metabolism, Zhongshan Hospital, in China.

The RL-DITR system was embedded in the insulin dosing interface of the health information system HIS , allowing real-time reading of patient clinical information and insulin dosage regimen recommendation Extended Data Fig.

An example of the AI recommendation report for the healthcare provider is presented in Extended Data Fig. Patients with T2D receiving subcutaneous insulin treatment were recruited and screened for the inclusion and exclusion criteria.

The pre-intervention initial insulin regimen served as reference for daily insulin regimen. Eligible patients received insulin dosage titration according to the AI model after the first cycle of insulin regimen, which was confirmed twice daily by the physician in charge.

The treating physician could reject the recommendation if deemed necessary. Throughout the trial, anti-hyperglycemic drugs remained unchanged; standard meals at usual mealtime were provided; and no physical activity was scheduled.

Capillary glucose concentration was measured at seven timepoints of fasting, after breakfast, before and after lunch, before and after dinner and before bedtime a day by a glucometer Glupad, Sinomedisite to estimate glucose control and to guide insulin regimen. The goal was to achieve preprandial capillary blood glucose of 5.

The CGM data were analyzed by physicians, and the treatment was not influenced by data gained by CGM. CGM alarms were not activated during the feasibility clinical trial.

The primary outcome was difference in glycemic control as measured by mean daily blood glucose concentration total, preprandial and post-prandial capillary blood glucose. The secondary endpoints included glucose concentration in the target range TIR of 3. Glycemic variability was determined by the CV of glucose values.

Safety was assessed as the number of hypoglycemic events. Serious adverse events included severe hypoglycemia, defined as a capillary glucose level of less than 2. The sample size calculation was based on the primary outcome.

PASS software version Clinical studies were analyzed using SAS 9. The matched t -test was used to compare the performance of RL-DITR and physicians.

The change from baseline measurements to the end of the trial was analyzed by two-sided paired t -test and a Wilcoxon signed-rank test for continuous measurements.

The seven-point blood glucose profiles were analyzed using a generalized linear mixed model. The model used a Noisy-OR approach to aggregate WTR predicted probabilities of points for daily WTR prediction.

Further information on research design is available in the Nature Portfolio Reporting Summary linked to this article. IRB approval was obtained from institutions for EHR data collection.

Individual-level patient records can be accessible with IRB consent and are not publicly available. De-identified data can be requested by contacting the corresponding authors. All data access requests will be reviewed and if successful granted by the Data Access Committee.

Data can be shared only for non-commercial academic purposes and will require a formal material transfer agreement. Individual-level data of the clinical trial ClinicalTrials. gov: NCT reported in this study are not publicly shared. Data can be available to bona fide researchers for non-commercial academic purposes and necessitate a data user agreement.

Requests should be submitted by emailing the corresponding authors Y. at chen. ying4 zs-hospital. cn or guangyu. wang24 gmail. Requests will be processed within a 2-week timeframe.

All data shared will be de-identified. The deep learning models were developed and deployed in Python version 3. The following standard model libraries were used: scikit-learn 1. Sun, H. et al. IDF Diabetes Atlas: global, regional and country-level diabetes prevalence estimates for and projections for Diabetes Res.

Article PubMed Google Scholar. Stratton, I. Association of glycaemia with macrovascular and microvascular complications of type 2 diabetes UKPDS 35 : prospective observational study.

BMJ , — Article CAS PubMed PubMed Central Google Scholar. Holman, R. Article CAS PubMed Google Scholar. ElSayed, N. Pharmacologic approaches to glycemic treatment: standards of care in diabetes— Diabetes Care 46 , S—S American Diabetes Association. Glycemic targets: standards of medical care in diabetes— Diabetes Care 44 , S73—S84 Article Google Scholar.

Martinez, M. Glycemic variability and cardiovascular disease in patients with type 2 diabetes. BMJ Open Diabetes Res. Care 9 , e Article PubMed PubMed Central Google Scholar.

Rodbard, D. Glycemic variability: measurement and utility in clinical medicine and research—one viewpoint. Diabetes Technol. Bi, W. Artificial intelligence in cancer imaging: clinical challenges and applications. CA Cancer J. Esteva, A.

Dermatologist-level classification of skin cancer with deep neural networks. Nature , — Kermany, D. Identifying medical diagnoses and treatable diseases by image-based deep learning.

Cell , — Wang, G. A deep-learning pipeline for the diagnosis and discrimination of viral, non-viral and COVID pneumonia from chest X-ray images.

Zhang, K. Deep-learning models for the detection and incidence prediction of chronic kidney disease and type 2 diabetes from retinal fundus images. Kaelbling, L. Reinforcement learning: a survey. Gottesman, O. Guidelines for reinforcement learning in healthcare. Komorowski, M. The Artificial Intelligence Clinician learns optimal treatment strategies for sepsis in intensive care.

Guo, H. Learning dynamic treatment strategies for coronary heart diseases by artificial intelligence: real-world data-driven study. BMC Med. Bothe, M. The use of reinforcement learning algorithms to meet the challenges of an artificial pancreas.

Expert Rev. Devices 10 , — Liu, Z. A deep reinforcement learning approach for type 2 diabetes mellitus treatment. Oh, S. Reinforcement learning-based expanded personalized diabetes treatment recommendation using South Korean electronic health records.

Expert Syst. Raheb, M. Subcutaneous insulin administration by deep reinforcement learning for blood glucose level control of type-2 diabetic patients. Thomas, M. Model-based reinforcement learning: a survey. Trends Mach.

Learn 16 , 1— Google Scholar. Huang, Q. Model-based or model-free, a review of approaches in reinforcement learning. Coronato, A.

Reinforcement learning for intelligent healthcare applications: a survey. Nemati, S. Optimal medication dosing from suboptimal clinical examples: a deep reinforcement learning approach.

IEEE Eng. PubMed Google Scholar. Vasey, B. Reporting guideline for the early-stage clinical evaluation of decision support systems driven by artificial intelligence: DECIDE-AI. Garg, S. Improved glycemic control in intensively treated adult subjects with type 1 diabetes using insulin guidance software.

Farajtabar, M. More robust doubly robust off-policy evaluation. In Proc.