Personal Blood Glucose Prediction Model: A 30K-Parameter Transformer with LoRA Fine-Tuning in Practice

A 30K-parameter Transformer with synthetic pretraining and LoRA fine-tuning achieves personal glucose prediction rivaling published models.
A developer trained a 31,253-parameter encoder-only Transformer on ~19,200 CGM-years of synthetic data, then LoRA-fine-tuned on 115 days of his own CGM data. Inputs include insulin and carbohydrate action curves for counterfactual reasoning; outputs are 2-hour glucose forecasts in Kovatchev risk space with a DILATE+MSE+pinball loss. The model imputes missing data, backtests, and learns time-of-day with ≥48h context. It still trails the developer at 30-minute hypoglycemia prediction, and counterfactual reasoning lacks benchmarks. Code is open source.
A developer used a 30K-parameter Transformer model to predict his own blood glucose, achieving results comparable to published models. This personal experiment demonstrates the potential of synthetic-data pretraining combined with LoRA fine-tuning for medical time-series forecasting.

Model Architecture and Training Strategy
The developer trained an encoder-only Transformer with just 31,253 parameters. The core idea is "synthetic-data pretraining + personal-data fine-tuning." The model was first pretrained on roughly 19,200 CGM-years (continuous glucose monitoring years) of synthetic data—records from 1 million simulated type 1 diabetes patients, 7 days each. He then used his own 115 days of CGM data (80 days for training the adapter, 35 days for testing) for LoRA fine-tuning.
The advantage of this two-stage strategy: synthetic data lets the model learn general glucose dynamics, while LoRA fine-tuning adapts it to individual characteristics without overfitting. According to the developer, the model is competitive with other published models.
LoRA (Low-Rank Adaptation) is a parameter-efficient fine-tuning technique: it freezes all original weights of the pretrained model and injects trainable low-rank matrix pairs (typically a down-projection matrix A and an up-projection matrix B) into the Transformer's attention layers, updating only these newly added parameters. Since the rank r is much smaller than the original dimensions, the number of trainable parameters often drops to 0.1%–1% of the original model, with memory and storage costs falling dramatically as well. For scenarios like personal health data—small sample sizes and privacy-sensitive—LoRA preserves the general glucose dynamics learned from synthetic data while adapting to individual characteristics with very few parameters, significantly reducing overfitting risk. After fine-tuning, the adapter can be saved and swapped independently, making it easy to reuse the same base model across different individuals or devices.
Multi-Input and Output Design
The model accepts not only historical CGM time series but also two additional inputs: insulin (U) and carbohydrates (G, GI), provided as action curves. Insulin dose U is an exact value, while carbohydrate intake (G, GI) is the developer's best estimate. This design enables counterfactual reasoning—for example, "what would happen to blood glucose if I took 1 more unit of insulin?"
On the output side, the model predicts blood glucose for the next 2 hours, represented in Kovatchev risk space. The training loss combines DILATE, MSE, and pinball loss, fused via learned Kendall-Gal weights. In addition, the model can impute missing data and perform backtesting (predicting the past).
The developer notes that the model also learned to predict time of day from historical glucose—a capability that emerged with context windows ≥48 hours. The model natively supports variable context windows from 3.5 to 7.0 days.
Kovatchev risk space is a nonlinear transformation framework for assessing glycemic control: it maps raw glucose values to risk values, assigning a steeper penalty slope to the hypoglycemic range than to the hyperglycemic range, thereby reflecting the acute danger of low blood glucose. Common forms include the Glucose Risk Index (GRI) and low/high blood glucose indices. Making predictions and computing losses in this space makes the model more sensitive to dangerous hypoglycemic excursions and better aligned with clinical concerns.
DILATE loss is a loss function for time-series forecasting that combines two differentiable components—shape and temporal distortion—allowing predictions to shift moderately along the time axis without being over-penalized. It suits signals like glucose curves that are sensitive to phase errors but whose morphology matters; pinball loss (quantile loss) is used to train quantile predictions, providing an interval rather than a point estimate for future glucose.
Limitations and Personal Experience
Despite the strong performance, the developer admits the model is still worse than himself at predicting hypoglycemia about 30 minutes ahead. This may be because hypoglycemic events involve complex individual physiological factors that are hard to fully learn from data.
Currently, there is no benchmark to validate the counterfactual reasoning capability, but his intuitive experience using the model on his phone is that when future insulin and carbohydrates are forcibly taken into account, the model responds sensibly. The source code is open-sourced on GitHub (0xdeadf1sh/T1DMAI) for interested developers.
This personal experiment shows that a small-scale Transformer combined with LoRA fine-tuning can reach a practical level on specific health prediction tasks, offering a low-cost, reproducible path for personalized medical AI.
Counterfactual reasoning here refers to the model's ability to predict how the glucose trajectory would change given different interventions (e.g., an extra 1 unit of insulin or a certain carbohydrate intake). Unlike ordinary prediction, it requires the model to learn a causal-style response between actions and outcomes, rather than merely fitting historical correlations. Since multiple interventions at the same moment cannot be observed simultaneously in the real world, counterfactual predictions lack direct labels and can usually only be validated indirectly via simulators or intervention trials—which is why the developer currently has no benchmark.
One-line Summary
A developer used a 30K-parameter Transformer with synthetic pretraining and LoRA fine-tuning to achieve personal glucose prediction comparable to published models.
Paragraph Summary
A developer trained an encoder-only Transformer with just 31,253 parameters to predict his own blood glucose. The model was first pretrained on roughly 19,200 CGM-years of synthetic data to learn general glucose dynamics, then adapted to the individual via LoRA fine-tuning on his own 115 days of CGM data. Inputs include insulin and carbohydrate action curves in addition to historical CGM, enabling counterfactual reasoning; outputs are glucose for the next 2 hours in Kovatchev risk space, with a loss combining DILATE, MSE, and pinball. The model can also impute missing data, backtest, and—with context ≥48 hours—exhibit an emergent ability to predict time of day. Limitations: it is still worse than the developer himself at predicting hypoglycemia about 30 minutes ahead, and counterfactual reasoning lacks benchmark validation. The code is open-sourced, demonstrating a low-cost, reproducible path for personalized medical AI.
Related articles

Why Microsoft CEO Nadella Still Backs Nvidia After Jensen Huang's Departure
Microsoft CEO Nadella publicly expresses confidence in Nvidia interim CEO Colette Kress. This article analyzes Nvidia's power transition after Jensen Huang's departure, Microsoft's compute dependency, and the trust logic of the AI supply chain.

Nadella: Critical Infrastructure Is 'Unpatchable' as AI Cybersecurity Arms Race Escalates
Microsoft CEO Nadella addresses AI cyberattack fears, saying much critical infrastructure is unpatchable, downplaying the Outlook outage, and analyzing AI's dual role in cybersecurity.

Musk Backs Jensen Huang: Firing Him Is Like Cutting Jordan
Musk says Jensen Huang is loyal to the U.S. and likens firing him to cutting Michael Jordan—highlighting politicization anxieties in the AI chip industry and defending talent value.