Electronic Health Record

The study suggests that the real breakthrough may lie in designing intermediate representations that can bridge clinical semantics and latent-space prediction, rather than changing the prediction target.

MLM does not explicitly encourage the model to learn high-level patient states (such as overall disease progression or risk stratification), which are the abstractions most needed for clinical downstream tasks.

In EHR sequences, high-level abstractions of patient state (such as disease progression, comorbidity burden, treatment response) lack an accepted, computable representational definition, making them difficult to convert into dense latent targets suitable for JEPA prediction.

For practitioners, in the short term MLM remains a reliable baseline for longitudinal EHR pretraining.

Masked language modeling (MLM) is the de facto standard pretraining objective for Transformer encoders processing longitudinal electronic medical records.

The paper points out that the bottleneck of JEPA on EHR is not the magnitude of prediction error, but whether the predicted latent target itself has clinical significance.

Reconstructing individual events may cause MLM to overemphasize noisy or low-information observations and waste its capacity on low-value signals.

A new study proposes using Block Diffusion (BD) instead of autoregression (AR) for zero-shot longitudinal EHR forecasting.

Longitudinal EHR data is characterized by events occurring in temporal clusters and being highly correlated with one another, and the strict sequential generation assumption of AR models does not fully align with this characteristic.

A semi-synthetic study based on real electronic health record covariates shows that a globally trained policy misgeneralizes on a high-risk subgroup, while FDR control limits the number of harmful treatments that the policy would otherwise have administered.