Hyperparameter Optimization Boosts AF Detection: Hyperband+XGBoost Reaches 96.95% F1

Atrial Fibrillation (AF) is one of the most common sustained cardiac arrhythmias, and its early detection is crucial for preventing stroke and heart failure. In recent years, machine learning and deep learning models have been widely used for automatic AF identification based on electrocardiogram (ECG), but model performance is highly dependent on hyperparameter configuration, and the value of systematic hyperparameter optimization (HPO) in AF detection remains insufficiently evaluated. A new study addresses this gap by comparing the combined effects of multiple classifiers and HPO strategies, and provides a clear performance baseline.

Study Design and Model Selection

The study covers both traditional machine learning and deep learning tracks. On the traditional side, three standard classifiers are used: decision tree, random forest, and XGBoost; on the deep learning side, recurrent neural networks (RNN), deep neural networks (DNN), and Siamese Neural Networks are included. This combination includes strong baselines for tabular feature scenarios as well as sequence modeling and metric learning approaches, facilitating a cross-sectional comparison of the sensitivity of different model families to hyperparameter tuning.

For optimization methods, the study selects four representative HPO strategies: Genetic Algorithm, Hyperband, Optuna, and SMAC3. These four represent different paradigms—evolutionary search, early-stopping-based resource allocation, Bayesian optimization, and model-based configuration search—providing a relatively comprehensive comparison for assessing whether the choice of optimizer affects final detection accuracy.

Hyperband is a resource-allocation-based hyperparameter optimization algorithm built on early stopping. Its core idea is to allocate a limited training budget to a large number of configurations, eliminate poorly performing ones round by round, and concentrate computational resources on promising candidates. Hyperband is essentially a repeated application of the Successive Halving strategy: each round starts with a different ratio of small budget to large budget, thereby balancing "exploring more configurations" and "fully training a few configurations." Compared with grid search or pure Bayesian optimization, Hyperband has a clear efficiency advantage when the hyperparameter space is large and the cost of a single training run is high—this is also the mechanistic basis for its best results on XGBoost.

XGBoost (eXtreme Gradient Boosting) is an ensemble learning algorithm based on gradient boosted decision trees (GBDT). It progressively fits residuals through an additive model, adding a new tree each round to correct the prediction errors of the preceding model, while incorporating both a loss term and a regularization term in the objective function to control overfitting. XGBoost supports parallelization and sparsity awareness, trains efficiently, and has long been a strong baseline in tabular feature scenarios. In AF detection, if the input consists of handcrafted ECG features (such as RR interval variability and frequency-domain metrics), XGBoost can effectively capture nonlinear interactions among features, which explains why it can rival more complex deep learning architectures.

Data Splitting and Class Imbalance Considerations

The experiments are based on the PhysioNet ECG dataset and adopt two data splitting schemes: inter-patient and intra-patient. The former requires the training and test sets to come from different patients, which is closer to the generalization scenario in clinical deployment; the latter splits within the same patient and usually yields more optimistic metrics. Reporting both splits side by side helps distinguish whether the model has truly learned cross-patient discriminative ability or merely fitted individual characteristics.

Class imbalance is an inherent difficulty in AF detection—in real-world long-term recordings, AF segments account for a very low proportion. The study examines model robustness under class imbalance in both splitting schemes, making the conclusions closer to actual clinical screening settings rather than idealized balanced-dataset experiments.

Key Results: Hyperband-Optimized XGBoost Leads

Among all model and optimizer combinations, XGBoost tuned with Hyperband achieves the best performance, with a peak F1 score of 96.95%. This result has two implications: first, under the given features and data conditions, gradient boosted tree models combined with efficient resource-allocation-based search are sufficient to match or even surpass more complex deep learning architectures; second, HPO is not an insignificant engineering detail—an appropriate optimization strategy can substantially change the accuracy and robustness of classification results.

It is worth noting that in class-imbalanced scenarios, the F1 score reflects actual detection quality better than accuracy because it takes both precision and recall into account. The study uses F1 as the core reporting metric, which is consistent with its focus on imbalanced data.

The F1 score is the harmonic mean of precision and recall in classification tasks, calculated as F1 = 2 × (Precision × Recall) / (Precision + Recall). In scenarios such as AF detection where positive and negative samples are severely imbalanced, if the model tends to predict the majority class (non-AF), accuracy may still be high, but recall will be extremely low—that is, many AF segments will be missed. The symmetric penalty of the F1 score on precision and recall enables it to effectively expose this "false high accuracy," making it the preferred core evaluation metric over accuracy in imbalanced data classification. The peak F1 of 96.95% reported in this paper means that both precision and recall are at very high levels.

Implications for Clinical Deployment

The core message of this study is that the clinical usability of AF classification models depends not only on network architecture or feature engineering, but also to a considerable extent on the choice of hyperparameter optimization strategy. The reason Hyperband performs prominently on XGBoost is related to its early-stopping mechanism, which concentrates the computational budget on promising configurations—this is particularly efficient when the hyperparameter space is large.

Another implication lies in the evaluation protocol. Robust performance under inter-patient splitting is the prerequisite for a model to enter real screening workflows. The study reports both splits, providing a reproducible comparison framework for future work. If this combination can be validated on more datasets and longer-term recordings in the future, the pairing of Hyperband and XGBoost is expected to become a practical baseline for automatic AF detection.

One-line Summary

Systematically evaluates the value of hyperparameter optimization in AF detection; Hyperband-tuned XGBoost leads with F1 96.95%.

Paragraph Summary

This study systematically evaluates the role of hyperparameter optimization (HPO) in automatic ECG-based atrial fibrillation detection. The experiments cover three traditional classifiers—decision tree, random forest, and XGBoost—and three deep learning models—RNN, DNN, and Siamese networks—and compare four HPO strategies: Genetic Algorithm, Hyperband, Optuna, and SMAC3. Using the PhysioNet dataset, it adopts both inter-patient and intra-patient splits and evaluates robustness under class imbalance. Results show that Hyperband-tuned XGBoost achieves the best performance, with a peak F1 score of 96.95%, outperforming all deep learning architectures. The study emphasizes that an appropriate HPO strategy can substantially change classification accuracy and robustness, and that the combination of Hyperband and XGBoost is expected to become a practical baseline for automatic AF detection.