Bridging the gap from clinical to home ECG: quantifying and overcoming accuracy loss in AI-enabled single-lead ECG models

Data source and ethical statement

This study was approved by the Institutional Review Board (IRB) of Tri-Service General Hospital (A202505026), which also granted a waiver of informed consent. The waiver was justified because this was a retrospective analysis of ECG recordings that had been collected by patients using consumer-grade devices—specifically, the Apple Watch (Taiwan Food and Drug Administration [TFDA] approval no. 033901) and the QOCA ECG102D (TFDA approval no. 005325)—and routinely integrated into patients’ electronic health records (EHRs) as part of standard care. Given the retrospective design and the use of existing EHR-linked data, the requirement for individual patient informed consent was waived by the IRB. Importantly, these single-lead ECGs represent patient-initiated recordings integrated into routine care via the EHR; we did not analyze continuous free-living recordings captured during daily activities. Therefore, generalizability to fully unsupervised home monitoring requires prospective evaluation.

The workflow and dataset details are summarized in Fig. 1. Briefly, we developed a base AI model using Lead-I ECG segments extracted from standard clinical 12-lead ECG recordings from the Philips TC series (TC30, TC50, TC70) acquired at the Neihu General Hospital site of Tri-Service General Hospital (academic medical center). A total of 246,874 patients were randomly assigned to a training set (172,827 patients, 474,130 ECGs) and an internal test set (74,047 patients, 202,062 ECGs). Additionally, an independent external test set comprising 65,338 patients (219,231 ECGs) from Tingzhou Branch Hospital (community hospital) was used for external validation.

To evaluate the generalizability of the base AI model to home-based ECG recordings, we collected single-lead ECG data from two types of consumer-grade devices: Apple Watch and QOCA ECG102D. For both consumer devices, we used the vendor-provided signal-quality assessment as the first-line quality gate and excluded ECGs flagged as “inconclusive”, “poor recording”, or “low quality” prior to model training and evaluation. We only included the “sinus rhythm” and “atrial fibrillation” recordings identified by vendor algorithms. This choice reflects real-world operation of these devices, where low-quality recordings are typically rejected or prompt reacquisition, and thus our AI models are intended to be applied to device-accepted recordings integrated into the EHR. Specifically, for the Apple Watch dataset, 8790 patients at Neihu General Hospital recorded Lead-I ECGs. Patients were randomly assigned to training (6113 patients, 7044 ECGs) and test (2677 patients, 3399 ECGs) sets. After excluding ECG recordings classified by the device vendor as inconclusive or poor quality (426 ECGs [6.0%] in the training set and 182 ECGs [5.4%] in the test set), we retained 6618 ECGs in the training set and 3217 ECGs in the test set. Similarly, the QOCA ECG102D dataset included 27,983 patients who recorded ECGs. After randomization into training (19,553 patients, 24,977 ECGs) and test (8430 patients, 10,942 ECGs) subsets, ECG signals labeled as low quality by the device vendor (3050 ECGs [13.9%] in the training set and 1352 ECGs [12.4%] in the test set) were excluded, resulting in final training and test sets comprising 21,927 ECGs and 9590 ECGs, respectively.

All 12-lead ECGs including recordings from both the Neihu and Tingzhou sites, were obtained before February 2022. In contrast, the Apple Watch and QOCA ECG102D cohorts were composed predominantly of recordings obtained after January 2023. Because these data were collected as part of longitudinal routine clinical care, there was partial patient overlap across cohorts. Specifically, 54% of patients in the QOCA ECG102D cohort and 58% of patients in the Apple Watch cohort had prior 12-lead ECG data in Cohort A. In addition, 15% of patients in the QOCA ECG102D cohort also appeared in the Apple Watch cohort, and 47% of patients in the Apple Watch cohort also appeared in the QOCA ECG102D cohort. To prevent patient-level leakage despite partial overlap across cohorts, patients were not randomized independently within each cohort. Instead, we generated a single master patient-level assignment from the union of all eligible medical record numbers across Cohorts A-C and allocated these medical record numbers to the development or test partition at a 70%/30% ratio. This same assignment was then applied to every cohort. Thus, any patient appearing in more than one cohort was assigned to the same partition across all cohorts; for example, a patient in the development set of Cohort A could only appear in the development set of the Apple Watch or QOCA ECG102D cohorts, and could not appear in their test sets. This design prevented patient-level information leakage between development and test partitions despite cross-cohort overlap.

ECG signal preprocessing

The original ECG format obtained from the Philips TC-series system was a 10-s, 12-lead recording, whereas ECG recordings from the Apple Watch and QOCA ECG102D were 30-s single-lead (Lead-I) recordings. To ensure consistency across datasets, we extracted Lead-I segment from the 12-lead ECG recordings and standardized the amplitude unit to 10 μV increments for all three ECG sources. Additionally, we uniformly resampled signals to 500 Hz, as the original sampling frequency was 512 Hz for Apple Watch data, whereas Philips TC-series and QOCA ECG102D data were sampled at 500 Hz.

Regarding filtering, the 12-lead ECGs from the Philips system had already undergone built-in filtering with a high-pass filter at 0.05 Hz and a low-pass filter at 150 Hz. In contrast, despite initial vendor-applied filtering of the Apple Watch and QOCA ECG102D signals, significant baseline wander and noise persisted. Therefore, we implemented additional digital filtering for these two datasets using second-order Butterworth high-pass (0.5 Hz) and low-pass (150 Hz) filters to effectively reduce baseline drift and high-frequency noise. To ensure consistency, given that the Apple Watch allows user recordings shorter than 30 s, we uniformly trimmed Apple Watch and QOCA ECG102D recordings to retain only the central 28-s segments. For the Philips TC-series ECG recordings, we utilized the entire 10-s duration without additional truncation.

Phenotypes used for prediction

Since each ECG recording had a precise timestamp logged within the EHR, we utilized this timestamp as the index time to merge the ECG data with patient demographic and clinical variables, echocardiographic measurements, and laboratory test results. Initially, we performed a literature review to identify appropriate phenotypes for prediction. The phenotypes selected had to meet two criteria: (1) previous studies had demonstrated the feasibility of predicting these phenotypes from Lead-I ECG recordings; (2) sufficient abnormal samples were available within our dataset. Based on these criteria, we identified a total of 9 phenotypes for prediction, categorized as follows: Demographic and clinical variables–gender, age, 90-day mortality; Echocardiographic outcomes–left ventricular ejection fraction (EF), pulmonary artery systolic pressure (PASP), left atrial diameter (LAD); Laboratory outcomes–N-terminal pro-B-type natriuretic peptide (NT-proBNP), hemoglobin (Hb), estimated glomerular filtration rate (eGFR). Gender and Age predictions were retained as a methodological control and domain-shift indicator rather than a clinically actionable target. We merged echocardiographic outcomes within ±7 days of the index ECG recording time³⁶. Laboratory measurements were merged using different time windows based on their clinical relevance and frequency of testing: NT-proBNP within ±24 h³⁷, Hb within ±12 h³⁸, and eGFR within ±30 days³⁹.

From these phenotypes, we further defined eight binary prediction tasks–Gender (male vs. female), death in 90 days (death vs. survival), low EF (≤40% vs. >40%), elevated PASP (≥50 mmHg vs. <50 mmHg), enlarged LAD (≥50 mm vs. <50 mm), elevated NT-proBNP (≥1000 pg/mL vs. <1000 pg/mL), anemia (Hb ≤10 g/dL vs. >10 g/dL), reduced renal function (eGFR ≤60 mL/min vs. >60 mL/min) –and seven continuous prediction tasks–age (years), EF (%), PASP (mmHg), LAD (mm), NT-proBNP (pg/mL) in log-transformed (log base 10) due to severe right skewness, Hb (g/dL), eGFR (mL/min).

Deep learning model training and inference

The deep learning architecture utilized in this study was based on the ResNet architecture, employing one-dimensional convolutional layers for feature extraction. The detailed network architecture has been previously described⁵¹. The only modification was the removal of the original module designed for merging 12-lead ECG information. High-level features extracted directly from Lead-I ECG segments were instead used for subsequent binary or continuous predictions. All models were trained using the Adam optimizer (standard parameters: β₁ = 0.9, β₂ = 0.999), with an initial learning rate of 0.0001, reduced by a factor of 10 at the 30th and 60th epochs, a mini-batch size of 32 ECG segments, and weight decay of 0.001. Training was performed for a total of 100 epochs. In each task, the training data were randomly partitioned, allocating 80% for model fitting and 20% to select the best-performing model based on lowest validation loss. For each prediction task, only a single optimal model was selected for final inference on the test set.

For data augmentation, random cropping and signal inversion (vertical flipping) were applied. Specifically, during training with Lead-I segments from 12-lead ECGs (original length: 5000 data points), random crops of 4096 points were used. At inference, predictions from four signal variations (original first and last 4096 points, and vertically flipped first and last 4096 points) were averaged. For Apple Watch and QOCA ECG102D signals (original length after preprocessing: 14,000 data points), similar random cropping to a length of 12,288 data points was performed. Inference similarly averaged predictions from four variations (original first and last 12,288 points and vertically flipped first and last 12,288 points). Due to the use of global average pooling in the network’s final layer, the model initially trained on 4096 data points was fully compatible with longer input lengths, including 12,288 data points, allowing direct application to Apple Watch and QOCA ECG102D data without further adjustment.

The initial weights for models trained on Lead-I segments derived from the 12-lead ECG dataset were randomly initialized. Conversely, all fine-tuned models employed weights from the corresponding 12-lead ECG-trained Lead-I model as initial parameters. Fine-tuning experiments utilized subsets of varying sample sizes (n = 100, 200, 500, 1000, and 2500 ECGs). For subsets with particularly small training samples, we ensured inclusion of at least two positive cases. For binary prediction tasks, oversampling was implemented to ensure balanced representation, enabling the model to learn discriminative features from both case and control samples. For continuous prediction tasks, the range of values from the minimum to the maximum was partitioned into 20 equal bins, and samples within each bin were oversampled to achieve equal counts⁵¹. Because this strategy alters the value distribution of the training set, we subsequently applied a linear regression-based calibration to adjust the predicted values and mitigate potential numerical offset. Moreover, we also performed a sensitivity analysis using an inverse regression calibration to address regression-to-the-mean. All calibration mappings were derived exclusively from the validating subset and then applied unchanged to the independent test set.

Statistical analysis and model performance assessment

Continuous variables were presented as means ± standard deviation, while categorical variables were summarized as counts and percentages. The performance of AI models for binary classification tasks was evaluated using the AUC along with corresponding 95% confidence intervals (CI). For continuous prediction tasks, the performance was quantified using Pearson’s correlation coefficient (r) and MAE with associated 95% CIs. To rigorously assess domain shifts from 12-lead ECG-trained models to home-based single-lead ECG devices, we compared the prediction performance between the original Lead-I model trained on 12-lead ECGs and fine-tuned models using data collected from Apple Watch or QOCA ECG102D devices. Model performance comparisons were tested using DeLong’s method for differences in AUC, Fisher’s Z-transformation for differences in Pearson’s correlation coefficients, and paired t-test for difference in MAE. To assess agreement and potential systematic bias for continuous endpoints, we additionally performed Bland–Altman analyzes, plotting prediction error (AI-ECG prediction minus ground truth) against the corresponding ground-truth measurement; the mean difference (bias) and 95% limits of agreement were defined as mean ± 1.96 standard deviations.

For 7 clinical phenotypes, we also reported sensitivity and specificity at a pre-specified operating threshold. For clinical interpretability, we additionally reported prior risk as the observed test-set prevalence, posterior risk after a positive AI-ECG screen as positive predictive value (PPV), and residual risk after a negative AI-ECG screen as 1 – negative predictive value (NPV). Two thresholds were evaluated. First, a Youden-index threshold was selected on the validating subset (20% split from the training set) to provide a balanced operating point for model comparison. Second, a sensitivity-prioritized threshold was selected on the validation subset to target 90% sensitivity, reflecting a screening-oriented workflow in which missed cases may be more harmful than additional confirmatory testing. Both thresholds were selected exclusively in the validation subset and then applied unchanged to the independent test set. Accordingly, the realized sensitivity in the test set, particularly in small subgroups, may differ from the nominal validation target.

Additionally, to evaluate the relationship between the fine-tuning sample size and improvement in prediction performance, we generated learning curves showing changes in AUC (binary tasks) and Pearson’s r (continuous tasks) relative to increasing fine-tuning dataset sizes. The relationship between sample size and predictive accuracy was modeled using a power-law function according to a previous study⁵². This analysis allowed us to quantify the minimal number of fine-tuning samples required to restore performance lost due to insufficient fine-tuning samples. All statistical analyses were performed using R software (version 3.4.4), and two-tailed p-values < 0.05 were considered statistically significant.

Benchmark dataset generation

We curated and released an evaluation benchmark, the Health records-linked Open Multi-consumer device Electrocardiogram (HOME) dataset, from the Apple Watch test set and the QOCA ECG102D test set. For each device, we constructed a patient-level subset of 1000 individuals, with exactly one single-lead ECG recording per patient to avoid within-subject correlation. We excluded the Apple Watch ECGs shorter than 30 s. Because 90-day mortality events and the availability of linked NT-proBNP measurements were sparse in the wearable test cohorts, we used an enriched two-stage sampling strategy to ensure adequate representation of these endpoints. Specifically, within each device-specific test cohort, we first included all eligible patients who (i) died within 90 days of the index ECG and/or (ii) had an NT-proBNP value available within the prespecified linkage window. If multiple eligible ECGs existed for a patient, one recording was randomly selected. After forcing inclusion of the above patients, the remaining slots were filled by simple random sampling from the rest of the device-specific test cohort until a total of 1000 patients was reached.

To mitigate privacy risks, the released HOME waveforms were downsampled to 200 Hz, resulting in 6000 samples per 30-s recording. For compatibility with the original inference pipeline (trained and evaluated using 500 Hz signals), we upsampled the released waveforms back to 15,000 samples (corresponding to 30 s at 500 Hz) and applied the same preprocessing and inference procedure as in the main analyses. Task-specific model predictions (before and after device-specific fine-tuning) are released alongside the waveform files to facilitate standardized benchmarking and reproducibility.