A Nomogram to Predict Patients with Obstructive Coronary Artery Disease: Development and Validation

Abstract

Objective: To develop and validate clinical prediction models for the development of a nomogram to estimate the probability of patients having coronary artery disease (CAD).

Methods and Results: A total of 1,025 patients referred for coronary angiography were included in a retrospective, single-center study. Randomly, 720 patients (70%) were selected as the development group and the other patients were selected as the validation group. Multivariate logistic regression analysis showed that the seven risk factors age, sex, systolic blood pressure, lipoprotein-associated phospholipase A₂, type of angina, hypertension, and diabetes were significant for diagnosis of CAD, from which we established model A. We established model B with the risk factors age, sex, height, systolic blood pressure, low-density lipoprotein cholesterol, lipoprotein-associated phospholipase A₂, type of angina, hypertension, and diabetes via the Akaike information criterion. The risk factors from the original Framingham Risk Score were used for model C. From comparison of the areas under the receiver operating characteristic curve, net reclassification improvement, and integrated discrimination improvement of models A, B, and C, we chose model B to develop the nomogram because of its fitness in discrimination, calibration, and clinical efficiency. The nomogram for diagnosis of CAD could be used easily and conveniently.

Conclusion: An individualized clinical prediction model for patients with CAD allowed an accurate estimation in Chinese populations. The Akaike information criterion is a better method in screening risk factors. The net reclassification improvement and integrated discrimination improvement are better than the area under the receiver operating characteristic curve in discrimination. Decision curve analysis can be used to evaluate the efficiency of clinical prediction models.

Main article text

Introduction

For 44 years, the Framingham Heart Study has shown that the incidence of coronary artery disease (CAD) is 49% in men and 32% in women. With increasing economic development, the number of individuals with CAD is increasing, of which more than one million die each year [1]. For diagnosis of CAD, coronary artery angiography (CAG) is now the first choice. The numbers of CAG examinations increased by 17 times in the past decade according to a report on the PEACE study [2]. However, CAG is not widely accepted by everybody due to traumatic effects and affordability. Noninvasive artery computed tomography can provide limited information in diagnosis, and composite plaque burden scores seem to increase the sensitivity for CAD. But there are still some risks one has to consider, such as radiation-induced cancer, nephropathy, and contrast medium extravasation [3].

With the high morbidity and mortality associated with CAD, clinical prediction models (CPMs) have attracted increasing attention in the current era of personalized medicine. The application of CPMs in left main coronary artery disease was demonstrated by Lee et al. [4]. Wilson et al. [5] developed a simple coronary disease prediction algorithm using categorical variables: blood pressure, total cholesterol (TC) level, and low-density lipoprotein cholesterol (LDL-C) level. The development and validation of CPMs in CAD depend on a complex combination of clinical factors, which also have the certain report templates according to the TRIPOD [6].

However, few studies of CPMs have been reported in Asian countries. The original Framingham Risk Score (FRS) overestimated the risk of CAD in the Chinese population. Although recalibration improved the estimates and demonstrated that the Framingham model is useful in the Chinese population, the same elements were selected rather than other factors [7]. We aimed to rescreen clinical factors to develop a simple diagnostic CPM, in comparison with the FRS, in the Chinese population and validate it.

Methods

Patient Selection

We analyzed retrospectively the clinical data of 1,025 patients who underwent CAG examination between November 2018 and August 2019 at the First Affiliated Hospital of Henan University of Science and Technology. Some patients were excluded: those with a history of CAG or percutaneous intervention with CAD, and those with a history of coronary artery bypass surgery with non-ST-segment elevation myocardial infarction or ST segment elevation myocardial infarction. Seven hundred twenty patients (70%) were enrolled in the development cohort randomly with the use of statistical software (R package); the other patients (30%) formed the validation cohort. All patients gave their written informed consent before CAG examination.

Data Collection

The blood pressures of the patients were obtained on their admission to the hospital. All blood samples were drawn before breakfast the next morning and subjected to laboratory inspection with standardization at the hospital’s laboratory. The variables studied were age, sex, height, weight, systolic blood pressure (SBP), diastolic blood pressure, hemoglobin A_1c (HbA_1c) level, hemoglobin level, white blood cell (WBC) count, platelet count, serum creatinine level, serum uric acid (UA) level, TC level, triglyceride level, high-density lipoprotein cholesterol (HDL-C) level, low-density LDL-C level, lipoprotein-associated phospholipase A₂ (Lp-PLA₂) level, left ventricular ejection fraction (LVEF), clinical symptoms (type of angina), history or no history of hypertension, history or no history of smoking, and history or no history of diabetes.

The type of angina was defined according to European Society of Cardiology guidelines [8], as was hypertension [9]. Diabetes mellitus was defined as fasting glucose levels greater than 7.0 mmol/L or postprandial blood glucose levels greater than 11.1 mmol/L after 2 hours of treatment with insulin or oral hypoglycemic medications. CAD was defined as the presence of at least one major artery or coronary artery branch with 50% or greater narrowing. In the case of unavailable data, the missing information was imputed by the mice package in R.

Statistical Analysis

Continuous variables were expressed as the mean ± standard deviation and compared by the independent t test. Categorical variables were expressed as a percentage and compared by Pearson chi-square tests among patients with CAD and patients without CAD. The statistical analyses, including logistic regression analysis, were performed with IBM SPSS Statistics (version 25.0) and R packages.

Model Development and Validation

In all patients, statistical analysis was performed to determine whether risk factors were significant; P < 0.1 was considered statistically significant. We used logistic regression analysis in the development cohort to choose the risk predictors that were significant (P < 0.05) in univariate analysis. The forward elimination approach was used in multivariate analysis until all of the factors were significant (P < 0.05). To indicate the efficiency of the model better, we adjusted some variables appropriately. In this way, the final CPM (model A) consisted of seven factors: age, sex, SBP, Lp-PLA₂, type of angina, hypertension, and diabetes. Model B consisted of the factors age, sex, height, SBP, LDL-C, Lp-PLA₂, type of angina, hypertension, and diabetes with the Akaike information criterion (AIC), which can decrease the maximum risk possibility of infinite sample sizes [10]. Model C consisted of factors that come from the FRS [5], which included age, sex, TC, HDL-C, SBP, history of smoking, and history of diabetes.

The discrimination of the CPM is most often done by c-statistics – calculating the area under the curve (AUC) of the receiver operating characteristic (ROC) curve. Net reclassification improvement (NRI) and integrated discrimination improvement (IDI) are new evaluation indicators for discrimination [11]. The CPMs were assessed by calibration plots. The clinical application has been showed with the decision curve analysis (DCA). We wished to determine which model is best by means of the AUC, NRI, IDI, and DCA in the population studied. The same assessment was done in the validation cohort.

Results

Baseline Characteristics

The cohort was divided into two groups according to coronary stenosis of 50% or greater or less than 50% as a categorical variable. A total of 1,025 eligible patients were analyzed: 745 had CAD and 280 did not have CAD. We considered the following as significant risk factors: age, sex, height, weight, SBP, HbA_1c, WBC count, hemoglobin, serum creatinine, serum UA, HDL-C, LDL-C, Lp-PLA₂, LVEF, clinical symptoms, history of hypertension, history of smoking, and history of diabetes. Baseline characteristics are summarized in Tables 1 and 2.

Table 1

Baseline Characteristics of All Patients with and without Coronary Artery Disease (CAD).

Baseline characteristics	Without CAD (n = 2 80)	With CAD (n = 745)	P
Age (years)	58.48 ± 10.69	62.23 ± 9.84	< 0.001
Height (cm)	163.93 ± 7.55	164.94 ± 8.24	0.046
Weight (kg)	67.52 ± 13.03	68.76 ± 11.78	0.021
SBP (mmHg)	131.84 ± 18.71	137.36 ± 19.46	< 0.001
DBP (mmHg)	83.09 ± 12.28	83.56 ± 12.11	0.481
HbA1c (%)	5.94 ± 0.79	6.31 ± 1.22	< 0.001
WBCs (109/L)	6.18 ± 4.16	6.40 ± 5.31	0.079
Hemoglobin (g/L)	130.88 ± 17.33	132.79 ± 16.11	0.088
Platelets (109/L)	213.86 ± 53.32	209.92 ± 59.01	0.131
Creatinine (μmol/L)	60.23 ± 12.82	69.39 ± 53.10	< 0.001
UA (μmol/L)	307.33 ± 82.50	318.08 ± 83.32	0.046
TC (mmol/L)	4.15 ± 1.02	4.14 ± 1.07	0.742
TG (mmol/L)	1.72 ± 1.41	1.68 ± 1.00	0.302
HDL-C (mmol/L)	1.17 ± 0.27	1.12 ± 0.28	< 0.001
LDL-C (mmol/L)	2.05 ± 0.69	2.14 ± 0.70	0.098
Lp-PLA2 (ng/mL)	144.18 ± 96.63	159.54 ± 125.22	0.071
LVEF (%)	62.86 ± 5.39	62.08 ± 5.36	0.011
Sex			< 0.001
Male	109 (38.93%)	427 (57.32%)
Female	171 (61.07%)	318 (42.68%)
Symptoms			< 0.001
No	223 (79.64%)	445 (59.73%)
Yes	57 (20.36%)	300 (40.27%)
Hypertension			< 0.001
No	164 (58.57%)	321 (43.09%)
Yes	116 (41.43%)	424 (56.91%)
Diabetes			< 0.001
No	257 (91.79%)	593 (79.60%)
Yes	23 (8.21%)	152 (20.40%)
Smoking history			0.008
No	220 (78.57%)	524 (70.34%)
Yes	60 (21.43%)	221 (29.66%)

DBP, diastolic blood pressure; HbA_1c, hemoglobin A_1c; HDL-C, high-density lipoprotein cholesterol; LDL-C, low-density lipoprotein cholesterol; Lp-PLA₂, lipoprotein-associated phospholipase A₂; LVEF, left ventricular ejection fraction; SBP, systolic blood pressure; TC, total cholesterol; TG, triglycerides; UA, uric acid; WBCs, white blood cells.

Table 2

Baseline Characteristics of Patients with and without Coronary Artery Disease (CAD) in the Development Group and the Validation Group.

Baseline characteristics	Development group			Validation group
	Without CAD (n = 192)	With CAD (n = 528)	P	Without CAD (n = 88)	With CAD (n = 217)	P
Age (years)	58.91 ± 10.79	62.42 ± 9.76	< 0.01	57.56 ± 10.49	61.77 ± 10.06	0.002
Height (cm)	164.08 ± 7.73	164.68 ± 8.34	0.242	163.59 ± 7.16	165.59 ± 7.95	0.058
Weight (kg)	67.52 ± 12.92	68.41 ± 12.03	0.151	67.53 ± 13.36	69.61 ± 11.14	0.050
SBP (mmHg)	132.01 ± 19.28	136.74 ± 19.09	0.001	131.49 ± 17.51	138.86 ± 20.30	0.008
DBP (mmHg)	82.89 ± 13.03	83.31 ± 12.05	0.348	83.55 ± 10.51	84.18 ± 12.27	0.934
HbA1c (%)	5.98 ± 0.84	6.32 ± 1.23	< 0.01	5.86 ± 0.68	6.27 ± 1.19	< 0.01
WBCs (109/L)	6.30 ± 4.88	6.39 ± 6.16	0.404	5.93 ± 1.75	6.41 ± 2.14	0.052
Hemoglobin (g/L)	131.44 ± 18.15	132.68 ± 16.41	0.472	129.64 ± 15.41	133.04 ± 15.38	0.044
Platelets (109/L)	214.24 ± 54.53	207.63 ± 58.75	0.093	213.05 ± 50.87	215.48 ± 59.41	0.860
Creatinine (μmol/L)	60.09 ± 12.71	70.33 ± 62.10	< 0.01	60.54 ± 13.12	67.10 ± 17.15	< 0.01
UA (μmol/L)	306.30 ± 83.36	315.82 ± 83.04	0.149	309.57 ± 81.02	323.58 ± 83.94	0.128
TC (mmol/L)	4.13 ± 0.98	4.12 ± 1.00	0.878	4.21 ± 1.12	4.19 ± 1.23	0.767
TG (mmol/L)	1.65 ± 0.92	1.67 ± 0.99	0.565	1.86 ± 2.12	1.71 ± 1.04	0.338
HDL-C (mmol/L)	1.16 ± 0.26	1.13 ± 0.28	0.064	1.20 ± 0.30	1.09 ± 0.27	< 0.01
LDL-C (mmol/L)	2.05 ± 0.67	2.14 ± 0.71	0.170	2.05 ± 0.71	2.15 ± 0.70	0.325
Lp-PLA2 (ng/mL)	140.47 ± 77.93	160.25 ± 124.42	0.107	152.27 ± 128.51	157.82 ± 127.43	0.397
LVEF (%)	62.61 ± 5.33	62.09 ± 5.51	0.206	63.42 ± 5.52	62.04 ± 5.00	0.007
Sex			< 0.01			0.003
Male	73 (38.02%)	297 (56.25%)		36 (40.91%)	130 (59.91%)
Female	119 (61.98)	231 (43.75)		52(59.09)	87 (40.09)
Symptoms			< 0.01			< 0.01
No	149 (77.60%)	320 (60.61%)		74 (84.09%)	125 (57.60%)
Yes	43 (22.40%)	208 (39.39%)		14 (15.91%)	92 (42.40%)
Hypertension			0.001			0.002
No	112 (58.33%)	235 (44.51%)		52 (59.09%)	86 (39.63%)
Yes	80 (41.67%)	293 (55.49%)		36 (40.91%)	131 (60.37%)
Diabetes			< 0.01			0.033
No	176 (91.67%)	414 (78.41%)		81(92.05%)	179 (82.49%)
Yes	16 (8.33%)	114 (21.59%)		7 (7.95%)	38 (17.51%)
Smoking history			0.029			0.127
No	153 (79.69%)	378 (71.59%)		67 (76.14%)	146 (67.28%)
Yes	39 (20.31%)	150 (28.41%)		21 (23.86%)	71 (32.72%)

DBP, diastolic blood pressure; HbA_1c, hemoglobin A_1c; HDL-C, high-density lipoprotein cholesterol; LDL-C, low-density lipoprotein cholesterol; Lp-PLA₂, lipoprotein-associated phospholipase A₂; LVEF, left ventricular ejection fraction; SBP, systolic blood pressure; TC, total cholesterol; TG, triglycerides; UA, uric acid; WBCs, white blood cells.

Nomogram Development and Validation

With the first step of data analysis, forward multivariate logistic regression analysis was used to calculate model A’s risk elements. They are shown in Table 3, and were considered to be significant statistically when associated with CAD. Some risk factors’ cross-sectional points were computed with optimal scaling regression (CATREG) in SPSS Statistics. In the development cohort, we obtained model B with the AIC and model C with the risk factors of the FRS. The variables x_i and the coefficients β_i are shown in the Appendix. SBP, HbA_1c level, serum creatinine level, clinical symptoms, hypertension, diabetes, and smoking were categorical variables, and the other factors were continuous variables.

Table 3

Univariate and Multivariate Logistic Regression Analysis in the Development Group.

Risk factor	Univariate logistic analysisc			Multivariate logistic analysis
	OR	95% CI	P	OR	95% CI	P
Age per 10 years	1.41	1.19–1.67	< 0.01	1.44	1.20–1.72	< 0.01
Sex (female vs. male)	0.48	0.34–0.67	< 0.01	0.36	0.25–0.52	< 0.01
Height	1.01	0.99–1.03	0.389	NA
Weight	1.01	0.99–1.02	0.390	NA
SBP				1.52	1.05–2.21	0.027
≤105 mmHg			0.020
106–145 mmHg	2.06	0.94–4.51	0.071
>146 mmHg	3.02	1.31–6.96	0.009
HbA1c				NA
≤5.8%			< 0.01
5.81–7.3%	1.52	1.07–2.15	0.018
>7.3%	4.35	2.10–9.00	< 0.01
Creatinine				NA
≤53.33 μmol/L			< 0.01
53.34–57.10 μmol/L	1.68	1.07–2.62	0.023
>57.10 μmol/L	2.19	1.45–3.30	< 0.01
UA	1.001	0.99–1.00	0.175	NA
HDL-C	0.64	0.35–1.15	0.137	NA
LDL-C	1.20	0.94–1.53	0.145	NA
Lp-PLA2 per 100	1.21	1.00–1.46	0.045	1.26	1.03–1.54	0.024
LVEF	0.98	0.95–1.01	0.263	NA
Symptoms (atypical vs. typical)	2.25	1.54–3.30	< 0.01	2.28	1.53–3.41	< 0.01
Hypertension (no vs. yes)	1.75	1.25–2.44	0.001	1.51	1.05–2.18	0.027
Diabetes (no vs. yes)	3.03	1.74–5.26	< 0.01	2.85	1.60–5.08	< 0.01
Smoking history (no vs. yes)	1.55	1.04–2.31	0.030	NA

CI, confidence interval; HbA_1c, hemoglobin A_1c; HDL-C, high-density lipoprotein cholesterol; LDL-C, low-density lipoprotein cholesterol; Lp-PLA₂, lipoprotein-associated phospholipase A₂; LVEF, left ventricular ejection fraction; NA, no answer; OR, odds ratio; SBP, systolic blood pressure; UA, uric acid.

We drew the ROC curves of predicted CAD probability (Figure 1) and calculated the AUC, NRI, and IDI in the development and validation groups. The AUC of the three models is shown in Table 4, with the Delong test used for comparing AUCs. The AUC in model B was 0.739 in the development group and 0.747 in the validation group, the model B was the best one of the models. The difference between the values was statistically significant (P < 0.05). The NRI and IDI are compared between model A and model B, between model A and model C, and between model B and model C in Table 5 by means of the Hosmer-Lemeshow test. The cutoff points of the comparison between two models for the NRI and IDI were obtained from the tangent points of the ROC curves. We chose the best one from both ROC curves for overall consideration with the R packages in the computer. The order of comparison could not be changed. For example, A ~ C means model A was the newly developed model and model C was the old model, which should be updated and calibrated with the change in the dietary habits of people even in a new population. If the order were reversed, the results for the evaluation index would be opposite for NRI and IDI. All of the results indicated that model B is the best of the three. The same was found in validation group.

Figure 1:

The ROC curves in development group (A) and in validation group (B).

Table 4

The Areas Under the Curve (AUC) in the Development Group and the Validation Group.

Model	Development group			Validation group
	AUC	95% CI	P	AUC	95% CI	P
A	0.725	0.68–0.77		0.747	0.69–0.81
B	0.739	0.70–0.78		0.733	0.67–0.80
C	0.699	0.66–0.74		0.692	0.63–0.76
A−B	0.014	−0.00 to 0.03	0.085	0.015	−0.01 to 0.04	0.207
A−C	0.026	0.00–0.05	0.049	0.055	0.02–0.09	0.005
B−C	0.040	0.01–0.07	0.003	0.041	0.002–0.08	0.041

CI, confidence interval.

Table 5

The Net Reclassification Improvement (NRI) and Integrated Discrimination Improvement (IDI) in the Development Group and the Validation Group.

Model	Development group			Validation group
	Cutoff*	NRI (categorical)	P	Cutoff	NRI (categorical)	P
A~C	0.72	0.050	0.143	0.73	0.172	0.110
B~C	0.81	0.081	0.011	0.79	0.124	0.029
B~A	0.81	0.049	0.052	0.79	0.037	0.082
		NRI (continuous)			NRI (continuous)
A~C	–	0.298	< 0.01	–	0.383	< 0.01
B~C	–	0.384	< 0.01	–	0.445	< 0.01
B~A	–	0.145	0.010	–	0.114	0.566
		IDI			IDI
A~C	0.72	0.034	< 0.01	0.73	0.066	< 0.01
B~C	0.81	0.051	< 0.01	0.79	0.071	< 0.01
B~A	0.81	0.018	0.002	0.79	0.005	0.334

*The cutoff point was one of the best points obtained from the tangent points of both receiver operating characteristic curves.

The calibration of the models was expressed as another aspect about ROC curves, which is shown in Figure 2. The P values from the calibration plots of the three models were 0.987, 0.196, and 0.098, respectively, with the Hosmer-Lemeshow test. Hence, the predicted probability of the models revealed model B has strong concordance performance consistent with the actual probability in the development group. In the validation group, the P values were 0.119, 0.008, and 0.533 respectively.

Figure 2:

The calibration plots in different models. (A was the plot of model A in development group; B was the plot of model B in development group; C was the plot of model C in development group; so as in validation group, they were A, B and C respectively).

Discrimination and calibration play important roles in the appraisal of CPMs, but they cannot reveal which prediction model is clinically useful [11]. If one wishes to use a prediction model to guide clinical diagnosis and treatment, decision curve analysis (DCA) should be used. The DCA has the advantage of evaluating alternative diagnostic and prognostic strategies [12]. Vickers et al. [13] describe the details. The DCA of models A, B, and C (Figure 3) showed the greatest net benefit of model B for physicians and patients.

Figure 3:

The DCA in development group (A) and in validation group (B).

From the analysis above, we established an individualized nomogram prediction model based on model B (shown in Figure 4) because model B was the best in aggregate.

Figure 4:

Nomogram to predict the probability of CAD.

Discussion

As far as we know, there have been no studies of CPMs with CAD reporting the models from so many aspects. Previous studies just developed a new model rather than contrasted it with the traditional model or performed recalibration except for new risk factors or only validated predicted models. No study showed that the FRS could predict the presence of CAD compared with CAG. Also, the CMCS prediction model from China and the European Risk Score both evaluate the risk of CAD at 10 years or more. In contrast, we hoped to obtain an immediate CPM with hospitalized patients with suspected diagnosis of CAD. In our study, we developed two new prediction models and compared them with the traditional FRS, resulting in a simple nomogram with nine risk factors: sex, age, SBP, height, LDL-C, Lp-PLA₂, clinical symptoms, hypertension, and diabetes. The discrimination for risk prediction of model B was similar for c-statistics in the development group and the validation group (P = 0.739 and P = 0.733), which provided adequate goodness (P = 0.88).

In the development group, the AUC of model B was greater than that of model A, but the opposite was found in the validation group, but both AUC values were greater than the AUC values in model C. To evaluate the differences in diagnostic accuracy between model A and model B, NRI and IDI can be used [14]. In the development group, the category-based and continuity-based NRIs of model A versus model C, model B versus model C, and model B versus model A were all positive, showing that model A is better than model C, model B is better than model C, and model B is better than model A. All of the P values for IDI have statistical significance (P < 0.05) and indicate that model B is the best. Pencina et al. [15] stated that NRI and IDI could be viewed as a universal measure of effect size with an example based on the Framingham Heart Study risk assessment model. With the less sample of validation population got a distinction. So the studies of CPMs should be involved in more numbers of patients as many as possible.

DCA is a novel algorithm for a predictive model’s utility for clinical decision making [16], and can assess the performance in clinical scenarios at a range of threshold probabilities. DCA has been considered as an alternative to the AUC [17]. Among our models, model B will be of more benefit for patients, because its point of intersection with the X coordinate is the largest of the three models. The weighted area under DCA will be given the improvement of the model comparison in various clinical scenarios in the future [18].

Recently, Lp-PLA₂ level has been used in diagnosis of cardiovascular events as a new biomarker of vascular disorders [19]. Our study showed the odds ratio for Lp-PLA₂ was 1.26, which suggests it is an important risk criterion. Mayala et al. [20] pointed out that there is a role for LDL-C levels in predicting the development of coronary microvascular dysfunction and CAD. That was one of the reasons why we chose model B rather than model A.

The Framingham Heart Study has paved the way for further epidemiological research in preventive cardiology. The Diamond-Forrester model and the FRS do not consider the type of angina [21], which is less of a risk factor than diabetes, but a greater risk factor that the other risk factors in our study. A prediction model developed with age, sex, chest pain, diabetes mellitus, smoking, and lipid levels based on the Chinese population showed that the strict identification of typical angina is beneficial to further optimize this model [22].

A nomogram was used to predict the probability of CAD via the values of every variable on each axis added together to give the total number of points. A total of 2,431 patients with suspected CAD and who underwent coronary angiography were used to develop an intuitive and practical nomogram with the total number of plaques and the area of maximum soft, hard, and mixed plaques [23]. In comparison with predicted risk scores, nomograms are easier and more convenient to apply for disease diagnosis and treatment. They are widely used, for example, for cancer prognosis [24].

Conclusion

We developed an individualized nomogram prediction model for diagnosis of CAD by comparison of three models. The AIC was better in screening risk factors. The proposed CPM of CAD should be evaluated and adjusted with new biomarkers or a larger population in future studies.

Study Limitations

All of the patients came from one medical center. The validation group in our study showed the repeatability of the risk prediction model but multicenter hospital data are need to determine the ­universality. Next, the study was a retrospective, case-control study, and the participants who underwent CAG with suspected CAD may cause a selection bias in diagnosis, which is impossible to avoid. The differing clinical experience of physicians could be another bias.

Conflict of Interest

The authors declare that they have no conflict of interest.

Contributions

Zesen Han was responsible mainly for collection and arrangement of the data and wrote the report. Zhaokun Pu and Lan Yang were participated in data collection and discussed the study with Zesen Han. All of the work was accomplished under the guidance of Lihong Lai.

Funding Statement

There was no funding.

Ethics Approval

This study was granted exemption from requiring ethics approval by the Ethics Committee of The First Affiliated Hospital of Clinical Medicine of Henan University of Science and Technology because this study was a retrospective observational study.

Data Availability

The deidentified participant data will be shared on a request basis.

Appendices

Appendix

Age per 10, Lp-PLA₂ per 100, height, TC, HDL-C, and LDL-C are continuous numbers, male sex is coded as 0 and female sex is coded as 1, SBP under 105 mmHg is coded as 0, SBP from 106 to 145 mmHg is coded as 1, and SBP greater than 145 mmHg is as coded 2, atypical angina is coded as 0 and typical angina is coded as 1, a history of hypertension is coded as 1 and no history of hypertension is coded as 0, a history of diabetes is coded as 1 and no history of diabetes is coded as 0, and a history of smoking is coded as 1 and no history of smoking is coded as 0.

Model A: f(x) = −2.107 + 0.362 × Age per 10 + (−1.019) × Sex + 0.421 × SBP + 0.233 × Lp-PLA₂ per 100 + 0.826 × Symptoms + 0.414 × Hypertension + 1.047 × Diabetes

Model B: f(x) = 5.035 + 0.348 × Age per 10 + (−1.602) × Sex + 0.445 × SBP + (−0.05) × Height + 0.324 × LDL-C + 0.231 × Lp-PLA₂ per 100 + 0.825 × Symptoms + 0.460 × Hypertension + 1.088 × Diabetes

Model C: f(x) = −1.969 + 0.398 × Age per 10 + (−1.061) × Sex + 0.485 × SBP + 0.142 × TC + (−0.070) × HDL-C + 1.144 × Diabetes + (−0.131) × Smoking

References

1. S Jia, Y Liu, J Yuan. Evidence in guidelines for treatment of coronary artery disease. Adv Exp Med Biol 2020;1177:37–73.

2. X Zheng, JP Curtis, S Hu, Y Wang, Y Yang, FA Masoudi, et al. Coronary catheterization and percutaneous coronary intervention in China: 10-year results from the China PEACE-Retrospective CathPCI study. JAMA Intern Med 2016;176(4):512–21.

3. M Kolossváry, B Szilveszter, B Merkely, P Maurovich-Horvat. Plaque imaging with CT-a comprehensive review on coronary CT angiography based risk assessment. Cardiovasc Diagn Ther 2017;7(5):489–506.

4. TH Lee, EF Cook, L Goldman. Prospective evaluation of a clinical and exercise-test model for the prediction of left main coronary artery disease. Med Decis Making 1986;6(3):136–44.

5. PW Wilson, RB D‘Agostino, D Levy, AM Belanger, H Silbershatz, WB Kannel. Prediction of coronary heart disease using risk factor categories. Circulation 1998;97(18):1837–47.

6. EW Steyerberg, Y Vergouwe. Towards better clinical prediction models: seven steps for development and an ABCD for validation. Eur Heart J 2014;35(29):1925–31.

7. J Liu, Y Hong, RB D‘Agostino Sr, Z Wu, W Wang, J Sun, et al. Predictive value for the Chinese population of the Framingham CHD risk assessment tool compared with the Chinese Multi-Provincial Cohort Study. J Am Med Assoc 2004;291(21):2591–9.

8. G Montalescot, U Sechtem, S Achenbach, F Andreotti, C Arden, A Budaj, et al. 2013 ESC guidelines on the management of stable coronary artery disease: the Task Force on the Management of Stable Coronary Artery Disease of the European Society of Cardiology. Eur Heart J 2013;34(38):2949–3003.

9. SE Kjeldsen, K Narkiewicz, S Oparil, T Hedner. 2013 European Society of Hypertension/European Society of Cardiology hypertension guidelines. Blood Press 2013;22(4):191–2.

10. SI Vrieze. Model selection and psychological theory: a discussion of the differences between the Akaike information criterion (AIC) and the Bayesian information criterion (BIC). Psychol Methods 2012;17(2):228–43.

11. EW Steyerberg, AJ Vickers, NR Cook, T Gerds, M Gonen, N Obuchowski, et al. Assessing the performance of prediction models: a framework for traditional and novel measures. Epidemiology 2010;21(1):128–38.

12. AJ Vickers, EB Elkin. Decision curve analysis: a novel method for evaluating prediction models. Med Decis Making 2006;26(6):565–74.

13. AJ Vickers, B van Calster, EW Steyerberg. A simple, step-by-step guide to interpreting decision curve analysis. Diagn Progn Res 2019;3:18.

14. J Li, B Jiang, JP Fine. Multicategory reclassification statistics for assessing improvements in diagnostic accuracy. Biostatistics 2013;14(2):382–94.

15. MJ Pencina, RB D‘Agostino Sr, OV Demler. Novel metrics for evaluating improvement in ­discrimination: net ­reclassification and integrated discrimination improvement for normal variables and nested models. Stat Med 2012;31(2):101–13.

16. B Van Calster, L Wynants, JFM Verbeek, JY Verbakel, E Christodoulou, AJ Vickers, et al. Reporting and interpreting decision curve analysis: a guide for investigators. Eur Urol 2018;74(6):796–804.

17. S Zastrow, S Brookman-May, TA Cong, S Jurk, I von Bar, V Novotny, et al. Decision curve analysis and external validation of the postoperative Karakiewicz nomogram for renal cell carcinoma based on a large single-center study cohort. World J Urol 2015;33(3):381–8.

18. R Talluri, S Shete. Using the weighted area under the net benefit curve for decision curve analysis. BMC Med Inform Decis Mak 2016;16:94.

19. A De Stefano, L Mannucci, F Tamburi, C Cardillo, F Schinzari, V Rovella, et al. Lp-PLA₂, a new biomarker of vascular disorders in metabolic diseases. Int J Immunopathol Pharmacol 2019;33:2058738419827154.

20. HA Mayala, Y Wang, J Hu, S-Y Liu, G-W Yi, C-X Qin, et al. Clinical characteristics and biomarkers of coronary microvascular dysfunction and obstructive coronary artery disease. J Int Med Res 2019:47(12):6149–59.

21. TS Genders, EW Steyerberg, H Alkadhi, S Leschka, L Desbiolles, K Nieman, et al. A clinical prediction rule for the diagnosis of coronary artery disease: validation, updating, and extension. Eur Heart J 2011;32(11):1316–30.

22. H Xu, Z Duan, C Miao, S Geng, Y Jin. Development of a diagnosis model for coronary artery disease. Indian Heart J 2017;69(5):634–9.

23. N Wu, X Chen, M Li, X Qu, Y Li, W Xie, et al. Predicting obstructive coronary artery disease using carotid ultrasound parameters: a nomogram from a large real-world clinical data. Eur J Clin Invest 2018;48(8):e12956.

24. A Iasonos, D Schrag, GV Raj, KS Panageas. How to build and interpret a nomogram for cancer prognosis. J Clin Oncol 2008;26(8):1364–70.