Background: Diabetes mellitus (DM), particularly if insulin-dependent, is a predictor of increased perioperative risk, whereas stringent metabolic control with insulin is beneficial in the critically ill. Methods: The impact of oral hypoglycemics (OH) vs. insulin on outcome was determined as a secondary retrospective analysis of a cohort study in patients with coronary artery disease (CAD) and DM undergoing major non-cardiac surgery. Primary end-point was 2-year all-cause mortality; secondary endpoints were perioperative myocardial ischemia and 2-year cardiac mortality. Results: Of 173 patients, DM was diagnosed in 42 (24%) based on pre-existing treatment with OH (15%) or insulin (9%). During follow-up, 40/173 (23%) patients died. All-cause mortality was similar in the non-diabetic (20%) and insulin groups (19%) but significantly higher in the OH group (42%; p = 0.025). Cardiac mortality tended to be higher in the OH group compared with the insulin and non-diabetic groups (27 vs. 19% and 11%, respectively; p = 0.066). Multivariate analysis revealed renal failure (odds ratio [OR] = 4.9, 95% confidence interval [CI] = 1.8–13.0), treatment with OH (OR = 3.3, 95% CI = 1.2–9.0), peripheral vascular surgery (OR = 2.7, 95% CI = 1.2–6.0), and prior diuretic therapy (OR = 2.6, 95% CI = 1.1–5.7) being independently associated with 2-year all-cause death. No difference existed in postoperative ischemia among the different groups. Conclusions: Long-term mortality after major non-cardiac surgery is elevated in patients with CAD and diabetes mellitus only if they are treated with OH, but not if they are treated with insulin. Further evaluation of the impact of perioperative anti-diabetic treatment on morbidity and mortality in CAD is warranted.