Coffee and caffeine consumption are thought to increase the risk of bladder cancer. However, few studies have stratified this risk by smoking status, which is a potential confounder. Here, we investigated the association between coffee, green tea (another major source of caffeine), and caffeine, and bladder cancer incidence in relation to smoking status. We conducted a population-based prospective study in a cohort of Japanese, comprising a total of 49 566 men and 54 874 women aged 40–69 years who reported their coffee and green tea consumption at baseline. During follow-up from 1990 through 2005, 164 men and 42 women were newly diagnosed with bladder cancer. Cigarette smoking was associated with an increased risk of bladder cancer, with a strong dose–response relationship. Coffee was positively associated with bladder cancer risk in men, without statistical significance. When stratified by smoking status, coffee and caffeine consumption were associated with an increased risk of bladder cancer in never- or former-smoking men, with hazard ratios (95% confidence interval) in the highest categories of coffee (one or more cups per day) and caffeine consumption compared with the lowest of 2.24 (95% CI = 1.21–4.16) and 2.05 (95% CI = 1.15–3.66), respectively. In conclusion, cigarette smoking was confirmed as a risk factor for bladder cancer. Coffee and caffeine may be associated with an increased bladder cancer risk in never or former smokers among Japanese men.