BACKGROUND: In many places, daily mortality has been shown to increase after days with particularly high or low temperatures, but such daily time-series studies cannot identify whether such increases reflect substantial life shortening or short-term displacement of deaths (harvesting). OBJECTIVES: To clarify this issue, we estimated the association between annual mortality and annual summaries of heat and cold in 278 locations from 12 countries. METHODS: Indices of annual heat and cold were used as predictors in regressions of annual mortality in each location, allowing for trends over time and clustering of annual count anomalies by country and pooling estimates using meta-regression. We used two indices of annual heat and cold based on preliminary standard daily analyses: a) mean annual degrees above/below minimum mortality temperature (MMT), and b) estimated fractions of deaths attributed to heat and cold. The first index was simpler and matched previous related research; the second was added because it allowed the interpretation that coefficients equal to 0 and 1 are consistent with none (0) or all (1) of the deaths attributable in daily analyses being displaced by at least 1 y. RESULTS: On average, regression coefficients of annual mortality on heat and cold mean degrees were 1.7% [95% confidence interval (CI): 0.3, 3.1] and 1.1% (95% CI: 0.6, 1.6) per degree, respectively, and daily attributable fractions were 0.8 (95% CI: 0.2, 1.3) and 1.1 (95% CI: 0.9, 1.4). The proximity of the latter coefficients to 1.0 provides evidence that most deaths found attributable to heat and cold in daily analyses were brought forward by at least 1 y. Estimates were broadly robust to alternative model assumptions. CONCLUSIONS: These results provide strong evidence that most deaths associated in daily analyses with heat and cold are displaced by at least 1 y. https://doi.org/10.1289/EHP1756.