A novel multi-timescale analysis method, Empirical Mode Decomposition (EMD), is used to diagnose the variation of the annual mean temperature data of the global, Northern Hemisphere (NH) and China from 1881 to 2002. The results show that: (1) Temperature can be completely decomposed into four timescales quasi-periodic oscillations including an ENSO-like mode, a 6–8-year signal, a 20-year signal and a 60-year signal, as well as a trend. With each contributing ration of the quasi-periodicity discussed, the trend and the 60-year timescale oscillation of temperature variation are the most prominent. (2) It has been noticed that whether on century-scale or 60-year scales, the global temperature tends to descend in the coming 20 years. (3) On quasi 60-year timescale, temperature abrupt changes in China precede those in the global and NH, which provides a denotation for global climate changes. Signs also show a drop in temperature in China on century scale in the next 20 years. (4) The dominant contribution of CO2 concentration to global temperature variation is the trend. However, its influence weight on global temperature variation accounts for no more than 40.19%, smaller than those of the natural climate changes on the rest four timescales. Despite the increasing trend in atmospheric CO2 concentration, the patterns of 20-year and 60-year oscillation of global temperature are all in falling. Therefore, if CO2 concentration remains constant at present, the CO2 greenhouse effect will be deficient in counterchecking the natural cooling of global climate in the following 20 years. Even though the CO2 greenhouse effect on global climate change is unsuspicious, it could have been excessively exaggerated. It is high time to re-consider the trend of global climate changes.