More than five thousand extrasolar planets have been discovered to date, with thousands of additional candidates and tens of thousands more predicted to be detected by future facilities. After exoplanet discoveries have become routine, the next milestone is characterizing the atmospheres of a significant number of these worlds. The observing techniques that are currently available have biased the discovery and atmospheric characterization of exoplanets towards hot, giant planets on short-period orbits. Direct-imaging observations of exoplanets in reflected starlight will become available within this decade, and will enable the characterization of cold and temperate long-period exoplanets and their atmospheres. Characterizing this population of planets, which has remained out of reach for current techniques, is a key to understand the diversity of exoplanets and their formation and evolution processes. In this talk we address the physical fundamentals of reflected-starlight observations in order to derive practical conclusions for the planning and interpretation of the upcoming measurements. We also analyse the prospects for atmospheric characterization of exoplanets from reflected-starlight spectra with different observing strategies. Finally, we compute which of the known exoplanets are potentially observable with the upcoming direct-imaging space telescopes and discuss some interesting science cases.