Recent experimental progress in table-top experiments or gravitational-wave interferometers has enlightened the unique displacement sensitivity offered by optical interferometry. As the mirrors move in response to radiation pressure, higher power operation, though crucial for further sensitivity enhancement, will however increase quantum effects of radiation pressure, or even jeopardize the stable operation of the detuned cavities proposed for next-generation interferometers. The appearance of such optomechanical instabilities is the result of the nonlinear interplay between the motion of the mirrors and the optical field dynamics. In a detuned cavity indeed, the displacements of the mirror are coupled to intensity fluctuations, which modifies the effective dynamics of the mirror. Such "optical spring" effects have already been demonstrated on the mechanical damping of an electromagnetic waveguide with a moving wall, on the resonance frequency of a specially designed flexure oscillator, and through the optomechanical instability of a silica micro-toroidal resonator. We present here an experiment where a micro-mechanical resonator is used as a mirror in a very high-finesse optical cavity and its displacements monitored with an unprecedented sensitivity. By detuning the cavity, we have observed a drastic cooling of the micro-resonator by intracavity radiation pressure, down to an effective temperature of 10 K. We have also obtained an efficient heating for an opposite detuning, up to the observation of a radiation-pressure induced instability of the resonator. Further experimental progress and cryogenic operation may lead to the experimental observation of the quantum ground state of a mechanical resonator, either by passive or active cooling techniques.
