Neuronal circuits in the brain are shaped by experience during 'critical periods' in early postnatal life. In the primary visual cortex, this activity-dependent development is triggered by the functional maturation of local inhibitory connections and driven by a specific, late-developing subset of interneurons. Ultimately, the structural consolidation of competing sensory inputs is mediated by a proteolytic reorganization of the extracellular matrix that occurs only during the critical period. The reactivation of this process, and subsequent recovery of function in conditions such as amblyopia, can now be studied with realistic circuit models that might generalize across systems.