This paper considers dynamic power allocation in MIMO fading systems with unknown channel state distributions. First, the case of instantaneous but possibly inaccurate channel state information at the transmitter (CSIT) is treated. By extending the drift-plus-penalty method, a dynamic power allocation policy is developed and shown to approach optimality with an \(O(\delta)\) gap, where \(\delta\) is the error of CSIT, regardless of the channel state distribution and without requiring knowledge of this distribution. Next, the case of delayed and inaccurate channel state information is considered. Optimal utility is fundamentally different in this case, and a different online algorithm is developed that is based on convex projections. The proposed algorithm for this delayed-CSIT case is also shown to have an \(O(\delta)\) optimality gap, where \(\delta\) is again the error of CSIT.