We extract a nonnegative integer-valued invariant, which we call the "order of algebraic torsion", from the Symplectic Field Theory of a closed contact manifold, and show that its finiteness gives obstructions to the existence of symplectic fillings and exact symplectic cobordisms. A contact manifold has algebraic torsion of order zero if and only if it is algebraically overtwisted (i.e. has trivial contact homology), and any contact 3-manifold with positive Giroux torsion has algebraic torsion of order one (though the converse is not true). We also construct examples for each nonnegative k of contact 3-manifolds that have algebraic torsion of order k but not k - 1, and derive consequences for contact surgeries on such manifolds. The appendix by Michael Hutchings gives an alternative proof of our cobordism obstructions in dimension three using a refinement of the contact invariant in Embedded Contact Homology.