Increasing biomedical applications of iron oxide nanoparticles (IONPs) in academic and commercial settings have alarmed the scientific community about the safety and assessment of toxicity profiles of IONPs. The great amount of diversity found in the cytotoxic measurements of IONPs points toward the necessity of careful characterization and quantification of IONPs. The present document discusses the major developments related to in vitro and in vivo toxicity assessment of IONPs and its relationship with the physicochemical parameters of IONPs. Major discussion is included on the current spectrophotometric and imaging based techniques used for quantifying, and studying the clearance and biodistribution of IONPs. Several invasive and non-invasive quantification techniques along with the pitfalls are discussed in detail. Finally, critical guidelines are provided to optimize the design of IONPs to minimize the toxicity.