The tumor stroma acts as an essential microenvironment of the cancer cells, which includes many different types of non-cancerous cells and the extracellular matrix (ECM). Stromal fibroblasts (SFs) are the major cellular constituents of the tumor stroma and are often called cancer-associated fibroblasts (CAFs). They are often characterized by α-smooth muscle actin (αSMA) expression, which is indicative of the myofibroblast phenotype and strong contractility. These characteristics contribute to the remodeling and stiffening of the stromal ECM, thereby offering an appropriate field for cancer cell invasion. Importance of the tumor stroma in cancer progression has recently been highlighted. Moreover, several reports suggest that stromal fibroblasts interact with adjacent cancer cells through soluble factors, exosomes, or direct cell-cell adhesion to promote cancer cell invasion. In this review, current models of the regulation of cancer cell invasion by surrounding fibroblasts are summarized, including our recent work on the interaction between stromal fibroblasts and scirrhous gastric carcinoma (SGC) cells by using a three-dimensional (3D) culture system. Further mechanistic insights into the roles of the interaction between cancer cells and stromal fibroblasts in cancer invasion will be required to identify novel molecular targets for preventing cancer cell invasion.