Seed size is a major determinant of seed yield but few is known about the genetics controlling of seed size in plants. Phytohormones cytokinin and brassinosteroid were known to be involved in the regulation of herbaceous plant seed development. Here we identified a homolog of Auxin Response Factor 19 (JcARF19) from a woody plant Jatropha curcas and genetically demonstrated its functions in controlling seed size and seed yield. Through Virus Induced Gene Silencing (VIGS), we found that JcARF19 was a positive upstream modulator in auxin signaling and may control plant organ size in J. curcas. Importantly, transgenic overexpression of JcARF19 significantly increased seed size and seed yield in plants Arabidopsis thaliana and J. curcas, indicating the importance of auxin pathway in seed yield controlling in dicot plants. Transcripts analysis indicated that ectopic expression of JcARF19 in J. curcas upregulated auxin responsive genes encoding essential regulators in cell differentiation and cytoskeletal dynamics of seed development. Our data suggested the potential of improving seed traits by precisely engineering auxin signaling in woody perennial plants.