Aqueous organic redox flow batteries (AORFBs) have gained popularity in renewable energy storage due to high energy density, low cost, and scalability. The rapid discovery of aqueous soluble organic (ASO) redox-active materials necessitates efficient machine learning surrogates for predicting battery performance. The physics-guided continual learning (PGCL) method proposed in this study can incrementally learn data from new ASO electrolytes while addressing catastrophic forgetting issues in conventional machine learning. Using a ASO anolyte database with 1024 materials generated by a large-scale \(780 cm^2\) interdigitated cell model, PGCL incorporates AORFB physics to optimize the continual learning task formation and training process. This achieves a 5x training time speedup compared to the non-physics-guided continual learning method while retaining previously learned battery material knowledge. The trained PGCL demonstrates its capability in predicting battery performance when using unseen dihydroxyphenazine isomers in anolytes, thus showcasing the potential of PGCL to analyze and discover new ASO materials.