The present study was designed to develop a practical strategy to tackle the problem of lacking standard compounds and limited references for identifying structure-related compounds in Streptocaulon griffithii Hook. f., especially those in trace concentrations, with a focus on antitumor activity. The cardiac glycosides (CGs)-enriched part was determined using in vitro bioactive assays in three cancer cell lines and then isolated using macroporous resins. The MS and MS/MS data were acquired using a high performance liquid chromatography coupled with hybrid quadrupole-time of flight (HPLC-Q-TOF-MS) system. To acquire data of trace compound in the extract, a multiple segment program was applied to modify the HPLC-Q-TOF-MS method. A mass defect filter (MDF) approach was employed to make a primary MS data filtration. Utilizing a MATLAB program, the redundant peaks obtained by imprecise MDF template calculated with limited references were excluded by fragment ion classification, which was based on the ion occurrence number in the MDF-filtered total ion chromatograms (TIC). Additionally, the complete cleavage pathways of CG aglycones were proposed to assist the structural identification of 29 common fragment ions (CFIs, ion occurrence number ≥ 5) and diagnostic fragment ions (DFIs, ion occurrence number < 5). As a result, 30 CGs were filtered out from the MDF results, among which 23 were identified. This newly developed strategy may provide a rapid and effective tool for identifying structure-related compounds in herbal medicines.