We have made a series of gradient-corrected relativistic full-potential density-functional calculation for Ca-substituted and hole-doped SrRuO\(_3\) in para, ferro, and \(A\)-, \(C\)-, and \(G\)-type antiferromagnetic states. Magnetic phase-diagram data for Sr\(_{1-x}\)Ca\(_x\)RuO\(_3\) at 0 K are presented. Neutron diffraction measurement combined with total energy calculations show that spin-glass behavior with short-range antiferromagnetic interactions rules in CaRuO\(_3\). The substitution of Sr by Ca in SrRuO\(_3\) decreases the ferromagnetic interaction and enhances the \(G\)-type antiferromagnetic interaction; the \(G\)-AF state is found to stabilize around \(x\) = 0.75 consistent with experimental observations. Inclusion of spin-orbit coupling is found to be important in order to arrive at the correct magnetic ground state in ruthenates.