We propose a chiral quark model that incorporates vector mesons and apply it to the study of the hadron spectrum. We consider the contributions of vector mesons within the framework of hidden local symmetry. Our results demonstrate a significant improvement in the masses of ground state baryons, including the nucleon, Λc, and Λb. We successfully reproduce the masses of all 45 experimentally confirmed ground states of mesons and baryons. Furthermore, our predictions for 21 ground states align well with the results obtained from lattice QCD analyses. This work represents the first successful achievement of all 45+21 ground states of mesons and baryons using a single set of parameters.