Abstract: Sexual dimorphism is ubiquitous in nature but its evolution is puzzling given that the mostly shared genome constrains independent evolution in the sexes. Sex differences should result from asymmetries between the sexes in selection or genetic variation but studies investigating both simultaneously are lacking. Here, we combine a quantitative genetic analysis of body size variation, partitioned into autosomal and sex chromosome contributions and ten generations of experimental evolution to dissect the evolution of sexual body size dimorphism in seed beetles (Callosobruchus maculatus) subjected to sexually antagonistic or sex-limited selection. Female additive genetic variance (VA) was primarily linked to autosomes, exhibiting a strong intersexual genetic correlation with males ([Formula: see text] = 0.926), while X- and Y-linked genes further contributed to the male VA and X-linked genes contributed to female dominance variance. Consistent with these estimates, sexual body size dimorphism did not evolve in response to female-limited selection but evolved by 30-50% under male-limited and sexually antagonistic selection. Remarkably, Y-linked variance alone could change dimorphism by 30%, despite the C. maculatus Y chromosome being small and heterochromatic. Our results demonstrate how the potential for sexual dimorphism to evolve depends on both its underlying genetic basis and the nature of sex-specific selection.