My thighs are really muscular and...?

I ride horses so i have very muscular thighs. I was wondering if i quit riding would my thighs shrink or would they get flabby because im not working them?