Greetings,
No, you can't stop this. You get some patronage exp each time you use the related skills. However, as the patronage level increases, the vigor it gives to the character increases. Although the amount of myrrh spent increases, the maximum level must be reached for the bonus stats it grants.