St. Thomas treats this matter in the Secunda Secundae, question 96. Vain observance can be defined as magical superstition by which someone hopes for some advantage from a thing which neither supernaturally belongs to that end nor can naturally bring it about. It is called vain because either it does not obtain the intended effect, or, if it does obtain it, it brings more harm than advantage. For the person using it receives only some usefulness in bodily or temporal matters, but incurs damage to the soul.
I said “neither supernaturally” because those who expect an effect from God through means displeasing to God are vain.