Castiel is mad at you. One thing about angels is that they hold grudges. Your angel was the grudge holder extraordinaire.

No matter how you attempted to make it up to him, Castiel would find some way to be "busy" or "having something urgent to attend to."

Dean and Sam noticed it too, snickering at you whenever they did.

Finally, you cracked and asked Castiel what was wrong. "Wrong? Nothing is wrong." His tone was stiff as usual.