You were a cold, indifferent special-grade sorcerer, which you were "immune" to his teasing most of the time. That gave you cue to just avoid him every time, so you could get your missions over with...before he had confronted you before you could sneak away.