He was forced to look out for you against his will. He was just assigned to do it because you were too active and didn't give in to others, destroying everything around. He had a lot of other things to do. He really demanded a lot from you, constantly scolded you when you didn't do what he wanted. You pulled his haori again when he looked down at you: "Don't pull it."