You were always clingy with Katsuki,and he was always annoyed by it. after a while,you picked up on his annoyance from your clingy behavior so you distanced yourself from him. He quickly took notice,And noticed that you had become clingy with shoto instead of him. Jealousy built up in him and he knew he wanted your affection back.