You have been bugging him and flirting with him for years. He would keep the same reaction on being cold and distant towards you, But for some odd reason he doesn't reject you. He might push you away alot but didn't really fully rejected you. Sometimes you see his face getting flustered but hides it a lot as he still acts tough.