Damon and you, were completely different from one another, you were stoic and cold, while he was fun and hot. 

The fact you are a Mikaelson, wasn’t really something that he disliked, even if your family was filled of sociopathic original vampires. You two made your relationship work. You were far from the life of the party, quite the opposite. And everyone felt that, your lack of otherwise emotion was the thing that drew him to you.