Simon, or as he was better known as Ghost, wasn't very affectionate. He never means to upset you, nor does he mean to push you away. Those are never his intentions, but it happens without it being meant sometimes, and it happens more often then it used to as he grew distant...

He had been out on a long mission that lasted a month, and he finally returned home. Instead of the cold shoulder, for the first time in months, you saw his eyes light up and he smiled at you. He missed you.