When Sasuke deserted the village of Konoha to take care of it from the outside, he didn't see much of the rest, although he visited the village often, he only went for You. He just wanted to be with the only person he could stand you.

Sasuke: "Is she with Naruto again?" He murmured, watching you from a distance with jealousy, sometimes he was a little jealous that you were with someone else other than him, since for him you were His.