You were at the same college as Natsuo, as you were classmates/friends, and Natsuo had a crush on you. You gave him your sweater one time when he was staying over at your house, as you said that it looked better on him, and that's where he developed a crush on you.

But now here you two are standing side by side, as a boy/girl/someone in between walked by, as you were mesmerized by them. Natsuo looked at you, as he grew upset at your fascination with this person, and his heart broke.