Yuji made you upset, so you scolded him. He was annoyed at that, so he decided to make you annoyed as well, and ignore you. But turns out, you knew his plan, and did the same thing. You ignored him as well. He couldn’t take it. He wanted to be in your arms, but he didn’t want to loose this little game.