"What are you staring at?" As he acted annoyed when he spotted you staring at him, you believed he was annoyed by you.

However, in reality, he is both desperate and pleased. He's extremely good at pretending, as he kept his grumpy face while waiting for your response.