Years 1938-1945, the years our dearest Tom spent his school years. The years you were sent back to. Dumbledore sent you back to time with the purpose of stopping Tom from what will he become. Dumbledore called you to his office countless times, asking how your mission is going. Everything went as expected, Tom and his group understand nothing and their suspicions of you decreased over time. Even Tom become warmer, almost affectionate. However, something unexpected happened. You develope feelings for Tom.
You were supposed to kill him by now, to stop him and his plans. Dumbledore start calling you to his office more frequently to speak about mission, asking when will you accomplish it. He's suspicious of your loyalty to him and mission. You have been putting off "Kill Tom" part with some miserable excuses. Like Tom isn't trusting you enough. Which is a total lie, he trust you with his life, he literally did. The infamous Tom is acting better with you, that surprised even Tom himself. The sudden fastening of his heart, the warmness in his body whenever he's with you is irrefutable. You loved him, he loved you more. The more time he spent with you, the more he find himself falling deeper for you. What he didn't know was your mission.
Tom is getting suspicious of Dumbledore calling you this frequently, of course he would. After another conversation with Dumbledore, you are now walking besides Tom to a library, to study.
"So, what was it about this time? Why did he called you to his office?"
Tom asked with a calm face, hiding the fact that he is rather offended and annoyed that you're doing something with Dumbledore that he doesn't know. He'd want to know what is happening and he doesn't know. He doesn't want to be cold with you or destroy the thing that you two have. He still trust you but he couldn't ignore the paranoid side of him. As much as he wants to, he couldn't ignore suspicions that grew with him.