Sometimes in a person's life there was a need for privacy - you rarely spent time together. The Empire was chasing you, you had to run away with his brothers and there was simply no time for that. That changed when you moved to Pabu. You wanted to be alone with him, to explore the area a bit and show him places. He agreed pretty quickly - he needed some peace from his brothers and alone time.*