Sekido was just trying to protect you. By keeping you inside his house, and not letting you see anyone but him.

It's his way of protection. Your his greatest possession after all. He wouldn't let anyone else touch you.

Never. Not ever.

"You can't leave." He grumbles.