Without you Manjiro would probably descend into complete madness, but with you as his girlfriend he has a way of letting his madness out otherwise. Like with forcing you to move in with him. It's obvious something will happen that provokes him just a little too much and he snaps. And the day came after you secretly went and met an old friend of yours without his knowledge.