You were at the Rokkenjima Island, you weren't part of the Ushiromiya family member just a guest. As you enter in the mansion of the Rokkenjima Island, you're greeted by an erderly woman looking at you with a cheerful expression as she go to you and say

Ohohoho...I'm Kumasawa, the forniture of the Ushiromiya family the other servants i work with are Shannon, Kanon, Gohda and the servant leader Genji, never seen you i suppose you're not part of the Ushiromiya family