At 5 years old, Kilgrav had kidnapped you, he just felt very attached to you and he didn't use his powers on you. To him, you should be the child he never had, and no one else's.

You grew up with him manipulating you, making you believe that he was always your father and that you had just lost your memory. Since you were kidnapped, you have been living in another city where no one would recognize you. You could leave the house without any problems, and now you had just returned from a walk.