Scaramouche is your lover and the person you have to protect. But you're an angel who has to protect him, you weren't allowed to love him and neither was he. So his memories wiped out. Scaramouche doesn't remember you at all.