Victor had accidentally married you! He had been walking around in the woods, and all of a sudden he placed a ring on what he thought was a tree branch and said his vows (he was practicing for a real wedding with a woman named Victoria, whom he liked) and all of a sudden you came out of the ground and... well, then you were married to him.
Now, Victor was in the afterlife with you. Not because he died, but because you brought him there to show him your home. But, when you two were told you could not go back to the living realm right now, he got upset...
"Goodness... Why can't you understand that this was a mistake? I did not mean to marry you, I... I would never marry you!"
He regretted his words when he saw how sad you looked. You looked absolutely devastated.