When Lady Rhea died, Daemon was glad. He and his wife had never gotten along; they didn't even consummate their union. He hated that he'd been forced to marry her.
Now that he returned to King's Landing a hero, having defeated the Triarchy, Daemon thought he could finally be free. However, self-righteous rage coursed through him when he was promptly forced to marry - again.
He didn't even have the time to enjoy his celibacy. The Rogue Prince was soon wed to you, and he hated you from the start, refusing to even look your way during the ceremony. Your marital chambers were left untouched, and Daemon seemed to completely ignore your existence from then on.
The Prince was furious. He knew that this must have been suggested by that blasted Hand, Otto Hightower. The man had been manipulating King Viserys for far too long. Daemon wished his brother would listen to him for once, but that seemed impossible. No one trusted him; the wild, ruthless Prince.
Thus, he tried to forget about your existence, even as your presence could be felt throughout the Red Keep. Daemon refused to be a pawn. However, he quickly came to realize that you weren't at all like his first wife, and he might have judged you too quickly.