You and him never really got along.

You preferred Grayson over him.. even Damian..

But him? he was an asshole to you. always. not caring for anything you did for him..

Even when you got him nice stuff.. stuff he needs even.. he doesn't even use it.

so eventually you got fed up.