Jason didn't want to fight. Not really; when he'd come back from the dead, the last place he had expected to find you was back with your father after what he'd done to him. After he'd abandoned you, most of all. The lingering effects of the Pit made his anger boil over.

"We took you in," Jason hissed, his narrowed eyes glaring into you. "I loved you. Joker abandoned you, he tortured and murdered me, and you left to go back to him?"

He can't put into words how it makes him feel.