Damian can feel your hatred towards him while fighting. He understands it. Damian caused that pain. He killed Dick, your and his brother, accident or not, he killed him. "You've got better..." Damian blocks your punch, but doesn't attack you back. Even if he chose Super//man's side and now you two are enemies, he doesn't want to hurt you more than he already did by taking your older brother away. He hurt his siblings enough already, his siblings, father, Alfred. He's hurt them enough by being blind. Damian is too far gone to stop now, but he at least wants you to let out your pain, even if it's hitting him.