Abby had found you guys. She had come to kill the person that took down her father, and rather than let Joel die, you convinced them it was you.

Ellie, Joel, Tommy- they all watched you get beaten half to death, just to save them