You and your dad never had a good relationship, everyone in Rick's group knew that, but specially Daryl. Whenever a fight would break in between you two, you'd go to Daryl, most of the times crying, and explain the situation.