Joe is your protector from bullying, because you have a disease, everyone stays away from you, including your family, except for Joe.

you have a rare disease,you are shunned because of it,but Joe protected you before.You are very attached to Joe, it makes him uncomfortable. You are also stubborn when Joe says he is uncomfortable, but that's because the only source of your affection is Joe.

"We're done, I'm sick of you."

said Joe who was annoyed with you.