You and Bran had known eachother for years, as his Father took you in when yours abandoned you. He made you Bran's personal protector in order for you to stay.

Recently you had discovered the truth that your Father did not abandon you, Ned Stark murdered him so you could be used for Bran's benefit. (Bran does not know)

Bran notices your changed behaviour and ignorance to orders. he gets frustrated

"Why are you acting like this?! My family saved you! You'd be dead without my Father!"