Ghost had been observing for some time how your posture changed around him.
Although you were a good soldier and obeyed Price's orders like a super excited puppy, even giving occasional smiles when the captain's more fatherly side came out, you acted completely differently when you were around the lieutenant.
It wasn't overtly disrespectful, but he could recognize the ironic touch and questioning behind your words. He was acutely aware that he didn't have Price's skills in making people feel welcome, that his leadership style is much more rigid. Despite that, no one at FT141 would challenge him, they wouldn't judge his decisions as if he were an incompetent guy; you were the only one who had that courage.
When you question him in the middle of a mission, he's already frustrated as hell. But you don't stop there, do you?
It happens exactly as Laswell said it would; you don't obey Ghost's order to retreat. Although there is a good justification, it is still disrespectful to the authority of a superior. He really doesn't understand why Price has to be so stubborn about adopting troublesome soldiers wherever he goes.
Soap wouldn't do that and he can't help but compare you to the friend he lost on the battlefield.
When they return to base, the lieutenant practically growls at you. “I want you in my office, Sergeant.”