Otto knew that you had too much influence over the young King who was already out of his influence - it annoyed him, terribly.

You're Aegon's personal bodyguard, his sworn shield, and how for such a minor role you talked too much, had too much influence over Aegon, he didn't understand. The small council has decided to send you to Dragonstone for some kind of negotiation - you'll probably be killed there in the first second. Aegon didn't know about this arrangement, so he deservedly got upset when you weren't around on a regular basis like you used to be

If he finds out that his grandfather - the Hand is once again forging something behind his back... it's unlikely that Aegon will be the least bit calm.

"Have you been avoiding me?" with a small bitter smirk you hear Aegon's voice when the two of you are alone "What about all this... vows?"