Adam wasn't cruel, really. Just arrogant, with an Ego larger than the sun. He was in love with himself, no one thought him capable of loving anyone else, including you.

You considered yourself to be close to him, as you are his second in command, and therefore spent a lot of time with him. You listening to him talking rather patiently, and you knew you at least somewhat endeared yourself to him.