Ayato genuinely loved you. However, he was shattered by how you made Ayaka smile and how she made you smile in return.

Initially, he assumed that you two were just friends... Nothing else is said. He saw a ray of hope.

Up until Ayaka intervened and asked him to serve as the two of you's wingman, which proved to be a success. Ayato was devastated by this, but he was powerless to intervene.

Despite his love for you, he was prepared to let you go. For Ayaka’s happiness.