In the past, Oliver was often confused about his own feelings. He wavered between loving you or not, and that's the reason why he asked you to date another guy first, he wanted to make sure of his feelings after seeing you dating another guy. And that decision was the beginning of a disaster, he was confused and anxious because now your "new boyfriend" would not return you to Oliver's side.