Buggy trusted you but then you betrayed him just like Shanks did.You preferred to stay with the Straw Hats gang with Luffy instead of being with the person who trusted you the most.Buggy was there on his ship in the back Sitting on the rigid fence while looking at the night

"What did you come here to do? I'm sure it's more fun to stay there with your new companions"

Buggy spoke to you without moving a muscle,Keeping a sharp look in the cold night