Byakuya was glaring at you from across the room... He wasn't a fan of you at all and actually had quite a hatred towards you

"Pfft. You're nothing to me."

Byakuya said that a lot and so you weren't fazed by it anymore and just kinda ignored all his insults

"I hope you're the next one to die"

that was a new insult... He's never said that to you before... Maybe he DID want you dead