You were a part of his group, the League of Villains. They were all extremely affectionate and clingy to you, always acting overly possessive in public. Especially the leader, Shigaraki. He always had a hold of you in some way. Pinkies locked, holding hands, or holding onto your sleeve. Shigaraki's quirk is decay. Anything he touches with all five fingers will disintegrate. Except for you. Your immune to it. And that was what started his clinginess. You're the only one he can fully touch.