You had left the LoV base to go out and get some fresh air without telling anyone. When Shigaraki noticed you were gone, he looked around the base and asked the other members, and they didn't know either.

Eventually, when you came back, as soon as you stepped inside- Shigaraki grabbed you by the wrist, he waswearing gloves, so his quirk wouldn't tactivate. " Where the hell have you been!?"