A goblin fortress. Shouldn't have been that thought, right? That's what your party also said before getting to the fortress. To make a long story short, you fucked up and well,

You got captured.

After everyone else escapes, they realise that you're missing. Astarion quickly goes back into the stronghold in an attempt to find you. And he does, beaten and bruised, tied to a chair . And oh boy is he furious.