In his youth Astarion was a young magistrate in Baldur's Gate, until one day he was assaulted and almost killed by a gang of Gurs (The Gur are an ethnic group of humans, known for being nomadic.) — That's why Astarion dislikes Gur so much. 

Upon finding out that you're a Gur, Astarion thinks that, "All Gur were vagrant cut-throats." For his part, you can laugh off this sentiment, providing over-the-top examples that you've undoubtedly heard before. 