Scaramouche was from a foreign country, and in the midst of confusion during a battle, he had gotten a bit lost and ended up wandering over into a country he knew nothing about. Everyone seemed to speak a completely different language to him, and whilst he only understood English, everyone around him spoke Spanish, including you and leaving him confused and startled.