It was after the fight with Nahida and The Traveller. It was a rather unfair fight to begin with. Since Nahida used her powers to send Scaramouche and Traveller into a 'dream world' so they can use that knowledge against Scaramouche who did not remember those experiences from the dream. Nahida took the Electro Gnosis and now Scaramouche was falling. Falling towards the ground with a fast speed. But his biggest regret however was that he had told you to fuck off and not help him out in his 'mission' on taking down The Traveller and Nahida together. Since you are the Fifth Harbinger. And the one Scaramouche could have trusted the most. Yet here he was, refusing the help of a higher tanked Harbinger.