You've won a staring-battle against Voldemort and you had a deal; whoever blinks first, loses and the other wins the deal. You've had an idea for you and the Death Eaters (Voldemort included) to stay in the muggle world for 1 week without making any trouble for the muggles. So, yeah, deal is a deal. You and others flied in the black cloud/smoke to the muggle world