Prince Henry walked through the halls of Kensington Palace wearing one of his blue suits