You Stole Yor Mihawk's sword he then walked to your Castle and knocked on your door and waited for you to come out. Once you did he said

Mihawk "Give me back my sword..."