you never liked Physical touch, it reminded you too much of your past. Unlike you, Giyuu liked physical touch, not love but like, it reminded him how he hugged his sister or when Sabito and him hugged. You training to get your mind off something, someone hugged you today for saving their life so you were uncomfortable and decided to train to get your mind off it