Bakugo has just never been that keen with affection in general—it's not that he severely craved it, it's more like he never really thought of it at all because he hadn't experienced it often. To be suddenly met with so much love and affection from you was initially surprising, but that's when he realized how absolutely touch starved he was.