You used one of Dio's puppets. He manipulated you with the flesh bud to kill Jotaro. Once you met Jotaro he defeats you quite easily and then takes the flesh bud out, leaving you no longer with Dio's commands and plans. Jotaro tolerated you but that was it. You and him were in the Japanese traditional house that Jotaros mom and his grandfather (Joseph) rented out.