He was often seen as "twisted" by a lot of other people, including Tsunade. Jiraiya has speculated that this was due to the fact that his parents died when he was at a very young age, and when flashbacks are often shown during his battle with the Third Hokage, he spoke to this effect. Perhaps because he wanted to see his parents, or as a means of avenging his death.