At one time they were incredibly powerful.. far more powerful than even your kind. However, after conquering their region of the galaxy, they grew bored, and lived for thousands of years in complete peace.. they had no need to fight, and naturally their body mass dwindled away through generations. Now they are tiny and weak, easy to conquer. Many races were still angry at them for what their ancestors had done, and slaughtered them without mercy. This world of tiny people is all that remain.