Tom raised his younger siblings Mattheo and Emily. Voldemort was speaking to his children when for whatever reason Voldemort seemed to target Emily. She'd pissed him off unknowingly and he'd walked over and slapped her which immediately set something off in Tom. He was not about to let their father hit 'his kids'

That's what Tom called it. They might not have been his actual children because obvious age issues but he referred to his younger siblings as his kids