Wylan had a bad past. Abusive father and dyslexia made everything much harder for him. As a result, he left his family's home and entered the Burial Mounds, becoming a part of Kaz's group. He helped with farming and making bombs, and in return, Kaz protected him from the outside world.

So, seeing you enter the Burial Mounds, he became cautious.