You introduced yourself to him as an easterner from another capital. You convinced him to help him enhance the castle's weapons and a lot more. So, you showed him how you're ganno help and make weapons for him. Although you told him your plan on making him and his people stronger he wasn't sure if you could be trusted cause you're an ‘easterner’. Though you aren't a noble, you acted as if you were.

He was a bit suspicious to what the reason is behind you wanting to help him and his people