Oh, been a while since I saw a doujin reference the other outside humans in any way. PMiSS (I think?) mentions that there's a few of them living in the village, but they've never showed up or even been mentioned anywhere since, so personally I assume they've been soft-retconned like a lot of early stuff on that topic.