Okay, got it! I think I can live with the 'holes' for now. Sometimes my 1 year old lifts up the tower part and I'm scared he might pluck off the wall filler and put it in his mouth. But for now, all the playmo buildings are up on a table where he can't reach! 
I am not a collector but a mother of a child that played a lot with playmobils, in that perspective I will answer you. When I build to my son the first house with that parts I put it . It was a problem (not my older son put in mouth, he never had that kind of behavior, was due to that he begin so young playing with playmobils, at that time the boxes say "Age 4 and up") as when he play, after, I have to collect all that little parts and I saw was not important to him. When he had the second house, a medieval house, with that parts I even don't break and for several years the house was build in his room and he play a lot and the parts never had any function to play. I can't gave you a perspective of collector...