I think it depends upon what you are going to do with it. If ‘this’ building is only ever going to be this building, then I’d leave it intact (or, at least, in the same storage container). And if ‘this’ piece of furniture is only ever going to be used with this particular building, then it may as well be stored with it.
But if you might use this thing with that thing, or maybe this other thing instead – then it is a lot easier to find if you keep all those same sort of things together.
I started organising by moving things into ziplock bags and then the ziplock bags into containers, all under the same theme. For example, all buckets went into a bucket bag, then the bucket bag went into the Farm container. When the contents of the ziplock bag became too much for the bag, or too much to locate one particular item, they got divided further. The smaller the part, the quicker it was divided – and put into those flat containers one might put nails and screws in.
My most precious things – and the most easily breakable – are stored in a cupboard. I’m fortunate in that I have big cupboards that can house most things. But only three pirate ships. I had nine of them. The rest sat on the floor, covered by a sheet, and became quite annoying. As much as I loved them, I sold them all in the end.
Klickies: I originally had them all together. Then I divided into female, male, and children. Now each of those categories are divided further - skin colour, summer, winter, click-on-dress females – with separate containers for theme klickies (construction, knights, fairies, clowns, whatever), and yet another for older ones (divided into their colours or themes in ziplock bags).
I have some rather vague-sounding containers (‘Outdoors 1’ and ‘Outdoors 2’ for example), and some that still could do with further division one day (‘Fences’ always daunts me – they are divided into ziplock bags by type, but it is a big container, and there are so many of them…), but as long as I know where something is, that is good enough in general.
Oh, and I don’t always put things away properly, and I currently have a pile of things to sort out – organising is an on-going task – but I think if you choose a system, make it ‘good enough for now’, you can make a start, and always fine-tune it later.