It's not actually inconsistent; after de the definite article is always included, but the partitive article is always excluded.

If you're afraid of spiders, that means you're afraid of spiders as a class, which requires the definite article = les araignées. 

J'ai peur de + les araignées -> J'ai peur des araignées.

But if you need flour, (or bizarrely spiders) you don't need all the flour in the world (or all spiders), just some flour, which implies the partitive article.

J'ai besoin de + de la farine. -> J'ai besoin de farine.

The reason, perhaps, is that "de de la farine" would sound a bit funny.

But if you were talking about some specific flour, i.e. "I need the flour", then you would use the definite article, and it would not be excluded.

J'ai besoin de + la farine. -> J'ai besoin de la farine.

Yes, very clear and pragmatic explanation. Thanks.

Wondered if having a "partitive" fear of spiders might corrupt the rule, but then I suppose it would be:

J'ai peur de certaines araignées