Your local fire code likely sets limits as outlined above. Enforcement is generally non-existent, and no, your insurance company won't deny your claim of you exceed code and your house burns down.
That said, the NFPA provides safe storage guidelines that a prudent reloader would follow. For example, storing smokeless powder in a 1" nominal thickness wood cabinet ensures that the cabinet will have ventilated (burned through) before the powder can ignite preventing explosions.