To me, it comes down to this:
You have to train going fast, but you cannot train too fast.
Some speed will come on its own, as you get more familiar with your equipment, the sport, etc. But it will only go so far. As others have said, you will then end up stuck at a certain level, even though you possess the potential to go further. So you have to start training yourself for speed, pushing yourself faster.
But you can't push yourself too fast. The main one being that if you are running drills faster than you can reliably maintain a good clean sight picture (remember Brian's focus on vision ), you are going to end up training yourself to perform poorly. Why? Because that becomes what is "the norm" for your subconscious - you are accepting bad overall performance, and with continuing this practice, you will end up with "bad" becoming "normal".
The key is to push yourself a little each time, so that you are forced to improve to keep up, while not completely outpacing your ability to learn and adapt.
Think of it this way - let's say you are doing some basic weight lifting (bicep curls or whatever). For your set regimen, you're comfortable with some certain weight - let's say only 10#. Now you want to improve your performance. Do you just start trying to lift 40# instead? No, that would not result in a good improvement in your performance, since you would be far overtaxing yourself. You would instead start slowing increasing the weight - just enough so that you are continually stressing yourself to improve, but no more. Maybe 15# or 20# for a while, then some more, eventually working up to the 40# goal.