Incremental design means doing just the right amount of work that will satisfy your client needs, and not complicate the design with premature optimizations.
We've all done at least a little extra work that we thought we were going to need in the future, only to find out later that we never did. In some cases you don't even remember writing the code, so it rots somewhere in your codebase. This waste also happens with design and architectural decisions.
Incremental design proposes that we do a continuous effort during the life of a system to gradually grow and improve it's design, as new features require it.
In a web application, for example, it's typical to have clients worried about how their system will perform when they have a million visits a day. Any premature optimizations are useless because you just don't know what the problem will be. You can't solve a problem that you don't have yet, it could be anything. A way to go about it could be incrementally exposing your system to higher demands, and identifying the bottlenecks. Until that moment when you know what the actual problem is, there's nothing you can do (see also The Theory of Constraints).
In Extreme Programming Explained, Kent Beck highlights the benefits of deferring design until it's actually needed:
Deploy software sooner
Since you don't wait for the perfect design (and good enough will do), you don't waste time optimizing for suppositions.
Make decisions with certainty
As you gather feedback and get to know your system, you can make better, more informed decisions about it.
Avoid living with bad decisions
The risk of making an error is at its highest when you have the least information about a system, which happens to be at the beginning of a project. It's obvious and yet the industry has been doing it backwards for years.
Maintain the pace of development
Advance with confidence, at a steady, predictable pace. Follow guidelines to write clean, modular code, embrace change and grow your design when you actually need it.