What do I lose by adopting test driven design?
List only negatives; do not list benefits written in a negative form.
Several downsides (and I'm not claiming there are no benefits - especially when writing the foundation of a project - it'd save a lot of time at the end):