I have successfully used Pocket PCs in the past (using the serial port) to control simple robots (small rovers).
Looking around here and on Apple's developer website, it seems that starting on 3.0, they do allow applications to communicate (and even use their own protocol) with custom hardware. I'd assume they have the same policy for Bluetooth communication.
But it seems the hardware developer program is geared towards big corporations.
Is it possible to make non-jailbroken iphones to communicate with custom hardware (a PIC microcontroller with some Bluetooth chip interface for example)?
The External Accessory framework added in iPhone OS 3.0 does give you the ability to communicate with outside devices connected either through the dock connector or Bluetooth, but those devices must be part of the Made for iPod program. That means that they must have special hardware in them, obtained from Apple, that allows you to communicate via the protocol Apple uses. If they do, it becomes very easy to interact with them through the External Accessory framework.
Apple has made a push to open up the Made for iPod program to much smaller companies than had traditionally been able to develop iPod accessories, but there are still some hurdles to overcome when designing such hardware. An approval process with Apple is involved, along with the hardware design you'll have to do. This means that it's still not suited for one-off projects like your robotic controller, unless you plan to sell your robotic controller in moderate volumes.
However, I have heard of a few companies who are working on generic digital-to-analog boards and the like, which might be able to act as a hardware interface for your robot. Pasco attempted to show off one such board during the WWDC 2009 keynote.