Would it be OK to put my public interfaces into their own package (for my organisation only).
for example
com.example.myprogram - contains all normal code
com.example.myprogram.public - contains public accessible interfaces
com.example.myprogram.abstract - contains abstract classes
Is this a good or a bad thing to do, are there any disadvantages?
I wouldn't like this practice at all. You should group classes, both abstract and concrete, and interfaces according to functionality.
Look at the Java API as an example. Did Sun separate the Collections interfaces from implementations? No. Sun's practices aren't always the best guide, but in this case I agree.
Don't do it.