I downloaded several software that provide virtual COM ports. These COM ports do appear in the Device Manager and can be selected for upload from the Arduino IDE, menu Tools -> Serial Port -> COM3. It starts uploading and reaches 90% and then it either times out or just does nothing.
I want to upload onto the virtual COM port so I could then read the compilation output files in another program. I don't want to use my Arduino at all, and I don't want to manually get the verbose output files when uploading. This problem doesn't happen when I upload on the real Arduino.
The application should work on all platforms. This task seems easy on Linux, and I am facing the stated problem on Windows and any help with Mac would also be useful.
The application will be an educational tool for hardware simulation and visualization trying to give a more hands-on experience for users than other simulators out there. So may be this will give you an idea of why I want to do so.
How can I get started?
I think you might be assuming that uploading code to Arduino is one-way communications: this would be like putting strawberries in a blender, and a Daquiri comes out. If that was true, you could just take whatever the IDE dumps to the serial port, save it to a file, and you have an Arduino binary. (Skip to TL;DR if not interested in details. Upshot: this assumption is not correct).
The 1-way communication assumption is not entirely correct: there is a program on the Arduino (called "a bootloader") which is responsible for communicating with the programmer ("programmer": a program that programs the Arduino, assume it is the Arduino IDE for now). In their most "natural" state, the Arduino CPUs cannot be programmed across serial lines. Rather these chips are programmed either via the in system programming (ISP) or via the JTAG protocol. The bootloader is a program that runs on an Arduino CPU and loading of sketches/programs over the serial port. This program runs at startup and looks for programming commands over the serial port.
If it discovers that a programmer is trying to communicate programming information, it will read the compiled Arduino binary coming over the serial link, store it in flash memory, send it back to over the serial link for verification, and if everything is successful, exit and launch the stored sketch. If no programming information appears on the serial port, that is, no programmer is trying to write a new sketch, then the bootloader simply quits and launches the program already stored in flash.
TL;DR: In order to implement a pseudo-Arduino on your serial port you must write a program some code that simulates an Arduino (bootloader) on the other end of your virtual serial port. So when a programmer/IDE says to Arduino "are you there?" your program will respond "yes!", just like an Arduino would.
The default Arduino bootloader is STK-500 compatible: that means that it implements STK-500 commands - the reference for which can be found here. If you decide to do this, then the easiest thing might be to start with an existing bootloader, such as Arduino's or AdaFruit's (there are others too), and modify it. Such a bootloader would have all the commands already implemented, and since it is written in C (I wouldn't choose an assembly bootloader to modify :), it should be easy enough to modify.
Alternatively, you might decide that STK-500 is too difficult to implement. If this is the case, you can use any programmer protocol that Avrdude supports: Avrdude is a program for programming AVR chips, and Arduino IDE uses Avrdude internally to send the sketch to the Arduino. If you do this, then you'd have to change the settings in Arduino IDE for which programmer you are using.
Personally, I think STK-500 compatible is the best option for this, but YMMV.