We are attempting to port a legacy application from an old monolithic RTOS to a unix based RTOS on a PowerPC 8360. In the old system, our large code base became dependent upon 1/0 returning zero, and 0/0 returning zero. Now in the new operating system 1/0 returns inf and 0/0 returns NaN, which breaks our application. We've tried playing around with the FPSCR register with no results.
Secondly, if there is a way to change it, can the change take affect for our application process and not the entire system? We do not want to change the div/0 behavior of other applications within the system.
Anticipating the inevitable "why are you doing it that way" question, we have to preserve the previous behavior so changing the app to actually not divide by zero is out of the question. This is a sore spot with us so please refrain from asking. Thanks in advance!
Note: it's been a while since I worked on this kind of stuff. Hopefully my answer is in the ballpark.
You'll need to trap the div-0 exception that occurs when this happens -- I believe MSR.FE0 and MSR.FE1 as well as FPSCR.VE and FPSCR.ZE will need to be manipulated in order to ensure the handling of this the way you want.
So once you get that set up and working, you'll need to:
take control of the exception handling for these scenarios (0/0 and 1/0). In most of the small real-time kernels I use, with all source code available, I would know what to do. Not sure what your RTOS is or how much control you have. Chances are, if it's a "heavierweight" OS, it's not going to let you fiddle with exception handler logic.
I think 0/0 will trigger an "Invalid Operation" exception (FPSCR.VE), whereas 1/0 will trigger an IEEE floating-point zero divide exception (FPSCR.ZE).
if you get the Invalid Operation exception you need to determine if the cause was 0/0 or something else. With 0/0, FPSCR.VXZDZ will be set (I think). There are other ways to trigger this exception so here FPSCR is your friend.
if you get the IEEE FP div-0 exception you need to determine if the cause was 1/0 or something else (e.g. 2/0). I think for this you'll have to examine the registers of the interrupted context to see if the numerator was 1 at the time of the division operation that caused the exception. The FPU doesn't care if you attempted 1/0 or 2/0, but apparently your application does.
next, you need to alter the returning context so that you get the desired result. This might be something like changing the FP registers used in the operation so that when you return from exception & the FP division is re-tried, it yields zero. For example, make the numerator 0 and the divisor 1.
Then when you return from the exception, you should get your desired result. Sorry I'm rusty on the specific registers and values, I hope this is enough to get the job done.
You also asked about selectively enabling this behavior just for your application process. I've had to do things like this before, but more in flat-address space, "single process, multiple thread" type kernels (where each task is really a thread, all running in the same flat address space). I've done it a couple different ways, here are some ideas that might work for you:
In the exception handler, examine the process ID / task ID and if it's your application process, handle it the special way, otherwise handle it the standard "system" way.
Alternatively, on a context switch into your application, install your "special" handling for this exception. On a context switch out of your application process, replace it with the standard handling. Note that the application itself won't be able to do this, you'll have to tap into the kernel to do it (there might be a context switch hook/callout that you can use, otherwise you'll probably be modifying kernel source).
I've inherited legacy code like this before & I feel your pain. You want to shake your fist at the people who installed such bone-headed behavior, but right now shaking your fist doesn't help you ship product. You need a solution. Good luck.