Some odd things going on with my affine transforms

Question

Now, I am not asking how to rotate my object, what I'm asking is why on earth my object (an SDL_Surface) stretches when rotated with the formula:

x' = cos(angle) * x - sin(angle) * y; y' = sin(angle) * x + cos(angle) * y;

Theoretically (I presume) this is correct. However, when I use the code below, of which uses this formula, I get this odd stretching and flipping after the angle goes outside 15 degrees! any idea what is causing this?

for (int y = -GAME_HEIGHT/2; y < GAME_HEIGHT/2; ++y) {
    for (int x = -GAME_WIDTH/2; x < GAME_WIDTH/2; ++x) {
        /*----------------Begin Mode7 FX-----------------*/
        float px, py, pz;
        px = x;
        py = FOV;
        pz = y - Xrot;

        float sx, sy;
        sx = x;
        sy = y;


        //sx = px != 0 && pz != 0 ? px / pz : 0;
        //sy = py != 0 && pz != 0 ? py / pz : 0;

        sx = cos(Yrot*PI/180) * sx - sin(Yrot*PI/180) * sy;
        sy = sin(Yrot*PI/180) * sx + cos(Yrot*PI/180) * sy;

        sx *= scaling;
        sy *= scaling;

        sx = (sx / GAME_WIDTH * 0.5f + 0.5f) * BG0.image->w;
        sy = (sy / GAME_HEIGHT * 0.5f + 0.5f) * BG0.image->h;

        sx = (float)wrap((int)sx, 0, BG0.image->w);
        sy = (float)wrap((int)sy, 0, BG0.image->h);

        /*------------------End Mode7 FX-----------------*/
        Uint32 grabPixel = getpixel(BG0.image, sx, sy);

        SDL_PixelFormat* myPixelFormat=backBuffer->format;
        putpixel(backBuffer, x+GAME_WIDTH/2, y+GAME_HEIGHT/2, grabPixel);
    }
}

Please help, as I have been really banging my head on the desk with this one. It's the only stumbling block to getting a perfect software-rendered mode-7 effect I'm doing (that's the reason for the commented out perspective transform)

EDIT: Solved this, below answer explains how I screwed this up. (Basically, I was accidentally feeding in the wrong value to the Y coordinate generation, and it was using the already-transformed X coordinate)

Answer 1

When you do sx = ... and then you do sy = ..., sy equation use sx.

I think you should do something like :

float ax = sx, ay = sy;
sx = cos(Yrot*PI/180) * ax - sin(Yrot*PI/180) * ay;
sy = sin(Yrot*PI/180) * ax + cos(Yrot*PI/180) * ay;