Search code examples
assembly64-bitnasmx86-64low-level

Program in x64 assembly modifying array passed from a C++ procedure in Linux does not work, though analogous solution worked for x86


I wrote a program in x64 assembly to replace a string's lower-case letters with stars. The assembly procedure is called from a C++ program and receives an array of chars. The similar logic applied for x86 worked (the difference being registers used etc.), but now the string remains unaltered when built for x64. I use Debian Linux and nasm.

section .text
global  func

func:
    push    rbp
    mov rbp, rsp

; zadanie jako takie
    mov rax, QWORD [rbp+8]
    loop:
        cmp BYTE [rax], 97
        jl increment
        cmp BYTE [rax], 122
        jg increment
        mov BYTE [rax], 42
        increment:
        add rax, 1
        cmp BYTE [rax], 0
        jne loop


    exit:
    mov rax, 0          ;return 0
    mov rsp, rbp
    pop rbp
    ret

It is called from the following C++ program:

#include <stdio.h>
#define LENGTH 1024

extern "C" int func(char *a);

int main(void)
{
  int result;

  char text[LENGTH];
  printf( "Write some string\n" );
  fgets( text, LENGTH -1, stdin );


  printf("String: %s\n", text);
  result=func(text);
  printf("String: %s\n", text);

  return 0;
}

If necessary, here's the makefile:

CC=gcc
ASMBIN=nasm

all : asm cc link
asm : 
    $(ASMBIN) -o func.o -f elf64 -l func.lst func.asm
cc :
    $(CC) -m64 -c -g -O0 main.cc
link :
    $(CC) -m64 -o test -lstdc++ main.o func.o
clean :
    rm *.o
    rm test
    rm errors.txt   
    rm func.lst

Additionally, any resources for porting from x86 to x64 would be appreciated.


Solution

  • The reason the program does not work is the different calling convention in x64. See: link for reference. The address of the string array wasn't passed on a stack, as it would be the case of x86, but stored in the rdi register. Therefore the solution is to change the instruction loading the array address into:

    mov rax, rdi