segmentation fault in x86 trying to do bubble sort

Question

I am trying to implement bubble sort in assembly. Here is my code. I keep getting segmentation fault. I have a function down below. I have been trying to figure this out but I couldn't find a compiler for x86 and I have been checking with my code to check what is wrong but to no avail.

here is my function code:

bubble:
    push ebp
    mov ebp, esp

    push ecx
    push edx
    push edi
    push esi
    push ebx
    push eax
    
   
    mov eax, 0
    mov ecx, [ebp+12] ; number of elements in the array
    mov edx, [ebp+8]; address of array
    mov edi, 0
    mov esi, 0
    dec ecx; n - 1

    mov esi, 1

    sortOuter:
        cmp esi, 0
        jg sort

        sort:

        mov esi, 0 ;count

        check:
        cmp edi, ecx ; i < n - 1
        jl sortInner

        sortInner:
            mov ebx, [edx+edi*4] ; mov array[i+1] to ebx
            cmp [edx], ebx   ; cmp array[i] to array[i+1]
            jle noswap

            swap:
                mov eax, ebx ; mov array[i+1] to eax
                mov ebx, [edx] ; mov array[i] to array[i+1]
                mov [edx], eax ; mov array[i+1] to array[i]
                
                inc esi ; count++

            noswap:

            inc edi ; i++
            jmp check
    
        
        jmp sortOuter

    done:


    call print_nl


    pop ebx
    pop esi
    pop edi
    pop edx
    pop ecx

    
    mov esp, ebp
    pop ebp
    ret

Answer 1

The segmentation error comes from the infinite loop that you have created with

check:
  cmp edi, ecx ; i < n - 1
  jl sortInner

  sortInner:

If EDI is less than ECX you jump to sortInner, but if it isn't you fall-through into sortInner. No matter, you always end up running the code at sortInner and because the memory addresses that the code uses keep growing, at some point you will be trying to read memory that you don't have access to, hence the segmentation error.

Now if you were to correct this, then there's a second infinite loop waiting.

sortOuter:
  cmp esi, 0
  jg sort

  sort:

Other errors include:

• Resetting ESI instead of EDI at the start of the inner loop
• Not swapping elements at all but always writing the smallest value in the first array element
• Forgetting to restore the register EAX

---

This is a working BubbleSort. Don't just copy it, but find out how it functions!

In an array with N elements we have to do N-1 comparisons at most (to have the greatest value bubble towards the rear).
Because the InnerLoop uses a counter that gets initialized from the ever decreasing OuterLoop counter, with each iteration of the OuterLoop the portion of the array that is processed in the InnerLoop gets smaller. The portion of the array that we then no longer process contains the greatest elements that have bubbled towards the end of the array, hence the name BubbleSort.
Provisions have been made for an empty array or an array that has but 1 element. Always include some code for the special cases!

bubble:
  push ebp
  mov  ebp, esp
  push ...

  mov  ecx, [ebp + 12] ; Number of elements in the array
  sub  ecx, 1          ; First time we do n = N-1
  jbe  Done            ; Array is empty or has but 1 element

OuterLoop:
  mov  edx, ecx        ; Current value of the OuterLoop counter (n)
  mov  esi, [ebp + 8]  ; Address of array
InnerLoop:
  mov  eax, [esi]
  mov  ebx, [esi + 4]
  cmp  eax, ebx
  jng  NoSwap
  mov  [esi], ebx
  mov  [esi + 4], eax
NoSwap:
  add  esi, 4
  dec  edx
  jnz  InnerLoop
  dec  ecx                ; Next times we do n = n-1
  jnz  OuterLoop

Done:
  pop  ...
  pop  ebp
  ret