Having a problem with inline assembly and calling conventions

[Psytherium]

I'm working on an assignment that creates a function using inline assembly to compute the value of an expression. The following is the signature of the function.

long evaluate(char *code, bool trace);

The expression pointed to by the code parameter computes a signed 32-bit value using a simple expression language. Each expression contains three fields. The first field is required but the other two vary in size depending on the value of the first field. The memory layout is shown below.

Expression encoding:
1-byte op-code | optional left operand bytes | optional right operand bytes

Op-codes include three fields as shown below.

Bits Description
7-6 Operation code
5-3 Left operand value
2-0 Right operand value

The operation code indicates how the operands will be transformed into the final resulting expression value.

Code Operation
00 minimum of left and right operand
01 maximum of left and right operand
10 sum of left and right operands
11 difference between left and right operands (left-right)

Each operation requires two operands (left and right). The value of each operand is given in the table below.

Code Value of the operand
000 constant 0
001 constant 1
010 constant 2
011 constant 4
100 1-byte signed value
101 2-byte signed value
110 4-byte signed value
111 an expression (as defined here) (We'll ignore this one).

Constants require no additional bytes. One-, two-, and four-byte values require the appropriate number of bytes.

Here are some example inputs and what the output should be.

Input | Output
00 | min(0,0) = 0
4A | max(1,2) = 2
9C 7F | 4 + 127 = 131


The following is my code. It compiles without any errors, but when i run it it just crashes.

//add2.cpp

long __declspec(naked) evaluate(const char *code, bool trace)
{
    _asm{
        push ebp
        mov esp, ebp
        push dword ptr [ebp+8]
        push code
        
        call eval
        
        add esp, 12
		ret
        
    eval:
        mov eax, 0
		mov ebx, 0
        mov ecx, 0
        mov edx, 0
		mov esi, [ebp+8]
        mov ebx, [esi] 
        mov dl, [esi]
        and dl, 56
        shr dl, 3
        cmp dl, 7
        jne notexp
              
    nextthreebits:
        cmp ecx, 1
        je op
        mov dl, [ebx]
        and dl, 7			;00000111
        cmp dl, 7
        mov ecx,1			;checked nextthreebits
        jne notexp
        
        jp op
        
    notexp:
        cmp dl, 4			;00000100 (check to see if byte value)
        jge signedvalue
        cmp dl, 3			;00000011 (check to see if const 4)
        je constant4
        and edx, 255		;11111111
		push edx
        jp nextthreebits
        
     constant4:
        push 4
        jp nextthreebits    
    
    signedvalue:
        add dl, 4
        cmp dl, 0
        je onebyte
        cmp dl, 1
        je twobyte
        and edx, 255
        push edx
        add esi, 4
        jp nextthreebits
        
    onebyte:
        inc esi
        mov dl, [esi]
        and edx, 255
        push edx
        jp nextthreebits
        
    twobyte:
        inc esi
        mov dx, [esi]
        and edx, 255
        push edx
        add esi, 2
        jp nextthreebits
        
   
    op:
        mov dl, [ebx]
        and dl, 192			;11000000
        cmp dl, 0
        je min
        cmp dl, 64			;01000000
        je max
		cmp dl,	128			;10000000
		je sum
		cmp dl, 192			;11000000
		je subtract
		jp done
        
    min:
        pop ebx				; right value
		pop eax				; left value
        cmp ebx,eax			; right-left
        jle right
        jp done
        
    right:
        mov eax, ebx
        jp done
        
    max:
        pop ebx
		pop eax
        cmp ebx,eax
        jg right
        jp done
                 
	sum:
		pop ebx
		pop eax
		add eax, ebx
		jp done

	subtract:
		pop ebx
		pop eax
		sub eax, ebx
		jp done
        
    recursive:          
        jp done
        
	done:
        }
}

Here is the test file.

//testeval.cpp

#include <stdio.h>
#include <assert.h>

long evaluate(const char *code, bool traceFlag);

int main()
{
    static const char test1[] = { 10 };
    static const char test2[] = { 0x4A };
    static const char test3[] = { 0x9C, 0x7F };

    long x = evaluate(test1,true);
    printf("test1 result %d\n\n",x);
  
    x = evaluate(test2,true);
    printf("test2 result %d\n\n",x);
    
    x = evaluate(test3,true);
    printf("test3 result %d\n\n",x);

	return 0;
}

Any help would be GREATLY appreciated. I need help on this ASAP. I don't necessarily need the code revised for me(though that would be excellent) but atleast just some help on where I'm going wrong or which parts in the code i need to look at, etc. Thanks!

[SpinningVertex]

I'm afraid I can't give you any exact instructions for a solution. But since it crashes, we can assume that you either cause an access violation or that you do not preserve the registers well enough for C so that the your thread starts trying to run code at some random place in the memory.

If the later is the case, it can easily be fixed by adding PushAd (Which pushes All general purpos registers to the stack) at the start of your asm and PopAd(which pops them all) before "ret".

If it still crashes after that,.. then you probably have an access violation. I usually make those when I treat a pointer as a value or the otherway around.

Oh, and try to avoid using the registers directly, if your compiler translates the parameters and result names to registers when compiling. It makes it a bit easier to keep track of all the variables.

Sorry if this isn't much help but it is all I can think of at the moment.

Hope it turns out well!