Challenge RE #17

Let’s start with the challenge #17. According to the description this should be a task

This is a quite esoteric piece of code, but nevertheless, the task it does is very mundane and well-known to anyone. The function has 4 32-bit arguments and returns a 32-bit one. What does it do?

The assembly code to understand it’s the following one.

f:
        sub     edx, edi
        mov     r8d, ecx
        mov     ecx, 63
        mov     eax, edx
        sub     r8d, esi
        sar     eax, cl
        and     eax, edx
        mov     edx, r8d
        sar     edx, cl
        add     edi, eax
        and     edx, r8d
        add     esi, edx
        sub     esi, edi
        mov     eax, esi
        sar     eax, cl
        and     eax, esi
        add     eax, edi
        ret

Analysis

The signature of the function f it’s not a great deal, given that it’s provided to us in the description of the challenge itself. We have

int f(int a, int b, int c, int d);

There are several operations performed here in this assembly code, one thing the notice right away it’s the constant 63. This constant it’s used to perform a right shift. The issue that I see here, is that if we perform a right shift to a 32 bit integer, with displacement of 63, we will have some problems. Actually I manage to take this code into C code, and try to compile it with -g -Wall -Werror flags to see if we will encounter any warning(made an error with -Werror flag), and yep we will get a warning like this:

main.c:4:22: warning: right shift count >= width of type [-Wshift-count-overflow]

The problem that I have now is that I don’t know how should the computer reacts to this. My human logic, will tell me, well at the end if you right shift more bits than the number has, you can assume will be zero. That’s my brain talking, now another thing is how is this implemented. Searching on Google I found this article, Hello ARM: Exploring Undefined, Unspecified, and Implementation-defined Behavior in C++, which talk in more detail on the expected behavior of these cases on the different architectures. Take a look at the section related to the Shift operators, and you will see several tables, displaying what we can expect on these cases.

It seems that my initial intuition, that will be 0, is not so crazy, take a look at this table from this same article:

Given a 32-bit integer with a value of 1:

Notice that if we perform a shift with an amount of 64, in ARM we will get 0, while in x86 and x64 we will get 1. With this in mind I cannot get to definitive solution.