Challenge RE #24
This challenge it’s more interesting. This time we have a binary file with the following description
Challenge Description
Here is the simplest possible calculator program, a simplified version of bc UNIX utility:
123+456
(unsigned) dec: 579 hex: 0x243 bin: 1001000011
120/10
(unsigned) dec: 12 hex: 0xC bin: 1100
-10
(unsigned) dec: 4294967286 hex: 0xFFFFFFF6 bin: 11111111111111111111111111110110
(signed) dec: -10 hex: -0xA bin: -1010
-10+20
(unsigned) dec: 10 hex: 0xA bin: 1010
It is known that it supports 4 arithmetic operations and negative numbers can be denoted with a minus before the number. But it’s also known that there are at least 7 undocumented features. Try to find them all.
In order to approach this problem, we need to take a look at the assembly code of this binary. For this you have several tools to extract the disassembled code, here is a list of them
For this challenge, I’ll just use Ida because is the one I’m more familiar with. The great advantage of these tools is that they even draw you through the flow of the program, and even disassemble the code to C/C++ code. I’m not going to disassemble anything, but I’m not mad for them to draw me some arrow flows, that’s sick, good stuff.
Analysis
Exploring the binary functionalities
Let’s start our reversing process running the binary, as always don’t do this if you are running with malware. Let’s run this program
./calc_x64
1
(unsigned) dec: 1 hex: 0x1 bin: 1
1+1
(unsigned) dec: 2 hex: 0x2 bin: 10
1*1
(unsigned) dec: 1 hex: 0x1 bin: 1
1/1
(unsigned) dec: 1 hex: 0x1 bin: 1
1-1
(unsigned) dec: 0 hex: 0x0 bin:
1**1
Error: unexpected token. Exiting.
As you see in this terminal output I explored the program running basic arithmetic operations, +, -, *, /. I also tried power in the Python style, and we received an error. The output in all cases, except the error one, is in the format (unsigned) dec: <number in decimal representation> hex: <number in hex representation> bin: <number in bin representation>. In all the cases we tried with positive numbers, let’s try with negative ones
-1
(unsigned) dec: 4294967295 hex: 0xFFFFFFFF bin: 11111111111111111111111111111111
(signed) dec: -1 hex: -0x1 bin: -1
-1 * 2
(unsigned) dec: 4294967294 hex: 0xFFFFFFFE bin: 11111111111111111111111111111110
(signed) dec: -2 hex: -0x2 bin: -10
-1 + 2
(unsigned) dec: 1 hex: 0x1 bin: 1
-1 / 2
(unsigned) dec: 0 hex: 0x0 bin:
-1 - 2
(unsigned) dec: 4294967293 hex: 0xFFFFFFFD bin: 11111111111111111111111111111101
(signed) dec: -3 hex: -0x3 bin: -11
In this case, we can see that the calculator can handle signed numbers as well. Ok, everything is as expected. These are 4 arithmetics operations mentioned in the description of the challenge, we are missing 3 more. One thing we can try is using bitwise operators and see what we get.
1 ^ 2
(unsigned) dec: 3 hex: 0x3 bin: 11
1 & 2
Error: bad character. Exiting.
Here we see the calculator supports XOR with ^ operator, but does not support AND with &. We have 5 operations, we are still missing 2 more. I also tried | but we received an error output as well. The error for this case it’s quite different, has the following output junk after expression, which might mean that we are just using badly this operator. For now, this exploration it’s quite good we will need to look at the assembly code to find out the other two.
Reading the code
Let’s start our analysis with the main thread of the program. Right away you can notice that we have a call to sub_400A3B, which we are going to look at a bit later. Followed by printings of a text with the following format, using the result of this subroutine to populate the formatted text. This is the formatted text (unsigned) dec: %u hex: 0x%X bin: <number in binary representation>\n. This matches with our previous exploration of the program. Here is the code snippet for the assembly code
; int __fastcall main(int, char **, char **)
main proc near
; __unwind {
push rbx
call sub_400A3B
loc_400696:
call sub_400AEE ;; ???
;; PREPARING FOR PRINTING RESULT OF sub_400AEE on different formats
;; decimal, and hexadecimal
mov esi, offset aUnsignedDecUHe ; "(unsigned) dec: %u hex: 0x%X "
mov ecx, eax ;; result from sub_400AEE
mov edx, eax
mov ebx, eax
mov edi, 1
xor eax, eax
call ___printf_chk
;; printing just 'bin: '
mov esi, offset aBin ; " bin: "
mov edi, 1
xor eax, eax
call ___printf_chk
mov edi, ebx
call sub_40082D
Now I personally don’t want to read the whole assembly code, I just want to find out these missing operations. One smart approach would be to look in Ida for the signs of operations we already know. For example, let’s look for the parsing of character +. In this way, we might identify other symbols as well in the code. The ASCII value for + it’s 0x2b, another way to represent this is 2bh which is the way Ida represents the hexadecimal numbers. To make the search you can press Alt+T or go to Search > Text, another way could be that if you previously disassembled the binary to a file you can use grep to search in the file where the disassembled code. When we made that search Ida added a comment with the value '+', this is done in the subroutine sub_400C07. Let’s analyze that particular function.
sub_400C07 proc near
; __unwind {
push rbx
xor eax, eax
call sub_400BBB
loc_400C0F:
mov ebx, eax
loc_400C11:
cmp cs:dword_602490, 1
mov al, cs:byte_602090
jnz short loc_400C69
loc_400C20:
cmp al, 2Ah ; '*'
jz short loc_400C43
cmp al, 2Fh ; '/'
jz short loc_400C54
cmp al, 25h ; '%'
jnz short loc_400C20
call advance
xor eax, eax
call sub_400BBB
mov esi, eax
mov eax, ebx
cdq
idiv esi
mov ebx, edx
jmp short loc_400C11
;; ... method continues
Right away in loc_400C20 we can see that apart from * and /, operations that we already know, there’s as well %. Let’s rerun our calc_x64 to see what this operation gives us, possibly just the remainder between two numbers, but let’s check.
5 % 2
(unsigned) dec: 1 hex: 0x1 bin: 1
4 % 2
(unsigned) dec: 0 hex: 0x0 bin: 0
Nice, so this is another supported operation, at this point, we have the +, -, *, /, ^ and %, just one operation it’s remaining. Now that we know Ida commented to us these special signs it would be smart on our part to make a smarter search, in this case, let’s look for the following regular expression:
; '.{1}'
With this, we are asking Ida to look for comments that are enclosed in '' and have exactly one character.
This search gave us several results, we only need our extra operation in order to solve the puzzle so I’m going to look right for that. Doing so, I spotted the following character ~, so this could be our last operation, which could be the negation.
~ 1
(unsigned) dec: 4294967294 hex: 0xFFFFFFFE bin: 11111111111111111111111111111110
(signed) dec: -2 hex: -0x2 bin: -10
Yep!! Now we have all the operations, +, -, *, /, ^, % and ~ are our seven arithmetics operations.
Conclusion
I think the lesson to learn here is to focus on your goal try to find what you want and not analyze every single line of code.