Decompiling tutorial part 1
First you have to get acquainted to the disassembly output of a script. There are two tools that will disassemble lua scripts: luadisasm and chunkspy. I usually prefer the first, because it's output is easier to read, but it doesn't output numbers in Q16.16 format. For chunkspy D-MAN made a Q16.16 version, but I don't really like chunkspy's output.
EDIT: luadec 1.9 and 2.0 can now diassembly scripts, so luadisasm is not needed anymore. Use luadec -dis filename to get a disassembly view of the lua files. Also luaconv is no more needed, if I refer to luaconv/luadisasm in this tutorial then don't care about them. Compare and luadecguess still works, but luadecguess is usually not needed anymore, because luadec 2.0 has built-in guessing capabilities (but they might break, so feel free to read theese sections too)
For scripts larger than 10kb, or for scripts where there are too many differences according to compare's output you simply can't live without the disassembled output
So first do this:
luadisasm.exe 1a335681_manila.luac > dis
If I'm disassembling at job where I have two monitors I usually put the disassembly on the secondary monitor, so I can always look at it.
Let's look at it's output. It is separated into functions, beginning with the main block. For each function (including the main block) it outputs the disassembly of the opcodes it encounters. Lets look at an opcode line:
50 [-]: LOADNIL R0 R0 ; R0 := nil
Each line of the disassembled output consits of the opcode number (50), the opcode (LOADNIL), the parameters of the operator (R0 K0) and a readable output after the semicolon (R0 := nil). I usually only read the part after the semicolon, because it usually tells you everything you have to know. As you have already lua uses registers to store information while running, and these registers are called R0, R1, R2, etc. Local variable declarations are also stored in registers, so if you declare a local in the file, lua will reserve the next free register to this new local variable. This is important, because it will show us where locals have been defined.
Let's look at another output we will analyze:
; Function #5:
; Defined at line: 170
; #Upvalues: 0
; #Parameters: 2 (R0, R1)
; Is_vararg: 0
; Max Stack Size: 16
; No locals information
; No upvalues
1 [-]: GETTABLE R2 R1 K0 ; R2 := R1["Namespace"]
2 [-]: SELF R2 R2 K1 ; R3 := R2; R2 := R2["FindName"]
3 [-]: LOADK R4 K2 ; R4 := "PhotoFrame"
4 [-]: CALL R2 3 2 ; R2 := R2(R3,R4)
5 [-]: GETTABLE R3 R1 K0 ; R3 := R1["Namespace"]
6 [-]: SELF R3 R3 K1 ; R4 := R3; R3 := R3["FindName"]
7 [-]: LOADK R5 K3 ; R5 := "Icon"
8 [-]: CALL R3 3 2 ; R3 := R3(R4,R5)
9 [-]: GETTABLE R4 R1 K0 ; R4 := R1["Namespace"]
10 [-]: SELF R4 R4 K1 ; R5 := R4; R4 := R4["FindName"]
11 [-]: LOADK R6 K4 ; R6 := "Title"
12 [-]: CALL R4 3 2 ; R4 := R4(R5,R6)
13 [-]: GETTABLE R5 R1 K0 ; R5 := R1["Namespace"]
14 [-]: SELF R5 R5 K1 ; R6 := R5; R5 := R5["FindName"]
15 [-]: LOADK R7 K5 ; R7 := "Select"
16 [-]: CALL R5 3 2 ; R5 := R5(R6,R7)
This is function number 5 (0 based, so this is function 5+1=6 in the decompiled output, with paramters like l_6_0, l_6_1, etc.), which has two parameters (R0, R1). Parameters, just like locals, take avay the free register slots. As you can see the first operator will operate on register R2, because R0 and R1 are taken.
Lua will always use the first free register to store it's data. If you have something like:
1st opcode: It will first load the "Namespace" component of R1 (l_6_1) into register 2.
2nd opcode: Then it will copy R2 to register R3, then replace R2 with "Namespace:Findname" (it will discard the old value of R2 (Namespace) because it's not needed anymore)
3rd opcode: It will load a string "PhotoFrame" into the next free register, which is R4, because both R2 and R3 are needed for the next opcode:
4th opcode: This will call the FindName function (stored in R2). This function takes two parameters: self (stored in R3) and "PhotoFrame" (stored in R4). After the call it will put it's result BACK into R2.
Why is the last sentence so important? Let's look at the 5th opcode. It will load l_6_1.Namespace into register R3. But why register R3? Because R2 is taken! And why is it taken? Because it seems we have declared a local at opcode 4!
This way the LDS for the first 5 opcode of this function would look like: ;;;;;;0,0,4 (declare R2 as local in the 6th function at opcode number 4. there are two zeros, which mean R0 and R1 are unneded, because they are parameters)
For knowing where to declare locals it's usually enough to look at the output and see which the first free register is. If, for a while you can only see things like R2 := xxx, or R2[xxx] := xxx, or R2(xxx,xxx,xxx) then R2 is usually not a local. But after a while you stop getting R2's and start gettin R3's then you can be sure that the last line you encountered an assignment to R2 (Something like R2 := xxx), then there is R2 defined as a local.