Parts

Part 1: Decoding the Gears of War 2 graphical settings.

Part 2: Writing the code to read and write the settings.

Introduction

In part 1 I showed you how I successfully decoded the binary settings of the Xbox 360 title Gears of War 2. This was the first step in my attempt to play the game without 2008 annoyances like bloom, depth of field and overly detailed textures that like to pop in. And don’t get me started on that FOV. The next step was to write an application that can read and write the settings. I dubbed the project Coalesced Inifier and it is released on GitHub. The following sections describe the development process.

A small recap: the settings are stored in a file called Coalesced_int.bin and a hash of said file is stored in a text file called Xbox360TOC.txt. The former is just a bunch of INI files stored in binary form, the latter basically just acts as a checksum to determine if something went wrong when reading the DVD.

Now that we have the format covered, let’s write a small tool to extract the files. For this project I chose Go because I wanted to get familiar with it, and because it compiles to a single binary I can actually release. Prior to this I have written maybe 100 lines of Go in a PR for an open source project so bear that in mind when you are disgusted by the code.

The code listings are abbreviated and don’t show all the error handling. For the full monty check out the project on GitHub.

Some libraries to help us out

It seems that every language I try has a novel approach to CLI argument parsing and Go is no exception. I, of course, tried to roll it myself, but it was very rudimentary. I then tried to make it nicer, but skill issues (and free time) are a thing, so I gave up and started looking for a library.

Another thing we need is the ability to parse an INI file. Now I remember INIs being very simple from ye olde days of Windows 98 and XP, perfectly editable in Notepad. Like many things in software it turns out INI files are extremely complicated to parse and any attempts at doing so will probably end in tears wasting a week trying to understand the 30 years worth of implicit convention that defines the format.

I ended up finding go-arg for argument parsing and ini.v1 for the INI file handling.

Writing the parser

Go structs look a lot like C structs which look a lot like the ones in the ImHex pattern language discussed in part 1. This made the translation fairly simple:

type BinaryIniKeyValue struct {
    Key   string
    Value string
}

type BinaryIniSection struct {
    Name   string
    Values []BinaryIniKeyValue
}

type BinaryIniFile struct {
    Name     string
    Sections []BinaryIniSection
}

type BinaryCoalescedIniFiles struct {
    fileCount uint32
    Files     []BinaryIniFile
}

Now that we have all the structs in order we need a couple of boilerplate-y things to handle the binary encoding in question. We need to be able to read big endian numbers, we need to be able to read nul-terminated UTF-16 strings and we need to read the string lengths as bitwise inverted big endian numbers. That’s a mouthful, but it’s basic IO/bit fiddling stuff.

Go has fairly nice support for bit fiddly stuff so reading a big endian number is easy. The string reading gets a little hairy:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

func ReadUtf16InvLen(r *bytes.Reader) (string, error) {
    var length uint32

    err := binary.Read(r, binary.BigEndian, &length)

    // There's a special case for missing values == 0x0 even if inverted
    if length == 0 || length == 0xFFFFFFFF {
        return "", nil
    }

    length = ^length

    // We are reading UTF-16 strings with a 2-byte nul string terminator
    b := make([]byte, length*2+2)
    n, err := r.Read(b)

    if uint32(n) != length*2+2 {
        panic(fmt.Sprintf("Read %d bytes, expected %d", n, length*2+2))
    }

    chars := make([]uint16, length)
    for i := 0; i < int(length); i++ {
        chars[i] = uint16(b[i*2]) | (uint16(b[(i*2)+1]) << 8)
    }

    runes := utf16.Decode(chars)
    return string(runes), nil
}

Now that’s a piece of code you wouldn’t be happy to see in a code review, but it actually works. At line 4 we see the trivial reading of a big endian number.

Jumping to line 10 we see the handling of the 0xFFFFFF encoding for “empty string”. This jumps at you as a WTF, that’s why it has a comment.

Lines 28-31 are the ones I am truly proud of. This is a for-loop that raw-dogs bytes in pairs into a UTF-16 rune. Before you start sending me e-mails or yelling at the screen, I did try to do this the right way with a UTF-16 reader thingy, but the array in question is apparently not what Go considers valid UTF-16. But I digress, the string is read fine and while it may not be the prettiest method on the planet, I am sure you’ve seen worse.

Now that we can read strings from the coalesced INI format we are actually good to go. Going bottom up we start by reading a key/value pair:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15

func readIniValue(r *bytes.Reader) (*BinaryIniKeyValue, error) {
    var result BinaryIniKeyValue

    key, err := gowenc.ReadUtf16InvLen(r)
    value, err := gowenc.ReadUtf16InvLen(r)

    result.Key = key

    if value == "\\\\\\\\" {
        value = fmt.Sprintf("`%s`", value)
    }
    result.Value = value

    return &result, nil
}

The weird if above is due to some weird escaping within the format, the rest is pretty straightforward. The method reads a single key/value pair from the stream. If you attempt to read a pair where there is none, you will most likely run out of bytes or memory. Now all we need to do is read the files and sections and then call this method for each key/value pair:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16

func readIniSection(r *bytes.Reader) (*BinaryIniSection, error) {
    var result BinaryIniSection

    sectionName, err := gowenc.ReadUtf16InvLen(r)
    result.Name = sectionName

    valueCount, err := gowenc.ReadUint32BE(r)

    result.Values = make([]BinaryIniKeyValue, valueCount)
    for i := 0; i < int(valueCount); i++ {
        value, err := readIniValue(r)
        result.Values[i] = *value
    }

    return &result, nil
}

The above shows how to read an INI section. It is fairly simple and the interesting parts are in line 7 and 11. Now we almost have the entire file written, we just need to loop over all the sections and read them:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16

func readIniFile(r *bytes.Reader) (*BinaryIniFile, error) {
    var result BinaryIniFile

    fileName, err := gowenc.ReadUtf16InvLen(r)
    result.Name = fileName

    sectionCount, err := gowenc.ReadUint32BE(r)

    result.Sections = make([]BinaryIniSection, sectionCount, sectionCount)
    for i := uint32(0); i < sectionCount; i++ {
        section, err := readIniSection(r)
        result.Sections[i] = *section
    }

    return &result, nil
}

Reading the sections looks a lot like reading the values, we read a count and then loop over the count and read some values. In the above we do the same just for sections. The last thing we need to do is to loop over the file count and call readIniFile:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

func ReadCoalescedIniFiles(r *bytes.Reader) (*BinaryCoalescedIniFiles, error) {
    var result BinaryCoalescedIniFiles

    binary.Read(r, binary.BigEndian, &result.fileCount)

    result.Files = make([]BinaryIniFile, result.fileCount, result.fileCount)
    for i := uint32(0); i < result.fileCount; i++ {
        iniFile, err := readIniFile(r)
        result.Files[i] = *iniFile
    }

    return &result, nil
}

Again it looks like the rest, read a count and loop over it. But that’s actually it. That reads all the settings from the game into the structs we wrote in the beginning of this post.

Writing the INI files

Once we have the INI files in memory we can just feed them to the INI library and write it to a file. The file names in the Gears of War 2 settings are prefixed with ..\ which we need to handle, otherwise it is straightforward.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34

func writeInitFileToDisk(file read.BinaryIniFile, baseDir string, prefix string) (string, error) {
    realFileName := strings.Replace(file.Name, prefix, "", 1)
    realFileName = strings.Replace(realFileName, "\\", "/", -1)

    dir, _ := path.Split(path.Join(baseDir, realFileName))
    err := os.MkdirAll(dir, 0755)

    join := path.Join(baseDir, realFileName)
    outputFile, err := os.Create(join)

    defer outputFile.Close()

    iniOut := ini.Empty(ini.LoadOptions{AllowShadows: true})
    for _, binarySection := range file.Sections {
        section := iniOut.Section(binarySection.Name)

        for _, values := range binarySection.Values {
            _, err := section.NewKey(values.Key, values.Value)
        }
    }

    _, err = iniOut.WriteTo(outputFile)(in theory)iFiles(r)

    for i := 0; i < len(coalesced.Files); i++ {
        file := coalesced.Files[i]

        fmt.Printf("Writing '%s' to disk...", file.Name)
        realFileName, err := writeInitFileToDisk(file, outputDir, prefix)
        fmt.Printf(" written to '%s'.\n", realFileName)
    }

    fmt.Printf("Unpacked %d files to '%s'.\n", len(coalesced.Files), outputDir)
    return nil
}

And that’s it! We can now extract all the settings from Gears of War 2 into simple INI files on disk. If we run the tool we get:

./GearGame/Config/Xe-GearUI.ini
./GearGame/Config/Xe-GearGame.ini
./GearGame/Config/GearWeaponMP.ini
./GearGame/Config/GearPlaylist.ini
./GearGame/Config/GearCamera.ini
./GearGame/Config/GearPawn.ini
./GearGame/Config/Xe-GearEngine.ini
./GearGame/Config/GearAI.ini
./GearGame/Config/Xe-GearInput.ini
./GearGame/Config/GearPawnMP.ini

If we poke into GearGame/Config/GearAI.ini we find the key EnemyDistance_Melee we saw in part 1:

» grep -B 10 EnemyDistance_Melee GearGame/Config/GearAI.ini 
[GearGame.GearAI]
MinDistBetweenMantles                 = 1024
EnemyDistance_Melee                   = 256.f

Converting the INI files to binary

Now that we have a way of peeking at the settings, we need to coalesce them back into the binary format. This consists of a couple of steps:

1. Listing all the INI files in a folder structure
2. Counting the files
3. Reading all the INI files
4. Writing the INI files as binary

This is not that complicated and is mostly IO stuff which I will not bore you with. The code is up on GitHub if you hate yourself.

The Coalesced Inifier CLI tool

The end product of this fine work is a small CLI tool that can unpack the INI files from Gears of War 2 and pack them again:

./coalesced-inifier unpack -i /path/to/Coalesced_int.bin -o  my-output-dir -g gow2

And packing them again:

./coalesced-inifier pack -i my-output-dir -o /path/to/coalesced_int.bin -g gow2

The last command will pack the INI files and print out the length and hash of the coalesced file:

[...]
Packing 'my-output-dir/GearGame/Localization/INT/locust_skorge_chatter.INT' as '..\GearGame\Localization\INT\locust_skorge_chatter.INT'... done.
Packing 'my-output-dir/GearGame/Localization/INT/locust_theron_chatter.INT' as '..\GearGame\Localization\INT\locust_theron_chatter.INT'... done.

File length: 1305998
  File hash: 3cab0a4f2237b00875ab02061bdd46a6

Successfully packed 63 files to 'coalesced.bin'.

Conclusion

This project started as a hunch but quickly turned into a working piece of whisky-fueled code that is actually useful. It enables changing engine and graphics settings of Gears of War 2 (and probably more UE3 games).

While fiddling with the settings I found some really amusing configurations such as Marcus running with supersonic speeds and locust rifles having 1000 bullets.

Next time we will look into actually changing the graphical settings and taking a look at their effect on the game using a cheap HDMI capture card.

Thanks for reading.

/J