Published by Slappey
Information by: Modzy, Altimit01, Jonathan, Sword, and various others I've acquired information from.
*This tutorial assumes you already know the basics on hex.*
First off, let's start with some terminology:
Offset: The sequence of bytes at a given address.
Address: The location in hex of an offset.
0x851974 (the address) contains the offset 2800F14B. (in the demo bloodgulch)
Endianness - The byte ordering in memory used to represent some kind of data in computing.
Big Endian - Describes data formatting (sequence of bytes) in which each field is addressed by referring to its most significant byte.
4B F1 00 28
Little Endian - Describes data formatting (sequence of bytes) in which each field is addressed by referring to its least significant byte.
28 00 F1 4B
Little endian describes a sequence of bytes stored in the computers RAM.
Most computers store RAM memory in little endian. However, PPC stores it in big endian.
The byte order is reversed from big endian. (see above examples)
Transition Offset - The space between the start of a tag's meta and the chunk itself. There are several transition offsets in a meta, and there are many chunks in a map.
Reflexive - the transition offset to a chunk count.
Chunks - a part of the tag's meta, that Halo uses to determine the tag's properties or characteristics. (The little independent pieces, like what contains the 00-4B-00, and each of first, second, and third. Example: Reflexive: 3-4B-0, The chunk count, would be the number of chunks in one group/array. (like 3 in this case))
See Figure 1 for reference.
Chunk Count - The number of chunks stored in an array as two different offsets. Both are 32 bit integers, (meaning 4 bytes) side by side. The first offset is how many chunks there are, and the second is the reflexive. A typical chunk count looks like
a [count | pointer | zero] tuple, where count is the number of items, pointer shows where the offset is, and zero is, well, zero...
Tag Array - A huge list of all the tags in the map, that includes all their name offsets, meta offsets, tag classes, etc.
- Things you should know:
Map Information & Structure:
Halo loads the map into memory all at once. It does all its calculations in the fast RAM memory. Because RAM data is stored in little endian, the maps need to be in little endian. So that when it's loaded into RAM, all its bytes are in big endian and easily readable. So, all that to say, maps are stored in little endian.
Halo Demo Map File Header:
Demo map offsets:
0x2C0: Head string
0x588: Game type
0x5E8: Decompressed map size
0x5EC: Index offset
0x2C4: Meta data size
0x58C: Map name
0x2C8: Map build date
0x02: Map type
0x5F0: Foot string
Head String: 0x2C0 begins the "Demo Mapfile Header" or the important parts. In ASCII it reads "dehE" and in hexidecimal it is 64 65 68 45.
Game type: 0x588 has the short 06 which most likely represents the map version number. In the full version it is 07, in XBox its 05 and in halo CE 609. Halo Demo will not accept maps unless there is 06 there.
Decompressed map size: 0x5EC this 32-bit integer expresses the full size of the mapfile in bytes. In the case of xbox maps, it indicates the size of the mapfile after it has been decompressed. If you're rebuilding for PC it's not required that you change this value but it is generally considered a good idea.
Index offset: 0x5EC this 32-bit integer is the most important piece of data contained in the header. It contains the address of the index header, and afterwards, the offset to the index. The offset to index is necessary to calculate the magic. Magic = (index magic - (offset to index + 40 )) 0x5EC is always the address, in any demo map for the index offset.
Meta data size: 0x2C4 this 32-bit integer is the other useful piece of data contained in the header. It tells us the size in bytes of all the meta data. Once again, it's not necessary to update this with rebuilding, but it is recommended. In an original, unmodified demo bloodgulch map there are 57 89 F4 bytes worth of metadata in the map.
Map name: 0x58C this string contains the internal name of the map, in this case bloodgulch, and goes on for about 20 bytes. Though there is one byte at the end of the 00's there, it has no effect on the map. However should be the same as the name of the map file.
Map build date: 0x2C8 this string indicates the build data/version of the map, which goes on for 0x1C bytes. Demo, full and CE each have different build dates but changing the dates around doesn't seem to have much effect. CE might be more specific about the build date according to some sources.
Map type: 0x02 this 32-bit integer indicates whether the map is a single player map (value = 0 ), a multiplayer map (value = 1) or a UI map (value = 2). These three types are the only ones that contain a normal mapfile header (yes you can open ui.map in most editors).
Foot string: 0x5F0 this string in a full map represents the head of the header data. Since demo scrambles the header it no longer is at the end of the data. On normal maps it is "toof" while on demo maps it is "tofG". It is necessary in the halo demo mapfile, otherwise Halo Demo will not read the map and throw a file not found error.
At this point all the data can just equal 00 until 0x800. Its just random, meaningless data that was probably used to confuse modders or people who are reverse engineering the files. Overall there are 0x7B8 meaningless bytes in the file.
- Map Reading:
Now, that there's been a bit of explanation, lets get started with the basics of map reading.
Open up a halo demo map with your favorite hexeditor. (I'll be using 0xED)
Also open HMT, (you don't need to open the map) and just open the Hex Calculator, (window/hex calculator)
Lets go to the index offset, which starts at the hex address: 5EC
The index offset, is a 32 bit integer, (meaning 4 bytes)
In a normal BG demo map, the byte sequence is as follows: (74198500).
Make it big endian, (00851974)
Let's go to that address. In hexedit, cmd+j, or in 0xED, just type it into the Goto Offset Field. Type in the address, and hit enter.
Welcome to the index header.
Take the first 4 bytes (2800F14B) and make it big endian, and subtract 28.
(4BF10028) - 28 = (4BF10000) (If your bad at math, or more comfortable using hex calculator, it looks like this:
Now we have the index magic: (4BF10000)
If you are on a demo map, the index magic will always be (4BF10000)
Now, lets get the map magic:
Take the index magic, and subtract the index offset from it. (4BF10000) - (00851974) = (4B6BE68C)
So now we have our map magic: (4B6BE69C)
Now, lets find the number of tags in the map.
Take the index offset, (00851974) and add (0C) to it.
This is the address to the offset of a 32 bit integer, which of course, is 4 bytes. (6A090000)
Make it big endian (0000096A)
Then, make it into a Decimal value. We get (2410)
We now have the number of tags in the map. (2410)
Now, we'll find the tag array.
Lets go back to the index offset, (00851974) and add 28 to it.
Welcome to the tag array. Which is started at the address: (0085199C)
The next 4 bytes: (726E6373) are actually meant to be read in string, not hex.
The string value as you can see, is (rncs)
Make the 4 bytes big endian: (73636E72) and the string value is (scnr)
You've found the tags.
Each of these are 20 bytes long
Split into 6 parts:
Main tag class (4 bytes)
Seconded tag class (4 bytes)
Third class (4 bytes)
Tag ID (4 bytes)
Name Offset (4 bytes)
Meta Offset (4 bytes)
Now, let's get some offsets.
The first tag, as you can see is scnr, which is the tag class.
Now lets get the full name.
First, we need to find the Name offset, right now, we're at the main tag class offset.
So, in hex, passed all the bytes before it,
Let me explain that better: Since we are already at the Main Tag Class, we don't need to consider those 4 bytes that we are already at.
The first 4, represents the seconded tag class byte sequence.
The second 4, represents the third class sequence of bytes.
The third 4, represents the tag ID sequence of bytes
The last 4, represents the name offset sequence of bytes.
All together, in hex, that makes 10. (4+4) =8 (8+4) =C (C+4)=10
However, if we were to find the meta offset instead of the name, then we would add the last 4 in there. Which naturally, is 14. But, since we're just going to find the name offset right now, we're gonna remember 10.
Ok, we've found the tag array already, and found the main tag class offset.
We now have the byte value that we need in order to find the name offset. (10)
We'll now repeat the step we used to get the tag array, except using the name offset byte value. (10)
So, again, lets take the index offset +28 (where we were previously(0085199C))
Add 10, and we get (8519AC) that's the address to the name offset.
Which in little endian, is (682DF24B)
Make it big endian, and we get. (4BF22D68)
[url=http://i288.photobucket.com/albums/ll19 ... ffsetA.png]Screenshot
Now, we take the map magic, (4B6BE68C) and subtract it from the name offset. (4BF22D68).
(4BF22D68) - (4B6BE68C) = (008646DC)
We now have the address to the name offset. (008646DC)
Go to that address, and there you'll see, the names of the tags. Or the Name Offset.
[url=http://i288.photobucket.com/albums/ll19 ... Offset.png]Screenshot
That is how you calculate just about any offset in a Halo map.
( The offset - Map Magic = The offset you want )
This will work for almost anything, from chunk counts, to tag references.
As stated before, to go to the tag's meta, you do the same thing, ( Meta offset - Map Magic = offset you want)
Now, just to avoid confusion, I'll also demonstrate how to get to the tag's meta.
Lets start with the tag array: (Index offset + 28)
With the hex address of (0085199C)
Remember how it was split up? Main(4) 2nd(4) 3rd(4) ID(4) Name(4) Meta(4)
We're at the main tag class byte sequence already, so we need to move ahead however many bytes we need to get the address for the meta offset. (4+4+4+4+4=14)
So, Take the index offset, and add 14. (0085199C) +14 = (008519B0)
Go to that address. (008519B0)
Make those 4 bytes big endian.
We now have (4BF3EDA0)
Subtract the map magic from it. (4BF3EDA0) - (4B6BE68C) = (00880714)
Now we have the address for the meta. (00880714)
Go to that offset
Welcome to the meta.
- Things we've learned!
Well, We've learned some information on RAM memory, that most computers store it in little endian, however if you are on ppc, It's big endian, and can be reversed to little endian using some techniques 'Little Endian Processing Mode, & Byte order Load ad stored instructions. "More info Here"
We've learned some terms, and some information on the map file.
We've learned how to calculate the Map Magic.
We've learned how to locate the address for the offset you want.
(index offset - map magic = offset you want)
Worked on this thing for 3 freakin days... practically nonstop. Sure hope it helps.