DTE
Required:
Files:
- Windhex
- Final Fantasy III (US, snes) rom.
- Final Fantasy III main tablefile.
- Final Fantasy DTE-less tablefile
- Snes emulator of your choice. Should support savestates (Snes9x does this).
In this tutorial:
What is DTE?
DTE is a type of text compression, used solely to cram as much text as possible into an area as small as possible (at best, you'll fit 50% more text than you would've done with uncompressed text). Unlike other type of compressions, this one is rather light. You don't need any previous experience with compression or ASM (nor do you need any other tools then a hexeditor) in order to take advantage of the DTE that can be found in some games. This is, on the contrary, something you can (and should!) do whenever possible.
DTE stands for Dual Tile Encoding. Just like the name implies (well, sort of), a DTE routine is used to output several (namely 2, hence the "dual") tiles using a single hexdecimal. This is quite handy, since many games store one letter per tile. Here's an example from Final Fantasy 3:
---
Take the word "where". It's a simple and common word in english, and games with lots of dialogue will most likely make good use of this word. In other words, it would be in our best interest if this word required as little space as possible to store inside the rom. Up until now, all text we've worked with (save for the Pointer-chapter) were written in a non-compressed form. Basically, this means that 1 letter = 1 hexadecimal in the code. Basics of the basics. It looks like this:
[w][h][e][r][e]
[] = 1 hexdecimal
As you can see, this word would require 5 hexdecimals to store in a rom, should it be uncompressed. But suppose we could use DTE in order to store two graphical tiles (i.e letters in this case) into a single hexadecimal?
Let's store "wh" and "re" into their own hexdecimals. Now we could type out "where" like this:
[wh][e][re]
As you can see, the number of hexdecimals required to type "where" has shrunk from 5 to 3. And not only "where". Every word in the game that uses "wh" and/or "re" now requires one or two hexdecimals less. You see the potential of an extensive DTE library, don't you?
*** TERMINOLOGY ALERT! ***
A DTE "library" is simply the term for all the DTE-combinations in the game. They aren't stored in an actual library or anything.
---
Finding DTE
All is not good and well with DTE, though. Like every other compression-scheme out there, the first hint of its presence you'll get, is that when you search for text (using a Relative Search, since you haven't prepared a tablefile yet) you won't get any (relevant) results.
This, however, doesn't mean that the game uses DTE, since virtually any compression will make it difficult to locate the text simply by searching. Just to be safe, here's the reason why words containing DTE compression won't turn up when you're searching:
---
Open your FF3 tablefile (the one containing DTE) if you want. Here are some of the DTE-combinations that the game uses:
80=e
81= t
82=:
83=th
84=t
85=he
86=s
87=er
88= a
89=re
8A=in
8B=ou
As you can see, some of them contains only one letter. That's because they come together with a blank-space. Basically every word in the entire game (at least in the dialogue) has a blank between every word, so it makes sense to include them as well.
Look at number 83, "th". What it means, is that the game uses the hexdecimal 83 in order to display "th" on screen.
Now go further down the file, and find the hexdecimals for "t" and "h":
4D=t
41=h
Obviously, the game also needs to output "t" and "h" apart from each other, so the "originals" are also present (along with the rest of the alphabet), using their own hexdecimals.
Let's type the word "there" without using DTE (check your tablefile if you want to):
[4D][41][3E][4B][3E]
[t][h][e][r][e]
Simple, huh? Windhex has no problems finding this line using a relative search. This is what it would look like if we used "th" as DTE:
[83][3E][4B][3E]
[th][e][r][e]
Now Windhex can no longer find the word, since DTE-combinations in no way has any relative positions compared to regular text. We could search for the last three letters, since they're still using regular hexdecimals, but we'd probably get too many results this way.
---
Ok, what now? Say we searched a rom, and got nothing. We don't know which hexdecimals are used to output the text. We can't search for any words, since the text most likely is compressed. Heck, we don't even know if the compression in question is DTE, or something else (in which case we're wasting our time). And even if we knew that we were dealing with DTE, we still can't search for any words since we don't know which parts of them that consists of DTE (not every game uses the same set of DTE combinations).
In this situation, we have to assume/hope that DTE is used. It might be a waste of time, but some people would say that romhacking in itself is just that, so who cares?
After that, we will try to locate some of the regular letters (non-DTE) in the rom. If we can build a tablefile consisting only of the regular letters, we can locate parts of the dialogue and work our way from there. Here are some alternatives to this:
---
1) The quick-fix
Check the internet if someone has alreay made a tablefile and put it up for grabs. This won't help you to learn anything about DTE, but it's the quick way. If the game you're working on is in japanese (and you find a tablefile), you might want to check if the game has already been translated.
2)
Check FF3's tablefile again. How many DTE-combinations (80 -> FF) can you find consisting of upper-case letters? The answer is none. Since upper-case letters are so scarce (compared to lower-case), there's usually no reason to include them unless the entire text consists if nothing but upper-case letters (and we know this isn't the case with FF3).
When you have come across what might be DTE in a rom, try to locate a word consisting only of uppercase letters, and see if you can find it using a Relative Search. If you can, proceed to build a tablefile using the data you've found. You can also edit the hexdecimals in the word, to see which hexdecimals that results in a DTE-combination, and continue to build your tablefile from there.
This would require you to try 00 all the way up to FF just to be sure. Not recommended unless you have a lot of spare time.
3)
Even though games aren't required to contain a certain set of DTE, you'll find that some combinations are more common the others. "th" and "wh" are examples of these. Because of that, searching for words like "this", "where", "light" etc. usually never work. Instead, try to find a word (or a part of a longer word) that doesn't seem to include any combinations worth defining as DTE.
An example would be "equilibrium", since "eq", "qu" or "ui" aren't all that common. With a little luck, "il" won't be defined by DTE either, so it might be worth it to search for "equil" or "equ" (and sort through the results).
---
To sum it up, this is what you should do when you believe you're working with DTE:
- Find a part of some text (in your hexeditor) that doesn't contain DTE (step 1-3 above). Make sure you can bring that text up in an emulator as well.
- Modify it a little (just replace some hexdecimal) and load it up in the emulator to make sure you're working with the right text.
- If you've found the right text, try to make a tablefile of the regular upper/lower-case alphabet (preferably both).
- Next, load that tablefile in your hexeditor. If you did it right, parts of words should pop up in the editor, since no text consist of 100% DTE.
- Now you can either look at the words in the hexeditor and try to guess the DTE being used (while inserting it in your tablefile and loading it to make sure you got it right). Little by little, the text should start to appear this way.
- Or, you can simply use your new-found text and insert a hexdecimal (start from 00 to be sure), and see if the result is a new DTE-combination on screen (in your emulator). If it is, add it to your tablefile. If it isn't, move on to 01 and so on (this takes time).
Looking at DTE
Just to steer clear of all the theoretical stuff for a while, we'll take a look at some DTE in FF3 before moving on. This is where you need your DTE-less tablefile. Also make sure that you're using the original copy of FF3, and not the one we modified in the chapter about Pointers.
Start up by loading FF3 and your non-DTE tablefile in Windhex. In case you're wondering, this tablefile is the same as the other one, exept all the DTE-combinations has been removed and thus won't show.
Now, go to offset (Ctrl+G) 0D02FF. If you did it right, this is what you should be looking at:
As you can see, even though FF3 uses quite a few DTE-combinations, regular letters are still found here and there. If we searched for "worry" or "ught", we would get accurate search-results. Of course, if we actually had to make a tablefile, we wouldn't have access to this one, so there wouldn't be any way for us to determine what to look for. This just goes to show how scarce long strings of non-DTE code can be in some games. You should also have noticed that "WEDGE" still displays fine, since his name consists of nothing but upper-case letters.
Next, we'll load our regular tablefile (the one containing DTE). It looks like this:
The red characters are DTE, as indicated by their color. Yellow ones are combinations that exceed 2 characters in length (even though they only represent one hexdecimal in the tablefile).
The reason why every line suddenly has begun to break the 16-characters-per-line-rule, is because this rule only applies to hexdecimals. There's still only 16 hexdecimals per line, but some of them outputs more than one letter, thus forcing the row to expand.
Modifying DTE
Part 1: Some editing
Next, we'll try to edit some DTE found in the game, and then we'll move on to adjusting the actual routine to better suit our needs.
If you've gone through the Pointer-chapter, you'll still be sitting on a savestate that takes you to this place in FF3:
If not (shame on you!), play until you reach this point in the game and take the savestate before any text is even loaded onto screen.
Now, in Windhex (with your regular tablefile, the one with DTE), go to offset 0D04EB (Ctrl+G). This is the text from the image above.
First we'll start with adding some DTE into WEDGE's name, using regular-onehex-characters only. Put the marker on "E" in "WEDGE", make sure you've gotten "hex / text -edit mode" unchecked, and type "9F". This way, "E" becomes "on". Still only using one hexdecimal. Two letters on screen. Neat.
Before you attempt to write entire sentances using DTE (go to "Edit > Enable / Disable DTE inserting"), let me warn you that Windhex makes it quite a hazzle. It lacks the ability to optimize the number of combinations (understandably, since it can't tell which input that comes next), and thus you'll most likely have to back-paddle in order to adjust previous errors. Thingy does this a little better, but otherwise falls short compared to Windhex.
DTE is, however, possible to work with when you're inserting scripts (tutorial coming), since whatever program you might use have access to the entire text at once.
Part 2: Modify the actual DTE
Ok, so inserting some random DTE-related hexdecimal wasn't all that difficult, was it? But ponder the fact that you might want to edit the DTE-combinations themselves. Perhaps you're translating a game to some other language then english, and find that you have absolutely no need for combinations like "th" and "wh".
Here's a rule: Even though you only use their assigned hexdecimals in the text, the "originals" can still be localized somewhere in the rom. In other words, when the game sees the hexdecimal "83", it looks - somewhere - in the rom in order to find the letters "t" and "h" in their original form, then outputs them onto the screen.
Now, we will find those originals, and edit them. Here are the the two main ways to locate them (a rom rarely stores DTE in both ways, so you'll have to find out which one that is used):
---
Note: Both methods assume that you have a complete tablefile at hand.
1)
Take a number of DTE-combinations from the tablefile that can be found next to each other (thus "ranked" ascending according to their hexadecimal, with no exeptions). The safest way is to take the ones in the beginning of your tablefile.
Example from FF3:
80=e[b]
81=[b]t
82=:[b]
83=th
84=t[b]
85=he
86=s[b]
87=er
Note: [b] = blank space, just to make it clear.
Take those combinations and write them next to each other in the same order:
e[b][b]t:[b]tht[b]hes[b]er
or
e t: tht hes er
Do a regular text-search (Ctrl+F in Windhex) for this exact line, and you'll find them (along with the remaining DTE). Easy as that.
2)
Take the same number of DTE-combinations from the tablefile as you did in the previous method.
Example from FF3:
80=e[b]
81=[b]t
82=:[b]
83=th
84=t[b]
85=he
86=s[b]
87=er
Take the first character/letter/blank-space/whatever from every combination, and type them into a row of their own:
e[b]:tthse
Then do the same with the second character in every combination:
[b]t[b]h[b]e[b]r
Do a regular text-search for both these lines, and you should find them. Usually they're located very close to each other, so you'll only have to search for one line. You'll then have to edit them one at the time.
Note: If a rom stores DTE this way, it is possible that said rows of text are divided to different parts of the rom. If your search won't give you any results, try to shorten the line ("e[b]:tthse" -> "e[b]:tt", for example). Since there's no way to tell where the line has been divided, it's safest to search for the first characters in your tablefile.
---
FF3 stores DTE according to method 1. Therefore, in Windhex, go to the beginning (top) of your rom, and search for "e t: tht hes er".
You should end up here, at offset E1A0:
Recognize those combinations? Check your tablefile carfully if you don't, since those are the DTE used by FF3!
Go to the end of line 1 (to "er") and make sure that "hex / text edit-mode" is checked. With normal, non-DTE, characters, write "qu" instead. Save and load FF3 into Snes9x.
And so it goes.
As you can see, "er" in "her" (and every other word where the hexdecimal 87 is used) has now been replaced with "qu" instead. You can edit the remaining combinations in the same manner, just remember to edit the tablefile as well, otherwise any programs you use won't reflect the changes you've made to the rom (the text will still read "her" in Windhex, since '87=er' in the tablefile).
Library DTE
Apart from the regular 2-byte combinations, some games also store entire words in a single (or sometimes more than one) hexdecimal. This is referred to as Library DTE, and works about the same way as normal DTE. The difference is that the words are stored like regular text, with pointers pointing to them (how to calculate these, can be found in the pointer-chapter depending on which format you're working with).
If you want to translate these, you'll probably have to modify said pointers. When it comes to tablefiles, they're added in the same way as normal DTE.
37=Castle
0218=Vengence
(^ for example)