Cheat Device Post-Mortem

tl;dr: I created a cheat device for PlayStation 2 similar to CodeBreaker, GameShark, and Action Replay. In this post I share my motivations and experiences while developing it.

Origins

Video game enhancers (or cheat devices) on 5th and 6th generation consoles have primarily existed as commercial products developed using reverse-engineered SDKs. In the case of ones created for the PlayStation 2, they even figured out how to press discs that can load on retail consoles without any modifications (!) or additional trickery. One of those cheat devices, and the one that is arguably the most popular among console modders due to its simplicity, is CodeBreaker. Published by Pelican Accessories (now Performance Designed Products), it offers straighforward cheat support for encrypted and unencrypted cheat codes, a large built-in cheat database, a save manager, and the ability to upgrade the cheat database using a flash drive.

Unfortunately, this feature suffers from a major flaw stemming from being developed with a reverse engineered SDK: it doesn’t work on slim-line PS2 models, leaving out a pretty large amount of consoles. Realizing this issue, in addition to others, I set out to create a new open source game enhancer using a newer SDK.

My main objectives were to:

  1. Create a game enhancer similar to CodeBreaker but with improved usability and compatibility.
  2. Allow for fast loading of large cheat lists.
  3. Allow for editing and saving of the cheat database on-console.
  4. Develop with the constraints of the system in mind: 32MB RAM and a slow USB 1.1 interface.

PS2RD GUI

Game menu in the PS2RD GUI

PS2RD (PlayStation 2 Remote Debugger) is a mature homebrew utility created by Mathias Lafeldt that can be used to enable cheats in retail PS2 games, among other cool features. It had a very simple text-based UI, so I decided to take a stab and create a new GUI similar to CodeBreaker.

This was my first “large” software product created on my own, and I didn’t really know what I was doing (I learned a lot though, and that’s what counts). It used PS2RD as a base and displayed a simple menu to select cheats and boot games. It was pretty ugly, with a simple blue background and text rendered with a mono-spaced font, but it was functional. Despite this, I remember someone on PSX-Scene defaced it and added a credits screen taking credit for the GUI. I guess someone liked it enough?

Eventually I ran into issues that would require a fairly substaintial rewrite to overcome, including:

  • Loading cheat lists was very slow, as each game, cheat, and code struct was allocated from the heap separately.
  • I dug myself into a hole by creating a messy, poorly thought out menu system primarily residing in a single source file. Remember, I learned a lot during the course of this project!

After realizing the issues at hand, I took it upon myself to create a new project from scratch: Cheat Device.

Cheat Device

Cheat Device is a clearer realization of my goals for a game enhancer and was developed on a clean slate. Rather than building a menu within the existing PS2RD code base, I created a new project that included PS2RD’s code engine as a seperate module.

Cheat Device features a pretty menu system.

Custom File Format: Cheat Database (CDB)

Quickly loading large cheat lists was the first goal I wanted to acomplish. PS2RD allocated structs one-by-one, so I decided to create a new binary file format that included fields for the number of games, cheats, and codes allowing each structure to be allocated at once rather than calling malloc thousands of times. To speed things up even more, the database was compressed using zlib to make up for the system’s slow USB 1.1 transfer speeds.

I admitedly didn’t think through the format very well. The format was designed to place all the game titles in a single chunk, followed by all the cheats in another chunk. Entries in the game chunk included an absolute offset to the start of the game’s cheats in cheat chunk, which required some trickery to get correct while generating the file. I was originally indending to load only the game titles and cheat offsets to create the game menu, then load the cheats “on demand” to create the cheat menu. Ultimately I deceided to load the games and cheats at the same time all at once, as it ended up being much simpler.

I can summerise the main issues with the format as follows:

  1. I was overestimating the constraints of having “only” 32MB ram (which is actually plenty for what it’s being used for here) and tried to optimize for the wrong reasons.
  2. I mistakenly thought If I placed the game title strings closer togther, it would compress better and the overall filesize would be smaller. I later discovered that zipping a text file of cheat codes actually yielded an even smaller file size!
  3. The final file is a raw zlib stream without any headers! This makes it difficult to determine if it’s a valid file or not without relying on decompress2() to fail if something is wrong.

Responding to User Feedback

I eventually realeased the first version in early 2015. It was mentioned on some homebrew scene sites such as psx-scene.com, eurasia.nu, GameHacking.org, and a few others. People seemed to like the interface and simplicity of being able to load games from disk or an ELF file.

One of the main requests from users was for a more simple method of creating cheat databases. At the time, Cheat Device required  using a seperate program (cdb-util) to create the cheat database file, and updating the cheat database required running this program each time. Users were confused as to the motivation behind using a custom file format and didn’t like having to use a command line utility to create it. Loading cheats from a text file seemed like a daunting task at first as I knew it would be quite slow, but I wanted Cheat Device to be straightforward and easy to use. Either I needed to:

  1. Read the text file in a single pass, allocating game and cheat structures one-but-one.
  2. Read the text file in 2 passes: the first pass would determine the token-type for each line (game title, cheat title, code line, comment, etc.), then allocate all the games, cheats, and codes at once using single calls to malloc() for each structure type. The second pass would populate each game, cheat, or code line using the token-list to determine the line type.

I decided to use the method described in 2., but it was still very slow. I needed to somehow allocate the structures on demand, but didn’t want to call malloc() thousands of times. Allocating everything at once (as I was doing for loading CDB files as well at the time) would make it very difficult to modify the cheat database on the console. How would I delete a structure if I can’t just free() it? I don’t want to overcomplicate things with manual bookkeeping either. So what did I do?

Answer: Object Pools

I decided to replace the calls to malloc() that allocated all the structures at once with two static object pools. Now allocating and deallocating games and cheats is O(1) with minimal overhead for bookkeeping. To delete an object, I push it’s pointer onto a “free list”, and to allocate an object I either take the last entry of the free list or increment a tail pointer. Allocating in this way ended up being thousands of times faster than using malloc to load cheats from a text file in a single pass. This is possible due to using a fixed-size struct for games and cheats.

Once object pools were added in, I could quickly load a text file of cheats in a single pass. Most recently I expanded upon this furthur by allowing a text file of cheats to be loaded from within a ZIP file, creating an easy way to compress a large, easily maintainable cheat file without relying on a seperate program or file format.

Lessons Learned

I learned more from the journey of creating this project than I would have expected! Here’s a few key points:

  1. Resist early optimizations. I ended up creating a poor, unmaintainable file format based on a false hypothesies (that putting a subset of the strings at the beginning would make it compress faster) to solve problems that could have been better solved in other ways. I’m being a little hard on myself here, but the experience gained from this was invaluable.
  2. Try to consider future use cases when picking data structures. I figured I could just call malloc() once and fill in a bunch of structs because I didn’t originally plan to allow for editing or saving the cheat database on-console. I ended up having to do a lot of refactoring to make that happen.
  3. Centralize user documentation. When I first release Cheat Device, I created posts on GameHacking.org, psx-scene, and my website linking to the program and providing documentation within the post itself. This made it a pain when I changed something important, as I would need to manually update the various posts about the project. Now I put virtually all documentation in a wiki hosted on GitHub and can link directly to that rather than repeating information that might eventually becomre outdated/incorrect across multiple websites.

Final Thoughts

I’m pretty happy with how the project has turned out and the experience I have gain in the process of developing it. I’m not an expert at PS2 development by any means, but I’ve enjoyed getting to know my favorite game console as a kid at a low level. I’ve spent hundreds of hours implanting features, fixing bugs, and trying my best to make the code readable and follow best practices. Maybe creating a similar project for Dreamcast, PS1, or another game console would be fun, who knows! Internationalization and support for PSV game saves would be a nice addition too.

You can download the newest version of Cheat Device here: https://github.com/root670/CheatDevicePS2/releases/tag/v1.4

Cheat Device approaching version 2.0!

I’ve been very busy with life stuff but have been working on Cheat Device occasionally for a few months and am ready to share a new update!

A feature many people have asked for is now available: console-side cheat database editing! This took me a while due to how I was allocating memory in older versions but the code has been refactored to allow for editing without imposing large loading times. This was accomplished by using a fixed-size object pool and replacing linked lists with pointer arrays for the menu items. Now you can add, edit, and delete games, cheats, code lines, and boot paths within Cheat Device. I’ve also polished the graphics routines after discovering I wasn’t handling alpha values correctly. Lots of other changes have been made, such as replacing button references in the bundled cheat database with button graphics. Ex: “Press circle” is rendered as “Press O”, where O is a circle button graphic. The game list is alphabetically sorted too.
Note that the cheat database file isn’t being updated by Cheat Device yet, so any games/cheats/codes you add/edit won’t persist after powering off the console. This will be implemented in the next release.
I will release updated documentation in the next few weeks, which isn’t a lot of work but I need to find some time to work on it. Once things seem to be stable based on feedback and testing, the new release will bump Cheat Device to version 2.0!
Download: https://wescastro.com/CheatDevicePS2…0-gcbb20ec.zip
Enjoy!

Cheat Device for PS2 Update

I’ve updated cdb-util (formerly makecdb) to support extracting text cheats from CDB files. CDB (CheatDataBase) is the format used by Cheat Device for storing its cheat database (as the name implies 🙂 ). A few users requested a way to extract cheats from existing CDB files so I figured it would be a useful addition.
Download: cdb-util v1.2

Initial D Progress

Some good progress has been made on the Initial D project. pactool seems more or less solid and has been able to extract and create PAC archives without issues. gim2png (really gimtool now) can convert GIM textures to PNG as well as inject a PNG back into a GIM. It’s much easier to inject a PNG rather than create the whole GIM structure from scratch since I don’t fully understand the format. When injecting a PNG, it will be palettized to fit either a 16 or 256 color RBGA palette depending on what type the target GIM uses.
Here’s what a modified texture will look like in game:

Texture injection test.

As for actually translating parts of the game, I’ve made some progress but it is very tedious. I don’t know any Japanese, but Google Translator has been surprisingly good at identifying characters I scribble in and spits out translations that (for the most part) seem accurate. I’d definitely like to sit down and learn parts of the language, but that’s a job for another day.

Initial D Special Stage Translation

ME0000340767_2
 Initial D Special Stage is a port of the popular arcade game released by Sega. There are English releases of the arcade versions, but none of the home versions have ever been released outside of Asia. This is too bad, since Special Stage was a great port considering the hardware limitations of the PlayStation 2 (the arcade version’s hardware uses TWO video processors and has 8 times more video memory than a PS2!). Nevertheless, I’ve decided to start work on a translation patch for this game. Yes it’s a racing game and the text is probably not that important to read, but it can be challenging to navigate the menus that are entirely Japanese.
Continue reading “Initial D Special Stage Translation”

ps2rd Cheat Device GUI Update

Last year I began work on a cheat device GUI for ps2rd (an excellent remote debugger for the PlayStation 2). The goal of the project is to create a menu-driven cheat device similar to CodeBreaker or Action Replay using ps2rd’s ASM cheat engine. I wanted this to be just a cheat device, so I removed most of the debugging functionality from ps2rd for my project.
I had some freetime over the past few days and made some updates to it. It’s now more stable than previous releases and has had more unneeded parts of ps2rd removed (such as videomod). BOOT2 selecting now works as it should, so you can boot to a ELF instead of directly to disk. I’ve also added preliminary support for compressed code lists using miniz. Basically, you use the included “compress-decompress.exe” program to compress/decompress your cheat files using zlib compression. When using a compressed cheat list, it must have a file extension of “.bin” or else it will try to load it as text. Update ps2rd.conf to reflect whichever cheat list you’re using. This should help cut down on loading times.
If you find any bugs, please let me know!
Todo:

  • Fix delete option (currently does nothing)
  • Implement less-awkard cheat list compression (maybe use zip/7z files instead)
  • More code cleanup
  • PS2-side codelist manipulation (add/modify/delete/etc.)

Latest version:
http://www.mediafire.com/download/5r…2092013-r2.rar
Updated source code will be available soon. Source on GitHub is not the latest version.

Text Encoding in Kingdom Hearts

Over the past few weeks, I’ve taken a look at the way text is stored in the game Kingdom Hearts. Like other games made by Square, Kingdom Hearts uses a special character map for strings throughout the game. The US and PAL versions of the game use a character map consisting of Latin characters and common symbols. The Japanese releases use a different table containing Japanese characters as well as uppercase Latin letters (there are a few lower case ones though, but not a full alphabet). Most text is compressed using a LZSS-like algorithm, but I was able to poke around at some of the non-compressed areas. For example, the menu items on the pause screen are not compressed and can be modified if you know the correct characters to enter. So far I am able to change these items as long as my modification’s don’t overlap any neighboring strings.

I started with the original Japanese release of Kingdom Hearts and was able to do this by modifying the game’s ISO:

Translated menu items in Kingdom Hearts NTSC-J

I created a tool that can convert plaintext to the character set used in the game. If you’re interested you can download it here. It requires .NET Framework 4.0 SP1 and currently cannot convert Japanese characters. A table created by Falo can be used to manually convert them by hand. I’ve tested it on these versions of the game:

  • Kingdom Hearts (SLUS-20370) USA
  • Kingdom Hearts (SLPM-66122) Japanese (Uses Shift-JIS encoding)
  • Kingdom Hearts Final Mix (SLPM-66123) Japanese (Uses Shift-JIS encoding)

I’m hoping someone will see this post and be motivated to continue work on a full translation patch some day for Final Mix. I don’t have the time to do it myself and it might be pointless to pursue it once Kingdom Hearts HD 1.5 is released in the US, since that contains the additions that were present in Final Mix. KH-Vids.net user Hidden Smithery has already offered help and was able to translate the following for the Japanese versions of the game:

Item: エーテル                    ETHER
Code: E3 55 F2 19 08             32 41 35 32 3F
Item:メガエーテル                  MEGA-ETHER (name is too long)
19 01 19 0E E3 55 F2 19 08      3A 32 34 2E 2C 32 41 35 32 3F
Item:ポーション                    POTION
Code:19 27 55 EB 19 2F 19 0D    3D 3C 41 36 3C 3B 01 01
Item:メガポーション                             MEGA-POTION
Code:19 01 19 0E 19 27 55 EB 19 2F 19 0D    3A 32 34 2E 2C 3D 3C 41 36 3C 3B 01
スロット                           SLOT
EC 19 0A 19 30 F3               40 39 3C 41 01 01
ソラ                             SORA (too long)
EE 19 06                        40 3C 3F 2E
たたかう                          ATK (the word "ATTACK" is too long)
9F 9F 95 92                     2E 41 38 01
まほう                            MGC (the word "MAGIC" is too long)
AE AD 92                        3A 34 30