Although I’d transferred across the score server files and the database, the score server was crashing for some reason. It took a bit of investigation as I was a bit confused as to why the server was crashing outright instead of throwing an exception. After a bit of poking around I stumbled across the following error in httpd’s error log:

PHP Fatal error: Class ‘DOMDocument’ not found in /var/www/html/ScoresServer/index.php on line 64

Turns out our server didn’t have the XML modules installed for PHP which was causing a fatal crash within PHP (no exception is thrown when a class can’t be found). Once I knew this the fix was of course as simple as SSHing into our server and entering the following:

yum install php-xml
service httpd restart

Anyway, the score server is now up and running again so go download Anti-Squish or Anti-Squish Lite from the Android Market!

I’m sure the details aren’t really of all the much interest to you non-developer folk, so I’ll save you all from boring details. However, I should probably save other developers from going through what I had to. If I get a chance a separate blog post that details how to get a cross-compiling environment up and running on OS X Lion is definitely in order.

Apple-specific issues aside…

Heimdall 1.3.1 is quite an important update. Apparently Heimdall’s repartitioning functionality was not working in prior releases. However with some help from the XDA developers community a fix has been implemented and verified as working.

On top of repartitioning fix there have been substantial improvements made to Heimdall’s –no-reboot functionality. Previously not rebooting your device was only really useful if you wanted to boot straight into recovery mode after performing a flash. However the changes made in version 1.3.1 allow Heimdall to detect for and resume an existing session with a device. This is particularly useful for Heimdall Frontend users who want to dump their PIT file and then immediately perform a flash. From a usability stand-point this is a very nice improvement.

Downloads are available from the Heimdall product page.

I don’t know how much Anti-Squish will appeal to you die-hard firmware flashing folk, but if you’ve got some time to kill then please give it a try. There’s a Lite edition with limited functionality as well as the fully functional $2.00 (AUD) paid version. Even if you’re not overly interested you could consider your purchase a small donation towards Heimdall’s cause. If you head over to the Anti-Squish product page you can see some screenshots and use the direct Android market links to download the app.

Now onto Heimdall Suite 1.3.0. Firstly, this is a enormous release. Some additional functionality has been added to Heimdall (command line) that adds the ability to download a device’s PIT file and Galaxy S II support has been vastly improved. However the real additions come in the form of a completely revamped Heimdall Frontend. However the changes aren’t just visual, in particular Heimdall Frontend 1.3.0 now includes support for the creation, loading and flashing of the brand new Heimdall Firmware Package format!

If you’ve come from using Heimdall Frontend 1.1.1 then you may be slightly intimidated by the new “Flash” tab. The “Flash” tab is the place where you can manually map individual files to partitions on a device and perform a flash. This is the same as general idea as the previous release of Heimdall Frontend. The new implementation is substantially more robust. Other than the user interface changes, the main difference is that in order to perform a “custom flash” you must always specify a PIT file. This is regardless of whether or not you intend to repartition. The PIT file contains device specific information that Heimdall Frontend utilises in order to ensure the same user interface can be used with all Galaxy S devices. This includes device that are presently available as well as devices that are yet to be released.

Galaxy Tab and Galaxy S II users in particular will have noticed that the old Heimdall Frontend 1.1.1 did not provide them with a means to flash all the partitions on their device, also some partitions listed may not have even existed for their device. The new PIT file based user interface ensures that you’re able to flash all the partitions on your device and you’ll never see partitions listed that aren’t for your device.

Of course the new PIT based user interface assumes that you have access to the PIT file that is presently installed on your device. Depending on where your firmware came from then you may have the correct PIT file laying around somewhere on your hard-disk. However that’s certainly not the case for all users and there’s always the potential of getting confused if you have multiple PIT files from different firmwares. This is where the brand new “Utilities” tab comes in.

The “Utilities” tab provides you with some additional functionality previously only available from Heimdall command line such as the ability to detect a device or close the infamous “phone <---> PC” screen. However most importantly, you can use this tab to download your device’s current PIT file straight from the device itself! No confusion, no mucking about, just connect your device to your PC in download mode, chose where you want to save the PIT file on your PC and download away. Now if you want to perform a custom flash it’s just as simple as returning to the “Flash” tab and selecting the PIT you just downloaded. No stress or confusion, you’re now able to flash exactly which partitions are presently available on your device!

The new PIT file based flashing is hugely important, but it’s not the killer feature. As mentioned above Heimdall Frontend now supports a brand new firmware distribution files known as Heimdall Firmware Packages. These are with out a doubt the preferred means of flashing. It takes the technical details of flashing of the shoulders of end-users and hands the responsibility over to the more knowledgeable firmware developers.

From the end-users point of view this new functionality is immediately available as soon as you open Heimdall Frontend, the tab is now the “Load Package” tab. After downloading a Heimdall Firmware Package all an end-user needs to do is click the “Browse” button and select the package they’ve downloaded. Heimdall Frontend will automatically decompress the package and display information about the package. The information available includes the firmware name, version, base Android version, developer names, buttons that link to the developer’s homepage and donation page, a list of included files, and most important for the end-user, a list supported device! Assuming the developer filled out the information correctly then straight away the end-user knows if the package they’ve downloaded will work on their device. If you don’t see your device listed then it’s pretty clear you don’t want to use the package, no more incorrect firmware bricks! Once you confirm your device is listed all you need to do is press the “Load / Customise” button and you’ll be swapped to the “Flash” tab, providing you with a chance to customise the package before flashing if you desire. Now assuming your device is connected to the PC in download mode then flashing becomes as simple as pressing the “Start” button.

This is absolutely great for the end users, however firmware developers are not forgotten.

Heimdall Frontend provides a clean interface for creation of a Heimdall Firmware Package. Simply fill out the “Flash” tab as you would when performing a custom flash, then make your way over to the “Create Package” tab, which will be available after you’ve filled out the “Flash” tab. Then it’s as simple as filling out the required information and pushing the “Build” button and Heimdall Frontend will spit out a fully redistributable compressed Heimdall Firmware Package! Not only does distributing your packages in this format guarantee that users can flash your firmware from Windows, Linux and OS X, the hugely simplified flashing procedure (over previous versions of Heimdall and current versions of Odin) means that you have the ability to help save a lot of bricks. There’s also the added benefit of ensuring users know who created the package and if you want you’re able to provide them with a simple means to donate to your cause as well as customer supported and detailed information from your homepage.

All in all Heimdall Frontend 1.3.0 includes some amazing new features that benefit end-users and developers alike. However compared to the previous release performing a custom flash is definitely a slightly more involved process. So the key feature of Heimdall Frontend 1.3.0 is without a doubt the new Heimdall Firmware Package functionality. As such I’m really hoping the Heimdall Firmware Package format will be heavily adopted by firmware developers.

Please remember to always read the README included in all Heimdall releases. The README includes detailed instructions for both end-users and developers alike. For Debian users the README will be installed to /usr/share/doc/heimdall/README. To download the latest release, make your way over to the Heimdall product page .

As always, a lot of effort goes into the development of Heimdall so please consider donating. Don’t forget to check out Anti-Squish, and have fun!

The update will be command line only (for now) and adds the following functionality:

Support for the Galaxy S II! (Hopefully, keep reading.)
• Support for SHW-M110S (Korean Galaxy S).
• Improved support for Galaxy Tab, Captivate and Vibrants (see below).
• No reboot option.
• Dump the devices PIT file and print the information in a human readable format.
• Support for more named partition parameters i.e. –hidden, –movinand.
• One more major feature (read below).

The biggest change is probably that proper support for PIT files is now included. This means that less things are hard-coded. Instead Heimdall will be grabbing data from your device’s PIT file or the PIT file you specify when repartitioning. This may seem like it has minor impact on the usefulness of Heimdall but it actually allowed me to add a second method of specifying files to be flashed to your phone.

You can now type:

heimdall flash --<partition identifier> <filename>

i.e.

heimdall flash --6 zImage

This essentially means you can flash any file your device supports as long as you know the partition identifier. Of course I don’t expect you to know them off the top of your head. This is where the PIT file dumping functionality comes in.

heimdall print-pit

Presto! You now have a complete list of all the flashable partitions on your device.

Another minor change is that I’ve removed the restriction on needing to specify bootloaders (or any files in particular) when repartitioning. However, this was in place as a safety feature. I honestly have no idea whether repartitioning without certain files is safe, and I have no immediate plans to find out. So please be careful!

As usual, to get your mitts on the latest release, just head over to the heimdall product page.

• Fixed modem flashes for Galaxy Tab i.e. Reverted from the “Kies method” back to the “Odin method” of flashing modem files.

System Support Changes:

• OS X binaries are now universal. Intel 32-bit, Intel/AMD 64-bit and PowerPC are all supported.

Installation Changes:

• The OS X package should now include Qt 4.7.1.
• Debian packages now install binaries to /usr/bin as opposed to /usr/local/bin.
• Debian packages now take care of udev rules in a more standard complaint fashion.

Compilation Changes:

• Heimdall Frontend can now be installed to any location by using “qmake OUTPUTDIR=/desired/output/dir”.
• On Linux running “make install” for Heimdall command line tool will now install the udev rules.

Whilst we’re talking about packages I should note that the packages listed here (in their present state) are not suitable for redistribution via regular repositories as the packages are almost certainly missing dependencies. The last 48 hours have been pretty hectic so it’s been a bit of a rush, but we’ll sort out the dependencies at some stage. I also have a strong feeling I’m going to end up having to statically link Qt into OS X binaries, bare with us though.

Okay, I’ve said this release is substantially more comprehensive than the previous release, if the packages aren’t great then what am I talking about? Well there are a quite a few new features, but I’m also referring to the shiny new git repository, courtesy off github. The github repository includes; source for both the Heimdall command line tool and Heimdall Frontend, a recent checkout of libusb-1.0, GNU build files, Visual Studio 2010 projects, drivers, scripts and documentation. There’s basically everything you need to get Heimdall compiled and running on any of the various supported platforms.

If you’re not a developer, you’re probably well and truly bored of all this source code, github and packaging rubbish by now, so I’ll get right to it. Here is a quick list of the new features and changes included in Heimdall 1.1.0:

• The inclusion of Heimdall Frontend.
• Galaxy Tab support.
• Simplified driver installation mechanism for Windows (zadig.exe).
• Better flashing progress feedback for non-verbose mode.
• Repartitioning.
• Removed the PIT file requirement when not repartitioning.
• Fixed the flashing progress bar on the phone/tab itself.
• Added the ability to flash the EFS, the recovery image and the backup copy of the secondary bootloader. It is intentional that only the recovery image is available from Heimdall Frontend.

Obviously the main feature is the inclusion of Heimdall Frontend. Tech-wizzes have already been getting great mileage out of Heimdall however the new GUI should make Heimdall a lot more accessible to your everyday user. The interface is designed to be similar enough to Odin so that users don’t have a heart-attack when they see it. However it also makes a lot more features immediately accessible, such as kernel-only flashes. It also includes safety-nets to prevent some of the more common flashing disasters, i.e. Repartitioning without flashing all the necessary files, or flashing an archive instead of individual files.

I’d like to extend a big thank-you to those of you what have donated, your support has made the last 48 hours somewhat more bearable.

Enjoy!

Anti-Squish is a simple touch controlled casual game. The goal is to score points by helping flies starting on the left or right side of the screen safely make their way across to the opposite side. With flies appearing all over the place be careful to make sure they don’t collide, and of course make sure they don’t get SQUISHED!

To download the app just click here (if you’re browsing on an android device), or search for “Anti-Squish” using the android marketplace. Feedback is very much appreciated so feel free to give us a buzz. We’ll be adding a product page shortly with more information (Update: Product Page). For now here are some screenshots:

… Well what are you waiting for, go buy Anti-Squish!

Getting Linux to run on the device is only half the battle, though. Drivers must be written for the audio system, the touchscreen, the wireless, the accelerometer, Bluetooth, etc. Given the dearth of publicly available data sheets on iOS-based device internals, the design of these drivers must be derived from the disassembly of iOS’ own drivers. Therein lies the challenge.

iOS is an OS X derivative, as Apple likes to claim. This means that it makes use of IOKit-based kernel extensions (“kexts”) to interact with hardware. However, unlike its desktop sibling, such kernel extensions are nowhere to be found on the (jailbroken) iOS filesystem. This is because iOS uses a prelinked kernel — for speed reasons and perhaps even to obfuscate its own internal workings.

Developers in the iDroid Project team have developed a work-around for this, but its not without its short-comings. The parsing process takes several hours (!) and is not amenable to significant changes in the drivers, i.e. adding / removing bytes. This is where we’ve started work.

There are some excellent open source tools required to even get to this stage, however. The prelinked kernel is distributed alongside iOS software updates (i.e. the ipsw files, zip files in disguise) as “kernelcache” files. These kernelcache files are wrapped in the IMG3 container format and compressed using LZSS. Both these encumbrances can be overcome using xpwntool. We can verify that we have successfully decrypted and decompressed the kernelcache file when the output file begins with 0xFEEDFACE, as detailed in the comprehensive Mach-O File Format reference.

With the raw prelinked kernel in our hands, we can start rolling up our sleeves and getting into the thick of it. The MachOView tool and 0xED are very handy for this purpose. Observant readers will note the presence of __PRELINK_TEXT, __PRELINK_STATE and __PRELINK_INFO segments in the file. Though undocumented in Apple’s reference library, their format can be discerned from the projects on Apple’s Open Source site. Of specific interest are IOKitUser (version 514.16.2 at time of writing), kext_tools (version 180.2) and xnu (version 1504.7.4), the kernel itself.

Though targetted at the OS X platform, information gleaned from these projects are relevant to iOS as well. Specifically, we can see that the __PRELINK_TEXT segment is simply a concatenation of all the kext Mach-O objects. There is no delimiter specified for these objects, but we can identify the start of a new object from the Mach-O object header “magic”.

The __PRELINK_INFO segment is host to a very informative property list in XML format. There are entries in this property list corresponding to each extension in the __PRELINK_TEXT segment. There is even enough information in this property list to reconstruct each kernel extension’s Info.plist and hence have “correct” kernel extensions, complete with the appropriate file hierarchy.

Unfortunately, the binary blobs in the __PRELINK_TEXT are not immediately usable. Specifically, they have been stripped of their symbol tables and can no longer be repacked into a new prelinked kernel. Despite this stumbling block, we can still proceed. There are numerous fields in the __PRELINK_INFO property list that do not appear in regular kext property lists. These prelink-specific properties are very useful to us: one such property contains the offset of the extension’s kmod_info_t data structure.

The kmod_info_t data structure contains pointers to the kext’s two entry points: one for loading the extension and another for unloading. Given that we know the kext’s location in memory (a benefit of prelinked kernels, much like prebound userland apps), we can determine their whereabouts in the executable for debugging purposes. However, to pack a specific kext into a kernel, the packing tool requires access to the kmod_info_t structure, so we must symbolicate it (i.e. give it an entry in the symbol table) .

With all this in mind, we can create an app to automate the process — and we did! Well, it’s a work in progress. If your particular religious affiliation allows you to associate with spaghetti code, feel free to download a copy and jump right in. The code appears to successfully unpack kernelcaches for all devices in the iPhone family, we have yet to test it with others. The repacking process can be accomplished using standard OS X tools, documented in the project’s wiki.

The code is in too rough a state to be used constructively just yet. Once all desired functionality has been completed, the code will be refactored considerably and have a proper release. We will also be porting it to Linux (and removing the dependency on OS X) to assist iDroid developers in their endeavour. In any case, we hope to soon start making use of this tool and begin development of some useful kernel extensions for iOS devices.

So what exactly does firmware flashing software do? Well, it enables you to upload any firmware, official or not, to your Galaxy S device. In the past those who have wanted to upgrade their device or flash custom firmware have been forced to do so via Samsung Kies or Odin. Both pieces of software only run on Windows and are closed source. Furthermore Samsung Kies is only meant for upgrading official firmware and Odin appears as though it is is leaked Samsung software not at all designed for consumer use.

The protocol used to flash firmware to the Galaxy S is provided by the secondary (second-stage) boot loader running on the device. The boot loaders on the Galaxy S appear to go by the collective name of Loke. Continuing on with the Norse mythology naming scheme (Odin and Loki) that Samsung appears to be using, I chose to name my software Heimdall. After the Norse god and guardian of the bi-frost bridge.

If you’re interested in the development then keep reading. If not, head over to the Heimdall product page, read the instructions and get to it!

First of all, before beginning the development of Heimdall I had to make sure what I was trying to achieve was actually plausible. As I had no information about the Windows drivers that Samsung provides the first step was to come up with a way of sending and receiving data over the USB port of my PC. After a bit of investigation I came across libusb-1.0, which just recently had Windows support integrated, perfect!

So I’d found how I was going to talk to my phone, but what exactly should I say? The protocol that Loke uses when flashing is not publicly documented. As such in writing this software I got to perform my first real piece of reverse engineering. In order to do so I had to monitor the data being sent and received through my PC’s USB port during the flashing process. After capturing the whole USB flashing process I was quite daunted, after all it’s just a bunch of ones and zeroes flying past. Numerous hours of starring blankly at the screen followed. However, after some time I slowly managed to piece together what some of these ones and zeros represented.

First of all, at the very beginning of the flashing process a handshake takes place. Luckily it wasn’t the type of secure handshake that was going to give me grief, just a simple “hello” really. So my first step was to write a simple piece of software using libusb-1.0, and just perform the initial handshake.

That went off without a hitch so the following few days included many hours of writing and rewriting sections of code as I began to understand the protocol. Data structures started to pop up and C++ class inheritance became the obvious way to cleanly express the protocol.

Eventually I got to the point where I could perform a simple flash without a problem. However, just as I was set to make the first alpha release on the XDA developers forums I experienced somewhat of a setback, in the form of bricking my phone! Now, as it turns out the Galaxy S is somewhat unreliable in the time it takes to respond to messages. The communication just happened to time-out just when the secondary boot loader was being written to my phone. Not having access to a device for over a month really slowed development but a few brave alpha testers gave me some invaluable information.

It didn’t take me long after getting my phone back from repair to get Heimdall into a reliable state with the timing-out more than accounted for. A bit of code cleaning and a few quick beta releases soon followed. Now that I’ve been able to verify that third-party Windows and Linux users are successfully able to flash firmware to various Galaxy S devices, I’m finally ready to make an official release. The release includes binaries for Windows, full source-code (MIT license) and GNU build files and can be downloaded here.

Have fun!