MD's Technical Sharing

Converting between NSString and C strings in an iOS project

2011-12-24T00:24:00.000-08:00

I recently had to use a C library in an iPhone project, which is mostly in Objective-C. Things were going on smoothly until I had some C functions that return C strings (wchar_t*, char*) and require conversions in order to work with Objective-C NSString* types.

There are 3 ways to declare a string in an iOS project in xCode:

NSString* str = @"hello world"; // declare an Objective-C string

wchar_t* str = L"hello world"; // declare a wide-character (Unicode) string

char* str = "hello world";     // declare an ANSI string

If you declare a Unicode string via the L"" syntax, the compiler defaults it to UTF32. The function wcslen() to get the length (e.g. number of characters) of a string may not work properly if the input string is not UTF8 encoded. For example, try the following code:

wchar_t* str1 = L"Giới thiệu về Google"; // "About Google" in Vietnamese

wchar_t* str2 = L"Gioi thieu ve Google"; // simplified with ANSI characters only

printf("str1 length: %d", wcslen(str1));

printf("str2 length: %d", wcslen(str2));  

The code will output wrong length for str1 and correct length for str2, even though they have the same number of characters. I think wcslen is confused by the UTF32 characters in str1 and counts some characters more than once. However, if I try the folowing code:

char* str3 = "Giới thiệu về Google";

setlocale(LC_ALL, "en_US.UTF-8"); 

int buflen = strlen(str3)+1;

wchar_t* buffer = malloc(buflen * sizeof(wchar_t));

mbstowcs(buffer, str3, buflen); 

printf("str3 length: %d", wcslen(str3));

free(buffer);

to declare an ANSI string and convert it to UTF8 wide string by using setlocale to ensure the correct Unicode encoding, wcslen will return the correct string length. Not knowing what the problem is, I have to make sure that all C strings in my project are UTF8 encoded.

Conversion from NSString* to an ANSI string (char*) is easy using the built in NSUTF8StringEncoding method. The returned value is valid as long as the original value is valid, so there is no need to release or free it. The following method (taken from my custom NSString category) shows how to achieve this:

- (const char*)getMultiByteString

{

return [self cStringUsingEncoding:NSUTF8StringEncoding];

}

It is a bit more complicated with C function mbstowcs to convert from NSString* to a wide string (wchar_t*):

- (wchar_t*)getWideString

{

const char* temp = [self cStringUsingEncoding:NSUTF8StringEncoding];

int buflen = strlen(temp)+1; //including NULL terminating char

wchar_t* buffer = malloc(buflen * sizeof(wchar_t));

mbstowcs(buffer, temp, buflen); 

return buffer;

}

It is the responsibility of the caller to free the returned buffer. To improve, one can free the return value in the dealloc() method of NSString. The return type should then be changed to const wchar_t* to indicate that the returned value is read-only.

Take note that wchar_t is 2 bytes on Windows but 4 bytes on Unix/Linux (including iOS) is 4 bytes. The above function uses sizeof to determine the size of wchar_t for the sake of generality.

Using stringWithUTF8String and wcstombs we can do the reverse and convert a C string into NSString:

+ (NSString*)stringWithWideString:(const wchar_t*)ws

{

    // Destination char array must allocate more than just wcslen(ws)

    // since unicode chars may consume more than 1 byte
// we do not yet know how many bytes the created array may consume, so assume the max.

int bufflen = 8*wcslen(ws)+1;

char* temp = malloc(bufflen);

wcstombs(temp, ws, bufflen);

NSString* retVal = [self stringWithUTF8String:temp];

free(temp);

return retVal;

}

I hope this will help other with similar problems.

HMC-35HD Media Player with NAS Server Review

2011-11-16T19:08:00.000-08:00

A few days ago I upgraded my home media player from the HMC-35HD made by Mele, a Chinese company, to the ASUS O!Play Air HDP-R3 to enjoy the many video formats supported by the HDP-R3. Before I resell the unit and earn back some cash, I am writing this short review which will hopefully help others who intend to purchase this player.

Overall the unit is very light, with a dimension of 220x196x73mm and weight of 1096g (without HDD) (see full specs here). It seemingly offers a wide range of features, including playing of music and videos (AVI, WMV, FLV, etc), accessing shared folder via Samba and uPnP using Ethernet or Wifi, Internet radio and an NAS server.

Storage devices can be connected via USB, or by inserting into the SD card slot. There are also 2 internal slots for 3.5" SATA or IDE hard disk. When connected, the device can be used as an NAS server. Judging based on the physical design, it seems that only one hard disk, either SATA or IDE, can be connected at any time.

This is the home screen of the player:

The five icons from the left to right are:

USB Devices: open media files on devices connected via USB and on storage card inserted into the card reader. Enabled only if external storage devices are detected.
Hard Disk: open media files from the internal HDD. Enabled only if a SATA/IDE HDD is connected.
Settings: configure system settings such as video output or wireless access point
Ethernet: access Samba/uPnP shared folder on LAN. Enabled only if the device is connected via Ethernet and assigned an IP address.
Wireless: access Samba/uPnP shared folder on wireless network. Enabled only if the device is connected to a wireless network and assigned an IP address.
Internet Radio. Enabled only if the device has an IP address on the network. I could never get this to work. Upon selection, it will attempt to load for a while and will just say "unable to connect".

The file explorer interface is fast and intuitive. However, if your storage device has a lot of files/folders, after selecting the icon, it may take up to a minute for the file list to show - I assume the player tries to index all files and folders on the drive as soon as the icon is selected. Sometimes if it takes too long, the operation will time out, resulting in an empty list and the device will erroneously report a huge drive:

The device supports many video output modes such as component, composite, VGA and HDMI as well as 16:9 and 4:3 aspect ratio on both PAL/NTSC systems. This can be configured in Settings:

Video Output: you can set to Composite, VGA or Component. HDMI output is automatic and will be enabled only when the unit detects an HDMI monitor connected. To change the video output without opening Settings, press the TV-OUT button on the player or on the remote control a few times.

Component: I could not get it to work with my TV. The TV will just flash the display from black to blue a few times and auto turn off. The setting is also grayed out unless a component cable is connected and video output is set to Component, making it difficult to change the settings to fit your TV if the TV does not like the default values.
Composite: this works without much hassle. Just set the correct aspect ratio for your TV and you're good to go. However, the maximum resolution is 480i, defeating the purposes of a high-definition media player.

TV Standard: support either PAL or NTSC. Affect only composite output. PAL has slighter higher resolution than NTSC. Modern TVs will handle both PAL/NTSC automatically so there is no need to change this setting.
TV Type: Support either 4:3 or 16:9 aspect ratio. However, the settings screen will be wrongly scaled while videos still playback with correct aspect ratio, leaving black gap on the left/right as in the above picture, if 16:9 aspect ratio is selected when it has been selected previously. 16:10 is not supported.
HDMI Mode: Set the HDMI resolution. Auto Detect is recommended. Notice that there may be HDMI handshaking failure, resulting in no output, if you're using a HDMI-to-DVI converter to connect to a monitor with DVI only. See also my previous post.
VGA Mode: Although VGA mode apparently only lists 640x480, 800x600, 1024x768 and 1280x1024, the resolution will be automatically adjusted to fit a 16:9 wide-screen monitor if the TV type is set to 16:9. VGA EDID is not used to identify the optimal resolution. Some TV may not show anything if the VGA resolution does not match its preferred resolution. To work around this, connect another LCD monitor and change the resolution.

At the back of the player, there are two identical rows of composite video and RCA audio output. Perhaps the designer wants you to be able to play on 2 different TVs at the same time, or perhaps for the sake of redundancy...

The player will always output audio via the RCA ports regardless of which video output mode is selected. This may be useful if you want to use a headphone. A RCA to 3.5mm cable is needed, obviously.

You can connect the player to your home wireless networks. WEP/WPA/WPA2 encryption modes are supported. However, it is time consuming to enter the wireless key using the remote control:

There is also a bug here. If the wireless network is using 26 hexadecimal characters as the WEP key, the last hex character must be a digit (0-9) and cannot be between A-F or you won't be able to enter it. I wish future firmwares will address this issue, but given how these Chinese companies work, I doubt there will ever be an update.

The device has basic support for subtitles (.SRT and .SUB files). Subtitle will be automatically displayed if it has the same file name and is in the same folder as the movie; otherwise it can be selected manually. While English subtitle displays well, Unicode is not supported and font size can't be changed. Word wrapping for subtitle is also not supported.

I also tried accessing Samba and uPnP shared folders, which works. Take note that your shared folder must not be protected with a password. In Windows 7, Simple File Sharing has to be turned on, otherwise Windows will require the system password to access the shared folder. Unfortunately, NAS did not work in all my attempts. After installing the hard drive and pressing the NAS button on the remote control, the media player appeared to be in NAS mode but the shared folder is not accessible. Until this day, I do not know what the problem was.

The Verdict

Despite some problems setting up and the fact that NAS does not work, I would still say that the HMC-35HD is worth considering, especially if you can get it second-hand. To be honest, I never expected a Chinese-made device to deliver all its promised features and therefore mainly use it as a media player. The only two things that irritated me were that most MKV files are not supported and that sometimes playback would stop half way and go back to the file list while showing "UNSUPPORT" on the screen. For this reason, I upgraded and the Asus HDP-R3 has served me well so far.

The user guide for the player (in Chinese only) can be downloaded here.

Dreambook W7: Cheap and apparently features-packed but unusable Chinese-made Android tablet

2011-10-14T07:33:00.000-07:00

Well, if you're intending to buy this tablet, the title says it all. This article serves as a reminder to watch out for things that are too good to be true. Save your money and buy a proper tablet!

It all looks promising at first - the Dreambook W7 is an Android 2.2 tablet (upgradeable to 2.3) running on a 1GHz Freescale IMX515 processor, 512MB RAM and 8GB internal storage. The device also has GPS, Bluetooth, SIM card slot (supports 3G, call and SMS), SD card slot supporting up to 16GB, mini-HDMI output and USB host capabilities, all for 350 SGD.

Upon first look, with the HOME button, the device looks like a giant iPhone with a lot of extra connection ports. It is also thicker and heavier than other tablets. No problem yet considering the price, I guess.

However, after some initial testing, problems soon emerged. The following list summarizes my findings:

The touchscreen is very sensitive, to the point that makes the device unusable. Scrolling through a list or a web page would just randomly clicks on items or links on the screen. I install this app, which reduces the sensitivity and seems to help a little, but it's still not good enough for web browsing.
Sometimes it is very hard to get passed the Android lock screen - the green button cannot move long enough to unlock the phone. I disable the lock by using the NoLock app.
In landscape mode, text box occupies entire screen area once the keypad is shown. This is a classic Android bug, reported here.
Even in landscape mode, the launcher home screen or program list still shows in portrait. But other screens appear correctly in landscape. Using a custom launcher such as ADW Launcher correctly changes the orientation, but the icon spacing is wrong.
No matter which firmware I tried, the FM radio doesn't work and just produces white noise, even when the headset is connected. (The headset is known to be used as an antenna in many phone radios)
GPS works intermittently, depending on which firmware I am using. On some firmwares, GPS seems to be working but reports fake location of 0N 0W. Where GPS is working, it reports 2X the speed so it can't be used when driving. Location determination using cell ID and wifi network doesn't work. Neither does the compass work.
Despite numerous sites saying that this device has USB host, I could not get it to work. Basically, it doesn't even supply power to the connected USB device such as a mouse or a thumb drive
Even with the SD card unmounted, hot-swapping the SD card may hang the device. Changing the SIM card while the device is on will definitely lock it. To reset the device, press and hold the reset button, which is also the power indicator light, on the bottom left side.
The device does not seem to support "reboot". The default "reboot" option in Android and all other tools will simply turn the device off.
Many firmwares will have problem receiving incoming SMS (the message is not shown and discarded) while sending outgoing messages is fine.
Finally, things got worse with the Android 2.3 Beta firmware. Wifi will not work until wifi sleep policy is turned off, and 3G works intermittently.

With the long list of problems, all with common use cases, I guess Synrgic, the manufacturer, does not have any kind of quality control over their products. There is also no good official tutorial to flash firmware for this device and all firmwares I could find are uploaded using various file sharing websites, not on the company official site like other devices.

Flashing the firmware

Flashing firmware is a pain on this device. First, you will need to find the correct ROM for your hardware version (either HD104 or HD105). Then, get the device into recovery mode: press and hold the HOME and BACK key and press the POWER key for a few seconds. Copying the bootimg.tgz and update.zip to /sdcard and use the recovery menu to start updating. Notice that on some firmwares, the internal memory is mounted as /sdcard and the SD card itself is mounted as /sdcard/extsd so you may need to copy the images to the device main memory.

Sometimes it just seems impossible to start in recovery mode despite pressing the correct key combination - the device will just perform a normal boot. You can get over this by using an app to reboot into recovery mode (the device needs to be rooted via z4root). The Dreambook will then turn off, and when started again, it will show the recovery loader.

Another way to flash is via MFGTool, a utility by Freescale, manufacturer of the device's processor. This should be used only at a last resort since incorrect usage may brick the device and turn it into a paperweight. The steps are as follows:

Used Windows XP. Windows 7 may require applications to have admin privilege and different drivers which will just interfere with the flashing process.
Download the MFG ROM package of the device. Some ROMs can be found here.
Start the device in recovery loader, then choose "Wipe for MFG". Connect it to the USB port of your computer, then press and hold the RESET button for a moment.
The PC will then find a new FreeScale BulkIO device which require driver. Install the device using the driver supplied in the MFG ROM package. If the device is identified as Mass Storage, disconnect everything, reboot the device and try again.
Run MFGTool.exe. Click “Options” menu at upper-right corner, and further click configuration item, you can find below pop-up window. Click “USB Ports” tab, you can view all the USB port. Choose the one to which your device is connected and click “OK” to close the window.

6. Click the green Start button and the flashing process should start. At the end of the process, the tool may say "Connect device" or report an "Updater Error". This is normal and the device should boot up to Android.

The latest firmware available is V25 (27 May 2011). Unfortunately it seems that Synrgic has stopped supporting the Dreambook W7 and decided to move on with their other products and no future firmwares will be available.

My conclusion is that the Dreambook W7, with a lot of out-of-the-box issues, can never be used as a productivity device, only perhaps as an expensive tool for advanced users to learn more about Android by trying different firmwares and applications in an attempt to solve these issues.

Rebuilding the Olevia LT19W 19" LCD TV Remote Control Unit

2011-10-09T02:04:00.000-07:00

I have here in my room an Olevia LT19W 19" LCD television that has no remote control unit. The buttons for this TV are located on the top panel so it would be a hassle to change volume or channel while watching. I decided to buy the RM-68 universal remote control from eBay hoping that I will be able to configure it to be used for my TV.

Unfortunately, I was wrong. None of the suggested codes I found online could make the remote control work with my TV. I also spent half a day trying most of the codes listed in the instruction manual, but none works. My last try is to use the auto-search mode, which automatically tries all supported codes to see which one works:

Finally after 20 minutes of pressing the VOLUME+ key, I got my TV not to show the volume control as expected, but to display "Normal" at the bottom of the screen. What should be the volume control key turned out to be the key to change the "Picture Mode" of the TV. The TV also recognized some other keys on the RM68 but the mapping was wrong. For example, pressing channel 2 on the RM68 would turn off the television! In other words, although the TV responds to some keys on the RM68, it can't be used.

I then had an idea. Since I had quite a few keys on the RM68 working, I may be able decode the remote control protocol for this TV. build a PIC-based remote control to simulate other keys and map them to a learning remote controller, which will be used in the long run.

Decoding the protocol

My initial research shows that most TV remote control use consumer IR which transmits using an infrared wavelength of 930 nm, 870 nm or 950 nm at carrier frequency between 33 to 40 kHz or 50 to 60 kHz, with 38kHz being the most common. To decode the protocol, I picked up a TFMS 5400 IR receiver from my junkbox, which is optimal for 950nm infrared @ 40kHz carrier frequency. Although the RM68 transmits at 38kHz, the "Relative Frequency" graph on page 3 of the datasheet shows that the TFMS 5400 would still detect 38kHz signal, just at 95% of the sensitivity. Following the datasheet, I built the receiver and fed the output to channel 1 of my Rigol DS1052E oscilloscope:

By pointing the remote controller at the receiver and pressing a few keys while observing the oscilloscope display, the protocol soon became clear. Each key will transmit a data stream, consisting of an ID byte identifying the TV and a data byte identifying the key, in the following format:

[header] [ID byte] [~ID byte] [data byte] [~data byte]
where ~ represents bit-wise NOT operation

The output of the receiver is active low and will remain high if there's no signal received. Each data stream starts with a header consisting of a low pulse for 9ms followed by high pulse for around 4.3ms. The ID byte (which is 8 for the Olevia TV) and its complement (e.g. bit-wise not) is sent following the header. After the ID byte is the byte for the key pressed, and also its complement. Binary 0 is represented by a pulse for 0.5ms and binary 1 by a pulse for 1.6ms. There is a delay of 0.6ms between every bit. Bytes are sent in little-endian format, with the least significant bit being sent first. Depending on the key pressed, the data stream may last anywhere between 50 and 90ms.

For example, the following data stream represents the POWER key, whose data byte is 0b00000010 (2 in decimal)

[9ms low] [4.3ms high] 00010000 11101111 01000000 10111111

The following screenshot from the Rigol oscilloscope shows the header and the ID byte:

Building the transmitter

To be able to simulate the rest of the keys, I used a PIC16F628 and a TSAL 7400 IR LED, with the information available here to build the transmitter circuit:

For the receiver to be able to pick up the transmitter signal, the transmitter needs to modulate the signal at around 38-40kHz. Since the receiver receives in active low, this means pulsing the output at 38-40kHz for around 0.5ms for a binary 0, and staying low for 1.6ms for binary 1.

But how do I actually pulse the output at the required frequency? Luckily the receiver is quite tolerant and any transmitted frequency within the the expected range will work. I used the assembly source code from here and modified it to suit my needs. NOP instructions are used to generate the required pulse frequency, as seen in the following code snippet:

IR_pulse
        MOVWF    count        ; Pulses the IR led at 38KHz
irloop        BSF    IR_PORT,    IR_Out
        NOP
        NOP
        NOP
        NOP
        NOP
        NOP
        NOP
        BCF    IR_PORT,    IR_Out
        NOP
        NOP
        NOP
        NOP
        NOP
        NOP
        NOP
        NOP
        NOP
        NOP
        NOP
        NOP
        NOP
        NOP
        DECFSZ    count,F
        GOTO     irloop
        RETLW    0

If you're not an assembly guru who knows exactly how many CPU cycles each instruction will take and is able to calculate the output frequency easily, you'll need an oscilloscope and lots of trial and error to get the desired output. This is when the Rigol DS1052E limited memory becomes a hassle. You will need to set the the TIME/DIV setting to a reasonable value that allows you to view the entire transmitted stream, but not at the expenses of losing too many sample points. Even at 1 mega samples of memory in long memory mode, signal aliasing may occur if the TIME/DIV is set so high that the oscilloscope can't capture enough samples. For example, the following puzzled me for a few minutes when I attempted to transmit 1 bit of data:

The code pulses the output at 38kHz, but from the oscilloscope screen, it seems much less. However there was nothing wrong with the code, all that was needed is to lower the TIME/DIV settings and capture the output again:

This is the transmitted signal:

Applying "invert" on the input channel on the Rigol oscilloscope, I got the transmitted signal to resemble the received signal:

The PIC-based transmitter now works properly and the TV responds to it as how it did for the universal remote controller.

Simulating other keys

By varying the data byte, I am able to simulate many other keys. The full list of keys and their data bytes can be found in the following table:

Key	Byte
1	10
2	9
3	8
4	18
5	17
6	16
7	14
8	13
9	12
0	22
Previous Channel	19
-/--	21
POWER	2
MUTE	4
MENU	3
Current Channel Display	24
Sound Mode	25
Picture Mode	26
Next Channel	29
Enter	28

With this, I am able to get my learning remote controller to learn the above keys and use it as the main remote controller for my TV.

The assembly source code can be downloaded here. It will repeatedly transmit the data stream for the "Channel 1" button. Unfortunately I could not find the data bytes for many other important keys, such as the volume control and TV/Video switch button. Perhaps these keys have data bytes outside the range I tried, or require a different protocol. I hope somebody with more time and expertise at PIC assembly programming can use my code and find out how to simulate these keys...

Reference:
1. Infrared TV Remote Decoder
2. PIC Tutorial Five - Infrared Communication

Compaq Contura 3/25C 80386 Laptop

2011-09-29T00:00:00.000-07:00

I recently picked up a free Compaq Contura 3/25c, a very old IBM-compatible laptop computer from the early 1990s, with a 25MHz 80386 processor, 4MB onboard memory and 3.5in 1.44MB floppy drive. The original 120MB hard drive has been removed by the previous owner. The laptop is also in very bad shape, with dust all over the place and the LCD panel removed:

The laptop must have looked like this when new with the 8.4inch black and white LCD:

I was trying to make it boot up by connecting an external monitor to the VGA port when I noticed that the VGA port is actually keyed via pin #9. No modern VGA cable would fit into this port until you break pin #9 on the cable or punch a hole on pin #9 of the port. Since there is actually no need to key a VGA DB9 connector because the shape of the port would prevent you from inserting it upside down anyway, later VGA ports used pin #9 as an optional +5V DC source.

The unit boots up with a "RTC Lost Power" error message (see this for a full list of Compaq POST error messages). This is quite common with old laptop when the CMOS battery finally dies after a few years. So I opened it up in order to replace the battery:

The laptop is using a CR2430 lithium coin battery placed in a socket to facilitate replacement. There is even a warning, in various languages, telling you about danger of explosion if the battery is incorrectly replaced. This surprised me, since modern laptops would probably just have the battery soldered onboard. Some may feature a motherboard socket for the battery connector, but the actual connections to the battery are permanently soldered, which means some wire-cutting and soldering needs to be done if the battery ever needs to be replaced. This is a CMOS battery from a newer laptop:

I quickly ordered a pack of 5 CR2430 cells for a dirt cheap price from eBay. Notice the funny English on the packaging. Whoever typed this probably had no ideas what it means:

With the CMOS battery replaced and a PS2 keyboard connected, I started the machine. Although many old laptops and PCs do not have an onboard BIOS and require the use of startup disks containing the BIOS utilities to change system settings, this machine has one, available in several languages, accessible via F10 at startup:

There are 2 startup disks for this laptop, formerly downloadable from Compaq website:

Disk 1: BIOS setup utility
Disk 2: Diagnostic utility

However, these downloads were removed when Compaq was merged with HP in 2003 and a lot of legacy support was gone. Although I eventually managed to find the disks with the help of archive.org, disk 1 comes with no COMMAND.COM and boots up to a DOS error "Bad or Missing Command Interpreter" error message. I fixed the error by putting MS-DOS 6.22 on it. The images for both disks as well as the original tool to create the disks (SP2054.exe) can be downloaded here. You will need WinImage to write the images to floppy disks.

Unlike the motherboard SETUP, the SETUP utility on the disks uses graphic mode and takes a long time to start up:

Since the original hard disk has been removed, my next task is to install a 2.5inch IDE hard drive before the notebook can be used for anything useful. Keeping in mind that laptops of this generation only use CHS addressing and does not support hard drive more than 512MB due to the 1,024 cylinders limitation, I have chosen a 270MB IBM DHAA-2270 hard drive:

Still, installing the hard drive turns out to be no easy tasks as the BIOS only supports a limited number of hard drives types (which does not include my drive). There is also no support for auto detection or user-defined types:

Interestingly, types 65 and 66 are empty, indicating that they are custom types and can be configured by either re-programming the BIOS as suggested in this forum discussion, or by writing a tool that modifies the BIOS hard disk type table stored in memory and hope that the BIOS will recognize the changes. Either way, it's a shoot in the dark as I could find no instructions on flashing this laptop's BIOS, and there is also no documented address where I can hope to find the type table.

However, there is an easier way by using a drive overlay such as ANYDRIVE or EZ-Drive. The trick is to specify a hard disk type in BIOS where the number of cylinders, heads and sectors is smaller than the actual value so that the BIOS will not report an error on POST and accept the hard disk. Once then, boot from a floppy disks with ANYDRIVE to set up the disk overlay, which will overwrite BIOS Int 13h (which is used by DOS to query hard drive info) and respond with the correct disk geometry values. Run FDISK to setup the hard disk and you will be able to use the full capacity.

This will work with DOS and Windows 3.1 or older which relies on Int 13h to access the hard disk. Operating systems such as Linux and some disk utilities may query the hard disk directly, resulting in possible data loss with the overlay installed. In Windows 3.1, 32-bit disk access must be turned off, otherwise Windows may also query the hard disk directly, resulting in similar problems.

With the overlay installed, all existing data on the disk is lost and some PCs may also fail to recognize the drive. Since I do not want to copy data and programs to the laptop using floppy disks, the only other way is to use a serial or parallel cable, with the help of Norton Commander 5.0 Link utility.

Note that you cannot simply use a male-to-male serial or parallel cable, but rather a null modem or a null printer (Laplink) cable respectively. Since these cables are extremely over priced, I decided to go for soldering a null modem cable myself using the connectors available in my junkbox, and the pinout from here. With my other PCs running Norton Commander (NC) from within Windows 98 acting as Master, I am able to copy the data to this laptop (acting as Slave). At a maximum theoretical speed of 115200bps, made slower by the noise-sensitive cable disrupting the copy process resulting in several misleading errors "There is not enough room to copy..." from NC, it took half a day to copy DOS programs, games and Windows 3.1 to the laptop.

However, before I could think of some useful purposes for this laptop, it dies and fails to POST with no beeps and no display output. On every boot, the system simply hung after attempting to seek the floppy drive. There were perhaps several symptoms that the laptop was dying, for example the fact that I needed to press the POWER button multiple times (although the button itself is fine, as checked by a multimeter) to turn the laptop on and the wrong memory count of 21885KBytes extended memory in BIOS:

Since all basic troubleshooting does not seem to help and there are no user replaceable parts inside the motherboard, I eventually removed the hard disk and ran FDISK /MBR to remove the overlay, and toss the machine. Everything has its time, I guess.

Reference:
Contura 3/25 Serie 2820A

Blogspot bug: Google+ Share button and Google Friend Connect widget

2011-09-19T01:03:00.000-07:00

Recently I observed that the Google Friend Connect widget that shows my site's followers on the left panel sometimes cannot be displayed. Refreshing the page did not help; clearing cookies and browser cache could only make it appear momentarily - the widget would disappear again eventually.

Firefox's error console (accessible via Tools>Error Console or Ctrl-Shift-J) and this post provide some hint:

The error messages are:

Error: window.googleapisv0 is undefined
Source File: https://apis.google.com/js/plusone.js
Line: 12

Error: google.friendconnect.container is undefined

The .js file used for Google "+1" button displayed at the end of every post is possibly conflicting with the js file for the Followers widget. As the both script files are obfuscated, it is not worthwhile for me to spend time investigating where the problem is. I ended up disabling the Share bar after every post by opening the Design tab of my blog settings, clicking Edit button at the bottom of the Blog Posts element, and unchecking Show Share Buttons

This would disable the entire set of share buttons even though I know I only need to disable the Google+ button. As there seems to be no way to show/hide each button individually, the only way to show only specific Share buttons (for example, share via Facebook) is to modify the page template and manually add in the code for the button.

Once you have manually modified the page template, always use the Edit HTML tab, and not the Page Elements tab to change the blog layout. Otherwise your changes may be lost and there may also be other unexpected results.

This is too much trouble, so the set of Share buttons at the end of every post in my blog will be disabled until this bug is fixed...

Xcode 4 bug: Crash when using Core Data Model from Xcode 3

2011-08-19T20:51:00.000-07:00

I recently upgraded my Xcode from version 3 to 4.0.2 (build 4A2002a) and notice that Xcode keeps on crashing whenever I right click on a core data model used in an iOS application:

The error message reads:

Xcode encountered an internal logic error. Choose "Continue" to continue running Xcode in an inconsistent state.  Choose "Crash" to halt Xcode and file a bug with Crash Reporter. Choosing "Crash" will result in the loss of all unsaved data.



ASSERTION FAILURE in /SourceCache/IDEKit/IDEKit-303/Framework/Classes/StructureEditingAdditions/IDEStructureEditing+Additions.m:587

Details:  Assertion failed: [indexes lastIndex] <= [_targetGroup.subitems count]

Object:   

Method:   -structureEditingCanGroupSubitemsAtIndexes:

Thread:   {name = (null), num = 1}

Hints:   None

Backtrace:

  0  0x0000000100949773 -[IDEAssertionHandler handleFailureInMethod:object:fileName:lineNumber:messageFormat:arguments:] (in IDEKit)

  1  0x000000010006d394 _DVTAssertionFailureHandler (in DVTFoundation)

  2  0x00000001009a683e -[IDEContainerItemStructureEditingTarget structureEditingCanGroupSubitemsAtIndexes:] (in IDEKit)

  3  0x000000011e9e5661 -[IDEStructureNavigator _testOrGroupSelected:useContextualMenuSelection:] (in IDEStructureNavigator)

  4  0x000000011e9e2617 -[IDEStructureNavigator validateUserInterfaceItem:] (in IDEStructureNavigator)

  5  0x00007fff834a538b -[NSMenu _enableItem:] (in AppKit)

  6  0x00007fff834a50e8 -[NSCarbonMenuImpl _carbonUpdateStatusEvent:handlerCallRef:] (in AppKit)

  7  0x00007fff8348899c NSSLMMenuEventHandler (in AppKit)

  8  0x00007fff884357f7 DispatchEventToHandlers(EventTargetRec*, OpaqueEventRef*, HandlerCallRec*) (in HIToolbox)

  9  0x00007fff88434d46 SendEventToEventTargetInternal(OpaqueEventRef*, OpaqueEventTargetRef*, HandlerCallRec*) (in HIToolbox)

 10  0x00007fff88452a81 SendEventToEventTarget (in HIToolbox)

 11  0x00007fff88481c35 SendHICommandEvent(unsigned int, HICommand const*, unsigned int, unsigned int, unsigned char, void const*, OpaqueEventTargetRef*, OpaqueEventTargetRef*, OpaqueEventRef**) (in HIToolbox)

 12  0x00007fff8849510f UpdateHICommandStatusWithCachedEvent (in HIToolbox)

 13  0x00007fff88431725 HIApplication::EventHandler(OpaqueEventHandlerCallRef*, OpaqueEventRef*, void*) (in HIToolbox)

 14  0x00007fff884357f7 DispatchEventToHandlers(EventTargetRec*, OpaqueEventRef*, HandlerCallRec*) (in HIToolbox)

 15  0x00007fff88434d46 SendEventToEventTargetInternal(OpaqueEventRef*, OpaqueEventTargetRef*, HandlerCallRec*) (in HIToolbox)

 16  0x00007fff88452a81 SendEventToEventTarget (in HIToolbox)

 17  0x00007fff8849476f SendMenuOpening(MenuSelectData*, MenuData*, double, unsigned int, __CFDictionary*, unsigned char, unsigned char*) (in HIToolbox)

 18  0x00007fff885b77ae PopUpMenuSelectCore(MenuData*, Point, double, Point, unsigned short, unsigned int, Rect const*, unsigned short, unsigned int, Rect const*, Rect const*, __CFString const*, OpaqueMenuRef**, unsigned short*) (in HIToolbox)

 19  0x00007fff885b7dc2 _HandlePopUpMenuSelection7 (in HIToolbox)

 20  0x00007fff835da99b _NSSLMPopUpCarbonMenu3 (in AppKit)

 21  0x00007fff835dad4b -[NSCarbonMenuImpl _popUpContextMenu:withEvent:forView:withFont:] (in AppKit)

 22  0x00007fff8372bb37 -[NSMenu _popUpContextMenu:withEvent:forView:withFont:] (in AppKit)

 23  0x00007fff836304c0 -[NSControl _rightMouseUpOrDown:] (in AppKit)

 24  0x00007fff834270c7 -[NSWindow sendEvent:] (in AppKit)

 25  0x00007fff8335bafa -[NSApplication sendEvent:] (in AppKit)

 26  0x000000010085b36e -[IDEApplication sendEvent:] (in IDEKit)

 27  0x00007fff832f26de -[NSApplication run] (in AppKit)

 28  0x00007fff832eb3b0 NSApplicationMain (in AppKit)

 29  0x0000000100000eec

I may click 'Continue' on the dialog box to ignore the error, but subsequently Xcode would just crash randomly and too unstable to be used. For example, clicking on the Help menu would crash Xcode, but not any other menu.

The root cause, after much investigation, was the hierarchy of the .h and .m files for the core data class objects. In my original project from Xcode 3, the .h and .m files stayed under the .xcdatamodeld file in the project tree:

This was the default behavior on Xcode 3. However, Xcode 4 does not allow you to place these files under the .xcdatamodeld file, which is perhaps why it crashed since the case was never handled. I removed all the .h and .m files, regenerated them and put them under the CoreData group, and Xcode stopped crashing (for now):

I thought that was it, but unfortunately the problem did not stop there. Xcode again crashed when I attempted to commit the project to SVN:

The reason for the crash was 'editorDocument should be an instance inheriting from IDE-QuickLookdocument, but...' Perhaps only Apple knows what this really means... Nothing really helped, including restarting Xcode, closing all open source code files or cleaning/rebuilding the project. In the end, I committed to SVN from command line, checked out a fresh copy of the project and Xcode no longer crashed when committing!

Finally, I am somewhat disappointed at Xcode 4.0.2. The UI changed significantly and some of the important behavior also changed, perhaps without any documentation at all! For example, for each core data relationship, the Edit>Create NSManagedObject subclass... creates the methods to add/delete objects to the relationship in the .m file, but forgets to to put them in the header file. This results in plenty of compilation warnings if your code is from Xcode 3 and the core data class files are regenerated using Xcode 4. I have to resort to manually modifying the header files to add the correct method signatures...

I hope these issues will soon be fixed in the next release of Xcode.

HTC HD2 Android: Working with ext2 images under Windows

2011-08-11T23:49:00.000-07:00

My HTC HD2 running on Android from SD card had been working well for some time until it could not boot to Android one day (booting to Windows Mobile was still fine). Obviously I could redownload the same Android build, but this would mean loss of text messages, applications, call history and any other data not backed up or synchronized with Google server. In an attempt to salvage these data, I looked at the Android folder on the SD card and notice the following files:

data.img: contains user data such as text messages, applications, etc.
system.ext2 and rootfs.img: Android system files

Although most guides suggest using a Linux live CD such as Ubuntu and use the mount command to mount them as drives, which actually works, I wanted to do the same in Windows. My first try is to use Daemon Tools. This obviously has no chance of success, as the tool is meant to mount CD/DVD images, and not disk images, not to mention the Ext2 filesystem used by Android. I then tried various Windows tools to open/mount Linux images, and summarized the results in the following table. Take note that the image may be easily damaged if mounted by a tool not supporting it, or if not unmounted properly. If you want to try this, remember to back up first!

	data.img	rootfs.img	system.ext2
Ext2Explore	R	R	R
Explore2fs	X	R	R
GizMo Tools	X	RW	RW
IMDisk Virtual Disk driver	X	RW	RW
DiskInternal Linux Reader	R	X	X
WinImage	X	R	R

Legend
X=failed
R=read-only access
RW=read/write access

You may need the Ext2 driver for Windows for some of the tools to work. Also read this tutorial on using IMDisk.

Looking at the table, Ext2Explore does the best by supporting all 3 images, albeit with read-only access. GizMo and ImDisk support read/write access for rootfs.img and system.ext2 (with IMDisk being more stable), but not for data.img, where most of the user data stays. DiskInternal Linux Reader supports data.img in read-only mode, but not the other two images.

So what is so special about data.img? Some websites say it is using YAFFS2 filesystem, which only applies for phones with Android on ROM. Since my Android is on SD card, I believe data.img still uses ext2, perhaps with some special format causing most Windows tools to fail mounting it.

Extending system.ext2

My next challenge is to use Windows to extend my system.ext2 as I want to add some fonts and background images and my system.ext2 does not have enough space. This time, despite finding no Windows tools that provide read/write access to data.img, I find a set of tools from xda-developers that does the job. The author compiled the source code of dd, dumpe2fs, e2fsck, mke2fs and resize2fs to run under Windows and built a user interface for it using Tclkit:

However, the TopoResize app is poorly written and crashes all the time. I resorted to resizing the image using the command line tools mentioned above:

dd if=/dev/zero bs=1M count=XXX >> system.ext2

XXX = Amount of space in MB to be added to the image

resize2fs -f system.ext2

Resize the filesystem in the image

e2fsck -f system.ext2 

Perform a check on the new filesystem. Without this, Android may hang while booting up.

The set of command line tools I used can be downloaded here. Next thing on my wishlist is to open data.img read/write on Windows; if anyone knows of a way, do leave a comment here. :)

Still alive after all these years: NEC D3835 45MB SCSI hard disk

2011-07-29T04:39:00.000-07:00

Recently I got around 20 old IDE/SCSI hard disks for free from a member of a Singapore technical forum and decided to try out all of them, one by one, with the help of an Adaptec AHA-2940UW, since my motherboard does not have SCSI ports. Utilizing the maximum available space in the chassis, I install 7 of them to my computer:

The 2 identical Seagate ST31276A drives are installed on the primary IDE channel while the IBM DARA-206000 6GB 2.5in drive is installed on secondary master with the help of a cheap 2.5" to 3.5" IDE converter purchased from eBay.

The 4 SCSI hard drives are detected separately by the SCSI BIOS. The last hard disk in the list, also the most notable, is the NEC D3835:

It's a 45MB 3500rpm SCSI hard disk drive, even older than the hard disk of my first computer (an old 80386 with a 106MB IDE hard disk). The rated speed is at 1.5MB/sec, way below the tens or hundreds megabytes per second which modern hard disks are capable of.

It is so old that HDTune refuses to test it:

The error message said "Read Error! Test Aborted" and popped up immediately after I clicked on the button to begin the test. But do not think that the hard disk is faulty - the error shows simply because the hard disk capacity is too small for HDTune to read/write data buffers onto it for testing. Perhaps the author did not expect such an old drive to be still alive today.

All the installed hard disks can be seen from Disk Management in Control Panel, with the NEC in the last position, certified as 'healthy' by Windows:

Unfortunately Windows is just plain wrong and the hard disk is actually on its last legs:

The label says the manufacture date is December 1990, which means the drive is getting 21 this year. That's equivalent to 100+ years for a human being. You will get nowhere near that for a modern hard drive, flash drive, or even solid state drive (SSD). Commercial pressure has forced manufacturers to use cheaper components, resulting in shorter life expectancy of their products, and you must be very lucky to see any modern storage device lasting more than the warranty period without issues.

The old new thing: mathematics of paper folding

2011-07-24T01:44:00.000-07:00

A few weeks ago I received the following quiz as one of the questions for an exam. The question is about paper folding, something that we all know from an early age, and seems simple but it seems that no-one could answer it. I presented it in this article with my proposed solution in an attempt to show how a frustrating mathematics problem could be set from something so simple in our everyday life.

The Problem

Fold a sheet of A0 paper (841mm x 1189mm) in such a way that the longer side (1189mm) is divided into half its length, you will get a sheet of A1 paper (594mm x 841mm). Do the same for the resulting piece of paper and you'll get a sheet of A2 paper (420mm x 594mm). Repeat the same twice and you will get the commonly used paper size, A4 (210mm x 297mm). This process could repeat over and over again to get a paper size of An after n times, as demonstrated in the following diagram:

Now let us define an "inner fold" as the line created on the original paper when you fold it into half, unfold it and look at it from the front:

On the contrary, if you look at the resulting paper from the back, you will see an "outer fold".

After n steps, unfold the original piece of paper and place it in front of you. Now calculate the number of "inner" and "outer" folds created on the paper (You position the paper and look at it the same way as how you did when you started folding it for the first time).

An example for up to n=3 is demonstrated in the following diagram. A normal dashed line indicates an inner fold while a bold dashed line indicates an outer fold.

The number of inner and outer folds for up to n=6 is shown in the following table. Notice that if a fold is intersected by another fold, it is counted as 2 folds.

n	Inner Folds	Outer Folds	Total
0	0	0	0
1	1	0	1
2	3	1	4
3	6	4	10
4	14	10	24
5	28	24	52
6	60	52	112

The first task is to find a general formula or algorithm to calculate the number of inner and outer folds after n times.

My Proposed Solution

As simple as it seems, there is no quick solution the problem. Since most people don't have an A0 sheet of paper at home to try to fold so many times, they can only attempt with the common A4 paper and get frustrated after a short while when the paper becomes too thick to fold or the number of folds too big to count.

I proposed a simple solution below. My research also shows that there are several other approaches which may yield seemingly different formulas. Do not read it until you have attempted the question.:)

First let us simplify the problem by calculating the total number of folds first. It is simply the total number of grid segments on the original piece of paper after n steps.

We first observe the total number of horizontal (from left edge to right edge) lines and vertical (from top edge to bottom edge) lines after every fold:

n	Horizontal	Vertical
1	1	0
2	1	1
3	3	1
4	3	3
5	7	3
6	7	7

This can be generalized into:

Now look at each horizontal and vertical line and calculate the number of segments on each line:

n	Segments /hori. line	Segments /vert. line
1	1	0
2	2	2
3	2	4
4	4	4
5	4	8
6	8	8

This can be generalized into (for n=2 and above)

The total number of folds after n steps is simply:

No. of horizontal lines x No. of segments on each + No. of vertical lines x No. of segments on each

With some efforts we are able to derive the formula for the total number of folds:

Now that we have found out the total number of folds, let's calculate the difference between the number of inner folds and outer folds after each step:

n	Inner Folds	Outer Folds	Diff.
1	1	0	1
2	3	1	2
3	6	4	2
4	14	10	4
5	28	24	4
6	64	56	8

Surprisingly enough, the difference is simply:

I leave this as an exercise for the reader to prove why. With the sum and the difference known, we can now calculate the number of inner and outer folds easily:

The first part of the problem is now solved. You can compare the results from our formulas with the example to make sure that the formulas are correct. Interestingly, the total number of folds after each step is equal to the number of outer folds in the next step.

Bonus Question

This is the next part of the problem:

"So far you have always been folding by the longer side of the resulting paper after every step (so that you could get an A1 paper from A0, A2 from A1, A3 from A2 and so on). Now you are told that this restriction is no longer necessary and you could fold the paper by either the shorter or the longer side. For example, fold an A0 paper (841mm x 1189mm) by the shorter side and you will get a piece of paper of size 420mm x 1189 mm.

How would the final formula for the number of inner and outer folds change? Derive a method to calculate the total number of inner and outer folds after a given set of fold steps, where one can fold either way in each step."

Although I managed to write a program to solve this, I believe there is no need to describe it here as most readers by now would have understood the concept and would know how to approach this extension of the original problem.

Afterthoughts

The concept which the problem is based on is very simple and does not require any fancy maths knowledge - anyone with perhaps a secondary school education can understand the problem. However, among many whom I have asked, almost all would immediately attempt the question but most would just become frustrated and give up without ever finding the solution, or even the correct approach. I believe it requires a high level of concentration and logical thinking as the number of folds grows exponentially. Setting this as an exam question where students work under stress and time constraints would therefore be unreasonable.

Samsung P2370 and ATI Moblity Graphics Card

2011-06-14T00:15:00.000-07:00

I recently bought a Samsung P2370 LCD monitor in order to watch 1080p HD movies on my HP G42 laptop, which is using an ATI Mobility Radeon HD5470 graphics card. Much to my surprise, this combination did not work at all and was a total waste of time. Here is why.

The monitor screen size is 23-inch, supporting full HD resolution at 1920x1080 and has only one DVI input. Interestingly, the on-screen display shows that the monitor searches for signal on both analog (likely referring to VGA) and digital (DVI) inputs. This is perhaps because the same firmware code base is shared with another similar model, the Samsung P2370HD that supports VGA input, among many other things including a TV tuner.

I first tried to connect the monitor to a DVD player with DVI output and everything worked well, so I proceeded to connect it to my HP G42 laptop HDMI port via a cheap DVI-to-HDMI converter purchased from eBay. Although the PC booted normally, with the monitor showing BIOS POST messages and Windows logo, the display is not detected once Windows finished booting. Screen Resolution settings in Windows 7 reported Another display not detected:

I tried to disconnect, reconnect, reboot Windows, reinstall driver to no avail. I also tried to install the Samsung driver for the LCD. The notebook graphics card has 3 outputs: LCD panel, VGA and HDMI. However, the monitor driver install utility only offers to install driver onto the LCD and VGA output, but not the HDMI output.

It then came to my attention that the P2370 was never detected - Windows played the "device connected" sound when I connected other monitors via HDMI, but not the P2370. The monitor worked well with other graphics cards, and the notebook graphics card HDMI output worked well with other monitors! So there must be something wrong with my special hardware combination. To my knowledge, unlike VGA where handshaking is optional and you can always force the graphics card to output to VGA port even when no monitor is connected, HDMI has some compulsory handshaking involved and the monitor has to be detected before the graphics card can output anything.

I then tried MonInfo and surprisingly the P2370 could be detected properly, with all the parameters correct. I tried to create a custom monitor driver to use with PowerStrip, which did not help either because the graphics card did not 'see' the monitor! Not giving up yet, I tried Windows XP which does not have automatic monitor detection. This time round, the Samsung driver installation utility could install the driver to the HDMI output, but the HDMI port was still not visible in Display Properties. I then tried to launch Windows in Safe Mode where only the VGA driver is loaded, and the P2370 always displayed a clone of the laptop LCD, much like what happened during boot time. The Fn key combination to switch between clone/extended desktop did not work.

All this observations suggested that the hardware was fine and the problem was with the ATI driver for Windows which could not see the P2370 via HDMI for some reasons. My final attempt was to try various Linux variants, hoping that some Linux drivers may fix the problems. However, the same problem persisted:

On Linux builds where the graphics card was detected properly, no output on HDMI.
On Linux builds with no driver for the graphics card, the P2370 displayed a clone of the laptop LCD

This is a dead end. In the end, I decided to sell the Samsung P2370 LCD and get a Dell S2409W instead, which has served me well so far. During the troubleshooting, my research revealed that this is a common problem (it's even in Google Autocomplete suggestion list, try to search for 'samsung p2370 ati problem') with no official answers from either Samsung or ATI, and no solutions available.

This problem may apply for other ATI card as well. The bottom line is not to buy Samsung P2370 and similar models if you're using an ATI graphics card!

Windows Phone 7 and Singapore IDD prefix 0XX

2011-05-07T20:54:00.000-07:00

When using my Windows Phone 7 today to make an international call using Singtel IDD 019, I realized that the call could not get through and the operator would always report "The number you have called is incorrect". Checking the call history showed that the prefix 019 has been replaced with "+9" by the Windows phone for unknown reasons.

After much googling, I found a way to disable this. This is part of a feature called International Assist by whoever designs the phone. To disable it, open Settings, click on Applications on the top bar, select Phone and turn off International Assist.

I am not sure which standard defines that "01" is the same as "+" as an IDD prefix or why this feature should be on by default. In any case, it's just another hassle when using Windows Phone 7!

See also:
25 Windows Phone 7 tips and tricks

Another customer service tale: "Your system connects me to the wrong number, please refund!"

2011-04-14T21:40:00.000-07:00

An early Monday morning at my company, which offers budget international calls via callback, I received a call from a user complaining he was connected to the wrong number and demanded a refund. Below is the conversation. The story is true with some details removed and/or altered to protect privacy.

....
User: Please help me, your system connects me to the wrong number.
Me: Can you describe what exactly happened?
User: I called your callback hotline at 6XXX XXXX, the system called me back, I answered the call and keyed in the number to call. The destination rang but the person picking up the call told me I had called the wrong number. Please refund the credit.
Me: Can I have your username and the number you wished to call?
User: My username is ..... and the number is +84XXXXX
Me: (after checking his account activity) According to our record, you have called +7484XXXXX, did you happen to key in the wrong number?
User: No, I entered it correctly.
Me: Can you tell me what you did exactly when the system prompted you to enter the destination number?
User: I entered +74 first, then realized it should have been +84, so I pressed Clear Display on my phone and re-entered the correct number.
Me: (almost started to laugh) I am sorry sir, the Clear Display only clears the display screen on you phone for your convenience and does not reset what you have entered into the system. Therefore you ended up calling the wrong number.
User: But I am the user and I don't care how your systems works. Please refund my credit!
...

This conversation should be normal and acceptable for a first time callback service user. However, I later found it to be disturbing once I knew that he is a frequent callback user with a PhD degree in computing. All Interactive Voice Response (IVR) system works based on DTMF which does not support a DELETE key or similar. All digits entered locally will be registered with the server, which is why sometimes you are asked to confirm sensitive information (via 'Press 1 to confirm or Press 2 to change') before any action is made. In any case, I believe it doesn't take a genius to know what he did (pressing the 'Clear Display' button) would not have worked. I have always thought that PhDs are supposed to be critical thinkers with good reasoning and whatnot, together with their professional skills. Oh well ...

See also:

Interfacing Nokia 3510i and 5110 LCD with PIC Microcontroller

2011-03-23T05:34:00.000-07:00

Recently I started to regain some interest in embedded systems, and start to experiment with PIC micro-controllers. After some successful attempts with standard character LCDs using using the HD44780 controllers, I decided to get some Nokia LCD modules from eBay to explore.

The 2 LCD modules I purchased are for the 3510i and 5110 models. Both have built-in controllers which use Serial Peripheral Interface (SPI). The following are the pinout for my modules, notice that pin assignments may vary slightly.

LCD pinout

Nokia 5110:

Nokia 3510i:

The only different here is pin #5 which is used as data/command selection for the 5110, and unused for the 3510i.

Voltage difference: 5.5v vs 3V

Both LCDs are designed to work with 3.3V, but due to an internal voltage clamp 5V can be used for SCLK, SDATA, REST, D/C and CS as long as a current limiting resistor (around 10k) is connected in series for each line. 3.3V should still be applied to VCC and the LED supply. I have tried using voltage dividers, which did not work, perhaps due to the LCD varying internal resistance and current consumption.

With the above connections we can only write to the LCD but can't read back the LCD response because 3.3v is not high enough to register as logic '1' in the PIC. Luckily reading from the LCD is not required for basic operations; all that is needed is sufficient delay after each operation to make sure the LCD is ready for next command.

I have chosen the PIC16f88 simply because it's available in my junk box. However, as the 16f88 does not have dedicated hardware SPI required by the LCD modules, bit-banging has to be used to simulate SPI. Although this usually means complicated code and lower throughput, it does not matter as all I wanted is to get the LCD to display something useful ;)

LCD Memory Map

The 5110 LCD is monochrome, uses the PCD8544 controller and has a resolution of 48 rows × 84 columns. Each 8 pixels on a single column consumes a single byte on the LCD memory map. It takes 504 bytes to fill the entire LCD.

The 3510i LCD has 97x66 resolution and can operate in either 256 or 4096 colors. Since there seems to be little difference between 256 and 4096 colors due to the small resolution, I have chosen 256 colors for simplicity. Each pixel on the LCD is represented by a single byte and filling the entire LCD takes 6402 bytes in 256-color (8-bit) mode.

Sample code: displaying test patterns

The following code shows how to display all black pixels on the Nokia 5110 LCD. Notice that LCD initialization code is not shown.

void lcd_5110_clear()
{
    for (int i=0; i<84;i++)
    {
        unsigned char row;
        for (row=0;row<6;row++)
        {
            //all black pixels
            char data = 0xFF;
            
            lcd_5110_send(0x40 + row,0); //Y address
            lcd_5110_send(0x80 + i,0);   //X address
            
            //write to display memory
            lcd_5110_send(data,1);
        }
    }
}

The following code shows how to display a selected color on the 3510i LCD:

void addset(unsigned char x1,unsigned char y1,unsigned char x2,unsigned char y2)
{
    send(0x2a,0);//column address set
    send(x1, 1);
    send(x2, 1);
    send(0x2B, 0);//page address set
    send(y1, 1);
    send(y2, 1);
    send(0x2C,0 );//memory write
}
void LCD_Clear(unsigned int value,unsigned char Color)
{
    unsigned char x, y;
    addset(0,0,97,66);
    for(y = 0; y < 67; y ++)
    {
        for(x = 0; x < 98; x ++)
        {
            send(Color, 1);
        }
    }
}

Displaying text and graphics

Up until now you can only display test patterns on the LCDs. The use of a bitmap font (and extra code) is required if you want to display any useful text. I have chosen a 8x12 font for the 3510i LCD, and a 5x8 font for the 5110 LCD. The font, together with any graphics to be displayed, will be stored in a 24C64 (8Kbytes) I2C EEPROM. To program the EEPROM, I use the I2C version of the PonnyProg programmer. Notice that this may not work on newer PCs where the available current from the serial port is limited and will never work with a USB-to-serial converter. In my experiment, I made a stupid mistake of adding a LED via a 470 ohm resistor to show activity during programming. This result in data corruption and verification errors after programming due to excessive current consumption. Changing the resistor to 2k worked fine, although the LED is much dimmer.

With the EEPROM to store font and graphics, the 5110 LCD could now display text and some monochrome bitmap:

The 3510i LCD could do a much better job ;)

Notice that the serial port connector is for debugging purposes only.

The entire source code is attached here. The contents of the EEPROM is included with the source code and named eeprom.bin.

VGA output with PIC Microcontroller

2011-02-15T19:55:00.000-08:00

I tried to use the instructions from here to make a PIC16F84A display some test patterns on my monitor. The circuit diagram is simple:

You should try with an old CRT monitor first, which is less sensitive to noises and will not be expensive to replace should you damage it due to connection errors. I tried to build this circuit on a breadboard and found out that it works every time with a CRT monitor but randomly will not work with my LCD monitor - the LCD simply stays in standby mode. Another important thing not mentioned in the article is that the monitor ID (pin #11) has to be grounded to indicate a low-resolution output (VGA 640X480) and the digital ground pin (pin #5) also needs to be grounded. The VGA pinout is detailed below for your reference:

I build the circuit on a veroboard and a breadboard:

Here is the result:

You can display different colors by connecting pin 1 (Red), pin 2 (Green), pin 3 (Blue) or any combinations of these to the 10k resistor.

Here is the horizontal sync and vertical sync signals on my oscilloscope:

Pin 7 output signal vs. horizontal sync:

Pin 7 output after the 10k resistor. The resistor is for brightness adjustment and can be omitted:

Experimenting with LCD modules and PIC microcontrollers

2011-02-08T20:13:00.000-08:00

I recently got a PIC16LF84A (free sample from Microchip) and a few LCD modules at a cheap price from eBay. With some interests and time at hand, I have successfully made these LCD modules work and display some test patterns, just for fun :)

16x2 Character LCD

This LCD is using the standard HD44780 controller so sample source code can be found everywhere on the Internet. I got it to work within 15 minutes:

20x4 Character LCD

This LCD is also using the standard HD44780 controller, but there is more display area, 20 columns and 4 lines. I also got it to work:

Download the source code to test the 16x2 and 20x4 character LCD here.

122x32 Monochrome Graphics LCD

This LCD is using the JHD12232D controller. I purchased it from sure-electronics, an eBay reseller:

Despite being a graphical LCD, its uses are pretty limited due to the small vertical resolution. The 32-pixel vertical resolution is divided into 2 blocks of 16 pixels each, having a space in between. This make it almost impractical to display any bitmaps or graphics on this LCD. The only graphical use for this LCD would be to display Chinese text, as demonstrated in the above picture.

Using the datasheet and sample source code available from the seller website, I am able to make this LCD display some test patterns:

Download the datasheet and source code here

Programming Nostalgia: revisiting Mike Wiering's Mario game written in Pascal

2011-01-18T15:45:00.000-08:00

I have always been a fan of Super Mario game (and its variants) ever since the first time I touched the computer keyboard. I remember the first time playing it on my old 80386 computer and could not get passed the big two fish in the middle of level 1:

After I managed to jump through the lava and proceed to higher levels, it seemed that I could not get through level 4:

I decided to give up and did not attempt the game until years later when I had an Internet connection at home and soon figured out that I was playing on an uncompleted version of the game. By then (around the year 2000), Mike Wiering, the original author of the Mario game for MS-DOS, has released the source code on his website. Unlike my version, which proceeds directly to level 1 upon startup, the full version supports 2 players (MARIO and LUIGI) and has a menu with some other options:

Compiling the source

After a few failed attempts due to a bug with Turbo Pascal CRT Unit that causes Runtime Error 200 when running on fast computer, I managed to patch the CRT unit, recompiled the game, ran it properly and finished all 6 levels! However, when I migrated to a faster laptop (1Ghz) around 2004, the game again could not run - it stopped at a black screen upon startup, perhaps due to some illegal VGA function calls. It also cannot run on modern Windows such as Windows Vista and above, or 64-bit version of Windows, due to a lack of 16-bit compatibility as well as full-screen support. On these computers, and my current computer in particular, DosBox is the only option if I want to play the game. Interestingly, this MARIO game, and similar games by Wiering Software such as Charlie II, Charlie the Duck or Super Angelo play fine on DosBox but seem to have timing issue (the speed is very fast) when run from inside a virtual machine such as Microsoft Virtual PC, VmWare or Sun VirtualBox.

With some Pascal programming knowledge and time at hand, I decided to have a closer look at the source code, to figure out how Pascal is used in game programming, and this article will discuss some interesting facts that I have found.

The first thing I learned was that the released executable was packed (as described in the README.TXT provided with the source code) to reduce file size to 57KB, perhaps with some MS-DOS packing utility. The compiled executable can be as big as several hundreds KB. In those days with 360KB floppy disk, this was probably a huge concern.

Source code organization

The source code is quite well organized into several Pascal unit files (*.PAS) and sprite include files (*.00?). Variables are well named and procedures are well structured. Although there are few inline comments provided since the code was never meant to release to public, the code can be understood and modified by anyone willing to do so.

The description of the main source code files can be found below:

MARIO.PAS: The main application

WORLDS.PAS: All level data are hard-coded here

BACKGR.PAS: Unit to support drawing the game background such as skylines.

BLOCKS.PAS: Assists in the drawing of animation

BUFFERS.PAS: Support reading of level and sprite data into buffers

CPU286.PAS: Halt the program if a CPU older than 286 is detected

ENEMIES.PAS: Define Mario's enemies, such as turtle, fish or moving objects

FIGURES.PAS: Define behavior of objects along MARIO's way, except for enemies

GLITTER.PAS and TMPOBJ.PAS: Display glitters such as stars that show when Mario hits coins or an object.

JOYSTICK.PAS: Support the use of a joystick

KEYBOARD.PAS: Process keyboard input

MUSIC.PAS: Play sound using PC speaker

PALETTES.PAS: The color palette used to draw the game

PLAY.PAS: Main game logic, e.g. how MARIO interacts with enemies, objects, earn coins, etc.

PLAYERS.PAS: Define the behavior of MARIO and LUJI.

STARS.PAS: Draw the stars on the sky

STATUS.PAS: The game status line

TXT.PAS: Text processing unit

VGA256.PAS: Custom Turbo Pascal VGA unit (Mode 13h, 320x200 256 colors)

Creating sprites

Sprites are first created using GRED.EXE (see GRED.TXT included in the source code):

It will be saved as a binary file (*.000, e.g: TREE.000), and then exported to a Pascal file that looks like the following:

The Pascal file is named TREE.$00. If a sprite has multiple states, as is the case for animated object, the extension is incremented, e.g. TREE.001 and TREE.$01. Sprites will be included as an include file in FIGURES.PAS:

{$I Tree.$00} {$I Tree.$01} {$I Tree.$02} {$I Tree.$03}

The point here is to store all sprites and level information into the code section, not the data section, of the program. The data segment in Pascal program can only contain up to 64K of data, and the game may grow beyond that. If slow read speed (earlier games ran on floppy disk) and having the game in multiple files was not an issue, an alternative would have been to store the data as external file.

Code would then be written to access the disguised data stored in the code segment by means of procedures consisting entirely of DB directives. The following will draw TREE000 at the specified location:

PutImage (XPos, YPos, W, H, TREE000^);

PutImage is defined in VGA256.PAS:

procedure PutImage (XPos, YPos, Width, Height: Integer; var BitMap);

Level data

As mentioned in README.TXT, there is no level editor for this game. All levels are coded in WORLDS.PAS:

A typical level consists of 2 procedures, a level data file (Level_1a) and an option files (Options_1a). Similar to sprites, they are just assembler procedures having only DB directives to store data. The option file will define how the level data will be interpreted. Take a look at Intro_0 and Options_0, for the 'intro' level, which is the background shown behind the selection menu:

Each assembler directive defines each vertical portion of the screen. One DB is a string of 13 characters. Each character defines an object on the screen, from bottom to top. The character 0 marks the end of a level (e.g. DB 0). The same character may be interpreted differently in different levels if the level options are different - see function ReDraw() in FIGURES.PAS. All level data will be loaded into variable WorldMap (found in BUFFERS.PAS) using ReadWorld(). Some levels may have certain pipes where Mario can dive in to enter a different area - these are defined as sub levels, for example, see Level_1b.

A modern approach

With some free time at hand, I decided to try out and see how the level data can be re-used to display an overview of each level, without re-writing everything from scratch. My first task is to save the sprites as an image file, which was easy since the GRED file format is documented. It wasn't long before I managed to write a tool in VB.NET that loads a sprite binary file and display in a PictureBox:

All sprites will then be converted to VB.NET resources. The next challenge would be to export the level data. Based on function PlayWorld in WORLDS.PAS, I wrote my LVL2BIN.PAS which exports all level data to a binary file (*.LVL):

It is used as follows:

WriteLevelToBin(@Level_1a^, 'Level1a.LVL');

For the level options, to facilitate modifications, I did not export it to binary file, but instead convert to an XML file:

Most of the code is available from function ReadWorld() in BUFFERS.PAS. I adapted them to VB.NET with some minor modifications to cater for zero-based array index in VB.NET (Pascal supports non-zero-based array) and the lack of set data types in VB.NET. For example, the following simple Pascal code:

var Ch: Set of Char;

.... 

Ch = [C] + [#1 .. #13];

turns complicated and probably more expensive in VB.NET - a List has to be used to emulate a set:

Dim Ch As New List(Of Char)

Ch.Add(C)

For i As Integer = 1 To 13  

If Not Ch.Contains(Chr(i)) Then Ch.Add(Chr(i))

The following shows the level viewer in actions. it reads level data (.LVL) and options (.XML) and display it on screen:

(For simplicity, enemies and background are not yet drawn)

When I was writing the code, there was something which surprised me. Despite the different look of the bricks between intro level (level 0) and level 2 (see image below), they actually come from the same sprite (BRICK0.000).

The tricks are in the following 2 functions in FIGURES.PAS:

procedure ReColor (P1, P2: Pointer; C: Byte);

procedure ReColor2 (P1, P2: Pointer; C1, C2: Byte);

Both were written in assembly:

All that the seeming complicated assembler code above will do is to loop through every pixel in the image and modify its color by 1 (for ReColor) or 2 (for ReColor2) constants to make the new image look different. This allows the same image to be used for 2 different levels yet still look different. I converted both of them to VB.NET:

Private Function ReColor(ByVal fig As Bitmap, ByVal factor As Integer) As Bitmap

Private Function ReColor2(ByVal fig As Bitmap, ByVal factor1 As Integer, ByVal factor2 As Integer) As Bitmap

However, despite using the exact same constant and same color palette, my resulting display of level 2 does not look exactly the same:

I am unable to locate the exact problem and can only assume it's due to something I might have overlooked.

At this point I can proceed to draw the background, animated sprites (turtles, fish, ...) and implement the proper game if I want.

Easter eggs

The game has some Easter eggs which can be found in PLAY.PAS. To activate cheat mode, press P to pause the game, then press TAB. Pressing 2305 while in cheat mode will get you through the next level. Also if you prefer to play in grayscale, press MONO:

Similar games: Charlie the Duck, Charlie II and Super Angelo

According to Wiering Software, these 3 games are developed based on the original Mario source code; however, their source code was never released. Charlie II (my favorite game) also has a Windows version which was perhaps written in C++. As of 2011, it's pretty clear that no future DOS versions of these games will ever be created, and any future version will perhaps only be in Flash. Click here and here if you want to try out. Nevertheless, I hope this article will be useful for those who want to port the game to other platforms, or to learn something about game development in Pascal.

By the way, if you're playing Charlie II and receive an error "Unexpected error no XXXX, please contact Wiering Software", don't think it's a bug with the game. Most likely you're using an activation key generated by a crack tool which does not satisfy all the requirements. In this discussion, Mike Wiering said that several checkings of the activation code are done at various places using various different algorithms in the game and a cracker may not have located all those places. So if you got the error, simple try a different key.

The full .NET source code to convert sprites and view the levels can be downloaded here.

References:

1. Open Game Source: Mario Clone
2. Wiering Software - Creating Games
3. Using haXe for Platform Games

Relisys RWT200EM (Exilis TR333) Thin Client

2010-12-20T20:49:00.000-08:00

A few months ago I was given a Relisys RWT200EM (also known as Exilis TR333) thin client (see detailed specs and review here).

I was expecting to "jailbreak" the thin client and get to install some sort of OS (DOS, Linux, early version of Windows on it). However, upon further testing, my disappointment soon came.

The thin client was manufactured by Exilis (also known as Relisys RWT200EM, see Relisys Thin Client) as shown in the splash screen:

It was re-branded by TOA E&I, a Singapore company, and manufactured in 2000 as shown in the about page:

All it will do is to boot up to a Windows CE Remote Desktop client:

Upon pressing F2 to enter terminal settings, it prompts me for a password. I tried all default passwords and common passwords, only to be replied with an error message. I guess whoever last used the client has set an unknown password.

Without the password, the connection settings cannot be changed. And the default connection settings is so odd that the client won't connect via a normal network using DHCP:

About the only way to get this unit to do something is use a router and setup a network with the expected parameters, then create a remote desktop session from the client to other computers in the network, which seems pretty useless.

The keyboard settings follow Windows CE standard:

It supports up to 1280x1024 @ 60Hz display resolution. However, changing the resolution again asks me for the password,which I do not know:

Apparently it supports remote firmware update if the correct password is provided. Where to find firmware, I don't know:

Again, the password cannot be changed if the original password is not provided:

All this seems to be a dead end. I tried to enter the BIOS to see if there's an option to reset the password there. The website provided at the beginning of this article suggests pressing F1 to enter BIOS,which doesn't work. What works is to connect a PS2 keyboard (a USB keyboard is only detected by Windows CE and not by BIOS) and press DEL repeatedly AFTER the power button is pressed but BEFORE the text "Starting System... please wait" appears. Pressing F1 during the "Starting system" phase will hang the boot process, with the keyboard still responsive - pressing Ctrl-Alt-DEL again will reboot the set. Probably reserved for some recovery ROM, etc. which was never implemented.

The unit is using the standard PC award BIOS:

Much like a PC BIOS, this bios has options to detect hard disks and set up floppy disk drives:

The motherboard also has empty connectors to solder an IDE connector (and also floppy connector, not shown)

I guess luck was not on my side. Unlike the website author's unit which has empty connector holes, my holes are pre-filled with existing solder. I tried with a 60W soldering iron to clear the solder from the holes but the temperature was not hot enough to melt the solder. The same thing applies for the floppy connector. So installing a hard disk or a floppy disk is out of the question.

I also tried booting from USB or LAN without success. The only option now would have been to upgrade the firmware on board. The Windows CE firmware is stored on a 16MB M-System DiskOnChip 2000. Up to 64MB DOC is available on eBay. However, no programmer/reader for these DOC seems publicly available. My research reveals the following possible methods to read/write the DOC, none of which I can do myself:

1. You can use an old ISA network card, enable Boot ROM, insert the DOC to the socket on the network card. The system can boot from the DOC and see it as a disk drive.

My system does not have an ISA slot and I do not have any ISA network card.

2. Build your own ISA card to access the DOC. I found the PCB here but however no schematics or software is provided.

3. Use the DOC evaluation board (available for ISA and PCI), see this and this. However, it seems that the evaluation board is no longer in production (M-Systems was eventually taken over by Sandisk)

4. Use a programmer. A user guide is available here but it seems the programmer is rather expensive. It is also unknown to me how to create the image to be programmed.

5. There seems to be an SDK of some sort for the M-Systems DiskOnChip 2000 such as this but there seems to be no good tutorial.

With all possible methods leading to a dead end, I decided to give up on the unit, package it and find someone who is willing to buy it. The board also does not have a lot of components useful for hobby projects. Among these few are a speaker and some audio/LAN connectors but the time taken to desolder these components would definitely outweigh any savings they may offer.

Casio PB-700 and FA-11 Printer & Tape Recorder

2010-12-02T21:02:00.000-08:00

I recently acquired an New Old Stock (NOS) Casio PB-700 with the FA-11 printer & tape recorder and an OR-4 memory module (4KB) from an eBay seller for a very cheap price. I know sometimes it's fun to play with old technology, and this article will share some of my findings.

The PB-700 packaging is still intact ever since 1984:

And so is the printer box. The left one is the paper cover, the right one is the plastic container for the printer and accessories:

The user manuals for the two units:

The scanned PDF copy can be downloaded from here (for PB-700) and here (for FA-11). More user manuals of similiar early computers can be downloaded here

However, when I first powered up the set, it did not work properly. The PB-700 LCD was not displaying properly (perhaps due to aging). The computer still seemed functional and responded to basic command such as BEEP.

The FA-11 did not seem to power up at all. Perhaps the rechargable battery inside the unit had died due to aging. I tried to leave it charging for 2 days and it still could not turn on. The AC adapter provided 8.5V, way beyond its specifications of 6V DC output. Some research suggests that the AC adapter is unregulated, so it may produce higher voltage when no load is applied. Whatever it was, the seller was nice enough to exchange a working set for me.

This time everything is working - the LCD is fine and the printer is able to print, although with difficulty (more on that later). The following photo shows the complete setup:

My first "Hello World" program on this device works fine, so I attempt to write a program to plot the graph of a sine wave. Luckily a sample is given in the user manual:

Take note that the line numbers must be keyed into the actual program. They are not there just for demonstration. If you don't key in the line number, many control commands (FOR, WHILE, IF) will not work.

It took me 10 minutes to key in this program and modify it slightly to display the graph title. Here is the execution result:

If you have never worked with early computers before, you''ll be surprised at the execution speed. The following video shows you why:

It takes 20 seconds to plot the graph - you can literally see every pixels being drawn. Nowadays, modern computer can plot this graph in less than a second.

My next try is to use a FOR loop with LPRINT to repeatedly print text to test the printer. The following video shows the printer in action:

It takes around 3 seconds to finish printing a single line of text, a slow speed thanks to the pen assembly.

There are a total of 4 pens, supporting 4 different colors. The horizontal stepper motor moves the pen assembly from left to right, a rotor will rotate the assembly to select the correct color, and a pin will push the required pen, touching the paper thus "plotting" the required text onto the screen. A very sophisticated piece of engineering! Unfortunately, this is an open-loop system - the printer does not have any ideas whether the pen assembly is correctly aligned. After a while, the pen assembly will no longer be calibrated causing the pen not to touch the paper and nothing will be printed. When this happens, I will have to stop the printing process and re-align. With care, the re-alignment can be done during printing when the pen assembly moves closer to the right.

Here is the printer output:

Because the pen slightly touches the paper when it moves from right to left during a carriage return and line feed, every line is underlined! The spacing between the lines is also not exactly equal thanks to the paper roller which doesn't care what happens to the paper during rolling. I am not sure if this is how the quality is supposed to be. Or perhaps I have an old unit...

My next try is to use the tape recorder to read and write programs:

The tape is just the normal cassette tape, used in the days of the Walkman. Program are stored as audio tones, with different tones for binary 0 and binary 1. Some more information is available here.

This is a closer look at the cassette tape. Interestingly, the tape comes with the erase prevention tab removed (thus the REC button can't be pressed). I have to put some paper in the holes to allow recording.

Using the SAVE command will store the current program as tones on the tape, to be retrieved later. The recorder plays the tones to be recorded during the process, making a sound similar to a dial-up modem connecting. Download the audio here.

Unfortunately, I am unable to read back the saved program using the LOAD command. The command simply hangs and I have to stop it. The recorder plays the tones of the saved program, which sound significantly different from what was recorded. Perhaps the audio head has malfunctioned due to aging, or the magnetic tape has degraded over the years and no longer suitable for recording.

Hope you will learn something new about old computer technology after reading this article. Except for having fun while testing it, the unit is not very useful to me. I have not even figured out how to insert a new line when editing a program (it's not mentioned in the manual). Therefore, after writing this article, the unit will probably soon be put in my closet.

More technical information about the Casio PB-700 computer series and accessories is available here

iPhone SDK on Mac OS X 10.6.4 under VmWare - Tips and Tricks

2010-11-30T23:29:00.000-08:00

This articles describes some common problems when using running Mac OS X 10.6.4 under VmWare using the instructions from my previous post and provides possible solutions/workarounds.

1. If you get an "rtclock_init panic" (shown below) and the system hangs while booting up:

press TAB at the Chameleon bootloader, select the Mac partition and enter -v busratio=18. OSX should then be able to boot. To make the busratio=18 permanent, edit /Library/Preferences/SystemConfiguration/com.apple.Boot.plist using nano/pico from Terminal (remember to do su/sudo to get admin right first), then look for the Kernel Flags section (add it in if it's not there). The final plist should look like the following:

 <key>Kernel Flags</key>

  

 <string>-v busratio=18</string>

Other useful bootloader parameters:
-v: verbose mode, which will show all boot messages on the screen. Without this you will only see the loading logo while booting up.
-x: boot in safe mode. Only kext critical for the system will be loaded.
-s: boot into single user mode, e.g. command line only.
acpi=off: turn off ACPI support.
cpus=1: tell the virtual machine to boot with only 1 CPU.
-x32 or arch=i386: boot in 32-bit mode

2. You may notice that some graphics applications won't be able to display graphics properly. In particular, the Preview application won't be able to display images, and pressing Apple-Shift-3 or Apple-Shift-4 (Apple key is mapped to Windows key) to capture screenshot will just return an empty black image. This is due to the lack of QE/CI support on the graphics card emulated by VmWare. Unfortunately, I found no workaround for it.

3. By default, the Apple key will map to the Windows key. If you want to map it something else (for example, the Alt key), follow the following steps:

* Go to System Preferences > Keyboard > Modifier keys

* Switch option key to command key (dropdown) and command key to option key (dropdown) - optional

4. For some reasons the hard disk image will be corrupted after some activity, and the virtual machine could not boot up, showing the following hard disk error:

I have not found out the root cause of this and therefore could not find any fix. A workaround is to set up a perfect working virtual machine that requires no further changes and set up the hard disk as non-persistent, which will discard all changes upon shutdown as per the following screenshot:

You may also want to add another persistent hard disk to virtual machine and only make change to the added hard disk. The main hard disk remains non-persistent. Alternatively, once you get the machine working as desired, take a snapshot and revert back to it should any problem occur.

5. To get sound working, install EnsoniqAudioPCI_v1.0.3_Common_Installer.pkg and reboot the virtual machine. Software updates will also work fine and I have used it to update successfully to 10.6.8 without any hassle.

I hope this helps those with similar problems. I will add to this list any other tricks that may be useful.

Singapore Bus Stop Database

2010-11-18T19:28:00.000-08:00

Attached to this post is something that developers for location-aware applications in Singapore may find useful - the location database of all bus stops in Singapore and the bus route. These databases were created some time back and may not be up-to-date, but should still serve as a good starting point for developers who wish to have local access to bus stop and bus route information without accessing the Internet.

The bus stop database is XML-formatted, and looks like this

   <lon>103.671627904274</lon>  
   <lat>1.30877958711704</lat>  
   <name>Opp Gul Way</name>  
   <road>Pioneer Rd</road>  
   <code>23229</code>

For each bus stop, its geocode (expressed as latitude and longitude), name, road, and code is provided.

The bus route database is also XML-formatted, and looks like this

   <busService>002</busService>  
   <busRoute1>99009 99131 99049 99039 99029 99019 98079 98069 97099 97089 97079 97069 97209 97059 97049 97039 97019 96069 96059 96049 96099 96089 96079 85079 85089 85069 85059 85099 84069 84059 84049 84039 84029 84019 83099 83079 83059 83049 83029 82069 82049 82029 82109 81069 81049 81029 80089 80069 80049 80039 80029 01319 01219 01129 01059 01119 01019 04159 04149 04229 04239 05059 05049 05039 05019 10589</busRoute1>  
   <busRoute2>10589 10017 05013 05023 04222 04142 01012 01113 01121 01211 01311 80011 80031 80051 80071 80091 81011 81031 81051 82011 82032 82061 83011 83031 83062 83081 84011 84021 84031 84041 84051 84061 85091 85051 85061 85081 85071 96071 96081 96091 96041 96051 96061 97011 97031 97041 97051 97201 97061 97071 97081 97091 98061 98071 99011 99021 99031 99041 99139 99009</busRoute2>

Routes are provided in terms of the codes of the bus stops that the bus service calls at. Usually each service has 2 routes (busRoute1 and busRoute2), but for looping services, only 1 route is provided.

Download the XML databases here

See also:

1. Google Maps API on Windows Mobile
2. Reverse-engineering gothere.sg API

My 5V home-made battery

2010-11-10T23:13:00.000-08:00

Inspired by various posts on how to create a battery from household items, I decided to create my version. here is the result:

There are a total of 11 cells - 10 slices of lemon and a bowl of Coke. Each cell provides around 0.47V. The 2 electrodes are the aluminum strip from the Coke can, and Singapore $1 coin.

Although it provides a 5V, the current is only enough to light up an LED briefly. The LED can light up brightly with a single 1.5V AAA cell.

I posted on electro-tech-online.com, my favorite electronics forum, and got some interesting replies from other members who have also made similar batteries. This is another version that provides around 2V:

This is a video showing a lemon battery and a salt-water batter in operation:

This website shows you how to make a Coke battery that can run a calculator:

and a battery from Soda water:

Dual booting Mac OS X Snow Leopard 10.6.4 and Windows XP on Asus 1000HE netbook

2010-10-29T06:11:00.000-07:00

This guide shows you how to install both Windows XP and Mac OS X 10.6.4 (retail version) on your Asus 1000HE netbook, with Chameleon bootloader to allow selection between these two OSes.

Pre-requisites:

1. Asus 1000 HE netbook with 2GB of RAM. Windows XP will work well with 1GB but Mac OS X will boot very slowly, making the whole process pointless.

2. A USB DVD drive (optional). Although you can install Windows XP and Snow Leopard from USB, certain troubleshooting steps (not mentioned here) will be easier if you have a DVD drive.

3. Windows XP SP2/SP3 installation CD. You will need to slipstream the Intel ICH7 drivers into the original installation CD using nLite, otherwise you'll get a blue screen during installation due to error INACCESSIBLE_BOOT_DEVICE. Read the instructions here

4. Mac OS X 10.6.0 and above retail installation DVD.

5. Mac OS X v10.6.4 Update (Combo) download from Apple.

6. Mac OS X 10.6.4 Atom Kernel patch from here.

7. A USB keyboard. This is needed as the onboard PS2 keyboard may require extra driver and will not be recognized by OS X after installation.

You'll also need some other patches and tools described in the section below.

Partition Table: GUID, Prototective MBR and Hybrid MBR

Before we start, let's get some facts clear first. All Mac OSX versions since 10.6.0 (Snow Leopard) by default will only install on GUID partition table (GPT) as opposed to Windows XP, which only supports booting from MBR partition table. There are ways to make Snow Leopard install on MBR, but in all my attempts I could not get a stable system, so let's stick with GPT.

GPT provides backwards compatibility with MBR using something known as protective MBR. With protective MBR, GPT hard disks will appear to non-GPT aware tools as having an unknown partition table, thus preventing these tools from altering and damaging the partition structure.

Protective MBR is not enough to install Windows XP on a GPT hard disk, so we'll need something called hybrid MBR. Up to 4 GPT partitions can be added to the hybrid MBR and can be seen by non-GPT aware tools. To create a hybrid MBR, you'll need something like GPT FDisk (gdisk). Disk Utility on OS X will also write a hybrid MBR by default. There is also another tool called GPTSync which is however less flexible and did not work properly in all my attempts.

The bootloader

Windows XP uses the NT bootloader (NTLDR) which will not load OS X. For XP and OSX to run smoothly together, we'll need the Chameleon 2 bootloader. Before we continue, you should get familiar with installing the Chameleon bootloader.

Partitioning the hard disk

Interestingly, Windows XP can only be installed on either the first or the last partition of a hard disk. For this reason, we partition the hard disk as below:

1. EFI Partition (FAT32). This is where Chameleon 2 bootloader wil be installed.

2. OSX Partition (Mac OS Extended Journaled). This is where Snow Leopard will be installed.

3. DATA Partition (optional). This is where we will store all the data to be shared between OSX and Windows XP once the dual boot has been set up. FAT32 (supported by both OS) is recommended. If you use HFS, you'll need MacDrive to access it from Windows. If you use NTFS, you'll need NTFS-3G to access it from Snow Leopard.

4. WINXP partition (NTFS). This is where Windows XP will be installed.

Installing Snow Leopard

There are plenty of guides on the Internet to install Snow Leopard on ASUS 1000HE using NetbookBootMaker so I will not repeat it here. Use the tool to create a USB installer boot disk and follow the guides to install OS X.

If the Asus 1000HE does not boot from USB, press F2 during startup to enter SETUP. Select the "Boot" menu, select "Boot Device Priority", and move the USB drive to the top of the list. Still under the "Boot" menu, select "Hard Disk Drives", and also move the USB drive to the top of the list. Save changes and your netbook should boot from USB.

If you get the error "The bless tool was unable to set the current boot disk" at the end of the installation process, ignore it.

Next, install Chameleon bootloader to EFI partition using this guide. When you reboot, Chameleon should show OSX as one of the partitions and you can boot from it.

If OSX boots up complaining that it cannot find a keyboard/mouse, connect a USB keyboard first in order to complete the boot process. We will install the driver for onboard keyboard (PS2) later.

Once Snow Leopard has finished booting, install the 10.6.4 Combo update from Apple. After the installation, it is expected that OSX will no longer boot up, since 10.6.4 does not officially support Intel Atom processor. You will see the following error message:

panic(cpu 0 caller 0x1c2a7459): "Unsupported CPU: family = 0x6, model = 0x17, stepping = 0x2"@/SourceCache/AppleIntelCPUPowerManagement/AppleIntelCPUPowerManagement-90/pnProcessor.c:210

You'll need to patch the kernel by booting from the installation media, launch Terminal, and replace the current mach_kernel on the OSX partition with the patched one that you downloaded earlier.

Once Snow Leopard 10.6.4 has booted up, install the following kexts with the help of KextHelper. This is needed to get the basic hardware working:

1. Display: AppleIntelGMA950.kext and AppleIntelIntegratedFramebuffer.kext. This is technically the same as the default driver, except that it has been modified to support Intel GMA950 chipset that has 0x27ae as the hardware ID.

If after installing the kext, OSX boots up to a blue screen, make sure you're using the correct DSDT files for your computer. In the same zip file you downloaded, there are 2 files for 1GB and 2GB of RAM.

2. Ethernet: install AttansicL1eEthernet.kext

3. Audio: VoodooHDA.kext and Voodoo.PrefPane. This is taken from here to solve the low volume issues.

4. Battery Indicator and Power Management: VoodooBattery.kext, VoodooPower.kext, VoodooPowerMini.kext.

5. Bluetooth: the default installation will support bluetooth but will have problem turning it on or off. to fix this, install IOBluetoothFamily.kext

6. Onboard mouse or keyboard: please install VoodooPS2Controller.kext.

7. Card Reader: install VoodooSDHC.kext

For Wifi to work, you'll need to install the RT2860 drivers for OS X. This will launch a tool called WirelessUtilityCardBusPCI which scans for Wifi network at startup.

Creating the Hybrid MBR

Boot from the installation media into Terminal, unmount the disk first and run gdisk. Make sure your device identifier (e.g. /dev/disk0) is correct. Select p to print the partition table. Select r for recovery and transformation options, follow by h for make hybrid MBR and choose to add the Windows XP, OSX and DATA partitions. When prompted to set the boot flags, select NO. (If you select YES, somehow the partition table is corrupted, and Chameleon will hang at "boot0: done" when your system reboots. To recover, you must format the EFI partition as FAT32 and reinstall Chameleon). Finally, type w to write the hybrid MBR.

Installing Windows XP

This is straight forward, just keep in mind that Windows XP will install the NTLDR on the first partition that it recognizes, which is the DATA partition. If this happens, after installation is done, you will need to select the DATA partition at the bootloader to boot to Windows XP! To avoid this, during Windows XP installation, format the DATA partition as HFS (you can reformat it to FAT32 later after everything is done).

After Windows XP is installed, reboot the computer and make sure that Chameleon bootloader can boot to both Windows and OSX. If it can boot to OSX but not to Windows XP, check the boot.ini file on the Windows XP partition to make sure the correct partition number is set.

Finalizing the Installation

At this point we are done, you can dual boot between Windows XP and Snow Leopard 10.6.4 without issues. You can customize the bootloader by editing /Library/Preferences/SystemConfiguration/com.apple.Boot.plist. The more useful keys are "Default Partition" to set default to either OSX or Windows XP and "Hide Partition" to exclude the DATA partition from the boot menu.

To backup the setup so that you won't have to repeat the process again in case of a failure, you'll need CloneZilla Live CD. Backup all EFI, Windows XP and OSX partitions in advanced mode, be careful to select the correct set of options. When restoring, remember to restore in advanced mode with the same set of options selected during backup. Also remember to re-write the hybrid MBR via GPT fdisk every time you make changes to the partition table such as adding/removing partitions.

Performance

With this dual boot configuration, Windows XP performs smoothly (it should!) while OSX encounters some strange behaviour despite booting up in less than 30 seconds. First, it would not wake up from sleep properly despite trying several sleepenabler.kext patches. Occasionally, especially after an improper shutdown, it would also hang at "Waiting for DSMOS" for around 3 minutes during boot up. Although applications such as Safari, iTunes, MSN/Yahoo/Skype Messenger and TextEdit are reasonably fast, office applications like Open Office, Microsoft Office 2011 or even AbiWord are too slow to be usable; you will only see what you type after almost a second! You can play MP3 smoothly, but playing videos via VLC or QuickTime is too slow. Surprisingly, xCode with iPhone SDK runs fast enough to develop some simple iPhone applications. To me, it isn't about the processor not being fast enough for the job, it's more likely many Snow Leopard applications are never optimized for the Intel Atom processor.

You may notice that some graphics applications won't be able to display graphics properly. In particular, the Preview application won't be able to display images, and pressing Apple-Shift-3 or Apple-Shift-4 (maps to Windows key) to capture screenshot will just return an empty black image. This is due to the lack of QE/CI support on the graphics card drivers. I have tried several kexts for Intel GMA 950 which claim support for QE/CI without success.

On a side note, if you install Microsoft Office 2011 on Snow Leopard on Asus 1000HE, it's not easy to get past the welcome screen the first time Word/Excel/PowerPoint is started after installation. The "Close" button is beyond the bottom of the screen due to the low 1024x600 resolution - you'll need an external monitor!

Conclusion

I hope this guide is useful for those interested in dual booting Windows and Snow Leopard. Except for the installation of the drivers which is specific to the Asus 1000HE, the rest of the guide should work for any computer. For simplicity, some troubleshooting steps in installing OSX has been left out. In case of any problems, you can search the InsanelyMac forum or leave a comment here and I will try to help if possible.

Web pages linking to my blog articles

2010-10-19T18:43:00.000-07:00

This post lists some of the web pages that link to my blog articles. I came across them during random Google searches, and via StatCounter's "Recent Came From" list.

1. VoIP linked from

Faxing over VoIP using Kapanga softphone (6 Feb 2010)
some fax calls on Trunk 25 (6 Feb 2010)

2. Free auto-expiring US DID numbers linked from Please help with CallerId Spoof (8 Feb 2010)

3. Retrieve ICC-ID (printed on SIM card) using SIM APIs linked from Retrieve ICC ID in Windows CE without cellcore.dll (20 April 2010)

4. Windows Mobile Pocket PC’s auto-suspend mode linked from Keeping GPRS alive in Windows Mobile App (7 June 2010)

5. SerializableDictionary and System.InvalidOperationException linked from XML Serializable Generic Dictionary (19 June 2010)

6. Read Write Call History linked from How can i Delete a entry in call history in Window mobile 6 using .Net ? (28 June 2010)

7. WebException on HTTPWebRequest with SSL connection: "Unable to read data from the transport connection" linked from CF 2.0/3.5 WebException when doing POST to any HTTPS server at certain intervals (4 July 2010)

8. Windows Mobile linked from Windows CE Programming (15 July 2010)

9. Wikipedia in MobiPocket ebook (PRC) format linked from Pocket Wikipedia (23 Sept 2010)

10. Identifying used and unused resources in a Visual Studio's project Resources.resx file linked from

Find Unused Resources in a .NET Solution (5 Oct 2010)
Detecting un-needed resources in a solution (24 Jan 2011)
Remove unused resources automatically (12 Nov 2011)

11. Intercept incoming SMS message on HTC Phones with HTC Sense linked from MessageInterceptor is not called on HTC HD2 (31 Oct 2010)

12. Restoring SMS Backup from CSV to Blackberry devices linked from

Backing up sms messages (20 Nov 2010)
Transferring SMS messages from a Nokia to a Blackberry (28 Jan 2011)

13. Dual booting Mac OS X Snow Leopard 10.6.4 and Windows XP on Asus 1000HE netbook linked from:

My Asus EeePC 1000HE and how I installed OS X on it (23 Nov 2010)
Can Chameleon boot Windows XP (x86) from GPT partition? (11 Jan 2011)
Dual booting XP and Mac Snow Leopard on Asus EEE 1000he pc - and getting to 10.6.6 (23 Jan 2011)
[Guide] - Dual booting XP and Mac Snow Leopard on Asus EEE 1000he pc - and getting to 10.6.6 (23 Jan 2011)
Installing Mac OS 10.6.4 on a Asus EeePC 1000HE (29 Jan 2011)
Dual Boot XP and Snow Leopard (26 Jun 2011)

14. Migrate SMS from Windows Mobile to Android linked from here (14 Dec 2010)

15. Transferring SMS messages from Windows Mobile to Blackberry devices linked from

[How to] - Transfer SMS from Windows Mobile to Blackberry (10 Dec 2010)
Thread Diskusi khusus untuk Bold 9780 aka ONYX2 (10 Jan 2011)

16. MAC OS System 8.0 on Basilisk linked from Quark XPress 4.0 + Passport + 4.11 updates
See comment by themacmeister, on 16 January 2011, 11:47pm

17. An RTF Editor Control for .NET Compact Framework linked from How to add RichTextBox control to Smartphone Device Application in vb.net2005?

I found this article linked from an experts-exchange answer on 19 Jan 2011. Previously you will be able to view answers to questions on their website without a paid account just by scrolling to the bottom of the page. But that is no longer the case. To view the answers now, you will need a tool such as User Agent Switcher for Firefox and switch the default user agent to Google bot. This way, many other online content such as forum posts can also be viewed without registering an account.

18. Accessing WIN32 API functions from C++ CLI linked from SendMessage в VS2010Exp для Windows Forms (24 Feb 2011)

19. Google Maps API on Windows Mobile linked from Windows Mobile Maps Application (10 March 2011)

20. Casio PB-700 and FA-11 Printer & Tape Recorder linked from Ordinateurs de poche & Calculatrice Casio - PB FX CFX- Pockets : Casio PB-700-Casio PB-700-Casio PB-700-Casio PB-700 (30 March 2011)

21. Programming Nostalgia: revisiting Mike Wiering's Mario game written in Pascal linked from Clone di Super Mario Bros con codice sorgente (Super Mario Bros clone with source code) (23 May 2011)

22. Acer Aspire Launch Manager linked from Is there a way to emulate combination of keystrokes? (7 July 2011)

23. Customize Listview Item Forecolor & Backcolor linked from Custom ListView with possibility to zoom in/out images (12 July 2011)

24. .NET forms on Windows 7 at 125% zoom level linked from Bug 671611 - increased spacing of lines in folder and thread pane, filters, compose and address book on ms-windows for TB6 (17 Sept 2011)

25. Interfacing Nokia 3510i and 5110 LCD with PIC Microcontroller linked from PIC and 5110 Interface with SPI hardware (26 September 2011)

26. TCPMP and AC3 codec for Windows Mobile 6.1 linked from

HTC HD2 - CorePlayer Audio Codec (AC-3) support (21 Oct 2011)
HD2 application (9 Dec 2011)

27. An RTF Editor Control for .NET Compact Framework linked from RTF on Windows CE (7 Nov 2011)

28. Attached the debugger to a running .NET Compact Framework process linked from Is it possible to debug my C# program Through LAN (8 Nov 2011)

29. Dreambook W7: Cheap and apparently features-packed but unusable Chinese-made Android tablet linked from Synrgic Dreambook W7 (3 Dec 2011)

iPhone SDK on Mac OS X 10.6.4 under VmWare

2010-10-08T07:41:00.000-07:00

This guide shows you how to get the latest iPhone SDK running on Mac OS X 10.6.4 under VmWare.

Pre-requisites:

1. A computer whose CPU supports hardware virtualization. You can check if your computer supports this by using CPUZ and check if your CPU supports VT-X instructions.
2. At least 3GB of RAM on host with at least 2GB of RAM allocated to VMWare image.
3. OS X 10.6.4 VMWare disk image which can be downloaded here
4. The latest iPhone SDK downloaded from here
5. A Windows tool to create ISO file, such as MagicISO or WinISO
6. A file splitter tool for Windows such as FFSJ or HJSplit
7. A file joiner tool for Mac such as AJoiner

Steps:

1. Download and start the VMX virtual machine.

The machine should boot up from the emulated CD-ROM, showing the Chameleon bootloader. Wait at least 15 seconds before pressing F5 and select the Apple icon to boot into Mac OS X from the virtual hard disk.

If you select the icon too soon, for some reason, the bootloader will not see the hard disk and you will be stuck at the loading screen forever. If VmWare reports no disk activity a few seconds after the loading screen comes up, reset the machine and try again. Try to wait longer and press F5 a few times before selecting the Apple icon.

2. Configure Mac OS X

The default language of the downloaded OS X image will be Russian. To change to English, you'll need to open the Preferences tab by clicking the top-left Apple logo and trying the various menu options until you get a panel that looks like the following:

Click on the International icon (it should be in the first row). After that, select English and drag it to the top of the list. It should look like the following:

After that, switch to Formats and Input menu and change everything to English (optional)

Disable Software Updates, Sleep (under Energy Saver), Screen Saver and Time Machine.

Also, go to Date & Time preferences, disable time synchronization and select a timezone so that the Mac system time is the same as what is displayed by Windows. This is needed because OS X assumes system time to be in GMT.

3. Increasing the hard disk size to 40 GB

The default hard disk size is too small (20GB) for the iPhone SDK. However, expanding the hard disk is not as simple as using VmWare to expand it to 40 GB and run Disk Utility to grow the partition. If you try it, you will get the "mediaKit reports partition map too small" error message. GParted also won't help as it can only shrink, not grow HFS+ partition.

I even tried to use CloneZilla to backup the original HFS+ partition from the 20GB hard disk, and restore it to the new HFS+ partition on the 40GB hard disk. This time, although Disk Utility shows the HFS+ partition size correctly as 40GB, Finder still reports the size as 20GB. Attempting to repair the disk under Disk Utility fails with error "invalid b-tree node size". A Google search suggests DiskWarrior which unfortunately does not boot up under VmWare using the Chameleon bootloader.

The solution (unbelievably simple) that I found was

a. Add a new virtual IDE (not SCSI) 40GB hard disk to the original OS X image.
b. Boot up from the original hard disk and use Disk Utility to create a single HFS+ partition on the 40GB hard disk.
c. Use Disk Utility to restore the original OSX partition to the newly created partition.
d. Shutdown OSX. Under the properties of the virtual machine, remove the newly created hard disk (not the image) and change the original hard disk to point to the new image.
e. Reboot. OSX should now see the hard disk as 40GB.

4. Getting the iPhone SDK installer on OSX

This is the tricky part. I have tried the following but failed:

a. Use Safari inside OS X to download the SDK.
For some reason the download was successful but the resulting DMG file was corrupted.
b. Download it under Windows to an external hard disk drive and use the Vmware's VM menu to map the drive to OS X.
OS X would recognize the file and seemed to be able to copy it over. The copy process however would hang Finder prematurely and I had to resort to a reboot.
c. Same as (b) but splitted the file to smaller parts before copying.
OS X would still hang while copying, no matter how small the size of each segment is. This seems to be a problem with how VmWare manages the hardware.

The solution I found is

a. On Windows, split the DMG file to many segments around 100 MB each using HJSplit/FFSJ.
b. Use MagicISO/WinISO to create an ISO file with all the splitted segments
c. Use VmWare to add a new virtual CD-ROM drive to the OS X which points to the ISO file.
d. Open the emulated drive in OS X and copy the segments over.
e. Use AJoiner to merge all the segments to create the installer DMG file.

5. Installing the iPhone SDK

This step is straight forward and should take around 30 minutes to 1 hour, depending on the speed of your computer.

That's all. You can now enjoy iPhone development on your PC. In my computer, I allocate 4GB of RAM to OSX and it's really fast. Except that there's no sound support, the rest is just fine:

I hope this guide will help those who want to develop iPhone apps but do not have a Mac. If you encounter problems setting up the virtual machine, refer to this post.

Restoring SMS Backup from CSV to Blackberry devices

2010-10-01T06:02:00.000-07:00

UPDATE (20 Jan 2012): Better support for Unicode characters. Sample CSV files with Unicode messages in various languages are included in the download. CSV2IPD will also report the number of successfully imported messages at the end of the conversion, not just the number of invalid records.
UPDATE (18 Nov 2011): Use .NET DateTime class instead VB6 CDate function to parse date strings (which may fail on non-English systems). Also added a log file (CSV2IPD.log) that can be used for debugging.
UPDATE (7 May 2011): Following a request from a reader, the program has been updated to handle CSV more robustly, supporting multi-line messages and messages having semicolons. It also reports the number of invalid records in the CSV file after the conversions. The source code and binary files have been updated.

Following my previous post on a Windows Mobile tool called SMS2IPD that would run on a Windows Mobile device and creates and IPD backup of all the SMS that can be restored to a Blackberry device via Blackberry Desktop Manager, a German reader contacted me asking how he could do the same for his Nokia phone.

Being no Symbian expert who could write a similiar tool running on Symbian phone, I advised him to use Nokia PC Suite to create a CSV backup of all his SMS. After that, I have written a tool that will help create an IPD from the CSV file.

The program, called CSV2IPD, accepts 2 CSV input files for Incoming SMS and Outgoing SMS, as demonstrated in the following screenshot:

The CSV for Inbox looks like:

sms;deliver;"+987654321";"";"";"2010.08.25 15:14";"";"Message 1"
sms;deliver;"+123456789";"";"";"2010.08.25 15:15";"";"Message 2"
sms;deliver;"+111111111";"";"";"2010.08.25 15:17";"";"Message 3"

The CSV for Sent Items looks like:

sms;submit;"";"+0123456789";"";"2010.08.25 15:15";"";"Hello world 1."
sms;submit;"";"+0123456789";"";"2010.08.25 15:16";"";"Hello world 2."
sms;submit;"";"+0123456789";"";"2010.08.25 15:17";"";"Hello world 3."

If your exported CSV file via Nokia PC Suite does not look exactly as above, you can use a text editor tool such as Notepad++ then use Find/Replace, perhaps with the help of regular expressions, to modify it. Make sure you do not have extra quotation marks at the beginning and the end of each line. If the delimiter is comma and not semicolon, use CSVEd to change it. Finally, if your time column is in a different format, try to tweak your Regional Settings (in Control Panel).

The generated IPD file can be imported using Blackberry Desktop Manager for Windows. It may not work on the Mac version of Blackberry Desktop Software. You will need .NET Framework 2.0 (already included with Windows Vista and above) to run the program. If you are using Windows XP, you can download .NET here.

Unicode text messages are supported, just make sure that your CSV files are saved using UTF8 encoding. As always, remember to backup all data on your phone prior to proceeding.

The source code for the tool can be downloaded here.

FINDSTR and Unicode strings

2010-09-18T05:27:00.000-07:00

Recently I had another look at my old post on finding unused resources in a .NET project, and realized that it would not work if any of the resource strings contain Unicode characters. The culprit turns out to be the FINDSTR command, which doesn't support Unicode text files. To solve the problem, I replace FINDSTR with FIND.EXE (which supports Unicode). However, all is lost when my usage of FOR to tokenize the input files stops working with Unicode input. I also tried a few suggestions here and here to no avail.

My simple conclusion is that, batch script was not designed with Unicode in mind. Perhaps Windows Powershell can probably do a better job...

Re-partitioning the HP G42 laptop

2010-09-07T05:23:00.000-07:00

I recently purchased a HP G42 laptop having an Intel Core i5 processor and 4GB DR3 RAM. Everything was good until I failed to create a few more partitions on the built in hard disk drive for software, documents, etc. The problem was with the default partition structure:

* SYSTEM (200MB hidden). This was apparently created during Windows 7 setup when the hard disk has no other partitions. The partition only contains the bootloader and its various language packs.
* WINDOWS (NTFS). This is where Windows 7 is installed.
* RECOVERY (NTFS hidden). If you press F11 during boot time, BIOS will boot from this partition to begin the recovery process.
* HP_TOOLS (hidden). Some BIOS utilities will require tools on this partition.

There are already 4 primary partitions so creating another one is impossible unless you have an extended partition. If you want to dual boot with another OS, you will have to delete one of the existing partitions as most OS cannot boot from an extended partition.

It's actually easy to delete the SYSTEM partition if you know how to do it correctly:

1. Use EasyBCD to write the bootloader to the Windows partition (and not the SYSTEM partition). Reboot the system and make sure that Windows can start.

If Windows fails to start, set the SYSTEM partition back to Active (boot) and there should be no damage. During my experiment, I used GParted Live CD for this purpose.

2. If step 1 is successful, delete the SYSTEM partition. As it's no longer in use, Windows should still boot properly.

Reclaiming the space occupied by the SYSTEM partition is tricky. My first attempt was to use GParted to extend the Windows partition to cover the free space. This process took almost half a day and eventually failed with an unspecified error, making the whole hard disk unusable with a partition table error. Only a low-level format would help. As of now, I am still not sure of the cause.

The solution was to use Norton Ghost, which only took around 1 hour:

1. Create an image of the working Windows partition (after the SYSTEM partition has been deleted).
2. Delete the Windows partition and create a new partition covering the original SYSTEM partition and the just deleted Windows partition.
3. Restoring the ghost image to the newly created partition in step 2 and the system should be able to boot into Windows

Why can't I simply just delete the SYSTEM and the Windows partition and use the HP recovery program to restore? Unfortunately it wasn't that simple, since the HP Recovery program does not ask you where you want your Windows partition to be and simply recreates the SYSTEM, WINDOWS and HP_TOOLS partition. It does not, however, re-create the RECOVERY partition if this partition does not already exist (e.g. the recovery program was launched from a USB or DVD backup).

The entire HP_TOOLS partition contents is stored on the RECOVERY partition, which is hidden from Windows Explorer and is only accessible via a third party file explorer such as Free Commander. This partition contains the pre-installed Windows 7 image, with a .wim compressed file (readable via 7Zip) that has installers to some bundled programs. The device drivers can also be extracted from a a folder named SWSetup in this .wim file.

One last thing about the HP Recovery Manager program is that it only allows you to backup the recovery image to DVD/USB once. In my case, I backup to a USB thumbdrive, which can later be restored to a bootable hard disk partition by simply copying all the files over and use EasyBCD to write the bootloader. If you backup to DVDs, be sure to make ISO copies of the disks.

Once you have finished re-partitioning the system and set up all your software, simple use Norton Ghost to backup/restore your setup. Do not ever use HP Recovery Manager again as it will destroy all your hard work!

Singtel Caller Alert

2010-08-28T06:58:00.000-07:00

Recently while checking my call divert settings, I noticed a strange number "+6519454" in my "Forward when unreachable" settings, which would always revert itself despite my several attempts to clear the settings:

It turned out that the number belongs to the free Singtel Caller Alert service, which sends you a free notification SMS if somebody tries to call you when your phone is unreachable. Although it's a nice thing to have, your caller will be charged for the call since the call is taken as connected, which is odd for Singtel since may other telcos provide the service for free. What's more, the setup is weird - why must Singtel enforce an always-on divert settings in order to activate this free service? Why can't they do it on their side?

Interestingly, if you try to call 19454 from a Singtel mobile phone, you'll hear the same announcement as if you were calling somebody with the free Caller Alert service and you will still be charged for the call!

Allowing only numeric input in a TextBox

2010-08-15T01:36:00.000-07:00

You may have seen a Windows forms textbox that would disallow non-numeric input, showing an error message like in the following screenshot:

This can be easily accomplished from .NET by applying the ES_NUMBER style to the textbox:

Public Sub SetNumericInputMode(ByVal txtBox As TextBox)
    Dim style As Int32 = GetWindowLong(txtBox.Handle, GWL_STYLE)
    SetWindowLong(txtBox.Handle, GWL_STYLE, style Or ES_NUMBER)
End Sub

Of course, SetWindowLong has to be declared properly:

<DllImport("user32.dll")>

Private Shared Function SetWindowLong( _
     ByVal hWnd As IntPtr, _
     ByVal nIndex As Integer, _
     ByVal dwNewLong As IntPtr) As Integer
End Function

Now what if you're using the RadTextBox control from the Telerik RadControls for WinForms? The following code works for a normal textbox, compiles but will not work for a RadTextBox:

SetNumericInputMode(textBox1.Handle);

The reason is that the RadTextBox is just a container for the native Win32 textbox. Any style changes have to be applied directly to the actual textbox, not the container. The following will work:

Public Sub SetNumericInputMode(ByVal txtBox As RadTextBox)
  'special handling for RadTextbox as the actual Win32 textbox is hidden underneath
  Dim hwnd As IntPtr = CType(txtBox.TextBoxElement.Children(0), RadTextBoxItem).HostedControl.Handle
  Dim style As Int32 = GetWindowLong(hwnd, GWL_STYLE)
  SetWindowLong(hwnd, GWL_STYLE, style Or ES_NUMBER)
End Sub

Notice that ES_NUMBER only prevents user from entering non-numeric (0..9) input for the textbox. It does not stop user from pasting random text. For more advanced features, I suggest something like MaskedTextBox

Migrate SMS from Windows Mobile to Android

2010-08-08T06:05:00.000-07:00

To migrate SMS from Windows Mobile to Android, you will need the SMS Exporter tool, which will export all SMS to an XML file that can be imported to Android using SMS Backup Restore

Same as my SMS2IPD tool that migrates SMS from Windows Mobile to Blackberry, SMS Exporter uses MAPIDotNet library found on codeproject, written by rwt33. The export speed on Windows Mobile is fast, but the import speed is very slow. On my HTC Dream flashed to Android 2.1, it took almost 2 hours to import 3000 messages - the process became slower towards the end and the phone would go to standby mode during the process. After the painful import , the message application is still responsive despite the huge message database.

SMS Exporter also supports backing up SMS to XML, which is useful if you want to read your SMS on a computer, or migrate your SMS to other phones.

An RTF Editor Control for .NET Compact Framework

2010-07-23T20:58:00.000-07:00

As you may have known, there is no RichTextBox control in .NET Compact Framework. This creates a lot of headaches for developers who want to display RTF-formatted text in their program. The only way is to use the RichInk control. However, there is no good free managed wrapper for it, even the OpenNetCF's InkX wrapper is still untidy and requires a lot of P/Invoke calls to make it work properly. IntelliProg used to provide a working control but the company has seemingly disappeared (their domain is parked). The only commecial solution available now is provided by dsrtech but it is expensive.

AgileNotesTouch with RichInk control

After some research, I found a freeware called AgileNotes Touch which allows user to write notes and save it into TXT, PWI or RTF. Its interface is similiar to Pocket Word:

Since the source code is not available, I decided to go ahead decompiling the executable using Reflector. This was an easy task - the decompiled source code, with only some minor modifications, compiled and ran as if it were the original. It turned out AgilesNotes Touch also uses the RickInk control, with some nice managed .NET wrappers which can be re-used easily.

Creating RTF programmatically: NRtfTree

Since I need to create the RTF document programmatically, I decided to use the open source project NRtfTree. Some basic formatting worked well until I tried to create an RTF document with images such as this. The document was created properly but to my disappointment the image was not displayed when the document was open with my program and with Pocket Word. On the other hand, Wordpad, and Office Word, displayed the image just fine. It took me a long time before I figured out the cause - the RichInk control (and the RichEd50W control used by Pocket Word) only supports binary images, and not hex images. For example the following picture will be discarded:

{\pict\pngblip\picw10449\pich3280\picwgoal5924\pichgoal1860 hex data}

but the following will work:

{\pict\pngblip\picw10449\pich3280\picwgoal5924\pichgoal1860\bin binary data}

And since NRtfTree only supports hex image, I had to modify the library to support binary image. What a hassle, but in the end everything worked well and images are displayed properly.

From RichInk to RichEdit50W

Next, I noticed that my control could not display images on the same line as text and seem to drop tables, while Pocket Word displayed them just fine. This is a hint that Pocket supports higher RTF versions then RichInk, so I use Remote Spy to find out which control Pocket is using:

PocketWord never used RichInk, but RichEd50W, similiar to what RichTextBox is using. I changed my code to use this window class (it stays in RichEd20.dll):

WindowsAPIs.LoadLibrary("RichEd20.dll");

this.m_RichInk = new WindowHost("RICHEDIT50W", 0, 0);

and images are displayed as intended. Surprisingly that's all I need - my program is now as good as Pocket Word.

Making the document read-only

My main purpose was to display documents - and not allowing user to modify them so I applied ES_READONLY to the window style:

this.m_RichInk = new WindowHost("RICHEDIT50W", 0, ES_READONLY);

There is still a minor problem: the window display may be distorted or incomplete after some extensive usage. I have yet to figure out the cause of the problem or how to fix it.

Source code

The sample source code is here. It includes the modified NRtfTree library, the decompiled source code of AgilesNotes Touch, and a sample form showing you how to read and write RTF files. NRtfTree was modified to insert binary images, instead of hex, and to support unicode characters. I have commented clearly where I modified the code.

I hope this will help other developers facing the same problem.

Beware when using DES/TripleDES for data encryption

2010-07-17T08:59:00.000-07:00

I recently worked on a secure device application which requires a PIN code to be entered as startup.The application database also needs to be encrypted. My first approach was to use a DESCryptoServiceProvider, which is among the few supported encryption methods in the .NET Compact Framework, to encrypt the database using the PIN code itself as the secret key.

My full code can be found here.

Soon after, I discovered the problem. The encrypted database can be decrypted if the secret key provided is slightly wrong. For example, if the secret key I used when encrypting is 12345678, the decryption will work with both 12345678 and 12345778 (wrong value). Thinking that this was almost certainly a bug, I went ahead to file a report at Microsoft Connect.

I did not have to wait until Microsoft responsed me (which was 3 days later) for an answer. Soon after I read the Wikipedia article on DES, I immediate realized my mistake. The key ostensibly consists of 64 bits; however, only 56 of these are actually used by the algorithm. Eight bits are used solely for checking parity, and are thereafter discarded. This explains why the wrong secret key can be used to decrypt the data as described above.

I changed to TripleDES only to observe the same behaviour. As AES is not supported on the compact framework, I end up encrypting the database using an MD5 hash of the PIN code. Although you may argue that MD5 has some chance of collision, the above hash + encryption combination will ensure, at least with ultimate possibility, that there is no way for the database to be decrypted without the correct password.

Intercept incoming SMS message on HTC Phones with HTC Sense

2010-07-14T20:29:00.000-07:00

In my previous post I have explained the problem with MessageInterceptor on new HTC Phones and how to temporary fix it it by installing the patch. Although this allows existing applications to work without any code changes, user will have to use Pocket Outlook, which has an inferior user interface compared to HTC Messages, to send SMS. MMS are not supported until the Arcsoft MMS client is installed. HTC Sense will not be able to show new incoming text messages, which is a major limitation.

Luckily there is a way to intercept incoming text messages without installing any patch. However, you will have to make some code changes to your application. The idea is that, although the .NET MessageInterceptor class does not work, the SDK MapiRule sample (C:\Program Files\Windows Mobile 6.5.3 DTK\Samples\Common\CPP\Win32\MapiRule) works just fine. This post shows you how to modify the MAPIRule sample code to integrate it with your .NET project

Modifying MAPIRule.dll

You will not have to touch most of the MAPIRule code, except for modifying the GUID (important!) and the ProcessMessage() function (the sample just displays the incoming text message received in a MessageBox). Although you can parse the incoming message and process it directly inside ProcessMessage(), I decided on the following to make things clearer:

When MapiRule.dll received a new SMS message in ProcessMessage(), it simply send a Win32 message via SendMessage to the host application.
Upon received the specific Win32 message, host application, which is a .NET application, will decide on whether or not to intercept the new SMS message (it will not appear in phone Inbox) or let the default messaging application handle it.
Based on the result of SendMessage(), MapiRule.dll will act accordingly.

This interprocess communication can be easily done by using the Win32 API SendMessage() and the .NET Compact Framework's MessageWindow class. Although you can specify your own message ID, I have chosen WM_COPYDATA since MapiRule.dll needs to send the SMS text and sender number to the host application, otherwise string pointers to message text and sender numbers cannot be shared. The sample code is below. Notice that newMsgWin is the handle to the host application's window. The sender number and message text is concatenated into a single string to be sent out.

HRESULT CMailRuleClient::ProcessMessage(IMsgStore *pMsgStore, ULONG cbMsg, LPENTRYID lpMsg, ULONG cbDestFolder, LPENTRYID lpDestFolder, ULONG *pulEventType, MRCHANDLED *pHandled)
{
............
        TCHAR msgInfo[500];
        COPYDATASTRUCT cds;

        // we concatenate the sender number and the message text, to be sent to the host application
        memset(msgInfo, 0, sizeof(msgInfo));
        wcscpy(msgInfo, pspvEmail->Value.lpszW);
        wcscat(msgInfo, L"\n");
        wcscat(msgInfo, pspvSubject->Value.lpszW);

        //length for the recipient to know
        cds.dwData = wcslen(pspvEmail->Value.lpszW) + wcslen(pspvSubject->Value.lpszW) + 1;

        cds.cbData = sizeof(msgInfo);          //msg size in bytes
        cds.lpData = &msgInfo;            //pointer to the information to be sent

        // tell the main app about the text message
        if (SendMessage(newMsgWin, WM_COPYDATA, 0, (LPARAM) &cds) == (LRESULT) 1)
        {
            // a LRESULT of 1 means that the message was processed by parent application
            // so we delete the message and mark it as handled so it won't show up in Inbox
            hr = DeleteMessage(pMsgStore, pMsg, cbMsg, lpMsg, cbDestFolder, lpDestFolder, pulEventType, pHandled);
        }
        else
        {
            // other LRESULT means message not handled by main app, pass it on
            *pHandled = MRC_NOT_HANDLED;
        }
............

}

Everything is straightforward, except that the lpData member of COPYDATASTRUCT structure has to be part of the structure memory area. This explains why an array of TCHAR, an not LPWSTR is used. If this is not followed, application may work or crash randomly without any indication why.

The .NET code is as follows:

    [StructLayout(LayoutKind.Sequential)]
    private struct CopyDataStruct
    {
        public int IntData;
        public int Length;

        [MarshalAs(UnmanagedType.LPWStr)]
        public string Data;
    }

public class NewMsgWindow : MessageWindow
{

..........

    protected override void WndProc(ref Message m)
    {
        if (m.Msg == WM_COPYDATA) //we received a message telling us that there is an incoming SMS
        {
            CopyDataStruct cs = (CopyDataStruct)Marshal.PtrToStructure(m.LParam, typeof(CopyDataStruct));

            if (cs.Data.Contains(delim))
            {
                string sender = cs.Data.Split(delim)[0];
                string messageText = cs.Data.Split(delim)[1];

                if (true)
                {
                    // a LRESULT of 1 marks message as processed.
                    // and native msg app will not show msg in Inbox
                    m.Result = new IntPtr(1);
                }
                else
                    // Other LRESULT indicates we did not intercept the message
                    // and the message will be passed on to native messaging application.
                    m.Result = new IntPtr(0);
            }
        }
.......

}

The challenge is to receive the WM_COPYDATA message and marshal it back to a string. Most sample codes use GetLParam(), but as this is not supported by .NET CF, we have to use Marshal.PtrToStructure(). As commented, the .NET code will response with a result of either 1 or 0.

Source code

The sample code is here. Some part of the code was taken from the Remote Tracker open source project which also intercepts incoming text messages. Remote Tracker source code, however, parses the text message directly inside MapiRule.dll.

Setting up

Some registry keys need to be modified for MapiRule.dll to be used. This can either be done via a CAB file, or via the CreateInterceptorMethod2() method in the code. A reboot is needed for changes to take effect. If you need to change MapiRule.dll, you will have to remove the registry keys via RemoveInterceptorMethod2(), reboot the device, update the dll, call CreateInterceptorMethod2() and reboot again! Terminating poutlook.exe and tmail.exe may also help to reduce the number of restarts on some devices. This makes the development process very time consuming. Debugging MapiRule.dll is possible by attaching the debugger to tmail.exe.

Issues

There are still some minor issues yet to be solved. On HTC HD2, for some reasons, all text messages received will have the string " - GSM" appended at the end (refer to this discussion). Also, the total length of the sender name and the message text cannot exceed 500 since the TCHAR array length is hard coded. If you need to receive longer text message, the interprocess communication algorithm has to be modified to send and receive the text message part by part. You cannot simply increase the array length, as it will cause a stack fault.

Last but not least, the approach described here works on all devices, including HTC Devices. So it can be used as a replacement for the MessageInterceptor class. In a sense, it's better as you can decide, at the point of receiving, which message to be processed, instead of pre-creating a set of MessageInterceptor conditions and re-creating them should the interception conditions change.

Common problems with .NET CF WebBrowser controls

2010-07-06T14:57:00.000-07:00

Although most .NET developers will immediately think of the WebBrowser control when it comes to displaying HTML-formatted document, that seems to be more true for the full .NET framework rather than the compact framework. In fact, if your smart device application has to support a wide variety of devices, below are some reasons why the use of the WebBrowser control is discouraged:

1. Inconsistent HTML support across different devices. Although simple formatting like bold, italic, underline and tables tend to work well, more complicated constructs (css, javascripts) will be dropped or simplified by many devices. There is no way to know for sure except for testing.

2. WebBrowser.Navigate() opens a new window. On many devices, especially on devices where Opera is the default web browser, calling WebBrowser.Navigate() on a local file will simply open a new Opera browser window and leave the application. This will prevent many applications relying on the WebBrowser control from working.

A workaround is to set back IE as the default web browser by modifying registry key, see this. Another workaround is not to use the extension ".htm" or ".html" when calling .Navigate(). For example, instead of

WebBrowser1.Navigate("\\test.htm");

use:

WebBrowser1.Navigate("\\test.tmp");

The document contents will still be rendered properly despite the ".tmp" extension. This trick, however, does not help if you want to the web browser to navigate to an external URL, e.g. http://www.google.com.

Also, on many devices, the WebBrowser.Navigating event never fires, which I could not find a workaround for it.

3. Local images not rendered when DocumentText is set. This may seem a trivial problem with image path or format, but it's actually not. The problem is particular to .NET Compact Framework 3.5 running on Windows Mobile 6.5. The web browser control will refuse to render any local images and only display placeholder if the HTML is set via the DocumentText properly. Strangely it will display remote images from a web server. Images are also displayed correctly if the DocumentText is written to a file and loaded by calling Navigate(). The problem never appears on older .NET Compact Framework or Windows Mobile versions.

All the above seemingly trivial problems are just making life harder for the Windows Mobile developers.

Blackberry and phone numbers with text

2010-07-02T09:22:00.000-07:00

Recently I received an advertisement SMS from Standard Chartered Bank having "StanChart" as the Sender ID. I accidentally pressed the dial (green) key and to my surprise my phone started to call 782624278. All I could get was, however, an automatic IVR announcement saying that the number is invalid. For a moment I wondered where the phone retrieved the number from, as I did not have an entry name 'StanChart' in my phonebook, and certainly would never store an invalid number in my phone book.

The answer is that, Blackberry supports auto-detection of text in phone numbers, For example, you can have a contact with 1800-CALL-LTA as its number, and the phone will automatically call 1800-2255582 (Singapore Land Transport Authority) for you. My case was a similiar example, the phone automatically translated "StanChart" to 782624278 and dialled the number for me.

Although the first case (1800-CALL-LTA) is no doubt very useful to assist users in memorising the phone number, automatically conversion of an SMS sender ID to a phone number is sort of an overkill and result in the wrong number being called.

The same behaviour can be seen on iPhone as well. Windows Mobile, however, does not support the feature.

.NET CF MessageInterceptor class fails to intercept incoming messages

2010-06-29T05:01:00.000-07:00

If you're writing an SMS application that intercepts incoming text messages using the MessageInterceptor class and observe that your application does not work on a particular phone, below is a list of things to check:

1. Most new HTC Phones will choose HTC Messages (part of HTC Sense), and not Pocket Outlook, as the default messaging spplication. HTC Message does not implement the MessageInterceptor class. To overcome this problem, you'll need to set Pocket Outlook as the default messaging application by using this patch.

Note: You may not be able to send/receive MMS after installing the patch. To use MMS, please install the Arcsoft MMS client.

Update (19 July 2010): I have found a way to intercept incoming messages on affected HTC phones without installing any patch by using MAPI Rule. For more information, see this post.

2. Some phones have a specific registry key to manually enable MessageInterceptor. Check HKEY_LOCAL_MACHINE\Software\Microsoft\Inbox\Svc\SMS\Rules and look for a key branch that looks like a GUID, e.g. {1000BC1C-F4A3-4210-B197-4AEBF2CEE6F5} Open that key, set the default value to 0 and reboot the phone to allow message interception.

3. You may have conflicting message interceptor rules left-over from other applications. To fix this, go to HKey_Local_Machine\Software\Microsoft\Inbox\Rules\, delete all sub-entries and reboot the phone. To avoid this from happening in the future, always call Dispose() on a MessageInterceptor when you finish using it or when your application closes.

Reference:

1. http://social.msdn.microsoft.com/Forums/en/vssmartdevicesvbcs/thread/e56f6db2-70d6-43a9-b3f2-7d476aabfa7e

2. http://stackoverflow.com/questions/2138650/messageinterceptor-is-not-called-on-htc-hd2

Wikipedia in MobiPocket ebook (PRC) format

2010-06-23T20:54:00.000-07:00

Following my previous post on a version of Wikipedia for Windows Mobile improved from the original Pocket Wikipedia 1.0 version by free-soft.ro, I decided to find a MobiPocket (PRC) version, to read on my Blackberry phone, Unfotunately I could not find a usable version - many versions I found, including PRC, are incomplete, with images stripped off, and not suitable for mobile viewing. There is also an expensive Wikipedia software for most mobile platforms. TomeRaider also offers a few free versions of wikipedia (with images, and compact without images around 50MB) of Wikipedia in its propietary format. As none of these suit my needs, I decided to go ahead with creating my own PRC version of Wikipedia.

The article database

I decided to use the same article database (Wikipedia.wi) as Pocket Wikipedia 1.0, which turns out to be the 2007 School Wikipedia selection. Although the source code was never released, the binary was not obfuscated and after a bit of decompiling using .NET Reflector, I was able to extract the articles and images from the 180MB SevenZip-compressed database.

Building the PRC ebook

My first thought was to rely on the MobiPocket Creator user interface. However, its UI is terrible - there is no way to add multiple HTML/image files at a time, you have to add them one by one. Even if drag and drop is supported, the application stops responding when a lot of files are added. I then decided to create the OPF file myself, then feed it into mobigen or kindlegen in order to create the final PRC file.

The source code to extract the articles and create the OPF file was written in .NET and can be downloaded here. Once the OPF is created, as there are more than 5000 articles and 24,000 images, kindlegen/mobigen takes more than 15 minutes on a 3Ghz processor to create the final PRC file.

Some of the articles contain Unicode characters (for example, various currency symbols) but were extracted and saved in ASCII format. I have tried various methods in System.Text.Encoding to convert to Unicode before saving without success. The only resolution I found is to use UTFCast Express (freeware) to convert the HTML files to Unicode before feeding them into kindlegen/mobigen.

The product: Wikipedia on a 214MB PRC file

The final compressed PRC file can be downloaded here. It's a multi-part RAR file, so you will need to download both parts to the same directory and use WinRar to extract the PRC file.

PocketWikipedia.part1.rar

PocketWikipedia.part2.rar

It contains all articles and images as in the original version, with a subject list, and an index where the titles of all articles can be looked up. As the title list is generated automatically by guessing the few words of the article, there are cases where the title are not retrieved properly, which can be resolved by editing the index manually in the OPF file before calling kindlegen to generate the PRC file.

Due to the large file size, some desktop versions of MobiPocker ebook reader may fail to open the file due to a Win32 exception. Mobile versions, in particular Blackberry and Windows Mobile, seem to open the PRC file properly.

UPDATE (8 Oct 2010): A Vietnamese reader has used my instructions to create an improved version of the Wikipedia ebook in PRC format. The new version, which fixes some font problems and has improved search support, can be downloaded here and here. It's a multi-part RAR file, so you will need to download both parts to the same directory and use WinRar to extract the original PRC file.

SerializableDictionary and System.InvalidOperationException

2010-06-19T06:08:00.000-07:00

I recently attempted to use Paul Welter's SerializableDictionary in one of my Windows Mobile projects, Everything seems to work well with no code motifications, until when I need to serialize a class containing 2 dictionaries:

public class AppConfig

{

public SerializableDictionary Dictionary1;
public SerializableDictionary Dictionary2;

}

Attempt to serialize the above class will fail with a System.InvalidOperationException: Two mapping for dictionary. Searching the web for the exact error message returns no useful result, except for a posting which described a similiar problem (Two mappings for SourceTree.DTO.InventoryItemCollection.) but did not provide a solution. The posting also explicitly mentioned that the error occurred only on .NET Compact Framework, and not on the full framework.

The root cause of the error is at the first few lines of the SerializableDictionary source code:

[XmlRoot("dictionary")]

public class SerializableDictionary
: Dictionary, IXmlSerializable

With this declaration, when 2 SerializableDictionary stay in the same class (which will be serialized), there will perhaps be a conflict somewhere as both of them are set up to use the same XmlRoot element ("dictionary").

A workaround is not to define XmlRoot inside SerializableDictionary class, but to create another class inheriting from SerializableDictionary and define XmlRoot in this class:

//parent class: SerializableDictionary

public class SerializableDictionary
: Dictionary, IXmlSerializable { .... }

//dictionary 1

[XmlRoot("SerializableDictionary1")]

public class SerializableDictionary1 : SerializableDictionary { }

//dictionary 2

[XmlRoot("SerializableDictionary2")]

public class SerializableDictionary2 : SerializableDictionary { }

//the class to be serialized

public class AppConfigNew

{

public SerializableDictionary1 Dictionary1;
public SerializableDictionary2 Dictionary2;

}

This requires a few more lines of code, but is the cleanest workaround I have found. Also, I have yet to figure out why the full framework does not have such a problem.

Extension method and VB.NET With construct

2010-06-13T08:37:00.000-07:00

Recently I came across a codeproject article about extension methods in VB.NET, where one reader complained that he could not get it to work when using VB.NET's With construct. The comment by the reader is very long and contains irrelevant code, but the issue is valid. This post will demonstrate the issue and summarize some of my findings.

Suppose you have the following code:

Snippet #1:

    <System.Runtime.CompilerServices.Extension()>
    Public Sub ModifyString(ByRef source As String)
        source += " modified"
    End Sub

    Sub Main()
        Dim str As String = "original"
        str.ModifyString()

Debug.WriteLine(str)
End Sub

When the code is run, you will get the value "original modified" as expected. Now change the extension method call to be inside a With construct:

Snippet #2:

    Sub Main()
        Dim str As String = "original"
        With str
            .ModifyString()
        End With

Debug.WriteLine(str)
End Sub

You will now get "original", e.g. the string "temp" is never modified! This is clearly a bug in the framework itself so no workaround is available. Neverthess, let's look at the IL code for the above 2 code samples.

Snippet #1 in IL (No 'With' is used):

         .method public static void Main() cil managed
        {
            .custom instance void [mscorlib]System.STAThreadAttribute::.ctor()
            .entrypoint
            .maxstack 1
            .locals init (
                [0] string str)
            L_0000: nop
            L_0001: ldstr "original"
            L_0006: stloc.0
            L_0007: ldloca.s str
            L_0009: call void ConsoleApplication1.Module1::ModifyString(string&)
            L_000e: nop
            L_000f: nop
            L_0010: ret
        }

Snippet #2 in IL ('With' is used):

         .method public static void Main() cil managed
        {
            .custom instance void [mscorlib]System.STAThreadAttribute::.ctor()
            .entrypoint
            .maxstack 1
            .locals init (
                [0] string str,
                [1] string str2)
            L_0000: nop
            L_0001: ldstr "original"
            L_0006: stloc.0
            L_0007: ldloc.0
            L_0008: stloc.1
            L_0009: ldloca.s str2
            L_000b: call void ConsoleApplication1.Module1::ModifyString(string&)
            L_0010: nop
            L_0011: ldnull
            L_0012: stloc.1
            L_0013: nop
            L_0014: ret
        }

The different is obvious when a text comparison tool such as ExamDiff is used. When a With construct is used, the compiler generates extra code to create a copy of the string variable (str is copied to str2) and pass it to the extension method ModifyString! So whatever changes made to the string have no effect on the original variable, in spite of the ByRef keywork to pass the string by reference. This explains why we get the original value of the string variable.

Now change the code to:

Snippet #3:

    Sub Main()
        Dim str As String = "original"
        With str
            str.ModifyString()
        End With

Debug.WriteLine(str)
End Sub

We still use the With construct, but instead of using shorthand to call the extension method, we explicitly refer to the string variable. Guess what, now you'll get the correct result "original modified"! Let's look at the IL code to see what happened:

Snippet #3 in IL:

         .method public static void Main() cil managed
        {
            .custom instance void [mscorlib]System.STAThreadAttribute::.ctor()
            .entrypoint
            .maxstack 1
            .locals init (
                [0] string str,
                [1] string str2)
            L_0000: nop
            L_0001: ldstr "original"
            L_0006: stloc.0
            L_0007: ldloc.0
            L_0008: stloc.1
            L_0009: ldloca.s str
            L_000b: call void ConsoleApplication1.Module1::ModifyString(string&)
            L_0010: nop
            L_0011: ldnull
            L_0012: stloc.1
            L_0013: nop
            L_0014: ret
        }

A copy of the original string variable (str2) is created as usual, but it was never used. Instead, the original string variable (str) is passed to the extension method. This explains why everything works as intended.

The conclusion is to never use With...End With together with extension method as you may get unexpected results. As for the solution, well, I'll leave it up to whoever designs the .NET framework...

UPDATE (17 June 2010): The issue was submitted to Microsoft Connect here. They acknowledge the issue, yet decided to do nothing, not even adding a compilation warning.

Error accessing VC80.idb during compilation of VC++ project

2010-06-03T04:13:00.000-07:00

If you're getting the following error

Could not delete file ‘c:\project\obj\release\vc80.idb
Make sure that the file is not open by another process and is not write-protected.

while building large VS solution with many VC++ projects, in a random and inconsitent manner, most likely you have just hit another Visual Studio bug. VS, by default, attempts to build multiple projects concurrently on system with multiple processors, which may cause the above error if different projects share the same IDB (VC++ Minimum Rebuild Dependency) file.

There are several solutions for this, but the simplest one would be to tell VS not to build multiple projects concurrently. To do this, open Tools>Options>Projects and Solutions>Build and Run, and set the value of Maximum number of parallel project builds to 1. However, with this method, your solution will take longer to build.

Full RSS feed for blogger posts/comments

2010-05-31T06:01:00.000-07:00

The official RSS feeds given by Google (http://www.google.com/support/blogger/bin/answer.py?hl=en&answer=97933) only retrieve the latest 50 posts and 20 comments, even if the settings under Site Feed has been set to Full. This default behaviour is useful as readers are usually only interested in latest posts, but will be a problem if you somehow want the RSS feed to show your entire blog.

The solution is simple, just add a HTTP GET parameter to the official RSS URL. For example, if your RSS feed is:

http://blogname.blogspot.com/feeds/posts/default?alt=rss

The feed that returns the full blog is:

http://blogname.blogspot.com/feeds/posts/default?alt=rss&max-results=N

Same for comments, add the max-results parameter to retrieve all comments:

http://blogname.blogspot.com/feeds/comments/default?alt=rss&max-results=N

where N is the maximum number of posts/comments you want to retrieve.

Identifying used and unused resources in a Visual Studio's project Resources.resx file

2010-05-02T20:22:00.000-07:00

If you often use project resources in a Visual Studio project, be it VB or C#, eventually you will end up with a big resource file (e.g. Resources) containing many unused items. This is because, when you remove the code that uses the resoure, very often you will forget to remove the actual resource item.

There are solutions to this problems such as using commercial code refactoring tool or making some minor modifications to Resources.Designer.vb/Resources.Designer.cs and relying on the compiler to generate warnings about unused resources. In this post, I choose to take a different approach: use a batch script.

My complete batch script will accept 4 parameters from the command line, listed from left to right:

1. Path to the VB.NET source code files, e.g. WindowsApplication1\*.vb
2. Name of project resource file, e.g. WindowsApplication1\My Project\Resources.resx
3. Name of file where all resources' usage are written, e.g. all_resources.txt
4. Name of file where all unused resources are written, e.g. unused_resources.txt

It makes use of the FOR extended syntax (FOR /F) to parse the resource file and FINDSTR to find where all the resources are referred to. It will only work properly with VB source code files, where most developers often use My.Resources.XXXX to access the resources. As FINDSTR simply performs a string search, the batch script will not care about code that are commented out, as well as resources that are accessed not by using the Vb's My namespace. If you want to use this with C# source code, you'll need to edit the call to FINDSTR to match your method of accessing the resources, e.g. Project1.Resources.Resource1 instead of My.Resources.Resource1.

The rest of this post will analyse some interesting points I encountered when writing it - hope this will be useful for those having similiar problems.

Batch script - powerful despite being crude

Many people may think that batch script is a thing of the past, since there are so many alternatives today - Windows PowerShell, Perl, or a .VBS file. Yet sometimes I still use it due to its powerful but simple nature. Take a look at the code to search for the resource:

FOR /F "tokens=*" %%a IN (%TMPFILE%) DO (

.........
        FINDSTR /S /P /N /C:"My.Resources.%%c" %SRCPATH% >> %OUT_ALLTOKENS%                   
        REM FindStr returns an errorlevel of 0 if the string was found, 1 and above if it was not found.
        IF ERRORLEVEL 1 (

.........
        )

.............
)
)

The call to FOR /F and FINDSTR can easily translate into 10-20 lines of .NET code, unless you use some existing text processing libraries. Here, almost everything is done, just simply remember the syntax, available via FOR /? or FINDSTR /?.

Another example is how to generate a random file name:

:GETTEMPNAME
set TMPFILE=%TMP%\mytempfile-%RANDOM%-%TIME:~6,5%.tmp
if exist "%TMPFILE%" GOTO :GETTEMPNAME

The equivalent .NET code would be:

string filename = Path.GetTempFileName()

However, batch scripts are still very crude. For example, although Windows XP batch scripts allow the use of block statement via (........), don't expect everything to work as they do in a modern programming language. The following would cause problem:

IF (%1) == () (
    ECHO Missing first parameter: path to the VB.NET source code files (e.g. WindowsApplication1\*.vb)
    EXIT /B 1
) ELSE (
    SET SRCPATH=%1
)

Execution would terminate shortly upon reaching ECHO, without any indication why. I believe it would take many people some time to figure out what causes it and fix it. The following will work:

IF (%1) == () (
    ECHO Missing first parameter: path to the VB.NET source code files, e.g. WindowsApplication1\*.vb     

    EXIT /B 1
) ELSE (
    SET SRCPATH=%1
)

Also the above example demonstrates how to check for missing parameter. This would have made more sense in C# or VB.NET, just compare against an empty string (e.g. "").

There are many other useful code snippets in this batch script, for example, how to use delayed environment variable expansion, or how to use double quotes as delimiters in FOR /F. I will not explain it in details - the code is well commented, take some time to study it yourself.

1440x900 resolution on Intel 945GM Express Chipset

2010-04-30T21:41:00.000-07:00

My laptop is using a Intel 945GM Express Chipset graphics card. Recently I purchased a 19" LCD with 1440x900 naive resolution. Believe it or not, it was very difficult to set the correct resolution for the LCD as my graphics card is quite old and does not officially support 16:10 resolution. The closest it supports is 1440x1050, causing distortion on the LCD.

I solved the problem by using PowerStrip. With some simple configuration as per this, I am able to add a new custom resolution and get it to work:

However, with this I encountered the next challenge. Media Player Classic, which is my favourite media player, is unable to play video on my LCD monitor when configured as the secondary monitor. I always got the error message

"Due to limitations of your computer's display device, this video can only play back on one of your monitors. Please move the video window completely onto that monitor"

whenever I attempt to play videos on the LCD.

After some research, I found out that the problem has to do with hardware overlay (see this). Media Play Classic seems to attempt hardware overlay on both the primary and secondary monitors, which, unfortunately, is not within the capabilities of the graphics card.

I decided to change to VLC media player. However, VLC, by default, uses software rendering and does not play HD movies smoothly. When configure to playback using hardware overlay, VLC encounters the same problem as Media Player Classic. Even worse, there is no clear error message - the application simply crashes or plays videos with huge delay/distortion. Windows Media Player works for some simple movies (.avi, .wmv, ...) but fails to play many other formats.

Finally, some googling on the best free media player leads me to BS Player. I switch to it and everything seems good so far. Sometimes, the player would crash upon closing but that does not seem to be a big problem for me.

Native SIP on Windows Mobile 6.1 and 6.5

2010-04-27T21:02:00.000-07:00

Although Windows Mobile 6.1 and above supports SIP natively, the user does not always have the ability to configure SIP settings. In particular, SIP (a.k.a Internet Call) needs to be enabled in the device firmware, and SIP server configuration needs to be set via registry. This post summarizes the step needed to enable SIP and make Internet Calls from the phone's native dialer.

First, you must check if your firmware enables Internet Call. There should be an Internet Call item on your Today Screen:

If you are not using the standard Today screen plugin, check your Phone settings, there should be a tab for Internet Call:

Your operator may have bundled the SIP configuration settings with the phone. If so, you're good to go, just select either "Whenever Available" or "Only if cellular is not available" from the drop down. The phone will attempt SIP registration whenever a connection is available and you'll see "Internet Call: Available" on the Today screen when SIP is registered. If SIP is not registered, you'll see "Internet Call: No Service".

If your phone does not have bundled SIP configuration settings, you will need to configure it yourself. You can do this manually, but I'd recommend using a tool called Sip Config Tool:

Input your SIP server configuration and choose to save. This will overwrite any existing configuration. By default, SIP over a cellular connection is disabled, but you can enable it via the menu if you want to.

You may need some configuration to make sure that the number you dial from the dialpad gets translated into a proper SIP address. Check \windows\ipdialplan.xml and you'll see something like this:

    <!-- International -->
    <rule pattern='(\d+)'
        dial='sip:\1@$host$'
        display='\1'
        transfer='sip:\1@$host$'
        />

pattern: the number you dial in the dialpad
dial: the actual SIP address where INVITE request is sent to
display: the number to be displayed during the call

The default configuration is only suitable for US number so you may need to change them as appropriate according to your country's telephone nunbering plan. In my case, I remove everything except one entry in the International section. To dial a number, I simple enter the full format, country code + area code + telephone number.

If your device does not support playing audio on the earpiece via RIL_SetAudioDevices, don't expect Internet calls to work with your earpiece. In my case (Acer F900), audio is played via loudspeaker unless a headset is connected, with a lower volume to simulate the earpiece. At the beginning of every call, a beep tone will be played to indicate an Internet call.

"csc.exe - Application Error" when shutting Windows with a .running NET application

2010-04-15T20:33:00.000-07:00

Recently a friend reported to me that my application written in .NET prevents his computer from shutting down. If the application is running when Windows tries to shut down, the following error message appears and Windows cannot shut down:

csc.exe - Application Error

The application failted to initialize properly (0xc0000142). Click on OK to terminate the application.

Some investigation reveals that the issue has to do with the .NET class XmlSerializer. My application stores user data into an XML file, which gets saved when the application is closed. In particular, the following code seems to cause the problem:

Dim serializer As New XmlSerializer(ConfigObj.GetType())
Dim writer As New StreamWriter(datafile)
serializer.Serialize(writer, ConfigObj)
writer.Close()

The error occurs as soon as XmlSerializer.Serialize() is called. A file system monitor tool such as Process Monitor shows me that csc.exe is creating temporary files in C:\Windows\Temp, which is perhaps interrupted by the shutdown process, causing the above problem. I am not even sure why csc.exe (the microsoft C# compiler) gets called even though I am using VB.NET!

Not sure how to fix the problem, I have to avoid calling XmlSerializer when Windows is shutting down by using:

Private Sub Form1_FormClosing(ByVal sender As Object, ByVal e As System.Windows.Forms.FormClosingEventArgs) Handles Me.FormClosing
...

If e.CloseReason <> CloseReason.WindowsShutDown

    Dim serializer As New XmlSerializer(ConfigObj.GetType())
   Dim writer As New StreamWriter(datafile)
   serializer.Serialize(writer, ConfigObj)
   writer.Close()

End If
...

End Sub

This way, user data may be lost, but at least my application does not prevent the system from shutting down. I have to find another way to save user data more frequently...

Reference:

Problem with CSC.EXE

MobileOne not M1

2010-03-30T07:50:00.000-07:00

This entry seems a bit "off", considering the technical nature of the blog. But nevertheless I decided to post it here. Enjoy :)

For those not familiar with the Singapore telecommunication industry, there are 3 telcos in Singapore, namely Singtel, Starhub and MobileOne. The last is also known as M1 by most people.

A friend of mine who is a fan of Google Products (GMail, Google Docs, Google Talk, Google Calendar, etc) recently told me that he changed his phone number. I asked him why, and he told me he needs to change from M1 to Starhub. "Why do you need to? All telcos in Singapore have good call quality, well, at least, for most of the time." I said. He then told me he could not get Google Calendar SMS notifications to work with his previous M1 SIM card. I said "Why not? It should be straight forward, just enable it in your Google calendar settings and enter your phone number." He replied "I have tried everything, and concluded that Google does not support my provider, that's why I need to change. And everything works fine now."

I was surprised as I used this feature with M1 SIM card before. So I checked http://www.google.com/support/calendar/bin/answer.py?answer=37226#S and of course M1 is listed in the supported provider list. I gave him the link, to which he said "Well I checked it before, it shows "MobileOne" but I am using M1."

It turned out that he only enabled Phone Notifications under his Google account settings after he purchased the new Starhub SIM card, and simply concluded that he was right, since it works with Starhub, and not M1.

FYI, he's planning to do an MBA in US, and needless to say, I am surprised that somebody who wants to do an MBA would rather go through the trouble of switching providers and changing phone numbers, instead of spending a few minutes checking if M1 and MobileOne are somehow related. I doubt he would ever be able to pass his MBA course with that kind of mindset...

See also:

Transferring SMS messages from Windows Mobile to Blackberry devices

2010-03-15T05:50:00.000-07:00

UPDATE (8 October 2010): For a more generic tool to restore SMS to a Blackberry device from a CSV backup, refer to this.

This may seem an easy task - one may immediately think of using the Device Switch Wizard in Blackberry Desktop Manager or do a quick search for some SMS backup/restore software which will do the job. However, while I was doing it, I came across some challenges which turns this into a difficult job:

1. The Device Switch Wizard found in the Blackberry Desktop Manager simply does not migrate the SMS messages.

2. There are plenty of applications that perform SMS backup/restore for Windows Mobile but there are none for Blackberry.

The first challenge is expected as many users simply do not need to migrate text messages to their new phones and it is therefore understandable that the Device Switch Wizard ignores SMS during the migration process but the second challenge disappoints me. I was realistic - I never expected to find a tool that transfers text messages on-the-fly, perhaps with the tool running on Windows, the Blackberry connected via Blackberry Desktop Manager and the Windows Mobile device connected via ActiveSync. BUT I was hoping to find some software that can restore Blackberry SMS from a backup, perhaps XML-formatted, which can be created by a tool running on Windows Mobile.

I was wrong - there are no such applications for Blackberry. This is explained by the fact that RIM does not provide an API for Blackberry applications to programmatically access the SMS messages store (there are, however, APIs to access email messages).

My approach

My ideas come from the fact that Blackberry Desktop Manager allows user to backup device memory (including contacts, calendar and SMS) into an IPD file, and restore from the IPD backup. Although the IPD file format is not documented by RIM, it is easy to reverse engineer (for example, by using a hex-comparison tool) due to its unencrypted and tokenized nature. There are previous attempts to do this, for example ipddump and MagicBerry. The latter is extremely useful as it allows you to view, edit, merge and split IPD files.

Keeping the above in mind, I came up with a tool that runs on Windows Mobile device and backs up all text messages into an IPD file. The file created can later be opened using Blackberry Desktop Manager and the text messages can be restored back onto the Blackberry device.

Final product: SMS2IPD

In the lack of a better name, let me call it SMS2IPD, short for SMS to IPD Converter. Here's how to use it:

1. Run the tool on your Windows Mobile device. You will need .NET Compact Framework 3.5 and above.

2. Select the correct time offset so that SMS timestamps will be correctly displayed. For example, if you are in GMT+8, select -8 for the time offset. The selection is needed as I am not sure which field to set to specify the timezone of the SMS messages.

3. Click on Start, choose the location to save the IPD file. Give the application some time to run, when completed, it will produce the IPD file.

4. Connect your Blackberry and start Desktop Manager, choose Backup/Restore, then click on Restore again. Select the IPD file created in step 1 and Desktop Manager will proceed to transfer all SMS to the device.

Note: this process may take a long time during which Desktop Manager may seem to hang. It will also overwrite all existing text messages on the Blackberry device, so back them up and merge them to the IPD first using MagicBerry. MagicBerry may also seem to hang when loading a large IPD file, so be patient.

Limitations and future improvements

I myself reverse engineer the IPD file format by using a hex editor, a hex comparison tool and MagicBerry, with some references to ipddump and MagicBerry source code, so there will be cases not handled properly. These include SMS with special characters, unicode SMS, and SMS sent from a timezone different from your device timezone.

Upon imported on the Blackberry device, all messages may appear as unread. You can mark all as read by scrolling to the top of the messages list (which displays the date), press the Blackberry key and choose Mark Prior Opened.

Some Blackberry devices by default will delete text messages older than 30 days. This behaviour can be changed by opening Options in Messages, choose General Options, and select a longer period for "Keep Messages".

I attached the source code for those interested to improve the application. You can modify and re-distribute the application and its source code as you like, but please mention that the source code was taken from my blog. You may not, however, use the source code and/or the application for any commercial purposes.

Credits

To read the SMS messages from a Windows Mobile device, I am using MAPIDotNet library found on codeproject, written by rwt33. The version I used was downloaded some time back, so it may differ significantly from the current version found in the project article.

Downloads

Source code:
http://www.mediafire.com/?ezj0eaa9kj5vqzk

SMS2IPD CAB Installer for Pocket PC and Smartphone, Windows Mobile 5 and above with .NET Compact 3.5 (updated 10 Oct 2010)
http://dl.dropbox.com/u/5881137/SMS2IPD_CAB.CAB

Windows Virtual PC vs. Microsoft Virtual PC 2007

2010-02-23T18:51:00.000-08:00

In an attempt to run Windows XP applications on Windows 7 via Windows XP Mode, I recently tried out Windows Virtual PC. However, while Windows Virtual PC is the next version of Microsoft Virtual PC and has some remarkably new features, the software as a whole still disappoints me. Here's why.

Improvements in Windows Virtual PC

1. Seamless integration with Windows Explorer. User can now browse through the list of virtual machines using an Explorer-like interface, instead of the ugly-looking window that can only support a maximum of 3 virtual machines (without scrolling) in the old Microsoft Virtual PC.

2. Support USB devices. USB devices plugged into the host machine can be detected by the guest operating system. This feature has long been present on other emulators (VMware, Sun Virtual Box, etc.) but is not present in Microsoft Virtual PC 2007.

What make it worse despite the improvements...

1. Windows Virtual PC requires hardware virtualization and will not run without it, even though it does not always use this feature. In other words, many older computers that can run Windows 7 smoothly will not be able to run Windows Virtual PC. Since Microsoft Virtual PC refuses to install on Windows 7, owners of these computers will have no choice but to use other emulator software, unless they plan to downgrade to Windows XP.

2. While I may not have used Windows Virtual PC extensively, the newly so-called Integration Features seems to be worse than its predecessor, Virtual Machine Additions:

Limited file sharing capabilites: Integration features only support sharing of entire hard disk disk; user cannot share a specific folder as they could in Microsoft Virtual PC 2007.
Integration Features is implemented as a remote desktop session into the Virtual Machine. This will make it unusable in many other older guest OS'es such as Windows 98 and will also reduce the speed.

This explains why the sharing settings look similiar to that of the Remote Desktop client that comes with every version of Windows - maybe Microsoft is just too lazy to re-write the code in a different way...

The older Virtual Machine Additions does not need a Remote Desktop session to communicate with the virtual machine; it instead uses some special communication mechanism, see this for more information.

3. No official support for OSes ealier than Windows XP.

In particular, Integration Features will not work with ealier OSes such as Windows 98 or DOS. Linux is out of the questions, too. In Microsoft Virtual PC 2007, even though DOS is not supported, one could still enable file and folder sharing by using the virtual machine addition floppy image that comes with Microsoft Virtual PC 2004.

4. Floppy Disk and parallel port emulations are removed.

While I seldom use the parallel port emulation and I doubt anyone is still using it frequently, unless they need to print to a parallel port printer or interface some parallel port peripherals from a virtual machine, the floppy disk emulation should not have been removed as it is still needed for some older operating system such as DOS.

5. Finally, it's overall slower than Microsoft Virtual PC and other emulators, when running on the same system configuration.

For example, using the same VHD image that comes with Windows XP mode, Sun VirtualBox starts the virtual machine within 5 seconds, while Windows Virtual PC spends 15 seconds staying at the "Starting the virtual machine..." screen.

Enough said... I have uninstalled Windows Virtual PC from my Windows 7 machine and use Sun VirtualBox or VMware for emulation purposes. If there is a need, I will just boot to Windows XP and use Microsoft Virtual PC 2007.

.NET forms on Windows 7 at 125% zoom level

2010-02-04T07:19:00.000-08:00

If you have ever tried to personalize your Windows 7 desktop and chose to zoom to 125%, as per the following screenshot, you'll perhaps notice that some applications do not display so well:

Believe it or not, the 125% or 150% zoom mode does not simply decrease the dpi of the display and make it look bigger, unlike the advanced display settings in Windows XP:

In fact, when the 125% or 150% zoom mode is set, Windows 7 does (at least) the following two things:

Increase the default font size
Tell every window (including child window) to increase its size

It's up to the application to handle the change in font size and window size gracefully. If it does not, the application interface may look distorted, with text truncated or displayed outside its designated area and graphics being tiled up or stretched.

Interestingly the 125% or 150% zoom mode cannot be used during a Remote Desktop session.

Effects on .NET forms

Under 125% or 150% zoom, the size of a .NET form will increase proportionally. The following changes will have to be made for the form to display properly:

All form controls should have their Anchor property set appropriately.
Most importantly, the AutoScaleMode property of every form should be set to Dpi and not Font. AutoScaleMode of user controls can be set to either Dpi or Inherit.

If set inappropriately, only the form will resize while the controls do not resize, causing display distortion.

Also, do not open the form designer from inside Visual Studio 2008 under 125% or 150% zoom mode. Otherwise, Visual Studio will "zoom" the form (perhaps in an attempt to obey the Windows settings faithfully) by increasing its Size property and forgets (or is unable) to restore the original size when the zoom mode is set to 100%. If this happens, you'll have to manually resize every form back to their original size.

I am not sure if it's a Visual Studio bug, or it's just supposed to be that way.

ArgumentOutOfRangeException when calling DateTime.ToString()

2010-01-17T01:17:00.000-08:00

Recently I encountered a strange problem with one of my .NET applications when running on phones having the language set to Chinese. If started by clicking the executable file, the program would just crash without warning at startup. If started via Visual Studio debugger, the debugging session will be terminated shortly with the error message "The remote connection to the device has been lost" and the program crashes with no further error messages.

For the device to show a more detailed error message, I uninstalled the .NET Compact Framework version 2.0 that comes bundled with the device and let Visual Studio automatically install the framework before debugging in started. With this change, the exact error message pops up, showing an ArgumentOutOfRangeException when the program tries to call DateTime.ToString(). With some further investigation, I realized this seems to be an issue with the .NET Compact Framework when running on phones where the user has changed the default language in Regional Settings to Chinese. In all my test devices, the problem seems to only happen on HTC phones with the language set to Chinese (Singapore) and strangely does not occur when a different Chinese dialect (e.g. Chinese (Taiwan)) is used.

To use Chinese (Singapore) as the phone's default localization, a workaround is to open Region Settings, in the Date tab, under Calendar Type, select another type of calendar such as 西历, instead of Gregorian Calendar.

The problem may have to do with the fact that HTC phones allow user to switch between 西历 and Gregorian Calendar. On other phones that do not have this problem the option Calendar Type is grayed out, default to 西历, and thus cannot be changed if the localization is set to Chinese.

Interestingly, when this problem happens, a Try/Catch around the related code section does not always prevent the crash. If it does, further execution of the program may have erratic behaviour such as throwing mysterious ArgumentOutOfRangeException or returning wrong results when calculating date/time values.

Upgrading to .NET Compact 3.5 does not fix the problem.

Just because you call it 'preprocessor directives', doesn't mean it is.

2010-01-08T22:03:00.000-08:00

A few months ago, one of my friends, a soon-to-be team leader in a software company, had a chance to assist me in debugging the C++ source code of a Windows application. The application could be configured (via a #define in a C++ header), to compile in either HOME_NETWORK or PUBLIC_IP mode. The first mode was used when we test it within our LAN, whereas the second one was needed to access the application remotely. Although what the application actually did is probably of little interest, it helps to know that my friend has a 3-year J2ME experience.

During the debugging process we needed to change some of the hard-coded IP addresses inside the application when running under HOME_NETWORK mode. Too busy with something else, I asked him to help me make the changes. I would assume that this would be an easy task for him without the need for any C++ experience. "Just edit the code, change the IP address, and re-compile", I told him.

Ten minutes later, he reported the job done. I tested it, and everything seemed to work fine, so I said "Thank you.". He then replied, "You're welcome. I have not touched C++ for a long time. You actually help me to revive my C++ knowledge. I also help you in commenting out old codes."

I did hear him fully, and actually questioned why he mentioned "commenting out". But as I was still busy debugging, I soon forgot about it and moved on to something else.

Three days later when I changed the compilation mode from HOME_NETWORK to PUBLIC_IP, the application suddenly stopped working. I started to check every single thing, including Windows firewall, antivirus prorgram, DNS, .... but could not find the reason why. Finally, suspecting that some typos must have been made while changing the IP addresses a few days back, I decide to check the source code. As soon as I re-compiled the source code, I noticed hundreds of warnings about unused variables and methods. Taking a further look, I could not believe that my source code had been changed to:

#ifdef HOME_NETWORK 

    /* Perform preparation while running in our LAN */

    PerformHomeNetworkCheck();
#else 

    /* Perform preparation while running in public IP mode */
    /* 

    PerformPublicIPCheck();

    .....

*/

#endif

In every single #ifdef block that checked if either HOME_NETWORK or PUBLIC_IP is defined, the section for PUBLIC_IP mode was commented out! This reminded of my friend mentioning "commenting out", so I called him to ask why. He told me "Well, those code sections were inactive but not commented out, so I thought your IDE must have forgotten to do that, and I helped you. My IDE, with Antenna, does a better job." Not having a single word to say, I hung up.

It turned out that he has always been using Antenna for pre-processing in J2ME, as Java does not have native pre-processing ability. In order to hide the inactive pre-processor block from the compiler, Antenna comments it out. My friend, a 3-year J2ME developer, was too familiar with this behaviour to assume the same thing for C++ preprocessor and commented out almost every single inactive pre-processor block in my C++ code, ignoring those where commenting out would cause compilation errors!

This reminds me of a few years ago when I saw a group of children in a supermarket, pointing at the frozen chicken (labelled 'CHICKEN') asking their parents "Why aren't those chicken clucking?" Those children had never seen a chicken alive before and thought what they saw was a real chicken!

But at least the children knew how to ask question. My friend did not even ask! I wanted to say "Well, thank you, maybe your boss should re-consider your promotion to be a team-leader". But, I'm polite.

Built-in card reader on Acer Aspire 5550

2009-12-26T13:18:00.000-08:00

In case you haven't known it, the built-in 5-in-1 card reader on the Acer Aspire 5550 can only support SD but not SDHC (high-capacity) cards. Although this wikipedia article shows that normal SD card can be as large as 4GB, most of my 4GB SD cards are high-capacity, which can't be read by the built-in card reader. The same applies for the free USB SD card reader that comes with unit - both can only support SD cards up to 2GB. For the built-in card reader, nothing will happen when you insert an unsupported card. For the USB card reader, attempting to use an unsupported card will result in a continuously flashing red light. On the other hand, when the USB reader does not detect any card, the indicator will flash for 3 seconds, pause for 1 second and flash again.

Using USSD on Windows Mobile to set call forwarding

2009-12-16T16:19:00.000-08:00

In one of my recent projects, I need to set the call forwarding programmatically. My first attempt was to use the RIL_AddCallForwarding and RIL_GetCallForwardingSettings APIs provided by RIL. However, on all phones I tested, these API mysteriously failed with either E_FAIL, E_NOTIMPLEMENTED or E_INVALIDARG. After a few attempts, I gave up and concluded that these APIs are supported only on Windows CE, but not Windows Mobile (or at least MSDN says so).

My next attempt is to send USSD via lineSendUSSD() method. For example, to divert all incoming calls to another number, use **21*«dest number»#. Since the USSD codes to set call forwarding status are fixed, this methos should work for all phones (see this for the exact USSD codes). However, after some testing, this method is not as reliable as it sounds. It works perfectly on some phones. On some others the call to lineSendUSSD() returns E_FAIL when I passed in the code to set call forwarding but the same API works fine for 'normal' USSD code (e.g. *100# to check account info). On other phones, I received a response from the mobile operator saying that the service code is invalid - it seems as if the phone literally sends the USSD string to the mobile operator, instead of processing it and gives me the call forwarding status.

On a side note, using the .NET method Phone.Talk() on any USSD string will simply display an error message, saying that the signal is unavailable. Also, lineSendUSSD() are not supported on all phones. On phones where it is supported, not all USSD codes are accepted and the response cannot always be retrieved easily. It will most likely fail if the mobile operator returns a menu response for your USSD code (e.g. Reply with '1' for account balance, '2' for expiry date), in which case I haven't figured out how to reply programmatically, or even to retrieve the response text. It looks as if it has to do with the registry branch HKEY_LOCAL_MACHINE\Services\USSD\.

At the moment, I have no ideas why Microsoft makes it so difficult for developers just to send a USSD code.

Useful country information database

2009-11-27T23:51:00.000-08:00

If you need a database of useful information on different countries in the world, e.g. country name, population, capital city, area, country code, timezone, etc, you can download from http://download.geonames.org/export/dump.

Each country's information is provided as a text file compressed into ZIP format. The text file is tab-separated and can be easily converted into some spreadsheet or database format such as Excel, Access or SQL.