Why is HotSync slow?

Why does it take so long to HotSync my database?

There's some per-record overhead in the synch protocol and in the Data Manager on Palm OS, so that a 100 KB database with 10 records will synch much faster than a 100 KB database with 100 records. The characteristics have changed from OS version to OS version, particularly in releases 3.5 and 4.0.

To illustrate this, Warren Young did some benchmarking on his Visor Deluxe with the USB cradle. The test database (see below) is a 113 KB Think DB 2 database with 2165 records. He did two main tests: regular HotSyncing, and ZBoxZ HotSyncing. ZBoxZ restructures a database so that it has fewer but larger records, so that it syncs faster.

These times are all approximate, since he used a PC-based stopwatch to do the timing.

Test # Operation Time
1 Delete the database from the Visor 7 sec
2 Synch the database back to the Visor 4 min, 5 sec
3 Back up the newly-synched database with BackupBuddy 40 sec
4 Synch a ZboxZ'd version of the database (still 113 KB) to the Visor 4 seconds
5 Un-box the database on the Visor 2 min 5 sec

He then re-did the tests on his Linux box, which has a serial cradle running at 57,600 bps. Everything else (the Visor, the database, etc.) remained the same.

Test # Operation Time
1 Delete the database from the Visor same
2 Reinstall the database to the Visor with 'pilot-xfer -i' 4 min, 29 sec
3 Fetch the newly-synched database with 'pilot-xfer -f' 1 min 10 sec
4 Install a ZboxZ'd version of the database (still 113 KB) to the Visor 27 seconds
5 Un-box the database on the Visor same

Test 4 shows that for bulk data transfers, the USB cradle is much faster than the serial one. But, test 2 shows that this speed is almost moot when the number of records in the database becomes large. Test 5 partially confirms this: installing 2,165 records takes about half of the total synch time for the plain-old database. The remaining 2 minutes were presumed to be in HotSync protocol overhead, possibly due to handshaking on each record. A decent amount of the time was also taken on the device, since early versions of the Data Manager were not optimized for allocating lots of small records, and thus slowed down.

This test needs to be repeated using different versions of Palm OS: 3.1, 3.5, 4.0 and 5.0 would be an ideal set to measure.

Today: Sep 8, 2010