Analysis of the iPhone Location Log

Yesterday, security researchers announced that iPhones and 3G iPads maintain a log of the phone’s position over time. Today, another researcher released a python program to dump the equivalent log from Android devices. While it has been common knowledge that cellular phone carriers store customer location data for a while now, it comes as a surprise to learn that client devices are carrying their own logs as well. These logs are included in backups on the user’s computer and rogue applications could easily steal the data if left unencrypted. Even more troublesome are reports that Michigan State Police are using a device from a company named Cellebrite to extract data from people’s cellular phones during routine traffic stops.

So what data, exactly, is contained in the consolidated.db file? And just how accurate is the positioning data contained within? After unencrypting my iPhone backups in iTunes, I used the information in Pete Warden’s documentation on GitHub to pull the information into a spreadsheet. I had 12,380 records in total starting on 9/3/2010. That’s the day I purchased my iPhone 4. On average, that’s 61 location entries per day! After spending some time with the data, here are some preliminary observations:

1. The log appears to be limited to cell phone tower data, not GPS data. The values in the Horizontal Accuracy column never fall below 500 (meters) which suggests this is approximate location data from cell towers. Moreover, the VerticalAccuracy and Speed columns are always -1, which is what the Significant Location Change service reports when GPS is not in use.

2. The MCC and MNC columns represent some combination of carrier and/or country code. These columns have values of 310 and 410, respectively, whenever I was in the United States. This includes trips to both coasts, so I’m fairly certain these values apply to AT&T anywhere in the country. In Europe, these columns take on different values for each country. It appears that the MCC column might be a country code and the MNC column is a carrier code.

3. Timestamps are only updated after a significant location change. Surprsingly, the same timestamp value is shared across many location entries. This behavior seems rather strange, as it is trivial to grab the current system time when writing locations to the database. Apparently the column was intended to represent the phone’s arrival in a certain area. Generally, travelling a distance of 5 kilometers is enough to trigger a new timestamp but this is not guaranteed. I have many timestamp updates at less than 1kmĀ traveledĀ and quite a few updates of over 5km without a timestamp change. It appears that driving somewhere quickly (i.e. using the freeway) is more likely to trigger a timestamp change than by slowly moving to a new location (i.e. on foot). This is not a reliable metric by any means, as I had some blocks of entries with the same timestamp span upwards of 20 miles within a city.

4. A stationary phone can generate many log entries. Building on the idea that timestamps only change when the phone is moved a significant distance, I started mapping blocks of entries with the same timestamp value. It turns out the phone logs quite a few location entries while the phone is technically stationary, even in buildings with great reception. This is not surprising, as received signal strength from nearby towers will vary significantly as I walk through a building. The following image shows location entries while I was in my apartment, which has excellent reception: (Note: I translated all of the values by the same offset for some anonymity)

These consolidated.db entries were logged over the course of 11 hours in one building with great reception.

5. Averaging entries with the same timestamp comes remarkably close to the phone’s actual location, but only if reception is good and cellular towers are dense. Averaging a random sample of consolidate.db entries with late-night timestamps yielded coordinates only 690 meters from the actual location of my apartment. The average of a block of entires on a Saturday gave me coordinates only 630 m from my girlfriend’s house. Both of these locations have great reception and are moderately-dense suburban neighbors. On the other hand, averaging entries during the week put me a full 8.1 km from the actual location of my office. We have terrible reception in my office, so this is not too surprising.

The average of these points with the same timestamp is within 650 meters of my actual location

6. 50% of location changes occur after 1 km of [perceived] movement. 33% occur after 500 m of movement. A full 90% occur within 5km. These distances are calculated from the estimated locations in consolidated.db. It would be interesting to log actual GPS location in parallel and compare the two values.

Relative frequency of the distance between updates

7. Locations with good reception have far more entries than locations with poor reception. I expected the opposite to be true, as buildings with poor reception tend to have higher received signal strength fluctuations and therefore trigger more perceived location changes. Instead, the phone infrequently logs locations whenever reception is poor. This could be a side effect of some built-in mechanism to only log locations when the phone is relatively stationary and receiving a stable signal. As a result, it’s very difficult to pin down the phone’s location if reception is poor.

8. Older entries are less frequent. This one is just a rough observation. It’s entirely possible that I’ve been more mobile lately than when I first bought my iPhone. However, my log has far more entries with timestamps in the past several months than it does from the beginning of the consolidated.db log. This could be due to changes in the iOS software to log more often. It could also indicate that the iOS software prunes older entries over time to keep consolidated.db from growing too large, but I’m not so sure this is the case.

So what does this mean for iPhone users? Not a whole lot, as long as you remember to encrypt your iPhone backup in iTunes to make it a bit more difficult for rogue applications to access. Still, I’m not sure why iOS and Android OS find it necessary to keep location logs. The only logical explanation I’ve heard so far is that it might help speed up future geolocation queries. It will be interesting to see how Apple and Google respond to this, if at all.

iPhone Background Location Service

Location Updates After 2 Days in San Diego

Apple’s iOS operating system famously doesn’t allow applications to run in the background. It does, however, allow them to perform seven narrowly-defined background services such as audio streaming and receiving ‘significant location change’ updates. The significant change updates don’t use the iPhone’s highly accurate but power-hungry GPS in the interest of preserving battery life. Instead, the phone uses the relative signal strengths from nearby cellular towers to triangulate a rough approximation of the iPhone’s location. The phone’s baseband must continuously communicate with the carrier’s cellular towers when your phone is turned on anyway, so gathering location data from these communications should have a negligible effect on battery life.

So just how accurate are these triangulated data points, and what exactly constitutes a ‘significant’ location change to the iPhone’s iOS? The only information I could find comes from a developer who tested phones in Tokyo, Chicago, and Melbourne. He concluded that 2500m (approximately 1.5 miles) as measured by the phone’s own background location service, was enough to trigger location updates 95% of the time. Moving 750m triggered an update 50% of the time. Good information, but it doesn’t tell us anything about the accuracy of the background updates.

The problem seemed interesting enough, so I spent a Saturday morning putting together a small test suite. I created a simple iPhone app that registered with the iOS significant location change service and logged all updates in an SQLite database and CSV file. I also built a Ruby on Rails app to receive real-time location updates from the phone and pushed it to Heroku for some quick (and free) hosting. Finally, I made a point to record every time I got into or out of my car in my notes for a week so I could compare the background location results to my actual whereabouts.

Some observations:

1) False location change events occur randomly when the signal is weak – At my apartment where the cellular signal is strong, the background location service is generally within 1km of my actual location. When I’m at my office building, where signal strength is so bad we struggle to complete phone calls indoors, the phone will randomly register location events up to 10km away. Usually these events are triggered as I walk from one side of the building to another, but occasionally they will occur while I’m seated at my desk.

2) It doesn’t use nearby WiFi networks for location – The above problem could be mitigated somewhat if the OS would use nearby WiFi networks to aid in location. After all, the iPhone already queries a massive database of WiFi router locations to assist with GPS lookups, and I’ve used my GPS at home and my office enough for the phone to associate my WiFi router’s BSSID with a physical location. On the other hand, scanning for WiFi networks would provide some additional battery drain. Dealing with mobile hotspots like the MiFi would be problematic for a WiFi-assisted location scheme as well.

3) Location updates occur at regular intervals while moving, but not always – For my first test of my location tracking app, I drove 13 miles across town to a coffee shop. Upon arrival, there were exactly two pins on my map: my apartment and the coffee shop across town. I assumed the background location tracking waited until I stopped moving and my location stabilized to register a location change event. Clever. Then, on the drive home, my app diligently registered location events every few kilometers along the freeway until I was home. So much for that theory. Since then, I’ve received updates while moving more often than not, but very rarely I won’t see any location update events until I arrive at my location.

In the screenshot to the left, I ended up driving all of the visible freeway segments (yellow lines) over the course of two days. For the upper (I-80) and rightmost (I-215) segments, the application received location change updates at roughly the same location each time. The segment on the left (I-15) did not register any location events in this snapshot, but on other days driving the same I-15 segment would register location change events every 2-4 km.

The good news is that I always seemed to register a location change event when I arrived at my destination and parked the car. In some cases, these location change events were 15km apart. This only seems to happen when traffic isn’t present, allowing me to maintain a constant 65mph on the freeway.

iPhone Navigation Active Indicator4) If the application is terminated, background location services will restart it automatically – Once the app calls [[CLLocationManager sharedLocationManager] startMonitoringSignificantLocationChanges] the familiar location ‘compass’ arrow shows up in the taskbar, even though the GPS isn’t running. Terminating the application does not remove this arrow as I expected. Rather, the OS will launch the application again when the next significant location change event occurs. The only way to remove the compass arrow and stop location change events is to call [[CLLocationManager sharedLocationManager] stopMonitoringSignificantLocationChanges] from within your application. Unfortunately, there is no way to list which applications are monitoring your location. I can imagine frustrated users killing all of their applications one by one in an attempt to turn off location updates.

5) Activating the GPS will register a location change event, even if the phone hasn’t moved – Opening the Maps application, checking in on FourSquare, or any other activity which uses the actual GPS will trigger a significant location change event. These are easy to distinguish from background location events because they have real values for CLLocation.VerticalAccuracy, CLLocation.Altitude, and CLLocation.Speed, whereas background location updates have values of -1, 0, and -1 respectively.