As consumers and developers get ready for the launch of iOS 7, at Appurify we wanted to understand how the performance of your favorite apps will change with the new operating system.
To that end, we’ve been conducting detailed experiments on several popular apps on the powerful Appurify mobile performance optimization platform. We were able to automate the measurement of key performance metrics—including load time, CPU and memory usage, FPS, network efficiency, crash rates, render times and multi-app interaction times—on both iOS 6 and iOS 7 devices.
The importance of load times
The first metric we focused on, and the topic of this post, is arguably the most important as well: app load time.
For this post, we compared the load times on iOS 6 and 7 for some of the top social networking apps in the Apple App Store. All of the apps we compared are native iOS apps that generate a lot of complex network side traffic when the user logs in. Because social networking apps have so much network activity, the condition of the underlying network is also a major driver of load time—logging into Facebook on WiFi isn’t the same as logging in on AT&T 2 bars. Again, Appurify’s unique and powerful platform allows us to conduct our automated performance tests on a variety of network conditions.
Our measurement methodology
We conducted our experiments on six of the most popular social networking apps: Facebook, Twitter, Vine, Instagram, LinkedIn and Foursquare. For Foursquare, we measured both load time when logging in through email, and logging in through Facebook Connect. For Instagram, we measured the time it takes to log in to the app, and the time it takes to load photos.
The two devices we used were an iPhone 5, iOS 6.1.4, and an iPhone 5, iOS 7 beta 6. We automated loading across multiple iterations, and four network conditions:
What we found
We expected Apple to include improvements in their networking TCP/IP stack and radio data handling, so it’s not surprising that all of the apps performed better on iOS 7 vs. iOS 6.
On average, the social networking apps tested loaded 11% faster on iOS 7 vs. iOS 6.
We also noticed some interesting differences in how apps react to poor network conditions and changes in iOS. As you can see, there is a trend where load times under poorer network conditions has improved more, indicating that Apple has started optimizing for variable or poor end-user conditions.
The improvement in iOS 7 performance under poor network conditions can be seen more clearly when comparing the normalized load times under different network conditions on iOS 6 and iOS 7. Even if the gap seems small, when it comes to load time, every bit counts. Also, under Poor and Bad network conditions, this could amount to a nearly 10-second speed up.
In the graph above, the average load times (across iterations) for different apps on the Good network conditions is shown. The two lines show load times in seconds on iOS 6 and 7, and the bar graphs show the speed improvement on iOS 7 vs. iOS 6.
As you can see, Twitter, Instagram and Vine are the fastest-loading apps on iOS 6 and iOS 7 under Good network conditions. On the other end, Facebook, LinkedIn and Foursquare using the Facebook Connect login are the slowest loading apps.
Using Foursquare’s email login is significantly faster than with the Foursquare Facebook Connect login for both iOS 6 and iOS 7. Facebook’s load time improves by at least 30% across all network conditions.
The good news about the slower apps though is they’ve significantly sped up between iOS 6 and 7, shaving off between 1 and 2 seconds on Good network conditions.
There’s still plenty of room for improvement under Poor network conditions
Users are not always going to be in Good network conditions, so it’s important to understand load performance on a range of networks. And although Twitter and Vine were among the fastest apps to load under Good network conditions, they were not able to maintain their lead when the network conditions degraded. Specifically, under Poor and Bad networks, the time it took for Twitter, Vine and Instagram photos to load skyrocketed. In contrast, Facebook, Instagram, LinkedIn and Foursquare email login registered only a moderate increase.
Fewer failure to load errors!
Finally, we noticed some apps failed to load at all under Bad network conditions, but this seems to be improving with iOS 7.
Instagram (and Instagram Photos), and LinkedIn failed to load 100% of the time under Bad network conditions on iOS 6. This improved for all three when moving to iOS 7 (LinkedIn had no failures on iOS 7, and Instagram and photos dropped the failure rate by 50%). However, Vine, which saw no issues on iOS 6, started seeing occasional issues on iOS 7.
Hypotheses—Why is iOS 7 faster?
Of course, the big question is why apps perform better on iOS 7, and what developers can do to extract the most benefit from the new OS. We’re currently conducting a technical deep-dive, and some of the hypotheses we’re exploring include:
- TCP/IP handling of network requests over poor network conditions;
- SSL encryption and decryption of requests;
- Low power handling of WiFi and the Radio. Typically, the longer these chips stay in low-power mode, the higher the battery life. However, this impacts the throughput. Hence, at a slight compromise to battery life, network throughput can be significantly improved;
- Better CPU scheduling, which lets the app run on CPU longer;
- Native app rendering improvements.
To get to the heart of the improvements, we will be using the Appurify platform to profile and de-SSL HTTPS traffic. This will be critical in understanding the network traffic and pinpointing bottlenecks.
Here is an example of LinkedIn app’s server-side HTTPS traffic:
And here is an example of Vine app’s server-side HTTPS traffic:
As we dive into these results further and conduct further experiments, we will verify our hypotheses and publish more data and best practices for app performance improvement.
Conclusions and next steps
Regardless of how and why it’s happening, it’s great that Apple is speeding up the performance of some of your favorite apps on iOS 7. And while our research focused on just social networking apps, our conclusions should hold true for other app categories as well.
At the same time, looking at each app’s individual results shows the amount of opportunity app developers still have to improve and enhance the performance of their apps. Specifically, we see the importance of testing and optimizing your apps under a variety of user conditions. You can’t guarantee that your users will always be using your app in a Good network, and designing resiliency to degrading network conditions is crucial for broad success of your app. You can use platforms like Appurify to automate the evaluation of your apps across not just a range of devices and OS versions, but equally important, across a range of user conditions.
Meanwhile, we will continue to explore the differences between iOS 6 and iOS 7. Stay tuned as we follow up with continued analysis on load performance, stability and bugs on the two iOS versions, and a comparison of HTML5, hybrid and native apps.