The Continued Rise of Cross Platform Tools

This morning I read this: learn to write Android applications in Swift and share code with your iOS app. The mobile market is maturing and unlike what many of us have been used to for a very long time, no one operating system came out on top. Neither Android or iOS will be likely to at this point. The proponents of Android or iOS will point to some movement around the edges, but that's mostly wishful thinking. Until the next revolution, multiple platforms will be the norm and now everyone wants to know how to code for multiple platforms.

This is good news for tools that are created for writing cross platform apps, really good news. We see a lot of them out there, Xamarin, ReactNative, NativeScript, Cordova, Progressive Web Apps, ReactXP, the list keeps going. Where do I think this will lead?

• The mobile platforms are maturing and the need to write apps for multiple platforms may be necessary for many companies.
• For internal enterprise apps where the main driver is cost, some organizations will either try to only hit one platform or make a bet on one of the cross platform tools like Xamarin.Forms.
• Native tools that can only hit one platform, like Xcode or Android Studio will loose traction in many scenarios to lower cost cross platform tools. Increasingly these will be used in scenarios where only one platform is needed or there is some need to stay abreast of the latest and greatest on a platform on the way that a non-platform vendor has difficulty keeping up with.
• First generation cross platforms tools will loose traction to better tools with many of the same advantages. In particular, Cordova is in danger of this happening in favor of something like ReactNative.
• The web is coming to mobile with Progressive Web Apps. They still have significant challenges on iOS devices but the popularity of the HTML5, CSS, Javascript platform will drive IT departments in this direction. This may have secondary impacts on if iOS will be able to continue to make inroads in many organizations as corporate devices due to their limitations unless they relent (unlikely at the current time due to the need to protect their app store revenue).
• Tools that can go cross platform, particularly on the UI layer, will become increasingly popular in scenarios where the UI can be good enough (Xamarin.Forms holds great promise here).
• Cross platform tools that can do a few platforms in a focused way may win out over cross platform tools that attempt to hit every platform. The coding situation becomes too tough and abstracted when too many platforms are at play; the situation also becomes too difficult for the maintainers to keep up with changes for all the different platforms. Cross platform tools that only target 2-4 platforms will likely do a much better job of it than ones that try to do every one under the sun.
• Cross platform quality control, build, life cycle, monitoring and tooling products will become more mature. Microsoft's recent attempt at this, Visual Studio Mobile Center, shows great promise for where it could go.

There is a lot of things still going on in mobile. But we are not so much waiting for the next version of the OS, the next announcements at WWDC or Google I/O. The changes there will likely be incremental. The changes that are exciting now are watching how we develop for these diverse platforms matures. Better tooling, better process, better quality control. If you are doing mobile development now you should be thinking about how to have a solid, controlled, and automated process and your tool set for most cases should be cross platform as well. Very exciting for those of us who develop the apps but it leaves me wondering where the excitement will be for the consumer of the devices themselves?

Mobile Development Platform Performance (Native, Cordova, Classic Xamarin, Xamarin.Forms)

Last month I published my Mobile Technology Decision Making White Paper and I feel that goes a long way to helping choose what technology to use in the mobile space.  One question I get asked frequently is how do the different mobile development platforms compare to each other from a performance perspective. While I've heard a lot of anecdotal information I have not seen a lot of formal comparisons.  At Magenic we normally work with native (iOS, Android, WP), Cordova and Xamarin so I wanted to compare these.  I plan on this being the first post on this topic.

For some background on testing methodology using Android and iOS.

The Development Platforms
- Native (Objective-C 64 bit and Java)
- Cordova (Multi-Device Hybrid Apps) using Intel's App Framework for the UI
- Classic Xamarin (64 bit unified beta for iOS)
- Xamarin.Forms (64 bit unified beta for iOS with beta version of Xamarin.Forms, note latest version of the unified API in the beta/alpha channels could not be used as it is not supported by Xamarin.Forms Note: 1.3.1 pre 1 was released Dec 24th so Xamarin.Forms may now work with the version of the unified iOS API in the alpha and beta channel)

The Devices
- iPad Mini (non Retina) running iOS 8.1.1 (12B435)
- ASUS K00F running Android 4.2.2

The Test Apps
Applications were made for each of the development platforms that are functionally similar. There was little (if no) effort to make them look exactly the same or even look "good".  But they looked about the same.  There were some differences such as Java, Classic Xamarin and Xamarin.Forms rendered the tabs on the top in Android as expected while the JavaScript library showed them on the bottom.

The Timing Methodology
Due to difficulties in knowing when things are "done", particularly with JavaScript, timings were handled via stopwatch.  Each timing was taken ten times and the results were averaged.  It should noted that hand timings have an accuracy of about 2/10 of a second so that does give us an approximate margin of error.

Test 1: Test App Size
The size of the application can impact how much bandwidth it takes to deploy and also have some impact on load times.  For Android the size of the APKs was examined.  For iOS I looked at Settings to find out how much space the apps took up on disk.

Development Platform	Size
Android
Java	166kb
Cordova	433kb
Classic Xamarin	3.5mb
Xamarin.Forms	4.7mb
iOS
Objective-C (64 bit)	644kb
Cordova	2.7mb
Classic Xamarin	12.1mb
Xamarin.Forms	16.9mb

When it comes to application size Xamarin shows the extra size involved in the overhead of the .Net framework.  There was an attempt to reduce the size of the deployed Xamarin application by using the "Link SDK assemblies only" setting.  I am surprised in a very small application how large the difference is.  However, from experience in "real" applications the difference is much less consequential as graphics and frameworks get added to the projects.

Test 2: Load Times
I wanted to see how long it took the application to load into memory.  While the initial load time is important, many mobile applications tend to stay in memory so it tends to have a limited impact.  For this test I made sure to close all applications before each timing.

Development Platform	Test Avg.
Android
Java	1.085
Cordova	3.978
Classic Xamarin	1.704
Xamarin.Forms	2.764
iOS
Objective-C	1.221
Cordova	1.715
Classic Xamarin	1.28
Xamarin.Forms	1.813

In all cases the vendor native technologies loaded the fastest.  Classic Xamarin loaded nearly as fast as the native languages.  Xamarin.Forms and Cordova had the slowest load times.  The Cordova load time on Android was particularly bad while on iOS the load times were close enough to not be a huge factor.

Test 3: Loading a List from Azure Mobile Services
In this test I wanted to look at getting data from an external service so I loaded 1000 records from Azure Mobile Services.  For Xamarin iOS 64 bit I had to modify the Azure Mobile Services to be compatible with the unified API.  The timings were taken from pushing the button to load the list until the results visibly came back and were displayed on a list on the screen.

Java:

public  void addRecord(String firstName, String lastName, int index, String misc) throws Exception {
    if (dbConn == null) {
        openConnection();
    }

    ContentValues values = new ContentValues();
    values.put("firstName", firstName);
    values.put("lastName", lastName + index);
    values.put("misc", misc);
    dbConn.insertOrThrow(TABLE_NAME, null, values);
}

Objective-C:

- (void)addRecord:(NSString*)firstName withLastName:(NSString*)lastName withIndex:(int)index withMisc:(NSString*)misc withError:(NSError**)error {
    NSString *sqlStatement = NULL;
    char *errInfo;
    *error = nil;

    if (dbConn == nil) {
        [self openConnection:error];
        return;
    }
    
    sqlStatement = [NSString stringWithFormat:@"%@%@%@%@%d%@%@%@", @"INSERT INTO testTable (firstName, lastName, misc) VALUES ('", firstName, @"', '", lastName, index, @"', '", misc, @"')"];
    
    int result = sqlite3_exec(dbConn, [sqlStatement UTF8String], nil, nil, &errInfo);
    
    if (result != SQLITE_OK) {
        NSMutableDictionary *errorDetail = [NSMutableDictionary dictionary];
        [errorDetail setValue:[NSString stringWithFormat:@"%@%s", @"Error writing record to database: ", errInfo] forKey:NSLocalizedDescriptionKey];
        *error = [NSError errorWithDomain:@"testDomain" code:101 userInfo:errorDetail];
    }
}

JavaScript:

db.executeSql("INSERT INTO testTable (firstName, lastName, misc) VALUES (?,?,?)", ["test", lastName, "12345678901234567890123456789012345678901234567890"], function (res) {
    successCount++;
    if (successCount === maxValue) {
        $.ui.popup({
            title: "Success",
            message: "All records written to database",
            doneText: "OK",
            cancelOnly: false
        });
        $.ui.unblockUI();
    }
}, function (e) {
    $.ui.popup({
        title: "Error",
        message: "An error has occurred adding records: " + e.toString(),
        doneText: "OK",
        cancelOnly: false
    });
    $.ui.unblockUI();
    return;
});

Xamarin (All Versions):

public void AddRecord(string fName, string lName, int i, string m)
{
    if (dbConn == null)
    {
        OpenConnection();
    }

    var testRecord = new TestTable {firstName = fName, id = 0, lastName = lName + i, misc = m};

    dbConn.Insert(testRecord);
}

Xamarin Classic Android Alternate:

public  void AddRecord(string firstName, string lastName, int index, string misc) 
{
    if (dbConn == null) 
    {
        OpenConnection();
    }

    ContentValues values = new ContentValues();
    values.Put("firstName", firstName);
    values.Put("lastName", lastName + index);
    values.Put("misc", misc);
    dbConn.InsertOrThrow(TABLE_NAME, null, values);
}

Development Platform	Test 1	Test 2	Test 3	Test 4	Test 5	Test 6	Test 7	Test 8	Test 9	Test 10	Test Avg.
Android
Java	22.71	17.5	18.04	17.7	18.63	20.33	2.68	2.42	2.16	2.34	2.369
Cordova	25.99	24.76	27.05	23.3	24.06	22.86	2.12	2.02	1.94	2.48	2.149
Classic Xamarin	34.07	27.38	32.05	38.77	29.27	34.63	1.61	1.63	1.84	1.85	1.738
Xamarin.Forms	1.99	1.76	2.32	1.91	1.9	1.58	1.93	2.02	2.03	1.64	1.908
iOS
Objective-C	2.38	2.44	2.24	2.3	2.34	2.32	2.32	2.35	2.2	2.27	2.316
Cordova	3.57	2.18	2.07	1.95	1.97	2.05	2.04	1.93	2.2	1.96	2.192
Classic Xamarin	2	1.87	1.88	2.06	1.74	1.9	1.81	1.94	1.75	1.96	1.891
Xamarin.Forms	2.11	2.01	2.23	1.96	1.95	2.07	2.12	2.16	2.08	2.1	2.079

*results in seconds

In many ways this test is showing how well the Azure Mobile Services libraries perform on the different platforms.  Unsurprisingly Xamarin, with it's underpinnings of a .Net implementation, performs the best in this test.  I was surprised to see the libraries for the native technologies perform the worst, both on Android and iOS.

I have heard that the Xamarin.Forms lists can perform poorly with large data sets.  These results did not show that, at least with lists of up to 1000 records.

Test 4: Prime Number Calculation
In the final of my first series of tests I wanted to try out a CPU intensive operation.  I created a Sieve of Eratosthenes on each of the platforms.  My first plan was to calculate all prime numbers up to 50,000,000.  This required some special handling of the method's array for both Objective-C and JavaScript.  In the case of Objective-C I had to malloc memory to support arrays that large.  Also for Objective-C and JavaScript I had to initialize the array items to 0.  To keep the timings the same I did the array item initialization to 0 on all platforms even though it could have been left out for .Net (and Java I believe).  It that was done, the .Net timings would have been even better.

I did end up having to settle for only calculating primes up to 5,000,000.  The reason for this is that the JavaScript performed so poorly that I was unwilling to wait for it to complete 10 times.

Java:

private int getPrimesFromSieve(int maxValue)
{
    byte[] primes = new byte[maxValue + 1];
    for (int i = 0; i <=maxValue; i++)
    {
        primes[i] = 0;
    }
    int largestPrimeFound = 1;

    for (int i = 2; i <=maxValue; i++)
    {
        if (primes[i - 1] == 0)
        {
            primes[i - 1] = 1;
            largestPrimeFound = i;
        }

        int c = 2;
        int mul = i*c;
        for (; mul <= maxValue;)
        {
            primes[mul - 1] = 1;
            c++;
            mul = i*c;
        }
    }
    return largestPrimeFound;
}

Objective-C:

- (int) getPrimesFromSieve: (int) maxValue {
    Byte *primes;
    primes = (Byte *) malloc(maxValue * sizeof(Byte));
    for (int i=1; i<=maxValue; i++)
    {
        primes[i-1] = 0;
    }    

    int largestPrimeFound;
    largestPrimeFound = 1;

    for (int i=2; i<=maxValue; i++)
    {
        if(primes[i-1] == 0)
        {
            primes[i-1] = 1;
            largestPrimeFound = i;
        }

        int c=2;
        int mul = i*c;
        for(; mul <= maxValue;)
        {
            primes[mul-1] = 1;
            c++;
            mul = i*c;
        }
    }
    return largestPrimeFound;
}

JavaScript:

function getPrimesFromSieve(maxValue) {
    var primes = new Uint8Array(new ArrayBuffer(Number(maxValue)));
    for (var i = 0; i <=maxValue; i++) {
        primes[i] = 0;
    }
    var largestPrimeFound = 1;

    for (i = 2; i <= maxValue; i++) {
        if (primes[i - 1] == 0) {
            primes[i - 1] = 1;
            largestPrimeFound = i;
        }

        var c = 2;
        var mul = i * c;
        for (; mul <= maxValue;) {
            primes[mul - 1] = 1;
            c++;
            mul = i * c;
        }
    }
    return largestPrimeFound;
}

Xamarin (All Versions):

public static int GetPrimesFromSieve(int maxValue)
{
    var primes = new byte[maxValue + 1];
    for (var i = 0; i <=maxValue; i++)
    {
        primes[i] = 0;
    }
    var largestPrimeFound = 1;

    for (var i = 2; i <=maxValue; i++)
    {
        if (primes[i - 1] == 0)
        {
            primes[i - 1] = 1;
            largestPrimeFound = i;
        }

        var c = 2;
        var mul = i*c;
        for (; mul <= maxValue;)
        {
            primes[mul - 1] = 1;
            c++;
            mul = i*c;
        }
    }
    return largestPrimeFound;
}

Development Platform	Test 1	Test 2	Test 3	Test 4	Test 5	Test 6	Test 7	Test 8	Test 9	Test 10	Test Avg.
Android
Java	4.31	4.31	4.2	4.33	4.39	4.37	4.32	4.45	4.34	4.4	4.342
Cordova	91.69	95	94.31	94.4	94.73	94.1	94.1	91.8	93.63	97.75	94.151
Classic Xamarin	4.27	4.25	4.15	4.32	4.51	4.41	4.22	4.12	4.14	4.19	4.258
Xamarin.Forms	4.21	4.17	4.31	4.3	4.2	4.34	4.29	4.36	4.22	4.19	4.259
iOS
Objective-C	5.04	5.49	5.38	4.86	4.8	5.02	5.03	4.83	4.84	4.85	5.014
Cordova	66.96	67.36	67.22	67.3	67.17	67.44	67.13	67.11	67.58	67.64	67.291
Classic Xamarin	4.41	4.42	4.35	4.34	4.49	4.37	4.17	4.27	4.39	4.28	4.349
Xamarin.Forms	4.51	4.33	4.31	4.31	4.33	4.4	4.41	4.4	4.33	4.46	4.379

*results in seconds

While I was expecting that JavaScript would be slower, I was unprepared for how much worse it was for this type of operation.  This would make Cordova problematic for highly CPU bound work were performance is important (I wonder what that means for server side Node.js...).  CPU bound work is not as important in many of today's mobile applications but given the history of increased performance for mobile CPUs and what happened in the PC market in the 1990's, it is likely that in the future CPU bound work will be more prevalent in mobile applications.

Having said that, using native HTML commands can be very fast.  In the past I've testing loading JSON using HTML commands vs. the JSON.Net library on Xamarin and found them nearly comparable.

I was also surprised that Xamarin performed better than Objective-C by a noticeable amount.  It also performed better than Java on Android but by a very marginal amount, well within the .2 second margin of error that manual timings give us.

That's it for my first installment of performance tests.  Much of this code was made more difficult for Xamarin due to the flux around the 64 bit Unified iOS API.  For next month I'll take a look at loading large JSON strings and perhaps something else.

I hope this is useful or you.  If you have any ideas for performance tests I can perform, I'd love to hear them.

Source code for the tests can be found here: Performance Tests Source