Wednesday, December 12, 2018

AI and the Productivity Paradox

Recently I was on a panel discussion at the Greenwich CIO Executive Leadership Summit (CIO Summit) on how innovation and new technologies like AI and machine learning can benefit the enterprise. During the discussion one of the audience members asked a question about the Productivity Paradox and how we can reconcile it with spending more on technology innovation. This is a type a question that a former coworker of mine would lovingly refer to as a "Stump the Chump" question as it is an economics question and none of us on the panel were economists; instead we were management, marketing and technology types.



I really don't feel any of us really nailed a great response but I did some research on the topic and have come to the conclusion that as an enterprise question it is somewhat meaningless. But it is something that we should understand as it tends to come up from time to time in industry and economics articles. Since it does some in senior management will always think that it is a reason, perhaps, to cut back or remove technology spending. It is not.

Before we get too far, what is the Productivity Paradox? In simple terms it is the idea that as technology spending has increased, from an economic perspective there has not been a similar increase in the productivity of the economy as a whole. The cynical interpretation is that our technology spending is wasted. This is now coming up in relation to new spending on artificial intelligence and machine learning.

Why is the Productivity Paradox happening? These are some of the common answers.

The technology isn't as impressive as we think it is - On the surface this is the most obvious answer, and as I said, the cynical one. We are spending all this money on technology and by extension AI and machine learning but these expenditures are not gaining us anything. Sure, could be.

We don't know how to use the technologies yet or productivity trails adoption - This isn't the first time we have heard of the Productivity Paradox. It had popped up in the 1980's as well. Then the late 1990's happened and economic productivity went through the roof. Not only did everyone have computers, but they were all connected through the internet and all kinds of new possibilities opened for us. This could be where we are at now, on the cusp of some large economic surge as other technologies come together with AI and ML to change the landscape. Similarly, it could be.

We are measuring the wrong things - The economic measure of productivity primarily deals with output of goods and services. Is economic productivity the only thing that makes or breaks a society. Probably not. Facebook, from a business perspective, is primarily an online ad revenue supported company. The economic measure of productivity would not measure in any way the benefit (or lack thereof) of social media. Similarly it doesn't measure a lot of subjective quality of life activities. It may measure the creation of a video game but not anything to do with the use of that game. Similarly if machine learning can make our shopping experiences subjectively "better" it would not measure that either. This could also be at play.



While these are all very interesting rationals, I don't think they have any day to day impact on your business. The real answer is, it just doesn't matter ... or at least it shouldn't matter. Why do I say this? Because the Productivity Paradox is dealing with a macroeconomic question of the best use of scarce resources. Individual companies probably do not measure their success on maximizing the productivity output of the world economy. Instead they likely do have a mission and that mission will be negatively impacted if they can't gain mind and/or market share.

For the sake of argument let's just say the emergence of companies like Uber and Lyft didn't change the size of the pay for ride market at all. Let's say their new business model enabled by new technology didn't increase overall productivity of the economy one bit. If you were a cab or other ride based transportation company, did you care?

My argument is they didn't laugh off Uber and Lyft saying, "look at those idiots spending all that money on technology, don't they know that isn't going to raise the productivity of the economy?". No instead they were pulling out their hair with the realization that Lyft and Uber were taking away large portions their market share. That is why the Productivity Paradox is, and will likely remain, an irrelevant question for the enterprise. Corporate missions and goals do not deal with overall economic productivity and as long as technology can deliver competitive advantage it will continue to be relevant for the enterprise.

Tuesday, August 14, 2018

A Confluence of Enabling Technologies

In the years leading  up to the release of the iPhone, all the technologies that made it possible were already in existence. We had touchscreens, cell phones, GPS devices, MP3 players, smart(ish) phones and even attempts at tablets. Everything was there, but no one could figure out how to put it all together. We had separate devices for GPS, listening to music, browsing the internet, reading email and making calls. While we had all the technologies and even could use them in commercial products, no one had figured out how to best put them together in a new and innovative product that really made these technologies work.

Now we've got new technologies and devices like this one (Microsoft's Hololens):


And this one (Amazon Echo with Alexa):


This guy (Nest):


And even this on our phones:

There are many, many more examples.

It feels like we are in a place very similar to where we were prior to 2007 and the introduction of the iPhone. There are a lot of enabling technologies like machine learning, bots, augmented reality, virtual reality, IoT and other ways for increasingly intelligent devices to interact with and change the world around us.

We might think, "Sure I can do a lot of this stuff now on my Alexa and Android or iPhone and I just got an Occulus Rift." But like the world prior to 2007 it doesn't feel like the current batch of products really has this all right. Not in the way that the early PCs did for in home computers, or the way the Mac fundamentally changed how we interacted with them in 1984, or even how battery, display, processor and OS improvements led the way to the laptop. It wasn't the existence of the internet that brought us online, it was the standardization of HTML, the web browser and distribution technologies like DSL that put it all together.

I'm not saying that tomorrow there is going to be some new revolutionary device and we're all going to stop using our smart phones. But it does feel like there is increasing momentum for something new. All these new technologies are not being used to their fullest and the current standard device platforms for delivering these technologies seem clunky at best. The smart phone is no longer the revolutionary platform that it was, it hasn't been for a while. It may take several years for this to bubble over in a usable way but it's time to start preparing. If you haven't started to learn machine learning or about augmented reality, perhaps now is the time to start investigating how these technologies work.

Wednesday, April 4, 2018

Even For Small Teams, AppCenter Helps Press the Easy Button



Opening up Xcode, Android Studio, Visual Studio, Atom or whatever integrated development environment we use and starting coding away is always easy enough. Sure, we can create an application, go through the process of manually signing it and get an app in the store. This is OK if we are working alone but at some point, many of us are going to be in a situation of working on a team or have a complex enough application where more formal testing and deployment is required. Then we start putting together disparate tools to do mobile DevOps, deploy test versions of the app, handle crash reporting, analytics, etc. Getting all this to work together can be difficult, particularly the build servers for automated builds which can be time consuming to setup and maintain for groups that don't have a lot of experience in that space.

This is the space where Microsoft's App Center really shines. For small teams, who may be new to mobile, they need to know very little about setting up these processes to use App Center. One of the strengths of this is that it falls under Microsoft's new philosophy of embracing technologies outside of their stack, so it isn't just for .Net folks. If you work in Xcode on Swift, no problem, Java in Android, no problem, React Native, no problem. App Center takes a variety of infrastructure needs common to most mobile apps and presents it as a all in one solution where all the parts are easy to use, work well together or can just be taken a-la-carte.

To see how many different capabilities Microsoft has provided on a variety of platforms consider the following:

Feature
Xamarin iOS and Android
Native Android
Native iOS
UWP
React Native
iOS and Android
Cordova
iOS and Android
MacOS
Build
 X
 X
 X
 X
 X

 X
Test
 X
 X
 X

 X
 X

Distribute
 X
 X
 X
 X
 X
 X
 X
Crash Reporting
 X
 X
 X
 X
 X
 X
 X
Analytics
 X
 X
 X
 X
 X
 X
 X
Push Notifications
 X
 X
 X
 X
 X
 X
 X
Code Push




 X
 X


Microsoft has gone to great lengths to make most of the features work with many different mobile development platforms. I don't have space to get into how these features work here, but if you want more information you can watch my AppCenter class on Lynda.com or Linked-In Leaning:



Here's how I think about the good, bad and ugly parts of App Center

The Good


  • We get continuous integration and continuous deployment setup quickly without needing to know a lot about how to do it
  • There is a lot of extendability and capability to tie in other process tools with API and REST interfaces
  • It has reasonable pricing for the features that are not outright free
  • If you are you blue about managing all the devices in your Ad-Hoc provisioning profiles, AppCenter can make this easy
  • Do you only have a few devices to test on? AppCenter has thousands
  • This even offers an entire framework for live code pushing React Native and Cordova Javascript code
  • Crash reporting and analytics can be set up easily on all supported development platforms
  • There just is a lot in this box to like. With AppCenter and a public git style repository of the right type there really is no reason not to have at least a basic CI process

The Bad


  • There are limitations around customization of build process (there are three places where custom scripts can be inserted)
  • No tie in for builds to kick off custom automated testing (must use scripts and API/REST interface)
  • Appium tests can only be written in Java
  • The testing feature does not allow for manual testing, there is no direct interaction with the devices for exploratory testing
  • There are some other limitations
    • Android React Native build only looks two directory levels deep in the repository for workspaces
    • Deployment feature still not as full featured as Hockey App
    • Features like iOS store deployment that require access to the Apple store account need two factor authentication to be turned off

The Ugly


  • To use the build services, it only has Git integration, if you use another source control tool, too bad. You also better be on VSTS, BitBucket or GitHub
  • On that note, repositories cannot be on-premise either
  • Builds cannot easily be tied into a gated check-in concept
  • The REST / API to create builds is fire and forget so it is hard to kick off an AppCenter build from another tool and know when the build is done to know if it failed.
That may seem like a lot of bad and ugly, but some of this is by design. The implication of making it super easy to set up builds that almost anyone can do, is that we may loose some flexibility. But that's kind of the point of AppCenters wide feature set. While organizations of any size and complexity may be tempted to use certain features of AppCenter, AppCenter also brings more complex things like the creation of build processes to even the smallest of teams. If your build needs are super complex then Microsoft has the much more flexible VSTS build services that can be used. But if your build needs are somewhat generic and simple, boy can you get that going with AppCenter quickly. The box of features is large, the breadth of toolsets supported is wide, there just is something for everyone to be found in there.

Tuesday, August 8, 2017

Visual Studio Mobile Center: Solving a Problem with Package Restore

Visual Studio Mobile Center is shaping up to be a great tool. Where before it had taken me a week to set up the builds in VSTS it now takes less than an hour. This speed and simplicity does come at a cost, mostly in terms of not having a lot of fine control over the build process. About a month ago this caused a problem for me.

I don't have full control of a repository I was trying to build using VSMC. That is to say the QA group added a directory with their test solutions. This was fine except that test solution required a Nuget package that was hosted on our company's internal Nuget server. Why was this a problem? Because VSMC will try and restore packages for any and all solutions it finds in the repository, even ones I'm not trying to build with VSMC and that meant the package restore step failed every time.

Luckily about the time this happened, VSMC implemented a feature I (and probably others) have been asking for. The ability to create custom build steps. There are three custom scripts that can be added, one right after the source code is retrieved (post-clone), one right before the solution is built (pre-build) and one right after the solution has been built (post-build). For Android and iOS apps the build happens on a Mac so bash scripts need to be made, for UWP apps the build happens on a Windows machine so Powershell scripts are needed. For more information you can check out their easy to read documentation:

VSMC Build Scripts

How does this help me and my error? Well as it turns out I can create a post-clone script that deletes the test project folder out of the cloned repository *BEFORE* the Nuget package restore happens. I don't need anything in there so no problem removing it. For my particular case I am doing Xamarin iOS and Android builds. That means I need to create a Bash script.

The script must be a specific name, mobile-center-post-clone.sh for a post clone shell script. It also has to be in the same directory as the solution or project to build. Because the solution that built my iOS project and the project for the Android application were in different locations, I needed two copies of this same script.

Here is the script I created:

 #!/usr/bin/env bash  
 echo “removing test automation directory”  
 set -e  
 rm -rf $MOBILECENTER_SOURCE_DIRECTORY/testAutomation  

The most important line in the script is the last one. It removes the entire directory from the build server where the QA project resides that is causing the problem. Notice that I am also using one of the built in build variables, $MOBILECENTER_SOURCE_DIRECTORY. This inserts the location of the root directory where the source code was copied to. It allows my script to work anywhere so I can use the same one for the iOS build and the Android build. It works like a charm.


Note: You may have to re-save your build configuration before it picks up that the scripts are there. The above picture shows that the build definition has detected my post-clone script.

That's it. If you are having issues with the Nuget restore step on a project you are not trying to build, you can just remove it using such a script. You could also do things with scrips like running UI tests after the build or changing the app Ids before the build. This is a great addition to Visual Studio Mobile Center.

Monday, June 5, 2017

How Software Development Reminds Me of the Things I Learned in the Military

A long time ago (seems like just a few years but the calendar says a few decades) I was in the military. I was very young when I joined, seventeen and freshly graduated high school. My time in the military has defined everything that came after. There were some great lessons there that I chuckle about when I consider my career in software development. Here are some of my favorites.

There is the right way, the wrong way and the Army way.
When on software projects you will see a lot of things that are good and bad. You may try and change them and people may or may not listen. At the end of the day you are still part of a project and you have to work as a team, even though that occasionally means doing things you know are not optimal. Tabs vs. spaces just isn't that big a deal, do what the team agrees on. Just running off and doing your own thing can be more harmful than the practice you dislike. Luckily, unlike the military, if you find the practices too poor and the project swirling the drain you can always leave a sinking ship.

Don't draw fire, it irritates those around you / don't be a blue falcon. 
Look we all mess up once in a while, but don't always be the guy who breaks the build or doesn't make his sprint commitments. Software development is portrayed as an industry of introverted loners but the reality of most our careers are far different. Well not the introvert part, that's true. But we are not loners. For the most part we work in teams and we have to depend on each other. You need to trust the people you work with and they need to trust you.

When the pin is pulled, Mr. Grenade is not our friend.
Having the credentials and ability to deploy software to production is not something that should be treated casually. Sure it has great power but it's dangerous as heck. They have the power to do good things for your company but can also unleash great destruction and if you are not careful it will be your fault. Did I mention don't be a blue falcon?

Cluster bombing from B-52s are very, very accurate. The bombs are guaranteed to always hit the ground.
Everything you change in the code is going to have an effect. Many times it is hard to understand what that effect will be. That little refactoring you want to do right before the release because you can do it better? Be prepared for that to come back at you in some way you don't expect. You can be sure it will change something, just not perhaps what you think. Code changes are in many cases better done with a scalpel instead of wholesale, "well, while I'm here" type changes.

Weather forecasts are horoscopes with numbers.
And so are large project estimates. I don't think I need to go too far into this, but they are rarely right. Humans are horrible estimators of things we haven't done before and almost all truly custom software is doing things we have not done before. The larger the estimate the more likely it is to be underestimated.

Flashlights are tubular metal containers kept in a flight bag for the purpose of storing dead batteries.
All those backups you have and contingency plans are useless when you don't use them or test them. Your computer will crash or something is going to go wrong. The cloud has made things so much easier to make sure you don't loose everything. These days there is no excuse to be stuck with dead batteries. But once in a while you still will be. Expect that too and at the most inopportune time.

Mountain? Hell, it was flat on the map!
Code changes always seem easy when talking about them. How hard could it be. Well that is until you start and find out all the complexities hiding under the hood...

X Days and a wake up.
Sometimes you get on that one horrible project, the death march. No matter how crummy it is, it will come to an end (in one way or another). Your time in purgatory will be limited and one day you will be done with it.

If it ain't raining, it ain't training.
You will learn more when things are going bad than when they are going right, usually much more. When things are going badly it should prompt you to start thinking, what can I learn from this?

Go to the flight line and get some rotor wash.
You can figure this one out. Don't forget to have fun, don't take yourself too seriously and never hesitate to give the rest of your team a good natured hard time.

Sunday, April 16, 2017

The Continued Rise of Cross Platform Tools

This morning I read this: learn to write Android applications in Swift and share code with your iOS app. The mobile market is maturing and unlike what many of us have been used to for a very long time, no one operating system came out on top. Neither Android or iOS will be likely to at this point. The proponents of Android or iOS will point to some movement around the edges, but that's mostly wishful thinking. Until the next revolution, multiple platforms will be the norm and now everyone wants to know how to code for multiple platforms.

This is good news for tools that are created for writing cross platform apps, really good news. We see a lot of them out there, Xamarin, ReactNative, NativeScript, Cordova, Progressive Web Apps, ReactXP, the list keeps going. Where do I think this will lead?


  • The mobile platforms are maturing and the need to write apps for multiple platforms may be necessary for many companies.
  • For internal enterprise apps where the main driver is cost, some organizations will either try to only hit one platform or make a bet on one of the cross platform tools like Xamarin.Forms.
  • Native tools that can only hit one platform, like Xcode or Android Studio will loose traction in many scenarios to lower cost cross platform tools. Increasingly these will be used in scenarios where only one platform is needed or there is some need to stay abreast of the latest and greatest on a platform on the way that a non-platform vendor has difficulty keeping up with.
  • First generation cross platforms tools will loose traction to better tools with many of the same advantages. In particular, Cordova is in danger of this happening in favor of something like ReactNative.
  • The web is coming to mobile with Progressive Web Apps. They still have significant challenges on iOS devices but the popularity of the HTML5, CSS, Javascript platform will drive IT departments in this direction. This may have secondary impacts on if iOS will be able to continue to make inroads in many organizations as corporate devices due to their limitations unless they relent (unlikely at the current time due to the need to protect their app store revenue).
  • Tools that can go cross platform, particularly on the UI layer, will become increasingly popular in scenarios where the UI can be good enough (Xamarin.Forms holds great promise here).
  • Cross platform tools that can do a few platforms in a focused way may win out over cross platform tools that attempt to hit every platform. The coding situation becomes too tough and abstracted when too many platforms are at play; the situation also becomes too difficult for the maintainers to keep up with changes for all the different platforms. Cross platform tools that only target 2-4 platforms will likely do a much better job of it than ones that try to do every one under the sun.
  • Cross platform quality control, build, life cycle, monitoring and tooling products will become more mature. Microsoft's recent attempt at this, Visual Studio Mobile Center, shows great promise for where it could go.
There is a lot of things still going on in mobile. But we are not so much waiting for the next version of the OS, the next announcements at WWDC or Google I/O. The changes there will likely be incremental. The changes that are exciting now are watching how we develop for these diverse platforms matures. Better tooling, better process, better quality control. If you are doing mobile development now you should be thinking about how to have a solid, controlled, and automated process and your tool set for most cases should be cross platform as well. Very exciting for those of us who develop the apps but it leaves me wondering where the excitement will be for the consumer of the devices themselves?