The Future of Software

Note: I originally wrote this post in February 2006 for my “corporate” blog, but since its very general in nature, I can’t see any reason to keep it there.

The Last 20

Here is a question to spark some discussion: What is the number one thing that will differentiate software 10 years from now from software today.

Over the last ten years, I say communications is number one. Though the Internet existed long before, this time was the fulcrum point (I'm fudging a bit to say 10 years but who wants to be exact). A secondary and connected area is data. Data is secondary because without communications none of the vast amounts of data currenlty available would have been collected or widely available. In fact, most of the data that fills up the vast coffers of the World Wide Web was around prior to it's inception, but was never so widely available.

The 10 years prior to that, I say user interfaces. Computers during this period evolved from being designed for "geeks" or narrowly trained workers (data entry workers). Users could be trained to use a single program very well, but once outside that area of specialized training there was little cross applicability. Developers generally designed software that "they" wanted and could use, and did not design software for users. Software came with thousand-plus page manuals and developers expected users to read them (at least once).

Eventually, developers began to realize that if software was easier to use more users would use it. Also, users would have the time (and memory power) to learn several different pieces of software. One way software became easier to use was through convergence. User interfaces started to share common elements, metaphors and more, and not just within a specific silo (like word processing), but across many areas. Also, things like online and context sensitive help encouraged users to "learn on the job". Lastly, there was graphics. Mice, pictures, graphics, all seemed more natural to users, and applications could use visual tricks well known from newsprint and magazine trades to draw users to important details.

The Next 10

Ten years from now, I think people will be saying the use of statistics. Over the past 10 years a huge amount of information has been collected and everyday more and more sources of data appear. RFID promises to create a vast abundance of data on things industrial, commercial and even personal. Wireless technologies promise to make this data available wherever you go, and also provide a path for more real time data to be collected.

So after talking so much about data, why do I think statistics will be number one? Well because honestly there is too much data already. Statistics however are what allow this data to become useful. I know, I know, this is nothing new. Search sites have long since been using statistics to allow users to dig through the vast pile of data. We internally have been using statistics to provide data to our clients. But the difference between today and ten years from now is that today such applications are few and generally specialized. In the future statistics will become an integral part of almost all applications, and will apply to a much greater number of a user's interactions with software.

The trends are already beginning. Amazon.com was an early innovator, and a great number of similar or connected sites are taking similar routes to simplifying a users search for what he wants by comparing his tastes with users of similar tastes. Estimating of many kinds (think projects, inventory control, etc., not just automotive) will become based upon statistics rather than heuristics, which will increase its accuracy and cut administrative costs dramatically.

So what does this mean for us? For one, developers who do not already work with statistics would be well advised to begin brushing up on their understanding of statistics. While it is likely that the process of implementing statistical models in software will greatly simplify, an understanding of the core concepts is always useful. Database software packages are where most statistics related development occurs today, but I expect a number of specialized tools will spring up as databases struggle to meet evolving statistical needs while still delivering on their core principles.

As I close, one cautionary note about using statistics. Statistics are a tool like all other data. People still need to rule, not numbers. If you don't understand what I mean, think of how unpopular Amazon.com would be if instead of suggesting books, CD or movies it thought you liked it automatically mailed and billed you for them... The website can save you time because it might take you much longer to find something you liked, but once you see it you are far better off making the decision on your own.