“Data Driven Thinking” is written by members of the media community and contains fresh ideas on the digital revolution in media.
Today’s column is written by Greg Keller, Chief Operating Officer at LinkSmart.
Green arrows up. Red arrows down. The distillation of large amounts of data into simple information that users can understand and take action on has come a long way. While part of this evolution can be attributed to the way companies have brought data-intensive applications to market, the shift is also largely influenced by the rising business culture founded on questioning data. Companies who follow this philosophy seek employees who have the keen ability to see a leaf’s singular detail all the while understanding the complete forest of data that is available. This unique characteristic enables companies to not only gather data, but to also productize and manage in a valuable way that delivers those simple green arrows, pie charts, and graphs.
First, how did we get here? Today’s SaaS-based products are a far cry from their earlier predecessors. Moreover, the development teams assembling them are not the same breed of workers who once built applications for the primitive web or desktop. In the early 2000s, web applications and the databases storing and managing them, were highly structured and you had to know first what you wanted to store before bolting on application layers.
It was a meticulously calibrated affair of “measuring twice and cutting once” to ensure the requirements were well known in advance and the application was prepared to on-board the needed data. When these calculated updates were made, the development teams faced dangerous risks from the unknown, rippling effects of the application, costing both time and money. Add to these factors sub-par databases that were never built to handle large amounts of data efficiently and companies had the makings of a perfect data disaster storm.
To the relief of developers everywhere, much has changed on the technology front in a short period of time to accommodate this fast growing scale. We’ve seen the Web inflate exponentially in a decade, witnessing it move from the desktops we were once tied to, right into the palms of our hands. With widespread access to the web, the data that can now be generated is so vast it’s almost incomprehensible. The old way of building software applications is now arcane if not impossible…like pushing the proverbial stone up a muddy hill. Instead, clicks, swipes, user interaction data and more can now be easily consumed through applications.
So what does this all mean for the future? With the rapid availability of data collection and tools there is a radical juxtaposition in the way teams plan and develop their application strategies. Today’s rule of thumb for many start-ups is ‘Just get the data. Fire-hose it in and we’ll learn how to productize it later.’ Large-scale data storage systems are now freely available to manage this seemingly endless stream of web-generated data and with open-source systems like Cassandra and Hadoop now responsible for storing, scaling and processing voluminous data, companies have many options to choose from. It has never been easier to utilize “the cloud” efficiently and inexpensively.
As our software building blocks have matured, the developer teams building the systems are trailblazing a new frontier. These folks and the culture they are inspiring within organizations are the most exciting things about this period of the Web — above and beyond the technologies available. Successful organizations will find ways to attract such dynamic, forward thinking people to thrive and compete in the emerging big data culture.
Follow LinkSmart (@linksmart) and AdExchanger (@adexchanger) on Twitter.
