When it comes to gaining insight from your data, traditional business intelligence and data warehouse solutions have leveraged years of innovation to arrive at intelligent solutions and methodologies to gain valuable insight from structured sets of data. The success of these solutions has conditioned organizations to the value of collecting and analyzing data— resulting in initiatives to collect and process even more. Today, the sheer volume and rate of data generated dwarfs that of days past and the potential value of this data leaves little incentive to throttle back. Organizations are now faced with the reality of staying ahead of the ever-growing scale and velocity of the data they are generating. To better do so, many have turned to big data solutions powered by Hadoop and data lakes for most or even all of their data discovery, organization, analytics, and reporting initiatives.
Big data solutions do not come cheaply, simply, or quickly. Not only is a large upfront monetary investment in hardware and supported software required, but also an upfront investment in time to plan, purchase, install, configure, and test the solution is needed before delivering any value to the business. Expert administrators and operators are required to administer the system. Storage capacity requirements grow rapidly and massive amounts of computing power are essential to processing data quickly. It is nearly impossible to decouple storage capacity and compute power to scale independently. Systems must be greatly overprovisioned for redundancy and future growth, and run the risk of being obsoleted quickly. Quite simply: Purchasing or building an on-premises big data solution comes at a big cost with a big risk of impacting time to insight.
