Gaining Perspective on the True Cost of Pricing Models for Analytics
I had a client years ago that was looking to add their International operations onto their existing data warehouse – which supported only the U.S. users at the time. They asked for an analysis on what it would take in terms of additional investment. Independently, the resources (and associated investment) would have been significant. As I looked at the workload and usage patterns, however, it turned out that nearly the entire new set of users would be able to leverage the current system without additional investment since international usage occurred during low utilization periods for the U.S. users and they could leverage many of the same workloads (i.e., we could eliminate workload redundancy).
Although not a great sales outcome, it did call out the advantages of sharing a production system and leveraging the investment as much as possible (i.e., “sweating the asset”). Interestingly, the biggest problem we ran into was how to share the costs; the International departments saw it as an environment that already existed, but the U.S. wanted to gain some financial benefits for having made the initial investments. Despite this being the best financial outcome for the overall company (the CFO was certainly happy with the answer), it did belie one of the biggest challenges when sharing an analytical platform: how do you equitably share the costs?
A Brief History of Pricing Analytics
Historically, this pricing dilemma led many to try to quantify the costs of analytics so that an equitable model could be established. Some of those that have been tried include:
- Allocated Storage – simple to quantify, but this was always a dubious way to apply cost since storage is the least costly component (vs. CPU and I/O) for any analytical platform;
- Number of Users – although users vary greatly in terms of query counts and complexity;
- Number of Queries Run – again, easily quantifiable but certainly not all queries are alike; and
- Derived Value – if you can quantify it, you may end up punishing organizations that are successful and rewarding those that misalign their analytical efforts.
Noting all the negatives associated with the methodologies above (and there are others), as well as an ongoing general movement from capital platform investments to
software subscription pricing models, there is now a wave of interest in a pure consumption-based pricing model for analytics. At first glance, this seems like an ideal scenario: with Consumption pricing, users will be charged based on how much consumption they place on the platform – i.e., based on compute and storage resources actually used, not simply provisioned and available for use. There are many advantages to Consumption: flexible to deploy, easy to quantify, and easy to charge back to the relevant organization. And for some use cases, this is entirely true. Charging based solely on usage, however, is not a solution for everyone because it can potentially turn into a very expensive method because analytical use cases vary greatly in terms of resources consumed.
So, what is the answer to pricing methodologies for analytics? One way is to look at how it can be applied in how we manage our personal transportation expenses.
What Can Transportation Teach us about Analytical Pricing Models?
On-demand ride-sharing services have exploded in popularity because they are nearly ubiquitous, flexible (use it when and where you want), and you only pay for what you need. Users that travel frequently to different locations, nevertheless, might prefer the flexibility and potential savings found in renting a vehicle where they pay a set rate for a specific set of capacity for a specified period of time and/or miles, and more if usage goes beyond that. Finally, there is the option of buying or leasing an automobile (primarily a fixed investment) and although you have the highest upfront overall commitment, the more you travel the more cost effective this option can become.
If we were to chart these three options in a simple depiction against an X-axis of “Asset Consumption” and a Y-axis denoted as “Travel Cost”, we would see something along the lines of the following:
Although an oversimplification of the cost/benefit of these three options, it does show that based on a particular usage profile, any of these options can be viewed as being the most cost effective: On-demand is best for low usage, variable is best for moderate usage, and fixed is best for high usage. The truth is that the best option depends on what your transportation needs are and how flexible you need to be within the confines of these needs. Note too: flexibility can come at a significant cost and, at some point, pure financial practicality can far outweigh any desire to accommodate usage ambiguity. So, what is a company to do when looking to invest in an analytical platform?
Embracing a Flexible Approach to Pricing
What was not highlighted in the previous chart is that we might choose a combination of all transportation options with a mixing of models to suit our needs. This same paradigm is true with respect to how we should look at our analytical investments. In different scenarios, our ability to invest in a specific platform and “sweat the asset,” can make a lot of sense (Fixed Capacity pricing). Other times, we need a specific level of capacity, but we do not need that capacity 24x7 (Variable Capacity pricing). Finally, there are times when we need to do things where our capacity needs might be significant, but our actual usage is sporadic (On-Demand pricing). The following chart provides a high-level overview of the pros and cons associated with each of the three models as well as typical use cases where each may be best applied.
Each pricing model can make sense under a certain set of circumstances and should be an integral part of any investment consideration. Much like our choice of transportation, it depends on what we need overall as well as what we need at a specific point in time. There are other considerations besides just the financial ones when deciding on which technology to choose.
3 Key Things to Consider when Making Analytical Investments
The cloud and associated technologies feature the ability to scale quickly to meet end users’ needs. As such, many analytical vendors now have creative ways to price based on this scalability. There is a lesson to be learned from
past hype cycles, like that of Hadoop. Hardware scalability was never the issue with Hadoop and people jumped into the technology without really evaluating if it supported the types of analytical use cases necessary to drive significant business value. The cloud certainly has rewritten the rules in terms of how infrastructure is procured. Nevertheless, the ability to scale hardware (and pricing for that scalability) is only one characteristic towards solving for your analytical needs. Any evaluation in this area should keep in mind these three points:
- Don’t Punish User Innovation
Analytical innovation is about exploration and is iterative by its very nature. End users will often explore, and run into dead ends, many times before they uncover something worthwhile. In some cases, charging end users for every query (and in some cases every block of data returned in that query) might lead to a stifling of innovation
if users are constantly reminded about the cost of asking a question versus the value of the potential answer. In other words, you wouldn’t want to be penny-wise and pound-foolish when it comes to analytical exploration.
- Don’t Reward a Vendor’s Lack of Innovation
Having the ability to charge back to a department or business unit based on consumption, and having the ability to grow capacity to meet that consumption, can provide significant advantages to corporations in how they manage analytical investments. Caution should be taken when a vendor touts this as their primary capability, however, as they might not be terribly interested in investing in functionality that will enable their system to do things better, faster, or simply do more with less.
If your vendor is primarily concerned with coming up with creative ways to charge you for consumption, then they might not be terribly motivated to develop a product that will improve in performance over time. Jobs paid by the hour often take longer than those paid by accomplishment.
- Embrace a Flexible Pricing Policy
There is no one-size-fits all when it comes to pricing analytics and how it can best be applied to your organization. Even with the same set of end users, you may find a need to leverage different pricing models for different workloads. Much like the transportation analogy, many of us use all three models to get where we want to go given our specific set of circumstances. You should expect the same in how you leverage your analytical investments.
There are great things to be had by embracing and investing in analytics, especially in the cloud. Analytics is a
hot topic and companies are putting more and more of an emphasis on analytics as a competitive advantage. Gone are the days where analytical functions were relegated to the storage closet (and yes, I literally started my analytics career working in a storage closet), but instead are front and center as a ubiquitous component to nearly every business function.
Like any investment, however, there are more aspects to considering how you make these investments than just the chargeback model. It is important to have flexibility and accountability in terms of analytical investments. The question is whether to spend most of your time getting sidetracked by
outlandish claims on how a particular technology will save you money, or focusing on the value a technology can deliver by executing in a
predictable, reliable and cost effective way. Whichever way you price it, do not forget why you invested in that analytical solution in the first place – to reinvent your business…not make it easier to apply your associate