Slimming the stack and audit categories

If you’re a practitioner and a data leader, you’ve doubtlessly been faced with the reality that every new tool or capability becomes yet another new thing you need to support. Putting aside just cost, there’s a practical limit to the amount of tools you can feasibly balance.

With that said, increasing the velocity of your office requires trimming down your toolset. I kind of chuckle when I receive these LinkedIn spam messages telling me that I cannot survive without some “master and reference data solution," and "data governance activities are incomplete without this bespoke solution.”

Not only am I not going to buy a new subscription or service, I’m actively cutting the cables on the ones I have.

Slimming the stack is absolutely critical to moving up the hierarchy of data capabilities, but it does add a new risk category. You will need to continue doing the work, but it’ll involve looking at your existing toolset.

In other words, trimming your data stack involves relying on the bigger contracts to do more (ie. using Azure, AWS, GCP, Databricks, and Snowflake features).

Now the unpopular part of my post:

It probably makes sense to create a couple new audit categories:

• Vendor lock-in, to assess the risk that reliance on any single vendor becomes so big that you can’t extricate yourselves from them without substantial difficulty.
• The corollary: that a vendor doesn’t change their business model between contract periods, and you end up being forced to accept a model that no longer benefits your business.

Both of those are risks that become more critical as single packages become harder and harder to break out of.

So, your decision becomes, 1. accept a lot of small contracts that are infeasible to manage, or 2. handling a small number of large contracts that may make you less fiscally nimble.