My apologies for not posting anything last week.  I was at a conference.

Much of the conference was about how in-house counsel measures the performance of outside counsel.

The mantra of the day: If you can’t measure it you can’t manage it.

Really?

What is the metric for confidence that your counsel (or your doctor or your veterinarian or your mechanic or whatever) will achieve that best outcome possible?  What is the metric for confidence that your service provider understands your issues and your goals?

There isn’t one.

Now I am a lawyer (this is Big Data and the Law after all – see the “About Me” tab) and this might seem like whining, but while it is an expression of annoyance it is also a more general expression of a concern about Big Data and in particular about how subjective agendas can affect the search for objective data.

Proving our economic worth is something we all struggle with, but that is not an excuse for doing a bad job in the effort to justify our existence.  Let’s consider an example of this trap – outside the context of the legal profession.  In large system development and installation projects it is common for the customer to engage a third-party consulting firm to help oversee the project and to manage the customer’s concerns as the project moves forward.

The problem with this is the third-party consulting firm is a cost that has to be justified. What happens if the project goes great and there is no need for the third-party consulting firm to raise any concern to the customer?  What happens is that the expense of the third-party consulting firm seems wasteful.

How does the third-party consulting firm avoid the appearance of irrelevance?  By finding problems whether there are any or not.  That’s the implicit metric for the value of the third-party consulting firm.   The more problems found, the more the consulting firm looks like a good investment.  (Oddly, often the same consulting firm participates in the selection of the vendor.  You wonder why customers don’t ask them why the consulting firm thought the vendor was a good choice to being with.)

This is stupid for all concerned.

The metric I use for evaluating the worth of my car insurance is not how many accidents I’ve had covered.  It’s the cost of the insurance in relation to the market, and the intangible value of knowing that the claim process will be smooth if I do have an accident.  Which, unfortunately, is an intangible value I have had the opportunity to measure.

Back to the system development and installation project.  The customer for that project should understand not only that the consulting firm’s participation is in part a form of insurance (yes, there should be other value adds but let’s keep this simple), but that creating a perverse incentive to create problems multiplies the costs of the consulting firm’s involvement.

Back to the conference.  Some of the discussion of managing legal cost made sense.  For example, there is a logical relationship between the cost of legal services and the sophistication of the subject legal work.  But one metric used by a presenter is a very good example of what I think is troubling.

This individual’s metric for success in negotiating agreements is the number of times his businesses gets the other party to accept his form agreement rather than the other party’s form.  That’s a goofy metric for any number of reasons, but let’s point out a couple of things for now.  That metric doesn’t measure cost or substance.  It might take ten times as long to come to finality on an agreement that is based on my form document than would have been the case starting with your document.  Are there appropriate limitations of liability?  How about warranties?

Let’s remain aware of the fact that we care about a great many things that are not susceptible of objective measurement.  Also, when someone tells you that you can’t manage something you can’t measure – ask them what the hell that’s supposed to mean and let me know what they say.