A lot of 'experiments' aren't.

What we often call “resistance to change” is really resistance to bad or poorly considered change. The word “experiment” turns up a lot in these moments.

I was talking to a friend about her work with a new-to-her team of managers. She had proposed “doing an experiment” with a change to how their teams were working, and they were all resistant.

The change she was asking for was small and incremental – there wasn’t any misplaced “change agent” or “boil the ocean” energy behind it. Still, the quiet ones just stopped interacting and the more vocal ones began listing all the reasons their teams weren’t up for yet more change. None of it was very concrete, and it left her thinking she’d picked up a team of people who just didn’t want to change how they work.

I remember running into the same problem with a new-to-me team.

It was frustrating because I’d felt like we were off on a good foot: They knew me by reputation before I joined and there was a foundation for trust. But when we came across a hitch in an old process that hadn’t changed with the business around us, I breezily said “well, why don’t we try an experiment” and proposed a potential improvement. I could see them shut down.

My first thought was “what on Earth is the problem here,” so I asked and they were forthcoming:

For years, “experiment” had been shorthand for “let’s just do this thing and see what happens.” Leaving aside the definitional problems Webster might raise, it came with other baggage.

• The “experiments” never seemed to be testing anything in particular. You could just as easily replace “let’s try an experiment” with “let’s throw some spaghetti at the wall.”
• They were never bound by time, success criteria, or retrospection.
• Because of that lack of structure, they’d eventually melt into the practice of the team whether they improved anything or not, becoming zombie processes or unexamined “that’s just how we do it” norms.

I had my own damage over undefined “experiments,” so it wasn’t too hard to wheel it back. They were right to be suspicious of the word, and I admitted that, then I laid out for them the same thing I suggested to my friend:

First, ask the question “what problem are we trying to solve?”

In a room full of people, you’ll probably get a few versions, so asking helps get the team aligned. As a manager or leader, you’ll give yourself an opportunity to unwind a few assumptions you’ve already made.

Second, decide how and when you’ll know the change had its intended effect.

• If it’s something you have metrics for, that’s great. If it’s not, keep any potentially helpful metric you identify light. You can go full Zeno’s Paradox over metrics.

• If it’s in the realm of the more qualitative give the change a little time, then periodically survey the team. 1:1s are better than team meetings for this. Just keep track of sentiment and lay out your observations when you lead the review.

Sometimes you’ll get immediate feedback, other times – if the change is going to touch a lot of things or lives in a complex process – you’ll have to give it more time. Leave room for a few repeats either way:

‘Once is happenstance. Twice is coincidence. The third time it’s enemy action a pattern.’
—James Bond

Third, set a concrete date to review the change and make a decision about it:

• It satisifed our success criteria, let’s keep it.
• It didn’t satisfy our success criteria, but there are ways we could adjust it. Let’s think about those changes and re-up the change for another period.
• It didn’t work at all. Let’s discuss why, then go to the next candidate.

Then stick to that decision.

If you decide to end it, end it, tell the team it is done and keep any documentation you have about what happened, how it worked, and why. Some teams keep a “decision registry” for this kind of thing. Other teams have an RFC process of some kind. However you do it, make it easy to find your notes: Future new team members or even your successor will thank you.

It’s important to stick to that decision because people who are hungry for change – or even just amenable to it – will lose their enthusiasm if they perceive that each “experiment” will become yet another geological layer of unexamined process or practice to contend with.