How to measure success in an Agile organization

**Re-posted from https://www.slalom.com/thinking/how-to-measure-success-in-an-agile-organization where this article was originally published.

Any organization adopting Agile faces the challenge of figuring out how to measure success. “How will we report to executives?” and “How can we standardize reporting across multiple teams?” are some of the first—and most important—questions that pop up. Answering these questions without compromising the spirit of an Agile transformation is critical to adoption.

Why? Because every metric you put in place will generate some type of behavior, be it positive or negative. That’s why it’s imperative to exercise a vast amount of diligence when working to define your metrics.

In the world of Agile, success can look very different than it does in a traditional Waterfall, fixed-scope environment. One of the cornerstones of Agile is to work on what’s most important to the business and deliver value as soon as possible. Here’s an example: the team goes into a sprint or release and commits to completing five items. But after work begins, the team learns that the number-one priority will require much more work than expected due to new requirements or bugs discovered during development. Poor planning isn’t to blame; the team just has to make adjustments based on new information. Everyone agrees to focus on the number-one item to complete by the end of the sprint, leaving items 2-5 unfinished. They refocus, successfully complete item number one, and deliver it on time. But the team only delivers 20% of its expected scope at the end of the sprint.

To many Agile teams, this could be considered a complete success, because ultimately the percentage of original scope isn’t as critical as delivering on what’s most important. So in this scenario, the traditional measure of committed vs. completed can be a poor measuring stick.

Let’s look at what could happen if the success of this team was measured by what it completed. The team learns that good numbers are more important than delivering on the top priority. They then start padding their estimates to ensure that they have plenty of time to finish without any risk of backlash. Now you have valuable energy being wasted on navigating a process rather than getting things done. You’ve invited some of the more negative aspects of the Hawthorne Effect to come and run rampant on your project and team.

For an Agile organization to be successful, there has to be a general assumption that people are working hard and want to do their best. The role of a good leader is to enable teams to work smarter, not harder.

Here are seven traditional and not-so-traditional ways to measure the success of your Agile team.

1. Measure teams rather than individuals. A study conducted by Team Builders Plus found that “individual performance data are of less value (judging from the ratings and number of organizations tracking it) while team data reinforces collaboration and problem solving.” Collaboration is a key part of any Agile team that focuses efforts on the outcome of a project rather than a single person’s output.

2. Keep track of the percentage of the highest priority items being delivered. If your team’s focus is on delivering what is most important first, you can keep track of this trend over time. If the team is always delivering on priority two and three, but missing number one, you can use it as a coaching opportunity to help increase collaboration.

3. Measure the team’s satisfaction. You can be leading a team that is delivering at a high level, but if they’re miserable, you’re in a dangerous place. Part of a team’s success is the overall satisfaction of its members.

4. Evaluate your software. There isn’t a universal solution for determining if you have “working software.” Some of this relates back to customer satisfaction, but you could also measure if the work being done is truly solving problems—or if, in fact, it’s creating more.

5. Show the business value delivered. This may require more work up front, but if you have an idea of the value for a particular story or project, you can tell the story of the value that was delivered.

6. Provide a narrative, not just data. Why spend time collecting, packaging, and sharing data that really has no bearing on a team’s success? Sometimes even providing 3-4 sentences can give much more useful information than a pie chart. Visualization of data is definitely a good thing, but think carefully before defaulting to visualizations.

7. Measure anything the business says is important. Easier said than done, but as a guiding principal, focus on what the customer is saying is important. If they only care about 2-3 things, why give them 15?

Now you’ve got the metrics, but what do you do when ground forces at the executive level are resistant to Agile? First off, don’t react when people are critical or have a different perspective about what should be measured. You’ll be more successful showing people how Agile can enable the metrics they want.

Of course, there’s still a chance you won’t be able to reconcile the expanse between the pro-Agile and anti-Agile parties. In VersionOne’s annual Agile survey, 53% of people said an organization’s inability to change its culture was a barrier to adoption. There’s no silver bullet here. Another revealing statistic from that same survey was that 35% said trying to fit Agile into a non-Agile framework was a barrier to adoption. The research speaks for itself:you can’t always expect the metrics that work for a Waterfall project to apply to an Agile project.

The workplace is full of spreadsheets and databases, amassing ever-increasing amounts of information. With that, the definition of success—specifically in information technology-centric groups—has changed dramatically over the last 10 years. I’m a firm believer that success in its purest form can be measured by how happy your customer base is with the product or service you provide for them. That’s why the Agile Manifesto emphasizes “individuals and interactions over processes and tools.”

Advertisement

“That’s Not Agile”

right_way_and_wrong_way_sign-other_zps8c090af3

If you work in an organization that utilizes one of the many agile methodologies, you have probably heard the statement “that’s not agile”. It could be about a proposed process change, or a critique of an existing way of doing things. It baffles me to no end when people get lost in the weeds about if something they want to try as a team is agile. This is unfortunately very prevalent in Agile Coaches and Scrum Masters who have a lot of experience.  Since agile is an emerging idea that is really starting to catch some momentum, some who have been advocates for a longer period of time can be standoff-ish about how to do things.  This should be a huge red flag, because there are so many variations of agile and not every organization is the same. You can know all the right things, but completely fail at helping a team adopt agile. That is why it is very important to be flexible, and not have a “better than” mindset.  The principles of agile/scrum are meant to be a guide, not a law that must be perfectly adhered to.  I’m not suggesting you should throw all caution to the wind and try any and everything, because “everything’s agile!”  No.  What I am primarily addressing is the way in which this topic is approached.

So, who’s right?  Maybe saying “that’s not agile” can be valid.  Or maybe we’ve lost sight about what agile is all about.  Let’s just take a quick peek at the Agile Manifesto for a refresher on what it’s all about:

Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan

To be brash: If someone tells you “that’s not agile” and it’s something that could enable one of the 4 agile main points of the manifesto, I would seriously question how well they truly understand agile.  And agile by nature is something that I believe will continue to evolve as people uncover better ways of doing things. (sound familiar?)

If your organization is looking for an Agile Coach or Scrum Master or you are just trying to improve your skills in that realm, here are some suggestions for what not to do, and what to do.

  1. Don’t try to impress anyone with your agile credentials.  Do try to identify things that have been successful on similar teams/organizations.
  2. Don’t be preachy when trying to present new ideas.   Do suggest ideas with a humble and flexible attitude.
  3. Don’t rely solely on past experiences of what worked for other teams.  Do spend a lot of time listening to people’s needs an challenges.
  4. Don’t react when people are critical of agile.  Do find a way to tie what people want back to agile principles and show them how they could enable those things.
  5. Don’t press your own agenda.  Do thoroughly understand how you can support management’s vision.
  6. Don’t treat the agile principles as a law.  Do use them as a guide for making continual improvement.
  7. Don’t be too critical of a team’s existing processes. You never know how much hard work they have put into building what’s there. Do make a point to try to build on existing foundations, and position it as an improvement rather than tearing down.
  8. Don’t be too quick to judge what will work best for  a team.  Do ask a lot of sincere questions and follow the breadcrumbs of how things became the way they are.

 

Hopefully the next time you want to say “That’s not agile” you can remember the big picture and help your teams find the very best solution.  What things are important to you as an agile proponent in these regards?  Do you have any good stories to share?