How to measure success in an Agile organization

**Re-posted from https://www.slalom.com/thinking/how-to-measure-success-in-an-agile-organization where this article was originally published.

Any organization adopting Agile faces the challenge of figuring out how to measure success. “How will we report to executives?” and “How can we standardize reporting across multiple teams?” are some of the first—and most important—questions that pop up. Answering these questions without compromising the spirit of an Agile transformation is critical to adoption.

Why? Because every metric you put in place will generate some type of behavior, be it positive or negative. That’s why it’s imperative to exercise a vast amount of diligence when working to define your metrics.

In the world of Agile, success can look very different than it does in a traditional Waterfall, fixed-scope environment. One of the cornerstones of Agile is to work on what’s most important to the business and deliver value as soon as possible. Here’s an example: the team goes into a sprint or release and commits to completing five items. But after work begins, the team learns that the number-one priority will require much more work than expected due to new requirements or bugs discovered during development. Poor planning isn’t to blame; the team just has to make adjustments based on new information. Everyone agrees to focus on the number-one item to complete by the end of the sprint, leaving items 2-5 unfinished. They refocus, successfully complete item number one, and deliver it on time. But the team only delivers 20% of its expected scope at the end of the sprint.

To many Agile teams, this could be considered a complete success, because ultimately the percentage of original scope isn’t as critical as delivering on what’s most important. So in this scenario, the traditional measure of committed vs. completed can be a poor measuring stick.

Let’s look at what could happen if the success of this team was measured by what it completed. The team learns that good numbers are more important than delivering on the top priority. They then start padding their estimates to ensure that they have plenty of time to finish without any risk of backlash. Now you have valuable energy being wasted on navigating a process rather than getting things done. You’ve invited some of the more negative aspects of the Hawthorne Effect to come and run rampant on your project and team.

For an Agile organization to be successful, there has to be a general assumption that people are working hard and want to do their best. The role of a good leader is to enable teams to work smarter, not harder.

Here are seven traditional and not-so-traditional ways to measure the success of your Agile team.

1. Measure teams rather than individuals. A study conducted by Team Builders Plus found that “individual performance data are of less value (judging from the ratings and number of organizations tracking it) while team data reinforces collaboration and problem solving.” Collaboration is a key part of any Agile team that focuses efforts on the outcome of a project rather than a single person’s output.

2. Keep track of the percentage of the highest priority items being delivered. If your team’s focus is on delivering what is most important first, you can keep track of this trend over time. If the team is always delivering on priority two and three, but missing number one, you can use it as a coaching opportunity to help increase collaboration.

3. Measure the team’s satisfaction. You can be leading a team that is delivering at a high level, but if they’re miserable, you’re in a dangerous place. Part of a team’s success is the overall satisfaction of its members.

4. Evaluate your software. There isn’t a universal solution for determining if you have “working software.” Some of this relates back to customer satisfaction, but you could also measure if the work being done is truly solving problems—or if, in fact, it’s creating more.

5. Show the business value delivered. This may require more work up front, but if you have an idea of the value for a particular story or project, you can tell the story of the value that was delivered.

6. Provide a narrative, not just data. Why spend time collecting, packaging, and sharing data that really has no bearing on a team’s success? Sometimes even providing 3-4 sentences can give much more useful information than a pie chart. Visualization of data is definitely a good thing, but think carefully before defaulting to visualizations.

7. Measure anything the business says is important. Easier said than done, but as a guiding principal, focus on what the customer is saying is important. If they only care about 2-3 things, why give them 15?

Now you’ve got the metrics, but what do you do when ground forces at the executive level are resistant to Agile? First off, don’t react when people are critical or have a different perspective about what should be measured. You’ll be more successful showing people how Agile can enable the metrics they want.

Of course, there’s still a chance you won’t be able to reconcile the expanse between the pro-Agile and anti-Agile parties. In VersionOne’s annual Agile survey, 53% of people said an organization’s inability to change its culture was a barrier to adoption. There’s no silver bullet here. Another revealing statistic from that same survey was that 35% said trying to fit Agile into a non-Agile framework was a barrier to adoption. The research speaks for itself:you can’t always expect the metrics that work for a Waterfall project to apply to an Agile project.

The workplace is full of spreadsheets and databases, amassing ever-increasing amounts of information. With that, the definition of success—specifically in information technology-centric groups—has changed dramatically over the last 10 years. I’m a firm believer that success in its purest form can be measured by how happy your customer base is with the product or service you provide for them. That’s why the Agile Manifesto emphasizes “individuals and interactions over processes and tools.”

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s