Agile By The Book?

Recently I was asked to introduce a process to a very senior team that joined our company not long ago. The team did not have much process established in the first place, so the goal was not necessarily to transition from traditional (waterfall) to Agile/SCRUM, but rather to establish a process. Having a very talented and senior team, with decades of software development experience presented a challenge: should we go ahead and implement Agile “by the book” or should we consider a different approach? Additional challenge was multiple other changes happening in the group, in terms of the personnel and the development infrastructure. The risk of imposing too many changes and too rigid process was very real and could impede progress rather than facilitate it.

Armed with experience of introducing Agile “by the book” to 6 teams in the last two years, and knowing the challenges that specific team would have I decided to try a different approach. First, I have done Agile/SCRUM training, similar to what I usually do for a team that is about to begin their SCRUM implementation. At that point I stressed, that this is an overview/introduction only, and by no means a prescription to how we will implement Agile. In addition I have communicated in advance, that the entire team will be involved in designing their own process, building on some of the practices they learn from the Agile overview. Setting this expectation has helped a lot: there was no antagonism towards the attempt to introduce yet another change, after the stage was set this way. However, there was also a risk that when the time comes to design their own process they would take an easy way out and will chose only few practices to implement, not realizing the real benefit of implementing SCRUM.

The next day after an overview we got together to design the process to be implemented. The guiding principles were the following:

1. We have new product development ahead of us with time to market being most important. Practices that we choose should help us focus on working software at all times and allow for making frequent changes and getting frequent feedback.
2. We want to implement enough process to benefit from what Agile/SCRUM has to offer, without introducing too much process overhead.
3. No decision is cast in stone: the team would inspect and adapt often.

With these guiding principles agreed upon with the entire team, we went ahead and examined each SCRUM ceremony, artifact and role and have designed our new process.

At the end, the team and I were very happy with where we’ve ended, in terms of amount of the changes the team will need to absorb, potential benefits and amount of overhead.

If you were to ask me 2 years ago whether the approach I described above is recommended, I would probably say “no”, because all we’ve heard from Agile practitioners at that time was “start by the book, then adjust as you go”. In retrospect, I can see how both of the approaches can work well, and more importantly, each situation has its own context and no approach should be applied blindly, without carefully analyzing that context.

Are we there yet?

Our entire Product Development organization has transitioned to Agile. Officially. Something tells me that I’m not ready to declare victory just yet. It’s extremely tempting to say “Yes” when people asking me whether we are Agile now. My answer is “we are working within Agile framework” but we are not Agile yet.

All 5 teams are gaining muscle memory, getting comfortable with Scrum terminology, roles, ceremonies, artifacts etc. But how do we know whether we are Agile or if not, what needs to be done to reach desired level of agility? That’s where the assessment  comes in.

I’ve seen several approaches to Agile assessments. Ones that I discarded focused on process rather than on results and the end results. Understanding that Agile is the means to an end, but not the end itself I have designed an assessment that focused on Agile values, principles and benefits. The idea here is to observe each team for a period of a single Sprint and interview team members as well as observe the process and evaluate the quality of outcomes. Each team would then receive detailed assessment of their “Agile Maturity” and will receive a list of suggestions for improvements. These suggestions will be discussed with the team and SMART action items will be developed.

The assessment that I propose consists of 6 composite Dimensions:

1.       Embracing change and feedback
2.       Predictability and Planning
3.       Learning and Adapting
4.       Working and Stable Software
5.       Technical Debt Control
6.       Teamwork and Communication

Each dimension has several Indicators with have different weights that are calculated based on observations and interviews (multiple questions/topics for discussions are associated with a single Indicator).

While Agile Maturity Assessment described above is internal to Product Development organization and is a team by team exercise, we also will be performing a Product Development Maturity that will focus on the vision and the goals we’ve set a year ago (see my earlier post about Vision).

Pattern for Agile Adoption

After leading a transformation for over a year I'd like to make one point. People often take Agile Transformation too lightly and think about it as yet another process being introduced into the Engineering organization. What I've experienced is that Agile Transformation is not much different than any other large-scale organizational change, and as such deserves to be managed accordingly. One has to consider that moving to agile is a significant mental shift in the way people approach problem solving and performing their daily jobs. People that are the core of the change process need to be well prepared and allowed to fail as they learn.
For us it took 1 year from introducing the idea to the management to starting with the 1st sprint. At times it seemed like we are dragging our feet, but in retrospective I can say it was worth investing all the time we spent learning the subject, talking to practitioners, analyzing applicability to our context and identifying obstacles, challenges and plans of attack. This due-diligence has paid off when we started the project.
If I would need to define a successful pattern for planning and managing Agile Transformation here would be the major steps in that pattern:
1. Get a focus group to self-learn Agile. The group has to include at least 1 member from each product development team (dev, qa, pm, ...)
2. Create a shared Vision (see my post “Vision matters”)
3. Focus group meets with practitioners, members attend events and share what they’ve learned
4. Information is digested, obstacles identified and plan of attack for these obstacles being developed
5. Propose a recommendation (go/no go)
6. Get Executive buy-in
7. Start with education for all relevant locations. I've held initial 2-day workshop for everyone, then invited coach to do SM and PO classes.
8. Spinoff teams on obstacles, challenges, processes identified in #4 above.
8. Hire a coach (see my post “Hiring a coach”)
9. Find a good pilot project (it's another topic on how to choose the right one)
10. Have coach facilitate 1st sprint
11. Communicate to the world about the progress

When Offshore meets Agile

This post belongs in both of my blogs “Leading Agile Transformation” and “Succeeding with Offshore Management”, so I’ll publish it in both…

When companies look at Agile the question of location often comes up: people are afraid to implement Agile with offshore teams due to the distributed nature of the teams. Of course it’s great when you have face-to-face communication, but I see no problem with implementation of Agile in distributed environment (ok, maybe some challenges).

There was a research done on the subject of communication in different office settings, and what they found is that best communication happens when people are sitting in the same area or in adjacent cubicles. Next best setup is an offshore model and the worst setup is when people are located in the same building but not in adjacent areas. The difference in communication efficiency was significant between 2^nd and 3^rd categories, but insignificant between 1^st and 2^nd. One of the reasons for these findings could be that people in the 3rd category think they are collocated and do not need to invest in improving communication channels and communication efficiency. On the other hand, folks in distributed setting are well aware of the challenges of communication and are investing a lot of effort in constant improvements.

Agile fosters frequent communication and helps distributed teams to advance to the next level of efficiency. Yes, there are challenges that are exacerbated with Agile: more communication requires more discipline and sometimes changing the ways we communicate and may require additional tools to support it. This is not different from any other problem that Agile helps to put a spot light on and that requires resolution. Agile will emphasize what we as an organization need to focus on to make sure required communication levels are met.

One thing we’ve been struggling with before was choosing the right timing for onsite visits: the environment was pretty chaotic and it was difficult to plan for a visit and to tie it to a certain event (planning/design review, etc). With clarity of SCRUM it’s obvious how we can leverage the framework to plan for visits: Release/Iteration planning and Demo sessions are a natural fit for this.

As part of our Agile transformation I’m making sure to have not only Agile enthusiasts on  the other side of the ocean, but someone in coaching capacity to augment my efforts on site and help teams when I’m not available due to time difference or any other reason.

Executive Commitment

“Is Commitment a direction to be pursued as long as it works for you or a direction to be pursued until it works for you?” –Anonymous

Getting executive commitment is crucial for Agile transformation success. The process is long and road mines will await you surely. It will be of utmost importance to have the executive team still supporting the initiative when things get tough and the pressure to get product out increases.

I have found that it’s not hard to “sell” Agile to the executive team: what is there not to like? The value proposition of Agile sells itself and all CEO hears from others outside the company is “you have to go Agile”. So on the face of it everything looks good and executive team is all in agreement that Agile is the way to go.

When I presented Agile to execs few months ago I took a different approach. Well, I had to do the selling part, not without it, but most important I have described all the challenges and obstacles we anticipate having in our way to become Agile:

1. Shared resources (PO/UxD/Architects),
2. PO availability to the team (considering other responsibilities of Product Manager)
3. Leading virtual teams
4. Too often/uncontrolled changes in direction
5. Inability to decide on priorities in timely fashion
…and the list goes on

One thing Agile does quickly and without a facelift is exposing existing problems in the organization, and when a company embarks on Agile transformation path these problems have to be dealt with, breaking status quo and being painful to resolve at times. For example, if the company has a problem on deciding on priorities prior to Agile, the problem will not be resolved just by going Agile. Agile will show the way if you will, but the company will have to address the issue immediately.

Although I still got a green light from execs, the real test of executive commitment will be when the push comes to shove.

Ready? Set Goal!

For task force to be successful it has to have a clear goal. The goal is to assess feasibility of the change. This means the group will need to answer following questions:

Can agile address our problems? Remember, we are not doing the change for the sake of the change.

Are the benefits tangible? You should be able to look back and show that specific problems got addressed and you better off than you started.

Do benefits outweigh risks? Agile transformation is a change like any other organizational change, you should expect turnover associated with it, going slower before going faster and need to consider that many companies fail attempting to make such change.

Should we go by the book or pick and choose specific practices? There will be people (yourself included) who will be tempted by such approach, as it seems to mitigate some risks, but this needs to be carefully studied.

It’s important to understand that Agile is not a one size fits all, so no prescriptions for success could be found in the pure theory. Any organization attempting Agile transformation needs to become creative about addressing many challenges awaiting it. One great way of becoming creative is engaging with Agile practitioners and coaches, learning about their challenges and the ways they have addressed these challenges. Of course their situation and context could be very different from yours, but learning from their successes and failures could be very helpful.

Task Force (or "Back to School")

Congrats! You’ve found the right time and convinced your boss that the change is needed. What’s the next step? Ideally you’d like to have the entire Product Development organization buy in to this idea and start implementing the change right away. Unless your organization consists of two guys writing software in garage, this needs to be carefully planned. Good place to start is to create a task force: a group of people that would learn about Agile methodologies and frameworks, will meet with Agile practitioners and coaches, will assess the feasibility of applying Agile to your organization and will recommend a course of action for the company. There are few key factors to the task force’s success. First, you need to have representation from all the groups primarily affected by the change (Product Management, Development, QA). As you select task force members try to include people who may resist the change the most and are key to your organization (senior managers, key engineers etc). Change initiatives, as just as they could be, may cause higher rate of turnover and no one wants to lose key talent as the change is being implemented. Having key people and potential resisters on your team will help guarantee the success of the transformation to be. Next, make sure that you include people who are seen as leaders in their respective groups and would be able to communicate the vision and help get buy-in to the change. In general, smaller teams will get things done faster, so team of up to 5 people should suffice.

Good task force should have pre-defined goals, timeframe and approach for achieving these goals.

Failure? To Launch!

Sometimes we read or hear about some new direction in the way people do business and tell ourselves: "Only if we could implement this in our company!". Then we rush implementing this new way of doing business at 100 mph and find ourselves in worse spot than we started.

So how one makes the change, if he/she feels strong that this change is a must?

Well, one of the obstacles is awareness of the problem that the change needs to address. After years of doing things certain way many people might become blind to the root cause of problems they constantly facing and assume that these problems are inevitable. This leads to attempts of patching the current processes and creating new processes, often adding significant overheads but not addressing the core problems. Surprisingly (or not so…), this approach is usually acceptable by most stakeholders as it is associated with much lower risks than the complete makeover.

First step in any change initiative is raising the awareness of the problem and the urgency for a change. Now, no one will ever bless a change initiative if the sole reason for the change is making the change. Unfortunately, in many occasions a “systematic failure” needs to occur for everyone else to see that they way things are being done now cannot scale and for the company to thrive, change must happen. At that point people might not know what exactly needs to be changed, but they already know that they cannot continue doing business as usual. For the change evangelist this creates an opportunity not to be missed.

If not successful otherwise, use the momentum of the “systematic failure” to raise the awareness of the problem and urgency of a change. Your management will gladly let you lead the task force charged with analysis of the current methods and proposal for the change.