Can we scale it? Yes we can!

Any company with enough focus and dedication can sustain a single pilot team. Ok, may be not any, but definitely many. When time comes to scale beyond the single pilot team various organizational impediments become evident and the real test for organization’s ability to go through with necessary changes begins.
Almost every process under the sky has to change and adjust to support multiple teams working on multiple releases simultaneously: Defect Management, Source Code Repository management (branches, builds, etc), UX, Development Documentation (design specs and other artifacts). Before we started with the first pilot team we have identified all these processes and created spin-off teams each focusing on a single process and identifying the way that process would need to adjust to meet the needs of the Agile environment.
The biggest concern with scaling Agile is shared resources: POs, DBAs, Architects, UX Designer, Release Manager, Documentation, and the list goes on. This was always our top concern and therefore it became a prerequisite to scaling beyond the pilot team.
1st shared resource concern that we’ve addressed was a Product Owner role. With Product Managers spread thin already it was obvious that without some scaling mechanism in place the Agile wouldn’t work. We have introduced the concept of Product Owner Proxy to augment the Product Owner role. PO Proxy can be anyone from the team who knows the product well and can make decisions 50% of the time on PO’s behalf and answer team’s questions in PO’s absence. We’ve seen this role being filled by architects, QA and developers in other companies successfully.
2nd shared resource concern that we just started to address is UX Designer. With the constraint of not being able to hire another UX Designer we had to become creative to address this issue. From what we’ve heard from other people it isn’t practical for a single UX Designer to support more than 2 Agile teams. Based on what we’ve learned during the pilot project it became evident that in our environment it will be impossible to support more than 1 team… 1st thing we’ve done was to identify the minimum process changes required to scale: identifying user stories with UX impact in advance so that UX Designer could see 2-3 sprints ahead and know what might land on his plate. Next, we’ve analyzed all the responsibilities and deliverables of the UX Designer and categorized them into 4 categories:
a)Has to be done by UX Designer
b)Can be delegated to another team member under close supervision
c)Can be delegated to another team member under loose supervision
d)Can be delegated to a PO
This analysis has shown that even though UX Designer has a lot on his plate not everything has to be done by him and in many cases can be offloaded even if under supervision to other folks.
Of course it doesn’t stop there: with multiple Scrum ceremonies for each team, UX Designer’s time management will also become an issue. Each organization should identify how to manage that time best with minimum waste on one hand, and without losing the UX insight on the other.
After all, hiring a new resource is not always an option. Even if hiring will have to be part of a solution, in my opinion it makes sense to assume you can’t hire and see what you can come up with. Best ideas come to mind when constraints are present.
p.s. To see how we are scaling the Offshore Manager in the new Agile world read my post at the offshore blog:  http://offshoremanagement.blogspot.com/2010/12/streamline-communication-accross-pond.html

Beginning with Agile in Distributed Environment

Here is my talk at SoftServe Innovations Conference, Florida, October 2010. In this talk I describe our experience to that moment. If you've been reading my blog posts from the last two years you'll hear the real live application of most of the concepts mentioned in my blog.

Daily Stand-ups: Workshop

Recently, after analyzing how we are doing on our daily stand up meetings I decided to run a small coaching workshop with the team. The purpose was to emphasize the value of a daily stand up meeting and figure out how to improve. In my opinion great place to run such workshops is a sprint retrospectives meeting. Everyone is already in the "learning mode" which makes the time right.
Here is a summary of the workshop:

• Open the discussion asking the team what value stand-ups have in their eyes. You will be surprised with the answers. Some folks will see certain benefits while others will see no benefits at all.
• Ask the team what the purpose of stand-ups is?
• Explain that daily stand-ups are for the team to:
• Daily sync-up: Be aware of the overall status (what's on track, what's behind, what's blocking, what the risks are, what new tasks were identified)
• Set direction and focus
• Perform a daily to weekly planning
• Share observations on how do our stand-ups look right now (our example):
• What I worked on, What I will be working on
• Some technical details of individual tasks
• Occasional planning during stand-up
• Occasional planning post stand-up
• Elaborate on how you'd like to see our stand-ups going forward:
• A place to self-organize on daily basis: For The Team, By The Team
• What I have accomplished since last stand up
• What I plan to accomplish by the next stand-up
• What needs to happen in order for me to accomplish the task (what blockers need to be removed, what coordination with others has to happen)
• This does happen on occasion, so I'd like to emphasize the value of this kind of stand-up.
• Problem solving: Parking Lot for post stand-up discussion
• Move from "Story telling" to "Headlines"
• Next move on to discussing the actual examples. Story telling exercises are the best because they create mental hooks that help people relate to these stories in the future easily. For each example ask the team following questions: 
• What is good about this stand-up meeting example?
• What can be improved?

Examples:
Show
Hide


Example 1
John: Since yesterday I was working on the new report; Today I will continue working on this report; Nothing is blocking.
Mary: I have nothing to test in this sprint yet, but I have some bug retests so this is what I've been doing pretty much in the last few days and planning on continuing with retests until User Story 123 is ready for testing.
Avanti: Yesterday I was fighting the Web Server configuration issue. Eventually I've found that there was an issue with database connection polling settings. Basically there was not threads pooled and this was affecting the throughput and therefore end user response time. I have found several articles and forum discussions about it and have left a message in few threads. Just few minutes before the stand up meeting I've got a response from some guy with reference to Weblogic article about suggested threads defaults in development vs. production.


Example 2
John: Yesterday I was working on a new report. I've implemented the UI based on the UX that Ivan designed and started implementing the query. This was a big task and I wasn't supposed to complete it yesterday. My plan for today is to complete the query and to complete the report. Mary should be able to get a first look at it later tomorrow.
Mary: Since yesterday I haven't done any work on this project since there is nothing ready to be tested. From what I just learned, John might be done with development of the new report by tomorrow, so today I will write the test cases and will review them with John.
Avanti: Yesterday I was fighting the Web Server configuration issue. Eventually I've found that there was an issue with database connection polling settings. I'm not blocked anymore and will plan on completing the configuration by tomorrow.


Example 2 - the next day
John: Since Yesterday I've completed the new report and have reviewed the test cases with Mary. Today I will be working on a spike about new authentication mechanism and plan to spend 2 days on it. By tomorrow I plan on reading framework documentation and setting environment.
Mary: Yesterday I've spend most of the time on bug retests and also had test cases written and reviewed them with John. I'm waiting on a build to perform the actual testing. I expect that I will have some time to look at any other completed functionality so folks, please let me know if there is anything for me to look at.
Avanti: Yesterday I've completed the web server configuration and it's ready to be replicated to our staging areas. Today I was planning to start working on US 234, but I need someone to bring me up to speed on it as I was on vacation during grooming and planning sessions.


Example 3
Alex: Yesterday I've completed the last task on US 323 which was to update the design documentation. Today I'm going to switch to US 454. No blockers.
Garry: Yesterday I continued working on US 323 and I keep getting these runtime errors that didn't occur even few days ago, so I'm pretty much stuck. Today I'll continue debugging this.
Jose: Yesterday I was working on the new interface to the provisioning system. I have completed this interface and will call a quick meeting to show it to few folks: it's important that we'll get it right, so I'd like to get some feedback early. Later today I have several meetings so I won't be able to complete much by tomorrow, although I'm planning to at least start writing the detailed design for the upcoming US 676.
Cathy (Scrum Master): Garry, what help you might need to move forward with this task?
Garry: I'd like to have another set of eyes with me looking at this issue as I'm going crazy here.
Alex: Garry, let me help you out here. I think it makes sense to declare victory on US323 before looking into next user story.
Garry: That would be great!

Complacency is evil

6th sprint is on and it seems like Agile framework is being followed and we are doing everything we should. Every retrospective we identify that one thing that we are going to improve going forward and planning on improving it in the next sprint.

1^st problem we are running into is constant deprioritization of improvement tasks over tasks related to the value bearing stories. The team doesn’t seem to have a problem with it initially as they see delivery of committed story points as a top priority for them during the sprint. The end result is not obvious at first, but few sprints later it becomes more evident: the improvement backlog starts forming and every sprint we have a carry-over of the improvement items.

2^nd problem is an increasing focus on tactical improvements that are the obvious obstacles to progress, while Agile values in general and SCRUM framework values in particular are losing focus and this is not obvious to the team. Our sprints look like small Waterfalls with massive acceptance at the end (usually during the last day of a 3-week sprint) and yet at retrospectives the team doesn’t pick this area as worth improving. Our daily stand-ups are all about the status and slew of technical details, and not about the daily planning. There are some good exceptions, and we do have post-stand-up planning meetings occasionally, but the actual stand-ups are not delivering the value to the team as they should in my opinion.

My plan is to hijack the next retrospective session and run it in a completely different manner. I’m going to turn it to a learning session where we will get abstracted from the tactical improvements and will get back to the framework values and will place the improvements we identify at the top of the backlog. One of the reasons we have got to this point is complacency that we all fall victims to at some point in time. That’s the reason to have a dedicated person (beyond the SCRUM master) to frequently inspect the way teams work and identify these complacency smells.

Knowing the path

In “Made to Stick” Chip & Dan Heath talk about “The Curse of Knowledge” as a term used to describe a situation where a person has a lot of knowledge on a subject, but has difficulty to assimilate it among his peers, or coachees. In “The Matrix” Morpheus and Neo exhibit classic coach – coachee relationship, where Morpheus asks the right questions at right times helping Neo to discover the answers instead of giving them out. During the past 2 years, and especially during last 3 months since the Agile pilot start I have experienced “The Curse of Knowledge” first hand and I saw how difficult it is to hold my tongue when the answer is obvious to me. I didn’t succeed to hold my tongue all the time and it was evident that when I did – the answers took longer to surface, but the effect was great: the sense of ownership and commitment from the group was much higher. It really seems easier that it is… I’m seeing every day my colleagues, whom I would define as excellent coaches, struggling with holding the tongue and rushing to give the answers right away.

With Agile mantra of self-organization, where managers “give-up” the decision making power delegating it to the team, the requirement for this new behavior becomes more evident and we all start paying more attention to the decision making process in the Engineering organization.
Here is an example. In the last sprint we had 3 user stories being worked on. At the sprint planning I have mentioned that the way to attack these 3 stories is one by one and not to work on all 3 at the same time (to the degree possible with all the constraints and dependencies). This idea seemed redundant and everyone was sure that if they work in the priority order of their tasks this will be achieved automatically. Few days before the sprint end the team has realized that none of the stories is likely to make it. This time, instead of bringing this idea up again, I asked few questions that have leped the team to come to this idea themselves. One team member suggested to abandoning the bigger story that had no chance in getting done in favor of other two stories that could be completed with the team effort. Another team member suggested next time the team attacks one story at a time – and this seemed reasonable. Eventually, we’ve attacked the last two stories with  as real team effort and got them accepted in time. I could have said “I told you so!”, but this of course would be counterproductive. When the team has come to this realization by themselves they felt ownership of this idea and were committed to try it again in the next sprint.

How to stay sane when the world is changing

Agile lives up to its promise to put a spot light on organizational problems and inefficiencies. In our focus group meetings we’ve identified several processes that will have to change once we move to Agile. The discussions on these processes have accelerated with first Agile pilot project kick-off as the new team had to know how to deal with defects, how branches and build environment will support Agile environment, what Definition of Done is and how development artifacts are created an managed. 

We approached this by creating spinoff teams that consisted of folks from different teams and locations and were lead by Agile Focus Group members. Each team spend some time on researching their corresponding subject and came up with proposal for next steps. These proposals were presented to Agile Focus Group and actual action plan was put in place. In addition to being very effective way to address many issues in short period of time, this approach presented an excellent opportunity to be engaged in the Agile transformation to the team members who otherwise wouldn’t be involved in the transformation for several months. In addition, the energy and momentum were maintained at high levels, which was essential for keeping the program going at the good pace.

While changes to existing processes will be driven by problems uncovered and challenges presented by Agile, there will be much temptation for modifying the Scrum framework as certain things will not make sense initially.

A word of caution when making changes to the framework. I particularly like the following quote by Ken Schwaber “I estimate that 75% of those organizations using Scrum will not succeed in getting the benefits that they hope for from it… The intention of Scrum is to make [their dysfunctions] transparent so the organization can fix them. Unfortunately, many organizations change Scrum to accommodate the inadequacies or dysfunctions instead of solving them”.

One needs to be careful when considering adapting the framework to their own context. In my opinion, its paramount to keep an eye on Scrum values when making changes, and the likelihood of success will be much higher.

Pattern for Agile Adoption

After leading a transformation for over a year I'd like to make one point. People often take Agile Transformation too lightly and think about it as yet another process being introduced into the Engineering organization. What I've experienced is that Agile Transformation is not much different than any other large-scale organizational change, and as such deserves to be managed accordingly. One has to consider that moving to agile is a significant mental shift in the way people approach problem solving and performing their daily jobs. People that are the core of the change process need to be well prepared and allowed to fail as they learn.
For us it took 1 year from introducing the idea to the management to starting with the 1st sprint. At times it seemed like we are dragging our feet, but in retrospective I can say it was worth investing all the time we spent learning the subject, talking to practitioners, analyzing applicability to our context and identifying obstacles, challenges and plans of attack. This due-diligence has paid off when we started the project.
If I would need to define a successful pattern for planning and managing Agile Transformation here would be the major steps in that pattern:
1. Get a focus group to self-learn Agile. The group has to include at least 1 member from each product development team (dev, qa, pm, ...)
2. Create a shared Vision (see my post “Vision matters”)
3. Focus group meets with practitioners, members attend events and share what they’ve learned
4. Information is digested, obstacles identified and plan of attack for these obstacles being developed
5. Propose a recommendation (go/no go)
6. Get Executive buy-in
7. Start with education for all relevant locations. I've held initial 2-day workshop for everyone, then invited coach to do SM and PO classes.
8. Spinoff teams on obstacles, challenges, processes identified in #4 above.
8. Hire a coach (see my post “Hiring a coach”)
9. Find a good pilot project (it's another topic on how to choose the right one)
10. Have coach facilitate 1st sprint
11. Communicate to the world about the progress

When Offshore meets Agile

This post belongs in both of my blogs “Leading Agile Transformation” and “Succeeding with Offshore Management”, so I’ll publish it in both…

When companies look at Agile the question of location often comes up: people are afraid to implement Agile with offshore teams due to the distributed nature of the teams. Of course it’s great when you have face-to-face communication, but I see no problem with implementation of Agile in distributed environment (ok, maybe some challenges).

There was a research done on the subject of communication in different office settings, and what they found is that best communication happens when people are sitting in the same area or in adjacent cubicles. Next best setup is an offshore model and the worst setup is when people are located in the same building but not in adjacent areas. The difference in communication efficiency was significant between 2^nd and 3^rd categories, but insignificant between 1^st and 2^nd. One of the reasons for these findings could be that people in the 3rd category think they are collocated and do not need to invest in improving communication channels and communication efficiency. On the other hand, folks in distributed setting are well aware of the challenges of communication and are investing a lot of effort in constant improvements.

Agile fosters frequent communication and helps distributed teams to advance to the next level of efficiency. Yes, there are challenges that are exacerbated with Agile: more communication requires more discipline and sometimes changing the ways we communicate and may require additional tools to support it. This is not different from any other problem that Agile helps to put a spot light on and that requires resolution. Agile will emphasize what we as an organization need to focus on to make sure required communication levels are met.

One thing we’ve been struggling with before was choosing the right timing for onsite visits: the environment was pretty chaotic and it was difficult to plan for a visit and to tie it to a certain event (planning/design review, etc). With clarity of SCRUM it’s obvious how we can leverage the framework to plan for visits: Release/Iteration planning and Demo sessions are a natural fit for this.

As part of our Agile transformation I’m making sure to have not only Agile enthusiasts on  the other side of the ocean, but someone in coaching capacity to augment my efforts on site and help teams when I’m not available due to time difference or any other reason.

Learning Pays Off

There is no entry in this blog since December ’09 not because I was lazy, but because so many things have happened in context of our Agile Transformation, that I decided to wait and organize my thoughts.

In February I’ve held a 2-day offsite workshop for the entire Product Development team. It was a workshop and not a mere PowerPoint overview of Agile practices. We’ve analyzed our challenges, our vision and goals, learned through games and read articles about Agile adoption. We’ve discussed success factors, success criteria and recipe for failure. This was my first public speaking in front of such a large audience for such long period of time, so as much as it was educational for the entire group it was quite an experience for me as well.

It was extremely beneficial to walk the entire team through the vision and the goals and showing how with help of Agile we can get there. Once the group understood that we are not doing Agile for the sake of “doing it”, but Agile is just a means to an end everyone became more comfortable with the idea of adopting it. Especially it resonated with senior engineers, some of whom had pretty bad experiences with Agile in previous jobs, where Agile was shoved down the throat by the management. Everyone has appreciated the fact that we’ve learned Agile for almost 12 months and didn’t want to rush with it right after reading a book.

Armed with the basic understanding of where we are marching to and how will we get there we had additional onsite education for Engineering Management, Product Management and key engineers on the roles of Scrum Master and Product Owner. These were lead by external coaches. Not to forget about our offshore team in Ukraine, we also made sure to organize SCRUM training over there (some delivered by me, and some by local coaches).

As our major release was coming to an end we were at the right place as organization to begin our 1^st pilot project.