Retrospective on Agile Transformations: Challenges to Adoption - Bottom-up Approach

Challenges to Adoption - Bottom-up Approach

Agile principles and values often easily adopted by development teams if provided effective coaching. These teams also understand and adopts the technical excellence practices e.g. XP (TDD, Pair Programming, CI etc.) with some initial resistance and start seeing their benefits sooner rather than later experiencing considerable improvement in quality of deliverable, customer satisfactions and timeliness of deliverable. Teams starts getting quite motivated by delivering right business value to its customers with quality software and people see a very positive buzz around them. Although this initial initiative gives lots of agile buzz within an organization but still its benefits are limited as rest of the business (Sales, HR, Finance, sometime even Product design) doesn’t join this agile journey. Slowly-slowly the fizz goes out of this agile initiative as team starts getting demotivated due to non-involvement of   other organization functions and little or no commitment seen from middle management and CxO layer. Senior and Middle Management often don’t get the new management paradigm that is required for an enterprise agile transformation. The main bottleneck is actually around their experience of power that is almost exclusively around the notion of being in charge, whereas, in new way of working power shifts from the notion of “being in charge” to the notion of “being connected”. This means that you don’t have to be in charge to exercise power.

Bringing management layer part of overall agile transformation enables a leaner organization that multi-fold the benefits of agile in a large organization. Once management joins the agile transformation journey, bringing other functions to join becomes little easier.

 

Retrospective on Agile Organizational Transformations: challenges to adoption

Abstract:
Agile transformations often fail or don’t deliver full benefits in large organizations. Lack of understanding of agile principles especially by management team leads to thinking that agile is only for development teams rather than an enterprise thought process. Despite development team using technical excellence practices (TDD, Pair Programming, Continuous Integration etc.) to produce quality deliverable the other part of the organization limits the benefit of agile by not adopting agile mindset. Often large organizations find difficult to implement agile principle & values because of its existing corporate structure, culture, mindset and lack of commitment from executive teams to support the agile transformation. 

Agile transformation started from bottom-up get some good initial success but derails later due to organizational constraints. Organization culture that promotes deep hierarchical (and dysfunctional) structure, political environment, policies (not aligned towards employee empowerment) and individual reward structures are often main reason stated by most of the fail agile transformations teams. The successful agile transformation initiatives are led by people right from the top CxO layer by putting “commitment before success” formula and supported by middle management using servant leader relationship to bring whole organization gradually towards agility to make organizations being agile rather doing agile.  

This is a series of article that I will continue writing on my blog and in each article will target few related reason for failed Agile transformations. Please watch this space...

Journey from Backlog Grooming to Sprint Planning

Scrum framework proposes Sprint Planning to plan the work for a given Sprint. The objective of Sprint planning is to discuss and plan the work with Product Owner and commit to the Sprint Backlog for a Sprint. Sprint planning meeting has two parts. Part 1 is to discuss with Product Owner to discuss WHAT stories can be completed in a Sprint (based on Sprint length) and Part 2 is to discuss within team HOW those stories would be implemented. 

Too much time in Planning Meeting

Teams new to Scrum process spends lots of time in both Sprint planning meetings and still fails to meet their Sprint commitments. Bit more mature teams’ starts backlog grooming sessions, Sprint planning sessions gets shorter but still miss to complete the commitments in a given Sprint consistently. This leads to frustration within Sprint teams and Product Owner as well.

So what’s the solution?

Grooming++ sessions

Grooming sessions are a perfect example of Team working with Product Owners to define and refine the User stories. Product Owner presents the business requirements and works with team to slice the stories into small stories. Because whole team is working together to define (and refine) the user stories, the team has a consistent understanding of stories discussed in the grooming session. So what’s the issue? When we know what needs to be done so clearly, why team is still missing the commitments (despite knowing very well their expected velocity)?

The problem is that grooming sessions are only focused on WHAT but no time is given to understand HOW the stories will be implemented. As team starts to discuss HOW to implement stories in Sprint planning part 2 meeting, but estimates has already been given either in Grooming sessions (most of the time) or Part 1 of Sprint Planning meeting. When team starts working on the implementation of the stories lots of unknown appears i.e. refactoring, lack of understanding of code, difficulty to work with legacy code etc. So most of these unknown work leads to more necessary work but missed commitments.

Try this. After a grooming session (I call it grooming++), quickly discuss how the story or stories going to be implemented at high level. So have a quick look on the code and understand if there are any unknowns that need to be factored in your estimates i.e. Refactoring. A quick engineering drawing on the board using either of Class Diagram, Sequence Diagram, Flow diagram or any other diagram that you are comfortable with and after the session. Once you have understood the high level implementation details and any unknowns that needs to be factored in, you would have better chance to give accurate (ok, close to accurate) estimates. These sessions are more useful when you have a legacy code or even a Greenfield project where a team joined late and don’t have enough understanding of existing implementation (kind of legacy code for them). This exercise is very useful even if your stories are small because it gives you

Team Empowerment

This is your first project using Scrum framework and someone announces (normally in one of training you receive before you start a project using Scrum framework), “Scrum teams are self organized, empowered and self-managed”. The first thought that goes in most of the people head, what exactly it means, especially if you have been following traditional waterfall methods all along your life. You ask to explain this statement bit more to the trainer and trainer explain that most of the decisions are taken by team rather than management. Now, you are bit confused (or grinned if you are a developer). The main reason of confusion (or grinning) is that nobody explains how it’s going to work in reality. Each member of team start thinking what does this statement means for them?

Let’s try to scan few people’s brain and see what’s going on their mind when they hear this statement.

Project Manager’s brain! What does it means? Scrum is a project management framework and supposed to solve project management issues. What is supposed to do with team’s self-Organisation, self-management and empowerment? I am the one who is responsible for the delivery of this project so how come team can be empowered to take decision.  Development team will manage themselves, you must be dreaming! Sounds familiar PMs?

Architect’s mind, what’s bull%£$t? Are you serious? Are you telling me that these so called developer and QA team will decide how the solution would/should be built? You must be joking aren’t you? These people have always worked what I have told them, so how can they be empowered to take those (architectural) decisions. What about sign off process?  Who will take ownership of decisions? I told you earlier, this whole Agile thingy is not going to work. I settle my case now.

Development/QA Manager’s mind, hmmm! I have no idea what does it means to my role. If team are self-organised, self-managed and also empowered, what am I going to do? Does it make my role redundant? Grrrr, I am not comfortable with it, so I should show resistance.

Developer’s mind, really? I mean do you really mean that I would be able to take those decisions that I never agreed with my PM, Architect etc. That would be awesome finally to take my own decisions. I am in dude!

Do you see a pattern here? I hope you do and if you have spotted it you are right. None of these people thought that they are part of ‘the team’.  They all thought that they have some individual responsibilities but no collective responsibilities.

That’s the beauty of most of the Agile methodology and frameworks specially Scrum. Scrum has only three roles Product Owner, Scrum Master and team and together they are one team.  So first thing that we must understand that ‘team’ includes everyone i.e. Architect, manager, QA, developer, DB architect, technical writers, customer etc. We all need to work together to achieve one goal; i.e. Deliver working software every sprint as per customer demands. Once we understand the concept of team, it’s easy to understand empowerment, self-organisation and self-management.

Now let’s go through Empowerment, self-organisation and self-management and understand what it means in reality.

Empowerment as per dictionary is “to give power or authority to; authorize, especially by legal or official means”. In Agile, the team has power or authority to take its own decisions to improve the quality, process and practice for the solution that they are working on. So for example, it’s up to team to decide what are the best development processes for them so that no unnecessary organisation processes are imposed them to create un-necessary bottleneck or overheads. The real advantage of empowerment gets realized when team starts using Inspect & Adopt process (using Sprint Retrospective meetings in XP/Scrum) and initiate improvement activities. Team should be empowered here to take those actions rather than going through a sign off process and someone else decide what actions needs to be worked and assigned them to individuals. This kind of behavior kills the momentum of drive to continual improvement.

Please note that empowerment also brings accountability along with it. So when teams are empowered they are also accountable for each decision they take that results in success (or failure) of a project/product.

Empowered teams become self-organised very quickly. Team starts owning actions and start interacting with each other to complete those actions. As I stated earlier accountability comes along with empowerment that helps team to become self-managed also. Self managed teams don’t look outside the team to decide or take decisions to drive improvement. They are motivated enough (through ‘empowerment’) to own the product and work towards continual improvement.

So as a Scrum Master, emphasize the importance of a team before start working on empowerment. Discuss principle and values behind agile teams i.e. trust, openness, integrity, transparency, courage etc. Once you have a team empower them to take their own decision but with active coaching (not managing). Also note, it can take up to 24 months before teams become fully self organised and self managed.