Wednesday, May 22, 2013

7 Agile Best Practices that You Don’t Need to Follow

There are many good ideas and practices in Agile development, ideas and practices that definitely work: breaking projects into Small Releases to manage risk and accelerate feedback; time-boxing to limit WIP and keep everyone focused; relying only on working software as the measure of progress; simple estimating and using velocity to forecast team performance; working closely and constantly with the customer; and Continuous Integration – and Continuous Delivery – to ensure that code is always working and stable.

But there are other commonly accepted ideas and best practices that aren’t important: if you don’t follow them, nothing bad will happen to you and your project will still succeed. And there are a couple that you are better off not following at all.

Test-Driven Development

Teams that need to move quickly need to depend on a fast, efficient testing safety net. With Test First Development or Test-Driven Development (TDD), there’s no excuse for not writing tests – after all, you have to write a failing test before you write the code. So you end up with a good set of working automated tests that ensure a high level of coverage and regression protection.

TDD is not only a way of ensuring that developers test their code. It is also advocated as a design technique that leads to better quality code and a simpler, cleaner design.

A study of teams at Microsoft and IBM (Realizing Quality Improvement through Test Driven Development, Microsoft Research, 2008) found that while TDD increased upfront development costs between 15-35% (TDD demands developers change the way that they think and work, which slows developers down, at least at first), it reduced defect density by 40% (IBM) or as much as 60-90% (Microsoft) over teams that did not follow disciplined unit testing.

But in Making Software Chapter 12 “How Effective is Test-Driven Development” researchers led by Burak Turhan found that while TDD improves external quality (measured by one or more of test cases passed, number of defects, defect density, defects per test, effort required to fix defects, change density, % of preventative changes) and can improve the quality of the tests (fewer mistakes in the tests, tests that are easier to maintain), TDD does not consistently improve the quality of the design. TDD seems to reduce code complexity and improve reuse, however it also negatively impacts coupling and cohesion. And while method and class-level complexity is better in code developed using TDD, project/package level complexity is worse.

People who like TDD like it a lot, so if you like it, do it. And even if you are not TDD-infected, there are times when working test first is natural – when you have to solve a specific problem in a specific way, or if you’re fixing a bug where the failing test case is already written up for you. But the important thing is that you write a good set of tests and keep them up to date and run them frequently – it doesn't matter if you write them before, or after, you write the code.

Pair Programming

According to the VersionOne State of Agile Development Survey 2012, almost 1/3 of teams follow pair programming – a surprisingly high number, given how disciplined pair programming is, and how few teams follow XP (2%) or Scrum/XP Hybrid (11%) methods where pair programming would be prescribed.

There are good reasons for pairing: information sharing and improving code quality through continuous, informal code reviews as developers work together. And there are natural times to pair developers, or sometimes developers and testers, together: when you’re working through a hard design problem; or on code that you’ve never seen before and somebody who has worked on it is available to help; or when you’re over your head in troubleshooting a high-pressure problem; or testing a difficult part of the system; or when a new person joins the team and needs to learn about the code and coding practices.

Some (extroverted) people enjoy pairing up, the energy it creates and the opportunities it provides to get to know others on the team. But forcing people who prefer working on their own or who don’t like each other to work closely together is definitely not a good idea. There are real social costs in pairing: you have to be careful to pair people up by skill, experience, style, personality type and work ethic. And sustained pair programming can be exhausting, especially over the long term – one study (Vanhanen and Lassenius 2007) found that people only pair between 1.5 and 4 hours a day on average, because it’s too intense to do all day long.

In Pair Programming Considered Harmful? Jon Evans says that pairing can have also negative effects on creativity:

Research strongly suggests that people are more creative when they enjoy privacy and freedom from interruption … What distinguished programmers at the top-performing companies wasn’t greater experience or better pay. It was how much privacy, personal workspace and freedom from interruption they enjoyed,” says a New York Times article castigating “the new groupthink”.

And in “Still Questioning Extreme Programming” Pete McBreen points out some other disadvantages and weaknesses of pair programming:

  • Exploration of ideas is not encouraged, pairing makes a developer focus on writing the code, so unless there is time in the day for solo exploration the team gets a very superficial level of understanding of the code.
  • Developers can come to rely too much on the unit tests, assuming that if the tests pass then the code is OK. (This follows on from the lack of exploration.)
  • Corner cases and edge cases are not investigated in detail, especially if they are hard to write tests for.
  • Code that requires detail thinking about the design is hard to do when pairing unless one partner completely dominates the session. With the usual tradeoff between partners, it is hard to build technically complex designs unless they have been already been worked out in a solo session.
  • Personal styles matter when pairing, and not all pairings are as productive as others.
  • Pairs with different typing skills and proficiencies often result in the better typist doing all of the coding with the other partner being purely passive.
And of course pairing in distributed teams doesn't work well if at all (depending on distance, differences in time zones, culture, working styles, language), although some people still try.

While pairing does improve code quality over solo programming, you can get the same improvements in code quality, and at least some of the information sharing advantages, through code reviews, at less cost. Code reviews – especially lightweight, offline reviews – are easier to schedule, less expensive and less intrusive than pairing. And as Jason Cohen points out even if developers are pair programming, you may still need to do code reviews, because pair programming is really about joint problem solving, and doesn’t cover all of the issues that a code review would.

Back to Jon Evans for the final word on pair programming:

The true answer is that there is no one answer; that what works best is a dynamic combination of solitary, pair, and group work, depending on the context, using your best judgement. Paired programming definitely has its place. (Betteridge’s Law strikes again!) In some cases that place may even be “much of most days.” But insisting on 100 percent pairing is mindless dogma, and like all mindless dogma, ultimately counterproductive.

Emergent Design and Metaphor

Incremental development works, and trying to keep design simple makes good sense, but attempting to define an architecture on the fly is foolish and impractical. There’s a reason that almost nobody actually follows Emergent Design: it doesn't work.

Relying on a high-level metaphor (the system is an "assembly line" or a "bill of materials" or a "hive of bees") shared by the team as some kind of substitute for architecture is even more ridiculous. Research from Carnegie Mellon University found that

… natural language metaphors are relatively useless for either fostering communication among technical and non-technical project members or in developing architecture.
Almost no one understands what a system metaphor is any ways, or how it is to be used, or how to choose a meaningful metaphor or how to change it if you got it wrong (and how you would know if you got it wrong), including one of the people who helped come up with the idea:
Okay I might as well say it publicly - I still haven't got the hang of this metaphor thing. I saw it work, and work well on the C3 project, but it doesn't mean I have any idea how to do it, let alone how to explain how to do it.
Martin Fowler, Is Design Dead?

Agile development methods have improved development success and shown better ways to approach many different software development problems – but not architecture and design.

Daily Standups

When you have a new team and everyone needs to get to know each other and more time to understand what the project is about; or when the team is working under emergency conditions trying to fix something or finish something under extreme pressure, then getting everyone together in regular meetings, maybe even more than once a day, is necessary and valuable. But whether everyone stands up or sits down and what they end up talking about in a meeting should be up to you.

If your team has been working well together for a while and everyone knows each other and knows what they are working on, and if developers update cards on a task board or a Kanban board or the status in an electronic system as they get things done, and if they are grown up enough to ask for help when they need it, then you don’t need to make them all stand up in a room every morning.

Collective Code Ownership

Letting everyone work on all of the code isn't always practical (because not everyone on the team has the requisite knowledge or experience to work on every problem) and collective code ownership can have negative effects on code quality.

Share code where it makes sense to do so, but realize that not everybody can – or should – work on every part of the system.

Writing All Requirements as Stories

The idea that every requirement specification can be written as User Stories in 1 or 2 lines on cards, that requirements should be too short on purpose (so that the developer has to talk to someone to explain what’s really needed) and insisting that they should all be in the same template form

“As a type of user I want some goal so that some reason…”
is silly and unnecessary. This is the same kind of simple minded orthodoxy that led everyone to try to capture all requirements in UML Use Case format with stick men and bubbles 15 years ago.

There are many different ways to effectively express requirements. Sometimes requirements need to be specified in detail (when you have to meet regulatory compliance or comply with a standard or integrate with an existing system or implement a specific algorithm or…). Sometimes it’s better to work from a test case or a detailed use case scenario or a wire frame or some other kind of model, because somebody who knows what’s going on has already worked out the details for you. So pick the format and level of detail that works best and get to work.

Relying on a Product Owner

Relying on one person as the Product Owner, as the single solitary voice of the customer and the “one throat to choke” when the project fails, doesn't scale, doesn't last, and puts the team and the project and eventually the business at risk. It’s a naïve, dangerous approach to designing a product and to managing a development project, and it causes more problems than it solves.

Many teams have realized this and are trying to work around the Product Owner idea because they have to. To succeed, a team needs real and sustained customer engagement at multiple levels, and they should take responsibility themselves for making sure that they get what they need, rather than relying on one person to do it all.


Simon said...

Really interesting stuff here Jim, thanks. Lots of references to work through too!

Simon Redfern said...

Nice post. :-)

Pierre Rasmussen said...

Great post!

Quite often you see people adopt trends from gurus without question it if it something for them.

Marie Haggberg said...

Great points on the need to always question standard processes. Thank you.

Anonymous said...

This is a great article because
1/ you back up with sources
2/ you do not just say "this best practice is bullshit" but "you should use this best practice if ..."

I may not agree with all your points but this is good matter for thoughts

Robert Patterson said...

I liked this artical too. I'm a fan of Agile, but it's nice to finally hear someone say it's okay to take it with a grain of salt. I've always felt that way.

For me the greatest value of Agile is simply to make the team work together, and promoting continuous delivery of working software.

Nirav Assar said...

This is a great article with many good points. I am an agile supporter and implementor, but still we should look at every thing we do with an eye for better ideas and critique.

Anonymous said...

A lot of the formal scrum practices also remove the possibility of exploitation and put responsibility on the people who need it to get things done.

You completely gloss over that entire aspect.

Sean Glover said...
This comment has been removed by the author.
Sean Glover said...

This is a very well researched post. I've been a member of a team that has practiced XP for a couple of years. We have arrived at a lot of the same conclusions to the use of these "best practices". That is, we've tried them and either tweaked them or made them optional. I feel that TDD and Pair Programming are the hardest to do well and consistently. I've found TDD requires a huge amount of discipline to do consistently, I can't think of many if any of the team actually use it still. Pairing is an entirely different story and it's something a lot of our team members practice regularly. We've made both practices optional.

Anonymous said...

Agile (with the big "A") suffers from many of the same problems as waterfall or other prescriptive software methodologies.

Agile (with a little "a") encourages teams to find their own way, be open to change and to not follow any best practices if they're found to hold the team back from their primary goal, which should be to deliver value.

Of course, for most teams, delivering value is defined as delivering quality software within reasonable timescales, and for them, the best practices you argue against here can be useful. I've read many of the articles you cite and I agreed with some of the points stated in them, just as I agree with yours.

In general, I've found agile practices to be:

a) Beneficial when done right
b) Worse than useless otherwise

I've worked with some clients who claim to be agile, but who still just don't get it - they want:

1) Fixed scope
2) Fixed budget
3) Fixed quality

...when it's well acknowledged (long before Agile with a capital A) that delivering on all three fronts is next to impossible.

Peter Modzelewski said...

First of all - great post. It made me to rethink some stuff in my teams process, which is always great.

As for the topic I believe the "it depends" is critical here.

In my experience, when subject is completely new to a team, it should be do done "canonically". For example, if a team wants to try Pair coding, it should invest weeks in doing it by the book (and yes, it takes weeks or even months to get free from mind blockades and to get it right). Only then, when they start to feel it, and honestly feel how it influences their work, they can modify it.

The problem is, that people who are practicing agile canonically for months, forget that agile is about agility, and there is *always* something to improve in the process.

I believe (and I might be wrong) that the reason is: the risk. People often are afraid to take the risk, and there is no ready framework explaining, how to manage the risk. That's why they prefer to stick with something that mostly works (and build their comfort zone around it) then risk searching for something a bit different.

cr0m said...

Great article, but I really don't agree with not needing scrum.

Sure, it's possible that a team can do away with scrum safely, but without it, it's all too easy for a team to slip out of the habit of effective communication. And you won't know until it's too late.

Chris Littlejohns said...

Great post - you eloquently and convincingly capture a lot of the sceptism I also have about naive application of techniques/processes.

Fitness Bite said...

sudha sri said...

Very informative and well written post! Quite interesting and nice topic chosen for the post.
mobile service centre
mobile service center in chennai
mobile service center chennai
mobile service centre near me
mobile service centre chennai
best mobile service center in chennai
best mobile service center

Star lucifer said...

Keto Burn Xtra

Supplement Portal

kanika azar said...

Perfect Keto Max Above all are the necessary and easy weight reduction tips for working girls At all times attempt to find jiffy for exercise and make yourself stuffed with healthy meals. Weight Loss basics: eat more energy than you utilize & you may placed on weight; use greater than you eat and you will lose it. Weight Loss is now a purpose which could be reached actually easily if we stick to a training regime, eating regimen plan. This occurs when the changes you initially made in terms of exercising more and maintaining a healthy diet are now not producing the fast weight loss you experienced at first. Once you go on a food regimen , you alter the form of foods you eat. On the off likelihood that you simply envision yourself shedding kilos and consuming calories amid typical daily action, you'll get in shape and muscle to fat quotients.

service care said...

I am hoping the same best effort from you in the future as well. In fact your creative writing skills has inspired me.
oneplus mobile service center
oneplus mobile service centre in chennai
oneplus mobile service centre
oneplus service center near me
oneplus service
oneplus service centres in chennai
oneplus service center velachery
oneplus service center in vadapalani

service care said...

Thanks for your valuable post... The data which you have shared is more informative for us...

apple service center near me
iphone service center in chennai
iphone service centre
iphone service centre chennai
iphone service center velachery

iphone service center vadapalani
iphone service center porur
ipad service center in chennai
ipad service center

service care said...

The blog is really satisfied to read the blog.keep sharing like this type of information.thanking you.
moto service center in chennai
motorola service center in chennai
moto service centre chennai
moto service center
motorola service center
motorola service center near me
motorola mobile service centre in chennai
moto g service center in chennai
motorola service center in velachery
motorola service center in t nagar
motorola service center in vadapalani

service care said...

Thanks for such a great article here. I was searching for something like this for quite a long time and at last, I’ve found it on your blog.thanks for sharing.
mobile service centre
mobile service center in chennai
mobile service center chennai
mobile service centre near me
mobile service centre chennai
best mobile service center in chennai
best mobile service center in chennai
mobile service center in velacherry
mobile service center in vadapalani
mobile service center in porur
best mobile service center

service care said...

Thanks for posting useful information.You have provided an nice article, Thank you very much for this one. And i hope this will be useful for many people.. and i am waiting for your next post keep on updating these kinds of knowledgeable things...Really it was an awesome article...very interesting to read..please share like this information......
samsung mobile service center in velachery
samsung mobile service center in porur
samsung mobile service center in vadapalani

swashthik said...

Great post. I was once checking constantly this weblog and I'm impressed! Extremely useful information specially the closing part. I maintain such information much. I was once seeking this specific information for a very long time. Many thanks and best of luck.
honor service center chennai
honor service center in chennai
honor service centre chennai
honor service centre
honor mobile service center in chennai
honor mobile service center
honor mobile service centre in Chennai
honor service center near me
honor service
honor service centres in chennai

Mobile App Development Company in India said...

Thanks for sharing such a great blog... I am impressed with you taking time to post a nice info.
Website Development Company in Delhi
Website Designing Company in Delhi
Mobile App Development Company
Mobile App Development Company in India

Chiến SEOCAM said...

Takk for at du har brakt en god artikkel. Håper du har bedre artikler. hilsener

Thu mua chó corgi

Thu mua chó Bull pháp

Thu mua chó pug

Thu mua chó poodle

mobile application development said...

Thanks for the fantastic info satisfied to read the blog. keep sharing like this .
Mobile App Development Company in chennai
enterprise mobile app development company
Mobile application development company in chennai

Java application development said...

Amazing Post. Your writing is very inspiring. Thanks for Posting.
Java application development company
Java development company
Java outsourcing company
Hire java developer
java web development services

Digitalmarketing said...

I have perused your blog its appealing and noteworthy. I like it your blog.
digital marketing company in chennai,
digital marketing agency in india,
digital marketing company in chennai,
online marketing company in chennai,
digital marketing company in india,
digital marketing services,
digital marketing company,

Software development company said...

I have scrutinized your blog its engaging and imperative. I like it your blog.
custom application development services
Software development company
software application development company
offshore software development company
custom software development company

best ppc company in india said...

I have inspected your blog its associating with and essential. I like it your blog.
ppc services in india
best ppc company in india
ppc services india
ppc advertising services
ppc services company

IELTS Coaching in Dwarka said...

I feel happy about and learning more about this topic. keep sharing your information regularly for my future reference. This content creates new hope and inspiration within me. Thanks for sharing an article like this. the information which you have provided is better than another blog.
Best IELTS training centre in Dwarka Delhi

Android iPhone Indonesian Service said...

Distributor Kuota
PT Lampung Service
Service HP Bandar Lampung
Service iPhone Lampung
Service Acer Lampung
PT Lampung Service

Creators Seo Master said...

super your blog
andaman tour packages
andaman holiday packages
web development company in chennai
Math word problem solver
laptop service center in chennai
Austin Homes for Sale

Aruna Ram said...

Innovative post! Very superb content, this is very helpful to gain my knowledge. Keep going...
Pega Training in Chennai
Pega Course in Chennai
Tableau Training in Chennai
Spark Training in Chennai
Oracle Training in Chennai
Oracle DBA Training in Chennai
Social Media Marketing Courses in Chennai
Unix Training in Chennai
Power BI Training in Chennai

sandhiya arav said...

Well written Blog, I really enjoy reading your blog. this info will be helpful for me. Thanks for sharing.
ccna Training in Chennai
ccna institute in Chennai
Angularjs Training in Chennai
PHP Training in Chennai
Web Designing Course in Chennai
ccna course in chennai
ccna training in chennai

Site Meter