Wednesday, December 10, 2014

If you could only do one thing to make better software, what would it be?

Good technical practices are what we have to do to make good software – this is the engineering part of software engineering. Design. Coding. Testing and Reviews.

If you could do only one thing to make better software, what would it be? Where would you get the most bang for your buck?

Continuous Integration – Making Code Run

Continuous Integration is an obvious place to start. You need to build the software and get it running before you can do anything useful with it.

Getting developers to check in and sync up with each other more often. Building the system more often – at least once a day to start, then on every check in. Which means simplifying and automating the steps to build the system. Making sure that the system builds successfully every time – without errors or warnings. Which means that people can run it and try it out whenever they want. Make sure that it will run correctly. Which means adding tests and checks as part of the build and deploy steps. Building information radiators so that everyone knows the status of the build and when the build is broken.

You can’t be Agile without Continuous Integration, and you need Continuous Integration in place before you can go down the Devops path to Continuous Delivery or Continuous Deployment.

And Continuous Integration works in sequential Waterfall delivery too. Developers in these environments might check in more code less often, but there is still real value in knowing that you can build and run the system and see it working sooner rather than later, especially in big enterprise systems and big programs where getting dependencies worked out and all the pieces working together is a huge challenge.

Developers testing their own work – Making Code Work

Making developers responsible for testing their own work, automating this as much as possible by building on Continuous Integration, is the only way to deliver software faster and keep costs down – depending too much on manual testing and hand-offs to a test team will slow you down too much.

Almost every organization that I have talked to over the past couple of years is pushing more responsibilities for testing onto developers, and pushing more testers into development teams (or out of the organization altogether), following the lead of Google and now Microsoft, to become "more Agile".

This means relying more on developers to write good automated tests (unit tests, basic UI regression using Selenium or Watir) and static analysis checking in Continuous Integration or the developer’s IDE to find common coding mistakes and security vulnerabilities.

But there are limits to what developers will catch in their testing, even good developers. Once you get developers to write tests (before, or after they write the code, it doesn't matter, now that TDD is dead), you’ll end up with mostly simple unit tests or UI regression tests that don’t stray far from the happy path, proving that the code does what the developer thinks it is supposed to do – because that is what they need to get their work done. Their assumptions and blind spots will be reflected in the tests as well as the code. Little or no negative testing. Or usability testing. Or security testing. Or stress testing. Or system-level integration testing. All of which still has to be done by somebody - unless you expect your customers to find your bugs for you.

It will take a long time before the team learns how to write good, efficient tests, and before they build up a set of tests that will catch real bugs, rather than just getting in the way. But if you do this right, you can deliver good code while still moving fast, and get better value out of testing.

Code Reviews or Pairing – Making Code Good

Another way to get better code is by getting developers to do code reviews.

Code reviews should be about finding problems in the code first – checking for correctness, defensive coding protection (error handling and API contracts and thread safety and data validation), security (using security libraries correctly for access control and output encoding, protecting confidential data, logging and auditing…). And about making the code better – more understandable, safer and easier to change.

Code reviews are expensive, so do them right: lightweight, risk-based, using static analysis first to catch low-level mistakes and bad coding practices so that reviewers can spend their time looking for more important problems.

Instead of code reviews, you could try pairing as a way to get another pair of eyes on the code.

Pairing isn’t the same as reviews – the goals and priorities are different. A good reviewer will find problems even in code developed through pair programming, because reviewers look for different things. But research proves that disciplined pair programming will give you better structured, cleaner code, with fewer bugs. And pairing is a much better way to teach programmers about the system than code reviews are.

The downsides of pair programming? The cost of having two people do the work of one person – a good pair will work faster than one person on their own, but the less experienced or less skilled team member will slow the pair down to what they can deal with. Focus fatigue. Pairing can be exhausting, which means people can’t do it for too long at a stretch, before their work becomes superficial or strained. And social problems. People who like it, like it a lot. But people who don’t like it won’t do it at all.

Refactoring – Making Code – and Design – Last

What about design? Collaborative design workshops? Design reviews? Threat modeling in design to take care of security and operational risks?

We do all of these things. But as we continue to iterate through the design and as our code base grows, refactoring – to retain, and sometimes restore, the design, and to keep the code maintainable – is becoming more and more important.

It’s easy to learn your IDE’s refactoring tools and the basic ideas behind refactoring. But it’s not easy to learn how do refactoring right (although you can learn a lot in a short time from Woody Zuill and Llewellyn Falco in their “2 Minutes to Better Code” video). Understanding why some refactoring approaches are better than others. How to save time refactoring. How to do it safely.

Mariusz Sieraczkiewicz does a good job of explaining how and when to do "everyday refactoring" using a matrix built on Michael Feathers’ work on brutal refactoring and the biology of code:

  1. Start by reading and annotating the code, maybe do some scratch (rapid, throwaway) refactoring to understand it better
  2. Find meaningful names for variables and conditionals
  3. Extract methods to break down big chunks of code and express the algorithm
  4. Get rid of obvious duplication
  5. Move methods and extract classes to isolate responsibilities.
I agree with Sieraczkiewicz that these simple steps “would heal most code bases on this planet”. He then goes on to describe larger and more fundamental “strategic refactoring” (aka “root canal refactoring"): refactoring to patterns, introducing new architectural constructs. Work that carries much higher risks and costs. This is where refactoring ends, and re-design and re-architecture starts.

What would you do, to make Better Software?

Continuous Integration can pay off quickly: the change in transparency and in the team’s focus is almost immediate.

Developer testing is a journey, not a goal. It will take a long time for most developers to get good at it, and a long time to build up a good set of tests that you can rely on. The sooner you start, the better.

Code reviews can also take a long time to pay off. Developers – and managers – need to make the time for reviews to be done and build the discipline, and developers need time to learn how to review code properly, and how to give and accept criticism. But code reviews – or pairing – will give you better code.

Refactoring is more of a compounding investment – you pay a little bit today to save a lot in the future.

If there is only one thing that you could do to make better software, what would it be? Where would you start?

13 comments:

  1. I was supposing to find something else here. Anyway, I am a web developer so I am developing websites which are mostly used by users of the whole world. Due to such global use, one of the biggest problems that web developers face is of timezones where date-time is to be stored, displayed and manipulated. So, I would like to solve this issue by removing timezones and setting one time for the whole world. Also, the DST should be removed. Date format should also be same worldwide. Year-Month-Day-Hour-Minute-Second a good option. However, if date-time is fully coverted in metric system then it would be the best option.

    ReplyDelete
  2. Make the software industry stick to one language and associated frameworks for the next 50 years. Adapt the existing languages and frameworks to new environments, instead of inventing entirely new frameworks and environments.

    ReplyDelete
  3. Get One Language, get it to work, WITHOUT millions of add-ons or self written-add-ons. And here I mean Angular-JS, QRCOde and ZXing, as examples. Why can't we have ONE language that does everything from one source? The myriad of requirements to make Java, JavaScript etc work is painful. And as Anonymous said, no more frameworks...

    ReplyDelete
  4. blah blah frameworks. That's called freedom dudes.

    Nikunj hits it on the head: Date handling is a massive problem. Ad something that real programmers of web software, bank software, medical software, etc, deal with daily.

    UX issues are another sore spot.

    Frameworks? Nah, I'd as soon have more than being tied to one.

    ReplyDelete
  5. Good points. Knowing your language (and frameworks) and tools is a critical technical practice that I should have included.

    ReplyDelete
  6. My coworker asked me, "If you could only do one thing to make better software, what would it be?" before sending me this article.

    My reply was, “program to interfaces and decouple components.” I guess I learned at least one thing from GoF. I think I stand by this statement after reading the article. If you don’t do this, it is very difficult to write the unit tests that go into a continuous integration process. The real value of continuous integration isn’t so much did the build succeed, but rather have I broken the logic within our software.

    ReplyDelete
  7. IBM once had a slogan "Think!". That's it.

    ReplyDelete
  8. Make sure that everything can be COUPLED, RECOUPLED AND DECOUPLED safely and easily. Decoupling does sacrifice writability and readability ( you have to know a zillion interfaces and a zillion lines to get one small thing done right). Tight Coupling is a sacrifice of flexibility and integrity for downright writability and at the time of first writing: readability.

    What we need is the FREEDOM to write tightly coupled code and decouple it later when we need to. We should also have the FREEDOM to not write to an interface now and write it later when we need to.

    For the moment, the only feature that respects these FREEDOMs is Refactor-extract method. I just can't write a massive XML-based component and refactor it later safely... heck there aren't many renames.

    I write a RenderTable function and then split it to RenderRow and RenderCell later and have the class implement IRenderer when I feel it's ready. OR I could have someone else split it up later without that much effort. OR I could write them as static methods at first and write instance method proxies for Dependency Injection. OR I could write them as instance methods at first and write static proxies later so people don't need to make a zillion objects to call one method. If book authors and copywriters could do this --- TWO-WAY abstraction-deabstraction -- why can't we do it in software? The fact that the best practice says: decouple everything and abstracize everything is costing the entire industry.

    ReplyDelete
  9. Better? Why?

    If software works it has to meet no other measure, does it? But now, what does working mean? Does working sometimes, or working for a while count? OF COURSE!

    Problems in software can often be fixed with hardware. NEW software is expensive. That is DEVELOPING new software is expensive. Hardware is cheap, way cheap in comparison.

    Well, let me rephrase that. Developing good software is expensive. Bad software is cheap, or cheaper. And some of it will work. And thus bad software rules, because the costs associated with making better software, that is creating new better software, are higher that those of gathering a lot of bad software and coders of it and using what may be used out of it.

    People are people. Supermarket shelves are full of "bad" food. iPods are full of bad music (or to be more up to date Pandora bandwidth has more bad music than good). The world is far more full of poor, poorly educated people than well-off, well, educated, literate people.

    Why would software be any different?

    What is better software? What is a better person?

    ReplyDelete
  10. i strongly believe that developing a software is entirely dependent on the client requirements and specifications. we can all rant about this or that but if the client wants a simple application to manage his retail shop, why would you as a developer go ahead and redesign your own. better software to me is one that does what the client wants whereby by the end of the day, everyone is happy.
    The language preference or tools are a means of product deliver.

    ReplyDelete
  11. Getting people in the workplace to just shut up so others can concentrate.

    ReplyDelete
  12. I would fire the incompetents.

    ReplyDelete
  13. Good software should be able to change as and if the business needs change. The long-term benefits of writing software that is already decoupled, or can easily be decoupled (modular, reactive, whatever) are greater as the software can evolve at a lower cost. Good software has a lower TCO than bad software. A lower TCO can be achieved by either not changing or by changing readily. Bad software is ok as long as it doesn't need to change or isn't critical.

    That being said, better software for the given situation can be achieved by connecting the developers/architects/engineers with the users and the business in a way that allows them (us) to understand the place of the software in the context in which it is intended to operate.

    If you were hiring a building architect and engineers and builders to build a structure, the purpose of the structure would define the design parameters. If the building is a shelter for farm equipment, then one could likely buy an off-the-shelf structure. Conversely, if the structure is intended to house offices and strive for energy efficiency, then the project managers must find adequate resources to design and build such a building.

    In the latter case, the costs and time will certainly be far greater. The product should be expected to net a positive gain in the course of it's tenure at the selected location.

    In software, where many parallels are made to structural architecture, it can be said that the expected cost and time should be reflected by the expected lifetime and ROI of the product. ROI can be actual profits or protected losses. These factors should drive the qualities of the code more than any other, regulatory compliance aside.

    Better software can and should be driven by better project management.

    ReplyDelete

Note: Only a member of this blog may post a comment.