Friday, June 5, 2015

Software Architecture in DevOps

A new book by Len Bass, Ingo Weber and Liming Zhu “DevOps: A Software Architect’s Perspective”, part of the SEI Series in Software Engineering, looks at how DevOps affects architectural decisions, and a software architect’s role in DevOps.

The authors focus on the goals of DevOps: to get working software into production as quickly as possible while minimizing risk, balancing time-to-market against quality.

“DevOps is a set of practices intended to reduce the time between committing a change to a system and the change being placed into normal production, while insuring high quality”
These fundamental practices are:
  1. Engaging operations as a customer and partner, “a first-class stakeholder”, in development. Understanding and satisfying requirements for deployment, logging, monitoring and security in development of an application.
  2. Engaging developers in incident handling. Developers taking responsibility for their code, making sure that it is working correctly, helping (often taking the role of first responders) to investigate and resolve production problems.
    This includes the role of a “reliability engineer” on every development team, someone who is responsible for coordinating downstream changes with operations and for ensuring that changes are deployed successfully.
  3. Ensuring that all changes to code and configuration are done using automated, traceable and repeatable mechanisms – a deployment pipeline.
  4. Continuous Deployment of changes from check-in to production, to maximize the velocity of delivery, using these pipelines.
  5. Infrastructure as Code. Operations provisioning and configuration through software, following the same kinds of quality control practices (versioning, reviews, testing) as application software.

Culture and collaboration between developers and operations, shared values and organizational issues, the softer people-side of DevOps, are considered only insofar as they are factors that could affect time-to-market delivery velocity or quality.

Cloud Architecture and Microservices

As a reference for architects, the book focuses on architectural considerations for DevOps. It walks through how Cloud-based systems work, virtualization concepts and especially microservices.

While DevOps does not necessarily require making major architectural changes, the authors argue that most organizations adopting DevOps will find that a microservices-based approach, as pioneered at organizations like Netflix and Amazon, by minimizing dependencies between different parts of the system and between different teams, will also minimize the time required to get changes into production – the first goal of DevOps.

Conway’s Law also comes into play here. DevOps work is usually done by small agile cross-functional teams solving end-to-end problems independently, which means that they will naturally end up building small, independent services:

“Having an architecture composed of small services is a response to having small teams.”

But there are downsides and costs to a microservice-based approach.

As Martin Fowler and James Lewis point out, microservices introduce many more points of failure. Which means that resilience has to be designed and built into each service. Services cannot trust their clients or the other services that they call out to. You need to add defensive checking on data and anticipate failures of other services, implement time-outs and retries, and fall back alternatives or safe default behaviors if another service is unavailable. You also need to design your service to minimize the impact of failure on other services, and to make it easier and faster to recover/restart.

Microservices also increase the cost and complexity of end-to-end system testing. Run-time performance and latency degrade due to the overhead of remote calls. And monitoring and troubleshooting in production can be much more complicated, since a single action often involves many microservices working together (an example at LinkedIn, where a single user request may chain to as many as 70 services).

DevOps in Architecture: Monitoring

In DevOps, monitoring becomes a much more important factor in architecture and design, in order to meet operations requirements.

The chapter on monitoring explains what you need to monitor and why, DevOps metrics, challenges in monitoring systems under continuous change, monitoring microservices and monitoring in the Cloud, and common log management and monitoring tools for online systems.

Monitoring also becomes an important part of live testing in DevOps (Monitoring as Testing), and plays a key role in Continuous Deployment. The authors look at common kinds of live testing, including canaries, A/B testing, and Netflix’s famous Simian Army in terms of passive checking (Security Monkey, Compliance Monkey) and active live testing (Chaos Monkey and Latency Monkey).

DevOps in Architecture: Security

Security is another important cross-cutting concern in software architecture addressed in this book. It looks at security fundamentals including how to identify threats (using Microsoft’s STRIDE model) and the resources that need to be protected, CIA, identity management, access controls. It provides an overview of the security controls in NIST 800-53, and common security issues with VMs and in Cloud architectures (specifically AWS).

In DevOps, security needs to be wired into Continuous Deployment:

  1. Enforcing that all changes to code and configuration are done through the Continuous Deployment pipeline
  2. Security testing should be included in different stages of the Continuous Deployment pipeline
  3. Securing the pipeline itself, including the logs and the artifacts
and security checks need to be part of monitoring (such as Netflix’s Compliance Monkey and Security Monkey).

Continuous Deployment Pipeline and Gatekeepers

Developers – and architects – have to take responsibility for building their automated testing and deployment pipelines. The book explains how Continuous Deployment leverages Continuous Integration, and common approaches to code management and test automation. And it emphasizes the role of gatekeepers along the pipeline – manual decisions or automated checks at different points to determine if it is ok to go forward, from development to testing to staging to live production testing and then to production.

DevOps and Modern Software Architecture

“DevOps: A Software Architect’s Perspective” does a good job of explaining common DevOps practices, especially Continuous Deployment, in a development, instead of operations, context. It also looks at contemporary issues in software architecture, including virtualization and microservices.

It is less academic than Bass’s other book “Software Architecture in Practice”, and emphasizes the importance of real-world operations concerns like reliability, security and transparency (monitoring and live checks and testing) in architecture and deployment.

This is a book written mostly for enterprise software architects and managers who want to understand more about DevOps and Continuous Deployment and Cloud services.

If you’re already deep into DevOps and working with microservices in the Cloud, you probably won’t find much new here.

But if you are looking at how to apply DevOps at scale, or how to migrate legacy enterprise systems to microservices and the Cloud, or if you are a developer who wants to understand operations and what DevOps will mean to you and your job, this is worth reading.

21 comments:

DevOps Online Training said...

Great Site, The Course were so simple and easy to understand.
DevOps Online Training

DevOps Online Training said...

nice post! Thanks for delivering a good stuff related to DevOps, Explination is good, nice Article
anyone want to learn advance devops tools or devops online training
DevOps Online Training

saran latha said...

The knowledge of technology you have been sharing thorough this post is very much helpful to develop new idea. here by i also want to share this.
advanced excel training in bangalore

Praylin S said...

You are sharing a lot of valuable information. I'm really glad that I came across this post. Really a worthy article. Regards.
Microsoft Dynamics CRM Training in Chennai | Microsoft Dynamics CRM Training institutions in Chennai | Microsoft Dynamics Training in Chennai | Microsoft Dynamics CRM Training

Unknown said...

Thanks for sharing this blog with us, keep sharing!
DevOps Online Training

James Dobson said...

The information regarding engineering you might have recently been revealing complete this kind of submit is Devops As a Service quite significantly beneficial to produce fresh thought. the following simply by my partner and I furthermore desire to discuss this kind of.

DevOps Online Training in Hyderabad said...

Good blog very nice information. Thank you for posting this interesting blog.

kavinilavu G said...

Such a Great Article!! I learned something new from your blog. Amazing stuff. I would like to follow your blog frequently. Keep Rocking!!
Blue Prism training in chennai | Best Blue Prism Training Institute in Chennai

Rithi Rawat said...

Very nice post here thanks for it .I always like and such a super contents of these post.Excellent and very cool idea and great content of different kinds of the valuable information's.

Best institutes for machine learning in chennai

machine learning certification in chennai

IT Tutorials said...

Thank you so much for your information,its very useful and helful to me.Keep updating and sharing. Thank you.
RPA training in chennai | UiPath training in chennai

amsa leka said...

Thanks for your great and helpful presentation I like your good service. I always appreciate your post. That is very interesting I love reading and I am always searching for informative information like this.AngularJS Training in Chennai | Best AngularJS Training Institute in Chennai

Roja Priya said...

Given so much info in it, The list of your blogs are very helpful for those who want to learn more interesting facts. Keeps the users interest in the website, and keep on sharing more
Our Credo Systemz Which is designed to offer you OpenStack Training skills required to kick-start your journey as an OpenStack Cloud Administrator.
Please free to call us @ +91 9884412301 / 9600112302

best openstack training in Chennai | openstack certification in chennai | openstack certification training in Chennai | openstack course fees in chennai

DJ PRASATH said...

Thanks for your post. This is excellent information. The list of your blogs is very helpful for those who want to learn, It is amazing!!! You have been helping many application.
best selenium training in chennai | best selenium training institute in chennai selenium training in chennai | best selenium training in chennai | selenium training in Velachery | selenium training in chennai omr | quora selenium training in chennai | selenium testing course fees | java and selenium training in chennai | best selenium training institute in chennai | best selenium training center in chennai

Praylin S said...

Very informative post! Thanks for sharing.
WordPress Training in Chennai | WordPress Training | WordPress Course in Chennai | Training institutes in Chennai with Placement | Tally Course in Chennai | Ionic Course in Chennai

kavinilavu G said...

Great post! I am actually getting ready to across this information, It's very helpful for this blog. Also great with all of the valuable information you have Keep up the good work you are doing well.DevOps Training in Chennai | Best DevOps Training Institute in Chennai

amsa leka said...

Wow!! Really a nice Article. Thank you so much for your efforts. Definitely, it will be helpful for others. I would like to follow your blog. Share more like this. Thanks Again.
iot training in Chennai | Best iot Training Institute in Chennai

Unknown said...

This is great post - so clear and easy to follow. And very useful information We offer devOps training with job assistance.
devOps training

cityspideyseo said...

CitySpidey is India's first and definitive platform for hyper local community news, RWA Management Solutions and Account Billing Software for Housing Societies. We also offer air quaity index and residential soceity news of Noida, Dwarka, Indirapuram, Gurgaon and Faridabad. You can place advertisement for your business on city spidey.

Gate Management System
Society Management App
Society Management
rwa Management App
Neighbourhood Management App
Apartment Management App
Apartment Management System
Visitors Management System
Apartment Management Software
Air Quality Index, Air Pollution
Noida News
Gurgaon News
Ghaziabad News
Delhi News
Indirapuram News
Dwarka News

amsa leka said...

Wow!! Really a nice Article. Thank you so much for your efforts. Definitely, it will be helpful for others. I would like to follow your blog. Share more like this. Thanks Again.
React js training in Chennai | Best React js training institute in Chennai | Best React js training near me | React js training online

write a paper for me said...

Thank you for this post. This is very interesting information for me.

research paper outline example said...

Thanks for this information, I think it will come in handy in the future.

Site Meter