Monday, March 06, 2017

Building is the easy part...

Agile Frameworks and books tell us how to build a product--that's the easy part... What we are not told are equally important things like maintaining, operating, fixing and extending the built product. When your agile philosophies fail to cover these areas, it greatly reduces agile's benefits. This is where DevOps comes into play.

DevOps is the combination of practices, cultural philosophies and tools; in order to increase the speed at which organizations deliver applications and services. This results in evolving and improving products at a faster pace than organizations using traditional software development and infrastructure management processes. This benefits the customers who will enjoy better services and the organization which will have a greater competitive edge.

DevOps means joining development and operations. These two teams form a single union, to oversee the total lifespan of an application: development, test, deployment, operations etc. and there are no limitations to the number of skills that can be developed when DevOps is involved. There's the possibility of quality assurance and security teams coming together in close conjunction with development and operations teams for the duration of the application's life cycle.

These teams use practices to automate processes that historically have been manual and slow. They use a technology stack and tooling which help them operate and evolve applications quickly and reliably. These tools also help engineers independently accomplish tasks (for example, deploying code or provisioning infrastructure) that normally would have required help from other teams, and this further increases a team’s velocity.

Benefits of DevOps

For customers to get quick services; for organizations to efficiently adapt to the changing market and for them to gain the needed competency for achieving their desired business results; a model that provides maximum speed is required. No better model than the DevOps model to make these results materialize. A quick release of updates is made possible as services are ultimately controlled by the teams.

Rapid delivery
When releases are fast because of an increase in speed; quick improvement and innovation of products follow suit. When there is a consistent release of new features and bugs are quickly fixed; you will retain your customers as their needs are promptly satisfied; thus, you don't lose them to your competitors. The software release process is automatic because of the uninterrupted integration and delivery practices.

While your delivery is fast; DevOps ensures that the application updates and infrastructure changes remain qualitative regardless of speed--the consumers continue enjoying a positive experience. To be sure of the functionality and safety of each change, you should test it with continuous delivery and continuous integration. Using practices like logging and Monitoring ceaselessly gives you information about the performance.

The risk that comes with managing complex or changing systems can be minimized when you operate and manage your infrastructure and development processes at scale. They can be efficiently managed with automation and consistency. Your development, testing, and productive environment for instance, can be more efficiently managed using infrastructure as code.

Improved collaboration
With ownership comes responsibility and accountability. When the teams are equipped with these, they become more effective--this is what the DevOps model brings about. Labor is divided between the development and operations teams. As they closely work together, time is saved and efficiency level is increased. For example, handover periods that involve writing codes that account for the background environment is placed at a minimal level.

Automated compliance policies, fine-grained controls and configuration management techniques keep your security intact as you adopt the DevOps model. Compliance at scale can be defined and tracked using infrastructure as code and policy as code.

DevOps practices

With the necessary tools, some key practices can boost the innovation speed of organizations. These practices involve automating and streamlining the software development and infrastructure management processes.

One fundamental practice is to perform very frequent but small updates. This is how organizations innovate faster for their customers. These updates are usually more incremental in nature than the occasional updates performed under traditional release practices. Frequent but small updates make each deployment less risky. They help teams address bugs faster because teams can identify the last deployment that caused the error. Although the cadence and size of updates will vary, organizations using a DevOps model deploy updates much more often than organizations using traditional software development practices.

Another essential practice is implementing a microservices architecture. It is like a breakdown tool; it breaks down complex systems and makes projects simple and independent. Organizations can use microservices architecture to create less rigid applications and ensure quicker innovations. There are varying individual services in an application, with different functions that work independent of each other. This architecture minimizes the coordination overhead of updating applications and as each service is paired and owned by agile teams, the speed of an organization’s progress is increased.

To attend to the operational challenges that might arise because of the combination of microservices and increased release frequency; DevOps practices like continuous integration and continuous delivery come into play to ensure that reliability and safety is maintained even as a rapid delivery remains in progress. Infrastructure as code and configuration management are infrastructure automation practices that help to ensure that frequent changes do not affect the elasticity and response of computing resources.

Furthermore, engineers can track applications and infrastructure's performance using monitoring and logging, to keep them actively reactive to problems. Delivery is fast, updates are reliable and customers are satisfied with organizations that makes use of these practices. Outlined below are important DevOps practices.

Continuous integration
With this software development practice; developers can as much as possible merge their code changes into a central repository before running automated builds and tests. Continuous integration ensures that bugs are quickly found and addressed, the quality of software improved, and time isn't wasted in validating and releasing new software updates.

Continuous delivery
Prior the release of a production, continuous delivery automatically builds, tests and prepares code changes. This software development practice deploys all code changes to a testing environment and a productive environment after the build stage, by expanding upon continuous integration. The right implementation of continuous delivery provides developers with a deployment-ready build artifact that has undergone series of standardized tests.

The microservices architecture runs a set of small services in a single application. These services are independent of each other and communication between these services is by means of a well-defined interface that uses a lightweight mechanism--an HTTP-based application programming interface (API). Each service has a single function which matches microservices with business needs. There are different Frameworks or programming languages that can be used to write microservices and they can also be set to either function as a single or group of services.

Infrastructure as code
Code and software development techniques like version control and continuous integration are used to merge and provision infrastructure under this practice. The interaction with infrastructure is programmer based and at scale rather than a manual set up and configuration resource. The API driven model of its cloud makes it possible for system administrators and developers to interact with the infrastructure as such. Code-based tools are used by engineers to interface with infrastructure; hence it is treated like an application code. Their being code based makes it possible for infrastructure and servers to be quickly deployed, using fixed standards, also the latest patches and versions can either be updated or repetitively duplicated.

Configuration management
Operating system, host configuration, operational tasks etc. are automated with codes by developers and system administrators. As codes are used, configuration changes become standard and repeatable. This relieves developers and system administrators of the burden of configuring operating system, system applications or server software manually.

Policy as code
The configuration of infrastructure and infrastructure itself are codified with the cloud. This makes it possible for organizations to dynamically monitor and enforce compliance. It enables the automatic tracking, validation and reconfiguration of infrastructure. In that way, organizations can easily control changes over resources and security measures are properly and distributively enforced. The fact that resources that do not comply can be flagged automatically for further investigation or automatically restarted to comply, increases the speed level of teams within an organization.

Monitoring and logging
To gauge the impact that the performance of application and infrastructure have on consumers, organizations monitor metrics and logs. The data and logs generated by applications and infrastructure are captured, categorized and then analyzed by organizations to understand how users are impacted by changes or updates. This makes it easy to detect sources of unexpected problems or changes. It is necessary that there be a constant monitoring, to ensure a steady availability of services and an increment in the speed at which infrastructure is updated. When these data are analyzed in real-time, organizations proficiently monitor their services

Communication and collaboration
This is the key feature of DevOps model; as development and operations come together and share their responsibilities, team spirit is enhanced and communication skills are facilitated. Chat applications, project tracking systems, and wikis are some of the tools that can be used by not just developers and operations but also other teams like marketing or sales. This brings all parts of the organization closely together as they cooperate to see to the realization of goals and projects.

Read more…

Sunday, January 15, 2017

Paying Technical Debt

The metaphor of technical debt in code and design can be defined as follows: You start at an optimal level of code. In the next release, you are adding a new feature. This would take an effort E. This of course assuming that estimations are somewhere near reality.

If the level of code was less than optimal, the effort will be E + T. Where T is the technical debt. Writing bad code is like going further into debt. You take the loan now, and you repay the debt later. The bigger the mess, the larger the delay in the next release.

The term “technical debt” was first introduced by Ward Cunningham. It was in the early 90s, when the disconnects between development and business was growing bigger and bigger. The business people would urge developers do release untested, ugly code in order to get their product or new features faster. The developers tried to explain why this was a bad mistake. Some things will never change...

Most products and projects are still released much earlier than the developers have wanted. Assuming that developers are not just being stubborn (I know, maybe an ever bigger assumption as decent estimations), you would think that we didn’t manage to get the message across to the business. We have done an awesome job explaining what technical debt is and what the results are going to be. The business people understand it. But they are just willing to take the loan now. Can you blame them? Business want something out there, in the field, that will sell now.

Let’s assume that there is a discussion at one point before a release. It’s about delaying the release and making the code better or releasing now. My experience is that most times the decision will be in favor of an early release at the cost of less quality. There are a number of reasons for this:

- The business would rather have something now, and take the debt, because they have something new to sell.

- Besides the business decision there are politics involved. People loose face with delayed releases because they promised their management that magic will happen.

- CapEx are payed from the project budget, but OpEx from a different budget. Meaning a lot of technical debt will not be payed by the project but a different department.

An example. On a recent project I was involved there were two very big debt creators.

1) Handling of a dependency on another System B. The team working on that system was completely overloaded. And instead of fixing the problems with that team, my team was forced by management to rebuild functionality that was already in place at System B. This was an architectural f*ç& up and caused many headaches later on.

2) The other one was not spending any time on test automation. Each regression test was done manually. This way we could never deliver each sprint something of good quality, nor could we deliver in fast cycles, cause after development manual testing started for multiple days.

Yes we released some features earlier as we should, but o boy, did the company payed for that later.

Read more…

Saturday, December 31, 2016

Agile Project Management Workshops Q1 2017

Data Solutions offers public workshops on various topics around Agile Project Management. Come and find the right workshop for you or contact us for an individual training recommendation.

We have currently fixed the following public workshop dates:

Workshop Date Location Price
Agile Budgets and Contracts February 24, 2017 Zurich CHF 990
Scaling Scrum in an Enterprise Environment March 17, 2017 Zurich CHF 990
Agile Budgets and Contracts March 31, 2017 Zurich CHF 990
Agile Estimation, Planning and Budgeting On request Zurich CHF 990

Read more…

Saturday, December 24, 2016

Essential rules for scaling agile

Based on my experiences with scaling agile in different contexts I am the opinion that the following guiding rules serve me well.

1. Do not scale, in most cases it is not necessary. Create well defined product boundaries with interfaces and most scaling efforts are futile.

2. Do not multi site. It makes simple things complex, and hard things even harder.

3. You cannot scale what you do not have, i.e. when you have no well functioning Scrum teams in your organization you should not talk about scaling Scrum.

4. Think products, not projects. This shift in thinking will help organizations a lot in making better decisions.

5. Continuous improvement. Never stop learning, and do not be afraid to stop what is not working.

6. Technical excellence is important for agile teams, but for scaling it is essential.

7. Take a modular approach, there is no one size fits all framework.

8. You need top down support, i.e. management wants to change there own way of working too.

9. Change the system, then culture and people's behavior will follow.

10. Tackle one product(group) at a time.

11. Feature teams, not component teams.

12. Empower your Product Owner (or similar role) to take the decisions he/she needs to be taking in that role.

Read more…

Friday, December 02, 2016

Outsourcing technical competence?

I have written about technical competence before in the context of Agile Engineering Practices and Three must have Technical Competencies for Scrum Teams. This article will shed some light on a different aspect of technical competency. Outsourcing it.

A number of companies I have worked for have started a large project based on a technology they are not competent in, or even completely unfamiliar with. This fits very nicely in the outsourcing strategy most of them have regarding IT. But does this makes sense?

In my opinion having technical competence (and excellence) in your organization has never been as crucial as it is today. Creating a technology strategy that allows you to not only deal with, but take advantage of the increasingly rapid pace of change separates the successful organizations from the obsolete ones.

Some observations I have made when your organization is not competent in a technology that is used for a big initiative:

- Your IT architecture team is dependent on external consultants for creating an architecture. You can have luck with your external consultants (honest, competent, independent), but I have seen the opposite more often (not competent, driven by sales, vendor lockin). Unfortunately when you have no competence in your organization you have no way of judging before it is to late.

- The cost of implementing new features (your business requirements) are determined by external consultants and/or vendors, and you have no way of judging if these are reasonable or not.

- The decision if certain new features are possible or not are made by external consultants and/or vendors, and you have no way of knowing if these decisions are justified or not.

- You have no healthy team atmosphere during the project. Internal employees always have to ask external ones for their opinion and input. This gives a very strange team dynamic where externals are making all the decisions one way or another (direct or indirect).

- You are not able to support the implemented solution after go-live without help from external consultants and/or vendors. This makes you very dependent, even after the project is finished

- It makes it very hard to switch vendor or implementation partner, because all of the knowledge of the solution is with them, so when you want to switch because you are not happy you have to work a while in parallel with two different companies. Besides the costs, this will give an even worse team dynamic.

- Your best external people can leave the project on a whimp because you have no influence on keeping them there. Supplier priorities may not be you, or they just leave your supplier.

In general I am the opinion that when a company is deciding on a certain technology for a big initiative the first thing they should do is hire two or three of the best people they can get, make them internal employees and give them key positions in your project setup (architect, QA, technical project lead). Leverage their experience by not making beginner mistakes, get second opinions on cost and feasibility estimations and get a judgement of skills of external consultants. This will put your organization in the driver seat instead of being driven by others.

Yes, your headcount goes up, and yes they do not work for free, but trust me on this one, this is money well invested and far much cheaper as not doing it.

PS: Although this was written based on a new technology decision, all the above is true for starting an agile project with a technology you know. When your internal employees are not familiar with Unit Testing, Test Automation and Continuous Integration, get people on board that do. You will never make a lasting change in your organization by outsourcing these.

Read more…

Friday, September 30, 2016

Swisscom CX Day: Agile. One day. Four sprints.

So next month (October 27) I am speaking at the Swisscom Customer eXperience Day. I am really looking forward to this event. Katja Leu and Christina Taylor have put together a fresh and interesting format on the topic "Agile".

"Agile" has long been the buzz word in software development for a new people-centred and efficient innovation culture that cuts across all methods. In four sprints we highlight "agile" insights as they relate to organisation, leadership and culture, environment and collaboration, and measuring success. We will discover together how an agile corporate culture can contribute to lasting success. 

The event is invite only, but you will find the presentations from previous CY Day events in their archive. There are some really interesting talks lined up for this event.

"Work? Question. Think. Learn." by Bastiaan van Roden, Founder @ Nothing Interactive

The way we think about work is stuck in the age of the industrial revolution. Linear, recurring routines have become less suited to producing adequate answers to the challenges of today when we need them. Bastiaan prefers open, authentic participation when it comes to people creating meaningful experiences for other people. It's not about work any more; it's about curiosity: only constantly asking questions, providing new answers and learning from them will ensure success.

Culture has a stronger impact than strategy." by Franziska Stebler and Rudolf Gysi, Agile Coaches @ SBB

The people in a company shape its culture. The culture is the shadow of the system and carries more influence than any strategy. For a company to change, it needs to start with the people. So the Agile Coaches train SBB IT employees in agile practices, empowering entire teams to develop new solutions faster. Using the "Iteration Zero" example, Franziska and Rudolf show how they boost the groundwork for product development with their people-centred approach.

"Goodbye boss. Hello trainer." by Heinz Herren, Head of IT, Network & Innovation @ Swisscom

You can see the full force of the butterfly effect in the networked world. When something happens in one corner of the world it is transmitted in the shortest of times to other areas and can trigger huge movements. Agile, people-centred collaboration is the answer to the shortcomings of hierarchical organisations in reacting quickly, flexibly, and diversely to changes. When it comes to management that means moving from rigid control to vibrant, learning organisations. Top and bottom was yesterday. Today it's about being a trainer and making others successful.

"The setting is crucial." by Thomas Bickel (Head of Sport) and Uli Forte (Manager) @ FC Zürich

What do agile forms of collaboration and top football have in common? What is the secret of a top-performing team that works in perfect harmony? What can influence a team positively and negatively in terms of its success? We find out from the perspective of the Head of Sport and the Manager of FC Zürich how important environment and communication are for perfect teamwork.

"Getting a little better every day." by me

Delivery is more important than all processes, frameworks, methods and tools. But it's not more important than the people involved in the project. Only the team knows what really works. So the team determines the success, not a book, a consultant or a manager. For every race, the team decides which activities from the backlog bring the greatest added value if they are pursued further. Too much guidance restricts any eagerness to experiment and stops you making mistakes, reducing your ability to learn. Quality is important from the first iteration. But it's not possible without technical excellence. Agility means small steps: getting a little better every day.

"Less is almost always more." by Head of Business Development @ Digitec Galaxus

Agility only works then the entire ecosystems works agilely together – not just some parts of the value chain. It's only when the entire system is operating agilely that its full power is released. It is important that everyone actively participates and can play their part. Focus is vital for this: it is only when we are prepared to give up some things willingly and leave them to the side that we can make everything faster and better. So less is almost always more.

CX-Day Panel 

How does agile collaboration change management? Is management even necessary in an agile environment? If so, what type of management? How is the new management role defined and what values is it based on? Moderator Carsten Roetz will put these and other questions to the following panel of experts.

Read more…

Wednesday, June 08, 2016

Antifragile at the Global Scrum Gathering Munich 2016

October 17 till 19 I will be at the Global Scrum Gathering in Munich. The theme of this gathering is

Business Agility: How to Thrive in a Constantly Changing Environment 

Questions that will be addressed are:

- How can executives of large- and mid-sized organizations set up their businesses to adapt and react faster to technological changes and challenges?
- How can they not only enable their teams to build things the right way but, even more importantly, ensure that they build the right things?
- What kind of structures are needed to involve and enable “smart creatives” to develop innovative and valuable products?
- To what extent do classic organizational structures support or contradict business agility?

The idea is to explore the different perspectives on what’s needed by whom.

- What are the needs of the executives?
- What are the needs of middle management?
- What are the needs of those who build the products in the end?
- Where are the gaps and how to bridge them?

The gathering is divided in four different tracks:

1) Dealing with Uncertainty: How to Build “Anti-fragile” Businesses in a Constantly Changing World

2) Effectiveness over Efficiency: Focus on Building the Right Thing Rather than Building Things Right

3) Empowering People: The Right Organizational Structure to Nurture and Foster “Smart Creatives”

4) Amplify Learning: How Do We Create the Right Experiments to Get the Right Learning Experience in Order to Reach the Next Level of Maturity — as a Product, as a Team, as a Corporation?

I will give a talk at the gathering titled "Dealing with Uncertainty: From Agile to Antifragile". The purpose of my talk will be:

- Gain a better understanding of what uncertainty is
- Understand Black Swan theory
- Understand what antifragile means
- Gain insight in some strategies that can be applied to deal with uncertainty

I am really looking forward to the event. Many smart and experienced people, cool conversations and new ideas will be guaranteed. When you want to meet up at the gathering and have one of those amazing German beers just contact me.

Read more…

Friday, April 29, 2016

Scaling Agile at LEGO

I have written articles before about scaling Scrum and scaling Agile because this is a challenge I am confronted with a lot. One way of approaching scaling Agile is the Scaled Agile Framework (SAFe). To put it mildly, SAFe is controversial in the Agile community. At first glance SAFe looks a bit like a big scary heavy-weight top-down RUP-zombie. But what about in real life?

Based on what I know about SAFe so far, I am not convinced it is something I would advice in most situations. Personally I am more of a LeSS guy myself. But that does not prevent me from learning more about it. So I decided to spend some time reviewing use cases of SAFe. One of the best documented use cases of SAFe is the use of it at LEGO. Henrik Kniberg is working as a coach at LEGO and he and the LEGO team are presenting about their learnings frequently.

In 2014, LEGO Digital Solutions turned to SAFe to improve their collaboration model. Much like creating something from LEGO bricks, they built their transformation one piece at a time, starting with inviting 20 managers to a 2-day Leading SAFe class. From there, they began training the teams; first one, then another until they had 20 teams trained in SAFe.

They approached every step as a learning journey, allowing for creativity along the way. When something didn’t seem like a good fit, they weren’t afraid to experiment. Taking results from Inspect and Adapt, they tweaked SAFe to their needs with a simple guiding principle, “Keep the stuff that generates energy.” Currently they work with 150 people in 20 teams using Scrum, Kanban and XP to deliver.

It is a fascinating use case, and no matter what you think about SAFe, you can learn from it. Below you will find the material I have reviewed.

Learnings from SAFe @ LEGO – Mattias Skarin & Eik Thyrsted Brandsgård at LKCE15

Is SAFe Evil? - Lars Roost & Henrik Kniberg at GOTO Copenhagen 2015

The slides for the talk Agile @ LEGO at at Passion for Projects in Uppsala.

Read more…