There are many articles about this topic: your company/team is working Agile and you notice soon that you miss Architecture… or at least a little bit of software-design-common-sense if you don’t like big words.
I would like to share my view on the problem, and also what I think is a pragmatic solution to deal with this (difficult) Agile & Architecture combination.
You work “Agile” so you are probably used to Scrum / Kanban. One of the “hidden rules” of Agile is that you can’t work in anything that can’t be mapped directly to business value.
Does the Architecture or even a little bit of Software Refactoring has business value? This is the big question. The answer depends on how good you are selling a software change or new design. No matter how good you know this change is, you need to prove it: is it going to save time for future development, for example?
If you pass the previous step and the Product Owner is convinced that this change or new design is for good then you will face the next challenge: can that task be completed within one Sprint? Can you split it into small chunks? If someone is really Agile-picky then he/she may wonder if you are able to write what you want to do in the format “As a [user/whatever] I want to [description of the goal] so [benefits of the goal]”. Let’s assume you are not dealing with radicals and you can skip this last one 🙂
OMG! And where are the Architects?
Agile methodology encourage architects to be part of the teams, so they should be working under the same “Agile rules”. But this fact itself is also causing problems, such as frustration in senior and creative people, and the fact that there is no technical ‘authority’ for solving conflicts. At this point I recommend you to read a post about someone who is really against Agile and Scrum: Why Agile and especially Scrum are terrible. Quoting Michael:
“[…] Like a failed communist state that equalizes by spreading poverty, Scrum in its purest form puts all of engineering at the same low level: not a clearly spelled-out one, but clearly below all the business people who are given full authority to decide what gets worked on. […]”
Even though I don’t fully agree with Michael I think he has got a point. In some companies Agile is being used to fight fires, without thinking in mid/long-term. Senior people and architects have no special treatment: they are team members. Some other companies managed to palliate this problem by having a really good communication between more skilled guys and the Product Owner, or even having Architects at the same hierarchy level.
Working with an example
Let’s imagine that you want to estimate a User Story in which you need to send an email. It’s the first time your application needs to do that. There are some guys in the team thinking that is a good opportunity to introduce Spring in your project, since your team could also benefit in the future from other Spring features, and in this case, you can use Spring integration with JavaMail.
The Product Owner may think that these guys are crazy: introducing Spring for such a small change! He knows that other team has some scripts in Python to send emails that you could integrate with your application, by running a command from your Java application (sounds clean…). Check the numbers for our example: estimation for using the scripts is a 5 (story points, bananas or whatever). Introducing Spring here would be a 13. These guys also say that maintenance of the scripts in future is going to be a nightmare compared to the good support from Spring. But, how are you estimating future maintenance issues in Agile?
Eventually, since the PO doesn’t know from how many places the application will need to send an email in future or if that feature will evolve, Python scripts are integrated.
If after one year that turned out to be a mistake and application is calling difficult-to-maintain scripts from everywhere: can it be changed? How much work is that? Who is going to change it? When? And some big bosses may ask: who is the responsible for having chosen the simplest implementation, and why wasn’t changed before?
The problem is that at the time of the decision maybe nobody knew how many email features would have the application. Agile it’s business oriented and also sometimes really short-term focused.
Is Agile a Technical Debt Generator?
Bad implemented Agile can be a Technical Debt generator where quick fixes prevail over any other solution. Sometimes it’s much more easy to copy and paste something than putting the code in common components and then reuse it. And sometimes there is no ‘technical authority’ to guard code quality.
This could be a summary of the problems I’ve seen:
- Agile has no room for Architecture, especially if there is no Project’s vision (team of firefighters).
- Sometimes the only way to introduce software improvements or pay technical debt is by associating these changes with a business feature that can really benefit from that, but then
- Technical improvements and/or refactoring usually make estimations higher, and then a discussion is needed to convince the Product Owner of the benefits, which is sometimes complicated since Agile can be a short-term trap.
Big changes don’t fit into Agile, so you need a plan.
Some pragmatic ideas
One of the first things you should do is to identify which parts of your legacy code are really critical and difficult to refactor. Since most probably it is a complex part of your code you could invest some effort in making it easy for everyone to understand it, so people can contribute to improving it while they are working on other issues.
Some ‘technical user stories’ that you can use are:
- Identify critical parts in your legacy code. Pay special attention to code that is really important, is constantly growing but has no documentation.
- Document these parts, even a basic diagram with the flow and technical overview is good enough. Ideally, you could also elaborate a step-by-step guide to do the most common task in your team’s daily work. It’s really weird but sometimes you need to think about how to translate your current design into a really basic diagram to figure out how to improve it in the most efficient way.
- Write integration tests for these core parts. They will be useful for refactoring. Avoid complex testing that needs lots of maintenance.
Another really important motivation booster that prevents frustration is to make goals clear to everybody. Any member of a team should know where is the project going in three months, six months and one year. That way people can take good functional and technical decisions.
To achieve that, you can propose some tasks that should fit within one sprint:
- Work with business representatives to define short Main Goal descriptions for these three milestones: 3, 6 and 12 months. This first draft should fit into a small presentation, four or five slides maximum.
- If needed, you can create a task per item that should be detailed a little bit further. But keep it from the general view to the specifics.
- Identify which of these goals are more challenging (or complex) from a technical perspective given your current situation. This task has to be done in close communication with the Product Owner or Business Users. A good outcome of these discussions would be a different prioritization of the goals, based on the Return of Investment and taking into account that you will need to accomplish those targets anyhow.
Make a Plan
If you pass the process of identifying and containing legacy code and getting the vision, then you should set up a Kanban board with the main technical improvements that the team wants to introduce. Once you have the topics, you will need to refine them so they can be split into small tasks. Some examples of vague topics that need to be detailed are ‘Migration to Java 8’, ‘Changing the UI to AngularJS’, ‘Make the build faster’, etc. You need to split those and make clear descriptions of the goals and how to achieve them.
Personally, I don’t like spikes but in case that the task is clearly related to a research that needs to be done, it makes sense. Anyway, my recommendation would be that the result must be always a wiki page or something that can be used by others.
Don’t forget the arguments behind that technical change: most of the times they are system quality attributes: performance, robustness, readability, testability… Depending on your project some of them may be much more important than others, but you need to link the improvements to them always and be able to communicate it to the Product Owner and team leaders.
The Architect role
With the introduction of Agile in many companies, Architects need to be more communicative than ever. It’s not that they are required to make presentations or convince people every day, but surely it’s about being able to let stakeholders know that technology-related improvements are important, together with keeping quality in software foundation within the project, having a good test base, etc.
Don’t you have an Architect in your team? Agile gives you something to take advantage of: it’s bottom up, and that means that you can start proposing improvements for your team, creating small tasks and explaining why it’s so important to have a good technical vision. What do you need to reach your goals? Energy, patience, passion for what you do, and a plan.