Escaping Developer Nightmares- 10 mins
A short write up of the bad things we do in software development and some suggestions on how to fix them.
Let’s take a look into what we can do to achieve a better development environment than an average development project – a project that most of us have seen at some point in our professional lives, or maybe even are a part of right now. We will also look into some tools and patterns that will help us convert those projects into a paradise for the developers.
Just a few decades ago, we were working in ways that might look like unproductive, in the best case. Our development models were predominated by waterfalls, our IDEs were basic and we were compiling our projects by hand, using
javac, or building up the
CLASSPATH depending on the
GOTO statements in a huge spaghetti code contained in countless
bat files. Our code lived in a very simple versioning systems that were not distributed or supported branching strategies that are praised by the developers today. Our documentation lived in
doc files on shared network drives, side by side with the simple issue tracking systems, that don’t even get close to what we have today.
Today, it is all different – we have Git, real issue tracking, IDEs, all that integrated with build servers and collaborative platforms. Yes, everything is much better, more effective and user-friendly, one might think that we are in the paradise already? Well… yes, things are fortunately getting better, however, we are still doing things in a way that might still give you nightmares, several decades from now.
Last years I have been working and invited to evaluate and help with an audit of various projects. Here are some of my observations and thoughts.
The Existing State of Affairs
This tweet describes it pretty well:
Some of the issues are, naturally, remnants of the past – the legacy systems; but even those systems and most of the other problems we see today can be avoided if we slightly change our view at some of the main parts of the development process. In most of the cases, we would be aware of those issues, but we might need to explain and motivate the others – often people are responsible for the projects and those who prioritize the development and maintenance backlog.
The Moving Parts
The road to a great nightmare-free future consists of three components: the code quality, the development and build tools, and a good documentation and collaboration systems. When evaluating systems I often start asking some simple questions listed below to get an idea of the system.
1. The Code
The Code Quality
First thing off is the general code quality. I often start by asking about simple things – if the project has a coding standard, and if it is being followed. I also ask to take a quick peek at the code and check minor things like file encodings and MIME types. I also follow up with a question if the team is practicing code reviews, and how they are doing that.
While those things alone don’t have to mean anything, and are minor issues individually, together with other factors they still are initial indicators of possible neglect. This gives me a possibility to map areas where to look further.
In addition to that, there are also some more specific parts that will be listed as sub-sections below.
The third-party libraries
The role of the third party libraries and their use is often forgotten and neglected when considering code and system quality. This is quite unfortunate as this is the part of the code that you might not be able to patch easily, and is harder to maintain compared to your own codebase. Here are some simple questions that might help with getting a better grip on third-party libraries:
- Do you keep track of your third-party libraries?
- Do you regularly check if there are known issues or vulnerabilities in them?
- Do you have a plan for keeping them updated?
- Are the libraries you are using being actively maintained by the authors?
- Are the libraries you are using compatible with each other?
- Do the libraries you are using have appropriate licenses that are compatible with your system? (This also applies to the open source software licenses.)
Issues and vulnerabilities are being found and patched all the time. As an example for this, let me point to Google’s OSS-Fuzz Project that has found numerous security vulnerabilities in several critical open source projects. Unfortunately, even though many people are aware of the security issues in software in general, the library updates still often tend to be forgotten.
It is also worth noting that while most of the issues on my list above are security related, the last one might be of a legal sort, and probably is the most neglected of the issues listed.
I am often being asked to assess a system and tell something about its architecture compared to more modern systems. The different aspects of the system’s architecture will tell a lot about its maintainability both when it comes to further development, bug fixing, and keeping the system running. Some questions that might help with determining the state of the system would be:
- Does your architecture support automated deployment?
- Does your architecture support continuous deploy and delivery?
- Does your architecture support load balancing?
- Does your architecture support microservices?
- How is the architecture implemented in the code?
Tools for Maintaining the Code Quality
Some of this might sound familiar to you when you think of one or several systems you had been working with and you might now be wondering what you can do to improve the code quality? Further steps here would be starting to use proper tools that will be able to tell more about the various aspects of your code. Some of them can be used as plugins to your build system (like Maven), and some could be stand-alone tools.
Stand-alone tools for code analysis:
Maven plugins to consider (more about plugins in my previous post):
Bonus: See this post on command line tools for Java projects.
2. Development Tools and Strategies
Now, let’s talk about the development tools. All that fancy code and great architecture will not bring you any closer to a developer’s paradise if there will not be some proper tools to support the development. The code should live in a proper version control system that supports collaboration and things like branching and tagging. There should also be tools that help you with code quality analysis, static code analysis, etc. A good starting point here would be to start with answering the following about the project in question:
- Do you use a proper code versioning tool – Git, or even SVN?
- Do you have a branching (and tagging) strategy?
- Do you have a way of measuring code complexity?
- Do you have a way of measuring test coverage and results?
- Do you run static code analysis?
Some tools that can help you here (again, for Java-based systems):
- IDEs and IDE plug-ins that can do checks at commits, integrate with test and QA tools, etc.
- Build tools: Maven, Gradle, etc.
- Continuous integration tools: Jenkins, TeamCity, Bamboo, etc.
- Frameworks and tools for testing: to run unit tests, integration tests, UI tests, and end-to-end tests.
There are some further strategies and questions to consider:
- Are your environments easy to reproduce with minimal efforts – can you rebuild it by simply running a script?
- Do you have a proper pipeline from packaging, to delivery, to deploy?
- To what environments can you deploy automatically? With the same script, or command?
- Are your environments (like development, testing, staging, pre-prod, production) similar to each other?
- Do you follow the same process to deploy to each environment?
- Are QA and production running on physically separate hardware?
- Are you monitoring all of the environments? (i.e. are you able to see errors before they make it to QA or even production?)
3. Documentation and Collaboration Tools
Last, but not least, we will need to talk about the tools for collaboration and documentation. Without these tools, we will be back to the way things were several decades ago – with documents on shared network drives and other horrors of the 90’s that I mentioned at the beginning of this post. However, good wikis, other collaboration tools, and proper issue tracking will bring your software to another level, encouraging continuous improvement of the system.
- Collaboration – chat, etc.
- Issue tracking tools
No matter how obvious it might seem, it is still important to note that one should avoid multiple documentation and issue tracking systems. Unfortunately, even though it might sound obvious, it is more common than you think – I have seen my share of systems for documentation and issue tracking resulting in fragmented information and confusion.
There are several challenges connected with having and maintaining the good code quality. The first challenge is that a good code quality is not something you can achieve overnight. It takes time and energy to achieve that and it is a continuous process. You will need some tools, techniques, and methodology to prevail, and it will probably be easier to introduce all that from the beginning of a project.
The second challenge would be that it might be hard to convince the stakeholders of the project to invest time and resources into something that does not bring any visible improvements to the table – things like new features and bug fixes are more likely to get prioritized over something that cannot be easily measured.
Actually, while presenting on this topic at JavaOne 2017 in San Francisco, several of the attendees asked me about the ways of getting to a beautiful nightmare-free code and infrastructure, and the ways of convincing the stakeholders that this is the way to go. Unfortunately, there is no one simple solution to this, and the most valuable thing, in this case, would be to show the real value of the good quality code.
The measurements parameters to show the value can be:
- time it takes from the code is written to deploy,
- system stability,
- how often bugs are reported compared to earlier, or
- frequency of errors in logs.
So, what can you do as a developer on a project that might need some help, you might ask? You can just start by continuously suggesting improvements and showing their value to the customer, or the project manager. Now you just need to keep going and gradually improving the system, one small bit at a time.