You generally should not wait until your dependencies become so outdated they no longer receive security updates anymore. Upgrading to newer versions are not only important from a security point of view, but it also gives the team access to new features that allow them to write cleaner and faster code (hopefully).
But our main application was stuck with PHP 7.2 after years of neglect, and our team decided to change that.
Approach
Upgrading several minor versions can be challenging, and so can be a major version upgrade. We decided to take a gradual approach and in Phase 1, we were planning to update our dependencies to the latest version and PHP to 7.4, then in Phase 2, upgrade to PHP 8.1.
There were several reasons for this. Many of our dependencies were unmaintained and we needed to find replacements for them, which introduced further code changes. We were also using a custom PHP extension that was compiled into a binary for each PHP version we used, and this custom extension was not yet compatible with PHP 8, so as a part of Phase 2, we also had to update this extension written in C++ and recompile the binary for PHP 8.
Challenges
Both stages turned out to be a bumpy road for us. While we were working on Phase 1, they simultaneously introduced a platform migration initiative as well, expecting us to move our project away from Cloudfoundry and onto GKE, specifically the Google Anthos Service Mesh, which was capable of deploying Cloudfoundry workloads to Kubernetes. This caused a delay in the PHP upgrade effort because we felt like it was safer to migrate the old version to the new platform first - because the application was not fully cloud-ready and was designed to run with a stable number of instances, and did not handle well frequently terminating and respawning pods.
After just finishing both the platform migration and the upgrade to PHP 7.4, they also announced a team downsizing, which meant the departure of 3 developers including our tech lead. This meant that I was in charge of the PHP 8.1 upgrade and the ad-hoc technical leadership of the remaining team of 3 (2 developers and 1 QA).
Problems, problems, problems
None of these problems were trivial. I am not a C developer and testing a C++ extension for memory leaks with Valgrind was uncharted territory for me. I have done it anyway, thankfully there are many resources available on updating older PHP extensions and this PHP internals book was a great help too.
This was actually pretty fun to do, I could learn new, more obscure things about PHP I haven't had to deal with before. Unfortunately, I was not able to update all of the dependencies to the latest version - this extension was not onboarded to PECL and it had a long list of C libraries that needed to be installed to make it work. After updating, memory leaks were detected, so I chose to downgrade to the newest safe versions of each dependency.
This is where I ran into our second problem: our inflexible pipeline!
You see, since we were deploying this application as a Cloudfoundry workload using a php buildpack and not a Docker image, we had limited means to install anything else in the environment. We did this anyway, with the help of some slightly hacy Groovy scripts that were supported by our pipeline.
These hacky scripts used mason, a package manager to install C dependencies. But mason is now unmaintained, so I could not use it to install newer versions of our dependencies.
Solutions
First I decided to look for mason alternatives, so I checked Conan and vcpkg. The problem with both of these was that they only listed fairly new versions of each package I needed.
As a second option, I checked if it was a possibility for us to switch to a Docker image based deployment, because this way I could install whatever I wanted inside the container. Our architecture absolutely supported deploying Docker workloads, but unfortunately, our pipeline scripts did not, not for PHP applications at least. This was a large scale enterprise pipeline system that served many different applications and had its own dedicated ops team, and because the organization worked primarily with Java and Javascript applications, currently only these to types were supported for Docker deployment.
I made some attempts to convince them that they should evaluate updating the pipeline for PHP applications and I offered my help - since the company had a few other PHP applications, there could have been business value in introducing this option for all PHP apps within the company.
However, I had to complete my task under a deadline and I could not wait for this decision to move forward, so I made a compromise and proposed to downgrade all dependencies to the last version still supported by mason and recompile the custom extension again.
Results
My last solution was accepted, especially that it had no memory leaks and it allowed us to upgrade the application to PHP 8. The regression testing did not reveal any problems, and the application could finally benefit from the performance improvements brought by the new version, and be safe from known vulnerabilities for the first time in long years.