DevOps Is Software Engineering for Your Builds
C++ developers often approach DevOps with a reasonable degree of skepticism, and it's not without reason. Their top five challenges revolve around build scripts, dependency management, and setting up CI pipelines—areas traditionally associated with DevOps responsibilities. Michael Xymitoulias articulated this sentiment well in a recent LinkedIn post writing:
Right now, it feels that C++ developers have to deal with way more than just writing business logic code. [...] Allowing devs to focus more on coding rather than trying to solve problems of the ecosystem would probably be liberating
Xymitoulias suggestion that allowing developers to focus more on coding, rather than grappling with ecosystem problems, would be liberating. He's not alone, Bill Hoffman, CTO behind CMake supports this ideal of simplifying the developer workflows, something we've seen put in place with recent improvements to the popular build system.
While building in C++ certainly comes with its challenges, streamlining this process is our collective goal. So, let's delve into the core focus as it relates to C++ and dispel some common misconceptions.
Learning from the Past
These ideas have been circulating for quite some time.
All aspects of software production Software engineering is not just concerned with the technical processes of software development but also with activities such as software project management and with the development of tools, methods and theories to support software production. "1.1.2 What is software engineering?" - Software Engineering (8th ed.) ISBN 978-0-321-31379-9
This quote, from a book originally published in 1982, highlights the broader scope of software engineering, encompassing not only technical development but also project management and the development of tools and theories supporting software production.
Yet, expecting individuals to be experts in every tool, method, and theory is unrealistic. The diversity of modern software tooling and platforms necessitates specialized team members with technical skills and a passion for these tools. A Software Engineering team should consist of various roles catering to different aspects of the development process.
Applying DevOps in C++ Development
Diego Rodriguez-Losada, the lead architect and co-found of Conan, one of the leading C++ package managers, presented the idea DevOps is dependency and package management over time in his 2022 CppCon presentation. I would open that up to include not just the toolchains and libraries you use but the infrastructure which supports the builds. This quote, succinctly captures the question and motivation for doing this.
We propose that "software engineering" encompasses not just the act of writing code, but all of the tools and processes an organization uses to build and maintain that code over time. What practices can a software organization introduce that will best keep its code valuable over the long term? How can engineers make a codebase more sustainable and the software engineering discipline itself more rigorous? "Programming Over Time." Software Engineering at Google (2020). ISBN 978-1-492-08279-8
As a DevOps Engineer, the aim is to support Software Engineering activities past, present and future, which for C++ include:
- Managing build environments: This involves detailing the contents, deployment methods, and update schedules of build environments. Infrastructure as code ensures repeatability and reliability, while versioning and storage solutions contribute to longevity. There are numerous tools for deployment environment and this is just an extension of those best practices.
- Supporting building and testing: This entails encapsulating build scripts from the environment, enabling their reuse as environments evolve. Binary management ensures that assets generated by builds are captured and saved for reuse, ensuring consistency from development to production.
- Releasing and shipping to production: This outlines translating internal engineering versions into business-level products, ready for deployment in assembly lines or cloud environments. Each product will be unique just as the target architectures they are compiled for.
This amalgamation constitutes the Software Development Lifecycle (SDLC), with security being a top priority though out. Strong collaboration among team members with diverse skills ensures that developers can focus on implementing product features, while DevOps practices uphold the integrity and reliability of the software.
Demystifying DevOps
This section is a small rant. @me on social media for discussion!
While criticism of a "fragile hodgepodge system of jerry-rigged things that were never intended to work together" resonates with many in the C++ community, I'd argue that is a bias based on the fragmentation within the ecosystem. It's essential to broaden our perspectives before drawing conclusions.
In many ecosystems, building projects is trivial, with straightforward commands like cargo build, npm build, or maven build. This is why we only hear about CI/CD with no "continuous building" concept which is completely skipped over. Once the challenge of building applications at scale for numerous developers multiple times a day is overcome, it becomes less of a focal point at conferences. This is why cmake --build --preset release is a huge step in the right direction.
So maybe it is understandable to see the remark "DevOps has such a full stack vibe about it"? Not really, fullstack means something completely different. DevOps is a bridging the gap between IT Operations and Software Development. Contrary to beliefs that C++ is excluded from this paradigm, various tools and platforms support C++ development within the DevOps framework. GitHub Actions and Azure DevOps platoformns have support for less common toolchains like MSYS2/Mingw-64. GitLab and Gitea have integrations for Conan packages. Jenkins has been used for C++ CI/CD for many years and helped pioneer concepts like Pipelines as Code.
Comments like "neverending fighting non-coding related bugs is fun but only from devops perspectives" miss the mark. A strong DevOps Engineer possesses knowledge of Infrastructure as Code, enabling them to develop well-designed applications compatible with distributed systems and third-party services. Learning HCL for Terraform, extending it with Golang, and developing test are absolutely fundamental just like any robust software stack. It's no different then developing on top of Boost.Asio.
Ecosystem and Evolution
To put it all into perspective, as a C++ developer, grappling with tedious build processes and platform-dependent multithreading complexities is a familiar struggle. However, embracing DevOps principles can alleviate these challenges, allowing developers to focus on crafting robust and efficient code.
In Xymitoulias post, Hoffman wrote:
It is all about creating an environment where C++ developers can focus on code and have the DevOps happen for them.
And I think that's the perfect place to aim for.