Version control in embedded projects: what still matters after ten years
Git in long-lived embedded projects: branches per hardware revision, handling vendor BSPs, binaries and toolchains, reproducible states through tags and manifests – and why a readable history only proves its worth years later.
In short: Version control in embedded projects differs less in its commands than in its time horizon. A web service is rolled out continuously and only knows the current state; a device is built, shipped and still supplied with fixes years later – alongside three other hardware revisions. The decisions that make this bearable are made at the start, but only felt after years.
What differs from software you deploy
Three characteristics shape the work:
There is not one current state but several at once. When revision C enters production, revisions A and B remain in the field and keep receiving security fixes. All three must be buildable at any time.
A substantial share of the code is not yours. Kernel, bootloader, the SoC vendor’s BSP, layers from the Yocto ecosystem. You do not maintain it, but you have to be able to ship it – even after the vendor eventually takes the repository offline.
The build process is part of the product. Whether a firmware image is bit-for-bit reproducible is not decided by application code alone, but by the combination of sources, layers, configuration and toolchain. Whatever of that is not versioned will be missing later.
Branches that match the product’s lifetime
The branch structure follows the hardware, not the calendar. A lean model works well: the main branch carries ongoing development. Every shipped device generation gets a release branch at production start, which from then on only takes fixes.
What matters is the direction of flow. A bug is fixed on the main branch and carried into the affected release branches – by cherry-pick or backport merge. The reverse route, fixing directly on the release branch and merging up later, works exactly until someone forgets the second step. Then the fix lives only on a branch nobody cares about in the next product, and the bug returns in the following generation.
Tags mark what was actually shipped. A tag without the matching build state, though, is worth little – more on that shortly.
Third-party code: submodule, copy or layer
For vendor BSPs and external sources there are three viable routes, and the choice comes down to who you trust to still serve that repository in ten years:
- Submodule. Cleanly separated, deliberately updatable, no noise in your own history. The price is a dependency on someone else’s URL – and SoC vendors tidy up their Git servers without warning.
- A copy in your own repository. Inelegant and bulky, but fully independent. For BSPs you patch anyway and touch once a year, that is often the more honest solution.
- A Yocto layer at a pinned revision. For anything built through Yocto this is the natural place: the layer stays external, the exact revision lives in your repository. Add an internal mirror and you get both separation and availability.
Kernel changes are a special case. Your own driver or a board device tree belongs in the product layer as a patch or source file, not as a local modification inside a kernel tree. Otherwise it is either gone at the next version jump or has to be reapplied laboriously.
Reproducibility: a tag is not enough
The question everything hinges on: can the state shipped three years ago still be built today? That needs three things within reach:
- The application code – the easy part, the tag covers it.
- The build description including every layer revision. In Yocto a manifest or a set of pinned
SRCREVvalues does this; without it the same layer branch points at a different commit today than it did back then. - The toolchain. It does not belong in the repository, but it must be pinned – through the Yocto recipe or a container image referenced by digest, not by
latest.
If one of the three is missing, the tag is a label without content. That gap rarely shows when it is created; it shows the moment a customer needs a fix for a device that never received an update.
Binaries, in moderation
Firmware blobs for radio modules, calibration tables, small reference images: such artefacts are part of the reproducible state and belong in the repository, sensibly through Git LFS. What does not belong are build outputs and complete SDKs – they can be recreated from source but bloat every clone permanently. The test is simple: can I recreate it exactly from what is versioned anyway? Then it does not belong in there.
Why the history should be readable
The value of a tidy history shows not while writing it but while searching it. A device in the field behaves differently since an update, nobody knows why, and git bisect is supposed to find the commit that caused it. The procedure is mechanical and reliable – on one condition: every commit must build on its own and carry a recognisable intent.
That is exactly where grown projects fail. Catch-all commits labelled “fixes”, intermediate states that do not compile, reformatting mixed with substantive changes – each of those makes the search impossible at the moment it would be worth most. For software replaced in two years that is tolerable. For a device with ten years in the field it is expensive.
Part of this is having CI build every commit, not just the state after a merge. Otherwise holes appear in the history that bisect later runs aground on.
Where to start
When cleaning up an existing project, the order pays off: first secure reproducibility – pin layer revisions and the toolchain – then adjust the branch structure to the device generations actually supported, and only then work on conventions for commits and reviews. The first two prevent concrete failures; the third makes the work permanently easier.
We work through these questions in our Git training directly on your repository – with your branching strategy and the conflicts that actually come up in your work. Where version control is part of a larger overhaul, we support that as part of our Embedded Linux development.
Frequently asked questions
- Should vendor BSPs be submodules or copied into the repository?
- Both are common, and both have a clear use case. A submodule stays cleanly separated and can be updated deliberately, but it assumes the source is still reachable in ten years – a bold assumption with SoC vendors. A copy versioned in your own repository is inelegant but independent. For anything whose availability you do not control, your own copy is the safer choice; for actively maintained open source layers, the submodule.
- How many branches does a product shipped over years need?
- As few as possible, but one per device generation still receiving updates. The main branch carries development, each shipped generation gets a release branch that only takes fixes. What matters is the direction: bugs are fixed on the main branch and carried into the release branches, never the other way round – otherwise a fix eventually lives only on a branch nobody looks at any more.
- Do binaries and toolchains belong in the Git repository?
- Firmware blobs, calibration data and similar small artefacts yes – they are part of the reproducible state, and Git LFS keeps the repository manageable. A full toolchain does not belong in it: pin it instead, through the Yocto recipe or a container image referenced by digest. The yardstick is always the same: can the state from three years ago still be built from what the repository contains?
- Is the effort for a tidy history really worth it?
- For short-lived software it is a matter of taste; for long-lived devices it is not. The moment of truth arrives years later: a device in the field shows a fault that did not exist two years ago, and `git bisect` is supposed to narrow it down. That only works if every commit builds on its own and has a recognisable intent. A history of catch-all commits labelled "fixes" makes the tool worthless exactly when it is needed most.
More on these topics
Managing Director, bitshift dynamics
Builds hardware-adjacent software for embedded products with his team – C++, Qt/QML, Embedded Linux and the Yocto Project. bitshift dynamics has worked in this field since 2005.