Is there software that tracks internal dependencies for CI/CD?
Is there software that tracks internal dependencies for CI/CD?
Here's a hypothetical scenario at a company: We have 2 repos that builds and deploys code as tools and libraries for other apps at the company. Let's call this lib1 and lib2.
There's a third repo, let's call it app, that is application code that depends on lib1 and lib2.
The hard part right now is keeping track of which version of lib1 and lib2 are packaged for app at any point in time.
I'd like to know at a glance, say 1 month ago, what versions of app is deployed and what version of lib1 and lib2 they were using. Ideally, I'm looking for a software solution that would be agnostic to any CI/CD build system, and doubly ideally, an open source one. Maybe a simple web service you call with some metadata, and it displays it in a nice UI.
Right now, we accomplish this by looking at logs, git commit history, and stick things together. I know I can build a custom solution pretty easily, but I'm looking for something more out-of-the-box.
It's probably more than you'd need for just two dependencies, but Apache's Ivy does more or less exactly that (although admittedly without a nice web UI)
Another solution for simple builds could be to use git submodules to include your libraries, that way for any commit of the top level you can see exactly what commit of the lower levels you're using
This is one of the situations where subversion is better than git - svn "extern" can tag a specific revision of the library to be associated with each branch & revision of each project. Svninfo in each directory will tell your tools the version currently in each library the program is using.
I realize that you're looking for audit style information. Disclaimer: I'm not offering that 😅
You may be interested in automation that helps manage these dependencies and their version. Github offers dependabot as a service for repos on github.com. There's also renovate which is a little newer and I think has a better open source reputation. You can use them to fully automate dependency patching. Though I suspect most teams use these tools to automatically build PRs for a human to review and test.
I believe you're already on the right track.
Your pipeline keeps track of the git commit that resulted in each build/deploy. You can use that (
curlyour CI/CD API and feed it intojq) to check out the build definition file forapp(egapp/build.gradle) from that particular revision, and simplygrepforlib1andlib2. It should technically be possible to do this in a few lines of shell script.The metadata you want is called a Software Bill of Materials, and there are a range of tools for generating them. Some generic ones include Trivy and Grype, but you may also find some for your language ecosystem by Googling ' + SBOM'.
One tool you can use to view these versions with a web UI is OWASP Dependency-Track.
All of the tools mentioned and linked above are F/OSS.