Skip Navigation
How to track automated "performance"-type tests over time?
  • My particular use case is actually for a hobby/fun project --- developing a bot in Rust to play a game (particularly, Screeps), and I want to track how fast it hits certain game thresholds with each newly developed feature. Gitea Actions for CI/CD, but it's all running on my local network/home lab so I'm happy to shift as needed.

  • How to track automated "performance"-type tests over time?

    I'm pretty familiar with automated tests where you're comparing a received value to an expected value (e.g. basically all unit/integration tests) --- in a CI/CD workflow, you handle test failures by failing the whole pipeline, and then that commit/PR/etc has a pipeline that failed next to it.

    However, what if I have some kind of "performance" measure I want to track, instead? Something that isn't pass/fail, but rather a set of experimental results over time? (e.g. speed of responses from an API, wins/draw/loss rates on chess bot, confusion matrix scores for a classifier, etc.) Is there a tool that can show that kind of "automated experiment" results in order by git commit, pull request, etc?

    I thought about sending the data to some kind of data store with a Grafana front-end, but I was hoping there might be some less "diy" method for creating such a display.

    7
    Lightweight CI/CD Tool (Targeting Docker on a Raspberry Pi)
  • Update --- I'm trying Gitea Actions. There's still a little weirdness with some actions (e.g. the checkoutv3 action doesn't work if your git server has a path prefix like http://my.server/some_git_specific_prefix/[rest of normal path]), but overall it seems to be working nicely.

    I'm currently running the act_runner daemon as a service on the bare raspberry pi/not in docker, since it has to interface with the docker socket anyway and there's no official act_runner docker image from gitea/I wouldn't be getting any "ease of update" benefits.

  • Lightweight CI/CD Tool (Targeting Docker on a Raspberry Pi)
  • I didn't realize that gitea was adding actions either --- it does look like their current setup expects the actions runner to be outside of docker, though: https://blog.gitea.io/2023/03/hacking-on-gitea-actions

    That's not to say it would be impossible to wrap it in a container, but their claim that they "recommend running it on a separate server from the gitea server for performance reasons" makes me wonder if it'll work on my little pi.

  • Lightweight CI/CD Tool (Targeting Docker on a Raspberry Pi)

    Anybody have recommendations for a lightweight CI/CD service that I can run on a Raspberry Pi with 8GB memory?

    I'd like to run it in Docker, and I'm the only client --- it doesn't matter to me if builds have to queue/if there's only one worker "node" on the pi, but I just want to be able to do github action-like stuff with my gitea server.

    11
    InitialsDiceBearhttps://github.com/dicebear/dicebearhttps://creativecommons.org/publicdomain/zero/1.0/„Initials” (https://github.com/dicebear/dicebear) by „DiceBear”, licensed under „CC0 1.0” (https://creativecommons.org/publicdomain/zero/1.0/)GA
    GaussianInteger @lemmy.world
    Posts 2
    Comments 4