Move benchmark orchestration from GH actions to a bot #5950
Replies: 1 comment
-
|
Should you wish to see the wonder 🌭 that underlies alamb-ghbot it is https://github.com/alamb/datafusion-benchmarking/tree/main/scripts#cookbook-requesting-benchmarks |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
-
Just moving this dicussion from Slack to make it part of the historic record.
After some effort, @joseph-isaacs and I managed to make the current benchmarking flow work for PRs coming from forks, but they can't comment on the PRs due to github's permission model and security concerns.
I've been thinking about moving that flow into a bot based one instead of using actions and labels, It will still be able to comment on PRs and do everything else we currently use, just without the limitations of gh actions permission model, and potentially it could be more flexible.
datafusion does something similar (AKA andrewlamb-bot), it looks pretty good imo, but I haven't looked at the code. IIRC Github's API is pretty well documented so I don't expect it to be a major effort once the initial issues are ironed out.
The bot will roughly have the following flow:
vortex-data/vortex./bot run benchmarkor something like it), it will trigger a GH action run on in that repo, but pulling the PR's branch using git or the gh cli.vortex-data/vortex.We can also use it for the post-merge benchmarks that update the website, it'll just require shifting some code over to that repo.
Beta Was this translation helpful? Give feedback.
All reactions