Support reporting status checks and comments from PR rules#6218
Support reporting status checks and comments from PR rules#6218krrish175-byte wants to merge 14 commits into
Conversation
|
You'll need to sync your proto changes with |
evankanderson
left a comment
There was a problem hiding this comment.
Very nice work, though we may want to consider adding either target_url or description as fields from the proto message, possibly with Go templating for the latter, and RFC6570 templates for the URL.
With respect to the target_url, I'm also trying to figure out whether we should offer some sort of "Rule Status URL" endpoint that could be auto-filled based on server-known parameters. We'd also need to figure out permissions on that endpoint, which is a separate work track. For now, I'd simply allow users to template their own URL, and we can add a separate automagic_url option (bool) in the future that's exclusive with the detail_url on the commit status message.
| optional RestType rest = 2; | ||
| optional GhBranchProtectionType gh_branch_protection = 3; | ||
| optional PullRequestRemediation pull_request = 4; | ||
| optional Alert.AlertTypePRComment pull_request_comment = 5; |
There was a problem hiding this comment.
Try pulling the AlertTypePRComment message out of the Alert message, and see whether buf flags that as a breaking change. We had some early use of nested messages that went a bit overboard for these use cases.
Also, if we are going to add PR comments as a remediation type, it would be good to understand (document) in an issue or the PR description when people should use the "remediate" vs "alert" type. A few options:
- We'll automatically convert "alert: pr_comment" to "remediate: pr_comment"
- What happens to people who have policies remediation enabled but not alerting, or vice-versa?
- We'll support all options on both types
- how do we explain the differences to people?
- We'll have different options on the two types -- e.g. only remediate can block PR merges
- Should this be a separate type then?
I'm not trying to be a pain, but it's a lot easier to jump into a mess than to get ourselves back out. 😁
| Remediate remediate = 6; | ||
|
|
||
| message Alert { | ||
| string type = 1 [ |
There was a problem hiding this comment.
We probably should have included a comment here about which types correspond to which entities as well.
| message AlertTypeCommitStatus {} | ||
| optional AlertTypeCommitStatus commit_status = 4; |
There was a problem hiding this comment.
What is this / what is it used for?
| zerolog.Ctx(ctx).Debug().Str("rule-type", ruletype.GetName()). | ||
| Msg("provider is not a GitHub provider. Silently skipping alerts.") | ||
| return noop.NewNoopAlert(ActionType) |
There was a problem hiding this comment.
Ooof, this is a cleanup opportunity. The Security Alerts feature is GitHub specific, but it should be implemented as a Provider API such that we can do the equivalent GitLab, Forgejo, etc operation. Commit Statuses should be handled similarly, though probably as a different "facet", so a provider could implement one but not the other.
Given the other set of changes here, I think filing an issue to track this in a future PR is the right step.
There was a problem hiding this comment.
Let's file this issue before we submit the PR, so we don't forget about it.
| // Context string for the commit status check (e.g. "minder/rule_name") | ||
| // If rule_name isn't available, we fallback to "minder" | ||
| contextStr := "minder" | ||
| if rule := actionParams.GetRule(); rule != nil { | ||
| contextStr = fmt.Sprintf("minder/%s", rule.Name) | ||
| } |
There was a problem hiding this comment.
Oh, this is interesting! I would have expected the name of the status check to be in the rule definition. You may also want to distinguish between rule type and rule name (our nomenclature here isn't always consistent, you basically have several "name-like" things:
- The
namefield from the profile, if set. If not set, I think there's some defaulting here from either 3 or 2, and I don't recall the exact rules - The
typefield from the profile, which matches thenamefrom the ruletype - The
display_namefrom the ruletype
|
While setting up testing for another PR, I realized that we may also need to change the GitHub App recommended permissions to allow updating status checks. You also have some lint errors that might be fixed by |
|
Good catch on both fronts. Lint errors: I will run make lint-fix and address any remaining issues manually in an upcoming push. GitHub App permissions: That makes sense, we would need statuses: write for the commit status API calls. I will file a separate issue to track the permissions update so it doesn't block this PR. |
evankanderson
left a comment
There was a problem hiding this comment.
Looking a little bit more deeply, I'm wondering if we should be using "Check Runs" instead of "Status Checks":
https://stackoverflow.com/questions/67919168/github-checks-api-vs-check-runs-vs-check-suites
Pros of Status Check
- Simpler GitHub API
- Works with other types of tokens than GitHub App
Pros of Check Runs
- Automatic grouping under the GitHub App (e.g. "minder")
- Ability to add line-by-line annotations
- Detail page with markdown, and summary images in addition to URL
| // type is the type of the remediation. | ||
| // * 'rest' can be used with any entity type. | ||
| // * 'gh_branch_protection' can only be used with the 'repository' entity type. | ||
| // * 'pull_request' and 'pull_request_comment' can only be used with the 'pull_request' entity type. |
There was a problem hiding this comment.
I think pull_request opens a new PR on the repo, so it applies to the "repository" type, not the "pull_request" type.
|
|
||
| replace github.com/mindersec/minder => ../. | ||
|
|
||
| replace github.com/docker/docker => github.com/docker/docker v28.0.0+incompatible |
There was a problem hiding this comment.
This is probably not right -- I'm assuming this was in hopes of fixing some of the security warnings?
| zerolog.Ctx(ctx).Debug().Str("rule-type", ruletype.GetName()). | ||
| Msg("provider is not a GitHub provider. Silently skipping alerts.") | ||
| return noop.NewNoopAlert(ActionType) |
There was a problem hiding this comment.
Let's file this issue before we submit the PR, so we don't forget about it.
|
|
||
| // Create template params | ||
| tmplParams := &CommitStatusTemplateParams{ | ||
| EvalErrorDetails: enginerr.ErrorAsEvalDetails(params.GetEvalErr()), |
There was a problem hiding this comment.
How would this be used / what does it show up as?
| contextStr := "minder" | ||
| contextNameTmpl := alert.alertCfg.GetStatusName() | ||
| if contextNameTmpl != "" { | ||
| // No templating specified for status_name, use as-is | ||
| contextStr = contextNameTmpl | ||
| } else if tmplParams.RuleName != "" { | ||
| contextStr = fmt.Sprintf("minder/%s", tmplParams.RuleName) | ||
| } | ||
| result.Context = contextStr |
There was a problem hiding this comment.
With Go 1.22 and later, you can use cmp.Or for some of this:
| contextStr := "minder" | |
| contextNameTmpl := alert.alertCfg.GetStatusName() | |
| if contextNameTmpl != "" { | |
| // No templating specified for status_name, use as-is | |
| contextStr = contextNameTmpl | |
| } else if tmplParams.RuleName != "" { | |
| contextStr = fmt.Sprintf("minder/%s", tmplParams.RuleName) | |
| } | |
| result.Context = contextStr | |
| contextStr := "minder" | |
| if tmplParams.RuleName != "" { | |
| contextStr = fmt.Sprintf("%s/%s", contextStr, tmplParams.RuleName) | |
| } | |
| result.Context = cmp.Or(alert.alertCfg.GetStatusName(), contextStr) |
| // 2. Evaluate description, using text/template | ||
| descTmplStr := alert.alertCfg.GetDescription() | ||
| if descTmplStr != "" { | ||
| descTmpl, err := util.NewSafeHTMLTemplate(&descTmplStr, "description") |
There was a problem hiding this comment.
Can "description" be HTML content? (I don't know, but the GitHub API doesn't say anything about HTML: https://docs.github.com/en/rest/commits/statuses?apiVersion=2026-03-10#create-a-commit-status)
| if err != nil { | ||
| return nil, fmt.Errorf("cannot parse description template: %w", err) | ||
| } | ||
| descStr, err := descTmpl.Render(ctx, tmplParams, 1024) |
There was a problem hiding this comment.
Extract the 1024 to a const parameter
| argsMap := map[string]any{ | ||
| "owner": result.Owner, | ||
| "repo": result.Repo, | ||
| "commit_sha": result.CommitSha, | ||
| "rule_name": tmplParams.RuleName, | ||
| } |
There was a problem hiding this comment.
This feels like it should include output, so you could have a rule that pokes some external system, gets back a URL for the status check, and then loads that URL into the status check.
|
Sorry for jumping around a bit on the status checks; I threw it out there as something which had been previously discussed, but I hadn't actually researched it until after you sent your PR, so I didn't really have any sort of design in mind. |
|
Also, it would be good to update https://github.com/mindersec/minder/blob/main/docs/docs/run_minder_server/config_provider.md alongside any changes to necessary GitHub app permissions, since they are in the same repo. |
evankanderson
left a comment
There was a problem hiding this comment.
I'm marking this PR as "request changes" (needs merge conflicts resolved) to help track which outstanding PRs need maintainer action vs contributor action.
|
This PR needs additional information before we can continue. It is now marked as stale because it has been open for 30 days with no activity. Please provide the necessary details to continue or it will be closed in 30 days. |
Summary
This pull request implements the enhancement requested in Issue #5977, allowing generic PR rules (e.g., Rego rules) to report status checks and leave PR comments natively without relying on custom Go evaluation engines like
vulncheck.Changes included:
commit_statusas an available Alert type.pull_request_commentas an available Remediate type.commit_statusaction underinternal/engine/actions/alert/commit_statuswhich interfaces with the GitHubSetCommitStatusAPI to surface a success/failure check run directly on the PR.pull_request_commentalert module into theRemediateregistry so it can be utilized in the remediation phase without duplicating logic.Fixes #5977
Testing
Outline how the changes were tested, including steps to reproduce and any relevant configurations.
commit_statusaction (commit_status_test.go) utilizingmockghclient.pull_request_commentcorrectly registers inside the Remediator engine.make buf-generateto ensure all generated protobuf Go bindings are clean and up to date.go test ./...across the entireinternal/engine/actions/...package, with all tests passing successfully.