[solution-checker] Fail job on Medium severity issue

[garrytrinder]

Not sure if this is documented anywhere but I'd like to be able to fail a job if the solution-checker returns medium severity issues.

When building Canvas Apps it is very easy to overlook the accessibility checker warnings which are raised as Medium severity warnings. I'd like to be able to ensure that we have quality gates in place so that developers cannot overlook this so failing the job will help enforce this.

[jjkinsey]

Seconding this! It would be great to e.g. have access to the counts of each error type as outputs of the action--that way we can create whatever checks are necessary for a given use of the check-solution action (maybe different standards for releasing to testing vs. production).

[echvsg]

@garrytrinder or @jjkinsey : Were you using service principal to run this check-solution action. For some reason it says the service principal is disabled but i can use the same service principal without an issue from pac cli check solution command.

Error: AADSTS500014: The service principal for resource 'https://service.powerapps.com/' is disabled. This indicate that a subscription within the tenant has lapsed, or that the administrator for this tenant has disabled the application, preventing tokens from being issued for it.

Any thoughts?