Skip to content

Acceptance tests for secrets validator#1987

Open
alessandro-Doyensec wants to merge 9 commits intogoogle:mainfrom
doyensec:acceptance-tests-for-secrets-validator
Open

Acceptance tests for secrets validator#1987
alessandro-Doyensec wants to merge 9 commits intogoogle:mainfrom
doyensec:acceptance-tests-for-secrets-validator

Conversation

@alessandro-Doyensec
Copy link
Copy Markdown
Collaborator

This PR adds acceptance validator tests.

After the addition of acceptance tests these plugin failed the test:

  • github.com/google/osv-scalibr/veles/secrets/awsaccesskey | An errResp.Error.Code was missing and the validator returned veles.ValidationFailed instead of veles.ValidationInvalid
  • github.com/google/osv-scalibr/veles/secrets/cloudflareapitoken | The service returns http.StatusBadRequest on malformed token
  • github.com/google/osv-scalibr/veles/secrets/cratesioapitoken | The service returns http.StatusBadRequest on malformed token
  • github.com/google/osv-scalibr/veles/secrets/grokxaiapikey | InvalidResponseCodes were not specified leading to false positives, needs further investigation
  • github.com/google/osv-scalibr/veles/secrets/salesforceoauth2access | The service returns 403 for bad ouauth token, needs further investigation
  • github.com/google/osv-scalibr/veles/secrets/salesforceoauth2client | The service is not testable online
  • github.com/google/osv-scalibr/veles/secrets/salesforceoauth2refresh | The request was poorly formatted and resulted in the validator always failing
  • github.com/google/osv-scalibr/veles/secrets/gcshmackey | The request creation was always failing because the host header was missing. I confirmed that the validator is now properly working.
Test output
➜  osv-scalibr git:(acceptance-tests-for-secrets-detector) ✗ ONLINE=1 go test \
github.com/google/osv-scalibr/veles/secrets/awsaccesskey \
github.com/google/osv-scalibr/veles/secrets/cloudflareapitoken \
github.com/google/osv-scalibr/veles/secrets/cratesioapitoken \
github.com/google/osv-scalibr/veles/secrets/grokxaiapikey \
github.com/google/osv-scalibr/veles/secrets/salesforceoauth2access \
github.com/google/osv-scalibr/veles/secrets/salesforceoauth2client \
github.com/google/osv-scalibr/veles/secrets/salesforceoauth2refresh \
-run 'TestAccept.*Validator' -v
=== RUN   TestAcceptValidator
=== RUN   TestAcceptValidator/cancelled-ctx
=== RUN   TestAcceptValidator/unreachable-service
=== RUN   TestAcceptValidator/true-negatives
StatusCode: 403 Forbidden
Response: <ErrorResponse xmlns="https://sts.amazonaws.com/doc/2011-06-15/">
  <Error>
    <Type>Sender</Type>
    <Code>InvalidClientTokenId</Code>
    <Message>The security token included in the request is invalid.</Message>
  </Error>
  <RequestId>e1de87b3-d8e9-4fec-873c-280dbd3bc366</RequestId>
</ErrorResponse>

    accept_validator.go:170: Validate() with true-negative secret returned status VALIDATION_FAILED, want VALIDATION_INVALID
    accept_validator.go:173: Validate() with true-negative secret returned unexpected error: unknown error code: "InvalidClientTokenId"
--- FAIL: TestAcceptValidator (0.47s)
    --- PASS: TestAcceptValidator/cancelled-ctx (0.00s)
    --- PASS: TestAcceptValidator/unreachable-service (0.00s)
    --- FAIL: TestAcceptValidator/true-negatives (0.47s)
FAIL
FAIL	github.com/google/osv-scalibr/veles/secrets/awsaccesskey	1.049s
=== RUN   TestAcceptValidator
=== RUN   TestAcceptValidator/cancelled-ctx
=== RUN   TestAcceptValidator/unreachable-service
=== RUN   TestAcceptValidator/true-negatives
StatusCode: 400 Bad Request
Response: {"success":false,"errors":[{"code":6003,"message":"Invalid request headers","error_chain":[{"code":6111,"message":"Invalid format for Authorization header"}]}],"messages":[],"result":null}
    accept_validator.go:170: Validate() with true-negative secret returned status VALIDATION_FAILED, want VALIDATION_INVALID
    accept_validator.go:173: Validate() with true-negative secret returned unexpected error: https://api.cloudflare.com/client/v4/zones: unexpected HTTP status 400
--- FAIL: TestAcceptValidator (0.30s)
    --- PASS: TestAcceptValidator/cancelled-ctx (0.00s)
    --- PASS: TestAcceptValidator/unreachable-service (0.00s)
    --- FAIL: TestAcceptValidator/true-negatives (0.30s)
FAIL
FAIL	github.com/google/osv-scalibr/veles/secrets/cloudflareapitoken	0.514s
=== RUN   TestAcceptValidator
=== RUN   TestAcceptValidator/cancelled-ctx
=== RUN   TestAcceptValidator/unreachable-service
=== RUN   TestAcceptValidator/true-negatives
StatusCode: 401 Unauthorized
Response: {"errors":[{"detail":"The given API token does not match the format used by crates.io. Tokens generated before 2020-07-14 were generated with an insecure random number generator, and have been revoked. You can generate a new token at https://crates.io/me. For more information please see https://blog.rust-lang.org/2020/07/14/crates-io-security-advisory.html. We apologize for any inconvenience."}]}
    accept_validator.go:170: Validate() with true-negative secret returned status VALIDATION_FAILED, want VALIDATION_INVALID
    accept_validator.go:173: Validate() with true-negative secret returned unexpected error: https://crates.io/api/v1/crates/osvscalibr361aa9c83e8d69e1/owners: unexpected HTTP status 401
--- FAIL: TestAcceptValidator (0.60s)
    --- PASS: TestAcceptValidator/cancelled-ctx (0.00s)
    --- PASS: TestAcceptValidator/unreachable-service (0.00s)
    --- FAIL: TestAcceptValidator/true-negatives (0.60s)
FAIL
FAIL	github.com/google/osv-scalibr/veles/secrets/cratesioapitoken	0.991s
=== RUN   TestAcceptAPIValidator
=== RUN   TestAcceptAPIValidator/cancelled-ctx
=== RUN   TestAcceptAPIValidator/unreachable-service
=== RUN   TestAcceptAPIValidator/true-negatives
StatusCode: 400 Bad Request
Response: {"code":"Client specified an invalid argument","error":"Incorrect API key provided: gr***00. You can obtain an API key from https://console.x.ai."}
    accept_validator.go:170: Validate() with true-negative secret returned status VALIDATION_VALID, want VALIDATION_INVALID
--- FAIL: TestAcceptAPIValidator (0.19s)
    --- PASS: TestAcceptAPIValidator/cancelled-ctx (0.00s)
    --- PASS: TestAcceptAPIValidator/unreachable-service (0.00s)
    --- FAIL: TestAcceptAPIValidator/true-negatives (0.19s)
=== RUN   TestAcceptManagementAPIValidator
=== RUN   TestAcceptManagementAPIValidator/cancelled-ctx
=== RUN   TestAcceptManagementAPIValidator/unreachable-service
=== RUN   TestAcceptManagementAPIValidator/true-negatives
StatusCode: 401 Unauthorized
Response: {"code":16, "message":"Invalid bearer token. Please ensure you use a valid management key.", "details":[]}
--- PASS: TestAcceptManagementAPIValidator (0.41s)
    --- PASS: TestAcceptManagementAPIValidator/cancelled-ctx (0.00s)
    --- PASS: TestAcceptManagementAPIValidator/unreachable-service (0.00s)
    --- PASS: TestAcceptManagementAPIValidator/true-negatives (0.41s)
FAIL
FAIL	github.com/google/osv-scalibr/veles/secrets/grokxaiapikey	1.343s
=== RUN   TestAcceptValidator
=== RUN   TestAcceptValidator/cancelled-ctx
=== RUN   TestAcceptValidator/unreachable-service
=== RUN   TestAcceptValidator/true-negatives
StatusCode: 403 Forbidden
Response: Bad_OAuth_Token
    accept_validator.go:170: Validate() with true-negative secret returned status VALIDATION_FAILED, want VALIDATION_INVALID
    accept_validator.go:173: Validate() with true-negative secret returned unexpected error: https://login.salesforce.com/services/oauth2/userinfo: unexpected HTTP status 403
--- FAIL: TestAcceptValidator (0.17s)
    --- PASS: TestAcceptValidator/cancelled-ctx (0.00s)
    --- PASS: TestAcceptValidator/unreachable-service (0.00s)
    --- FAIL: TestAcceptValidator/true-negatives (0.17s)
FAIL
FAIL	github.com/google/osv-scalibr/veles/secrets/salesforceoauth2access	1.282s
=== RUN   TestAcceptValidator
=== RUN   TestAcceptValidator/cancelled-ctx
=== RUN   TestAcceptValidator/unreachable-service
=== RUN   TestAcceptValidator/true-negatives
    accept_validator.go:170: Validate() with true-negative secret returned status VALIDATION_FAILED, want VALIDATION_INVALID
    accept_validator.go:173: Validate() with true-negative secret returned unexpected error: https://yuvrajapp.my.salesforce.com/services/oauth2/token: HTTP request failed: Post "https://yuvrajapp.my.salesforce.com/services/oauth2/token": dial tcp: lookup yuvrajapp.my.salesforce.com: no such host
--- FAIL: TestAcceptValidator (1.05s)
    --- PASS: TestAcceptValidator/cancelled-ctx (0.00s)
    --- PASS: TestAcceptValidator/unreachable-service (0.00s)
    --- FAIL: TestAcceptValidator/true-negatives (1.05s)
FAIL
FAIL	github.com/google/osv-scalibr/veles/secrets/salesforceoauth2client	2.333s
=== RUN   TestAcceptValidator
=== RUN   TestAcceptValidator/cancelled-ctx
=== RUN   TestAcceptValidator/unreachable-service
=== RUN   TestAcceptValidator/true-negatives
StatusCode: 400 Bad Request
Response: {"error":"unsupported_grant_type","error_description":"grant type not supported"}
    accept_validator.go:170: Validate() with true-negative secret returned status VALIDATION_FAILED, want VALIDATION_INVALID
    accept_validator.go:173: Validate() with true-negative secret returned unexpected error: https://login.salesforce.com/services/oauth2/token: unexpected HTTP status 400
--- FAIL: TestAcceptValidator (0.18s)
    --- PASS: TestAcceptValidator/cancelled-ctx (0.00s)
    --- PASS: TestAcceptValidator/unreachable-service (0.00s)
    --- FAIL: TestAcceptValidator/true-negatives (0.18s)
FAIL
FAIL	github.com/google/osv-scalibr/veles/secrets/salesforceoauth2refresh	1.112s
FAIL

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant