Test attestation Package

Produce a violation if any test result attestation has an erred result. The result type is configurable by the "erred_test_attestation_results" key in the rule data.

Solution: A test attestation has an erred result, indicating an infrastructure or execution failure. Review the test attestation and re-run the test.

Produce a warning if any informative test attestation has a failed result. Informative tests produce warnings instead of violations, allowing teams to roll out new tests without blocking releases. The list of informative tests is configurable by the "informative_test_attestations" key, and the result type by the "failed_test_attestation_results" key in the rule data.

Solution: An informative test attestation has a failed result. While this does not block the release, review the test attestation output for details.

Produce a violation if any non-informative test result attestation has a failed result. Failed test names from the attestation predicate are included in the message when available. The result type is configurable by the "failed_test_attestation_results" key, and the list of informative tests by the "informative_test_attestations" key in the rule data.

Solution: Ensure all test attestations have a passing result. Review the test attestation output for details.

Produce a violation if any test result attestation has a skipped result. A skipped result means a pre-requirement for executing the test was not met. The result type is configurable by the "skipped_test_attestation_results" key in the rule data.

Solution: A test attestation was skipped, indicating a missing prerequisite such as a scanner license. Ensure prerequisites are available and re-run the test.

Produce a warning if any test result attestation has a warned result. Warned test names from the attestation predicate are included in the message when available. The result type is configurable by the "warned_test_attestation_results" key in the rule data.

Solution: Review the test attestation output for warning details.

Ensure the result field of each test result attestation is a recognized value. Valid values are configurable by the "supported_test_attestation_results" key in the rule data. Defaults are PASSED, WARNED, FAILED, ERROR, and SKIPPED per the in-toto test-result predicate specification.

Solution: The test result attestation contains an unrecognized result value. Valid values are configurable via rule data.

Confirm the expected rule data keys have been provided in the expected format. The keys are "supported_test_attestation_results", "failed_test_attestation_results", "erred_test_attestation_results", "skipped_test_attestation_results", "warned_test_attestation_results", and "informative_test_attestations".

Solution: If provided, ensure the rule data is in the expected format.

Each test result attestation must include a result field in its predicate. Verify that the result field is present.

Solution: The test result attestation predicate must include a "result" field with a recognized value such as PASSED, WARNED, or FAILED.

Ensure every signature-verified test-result attestation provides a valid RFC 3339 timestamp. The timestamp is required to select the latest retry deterministically.

Solution: Set predicate.timestamp to the RFC 3339 instant when the test result was produced.

Ensure every test-result attestation provides a non-empty string in predicate.configuration[0].name. The name identifies repeated executions of the same integration test when selecting its latest result.

Solution: Set predicate.configuration[0].name to the stable name of the integration test that produced the attestation.

Verify that each test-result attestation’s subject includes the digest of the image being evaluated. An attestation produced for a different image should not satisfy this image’s test requirements.

Solution: The test result attestation was produced for a different image than the one being evaluated. Ensure the test pipeline produces attestations with the correct subject digest.