USER STORY:

As a Support Engineer working to review the partner results provided by OPCT, I would like to be able to extract insights from the results archive submitted by the partner, digging into the errors and details of the failed tests, and provide a feedback to the partner guiding to solve the problem, before scaling to the engineering teams.

As a Partner I would like to have more accuracy on the results, specially the failed tests, to keep the focus on the problems in my environment, so I can fix it proactively before sending to Red Hat.

DESCRIPTION:

Automate the many steps as possible used while reviewing the certification results. The idea is to decrease the manual steps and keep the focus on the results.

Due the amount of failures, many false-negative or flakes, we've identified many manual steps until keep the focus on the problems identified on the Partner's cluster. The steps have been discussed with the team and taking the notes on the draft document[1]. The idea here is to automate all the steps as possible identified on the draft, then document it to be used internally for the certification reviewers.

In the future, when the dependencies (sub-tasks) have been solved, it can be used by the partner before submitting the results, so we can decrease the interaction and provide the partners the autonomy to identify and fix problems before submitting the results.

Required:

• the command `report` should automate the many manual steps when running the Provider Certification archive review.
• both automated and manual steps should be described on the documentation
•  

Nice to have:

...

ACCEPTANCE CRITERIA:

• CI tests passing
• PR for the `report` command merged

ENGINEERING DETAILS:

 [1]  draft document with the ideas to review the certification results.