[4.12][sig-arch] Check if alerts are firing during or after upgrade success - alert NoRunningOvnMaster fired

[sig-arch] Check if alerts are firing during or after upgrade success is failing in 4.12 z-stream upgrade-rollback job. The job is testing z stream rollback by installing 4.12.0, updating towards a recent 4.12 nightly, and then, at some random point during that update, rolling back to 4.12.0.

The error is

{Sep  4 14:21:09.606: Unexpected alerts fired or pending during the upgrade:

alert NoRunningOvnMaster fired for 30 seconds with labels: {namespace="openshift-ovn-kubernetes", severity="critical"} Failure Sep  4 14:21:09.606: Unexpected alerts fired or pending during the upgrade:

alert NoRunningOvnMaster fired for 30 seconds with labels: {namespace="openshift-ovn-kubernetes", severity="critical"}

github.com/openshift/origin/test/extended/util/disruption.(*chaosMonkeyAdapter).Test(0xc005855540, 0xc001749170)
	github.com/openshift/origin/test/extended/util/disruption/disruption.go:197 +0x315
k8s.io/kubernetes/test/e2e/chaosmonkey.(*Chaosmonkey).Do.func1()
	k8s.io/kubernetes@v1.25.0/test/e2e/chaosmonkey/chaosmonkey.go:94 +0x6a
created by k8s.io/kubernetes/test/e2e/chaosmonkey.(*Chaosmonkey).Do
	k8s.io/kubernetes@v1.25.0/test/e2e/chaosmonkey/chaosmonkey.go:91 +0x8b}

The failed job is https://prow.ci.openshift.org/view/gs/origin-ci-test/logs/periodic-ci-openshift-release-master-nightly-4.12-e2e-aws-ovn-upgrade-rollback-oldest-supported/1698669458735763456

4.12.0-0.nightly-2023-09-04-115546

Flaky

1.
2.
3.

{Sep  4 14:21:09.606: Unexpected alerts fired or pending during the upgrade:

alert NoRunningOvnMaster fired for 30 seconds with labels: {namespace="openshift-ovn-kubernetes", severity="critical"} Failure Sep  4 14:21:09.606: Unexpected alerts fired or pending during the upgrade:

alert NoRunningOvnMaster fired for 30 seconds with labels: {namespace="openshift-ovn-kubernetes", severity="critical"}

github.com/openshift/origin/test/extended/util/disruption.(*chaosMonkeyAdapter).Test(0xc005855540, 0xc001749170)
	github.com/openshift/origin/test/extended/util/disruption/disruption.go:197 +0x315
k8s.io/kubernetes/test/e2e/chaosmonkey.(*Chaosmonkey).Do.func1()
	k8s.io/kubernetes@v1.25.0/test/e2e/chaosmonkey/chaosmonkey.go:94 +0x6a
created by k8s.io/kubernetes/test/e2e/chaosmonkey.(*Chaosmonkey).Do
	k8s.io/kubernetes@v1.25.0/test/e2e/chaosmonkey/chaosmonkey.go:91 +0x8b}