Description of problem:

Cusotmer configured a custom certificate with a custom "secretName" in their "DataScienceCluster":

apiVersion: datasciencecluster.opendatahub.io/v1
kind: DataScienceCluster
metadata:
  name: default-dsc
spec:
  components:
    ...
    kserve:
      managementState: Managed
      serving:
        ingressGateway:
          certificate:
            secretName: example-wildcard-certificate
            type: Provided
        managementState: Managed
        name: knative-serving

However, it seems RHOAI / knative is ignoring the provided Secret name and still searched for the default secret which is automatically generated when one does not specifiy the type "Provided". This lead to the knative Service staying in "Uninitialized". In this case the secret is automatically generated by RHOAI / knative and put into the "istio-system namespace".

As a workaround, customer created the certificate secret with the name "knative-serving-cert" in the "istio-system namespace". After this step (and after restarting the istiod-data-science-smcp-xxx pod), the knative service became ready.

Version-Release number of selected component (if applicable):

rhods-operator.2.7.0

OpenShift Container Platform 4.13.17

How reproducible:

Always

Steps to Reproduce:

    1. Install OpenShift AI 2.7.0 according to documentation
    2. To configure a custom certificate, follow the steps outlined in https://access.redhat.com/documentation/en-us/red_hat_openshift_ai_self-managed/2.7/html/serving_models/serving-large-models_serving-large-models#configuring-automated-installation-of-kserve_serving-large-models
    3. Instead of naming the Secret "knative-serving-cert", name it something else and still configure the "secretName" in the DataScienceCluster

Actual results:

Secret is not used as expected. Istio fails with the following error message:

oc logs -n istio-system istiod-data-science-smcp-6767877fbb-r2q2v | grep warn | grep secret | tail -n 5
2024-03-13T13:50:25.737407603Z 2024-03-13T13:50:25.737377Z	warn	ads	failed to fetch key and certificate for kubernetes://knative-serving-cert: secret istio-system/knative-serving-cert not found

Expected results:

The custom certificate is being used.

Additional info:

• must-gather is available in attached Support Case