Multi-tenant Installations

Multi-tenant Guidance

We suggest running Seldon Deploy in cluster-wide mode. If the full stack of suggested components is used then this makes it simpler to scale by adding new namespaces dynamically for new departments/users.

If you are able to use cluster-wide permissions, you don’t need to read this guide. Use one of the other install types.

The below is guidance on restricting the use of cluster-wide permissions.

Optional Components

Istio

Istio currently has very limited support for multi-tenancy.

There are alternative flavours of istio. OpenShift Service Mesh does have some multi-tenancy support.

An alternative is Ambassador, which can be installed namespaced.

If a non-default ingress setup is used then the requestForm and curlForm sections in the values file need to be configured with templates applicable to the ingress URL rules being used.

Argocd

From our testing, argocd does require read permissions at a cluster-wide level on all objects.

Write permissions can however be entirely namespaced. Using this means configuring roles for each new namespace, as explained in the Argocd GitHub issue.

Argocd is only required in order to use Gitops. Seldon Deploy can be used without Gitops.

We have a MULTITENANT flag in the deploy config file for a Trial Installation. This is used in the scripts provided in the download of installation resources.

Included in those scripts is seldon-deploy-install/sd-setup/gitops-setup.sh. The Trial Installation docs explain how to use it. Some of its steps read the MULTITENANT flag.

If using the Product Installation with gitops then this script should be consulted to adapt the steps for a multi-tenant cluster. The main differences are:

• The argocd installation is different as it uses different roles from the manifests that Argocd publishes.
• For each new namespace added to gitops, we have to add roles for Argocd to be able to sync it.
• For each new namespace a new instance of Seldon Core is installed, dedicated to that namespace.
• For each new namespace new roles are added for Seldon Deploy to be able to operate on that namespace.

The details of these differences can be found in the script.

Seldon Core

Seldon Core supports a namespaced install with a singleNamespace setting in its helm chart. There is an example in seldon-deploy-install/sd-setup/gitops-setup.sh.

KFServing

Only cluster-wide is currently supported. Also requires knative.

Minio

We suggest installing with the minio helm chart. By default we setup one instance per cluster but that could be configured and at the time of writing ClusterRoles are used only on OpenShift

Argo

We suggest using argo workflows default installation which is cluster-wide but namespaced installations are possible.

Note that each Seldon namespace that runs batch jobs would need the ability to start argo workflows. Batch jobs are optional.

Knative

At the time of writing neither knative serving nor knative eventing seems to officially support multi-tenancy. See below.

Configuring Seldon Deploy

Some functionality depends on knative. If knative is not used then this affects the request logger configuration. (Not using knative also means the outlier and drift detector demos won’t work.)

Seldon Core has a default URL that its install instructions point to a knative broker in seldon-logs namespace (executor.requestLogger.defaultEndpoint for seldon core). This can be changed to point to seldon-request-logger.seldon-logs (or your request logger location).

Detector component wizards and the Data Science metrics component wizard do need knative 0.18.

The Seldon Deploy helm chart has a rbac.clusterWide setting. The seldon-deploy-install/sd-setup/sd-install-default script uses the MULTITENANT flag to set this.

There are two options from here.

Option 1 - One Deploy, per-Namespace RBAC

This is reduced RBAC rather than true multi-tenant.

If rbac.readNamespaces is true then a single Deploy instance should be used. This option requires a cluster-level read role to read namespaces.

When each namespace is added then new roles can be added for that namespace using the apply-role-namespace.sh or gitops-setup.sh script.

Note that the gitops script restarts Deploy each time a namespace is added.

Option 2 - Deploy Instance Per Namespace

This is full multi-tenant using only namespaced roles.

If rbac.readNamespaces is false then a Deploy per namespace should be used. Each Deploy then looks only at its own namespace.

The request logger namespace is configurable. This could be a dedicated namespace per Deploy instance or the same namespace as Deploy.