Deployment Models

Deployment models for Seldon Deploy.

When it comes to deploying Seldon Deploy for a production workload, there are different deployment models that can apply depending on your use case. For example, you can consider the following questions:

  • Do you want to support multiple environments (e.g. production, staging, etc.)? Will these be split across multiple clusters? Or across different namespaces?
  • Are you planning on supporting multiple tenants in your infrastructure? How will you manage permissions for each one of them?

This section will showcase some of the most common deployment models to set up Seldon Deploy in your cluster.

Multiple environments

When you are managing machine learning models, a common use case is to test them first on some kind of test environment, isolated from production workloads. These test environments are usually known as staging, qa, pre-prod, etc.

In some cases, these environments will form a deployment (or continous delivery) pipeline, where each environment is considered more stable than the previous one:

Dev > ... > Staging > Production

Seldon Deploy allows you to have multiple environments by separating them as namespaces or clusters.

Namespace environments

The official Kubernetes docs define namespaces as virtual clusters. Therefore, it makes sense to treat each separate environment as a separate namespace. Following the example above, that would mean creating a dev, staging and production namespaces.

By default, Seldon Deploy will treat each namespace as a separate environment. Therefore, this is the suggested approach.

Out of the box, this gives you easy control over:

  • Where is each model deployed (on which environments).
  • Who is authorised to view or create models.
  • Visibility over all your models across every environment from a single instance of Seldon Deploy.

Cluster environments

When you are dealing with multiple environments, it makes sense to keep them completely isolated of each other. That way, any cluster-wide effects will only affect one of them.

Using separate Kubernetes clusters for each of your environments allows you to achieve a higher-level of separation than namespaces. However, this separation also comes with a few caveats:

  • All your infrastructure resources will need to be duplicated and managed across every environment (e.g. auth, ingress, logging, metrics, …).
  • Operations spanning multiple environments become more complicated (e.g. promotion of models between different environments).

This approach is usually recommended to test new versions of cluster-wide components at the infrastructure level. For example, a new version of Seldon Deploy or a new ingress system. However, due to the caveats listed above, we don’t suggest this approach to segregate different versions of your models.

Having said so, you can still leverage the GitOps paradigm to communicate between different clusters.

Last modified September 5, 2020: Update Demos section (91a072d)