カテゴリー
note

What is Multi-Tenancy

just a note

What is tenant?

  • A person who occupies land or property rented from a landlord.
  • A consumer who consumers services (as SaaS, IaaS or *aaS) offered by Cloud service Providers.

Multi-Tenancy

  • Have multiple tenants/occupants.
  • Software that is shared by several consumers.
  • Each consumers is unaware of the others.

Approaches

  • Infra Level Tenancy
  • Schema Level Tenancy/Shared Services (API)
    • DB sharding
    • Approaches Tenant, Service, Geo, etc
    • Tenant_id, Tenant_name, User_login, Location

Kubernetes Multi-tenancy

Providing isolation and fair resource sharing between multiple users and their workloads within a cluster.

It’s important that it is in a single cluster.

Trust

  • Trust, Semi-Trust, Non-trusted

Kubernetes Engine Multi-Tenancy Primitives

  • Cluster vs Namespace boundary

Cluster per Tenant

  • Pros
    • Separate control plane (API) for each tenant (for free)
    • Strong network isolation (if it;s per-cluster VPC)

Namespace per tenant(intra-cluster multi-tenancy)

Namespaces provide logical isolation between tenants on a cluster.

Kubernetes policies are namespace-scoped.

  • Logical isolation between tenants
  • Policies for API access restrictions & resource usage constraints
  • Pros
    • Tenants can reuse extensions/controllers/CRDs
    • Shared control plane(=shared ops, shared security/auditing)

Kubernetes engine primitives

  • Access Control
    • Admission Control
    • RBAC (Role-based access control)
    • IAM (Identity and Access Management)
  • Resource Sharing
    • Quotas
    • Pod Priority
    • Pod Affinity/Anti-Affinity
    • Limit Range
  • Runtime Isolation
    • Sandbox Pods
    • Pod Security Policy
    • Network Policy

Use cases

  • Enterprise
    • All user s from the same company/organization
    • Namespaces=Tenants=Teams
    • Semi-trusted tenants
    • Cluster Roles
      • Cluster Admin
        • CRUD any policy objects
        • Create/assign namespaces to Namespace Admin
        • Manage policies (resource usage quotas, networking)
      • Namespace Admin
        • Mange users in the namespace they own
      • User
        • CRUD non-policy objects in the namespace(s) they have access to
    • Many apps from different teams, semi-trusted
      • Vanilla container isolation may suffice
    • Network isolation
      • allow all traffic within a namespace
      • Whitelist traffic from/to other namespaces(=teams)
  • Software as a Service
    • Consumer deploys their app through a custom control plane
    • SaaS API is a trusted client of Kubernetes
    • Cluster admins can access the Kubernetes API directly.
    • Tenant workloads may have untrusted pieces:
      • such as WordPress extensions
      • may require sandboxing with gVisor etc
  • Kubernetes as a Service
    • Untrusted tenants running untrusted code.
      • Platform as a Service or hosting companies
    • tenants may create their namespaces, but canot set policy objects.
    • Stronger isolation requirements than enterprise/SaaS.

Kubernetes engine multi-tenancy primitives

  • Auth related
    • RBAC (Role-based access control)
      • which users/groups/Service Accounts
      • can do which operations
      • on which API resources
      • in which namespaces
      • mostly useful for
        • giving access to pods calling Kubernetes API (with Kubernetes Service Accounts)
        • giving fine-grained access to people/groups calling Kubernetes API (with Google accounts)
    • IAM (Identity and Access Management)
      • practical for giving Google users/groups project-wide access
    • Admission Control
      • Intercept API request before resource is persisted
      • can mutate and allow/deny
      • compiled into Kubernetes apiserver binary
      • Extending Admission Controls
        • can develop webhooks to create your own Admission Controllers
    • Pod Security Policy
      • Restricts access to host{filesystem, network, ports, PID namespace, IFS namespace}…
      • Limits privileged containers, volume types, enforces read-only filesystem etc
      • Enforced through its own admission plugin
    • Network Policy
      • Which pods can talk to which other pods(based on their namespace/labels) or IP ranges.
      • Available on Kubernetes Engine with Calico Network Plugin
  • Scheduling related
    • Pod Priority
      • Puts high priority pods waiting in pending state in front of the scheduling queue
    • Pod Preemption
      • Evicts lower priority pods from a Node, if high priority pod cannot be scheduled due to not enough space/resources in the cluster
    • Resource Quotas
      • Limits total memory/cpu/storage that pods can use, and how many objects of each type(pods, load balancers, ConfigMaps, etc.) on a per-namespace basis
    • Pod Affinity/Anti-Affinity
      • constrain scheduling of pods, baszed on the labels of other pods scheduled on the node
    • Limit Range
    • Sandbox Pods
      • Linux kernel bugs and security vulnerabilities may bypass container security boundaries
        • Kata Containers
        • gVisor(google approach)
    • Pod Security Context

Kubernetes Multi-tenancy Limitations Today

  • in 2018
  • Kubernetes API
    • Currently API calls are not rate limited, open to DoS from tenants, impacting others.
  • Networking:
    • Networking is not a scheduled resource in Kubernetes, yet(cannot use with limits/requests)
    • Tenants can still discover each other via Kubernetes DNS

Key Takeaways

  • Determine your use case
    • How trusted are your tenant users and workloads?
    • What degree and kinds of isolation do you need?
  • Namespace-centric multi-tenancy
    • Utilize Policy objects for scheduling and access control
    • Think about personas and map them to RBAC cluster roles
    • Automate policies across clusters with GKE Policy Management(alpha)

コメントを残す

メールアドレスが公開されることはありません。 * が付いている欄は必須項目です