Docs Developer Site
Kubernetes operator for XPUsage

Using the operator

Contents

This section covers the operator functionality and how to use it, once installed on a Kubernetes cluster.

Custom Resources

The operator introduces a couple of new resources into your Kubernetes cluster.

Xp7Deployment

This is the main custom resource that defines your deployment. It is namespaced, so you have to create a namespace for it. Only 1 Xp7Deployment can reside in each namespace.

apiVersion: enonic.cloud/v1
kind: Xp7Deployment
metadata:
  name: my-deployment
  namespace: my-namespace
spec:
  enabled: true     # Should the pods be running
  xpVersion: 7.13.2  # XP version, must be >=7.13.2

  # List of disks shared by all nodes (see description of disks below)
  nodesSharedDisks:
    - name: blobstore
      size: 1Gi
    - name: snapshots
      size: 1Gi
    - name: export
      size: 1Gi

  # List of nodegroups
  nodeGroups:
      # Name and number of replicas
    - name: main
      replicas: 1

      # Role of the node
      data: true
      master: true

      initContainers:
        # Preinstall Snapshotter app
        - image: enonic/snapshotter:3.1.0
          name: snapshotter-deploy
          mounts:
            - name: deploy
              mountPath: /deploy

      # Environment variables
      env:
        - name: MY_ENV_VAR
          value: hey

      resources:
        # Cpu and memory allocation
        cpu: "1.5"
        memory: 512Mi
        dirs:
          # Snapshotter app copied to deploy folder
          - name: deploy
            size: 100Mi
            mountReadOnly: true
        # List of disks private to each and every node
        disks:
          - name: deploy
            size: 1Gi
          - name: index
            size: 1Gi
          - name: work
            size: 1Gi

Disks

In the Xp7Deployment resource we have names of disks that you can define to be shared, private or non existant.

blobstore

All blob objects. This should always be shared in a cluster.

snapshots

Index snapshots. This should always be shared in a cluster.

export

Data dumps and other data. This should be shared in a cluster.

index

Private node index. This should never be shared.

work

Private node cache. This should never be shared.

deploy

Locally installed apps on this single node. This should never be shared.

Xp7Config

This custom resource manages the configuration of XP nodes. This resource namespaced, and should be created in a namespace that already contains a Xp7Deployment.

apiVersion: enonic.cloud/v1
kind: Xp7Config
metadata:
  name: my-config
  namespace: my-namespace
spec:
  #nodeGroup: all
  #dataBase64: false
  file: my.app.name.cfg
  data: |
    my = custom
    config = file
The nodegroup field is optional and defaults to all nodegroups. If you want to apply you configuration only to a single node group, set the field appropriately.
The dataBase64 field is optional and defaults to false.

You can also create Xp7Config files that hold binary data. To do that you have to base64 encode the data and set the dataBase64 field to true like so:

apiVersion: enonic.cloud/v1
kind: Xp7Config
metadata:
  name: my-config
  namespace: my-namespace
spec:
  #nodeGroup: all
  dataBase64: true
  file: my.app.name.cfg
  data: SGVpISBZb3UgYXJlIG9uZSBub3N5IGZveC4gVGhpcyBpcyB0b3Agc2VjcmV0IGRhdGEuIEdldCBvdXQgb2YgaGVyZS4gU2hvb28uLi4uLi4uLi4uLg==
It can vary how fast XP registers the Xp7Config changes. It can be instant, but it can also take up to a couple of minutes, depending on the Kubernetes cluster setup.

Xp7App

This resource is to manage apps with the operator. While you can manage them with XP, this provides you with the option to create a deployment complete with your custom apps using the operator. This resource namespaced, and should be created in a namespace that already contains a Xp7Deployment.

apiVersion: enonic.cloud/v1
kind: Xp7App
metadata:
  name: contentstudio
  namespace: my-namespace
spec:
  url: https://repo.enonic.com/public/com/enonic/app/contentstudio/5.0.3/contentstudio-5.0.3.jar
  #sha512: ba6b40ebf0808c6fa619ba2a05d07ce3e566eada7e9f3f34c1d280e9d1dcbd1e3c25eff9896d1057c4578ff3f516afa7a905c9e42ddc5e08f1cdf06f7e89774c
The sha512 field is optional, but if provided, XP will validate the sha512 sum of the jar before installing it. This prevents installing of potential malicious apps from the internet.

Added functionality

In addition to new resources, there are also new annotations that add some functionality.

Ingresses

To create virtual hosts for XP you use ingress annotations. These follow the format of:

enonic.cloud/xp7.vhost.mapping.<MAPPING_NAME>.source: /admin
enonic.cloud/xp7.vhost.mapping.<MAPPING_NAME>.target: /admin
enonic.cloud/xp7.vhost.mapping.<MAPPING_NAME>.idproviders: <DEFAULT_IDPROVIDER>,<OTHER_ENABLED_IDPROVIDER>

A very important thing to keep in mind is that the annotation enonic.cloud/xp7.vhost.mapping.<MAPPING_NAME>.source has to match a defined spec.rules[?].http.paths[?].path in the same ingress. That is so the operator knows what node groups it needs to update. That brings us to the second point. The spec.rules[?].http.paths[?].backend.serviceName has to match a node group name defined in your Xp7Deployment.

An example of a valid ingress, assuming you have a nodegroup main, would look something like this.

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: my-domain-com
  namespace: my-namespace
  annotations:
    enonic.cloud/xp7.vhost.mapping.my-mapping-site.source: /
    enonic.cloud/xp7.vhost.mapping.my-mapping-site.target: /site/default/master/homepage

    enonic.cloud/xp7.vhost.mapping.my-mapping-admin.source: /admin
    enonic.cloud/xp7.vhost.mapping.my-mapping-admin.target: /admin
    enonic.cloud/xp7.vhost.mapping.my-mapping-admin.idproviders: system
spec:
  rules:
    - host: my-domain.com
      http:
        paths:
          - path: /
            pathType: ImplementationSpecific
            backend:
              service:
                name: main
                port:
                  number: 8080

          - path: /admin
            pathType: ImplementationSpecific
            backend:
              service:
                name: main
                port:
                  number: 8080
Like Xp7Config, changes to virtual hosts can take a couple of minutes to register in XP.

Namespaces

It can be desireble to delete all created resources that are associated with an Xp7Deployment once its deleted. That is quite easy to do with this namespace annotation:

apiVersion: v1
kind: Namespace
metadata:
  name: my-namespace
  annotations:
    enonic.cloud/remove.with.xp7deployment: my-deployment

Common operations

Get deployment status

$ kubectl -n my-namespace get xp7deployments.enonic.cloud

NAME            ENABLED   VERSION   STATE     MSG
my-deployment   true      7.13.2     RUNNING   OK

Get config status

$ kubectl -n my-namespace get xp7configs.enonic.cloud

NAME                 NODEGROUP   FILE                                 STATE   MSG
all-admin            all         com.enonic.xp.app.main.cfg           READY   OK
all-appstatus        all         com.enonic.app.status.cfg            READY   OK
all-cluster          all         com.enonic.xp.cluster.cfg            READY   OK
all-logback          all         logback.xml                          READY   OK
all-sessionstore     all         com.enonic.xp.web.sessionstore.cfg   READY   OK
all-system           all         system.properties                    READY   OK
main-elasticsearch   main        com.enonic.xp.elasticsearch.cfg      READY   OK
main-vhosts          main        com.enonic.xp.web.vhost.cfg          READY   OK
my-config            all         com.my-app.cfg                       READY   OK

Get app status

$ kubectl -n my-namespace get xp7apps.enonic.cloud

NAME            KEY                            VERSION   STATE     MSG
contentstudio   com.enonic.app.contentstudio   3.2.0     RUNNING   OK

Fetching SU password

$ kubectl -n my-namespace get secret su -o go-template="{{ .data.pass | base64decode }}"

NGDDlGdFYkX8i@#49#Z6N45tfhX6#3Rw

Access admin (bypassing ingress)

$ kubectl -n my-namespace port-forward main-0 8080

Forwarding from 127.0.0.1:8080 -> 8080
Forwarding from [::1]:8080 -> 8080

Then open up localhost:8080 in your browser.

View XP logs

$ kubectl -n my-namespace logs -c exp main-0
                         _____
____________________________(_)______   ____  _________
_  _ \_  __ \  __ \_  __ \_  /_  ___/   __  |/_/__  __ \
/  __/  / / / /_/ /  / / /  / / /__     __>  < __  /_/ /
\___//_/ /_/\____//_/ /_//_/  \___/     /_/|_| _  .___/
                                               /_/

# Enonic XP 7.13.2
# Built on 2021-02-02T15:28:02Z (hash = 632195fda1bf0e9ce4a314d70b403ef731955ad0, branch = e4ea190187bd27bc1143a29c4ff2c80e564f58c0)
# OpenJDK 64-Bit Server VM 11.0.10 (AdoptOpenJDK)
# Linux 4.19.157 (amd64)
# Install directory is /enonic-xp
# Home directory is /enonic-xp/home
....

Edit configuration

$ kubectl -n my-namespace edit xp7configs.enonic.cloud my-config

xp7config.enonic.cloud/my-config edited

Watch configuration

$ kubectl -n my-namespace get xp7configs.enonic.cloud -w

NAME                 NODEGROUP   FILE                                 STATE     MSG
all-admin            all         com.enonic.xp.app.main.cfg           READY     OK
all-appstatus        all         com.enonic.app.status.cfg            READY     OK
all-cluster          all         com.enonic.xp.cluster.cfg            READY     OK
all-logback          all         logback.xml                          READY     OK
all-sessionstore     all         com.enonic.xp.web.sessionstore.cfg   READY     OK
all-system           all         system.properties                    READY     OK
main-elasticsearch   main        com.enonic.xp.elasticsearch.cfg      READY     OK
main-vhosts          main        com.enonic.xp.web.vhost.cfg          READY     OK
my-config            all         com.my-app.cfg                       PENDING   Not loaded

Simple example

Let’s deploy a simple example. Create a file called simple.yaml and paste these contents to it:

simple.yaml 
# Create a namespace
apiVersion: v1
kind: Namespace
metadata:
  name: my-namespace
  annotations:
    # Delete this namespace if the deployment is deleted
    enonic.cloud/remove.with.xp7deployment: my-deployment
---
# Create deployment in the namespace
apiVersion: enonic.cloud/v1
kind: Xp7Deployment
metadata:
  name: my-deployment
  namespace: my-namespace
spec:
  enabled: true
  xpVersion: 7.13.2

  # Create one node
  nodeGroups:
    - name: main
      replicas: 1

      data: true
      master: true

      initContainers:
        # Preinstall Snapshotter app
        - image: enonic/snapshotter:3.1.0
          name: snapshotter-deploy
          mounts:
            - name: deploy
              mountPath: /deploy

      resources:
        cpu: "1"
        memory: 512Mi
        dirs:
          # Snapshotter app copied to deploy folder
          - name: deploy
            size: 100Mi
            mountReadOnly: true
        # Volumes private to the node
        disks:
          - name: blobstore
            size: 1Gi
          - name: snapshots
            size: 1Gi
          - name: export # Dumps and other data
            size: 1Gi
          - name: index # Node ES index
            size: 1Gi
          - name: work # Node cache
            size: 1Gi
---
# Install content studio
apiVersion: enonic.cloud/v1
kind: Xp7App
metadata:
  name: contentstudio
  namespace: my-namespace
spec:
  url: https://repo.enonic.com/public/com/enonic/app/contentstudio/5.0.3/contentstudio-5.0.3.jar
  sha512: ba6b40ebf0808c6fa619ba2a05d07ce3e566eada7e9f3f34c1d280e9d1dcbd1e3c25eff9896d1057c4578ff3f516afa7a905c9e42ddc5e08f1cdf06f7e89774c
---
# Add your own custom config
apiVersion: enonic.cloud/v1
kind: Xp7Config
metadata:
  name: my-config
  namespace: my-namespace
spec:
  nodeGroup: all
  file: com.my-app.cfg
  data: |
    my = config
---
# Expose XP through an ingress
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: my-domain-com
  namespace: my-namespace
  annotations:
    enonic.cloud/xp7.vhost.mapping.my-mapping-site.source: /
    enonic.cloud/xp7.vhost.mapping.my-mapping-site.target: /site/default/master/homepage

    enonic.cloud/xp7.vhost.mapping.my-mapping-admin.source: /admin
    enonic.cloud/xp7.vhost.mapping.my-mapping-admin.target: /admin
    enonic.cloud/xp7.vhost.mapping.my-mapping-admin.idproviders: system
spec:
  rules:
    - host: my-domain.com
      http:
        paths:
          - path: /
            pathType: ImplementationSpecific
            backend:
              service:
                name: main
                port:
                  number: 8080

          - path: /admin
            pathType: ImplementationSpecific
            backend:
              service:
                name: main
                port:
                  number: 8080

Deploy this by running:

$ kubectl apply -f simple.yaml

namespace/my-namespace created
xp7deployment.enonic.cloud/my-deployment created
xp7app.enonic.cloud/contentstudio created
xp7config.enonic.cloud/my-config created
ingress.networking.k8s.io/my-domain-com created

Once the XP pods have started you can open up the admin by following the Access admin (bypassing ingress) section or call the ingress controller, if you have one set up.

Cluster example

The values cpu and memory setting in this example are too low for a good cluster setup. They are set this way, so you can try it out on a low resource Kubernetes cluster.

Let’s deploy a cluster example. Create a file called cluster.yaml and paste these contents to it:

cluster.yaml 
# Create a namespace
apiVersion: v1
kind: Namespace
metadata:
  name: my-namespace
  annotations:
    # Delete this namespace it the deployment is deleted
    enonic.cloud/remove.with.xp7deployment: my-deployment
---
# Create deployment in the namespace
apiVersion: enonic.cloud/v1
kind: Xp7Deployment
metadata:
  name: my-deployment
  namespace: my-namespace
spec:
  enabled: true
  xpVersion: 7.13.2

  # Create volumes shared by all nodes in this deployment
  nodesSharedDisks:
    - name: blobstore
      size: 1Gi

    - name: snapshots
      size: 1Gi

    - name: export # Dumps and other data
      size: 1Gi

  # Create nodes
  nodeGroups:
    # 3 master nodes
    - name: master
      replicas: 3

      data: false
      master: true

      resources:
        cpu: "0.5"
        memory: 1Gi

        # Volumes private to the node
        disks:
          - name: index # Node ES index
            size: 1Gi

    # 2 data nodes
    - name: worker
      replicas: 2

      data: true
      master: false

      initContainers:
        # Preinstall Snapshotter app on worker nodes
        - image: enonic/snapshotter:3.1.0
          name: snapshotter-deploy
          mounts:
            - name: deploy
              mountPath: /deploy

      resources:
        cpu: "1"
        memory: 1Gi

        dirs:
          # Snapshotter app copied to deploy folder
          - name: deploy
            size: 100Mi
            mountReadOnly: true
        # Volumes private to the node
        disks:
          - name: index # Node ES index
            size: 1Gi
---
# Install content studio
apiVersion: enonic.cloud/v1
kind: Xp7App
metadata:
  name: contentstudio
  namespace: my-namespace
spec:
  url: https://repo.enonic.com/public/com/enonic/app/contentstudio/5.0.3/contentstudio-5.0.3.jar
  sha512: ba6b40ebf0808c6fa619ba2a05d07ce3e566eada7e9f3f34c1d280e9d1dcbd1e3c25eff9896d1057c4578ff3f516afa7a905c9e42ddc5e08f1cdf06f7e89774c
---
# Add your own custom config
apiVersion: enonic.cloud/v1
kind: Xp7Config
metadata:
  name: my-config
  namespace: my-namespace
spec:
  nodeGroup: all
  file: com.my-app.cfg
  data: |
    my = config
---
# Expose XP site on frontend nodes through an ingress
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: my-domain-com-site
  namespace: my-namespace
  annotations:
    enonic.cloud/xp7.vhost.mapping.my-mapping-site.source: /
    enonic.cloud/xp7.vhost.mapping.my-mapping-site.target: /site/default/master/homepage
spec:
  rules:
    - host: my-domain.com
      http:
        paths:
          - path: /
            pathType: ImplementationSpecific
            backend:
              service:
                name: worker
                port:
                  number: 8080
---
# Expose XP admin on admin nodes through an ingress
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: my-domain-com-admin
  namespace: my-namespace
  annotations:
    # Enable sticy sessions with nginx
    kubernetes.io/ingress.class: "nginx"
    nginx.ingress.kubernetes.io/affinity: "cookie"
    nginx.ingress.kubernetes.io/session-cookie-name: "stickyXpAdmin"
    nginx.ingress.kubernetes.io/session-cookie-expires: "129600" # 36 hours
    nginx.ingress.kubernetes.io/session-cookie-max-age: "129600" # 36 hours
    nginx.ingress.kubernetes.io/session-cookie-change-on-failure: "true"

    enonic.cloud/xp7.vhost.mapping.my-mapping-admin.source: /admin
    enonic.cloud/xp7.vhost.mapping.my-mapping-admin.target: /admin
    enonic.cloud/xp7.vhost.mapping.my-mapping-admin.idproviders: system
spec:
  rules:
    - host: my-domain.com
      http:
        paths:
          - path: /admin
            pathType: ImplementationSpecific
            backend:
              service:
                name: worker
                port:
                  number: 8080

Deploy this by running:

$ kubectl apply -f cluster.yaml

namespace/my-namespace created
xp7deployment.enonic.cloud/my-deployment created
xp7app.enonic.cloud/contentstudio created
xp7config.enonic.cloud/my-config created
ingress.networking.k8s.io/my-domain-com-site created
ingress.networking.k8s.io/my-domain-com-admin created

Once the XP pods have started you can open up the admin by following the Access admin (bypassing ingress) section or call the ingress controller, if you have one set up.

Contents