PodTransitionRule

In normal pod lifecycle, some phases are defined. For example, K8s Pods follow a defined lifecycle，starting in the Pending phase, moving through Running if at least one of its primary containers starts OK, and then through either the Succeeded or Failed phases depending on whether any container in the Pod terminated in failure.

These phase definitions can fulfill basic Pod change scenarios, but it are ambiguous. Actually, before pod upgrade or ready, it is necessary to have some check mechanisms in place to ensure the safety of pod changes. Fortunately, PodOpsLifecycle extends and supports some check stages: PreCheck before pod upgrade and PostCheck before pod ready.

To ensure a more fine-grained and controlled change process for Pods, we introduce custom rules or perform additional tasks as prerequisites for state transitions before the desired state of a Pod is achieved. Similar to the Pod readinessGates, where certain conditions must be met for a Pod to be considered readiness. For example, we consider a Pod ready for the PostCheck phase only if it has specific labels. For this purpose, we introduce the PodTransitionRule as a prerequisite for the state transition of a Pod.

Rule Definition

You can use PodTransitionRule to define a set of transition rules for your workload pods. Each rule will be executed at the corresponding stage, and it will be blocked if the conditions are not met.

Here is an example:

apiVersion: apps.kusionstack.io/v1alpha1
kind: PodTransitionRule
metadata:
  name: podtransitionrule-sample
spec:
  rules:
  - availablePolicy:
      maxUnavailableValue: 50%
    name: maxUnavailable
  - stage: PreCheck  # stages are supported by PodOpsLifecycle. Defaults to PreCheck.
    labelCheck:
      requires:
        matchLabels:
          app.custom/ready: 'true' 
    name: labelCheck
  - stage: PostCheck
    webhook:
      clientConfig:
        url: https://1.1.1.1:8089/post-stop
        caBundle: Cg==
        poll:
          url: http://1.1.1.1:8089/fetch-result
          rawQueryKey: task-id   # URL parameter key to carry trace ID when fetching result. Defaults to task-id in form 'QueryUrl=URL?rawQueryKey=<task-id>'
          intervalSeconds: 5
          timeoutSeconds: 60
      failurePolicy: Fail
      parameters:
      - key: podIP
        valueFrom:
          fieldRef: 
            fieldPath: status.podIP
    name: webhookCheck
  selector:        # select pods in effect
    matchLabels:
      app: foo

Available Policy

An availablePolicy rule defines the availability strategy during the Pod update process.

maxUnavailable

availablePolicy:
  maxUnavailable: 
    value: 50%  # int or string 

maxUnavailableValue is the maximum number of pods that can be unavailable during the update. Value can be an absolute number (ex: 5) or a percentage of desired pods (ex: 10%). Absolute number is calculated from percentage by rounding down. This can not be 0.

minAvailable

availablePolicy:
  minAvailable:
    value: 5  # int or string 

minAvailableValue is the minimum number of pods that should be available during the update.

Label Check

A labelCheck rule is used to check if labels are satisfied. You can define your own labels as change check conditions and modify the labels according to your needs.

labelCheck:
  requires:
    matchLabels:
      app.custom/ready: 'true' 
    matchExpressions:
    - key: app.custom/forbidden 
      operator: DoesNotExist

Webhook

A webhook is an HTTP callback, based on which a external web application can determine whether a pod can pass this check.

• An HTTP POST occurs first when pods entries the configured stage which defaults PreCheck.
• If poll is provided, this rule then keeps calling polling url to fetch a long running job result. This job can be located by task-id returned from the response of the first request.

webhook:
  clientConfig: # custom server config
    url: https://1.1.1.1:8089/post-stop
    caBundle: Cg==
    poll:
      url: http://1.1.1.1:8089/fetch-result
      rawQueryKey: task-id
      intervalSeconds: 5
      timeoutSeconds: 60
  failurePolicy: Fail
  parameters:
    - key: podIP
      valueFrom:
        fieldRef:
          fieldPath: status.podIP

Protocol without poll

Request:

// URL: https://1.1.1.1:8089/post-stop
// Method: POST

{ 
  "traceId": "<trace-id>",    // <trace-id> is generated by Operating, which can be used to track request
  "stage": "PreTrafficOff",
  "ruleName": "webhookCheck",
    "resources": [            // Information of Pods which are in this stage
    {
      "apiVersion": "v1",
      "kind": "Pod",
      "name": "pod-a",
      "parameters": {
        "podIP": "1.0.0.1"    // Customized information users can indicate from rule paramter
      }
    },
    {
      "apiVersion": "v1",
      "kind": "Pod",
      "name": "pod-b",
      "parameters": {
        "podIP": "1.0.0.2"
      }
    }
  ]
}

Response:

{
  "success": false,
  "message": "msg", 
  "finishedNames": ["pod-a", "pod-b"]
}

Response success indicating all pods approved or not. If it's false, the finishedNames field can be used to approve partial pods.

Protocol with poll

Request:

// URL: https://1.1.1.1:8089/post-stop
// Method: POST

{ 
  "traceId": "<trace-id>",    // <trace-id> is generated by Operating, which can be used to track request
  "stage": "PreTrafficOff",
  "ruleName": "webhookCheck",
    "resources": [            // Information of Pods which are in this stage
    {
      "apiVersion": "v1",
      "kind": "Pod",
      "name": "pod-a",
      "parameters": {
        "podIP": "1.0.0.1"    // Customized information users can indicate from rule paramter
      }
    },
    {
      "apiVersion": "v1",
      "kind": "Pod",
      "name": "pod-b",
      "parameters": {
        "podIP": "1.0.0.2"
      }
    }
  ]
}

Response:

{
  "success": true,
  "poll": true,         // required to indicate polling calls is necessary
  "taskId": <task-id>,  // required to to fetch polling result
  "message": "msg"
}

Response success indicating whether the first request is success or not. If true and field poll in response is true (or field async in response is true), PodTransisionRule will then begin to keep calling poll URL to fetch process result. Field taskId is required for polling.

The request for polling is GET method and in form of QueryUrl=URL?task-id=<task-id>. The parameter key in this URL defaults task-id, if using poll in above response. It would be trace-id if using async in above response. Users can also indicate the key by field poll.rawQueryKey.

The response from polling call is expected like following:

{
  "success": true,
  "message": "msg",
  "finished": false,
  "finishedNames": ["pod-a", "pod-b"]
}

success is supposed to be true, if there is no error. If all pods is approved, finished should be true. If finished is false, finishedNames can be used to allow partial pods to be approved.