Synchronize Cluster Resources

  4 minute read  

The main function of Clusterpedia is to provide complex search for resources in multiple clusters.

Clusterpedia uses the PediaCluster resource to specify which resources in the cluster need to support complex search, and synchronizes these resources onto the Storage Component via Storage Layer in real time.

# example
apiVersion: cluster.clusterpedia.io/v1alpha2
kind: PediaCluster
metadata:
  name: cluster-example
spec:
  apiserver: "https://10.30.43.43:6443"
  syncResources:
  - group: apps
    resources:
     - deployments
  - group: ""
    resources:
     - pods
     - configmaps
  - group: cert-manager.io
    versions:
      - v1
    resources:
      - certificates

Synchronize built-in resources

In order to manage and view these synchronized resources through PediaCluster, you need to configure resources in groups

syncResources:
 - group: apps
   versions: []
   resources:
     - deployments
     - daemonsets

For built-in resources, versions is not required.

Clusterpedia will automatically select the appropriate version to synchronize based on the resource version supported in the cluster.

Also, you do not need to worry about version conversion because Clusterpedia will open all version interfaces for built-in resources.

kubectl get --raw="/apis/clusterpedia.io/v1beta1/resources/apis/apps" | jq

{
  "kind": "APIGroup",
  "apiVersion": "v1",
  "name": "apps",
  "versions": [
    {
      "groupVersion": "apps/v1",
      "version": "v1"
    },
    {
      "groupVersion": "apps/v1beta2",
      "version": "v1beta2"
    },
    {
      "groupVersion": "apps/v1beta1",
      "version": "v1beta1"
    }
  ],
  "preferredVersion": {
    "groupVersion": "apps/v1",
    "version": "v1"
  }
}

Clusterpedia supports three versions of Deployment: v1, v1beta2, and v1beta1.

Synchronize custom resources

Compared with built-in resources, custom resources have slightly different configuration on resource versions.

syncResources:
 - group: cert-manager.io
   versions: []
   resources:
    - certificates

You can also ignore the versions field and then Clusterpedia will synchronize the previous three cluster versions in the Group.

Take cert-manager.io as an example to get the Group supported by cert-manager.io in an imported cluster

# Run the command in an imported cluster
kubectl get --raw="/apis/cert-manager.io" | jq

{
  "kind": "APIGroup",
  "apiVersion": "v1",
  "name": "cert-manager.io",
  "versions": [
    {
      "groupVersion": "cert-manager.io/v1",
      "version": "v1"
    },
    {
      "groupVersion": "cert-manager.io/v1beta1",
      "version": "v1beta1"
    },
    {
      "groupVersion": "cert-manager.io/v1alpha3",
      "version": "v1alpha3"
    },
    {
      "groupVersion": "cert-manager.io/v1alpha2",
      "version": "v1alpha2"
    }
  ],
  "preferredVersion": {
    "groupVersion": "cert-manager.io/v1",
    "version": "v1"
  }
}

The imported cluster supports four versions for cert-manager.io: v1, v1beta1, v1alpha3, v1alpha2.

When syncResources.[group].versions is left blank, Clusterpedia will synchronize three versions v1, v1beta1, v1alpah3 in the order of the APIGroup.versions list except for v1alpha2.

Specify a sync version for custom resources

If you specified versions, the specific resource would be synchronized by versions.

syncResources:
 - group: cert-manager.io
   versions:
    - v1beta1
   resources:
    - certificates

The above snippet only synchronizes v1beta1.

Usage notes

The custom resource synchronization does not support version conversion currently. The versions are fixed after synchronization.

If cluster-1 only synchronizes v1beta1 resources when you are searching for multi-cluster resources, the request to search for version v1 will ignore the version v1beta1.

You are required to learn and handle the different versions in multiple clusters for custom resources.

Sync all custom resources

The custom resource types and versions change with the CRD, so when a CRD is created and we don’t want to modify spec.syncResources to sync resources as the same time, we can set spec.syncAllCustomResources to sync all custom resources.

spec:
  syncAllCustomResources: true

However, it should be noted that to use this feature, you need to enabled the corresponding Feature Gate in clustersynchro-manager, which can be found in Sync All Custom Resource

Using wildcards to sync resources

Group Wildcard

spec:
  syncResources:
  - group: "apps"
    resources:
    - "*"

Use Group Wildcard to sync all types of resources under the specified group.

In the above example, all resources under apps will be synced.

All-resources Wildcard

spec:
  syncResources:
  - group: "*"
    resources:
    - "*"

The All-resources Wildcard allows we to sync built-in resources, custom resources and aggregated API resources in the imported cluster.

This feature creates a large number of long connections, so use it with caution and enable the corresponding Feature Gate in the clustersynchro-manager, as described in Sync All Resources

Reference ClusterSyncResources

ClusterSyncResources is used to define cluster resource synchronization configurations that are commonly referenced by multiple PediaClusters, see Public Configuration of Cluster Sync Resources for more information about ClusterSyncResources

PediaCluster sets the referenced ClusterSyncResources by spec.syncResourceRefName.

apiVersion: cluster.clusterpedia.io/v1alpha2
kind: ClusterSyncResources
metadata:
  name: global-base
spec:
  syncResources:
    - group: ""
      resources:
        - pods
    - group: "apps"
      resources:
        - "*"
---
apiVersion: cluster.clusterpedia.io/v1alpha2
kind: PediaCluster
metadata:
  name: demo1
spec:
  syncResourcesRefName: "global-base"
  syncResources:
    - group: ""
      resources:
        - pods
        - configmaps

If PediaCluster has both spec.syncResourcesRefName and spec.syncResources set, then the concatenation of the two will be used.

In the above example, clusterpedia synchronizes the pods and configmaps resources, and all resources under the apps group in the demo1 cluster.

View synchronized resources

You can view resources, sync versions, and storage versions by using Status of the PediaCluster resource.

For Status, a resource may have Sync Version and Storage Version:

  • Sync Version refers to the resource version from a synchronized cluster by Clusterpedia
  • Storage Version refers to the version stored at the storage layer by Clusterpedia
status:
  syncResources:
  - group: apps
    resources:
    - name: deployments
      kind: Deployment
      namespaced: true
      syncConditions:
      - lastTransitionTime: "2022-01-13T04:34:08Z"
        status: Syncing
        storageVersion: v1
        version: v1

In general, Sync Version is same as Storage Version for a cluster resource.

However, if an imported cluster only provides the Deployment resource of the v1beta1 version, the Sync Version is v1beta1 and the Storage Version is v1.

For example, when synchronizing a Deployment of Kubernetes 1.10, the synchronization status is as follows:

status:
  syncResources:
  - group: apps
    resources:
    - name: deployments
      kind: Deployment
      namespaced: true
      syncConditions:
      - lastTransitionTime: "2022-01-13T04:34:04Z"
        status: Syncing
        storageVersion: v1
        version: v1beta1

For a custom resource, Synchronized Version is same as Storage Version

Next

After resource synchronization, you can Access the Clusterpedia to Search for Resources