Features

• 1: Return RemainingItemCount
• 2: Raw SQL Query
• 3: Resource Field Pruning
• 4: Standalone TCP for Health Checker
• 5: Sync All Custom Resources
• 6: Sync All Resources

# ignore other flags
./bin/clustersynchro-manager --feature-gates=SyncAllResources=true

1 - Return RemainingItemCount

We can require the number of remaining resources to be included in the response by search label or url query when querying.

Detailed use can be referred to Response With Remaining Count

You can set the number of remaining resources to be returned by default via Feature Gates – RemainingItemCount, so that the user does not need to use a search label or url query to display the request at each request.

When the remaining item count is returned by default, you can still request that the remaining item count not be returned via search label or url query.

kubectl get --raw="/apis/clusterpedia.io/v1beta1/resources/apis/apps/v1/deployments?withRemainingCount=false&limit=1" | jq

Feature Gate dedicated to clusterpedia apiserver

This feature is turned off by default because it may have an impact on the behavior or performance of the storage layer.

For the default storage tier, returning the number of remaining resources results in an additional COUNT query

2 - Raw SQL Query

Different users may have different needs, and although clusterpedia provides many easy search options, such as specifying a set of namespaces or clusters, or specifying an owner for a query, users may still have more complex queries.

In this case, you can use the Raw SQL Query provided by the default storage layer to pass more complex search conditions.

URL="/apis/clusterpedia.io/v1beta1/resources/apis/apps/v1/deployments"
kubectl get --raw="$URL?whereSQL=(cluster='global') OR (namespace IN ('kube-system','default'))"

In the example, we pass a SQL statement for a WHERE query —— (cluster=‘global’) OR (namespace IN (‘kube-system’,‘default’)),

This statement will retrieve deployments under all namespaces in the global cluster and under the kube-system and default namespaces in other clusters.

The sql statement needs to conform to the SQL syntax of the specific storage component(MySQL, PostgreSQL).

This feature gate is exclusive to the clusterpedia apiserver

Raw SQL queries are currently in alpha and are not well protected against SQL injection, so you need to enable this feature via Feature Gate.

3 - Resource Field Pruning

There are some fields in the resource’s metadata that are usually not very useful in the actual search, so we prune these fields by default when syncing.

We use feature gates to separately control whether thess fields are prunned during resource synchronization, these feature gates are exclusive to the clustersynchro manager component

4 - Standalone TCP for Health Checker

When client-go creates any number of Clients with the same configuration, such as certificates, it reuses the same TCP connection. https://github.com/kubernetes/kubernetes/blob/3f823c0daa002158b12bfb2d53bcfe433516659d/staging/src/k8s.io/client-go/transport/transport.go#L54

This results in the cluster health check interface using the same TCP connection as the resource synchronized informer, which may cause TCP blocking and increased health check latency if a large number of informers are started for the first time. We add a feature gate - HealthCheckerWithStandaloneTCP to allow users to use a standalone tcp for health checks

./clustersynchro-manager --feature-gates=HealthCheckerWithStandaloneTCP=true

Note: When this feature is turned on, the TCP long connections to member clusters will change from 1 to 2. If 1000 clusters are imported, then ClusterSynchro Manager will keep 2000 TCP connections.

5 - Sync All Custom Resources

The custom resources differ from kube’s built-in resources in that kube’s built-in resources do not usually change on a regular basis (there are still two cases where the native resource type can change).

The custom resource types can be created and deleted dynamically.

If you want to automatically adjust the synchronized resource types based on changes in the CRD of the imported cluster, you specify in PediaCluster to synchronize all custom resources.

spec:
  syncAllCustomResources: true

This feature may cause a lot of long connections, so you need to enable Feature Gate in the clustersynchro manager.

6 - Sync All Resources

You can synchronize all types of resources with the All-resource Wildcard, and any resource type change in the imported cluster (e.g. kube version upgrade, group/version disabled, CRD or APIService change) will cause the synchronized resource type to be modified.

spec:
  syncResources:
  - group: "*"
    resources:
    - "*"

Please use this feature with caution, it will create a lot of long connections, in the future Clusterpedia will add the Agent feature to avoid the creation of long connections

It is recommended to specify specific resource types, if you need to dynamically synchronize custom resources, you can use Sync all custom resources.

To use All-resources Wildcard you need to enable Feature Gate in clustersynchro manager.