Edit This Page

Migrate a ThirdPartyResource to CustomResourceDefinition

This page shows how to migrate data stored in a ThirdPartyResource (TPR) to a CustomResourceDefinition (CRD).

Kubernetes does not automatically migrate existing TPRs. This is due to API changes introduced as part of graduating to beta under a new name and API group. Instead, both TPR and CRD are available and operate independently in Kubernetes 1.7. Users must migrate each TPR one by one to preserve their data before upgrading to Kubernetes 1.8.

The simplest way to migrate is to stop all clients that use a given TPR, then delete the TPR and start from scratch with a CRD. This page describes an optional process that eases the transition by migrating existing TPR data for you on a best-effort basis.

• Before you begin
• Migrate TPR data
• What’s next

Before you begin

• Make sure your Kubernetes cluster has a master version of exactly 1.7.x (any patch release), as this is the only version that supports both TPR and CRD.
• If you use a TPR-based custom controller, check with the author of the controller first. Some or all of these steps may be unnecessary if the custom controller handles the migration for you.
• Be familiar with the concept of custom resources, which were known as third-party resources until Kubernetes 1.7.
• Be familiar with CustomResourceDefinitions, which are a simple way to implement custom resources.
• Before performing a migration on real data, conduct a dry run by going through these steps in a test cluster.

Migrate TPR data

1. Rewrite the TPR definition

   Clients that access the REST API for your custom resource should not need any changes. However, you will need to rewrite your TPR definition as a CRD.

   Make sure you specify values for the CRD fields that match what the server used to fill in for you with TPR.

   For example, if your ThirdPartyResource looks like this:

   apiVersion: extensions/v1beta1
   kind: ThirdPartyResource
   metadata:
     name: cron-tab.stable.example.com
   description: "A specification of a Pod to run on a cron style schedule"
   versions:
   - name: v1
   

   A matching CustomResourceDefinition could look like this:

   apiVersion: apiextensions.k8s.io/v1beta1
   kind: CustomResourceDefinition
   metadata:
     name: crontabs.stable.example.com
   spec:
     scope: Namespaced
     group: stable.example.com
     version: v1
     names:
       kind: CronTab
       plural: crontabs
       singular: crontab
   

2. Install the CustomResourceDefinition

   While the source TPR is still active, install the matching CRD with kubectl create. Existing TPR data remains accessible because TPRs take precedence over CRDs when both try to serve the same resource.

   After you create the CRD, make sure the Established condition goes to True. You can check it with a command like this:

   kubectl get crd -o 'custom-columns=NAME:{.metadata.name},ESTABLISHED:{.status.conditions[?(@.type=="Established")].status}'
   

   The output should look like this:

   NAME                          ESTABLISHED
   crontabs.stable.example.com   True
   

3. Stop all clients that use the TPR

   The API server attempts to prevent TPR data for the resource from changing while it copies objects to the CRD, but it can’t guarantee consistency in all cases, such as with multiple masters. Stopping clients, such as TPR-based custom controllers, helps to avoid inconsistencies in the copied data.

   In addition, clients that watch TPR data do not receive any more events once the migration begins. You must restart them after the migration completes so they start watching CRD data instead.

4. Back up TPR data

   In case the data migration fails, save a copy of existing data for the resource:

   kubectl get crontabs --all-namespaces -o yaml > crontabs.yaml
   

   You should also save a copy of the TPR definition if you don’t have one already:

   kubectl get thirdpartyresource cron-tab.stable.example.com -o yaml --export > tpr.yaml
   

5. Delete the TPR definition

   Normally, when you delete a TPR definition, the API server tries to clean up any objects stored in that resource. Because a matching CRD exists, the server copies objects to the CRD instead of deleting them.

   kubectl delete thirdpartyresource cron-tab.stable.example.com
   

6. Verify the new CRD data

   It can take up to 10 seconds for the TPR controller to notice when you delete the TPR definition and to initiate the migration. The TPR data remains accessible during this time.

   Once the migration completes, the resource begins serving through the CRD. Check that all your objects were correctly copied:

   kubectl get crontabs --all-namespaces -o yaml
   

   If the copy failed, you can quickly revert to the set of objects that existed just before the migration by recreating the TPR definition:

   kubectl create -f tpr.yaml
   

7. Restart clients

   After verifying the CRD data, restart any clients you stopped before the migration, such as custom controllers and other watchers. These clients now access CRD data when they make requests on the same API endpoints that the TPR previously served.

What’s next

• Learn more about custom resources.
• Learn more about using CustomResourceDefinitions.