IBM Cloud Docs
Why can't I assign hosts to a cluster?

Why can't I assign hosts to a cluster?

You try to assign a host to IBM Cloud Satellite resource such as a cluster, but the assignment does not succeed.

When you check your host, the health state might be unresponsive, unknown, or reload-required.

Your host might have encountered an issue during the bootstrapping process. For example, the underlying infrastructure of the host machine changed and no longer meets the minimum requirements, such as for network connectivity.

You might have set up a firewall or other change that prevents access to a dependency.

In particular, the bootstrapping process depends upon the following access.

  • Access to RHEL Satellite servers and the required packages installed on the host machine.
  • Access to IBM Cloud Container Registry endpoints to pull down required images.
  • Access to the Kubernetes master of the Satellite cluster that you want to assign the host to. Access might be blocked because the host cannot communicate with the service endpoint of the cluster, or because a Kubernetes resource within the cluster such as a webhook intercepts and blocks communication with the Kubernetes API server.

Debugging hosts for connectivity issues

If you want, you can debug the connectivity issues for your host.

Otherwise, remove the host, reload the operating system, and attach the host back.

  1. Get the location ID where your host is attached, and note the IBM Cloud multizone metro that the location is managed from. From the console, click your location, and then click the Overview tab. From the CLI, run the following command.
    ibmcloud sat location ls
    
  2. Confirm that your host meets the minimum requirements and verify that the hostname contains only lowercase alphanumeric characters, -, or ..
  3. Check your host for connectivity issues.
    1. Log in to your host machine, such as via SSH.
    2. Check your host network settings to ensure that your host can access the required ports and IP addresses, which might be blocked by a security group or firewall.
    3. Check access to the required IBM Cloud multizone metro endpoints.
    4. For hosts that are assigned to clusters, get the details of the cluster master endpoint.
      ibmcloud ks cluster get -c <cluster_name_or_ID> | grep "Master URL"
      
    5. Check connectivity to the cluster master. If the curl request fails, your host might not have access to the endpoint, such as blocked by a security group, firewall, or private network.
      curl -k <master_URL>
      
    6. If you think you might have a webhook in the cluster that block access to the API server, see Cluster cannot update because of broken webhook. Webhooks are often components for additional capabilities in your cluster, such as Cloud Paks, Istio, or container image security enforcement.
  4. After you resolve any connectivity issues, check the health of your host for further information.
  5. Reassign your hosts if you continue to have issues.
    1. Remove the host from your Satellite location.
    2. Reload the operating system of your host by following the procedure of the underlying infrastructure provider.
    3. Verify that you reloaded the host machine by logging in to the machine and checking for the following file.
      file /etc/satelittemachineidgeneration/machineidgenerated
      
      If the file does not exist, you see a message similar to the following. Your host was reloaded and you can continue to the next step.
      /etc/satelittemachineidgeneration/machineidgenerated: cannot open (No such file or directory)
      
      If the file exists, you see a message similar to the following. You must reload your host machine operating system before continuing to the next step.
      /etc/satelittemachineidgeneration/machineidgenerated: empty
      
    4. Confirm that your host meets the minimum requirements.
    5. Attach the host back to your Satellite location.
    6. Check that the host is attached to your location and unassigned. From the console, click your location, and then click the Hosts tab. From the CLI, run the following command.
      ibmcloud sat host ls --location <location_name_or_ID>
      
    7. Assign the host to your Satellite resource, such as a cluster.
    8. Check that the host is assigned to your cluster. The process might take an hour to complete. From the console, click your location, and then click the Hosts tab. From the CLI, run the following command.
      ibmcloud sat host ls --location <location_name_or_ID>
      

Endpoints to verify connectivity by IBM Cloud multizone metro

Review the following table to help troubleshoot network connectivity issues to IBM Cloud endpoints that are required for the bootstrapping process of a Satellite host.

Endpoints to test when your location is managed from Dallas.
Endpoint Command to check endpoint
Public regional endpoint nslookup origin.us-south.containers.cloud.ibm.com
Public regional bootstrap endpoint curl -v https://origin.us-south.containers.cloud.ibm.com/bootstrap/firstboot
Private regional bootstrap endpoint curl -v https://private.us-south.containers.cloud.ibm.com/bootstrap/firstboot
IBM Cloud Container Registry region curl -v https://us.icr.io
Endpoints to test when your location is managed from Frankfurt.
Endpoint Command to check endpoint
Public regional endpoint nslookup origin.eu-de.containers.cloud.ibm.com
Public regional bootstrap endpoint curl -v https://origin.eu-de.containers.cloud.ibm.com/bootstrap/firstboot
Private regional bootstrap endpoint curl -v https://private.eu-de.containers.cloud.ibm.com/bootstrap/firstboot
IBM Cloud Container Registry region curl -v https://de.icr.io
Endpoints to test when your location is managed from Sao Paulo.
Endpoint Command to check endpoint
Public regional endpoint nslookup origin.br-sao.containers.cloud.ibm.com
Public regional bootstrap endpoint curl -v https://origin.br-sao.containers.cloud.ibm.com/bootstrap/firstboot
Private regional bootstrap endpoint curl -v https://private.br-sao.containers.cloud.ibm.com/bootstrap/firstboot
IBM Cloud Container Registry region curl -v https://br-sao.icr.io
Endpoints to test when your location is managed from London.
Endpoint Command to check endpoint
Public regional endpoint nslookup origin.eu-gb.containers.cloud.ibm.com
Public regional bootstrap endpoint curl -v https://origin.eu-gb.containers.cloud.ibm.com/bootstrap/firstboot
Private regional bootstrap endpoint curl -v https://private.eu-gb.containers.cloud.ibm.com/bootstrap/firstboot
IBM Cloud Container Registry region curl -v https://uk.icr.io
Endpoints to test when your location is managed from Tokyo.
Endpoint Command to check endpoint
Public regional endpoint nslookup origin.jp-tok.containers.cloud.ibm.com
Public regional bootstrap endpoint curl -v https://origin.jp-tok.containers.cloud.ibm.com/bootstrap/firstboot
Private regional bootstrap endpoint curl -v https://private.jp-tok.containers.cloud.ibm.com/bootstrap/firstboot
IBM Cloud Container Registry region curl -v https://jp.icr.io
Endpoints to test when your location is managed from Toronto.
Endpoint Command to check endpoint
Public regional endpoint nslookup origin.ca-tor.containers.cloud.ibm.com
Public regional bootstrap endpoint curl -v https://origin.ca-tor.containers.cloud.ibm.com/bootstrap/firstboot
Private regional bootstrap endpoint curl -v https://private.ca-tor.containers.cloud.ibm.com/bootstrap/firstboot
IBM Cloud Container Registry region curl -v https://jp.icr.io
Endpoints to test when your location is managed from Washington DC.
Endpoint Command to check endpoint
Public regional endpoint nslookup origin.us-east.containers.cloud.ibm.com
Public regional bootstrap endpoint curl -v https://origin.us-east.containers.cloud.ibm.com/bootstrap/firstboot
Private regional bootstrap endpoint curl -v https://private.us-east.containers.cloud.ibm.com/bootstrap/firstboot
IBM Cloud Container Registry region curl -v https://us.icr.io