MrPointy's journey designing, building and deploying private clouds

pyVmomi - Adding Hosts to Clusters

2015-10-17

This post is part 3 of a small series of articles about how to use pyVMomi to build, configure and manage vmware vsphere environments.

For more information on the earlier posts please refer to Part 1 or Part 2

Updating our YAML file for Clusters with member Hosts

You will recall that the strategy for our sample program is to encapsulate the configuration we want to achieve in a simple YAML file.

In this section we will extend that YAML file structure such that Clusters with member Hosts can be represented.

YAML configuration file

We will now extend our previous YAMl definition by adding member ESXi hosts to the Cluster.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
cat labconfig.yaml
lab:
vsphere:
host:
-
ip: 192.168.10.101
hostname: "beast1.example.com"
user: root
pw: vmware123
-
ip: 192.168.10.102
hostname: "beast2.example.com"
user: root
pw: vmware123
vcenter:
ip: 192.168.10.11
user: administrator@vsphere.local
pw: vmware
topology:
- dc:
name: LABDC
clusters:
- name: LABC
members:
- ip: 192.168.10.101
- ip: 192.168.10.102

As you can see, a simple change has been made to add a members and ip section to the YAML file.

Additionally, a section has been added to contain host details. This section is cross-referenced with the member ip’s so that we can access the required userid and password details for the new hosts. The structure exists this way as it’s simpler to change the topology without having to shuffle around all the host details and keeps that information in one place.

Now, let’s look at the code side of things. The YAML parser will take care of capturing our configuration from the labconfig.yaml file. We need to use that information to make the correct calls to the vSphere SDK via pyVmomi.

A quick browse shows us that we have documentation on Clusters and that there is a method on that class called AddHost. AddHost in turn utilises a vim.host.ConnectSpec to define the characteristics of the host we’re adding. The important thing to note here is that the method returns a vim.Task

Managing Tasks

vSphere allows us to submit multiple long running tasks. Each task is represented by a vim.Task that can be queried to determine the status, success or otherwise of the requested action.

To manage this a simple helper method is added to out Vcenter class to allow us to track a task till completion. In our application we don’t run tasks in parallel so it’s a simple case of waiting for a launched task to either succeed or fail.

1
2
3
4
5
6
7
8
9
10
11
12
13
def wait_for_task(self, task):
while task.info.state == (self.pyVmomi.vim.TaskInfo.State.running or self.pyVmomi.vim.TaskInfo.State.queued):
time.sleep(2)
if task.info.state == self.pyVmomi.vim.TaskInfo.State.success:
if task.info.result is not None:
out = 'Task completed successfully, result: %s' % (task.info.result,)
print out
elif task.info.state == self.pyVmomi.vim.TaskInfo.State.error:
out = 'Error - Task did not complete successfully: %s' % (task.info.error,)
raise ValueError(out)
return task.info.result

As you can see we basically spin in a sleep(2) loop until the task succeeds or fails.

Adding a Host

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
def add_host(self, cluster_name, hostname, username, password):
host = self.get_obj([self.pyVmomi.vim.HostSystem], hostname)
if host is not None:
print("host already exists")
return host
else:
if hostname is None:
raise ValueError("Missing value for name.")
cluster = self.get_obj([self.pyVmomi.vim.ClusterComputeResource], cluster_name)
if cluster is None:
error = 'Error - Cluster %s not found. Unable to add host %s' % (cluster_name, hostname)
raise ValueError(error)
try:
hostspec = self.pyVmomi.vim.host.ConnectSpec(hostName=hostname,userName=username, password=password, force=True)
task=cluster.AddHost(spec=hostspec,asConnected=True)
except self.pyVmomi.vmodl.MethodFault as error:
print "Caught vmodl fault : " + error.msg
return -1
self.wait_for_task(task, self.service_instance)
host = self.get_obj([self.pyVmomi.vim.HostSystem], hostname)
return host

Firstly, our add_host() method checks to see if there is already a host object with the name we’re trying to add. If it does exist we simply return with the pointer to that Host object. In theory we could move the host to the cluster selected but that is left as an exercise to the reader. ie. The use case is that Hosts have already been added to the vCenter but the task is to move them into an existing Cluster definition.

If the host doesn’t exist we get a pointer the the desired cluster and add the Host to the cluster using a HostSpec that contains the credentials for the Host.

Trying it out

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
$ ./buildLabConfig.py
Connecting to 192.168.10.11 using username administrator@vsphere.local
/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:769: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html
InsecureRequestWarning)
Connected to 192.168.10.11, VMware vCenter Server 5.5.0 build-2183111
Creating Datacenter LABDC
Creating Cluster LABC
Traceback (most recent call last):
File "./buildLabConfig.py", line 159, in module
main()
File "./buildLabConfig.py", line 152, in main
vc.add_host(j['name'],iptoadd,user,pw)
File "./buildLabConfig.py", line 78, in add_host
self.wait_for_task(task)
File "./buildLabConfig.py", line 105, in wait_for_task
raise ValueError(out)
ValueError: Error - Task did not complete successfully: (vim.fault.SSLVerifyFault) {
dynamicType = unset,
dynamicProperty = (vmodl.DynamicProperty) [],
msg = "Authenticity of the host's SSL certificate is not verified.",
faultCause = unset,
faultMessage = (vmodl.LocalizableMessage) [],
selfSigned = false,
thumbprint = 'E2:B0:AE:1A:EE:D1:DA:EE:16:99:FA:9D:D9:90:F3:3A:D1:28:D6:17'
}

Hmm, what happened? Our AddHost task has failed with a SSLVerifyFault.

The documentation shows us that for the vim.host.ConnectSpec that we pass through to our AddHost() method has an optional attribute called sslThumprint. This attribute is only ‘optional’ if you have certificates that have been signed by a valid CA. In this case we’re using self-signed certificates so we need to supply the sslThumbprint.

How do we find the sslThumbprint?

A quick google leads us to this article which gives us two options.

  1. Execute an openssl command on the ESXi host
  2. Remotely connect to port 443 on the ESXi host using openssl and extract the thumbprint.

Option 2 is a relatively simple option as it doesn’t require credentials for the host

From a bash perspective the following command can extract the sslThumbprint.

1
echo -n | openssl s_client -connect 192.168.10.101:443 2>/dev/null | openssl x509 -noout -fingerprint -sha1

To perform this we use python subprocesses and pipes to reflect the above command in a python construct.

Here is the complete program :

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
#!/usr/bin/python
"""
Using the pyVmomi python bindings for vSphere connect to a VCSA and configure Datacenters and Clusters as per th spefified configuration file.
"""
import atexit
import yaml
import inspect
import time
import subprocess
class Vcenter(object):
def __init__(self, vcenter_params):
self.pyVmomi = __import__("pyVmomi")
self.server = vcenter_params['ip']
self.username = vcenter_params['user']
self.password = vcenter_params['pw']
self.connect_to_vcenter()
def create_datacenter(self, dcname=None, folder=None):
datacenter = self.get_obj([self.pyVmomi.vim.Datacenter], dcname)
if datacenter is not None:
print("datacenter %s already exists" % dcname)
return datacenter
else:
if len(dcname) > 79:
raise ValueError("The name of the datacenter must be under 80 characters.")
if folder is None:
folder = self.service_instance.content.rootFolder
if folder is not None and isinstance(folder, self.pyVmomi.vim.Folder):
print("Creating Datacenter %s " % dcname )
dc_moref = folder.CreateDatacenter(name=dcname)
return dc_moref
def create_cluster(self, cluster_name, datacenter):
cluster = self.get_obj([self.pyVmomi.vim.ClusterComputeResource], cluster_name)
if cluster is not None:
print("cluster already exists")
return cluster
else:
if cluster_name is None:
raise ValueError("Missing value for name.")
if datacenter is None:
raise ValueError("Missing value for datacenter.")
print("Creating Cluster %s " % cluster_name )
cluster_spec = self.pyVmomi.vim.cluster.ConfigSpecEx()
host_folder = datacenter.hostFolder
cluster = host_folder.CreateClusterEx(name=cluster_name, spec=cluster_spec)
return cluster
def add_host(self, cluster_name, hostname, sslthumbprint, username, password):
host = self.get_obj([self.pyVmomi.vim.HostSystem], hostname)
if host is not None:
print("host already exists")
return host
else:
if hostname is None:
raise ValueError("Missing value for name.")
cluster = self.get_obj([self.pyVmomi.vim.ClusterComputeResource], cluster_name)
if cluster is None:
error = 'Error - Cluster %s not found. Unable to add host %s' % (cluster_name, hostname)
raise ValueError(error)
try:
hostspec = self.pyVmomi.vim.host.ConnectSpec(hostName=hostname,userName=username, sslThumbprint=sslthumbprint, password=password, force=True)
task=cluster.AddHost(spec=hostspec,asConnected=True)
except self.pyVmomi.vmodl.MethodFault as error:
print "Caught vmodl fault : " + error.msg
return -1
self.wait_for_task(task)
host = self.get_obj([self.pyVmomi.vim.HostSystem], hostname)
return host
def get_obj(self, vimtype, name):
"""
Get the vsphere object associated with a given text name
"""
obj = None
container = self.content.viewManager.CreateContainerView(self.content.rootFolder, vimtype, True)
for c in container.view:
if c.name == name:
obj = c
break
return obj
def wait_for_task(self, task):
while task.info.state == (self.pyVmomi.vim.TaskInfo.State.running or self.pyVmomi.vim.TaskInfo.State.queued):
time.sleep(2)
if task.info.state == self.pyVmomi.vim.TaskInfo.State.success:
if task.info.result is not None:
out = 'Task completed successfully, result: %s' % (task.info.result,)
print out
elif task.info.state == self.pyVmomi.vim.TaskInfo.State.error:
out = 'Error - Task did not complete successfully: %s' % (task.info.error,)
raise ValueError(out)
return task.info.result
def connect_to_vcenter(self):
from pyVim import connect
print("Connecting to %s using username %s" % (self.server, self.username))
self.service_instance = connect.SmartConnect(host=self.server,
user=self.username,
pwd=self.password,
port=443)
self.content = self.service_instance.RetrieveContent()
about = self.service_instance.content.about
print("Connected to %s, %s" % (self.server, about.fullName))
atexit.register(connect.Disconnect, self.service_instance)
def getsslThumbprint(self,ip):
p1 = subprocess.Popen(('echo', '-n'), stdout=subprocess.PIPE, stderr=subprocess.PIPE)
p2 = subprocess.Popen(('openssl', 's_client', '-connect', '{0}:443'.format(ip)), stdin=p1.stdout, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
p3 = subprocess.Popen(('openssl', 'x509', '-noout', '-fingerprint', '-sha1'), stdin=p2.stdout, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
out = p3.stdout.read()
ssl_thumbprint = out.split('=')[-1].strip()
return ssl_thumbprint
def main():
config = yaml.load(open("labconfig.yaml"))
vc=Vcenter(config["lab"]["vsphere"]["vcenter"]);
dc=config["lab"]["vsphere"]["topology"]
hostlist = config['lab']['vsphere']['host']
# for each datacenter, get the name and optionally if there is a key for clusters then create matching clusters
# in this datacenter, otherwise just create the datacenter.
for i in dc:
datacenter = vc.create_datacenter(dcname=i['dc']['name'])
if i['dc'].has_key('clusters') :
for j in i['dc']['clusters']:
cluster=vc.create_cluster(j['name'],datacenter)
if j.has_key('members') :
for k in j['members']:
iptoadd=None
user=None
pw=None
for l in hostlist: # lookup the member details in the hosts section in the YAML to find the
if l['ip']==k['ip']: #
iptoadd=l['ip'] #
user=l['user'] # user and pasword details
pw=l['pw']
if iptoadd == None:
print("Couldnt find credentials for ip %s" % k['ip'])
else:
sslthumbprint=vc.getsslThumbprint(iptoadd)
vc.add_host(j['name'],iptoadd,sslthumbprint,user,pw)
return 0
# Start program
if __name__ == "__main__":
main()

Here is what the VCSA looked like before the job executed.

before
1
2
3
4
5
6
7
8
9
./buildLabConfig.py
Connecting to 192.168.10.11 using username administrator@vsphere.local
/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:769: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html
InsecureRequestWarning)
Connected to 192.168.10.11, VMware vCenter Server 5.5.0 build-2183111
Creating Datacenter LABDC
Creating Cluster LABC
Task completed successfully, result: 'vim.HostSystem:host-142'
Task completed successfully, result: 'vim.HostSystem:host-148'

and this is what it looks like after execution.

before

Other parts in the series can be found below :

  1. Part 1 :- Connecting to the VCSA
  2. Part 2 :- Creating Datacenters and Clusters
  3. Part 3 :- Adding Hosts to Clusters
  4. Part 4 :- Adding a NFS Share to a Cluster of Hosts