Last week I was helping a customer with installing and configuring VMware vCloud Director 5.1. Things were going pretty smooth until I noticed that we could not get any traffic to get out of the organisation to the external network. The Organisation network was connected to a Edge Gateway but it did not let any traffic through. In vCloud 1.5 this just worked out of the box. It used to have IP Masquerade and NAT enabled per default. After some troubleshooting we figured this was not the case in vCloud 5.1 on Edge Gateways. To enable IP Masquerade and NAT you have to do the following.

1. Sub allocate an IP pool to the Edge Gateway
2. Create an SNAT rule on the Edge Gateway 

VMware vCloud Director has several use cases. One of them is to use vCloud director for “lab management” purposes. For instance you can run several vApps simultaneous with the same IP and MAC addresses completely isolated or “fenced” We are currently building our own vApps for VMware training purposes. The ability to build one vApp for one student – add it to the catalog and then deploy it X amount of times is essential in our environment. 

To understand fencing in vCloud Director you really need to understand the basic network concepts in vCloud director. vCloud director has three kind of networks

External Network: This is basically a portgroup on a standard vSwitch, Distributed vSwitch or Nexus1000. This is where you get “into” the cloud. The external network is a network where you physically have allocated a vlan and an IP segment. The IP segment can either be public internet ip addresses or private ip addresses. In our case we are using the private subnet “10.10.10.0/24” When you define your External Network you have to define this IP segment gateway and subnet mask. Besides that you define a “static IP pool”. This pool is addresses vCloud director can manage and use for vShield edge devices and virtual machines.

Organisation network: An organisation network is a network only available to the organisation it is deployed to. This network is automatically created in vCloud Director and you don’t need to manually create the portgroup in vSphere. The Organisation network can either be 1) direct connected to an external connected (in this case we use 10.10.10.0/24) 2) routed connected with a vShield edge device (in this case a vShield edge device will have an IP on the external network example: 10.10.10.20 and another ip address on the organisation network – for instanse 192.168.0.1) or 3) no connection to an external network.

vApp network: a vApp network available to only virtual machines in the same vApp. This network is automatically created when defining it. The vApp network can either be 1) routed connected to the organisation network or 2) no connection to organisation network.

If you need more information on vCloud network concepts check out Duncans great two series post on the matter: vcd network post 1 |  vcd network post 2

Fencing in vCloud Director

Option 1:

You can do fencing in vCloud director in two ways. When looking at the GUI it would seem only one. But you can actually do fencing without putting a vApp into “Fence vApp” mode. An example of this. Look at the following two screenshots. I am building a vApp that is connected to my organisation network. That means I am using my organisation IP addresses for all virtual machines in the vApp. (10.10.10.0/24). In the first screenshot I have not selected the “Fence vApp” checkmark. In the second I have selected it.

The first one is direct connected to the organisation network – and in this option I have no way of running with duplicate IP and MAC for my virtual machines. After selecting the “Fence vApp” checkmark you should notice that it changes connection type to “Fenced” and you will see NAT and FIREWALL enabled. In this case we can now copy the vApp without customization and run it serveral times with the same IP and MAC addresses.

One thing to notice about this way of doing fencing is that you can only use the “Fence” button on organisation networks. NOT on vApp networks. This will give a situation where a vShield Edge will be deployed where the internal and external interface has the same IP subnet(10.10.10.0/24), and the edge does proxy-arp and NAT 

This option is probably the first one you would go to because you acutally see the “Fence” option inside vCloud Director. The truth is though, that you can do fencing without ever setting that option. And you would probably do option 2 instead.

Option 2: 

When looking at option1 we see that the vShield edge device is the one making sure we can do the fencing. In vCloud director when we create “vApp” network we can choose to make it “routed” to an organisation network. To do “manually” fencing you would create a vApp network when creating your vApp. This vApp you would connect with a routed connection to the External network. In our case we would get a “vApp” network with IP 192.168.2.0/24 with a vShield Edge Device connected and on the other side of the vShield edge you would have the 10.10.10.0/24 network. 

Look at screenshot 2 and 3. This is what happens when you copy a vApp and don’t customize it. The vApp network will be the exact same. The only difference is the external IP address of the vShield Edge device. This means that you can use the external address to get “into” your vApps virtual machine with a simple NAT rule.

Summary

Fencing can be done in two ways. Both use the vShield Edge Device.

Option 1: Use the “Fence” option and use your organisation IP range directly on your fenced virtual machines. (same inside and outside IP range 10.10.10.0/24)

Option 2: Create a vApp network and make it “routed”. In this case you have one internal IP segment192.168.0.0/24 and one external IP segment 10.10.10.0/24)