CloudStack Advanced Zone With Security Groups

A Shared Network in CloudStack can be used to assign public IPs to guest VM instances instead of internal private IPs. A Guest VM with a routable public IP is directly reachable from the Internet without using any NAT services. Services providers usually create a one or more shared networks with public subnets to provide VPS/shared hosting services for their customers. Please also see earlier post on CloudStack Shared Networks.

For guest instances which are on a VLAN based Isolated Network, the CloudStack managed virtual router (VR) provides NAT/SNAT and firewall services. Users can modify the Network ACLs directly from the CloudStack UI. Virtual routers however, cannot be used as firewall on shared networks as they are directly accessible from the Internet.

So what options are available to provide firewall services in shared networks?

1. Manually manage firewall entries on the physical router in a non-cloudy way. This would be the good old way where network admins modified the network ACLs on the physical firewall as per customer’s request. You will still have to add local iptables rules on the VM to block traffic from other customer’s guest VM instances
2. Integrate CloudStack with a supported external gateway device like a Juniper SRX. The list of supported devices in CloudStack 4.2.0 is at http://cloudstack.apache.org/docs/en-US/Apache_CloudStack/4.2.0/html-single/Installation_Guide/#hardware-firewall. Caveat: I have no personal experience in using CloudStack with external devices to confirm if it actually works
3. The Guest VMs OS native firewall capabilities can be used to filter traffic. Linux users can filter traffic using IPTABLES, FreeBSD users can use PF/IPFW and Windows users would use the Windows firewall services. The filtering rules would be internal to the guest instance and completely under the control of the user
4. Another option (and the topic of this post) is to use CloudStack’s SecurityGroups feature to provide firewall services. You can read about SecurityGroups from the CloudStack Installation Guide. SecurityGroups are essentially IPTABLES rules inserted on the hypervisor host. According to CloudStack Install Guide, SecurityGroups in an Advanced Zone is supported only on KVM Hypervisors as of CloudStack 4.2.0. This should hopefully change in future releases.

In order to create a Shared Network with SecurityGroups enabled…

1. Create your Advanced Zone with the SecurityGroup feature enabled and DefaultSharedNetworkOfferingWithSGService. According to the mailing list discussions, SecurityGroups cannot be enabled once the Zone is created.
2. While a SecurityGroup disabled Advanced Zone has the usual 4 traffic types (Management, Storage, Guest and Public), a SecurityGroup enabled Advanced Zone has only 3 traffic types – Management, Storage and Guest. There is no Public traffic type.
3. When you reach the Guest Traffic settings screen, provide the network details of your routable subnet. You can add more public routable guest networks later.
4. Once the zone is created, you can now launch new instances into the shared defaultGuestNetwork
5. Modify the SecurityGroup Ingress/Egress rules as applicable from the Network TabBy default Egress traffic is allowed while Ingress traffic is dropped. Internet access from the VM will work by default provided the network routing is working
6. Add an ICMP Ingress rule to allow pings anywhere (0.0.0.0/0)
7. Finally, ping the VM using its public IP. Once basic ICMP ping is confirmed to be working, whitelist service ports (like SSH) as applicable for your environment

For the curious, you can ssh to the KVM hypervisor(s) and view the IPTABLES rules inserted which are related to the SharedNetwork’s SecuritGroup rules. Look for the instance internal name (i-x-yy-VM) in the iptables -L -v -n output.

Chain i-2-19-VM (1 references)
target     prot opt source               destination         
ACCEPT     icmp --  0.0.0.0/0            0.0.0.0/0           icmp type 255 
DROP       all  --  0.0.0.0/0            0.0.0.0/0           

Chain i-2-19-VM-eg (1 references)
target     prot opt source               destination         
RETURN     all  --  0.0.0.0/0            0.0.0.0/0           

Chain i-2-19-def (2 references)
target     prot opt source               destination         
ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0           state RELATED,ESTABLISHED 
ACCEPT     udp  --  0.0.0.0/0            0.0.0.0/0           PHYSDEV match --physdev-in vnet4 --physdev-is-bridged udp spt:68 dpt:67 
ACCEPT     udp  --  0.0.0.0/0            0.0.0.0/0           PHYSDEV match --physdev-out vnet4 --physdev-is-bridged udp spt:67 dpt:68 
RETURN     udp  --  0.0.0.0/0            0.0.0.0/0           PHYSDEV match --physdev-in vnet4 --physdev-is-bridged match-set i-2-19-VM src udp dpt:53 
i-2-19-VM-eg  all  --  0.0.0.0/0            0.0.0.0/0           PHYSDEV match --physdev-in vnet4 --physdev-is-bridged match-set i-2-19-VM src 
i-2-19-VM  all  --  0.0.0.0/0            0.0.0.0/0           PHYSDEV match --physdev-out vnet4 --physdev-is-bridged 

SecurityGroups And XenServer

While the official documentation claims that Advanced Zone security groups is supported only on KVM hypervisors, the Add Zone creation wizard does show KVM and XenServer hypervisors.

The following CloudStack Wiki links also mentions support for XenServer:

Isolation+based+on+Security+Groups+in+Advance+zone
Security+Groups+Isolation+in+Advanced+Zone

However in my tests, VMs which were spun on the XenServer hypervisors were not reachable after adding Ingress permit rules. My XenServer hosts are using the default OpenVswitch backend instead of bridged networking which could be the reason. Will update this post once I have had a chance to test the bridged backend also.

Conclusion

While SecurityGroups (in an Advanced Zone) seems like a nice thing to have, you will however miss out on lot of nice features of an Advanced Zone. Please see the Design Document to fully understand the impact.