CloudStack “simple” Advanced Network Example

Before we begin, please do read the following links for a basic understanding of CloudStack’s networking concepts.

  1. CloudStack Manual About Physical Networks Section
  2. CloudStack Advanced Network Tutorial Step by Step
  3. Understanding CloudStack’s Physical Networking Architecture
  4. Chapter 15. Managing Networks and Traffic

Confused yet? 🙂

Here is one more article for creating an Advanced Zone in CloudStack but follows a keep-it-simple principle. The assumptions are bare minimum must have requirements for a Advanced Zone.

Pre-requisites

  1. One CloudStack Management Server fully installed (csman1)
  2. One or more KVM hypervisors with 1 physical network interface (NIC)
  3. One RFC1918 private /24 subnet (192.168.44.0/24) to be used for “management” traffic with Internet access
  4. One public /24 public subnet (192.168.65.0/24 on VLAN 65) to be used for “public” traffic with Internet access and is assigned a VLAN. 192.168.65.10 – 192.168.65.199 range will be assigned for CloudStack use
  5. One RFC1918 /24 subnet (10.1.1.0/24) to be used for “guest” traffic
  6. A storage device whose interface is also on the management network (192.168.44.0/24) and is reachable from the management server and the KVM hypervisors directly
  7. An L3 capable Switch whose ports are configured to allow tagged VLAN traffic for ranges 100-109 (guest traffic) and 65 (public traffic)

Now, lets begin with the IP assignments. Please do refer the Network Interfaces configuration section from the RHEL manual if required.

  1. CloudStack management server (csman1) running on CentOS 6.5: 192.168.44.1
  2. The storage server (nas1): 192.168.44.2
  3. The CentOS 6.5 KVM hypervisor (kvm1): 192.168.44.3
  4. The default gateway for the management network: 192.168.44.254
  5. The default gateway for the public network: 192.168.65.254
  6. Management IP range dedicated to System VM communication: 192.168.44.20 to 192.168.44.30
  7. Public IP range dedicated to CloudStack use: 192.168.65.10 to 192.168.65.199

Next, lets configure bridged networking on the KVM host(s):

On the KVM host(s):

  1. Update /etc/sysconfig/network-scripts/ifcfg-eth0 as below:
    DEVICE=eth0
    TYPE=Ethernet
    IPV6INIT=no
    IPV6_AUTOCONF=no
    ONBOOT=yes
    HOTPLUG=no
    BOOTPROTO=none
    NM_CONTROLLED=no
    BRIDGE=cloudbr0
    
  2. Update /etc/sysconfig/network-scripts/ifcfg-cloudbr0 as below:
    DEVICE=cloudbr0
    TYPE=Bridge
    ONBOOT=yes
    USERCTL=no
    HOTPLUG=no
    NM_CONTROLLED=no
    IPV6INIT=no
    IPV6_AUTOCONF=no
    BOOTPROTO=none
    IPADDR=192.168.44.3
    NETMASK=255.255.255.0
    GATEWAY=192.168.44.254
    DELAY=0
    STP=no
    

Once you restart networking service, the bridge interfaces would be created as below where cloudbr0 is mapped to eth0.

[root@kvm1 ~]# brctl show
bridge name	bridge id		STP enabled	interfaces
cloudbr0	8000.000c293bb945	no		eth0

Once you have KVM reconfigured, please do verify connectivity (via ICMP ping) to:

  1. Between the management server and the hypervisors
  2. To the storage device/NAS
  3. To the default gateway
  4. A public IP address like Google DNS 8.8.8.8 to confirm Internet routing

Please ensure that you have configured your L3 switch to allow tagged VLAN traffic for VLAN 65 and VLAN 100 to VLAN 109. There are plenty of guides on the Internet which can help you configure VLANs for your network device.

Now, finally to the CloudStack Add Zone configuration wizard:

  1. Select Advanced ZoneScreen Shot 2014-01-02 at 9.41.08 am
  2. Set the Guest CIDR to “10.1.1.0/24”Screen Shot 2014-01-02 at 9.42.37 am
  3. The default traffic types “Management”, “Guest” and “Public” are assigned to “Physical Network 1”. Keep it that way.Screen Shot 2014-01-02 at 9.44.26 am
  4. This step is where all the magic happens – assign traffic labels correctly by clicking on the edit button for “Management”, “Guest” and “Public Traffic” icons. The KVM traffic label should be cloudbr0
  5. Add public traffic range 192.168.65.10 – 192.168.65.199 on VLAN65Screen Shot 2014-01-02 at 9.52.08 am
  6. Add the first POD with the small reserved IP range (192.168.44.20-30 ) from the management subnet 192.168.44.0/24Screen Shot 2014-01-02 at 9.56.37 am
  7. Assign the VLAN range (100-109) for guest traffic useScreen Shot 2014-01-02 at 9.58.22 am
  8. Complete the steps for adding a Cluster, Host, primary storage and secondary storage and create the zone.

Thats about it.

Where to go from here…

  1. Add more hypervisor types like XenServer and VMware ESXi
  2. Add additional physical network interfaces
  3. Add dedicated NIC for storage traffic
  4. Add dedicated NIC for public traffic
  5. Add dedicated NIC for guest traffic
  6. Use bonded interface (LACP/LAGG) for fault tolerance and improve throughput
  7. Use tagged VLAN for management traffic on KVM hypervisor
  8. Add SDN technologies instead of VLAN isolation like GRE tunnels
  9. Enable security groups in an advanced network

Shanker Balan

Shanker Balan is a devops and infrastructure freelancer with over 14 years of industry experience in large scale Internet systems. He is available for both short term and long term projects on contract. Please use the Contact Form for any enquiry.

More Posts - Website

Follow Me:
TwitterLinkedIn

Published by

Shanker Balan

Shanker Balan is a devops and infrastructure freelancer with over 14 years of industry experience in large scale Internet systems. He is available for both short term and long term projects on contract. Please use the Contact Form for any enquiry.

15 thoughts on “CloudStack “simple” Advanced Network Example”

  1. Why is the guest network assigned a class A network, whilst the public network is assigned a class C ?

  2. Hi Jon,

    The guest and public network ranges are only examples. You would ideally choose a range from the RFC1918 block for guests and use a non RFC1918 range as public. Usually, the public range is allocated by your ISP.

    Hth.

  3. One question from my side, why do we need internet access on the management network?

    “One RFC1918 private /24 subnet (192.168.44.0/24) to be used for “management” traffic with Internet access”

    btw. nice tutorial!

  4. Hi Konstantinos ,

    Thank you for the positive feedback. Glad you found the content useful.

    You would require Internet access for the management network to seed the cloudstack templates. If you don’t have management network access to the network, it makes it a little tricky to seed the templates.

    Hth.

  5. Hello there,

    It may be obvious but are there any best practices to setup the “internet access” to these RFC1918 private /24 subnets. Can’t figure that out, there is likely the need for some kind of NAT-box. Thank you for this tutorial.

  6. Hi Giver,

    Glad you found this post useful.

    Ideally, you would be using your ISP assigned routable subnet as your “public” addresses.

    In my case, I dont require the traffic to leave my internal lab network. So I use pfSense for inter VLAN
    routing of the RFC1918 assigned “public” network with static routes on my edge firewall. There is no need
    to NAT when you can do internal routing.

    192.168.55.1 is my pfSense router doing inter VLAN routing while 192.168.44.1 is my edge router which finally does the NAT for outbound traffic.

    root@s-2-VM:~# traceroute -n 8.8.8.8
    traceroute to 8.8.8.8 (8.8.8.8), 30 hops max, 60 byte packets
    1 192.168.55.1 3.859 ms 3.844 ms 3.820 ms
    2 192.168.44.1 3.806 ms 3.789 ms 3.772 ms
    3 103.5.132.54 17.844 ms 17.778 ms 17.810 ms
    4 103.5.132.49 17.819 ms 17.795 ms 17.771 ms
    5 115.249.242.98 17.745 ms 17.728 ms 17.700 ms
    6 * * *
    7 72.14.212.118 37.965 ms 37.885 ms 37.921 ms
    8 209.85.245.167 37.912 ms 48.129 ms 48.026 ms
    9 8.8.8.8 49.933 ms 49.872 ms 49.846 ms

    I am not aware of any best practices around this but I quite like using pfSense to handle Inter VLAN routing for RFC1918 space.

    Hth.

  7. Thank you for your reply. The advanced networking in CloudStack is a kind of confusion. I’m at the very beginning of my cloudstack episode and couldn’t find any helpful documentation. With basic networking everything works fine, but when using the advanced networking I have no connection to the System VM. Why couldn’t CloudStack not just fire up one of these magic Virtual Router VM’s? 🙂

  8. Hi Giver,

    Most people who have had a requirement to work with CloudStack usually come from a specific vertical like virtualisation or Linux systems administration and have a passing knowledge of networking technologies. A basic cloudstack zone builds on top of a server virtualisation technology and thus virtualisation admins and Linux admins find it simple to understand and deploy.

    CloudStack Advanced zone however builds on top of network virtualisation technologies like L3 VLANs and SDN. Most admins I have come across have never had to deal with switches and routers and this makes it a lot harder for them to deploy a CloudStack Advanced Zone. They have relied on Network Admins to do the VLAN management on the network devices in the past. Network admins don’t understand network virtualisation of CloudStack either.

    CloudStack Basic Zone Admin == Virtualisation / Linux Server Admin
    CloudStack Advanced Zone Admin == Virtualisation / Linux Server Admin + Network Admin

    CloudStack Advanced zones are really simple if you can wrap your head around network virtualisation.

  9. Thanks for the tutorial.

    Your comments that “CloudStack Advanced Zone Admin == Virtualisation / Linux Server Admin + Network Admin” hits the nail on the head!

    However, many network admins themselves are unfamiliar with network virtualization and SDN, so we sysadmins get little help from them.

    I have been wondering, and have not seen any tutorials or documents, if we can deploy a cloudstack instance with advanced networking using little or no network virtualization and/or SDN, relying as much as possible on exiting physical network gears, routers/switches/physical NICs etc? Obviously, in such a set up, we may have to skip many features targeted for multi-tenancy, strict guest VLAN isolation, security group/iptables etc. But many private clouds are for enterprise internal users only, they don’t need all those extra features to complicate the setups !

    A side by side comparing of a normal CS advanced networking setup versus one using only physical network setup (in traditional sense) would greatly help people understand the differences and help those enterprise users who are new to CloudStack and have to evaluate CloudStack on their existing networking environments.

    Yiping

  10. Hi!

    I saw you know everything about cloudstack so I would like to ask you a question. I tried to install advanced mode for cloudstack. But at the first step I didn’t check the option Security Groups. Everything is OK in my installation. I can create guest networks but I cannot create Isolated Networks.Precisely, an isoleted network can be created but it doesn’t work correctly. I cannot use it to create instances. Moreover when I log in to the UI as a simple user, I can create isoletes networks but again when I try to create an instance, I cannot. The cloudstack tries to create it but the instance state is ERROR. I believe that the problem is that I didn’t check the option that I told you(security groups). Have you any idea what I can do???Please give a hint or something!

    Thank you very much

  11. Hi Shanker, thanks or the post. It works for me. But have questions is , i need to create a VM to a user with Public IP address. What should I do ?
    Cause the public IP network to Guest network , I need to do NAT, from Public to Guest Network ? How to do this?
    i do a work around here is that , I create public IP addresses as Guest Network, and when allocation of IP, it will directly allocate as VM IP for public IP.
    However above work around create issue whereby, the network shaping of bandwidth not working. I guest the shopping of network allocation will only work for Public to Guest NAT , right ?

Leave a Reply