Now that I have started studying for my MCITP certification I found that I am going to need some windows servers in order to complete all the labs. Not to mention that I will probably to need to setup a network inside of my network so that I can test things like AD replication between “New York” and “San Francisco” or whatever make believe location the book listed. The problem is that I don’t have too many spare servers capable of running Windows 2008 R2 just sitting around. I also don’t want to pay for the power to run all them while they idle along because ACME Inc’s workload isnt very high. However I do have an ML370 G5 with more then enough RAM sitting on my desk…
Enter VMware ESXi Hypervisor (I guess I should use the new terms).
Basically what I will explain is how to take …
And make it all run on a single server while making sure that each “site” has its own subnet and remains isolated from my employers network. The other advantage is that we can do this all with free software, and still provide internet access to each virtual site so that if we wanted to we can download windows updates, activate windows, and basically just emulate a real network. So lets get started.
The first thing that we will need to do is load VMware’s free hypervisor on to the hardware. I wont go into details on that, but there are many good places and even some VMware white papers on how to do this. After installing the hypervisor we will want to go ahead and configure it as we normally would, so go ahead and give it a static IP, a password, and configure your storage and other required stuff so that we can run VM’s.
After all the setup stuff is complete you will want to go to the configuration tab, and navigate down to the networking section over on the left side of the config tab. You will find that you have a single vSwitch0 listed which will probably have just the standard vmk0 interface.
So what we will want to do is click “Add Networking” in the top right corner. On the first screen select “Virtual Machine” as the connection type then press ‘Next’. On the next screen you are asked to select a physical network adapter to associate with this new vSwitch. We do now want a physical NIC on this vSwitch so un-check anything that is checked and press ‘Next’. On the next screen you will need to name the vSwitch, you can make it whatever you want… you can also change this later, but to make it simple I am naming mine the “City” that they represent. So Chicago, San Francisco, or New York. Click ‘Finish’, and we now have our first site. Do this as many more times as you need sites. Eventually you have something that looks like this…
So to plan this out we are going to associate “New York” with a subnet… in my case I picked 192.168.10.0/24. I then picked 192.168.20.0/24 for Chicago, and 192.168.30.0/24 for San Francisco. If we were to spin up a VM and put its network adapter in Chicago and one in New York, they would not be able to talk to each other because there is no physical or virtual connection out of that vSwitch. Also your VM’s would not be able to talk to the internet either… clearly a problem if you want to activate them, or download updates, etc etc.
In order to make the “sites” internet accessible as well as route traffic between sites we need a router… a virtual router to be exact. For this task we are going to use Debian Linux, you could substitute in Ubuntu or whatever you want really, I just like Debian. What we will do is create a virtual machine, and we will put one virtual network card in vSwitch0, this network card will pull a DHCP address from your LAN. Then we will also add more network cards… one for each vSwitch/Site you created. So in our case we will have 4 virtual network cards total in the virtual machine.
Here is what the router’s settings should look like:
Install Debian, there are a bunch of tutorials out there so I wont go into detail, but basically you can remove all roles except the base system. After you reboot, login as root and install the ssh server by issueing the following command.
apt-get install openssh-server
After installing ssh we can use Putty to connect to the host so that we can copy and paste in the rest of the config. So now we should have a linux machine that has 4 network ports, eth0-eth3. Eth0 should be pulling a DHCP address (or configured however you need for your LAN) and we will now configure eth1-3 to be the gateway address for each of our sites.
Network configuration in Debian (and Ubuntu) is stores in /etc/network/interfaces, inside Putty open this file so we can edit it.
nano /etc/network/interfaces
append the following to the existing file.
iface eth1 inet static
address 192.168.10.254
netmask 255.255.255.0
broadcast 192.168.10.255
network 192.168.10.0
iface eth2 inet static
address 192.168.20.254
netmask 255.255.255.0
broadcast 192.168.20.255
network 192.168.20.0
iface eth3 inet static
address 192.168.30.254
netmask 255.255.255.0
broadcast 192.168.30.255
network 192.168.30.0
Save the file by pressing ‘Ctrl + O’ then press ‘Enter’ then ‘Ctrl + X’ to exit.
The next thing we need to do is configure linux to allow IPv4 Forwarding. This can be done by editing the /etc/sysctl.conf file
nano /etc/sysctl.conf
Seach for ‘ip_forward’ by pressing ‘Ctrl + w’ and typing ‘ip_forward’ then press enter. This should take you to the line in the config file that controls ipv4 forwarding, turn it on by changing the 0 (zero) to a 1 (one). Then save and exit the same way you did for the interfaces file. One step left, this is to configure NAT so that we can get out to the internet, and the best way I have found to do this is to install webmin and use its GUI. This is pretty easy on Debian.
wget http://prdownloads.sourceforge.net/webadmin/webmin-1.520.tar.gz
tar xzvf webmin-1.520.tar.gz
cd webmin-1.520
./setup.sh
Answer the installer’s questions and then go to your browser and open webmin. If you do not know what the dhcp ip address is on your router type ‘ifconfig’ and look for the eth0 interface, this will be the LAN ip of your router. The web URL you will need is http://:10000 Login with the credentials your created during the installer. Then on the left, click on Networking and then Linux Firewall… you should see the following:
Click on the second radial button “Do network address translation on external interface”. Click ‘Setup Firewall’. The last part is to click the “Activate at Boot” button, this will save the configuration and activate it ever time the router is rebooted. Now reboot the linux router, after it comes back up create a VM in one of the “Sites” and set its gateway to the ip of the router in the subnet and as a dns server use either your LAN’s DNS server or one from the internet like 4.2.2.1
That is it! you now have a multi-site, routed network, with internet access… ALL ON ONE SERVER! it doesn’t get much better then that. If you get stuck at any point feel free to leave a comment or shoot me an email, as always I would be glad to help.
Thanks! Worked perfectly!
Good article, but this won’t seem to be compatible with my router ip, any advice?
Hai I am also Looking for same type of setup.
Let me Explain my Lab Senirio.
Dedicated Physical Box for Hosting my Vm’s (VMWARE-GSX) It is running vmware server and hosting my FILESERVER VM.
Dedicated Physical Box for ADS.
Dedicated Box for Administration
Now i have planed to install vmware workstaion in my Administration box and host couple of vm’s there for multi site envernoment.
But Please suggest me i need to setup a vm as router how i can do it.
Hi,
I’m a bit of a newbie with linux and routing. I’ve setup my VM environment but something has gone wrong along the way. I think it maybe the the interfaces file or something. Anymore screen shots\help available?
Try the simple router Zeroshell.
I have a win 7 64 , 8 gb ram, amd fx 3.3 ghz pro,
can i install a vm work station and create a vm with a win 2008 server on it and create other vms with win 7 clients.. and set up client -server connectivity?
yeah, 8GB of ram will be your limitation. 16Gb would be better, but with 8 you should be able to spin up a couple servers if you only give them 1-2 GB of ram. For a lab they should still run fine.
What if I have a mac book pro with 8gb ram, and its i5 intel, if i install windows 7 is it possible to have vmware? Or if im studying for an MCITP cert i need to buy a PC
This post was based on VMware running on a server but you could get yourself VMware Fusion or Parallels and run virtual machines too. 8GB of ram should allow you to run two or three windows machines if you cut the ram down to about 2GB each. Disk speed would probably be the biggest bottleneck unless you have eSATA or USB3.0
I tried to follow the tutorial using VMware workstation 8. I cannot access the internet from the virtual machines, I cannot ping a virtual machines located in a different network.
My settings are the following:
In VMware workstation 8 network editor
VMnet0 is Bridget to my wireless card
VMnet 2 host-only 192.168.10.0/24
VMnet 3 host-only 192.168.20.0/24
VMnet 4 host-only 192.168.30.0/24
In Webadmin I had to got to network to active the eth1, eth2, eth3
I followed your setting below. But when I ping a machine from 192.168.10 network to 192.168.20 network it fails.
The networking in VMware workstation will be completely different than it is in ESXi. without knowing what is bridged to where it will be hard to troubleshoot. There could also be something wrong with your linux router too.
I am using ubuntu 12.04. eth0 is working fine. eth1 and eth2 are not. I set them up identically as you did inside /etc/network/interfaces. However I cannot ping 192.168.10.254 from the “linux router” or ubuntu itself after a reboot. should I be able to?
did you set the interfaces to auto eth1 and auto eth2 so that they are brought online after a reboot ?
do an ifconfig and see if they exist
Yes they exist in the linux VM. I have webmin installed and configured to NAT. I now have a second VM in one of the other sites with that NIC applied to it. That NIC being eth1 in the firewall VM. I have assigned it an IP address with the correct site and the firewall cannot ping the server in that site and the server cannot ping the firewall. The VM in the other site cannot ping the gateway either which is the firewall of course but acting as a router too, the gateway for its site 192.168.10.254, it cannot ping.
The only difference is I setup the network via the network manager instead of using /etc/network/interfaces. When I setup eth1 and eth2 inside of /interfaces and then did a ifconfig they didn’t exist. I did a ifconfig -a and I could see them but they didn’t show an ip address assigned to them. The were also unping-able from the firewall/itself also. Should I setup the interfaces in /etc/network/interfaces a different way than you do in debian?
Or is ubuntu network manager interfering with /etc/network/interfaces? I setup the network just like you have it up top inside /etc/network/interfaces and its still not working. No ping between the firewall router(ubuntu) and my windows server VM connected via eth1 on the firewall(nic2 in vmware).
Never mind! It’s been working this whole time. The other VM I put in one of the sites is a Win2k8 DC and it has a route stuck in its route table of itself. I can’t get it out but that’s why its not communicating on the new vlan/ip. I put another server on that NIC/vlan and it is working just fine.
Awesome! did you make the route that is stuck or did something else? there is a command line option to delete persistent static routes but id have to google it…
No I found out you cannot clone a DC, it screws up all kinds of things makes them un-useable. I am building a new DC and then going to simply switch its network over to my lab switches and changes its IP address.