26 Oct

How To Set Up A Loadbalanced High-Availability Apache Cluster Based On Ubuntu 8.04 LTS

Please note that my main reference and source is Falko’s article here just modified for Ubuntu 8.04 LTS. Also some parts were taken from Falko’s article “The Perfect Server – Ubuntu 8.04 LTS” here.

Version 1.5
Author: Mohamed Ghaleb <Mohamed_Ghaleb [at] msn [dot] com>  (English and German only please)
Revised Author: William Chace <wchace [at] ncspros [dot] com>  (English only please)
Last edited 10/024/2011

This tutorial shows how to set up a two-node Apache web server cluster that provides high-availability. In front of the Apache cluster we create a load balancer that splits up incoming requests between the four Apache nodes. Because we do not want the load balancer to become another “Single Point Of Failure”, we must provide high-availability for the load balancer, too. Therefore our load balancer will in fact consist of two load balancer nodes that monitor each other using heartbeat, and if one load balancer fails, the other takes over silently.

The advantage of using a load balancer compared to using round robin DNS is that it takes care of the load on the web server nodes and tries to direct requests to the node with less load, and it also takes care of connections/sessions. Many web applications (e.g. forum software, shopping carts, etc.) make use of sessions, and if you are in a session on Apache node 1, you would lose that session if suddenly node 2 served your requests. In addition to that, if one of the Apache nodes goes down, the load balancer realizes that and directs all incoming requests to the remaining node which would not be possible with round robin DNS.

For this setup, we need four nodes (two Apache nodes and two load balancer nodes) and five IP addresses: one for each node and one virtual IP address that will be shared by the load balancer nodes and used for incoming HTTP requests.

I will use the following setup here:

  • Apache node 1: webserver1.tm.local (webserver1) – IP address: 192.168.0.103; Apache document root: /var/www
  • Apache node 2: webserver2.tm.local (webserver2) – IP address: 192.168.0.104; Apache document root: /var/www
  • Load Balancer node 1: loadb1.tm.local (loadb1) – IP address: 192.168.0.101
  • Load Balancer node 2: loadb2.tm.local (loadb2) – IP address: 192.168.0.102
  • Virtual IP Address: 192.168.0.105 (used for incoming requests)

Have a look at the drawing on http://www.linuxvirtualserver.org/docs/ha/ultramonkey.html to understand how this setup looks like.

In this tutorial I will use Ubuntu 8.04 LTS for all four nodes, just install basic Ubuntu 8.04 LTS on all four nodes.

I want to say first that this is not the only way of setting up such a system. There are many ways of achieving this goal but this is the way I take. I do not issue any guarantee that this will work for you!

I also recommend you to have a DNS server in place.

Step 1 to 6 should be done on all four servers.

 

1 Enable The root Account

Run

sudo passwd root

and give root a password. Afterwards we become root by running

su

 

2 Install The SSH Server

If you did not install the OpenSSH server during the system installation, you can do it now:

apt-get install ssh openssh-server

From now on you can use an SSH client such as PuTTY and connect from your workstation to your Ubuntu 8.04 LTS server and follow the remaining steps from this tutorial.

3 Install vim-full

I’ll use vi as my text editor in this tutorial. The default vi program has some strange behavior on Ubuntu and Debian; to fix this, we install vim-full:

apt-get install vim-full

(You don’t have to do this if you use a different text editor such as joe or nano.)

4 Configure The Network

Because the Ubuntu installer has configured our system to get its network settings via DHCP, we have to change that now because a server should have a static IP address. Edit /etc/network/interfaces and adjust it to your needs (in this example setup I will use the IP address 192.168.0.101):

vi /etc/network/interfaces

* This file describes the network interfaces available on your system
* and how to activate them. For more information, see interfaces(5).

# The loopback network interface

auto lo
iface lo inet loopback

# The primary network interface
auto eth0
iface eth0 inet static
address 192.168.0.101
netmask 255.255.255.0
network 192.168.0.0
broadcast 192.168.0.255
gateway 192.168.0.1

Please make sure your network configuration are set correctly, feel free to change that based on your network configuration.

Then restart your network:

/etc/init.d/networking restart

Then edit /etc/hosts. Make it look like this:

vi /etc/hosts
127.0.0.1       localhost.localdomain   localhost
192.168.0.101   loadb1.tm.local         loadb1

# The following lines are desirable for IPv6 capable hosts
::1     ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
ff02::3 ip6-allhosts

Now run

echo loadb1.tm.local > /etc/hostname
/etc/init.d/hostname.sh start

Afterwards, run

hostname
hostname -f

Both should show loadb1.tm.local now.

If you have a DNS server in place (recommended) make sure the 4 servers configured to use it, if you don’t have a DNS click here

vi /etc/resolv.conf
search tm.local
nameserver 192.168.0.100

5 Edit /etc/apt/sources.list And Update Your Linux Installation

Edit /etc/apt/sources.list. Comment out or remove the installation CD from the file and make sure that the universe and multiverse repositories are enabled. It should look like this:

vi /etc/apt/sources.list
#
# deb cdrom:[Ubuntu-Server 8.04 _Hardy Heron_ - Release i386 (20080423.2)]/ hardy main restricted
#deb cdrom:[Ubuntu-Server 8.04 _Hardy Heron_ - Release i386 (20080423.2)]/ hardy main restricted
# See http://help.ubuntu.com/community/UpgradeNotes for how to upgrade to
# newer versions of the distribution.
deb http://de.archive.ubuntu.com/ubuntu/ hardy main restricted
deb-src http://de.archive.ubuntu.com/ubuntu/ hardy main restricted
## Major bug fix updates produced after the final release of the
## distribution.
deb http://de.archive.ubuntu.com/ubuntu/ hardy-updates main restricted
deb-src http://de.archive.ubuntu.com/ubuntu/ hardy-updates main restricted
## N.B. software from this repository is ENTIRELY UNSUPPORTED by the Ubuntu
## team, and may not be under a free licence. Please satisfy yourself as to
## your rights to use the software. Also, please note that software in
## universe WILL NOT receive any review or updates from the Ubuntu security
## team.
deb http://de.archive.ubuntu.com/ubuntu/ hardy universe
deb-src http://de.archive.ubuntu.com/ubuntu/ hardy universe
deb http://de.archive.ubuntu.com/ubuntu/ hardy-updates universe
deb-src http://de.archive.ubuntu.com/ubuntu/ hardy-updates universe
## N.B. software from this repository is ENTIRELY UNSUPPORTED by the Ubuntu
## team, and may not be under a free licence. Please satisfy yourself as to
## your rights to use the software. Also, please note that software in
## multiverse WILL NOT receive any review or updates from the Ubuntu
## security team.
deb http://de.archive.ubuntu.com/ubuntu/ hardy multiverse
deb-src http://de.archive.ubuntu.com/ubuntu/ hardy multiverse
deb http://de.archive.ubuntu.com/ubuntu/ hardy-updates multiverse
deb-src http://de.archive.ubuntu.com/ubuntu/ hardy-updates multiverse
## Uncomment the following two lines to add software from the 'backports'
## repository.
## N.B. software from this repository may not have been tested as
## extensively as that contained in the main release, although it includes
## newer versions of some applications which may provide useful features.
## Also, please note that software in backports WILL NOT receive any review
## or updates from the Ubuntu security team.
## deb http://de.archive.ubuntu.com/ubuntu/ hardy-backports main restricted universe multiverse
## deb-src http://de.archive.ubuntu.com/ubuntu/ hardy-backports main restricted universe multiverse
## Uncomment the following two lines to add software from Canonical's
## 'partner' repository. This software is not part of Ubuntu, but is
## offered by Canonical and the respective vendors as a service to Ubuntu
## users.
## deb http://archive.canonical.com/ubuntu hardy partner
## deb-src http://archive.canonical.com/ubuntu hardy partner
deb http://security.ubuntu.com/ubuntu hardy-security main restricted
deb-src http://security.ubuntu.com/ubuntu hardy-security main restricted
deb http://security.ubuntu.com/ubuntu hardy-security universe
deb-src http://security.ubuntu.com/ubuntu hardy-security universe
deb http://security.ubuntu.com/ubuntu hardy-security multiverse
deb-src http://security.ubuntu.com/ubuntu hardy-security multiverse

Then run

apt-get update

to update the apt package database and

apt-get upgrade

to install the latest updates (if there are any).

6 Disable AppArmor

AppArmor is a security extension (similar to SELinux) that should provide extended security, which usually causes more problems than advantages. Therefore I disable it.

We can disable it like this:

/etc/init.d/apparmor stop
update-rc.d -f apparmor remove

7 Install Apache (Only on the Webservers)

webserver1/webserver2:

apt-get install apache2

8 Enable IPVS On The Load Balancers

First we must enable IPVS on our load balancers. IPVS (IP Virtual Server) implements transport-layer load balancing inside the Linux kernel, so called Layer-4 switching.

loadb1/loadb2:

echo ip_vs_dh >> /etc/modules
echo ip_vs_ftp >> /etc/modules
echo ip_vs >> /etc/modules
echo ip_vs_lblc >> /etc/modules
echo ip_vs_lblcr >> /etc/modules
echo ip_vs_lc >> /etc/modules
echo ip_vs_nq >> /etc/modules
echo ip_vs_rr >> /etc/modules
echo ip_vs_sed >> /etc/modules
echo ip_vs_sh >> /etc/modules
echo ip_vs_wlc >> /etc/modules
echo ip_vs_wrr >> /etc/modules

Then we do this:

loadb1/loadb2:

modprobe ip_vs_dh
modprobe ip_vs_ftp
modprobe ip_vs
modprobe ip_vs_lblc
modprobe ip_vs_lblcr
modprobe ip_vs_lc
modprobe ip_vs_nq
modprobe ip_vs_rr
modprobe ip_vs_sed
modprobe ip_vs_sh
modprobe ip_vs_wlc
modprobe ip_vs_wrr

9 Install Ultra Monkey (packages) On The Load Balancers

Ultra Monkey is a project to create load balanced and highly available services on a local area network using Open Source components on the Linux operating system; the Ultra Monkey package provides heartbeat (used by the two load balancers to monitor each other and check if the other node is still alive) and ldirectord, the actual load balancer.

In the Original article Falko uses Debian and thus there are directly Debian repositories from Ultra Monkey, however as here we are using Ubuntu we will have to install ipvsadm ldirectord heartbeat.

loadb1/loadb2:

apt-get install ipvsadm ldirectord heartbeat

If you see this warning:

  ¦ libsensors3 not functional                                               ¦
  ¦                                                                          ¦
  ¦ It appears that your kernel is not compiled with sensors support. As a   ¦
  ¦ result, libsensors3 will not be functional on your system.               ¦
  ¦                                                                          ¦
  ¦ If you want to enable it, have a look at "I2C Hardware Sensors Chip      ¦
  ¦ support" in your kernel configuration.

you can ignore it, I didn’t see it on Ubuntu, but as it was in the original article I though to include it just in case.

10 Enable Packet Forwarding On The Load Balancers

The load balancers must be able to route traffic to the Apache nodes. Therefore we must enable packet forwarding on the load balancers. Add the following lines to /etc/sysctl.conf:

loadb1/loadb2:

# Enables packet forwarding
net.ipv4.ip_forward = 1

Then do this:

loadb1/loadb2:

sysctl -p

11 Configure heartbeat And ldirectord

Now we have to create three configuration files for heartbeat. They must be identical on loadb1 and loadb2!

loadb1/loadb2:

vi /etc/ha.d/ha.cf
logfacility local0
bcast eth0 # Linux
mcast eth0 225.0.0.1 694 1 0
auto_failback off
node loadb1.tm.local
node loadb2.tm.local
respawn hacluster /usr/lib/heartbeat/ipfail
apiauth ipfail gid=haclient uid=hacluster

Important: As nodenames we must use the output of

uname -n

on loadb1 and loadb2.

loadb1/loadb2:

vi /etc/ha.d/haresources
loadb1.tm.local \
ldirectord::ldirectord.cf \
LVSSyncDaemonSwap::master \
IPaddr2::192.168.0.105/24/eth0/192.168.0.255

Please note that the last line above has my virtual IP which is: 192.168.0.105, my netmask is 255.255.255.0 and as its class C my IP should be followed by /24 then at the end my broadcast IP 192.168.0.255, please make sure you use the correct IP configuration.

The first word in the first line above is the output of

uname -n

This file should be the same on both nodes, no matter if you start to create the file on loadb1 or loadb2! After IPaddr2 we put our virtual IP address 192.168.0.105.

loadb1/loadb2:

vi /etc/ha.d/authkeys
auth 3
3 md5 somerandomstring

somerandomstring is a password which the two heartbeat daemons on loadb1 and loadb2 use to authenticate against each other. Use your own string here. You have the choice between three authentication mechanisms. I use md5 as I believe it is the most secure one.

etc/ha.d/authkeys should be readable by root only, therefore we do this:

loadb1/loadb2:

chmod 600 /etc/ha.d/authkeys

ldirectord is the actual load balancer. We are going to configure our two load balancers (loadb1.tm.local and loadb2.tm.local) in an active/passive setup, which means we have one active load balancer, and the other one is a hot-standby and becomes active if the active one fails. To make it work, we must create the ldirectord configuration file /etc/ha.d/ldirectord.cf which again must be identical on loadb1 and loadb2.

loadb1/loadb2:

vi /etc/ha.d/ldirectord.cf
checktimeout=10
checkinterval=2
autoreload=no
logfile="local0"
quiescent=yes

virtual=192.168.0.105:80
 real=192.168.0.103:80 gate
 real=192.168.0.104:80 gate
 fallback=127.0.0.1:80 gate
 service=http
 request="ldirector.html"
 receive="Test Page"
 scheduler=rr
 protocol=tcp
 checktype=negotiate

In the virtual= line we put our virtual IP address (192.168.0.105 in this example), and in the real= lines we list the IP addresses of our Apache nodes (192.168.0.103 and 192.168.0.104 in this example). In the request= line we list the name of a file on webserver1 and webserver2 that ldirectord will request repeatedly to see if webserver1 and webserver2 are still alive. That file (that we are going to create later on) must contain the string listed in the receive= line.

Afterwards we create the system startup links for heartbeat and remove those of ldirectord because ldirectord will be started by the heartbeat daemon:

loadb1/loadb2:

update-rc.d heartbeat start 75 2 3 4 5 . stop 05 0 1 6 .
update-rc.d -f ldirectord remove

Finally we start heartbeat (and with it ldirectord):

loadb1/loadb2:

/etc/init.d/ldirectord stop
/etc/init.d/heartbeat start

12 Test The Load Balancers

Let’s check if both load balancers work as expected:

loadb1/loadb2:

ip addr sh eth0

The active load balancer should list the virtual IP address (192.168.0.105):

2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast qlen 1000
    link/ether 00:0c:29:4e:67:1a brd ff:ff:ff:ff:ff:ff
    inet 192.168.0.101/24 brd 192.168.0.255 scope global eth0
    inet 192.168.0.105/24 brd 192.168.0.255 scope global secondary eth0

The hot-standby should show this:

2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast qlen 1000
    link/ether 00:0c:29:34:d7:7e brd ff:ff:ff:ff:ff:ff
    inet 192.168.0.102/24 brd 192.168.0.255 scope global eth0

loadb1/loadb2:

ldirectord ldirectord.cf status

Output on the active load balancer:

ldirectord for /etc/ha.d/ldirectord.cf is running with pid: 5321

Output on the hot-standby:

ldirectord is stopped for /etc/ha.d/ldirectord.cf

loadb1/loadb2:

ipvsadm -L -n

Output on the active load balancer:

IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  192.168.0.105:80 rr
  -> 192.168.0.103:80               Route   1      0          0
  -> 192.168.0.104:80               Route   0      0          0

Output on the hot-standby:

IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn

loadb1/loadb2:

/etc/ha.d/resource.d/LVSSyncDaemonSwap master status

Output on the active load balancer:

master running
(ipvs_syncmaster pid: 5470)

Output on the hot-standby:

master stopped

If your tests went fine, you can now go on and configure the two Apache nodes.

13 Configure The Two Apache Nodes

Finally we must configure our Apache cluster nodes webserver1.tm.local and webserver2.tm.local to accept requests on the virtual IP address 192.168.0.105.

webserver1/webserver2:

apt-get install iproute

Add the following to /etc/sysctl.conf:

webserver1/webserver2:

vi /etc/sysctl.conf
# Enable configuration of arp_ignore option
net.ipv4.conf.all.arp_ignore = 1

# When an arp request is received on eth0, only respond if that address is
# configured on eth0. In particular, do not respond if the address is
# configured on lo

net.ipv4.conf.eth0.arp_ignore = 1

# Ditto for eth1, add for all ARPing interfaces
#net.ipv4.conf.eth1.arp_ignore = 1

# Enable configuration of arp_announce option
net.ipv4.conf.all.arp_announce = 2

# When making an ARP request sent through eth0 Always use an address that
# is configured on eth0 as the source address of the ARP request. If this
# is not set, and packets are being sent out eth0 for an address that is on
# lo, and an arp request is required, then the address on lo will be used.
# As the source IP address of arp requests is entered into the ARP cache on
# the destination, it has the effect of announcing this address. This is
# not desirable in this case as adresses on lo on the real-servers should
# be announced only by the linux-director.
net.ipv4.conf.eth0.arp_announce = 2

# Ditto for eth1, add for all ARPing interfaces
#net.ipv4.conf.eth1.arp_announce = 2

Then run this:

webserver1/webserver2:

sysctl -p

Add this section for the virtual IP address to /etc/network/interfaces:

webserver1/webserver2:

vi /etc/network/interfaces
auto lo:0
iface lo:0 inet static
 address 192.168.0.105
 netmask 255.255.255.255
 pre-up sysctl -p > /dev/null

Then run this:

Please Note after the following step you will probably get this error: SIOCSIFFLAGS: Cannot assign requested address

That is a normal bug and you can ignore it.

webserver1/webserver2:

ifup lo:0

If you change the IP at a later stage its recommended to do ifup lo:0 then ifdown lo:0 then again ifup lo:0

Finally we must create the file ldirector.html. This file is requested by the two load balancer nodes repeatedly so that they can see if the two Apache nodes are still running. I assume that the document root of the main apache web site on webserver1 and webserver2 is /var/www, therefore we create the file /var/www/ldirector.html:

webserver1/webserver2:

vi /var/www/ldirector.html
Test Page

14 Further Testing

You can now access the web site that is hosted by the two Apache nodes by typing http://192.168.0.105 in your browser.

Now stop the Apache on either webserver1 or webserver2. You should then still see the web site on http://192.168.0.105 because the load balancer directs requests to the working Apache node. Of course, if you stop both Apaches, then your request will fail.

Now let’s assume that loadb1 is our active load balancer, and loadb2 is the hot-standby. Now stop heartbeat on loadb1:

loadb1:

/etc/init.d/heartbeat stop

Wait a few seconds, and then try http://192.168.0.105 again in your browser. You should still see your web site because loadb2 has taken the active role now.

Now start heartbeat again on loadb1:

loadb1:

/etc/init.d/heartbeat start

loadb2 should still have the active role. Do the tests from chapter 5 again on loadb1 and loadb2, and you should see the inverse results as before.

If you have also passed these tests, then your loadbalanced Apache cluster is working as expected. Have fun!

15 Further Reading

This tutorial shows how to loadbalance two Apache nodes. It does not show how to keep the files in the Apache document root in sync or how to create a storage solution like an NFS server that both Apache nodes can use, nor does it provide a solution how to manage your MySQL database(s). You can find solutions for these issues here:

 

 

16 Links

 

17 References and Sources

GUI Installation for Ubuntu LAMP Server

Option1

If you are a new user and not familiar with command prompt you can install GUI for your ubuntu LAMP server for this you need to make sure you have enabled Universe and multiverse repositories in /etc/apt/sources.list file once you have enable you need to use the following command to install GUI

sudo apt-get update
sudo apt-get install ubuntu-desktop

The above command will install GNOME desktop if you want to install KDE desktop use the following command

sudo apt-get install kubuntu-desktop

Installing Webmin in Ubuntu Hardy Heron

Option2

Webmin is a web-based interface for system administration for Unix. Using any modern web browser, you can setup user accounts, Apache, DNS, file sharing and much more. Webmin removes the need to manually edit Unix configuration files like /etc/passwd, and lets you manage a system from the console or remotely.

You can install webmin for your server web interface to configure apache,mysql servers.Now we will see how to install webmin in Ubuntu 8.04

Preparing your system

First you need to install the following packages

sudo apt-get install perl libnet-ssleay-perl openssl libauthen-pam-perl libpam-runtime libio-pty-perl libmd5-perl

Now download the latest webmin using the following command

wget http://prdownloads.sourceforge.net/webadmin/webmin_1.420_all.deb

Now we have webmin_1.420_all.deb package install this package using the following command

sudo dpkg -i webmin_1.420_all.deb

This will complete the installation.

Ubuntu in particular don’t allow logins by the root user by default. However, the user created at system installation time can use sudo to switch to root. Webmin will allow any user who has this sudo capability to login with full root privileges.

Now you need to open your web browser and enter the following

https://your-server-ip:10000/

Now you should see similar to the following Screen

After login if you want to configure Apache,Mysql server you need to click on Servers on your lefthand side you should many servers are ready to configure

This is very Easy to configure most of the servers and Enjoy your new Ubuntu Hardy Heron LAMP Server.