02 - Workbook - Teachers Edition

! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! !
0*,2F!"#%&,+>+G*#(!
!"#"$%&#%'()*+,#-*&.(
"#$%&#'(!
A+'BC++B!
!
8/-8D8/-E!
!
)#'(*+,!-./! 0$(1!234$1#5!/67-878/-8! 9(#5!"#$%&#'(!:!;124#,1(! <21&+'5!;$=2#>!?9@00<! ! ! !
Datacenter Solutions
Labs
Table of Contents
1. OpenVPN ................................................................................................................................... 1 1.1. Common configuration ...................................................................................................... 1 1.2. Bridged VPN .................................................................................................................... 7 1.3. Routing VPN .................................................................................................................. 11 2. Squid ...................................................................................................................................... 16 2.1. Preliminary Setup ........................................................................................................... 16 2.2. Squid as a caching proxy .................................................................................................. 16 2.3. Squid as a surrogate ........................................................................................................ 19 3. Mail Services ............................................................................................................................ 21 3.1. Local only mail delivery ................................................................................................... 21 3.2. Mail domains ................................................................................................................. 23 3.3. Virtual domains .............................................................................................................. 48 4. iSCSI ....................................................................................................................................... 60 4.1. Simple iSCSI ................................................................................................................... 60 4.2. Multipath iSCSI ............................................................................................................... 63 5. DRBD: Distributed Replicated Block Device .................................................................................... 67 5.1. Two-nodes ..................................................................................................................... 67 5.2. Three-nodes .................................................................................................................. 74 6. High Availability ........................................................................................................................ 79 6.1. High available website ..................................................................................................... 79 6.2. High available iSCSI-based SAN .......................................................................................... 85 7. SELinux .................................................................................................................................... 94 7.1. Enabling SELinux ............................................................................................................. 94 7.2. Working with booleans .................................................................................................... 95 7.3. Using audit2allow ........................................................................................................... 97 7.4. Creating a policy ............................................................................................................. 99 8. Quality Of Service .................................................................................................................... 105 8.1. Bandwidth fairness ........................................................................................................ 105 8.2. Traffic Shaping .............................................................................................................. 107 9. IPSec ..................................................................................................................................... 110 9.1. Host to host ................................................................................................................. 110 9.2. Network to network ...................................................................................................... 120 10. RADIUS ................................................................................................................................ 123 10.1. Setting up the RADIUS Server ........................................................................................ 123 10.2. Setting up dd-wrt ........................................................................................................ 126 10.3. Tests with the client ..................................................................................................... 127 11. Puppet ................................................................................................................................. 129 11.1. Preparing the DNS Server ............................................................................................. 129 11.2. Working with Puppetmaster .......................................................................................... 129
Page ii
Labs
1. OpenVPN
In this lab, you'll learn how to configure a VPN to allow clients to connect to a private network from the outside. The network topology will be the following:
For the lab testing purposes, you will have to work with the host-only and NAT vmware networks. Create three (clone) virtual machines and configure them to match (nearly) the topology: vpn-client will have two network cards (one not shown on the schema). The first network card will be bound to the host-only network whereas the second network card will be connected to the NAT'ed network. This second network card will be only used to install the required packages and deconnected/removed on purpose. vpn-server also needs two network cards: The first one will be connected to the host-only network and the second one will be connected to the NAT'ed network, which represent the remote private network you want to establish a VPN to. network-host will figure a random network host in the private network. It only needs one network card, connected to the NAT'ed network.
Note
This host is not mandatory and will be used for ping purposes only. If you need to spare resources, you can omit this virtual machine and use the host operating system itself: It has an IP address in the virtual NAT'ed network too. Find it using your host operating system tools. The switch and router roles will be actually undertaken by the host operating system and VMware services. You don't have to create that part by yourself. Configure VMware host-only network to be 172.17.2.0/24 and disable the DHCP service on that network too.
1.1. Common configuration

1.1.1. Server 1.1.1.1. Host setup
Q: A: Set the hostname to vpn-server and configure your host-only interface (first one, eth0 if you've followed the setup) with the 172.17.2.2 address. Also make sure that your second NIC gets its address via DHCP. The following answer is Debian-specific. Feel free to adapt it if you're using another distro.
Page 1
Labs
root@debian-master:~# echo "vpn-server" > /etc/hostname root@debian-master:~# /etc/init.d/hostname.sh root@debian-master:~# hostname vpn-server
Note
Your shell prompt will not be updated until you log off and in or start a new shell.
Edit /etc/network/interfaces as follows: [...] allow-hotplug eth0 iface eth0 inet static address 172.17.2.2 netmask 255.255.255.0 allow-hotplug eth1 iface eth1 inet dhcp
root@vpn-server:/home/supinfo# ifdown eth0 eth1 && ifup eth0 eth1 Q: A: Install the openvpn package. root@vpn-server:~# apt-get install openvpn
Note
This is Debian-specific. Feel free to adapt to your system if you're using another distro.
1.1.1.2. PKI Setup

In this section, you will use the easy-rsa tool to create a PKI with a certification authority and key/certificates for server and clients. OpenVPN ships easy-rsa in the /usr/share/doc/openvpn/examples/easyrsa/2.0 directory. Create a local copy of this tool to work with in /root/easy-rsa:
root@vpn-server:~# cp -R /usr/share/doc/openvpn/examples/easy-rsa/2.0 easy-rsa
Q:
Configure easy-rsa with the following values: Item Country Value US
Page 2
Labs
Item Province City Organization Email A: Edit the vars file as follows: export export export export export Q: A: KEY_COUNTRY="US" KEY_PROVINCE="UT" KEY_CITY="SaltLakeCity" KEY_ORG="Utopia" KEY_EMAIL="certmaster@utopia.net" Value UT SaltLakeCity Utopia certmaster@utopia.net
Build your CA with Utopia CA as the common name. Don't forget to initialize and the environment before.
root@vpn-server:~/easy-rsa# . vars NOTE: If you run ./clean-all, I will be doing a rm -rf on /root/easy-rsa/keys root@vpn-server:~/easy-rsa# ./clean-all root@vpn-server:~/easy-rsa# ./build-ca Generating a 1024 bit RSA private key .....++++++ ...............................++++++ writing new private key to 'ca.key' ----You are about to be asked to enter information that will be incorporated into your certificate request. What you are about to enter is what is called a Distinguished Name or a DN. There are quite a few fields but you can leave some blank For some fields there will be a default value, If you enter '.', the field will be left blank. ----Country Name (2 letter code) [US]: State or Province Name (full name) [UT]: Locality Name (eg, city) [SaltLakeCity]: Organization Name (eg, company) [Utopia]: Organizational Unit Name (eg, section) []: Common Name (eg, your name or your server's hostname) [Utopia CA]: Name []: Email Address [certmaster@utopia.net]:
Q: A:
Generate the Diffie-Hellman parameter root@vpn-server:~/easy-rsa# ./build-dh Create a certificate/key for your server, vpn-server.
Q:
Page 3
Labs
A:
root@vpn-server:~/easy-rsa# ./build-key-server vpn-server Generating a 1024 bit RSA private key ..........++++++ .....++++++ writing new private key to 'vpn-server.key' ----You are about to be asked to enter information that will be incorporated into your certificate request. What you are about to enter is what is called a Distinguished Name or a DN. There are quite a few fields but you can leave some blank For some fields there will be a default value, If you enter '.', the field will be left blank. ----Country Name (2 letter code) [US]: State or Province Name (full name) [UT]: Locality Name (eg, city) [SaltLakeCity]: Organization Name (eg, company) [Utopia]: Organizational Unit Name (eg, section) []: Common Name (eg, your name or your server's hostname) [vpn-server]: Name []: Email Address [certmaster@utopia.net]: Please enter the following 'extra' attributes to be sent with your certificate request A challenge password []: An optional company name []: Using configuration from /root/easy-rsa/openssl.cnf Check that the request matches the signature Signature ok The Subject's Distinguished Name is as follows countryName :PRINTABLE:'US' stateOrProvinceName :PRINTABLE:'UT' localityName :PRINTABLE:'SaltLakeCity' organizationName :PRINTABLE:'Utopia' commonName :PRINTABLE:'vpn-server' emailAddress :IA5STRING:'certmaster@utopia.net' Certificate is to be certified until Mar 14 12:47:54 2022 GMT (3650 days) Sign the certificate? [y/n]:y
1 out of 1 certificate requests certified, commit? [y/n]y Write out database with 1 new entries Data Base Updated
Q:
Create a key/certificate for your client, vpn-client.
Page 4
Labs
A:
root@vpn-server:~/easy-rsa# ./build-key vpn-client Generating a 1024 bit RSA private key ......++++++ ......++++++ writing new private key to 'vpn-client.key' ----You are about to be asked to enter information that will be incorporated into your certificate request. What you are about to enter is what is called a Distinguished Name or a DN. There are quite a few fields but you can leave some blank For some fields there will be a default value, If you enter '.', the field will be left blank. ----Country Name (2 letter code) [US]: State or Province Name (full name) [UT]: Locality Name (eg, city) [SaltLakeCity]: Organization Name (eg, company) [Utopia]: Organizational Unit Name (eg, section) []: Common Name (eg, your name or your server's hostname) [vpn-client]: Name []: Email Address [certmaster@utopia.net]: Please enter the following 'extra' attributes to be sent with your certificate request A challenge password []: An optional company name []: Using configuration from /root/easy-rsa/openssl.cnf Check that the request matches the signature Signature ok The Subject's Distinguished Name is as follows countryName :PRINTABLE:'US' stateOrProvinceName :PRINTABLE:'UT' localityName :PRINTABLE:'SaltLakeCity' organizationName :PRINTABLE:'Utopia' commonName :PRINTABLE:'vpn-client' emailAddress :IA5STRING:'certmaster@utopia.net' Certificate is to be certified until Mar 14 12:51:47 2022 GMT (3650 days) Sign the certificate? [y/n]:y
1 out of 1 certificate requests certified, commit? [y/n]y Write out database with 1 new entries Data Base Updated
Q: A: Q:
Copy the required the Diffie-Hellman parameter and all required keys and certificates to /etc/openvpn.
root@vpn-server:~/easy-rsa# cp keys/{dh1024.pem,ca.crt,vpn-server.crt,vpn-server.key} /etc/openvpn/
Take a snapshot of your virtual machine. This is not a mandatory step, but it'll help you later.
1.1.2. Client
In this part, you are going to prepare the second virtual machine to act as a vpn client through the host-only network. However, you need to install the openvpn package and thus to access the internet. To do so, connect the first network card to the host-only network and add a second one to the virtual machine and connect it to the NAT'ed network. The second card will be shutdown before connecting to the VPN.
1.1.2.1. Host setup
Page 5
Labs
Q: A: Set the hostname to vpn-client and configure your host-only interface (first one, eth0 if you've followed the setup) with the 172.17.2.23 address. Also make sure that your second NIC gets its address via DHCP. The following answer is Debian-specific. Feel free to adapt it if you're using another distro. root@debian-master:~# echo "vpn-client" > /etc/hostname root@debian-master:~# /etc/init.d/hostname.sh root@debian-master:~# hostname vpn-client
Note
Your shell prompt will not be updated until you log off and in or start a new shell.
Edit /etc/network/interfaces as follows: [...] allow-hotplug eth0 iface eth0 inet static address 172.17.2.23 netmask 255.255.255.0 iface eth1 inet dhcp
root@vpn-client:/home/supinfo# ifdown eth0 eth1 && ifup eth0 eth1 Q: A: Install the openvpn package. root@vpn-client:~# apt-get install openvpn
Note
This is Debian-specific. Feel free to adapt to your system if you're using another distro.
Q:
Remove (or unplug) eth1 from your virtual machine.
1.1.2.2. Keys
Q: A: Use ssh to transfert all needed keys and files from the server to the client. Store these files in /etc/ openvpn.
root@vpn-client:~# scp root@172.17.2.2:/root/easy-rsa/keys/vpn-client.key /etc/openvpn/ root@vpn-client:~# scp root@172.17.2.2:/root/easy-rsa/keys/vpn-client.crt /etc/openvpn/ root@vpn-client:~# scp root@172.17.2.2:/root/easy-rsa/keys/ca.crt /etc/openvpn/
Page 6
Labs
Q: Take a snapshot of your virtual machine. This is not a mandatory step, but it'll help you later.
1.2. Bridged VPN

1.2.1. Server configuration
In this section, you're going to create a bridged VPN. In such a setup, the VPN service will listen on the WAN interface (the 172.17.2.0/24 network in this lab) and bridge incoming traffic with the NAT'ed networtk. The NAT'ed network address depends on your environment. Use the ifconfig eth1 command on your server to get its address. In my test environment, the address is 192.168.82.155/24. I'll be using this address all along the following answers. Please adapt to your setup. Outgoing traffic from the VPN server will go through a special TAP device, that you have to create. The bridging will also be done manually to bridge the tap device and the NIC connected to the NAT'ed network. This means all traffic coming in and out from the VPN will be indistinguishable from the 'real' LAN traffic. Therefore, you can either choose to let the existing LAN DHCP server distribute network parameters, or have OpenVPN managing its own IP pool. In a real setup if you choose the later option, be sure to configure the LAN DHCP to exclude these VPN-reserved IP adresses from its allocation pool.
1.2.1.1. Creating the bridge

Q: A: Q: A: Q: A: Use the openvpn binary to create a tap0 interface. root@vpn-server:~# openvpn --mktun --dev tap0 Create a new br0 bridge root@vpn-server:~# brctl addbr br0 Get current eth1 network settings and write them down.
root@vpn-server:~# ifconfig eth1 eth1 Link encap:Ethernet HWaddr 00:0c:29:0f:94:c0 inet addr:192.168.82.155 Bcast:192.168.82.255
Mask:255.255.255.0
Q: A:
Configure tap0 and eth1 with the 0.0.0.0 address and put them up and in promiscious mode. If you don't know how to put an interface into promiscious mode, search the ifconfig manpage for "prom". root@vpn-server:~# ifconfig tap0 0.0.0.0 up promisc root@vpn-server:~# ifconfig eth1 0.0.0.0 up promisc Add these two interfaces to the bridge root@vpn-server:~# brctl addif br0 tap0 root@vpn-server:~# brctl addif br0 eth1 Configure the br0 bridge with the network settings previously set on eth1. root@vpn-server:~# ifconfig br0 192.168.82.155 netmask 255.255.255.0
Q: A:
Q: A:
Page 7
Labs
1.2.1.2. Service configuration
Q: A: Use the server.conf.gz example config file that ships with OpenVPN to create /etc/openvpn/ server.conf.
root@vpn-server:~/easy-rsa# zcat \ >/usr/share/doc/openvpn/examples/sample-config-files/server.conf.gz > /etc/openvpn/server.conf
Q:
Configure OpenVPN to use the keys you've copied previously, and enforce the following requirements: Listen on the standard UDP port, only on the interface connected to the 172.17.2.0 network Act as a bridged VPN, distributing adresses from .10 to .20 in the NAT'ed network range. Push the NAT'ed network default gateway Run as nobody/nogroup
A:
Edit /etc/openvpn/server.conf as follows: local 172.17.2.2 port 1194 proto udp dev tap0 ca ca.crt cert vpn-server.crt key vpn-server.key dh dh1024.pem server-bridge 192.168.82.2 255.255.255.0 192.168.82.10 192.168.82.20 push "route 192.168.82.0 255.255.255.0"
Warning
Be sure to use "tap0" and not just "tap" as the tap device. What's the difference? "tap0" is the device you've actually created and bridged with eth1. "tap" is a random tap device, there is no guarantee that OpenVPN will use the tap0 you've created.
Q: A:
Start the service. root@vpn-server:/etc/openvpn# /etc/init.d/openvpn start Starting virtual private network daemon: server.
1.2.2. Client configuration 1.2.2.1. Service configuration

Q: Use the client.conf example file that ships with OpenVPN to create the /etc/openvpn/ client.conf file.
Page 8
Labs
A: Q: A:
root@vpn-client:~# cp /usr/share/doc/openvpn/examples/sample-config-files/client.conf /etc/openvpn/
Edit the service configuration to use the right keys and match your server's configuration. Also run the service as nobody/nogroup. Edit /etc/openvpn/openvpn.conf as follows: client dev tap proto udp remote 172.17.2.2 1194 user nobody group nogroup ca ca.crt cert vpn-client.crt key vpn-client.key
Note
On the client side, there is no need to specify a given tapX device. Just use "tap" that will let OpenVPN create the tap device as needed.
Q: A:
Start the service root@vpn-client:~# /etc/init.d/openvpn start Starting virtual private network daemon: client.
1.2.3. Routing
Q: A: On the client, show your routing table. Try to ping the server (using its NAT'ed network address) and your host operating system if you know its address on the NAT'ed network (usually .1).
supinfo@vpn-client:~$ /sbin/route Kernel IP routing table Destination Gateway Genmask Flags Metric Ref 192.168.82.0 * 255.255.255.0 U 0 0 172.17.2.0 * 255.255.255.0 U 0 0 supinfo@vpn-client:~$ ping -c 1 192.168.82.155 64 bytes from 192.168.82.155: icmp_req=1 ttl=64 time=0.841 ms supinfo@vpn-client:~$ ping -c 1 192.168.82.1 64 bytes from 192.168.82.1: icmp_req=1 ttl=64 time=3.27 ms
Use Iface 0 tap0 0 eth0
Q: A:
Set your DNS server to 8.8.8.8 root@vpn-client:~# echo "nameserver 8.8.8.8" > /etc/resolv.conf
Page 9
Labs
Q: A: Try to ping google. Does it works? Why? supinfo@vpn-client:~$ ping google.fr ping: unknown host google.fr It doesn't work, because the 8.8.8.8 DNS is unreachable. It can't be reached because there is no route to it nor default route in the routing table. As you've configured the server to push its default route, you could expect that route to be applied on the client. Moreover, this is the expected behavior. However, OpenVPN seems to have a bug in this part: The route is pushed and parsed on the client but not applied. A patch as been sent to the OpenVPN team to fix this. Q: A: Fix the problem.
root@vpn-client:/home/supinfo# route add default gw 192.168.82.2 root@vpn-client:/home/supinfo# ping -c1 google.fr PING google.fr (74.125.230.248) 56(84) bytes of data. 64 bytes from par08s10-in-f24.1e100.net (74.125.230.248): icmp_req=1 ttl=128 time=22.4 ms
1.2.4. Disabling automatic IP distribution

Q: A: On the server, edit your configuration to disable IP distribution, and restart the service. Edit /etc/openvpn/server.conf as follows(change server-bridge directive): #server-bridge 192.168.82.2 255.255.255.0 192.168.82.10 192.168.82.20 server-bridge And restart the service root@vpn-server:~# /etc/init.d/openvpn restart Q: A: Q: A: Restart the service on the client as well. root@vpn-client:~# /etc/init.d/openvpn restart On the client, show your routing table. What's missing? Why? Show tap0 details to confirm.
root@vpn-client:~# route Kernel IP routing table Destination Gateway Genmask Flags Metric Ref 172.17.2.0 * 255.255.255.0 U 0 0 root@vpn-client:~# ifconfig tap0 tap0 Link encap:Ethernet HWaddr 8e:8a:0b:06:67:93 BROADCAST MULTICAST MTU:1500 Metric:1
Use Iface 0 eth0
There is no route to the NAT'ed network you're bridged to, because OpenVPN doesn't give addesses itself anymore. The situation is exactly the same as if had just plugged a network cable in a switch. Q: A: Query the DHCP to get an address on tap0. root@vpn-client:~# dhclient tap0
Page 10
Labs
Q: A: Show your routing table and try to ping google.
supinfo@vpn-client:~$ /sbin/route Kernel IP routing table Destination Gateway Genmask Flags Metric Ref Use Iface 192.168.82.0 * 255.255.255.0 U 0 0 0 tap0 172.17.2.0 * 255.255.255.0 U 0 0 0 eth0 default 192.168.82.2 0.0.0.0 UG 0 0 0 tap0 supinfo@vpn-client:~$ ping -c 1 google.fr 64 bytes from fa-in-f94.1e100.net (173.194.70.94): icmp_req=1 ttl=128 time=37.3 ms
1.3. Routing VPN

In this section, you're going to experiment with the second - and most used - OpenVPN operation mode: routing. Before going any further, make sure to reset your configuration. To do so you can either (on both client and server): Restore from the snapshot you've taken right after the "Common" section Start from fresh virtual machines and redo the "Common" section steps Manually undo what you've done in the brigded setup (bridge, routes, configuration, ...)
1.3.1. Server configuration 1.3.1.1. Service setup

In this section, you're going to configure the OpenVPN service to: Create a Vitrual Private Network with a dedicated network address Accept connections The purpose of this VPN is to accept connections from "foreign" clients coming from the 172.17.2.0/24 network and give them access to the VMware NAT'ed network which is connected to your server on the second network card. The NAT'ed network is managed by VMware and has a different address on any machine. Use the ifconfig eth1 command to get the NAT'ed network address. In the test environement where this lab is writtent, the VMware NAT'ed network is 192.168.82.0/24. The VPN also need a network address of its own. Please adhere to the following convention: If NAT'ed network address is x.y.z.0/24, then the VPN network address will be x.y.z-1.0/24. Having a 192.168.82.0/24 NAT'ed network address, the VPN network address will be 192.168.81.0/24 Q: A: Use the server.conf.gz example config file that ships with OpenVPN to create /etc/openvpn/ server.conf.
root@vpn-server:~/easy-rsa# zcat \ >/usr/share/doc/openvpn/examples/sample-config-files/server.conf.gz > /etc/openvpn/server.conf
Q:
Configure OpenVPN to use the keys you've copied previously, and enforce the following requirements: Listen on the standard UDP port, only on the interface connected to the 172.17.2.0 network Act as a routing VPN, using a dedicated network address computed as previously defined. Push the route to the NAT'ed network to VPN clients Run as nobody/nogroup
A:
Edit /etc/openvpn/server.conf as follows:
Page 11
Labs
local 172.17.2.2 port 1194 proto udp dev tun ca ca.crt cert vpn-server.crt key vpn-server.key dh dh1024.pem server 192.168.81.0 255.255.255.0 push "route 192.168.82.0 255.255.255.0" Q: A: Start the service. root@vpn-server:/etc/openvpn# /etc/init.d/openvpn start Starting virtual private network daemon: server.
1.3.2. Client configuration 1.3.2.1. Service configuration

Q: A: Q: A: Use the client.conf example file that ships with OpenVPN to create the /etc/openvpn/ client.conf file.
root@vpn-client:~# cp /usr/share/doc/openvpn/examples/sample-config-files/client.conf /etc/openvpn/
Edit the service configuration to use the right keys and match your server's configuration. Also run the service as nobody/nogroup. Edit /etc/openvpn/openvpn.conf as follows: client dev tun proto udp remote 172.17.2.2 1194 user nobody group nogroup ca ca.crt cert vpn-client.crt key vpn-client.key
Q: A:
Start the service root@vpn-client:~# /etc/init.d/openvpn start Starting virtual private network daemon: client.
Page 12
Labs
1.3.2.2. Tests
Q: A: Check if a new network interface has been created.
root@vpn-client:~# ifconfig tun0 Link encap:UNSPEC HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00 inet addr:192.168.81.6 P-t-P:192.168.81.5 Mask:255.255.255.255 UP POINTOPOINT RUNNING NOARP MULTICAST MTU:1500 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:20 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:100 RX bytes:0 (0.0 B) TX bytes:2910 (2.8 KiB)
Q: A:
Show the current routing table (no dns resolution).

root@vpn-client:~# route -n Kernel IP routing table Destination Gateway 192.168.81.5 0.0.0.0 192.168.81.1 192.168.81.5 192.168.82.0 192.168.81.5 172.17.2.0 0.0.0.0
Genmask 255.255.255.255 255.255.255.255 255.255.255.0 255.255.255.0
Flags UH UGH UG U
Metric 0 0 0 0
Ref 0 0 0 0
Use 0 0 0 0
Iface tun0 tun0 tun0 eth0
Q:
Try to ping the vpn server on its 3 ip addresses: Host-only network (172.17.2.2) VPN (first address on the subnet: in this example 192.168.81.1) NAT'ed address (read it on the server)
A:
root@vpn-client:~# ping -c 1 172.17.2.2 PING 172.17.2.2 (172.17.2.2) 56(84) bytes of data. 64 bytes from 172.17.2.2: icmp_req=1 ttl=64 time=0.287 ms --- 172.17.2.2 ping statistics --1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.287/0.287/0.287/0.000 ms root@vpn-client:~# ping -c 1 192.168.81.1 PING 192.168.81.1 (192.168.81.1) 56(84) bytes of data. 64 bytes from 192.168.81.1: icmp_req=1 ttl=64 time=0.635 ms --- 192.168.81.1 ping statistics --1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.635/0.635/0.635/0.000 ms root@vpn-client:~# ping -c 1 192.168.82.155 PING 192.168.82.155 (192.168.82.155) 56(84) bytes of data. 64 bytes from 192.168.82.155: icmp_req=1 ttl=64 time=0.608 ms --- 192.168.82.155 ping statistics --1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.608/0.608/0.608/0.000 ms
Q:
Boot the third virtual machine while making sure that the virtual network card is connected to the NAT'ed network. Get it's IP address and try to ping it from the vpn client. Does it works? Why?
Page 13
Labs
A:
root@vpn-client:~# ping -c 1 192.168.82.148 PING 192.168.82.148 (192.168.82.148) 56(84) bytes of data. --- 192.168.82.148 ping statistics --1 packets transmitted, 0 received, 100% packet loss, time 0ms It doesn't work, because the VPN server hadn't been configured to route packets from a network to another.
1.3.3. Routing
Q: A: Configure the server to route packets between the VPN (172.17.2.0/24) and the VMware NAT'ed network. root@vpn-server:~# echo "1" > /proc/sys/net/ipv4/ip_forward
Note
You can also use sysctl. Please also note that this setting will be restored to default (0) upon reboot. To avoid that and keep routing enabled, you can edit /etc/sysctl.conf.
Q: A:
Try to ping the third virtual machine again from the vpn-client. Does it works? Why? supinfo@vpn-client:~$ ping -c 1 192.168.82.148 PING 192.168.82.148 (192.168.82.148) 56(84) bytes of data. --- 192.168.82.148 ping statistics --1 packets transmitted, 0 received, 100% packet loss, time 0ms It doesn't work, because of the network topology: The vpn client as a route to the 192.168.82.148 third virtual machine through the VPN tunnel, but the 192.168.82.148 third virtual machine as no route to respond back to the 192.168.81.6 vpn client. Therefore the third virtual machine will send the ping response to its default gateway (your host operating system) which doesn't have a route to 192.168.81.0/24 either. To make the VPN network and the LAN communicate, the LAN default gateway must be aware of the VPN and have a route to that VPN. In this case, you could configure your host operating system to route the packets correctly. However, there is a quick fix alternative: Manually adding a route from the third virtual machine to the VPN.
Q: A:
Implement a quick fix for the routing problem by adding a route from the third virtual machine to the VPN.
root@debian-master:~# route add -net 192.168.81.0 \ >netmask 255.255.255.0 gw 192.168.82.155
Q: A:
Try to ping the third virtual machine again from the vpn-client. supinfo@vpn-client:~$ ping 192.168.82.148 PING 192.168.82.148 (192.168.82.148) 56(84) bytes of data. 64 bytes from 192.168.82.148: icmp_req=1 ttl=63 time=6.44 ms 64 bytes from 192.168.82.148: icmp_req=2 ttl=63 time=1.26 ms
Page 14
Labs
1.3.4. Internet access through the VPN
Q: A: Q: A: Configure the vpn client to use the Google DNS (8.8.8.8) as its main (and only) DNS server. root@vpn-client:~# echo "nameserver 8.8.8.8" > /etc/resolv.conf Configure the VPN server to NAT outgoing VPN traffic. root@vpn-server:~# iptables -t nat -A POSTROUTING \ >-s 192.168.81.0/24 -o eth1 -j MASQUERADE
Note
As eth1 ip address is configured dynamically by a DHCP I'm using the MASQUERADE target. However a SNAT target is perfectly working & valid.
Note
As a side effet, all traffic coming from the VPN is now going to be NAT'ed regardless of its destination (to the internet or within the VMware virtual NAT'ed network). This means that now when a VPN client will try to communicate with another host located on the VMware NAT'ed virtual network, it will do it through the VPN server that will NAT it's query. The network host will not know about the VPN client: From the network host perspective, it is communicating with the VPN server itself. Therefore, there is no need for additional routing information between the VPN and the virtual network.
Q: A: Q: A:
Configure the VPN client to use the tunnel as its default route. root@vpn-client:~# route add default tun0 Try to ping Google.
supinfo@vpn-client:~$ ping google.com PING google.com (74.125.230.228) 56(84) bytes of data. 64 bytes from par08s10-in-f4.1e100.net (74.125.230.228): icmp_req=1 ttl=127 time=23.4 ms
Page 15
Labs
2. Squid
In this chapter, you're going to configure a squid server to act as a caching proxy and also as an accelerator. You'll need a virtual machine to run squid. This virtual machine will be connected to the NAT'ed virtual network. The client will be your host operating system browser.
2.1. Preliminary Setup

Q: A: Set your machine hostname to squid-server. root@debian-master:~# echo "squid-server" > /etc/hostname root@debian-master:~# /etc/init.d/hostname.sh Install the following packages: squid squidclient
Q:
Note
This is Debian-specific. If you're using another distro, you may have to install more or less packages. Refer to your documentation.
A:
root@squid-server:~# apt-get install squid squidclient Create a backup copy of /etc/squid/squid.conf and empty this file. root@squid-server:~# cp /etc/squid/squid.conf . root@squid-server:~# cat /dev/null > /etc/squid/squid.conf
Q: A:
2.2. Squid as a caching proxy

2.2.1. Base configuration
Q: Configure squid to use the following settings: Listen on port 3128 on all interfaces Store a 4GB cache using ufs in /var/cache/squid. Organize it using 16 top-level directories and 256 second level directories. Enable cache_object protocol for localhost (127.0.0.1/32) only. Enable ports 80 and 443
Page 16
Labs
Only tunnel HTTPS traffic Only accept requests from the NAT'ed virtual network. Proxy runs as proxy/proxy. Don't forget to set permissions on /var/cache/squid accordingly. A: Create directories as needed: root@squid-server:~# mkdir /var/cache/squid root@squid-server:~# chown proxy:proxy /var/cache/squid/ Edit your configuration as follows: http_port 3128 cache_dir ufs /var/cache/squid 4096 16 256 acl acl acl acl acl acl acl all src 0/0 cacheProto proto cache_object localhost src 127.0.0.1/32 authorizedPorts port 80 443 SSLPort port 443 connect method CONNECT myNetwork src 192.168.82.0/24 allow cacheProto localhost deny cacheProto deny !authorizedPorts deny connect !SSLPort allow myNetwork
http_access http_access http_access http_access http_access
http_access deny all Q: A: Test your configuration and (re)start the service root@squid-server:~# squid -k parse root@squid-server:~# /etc/init.d/squid restart Edit your host operating system browser settings to use your squid virtual machine as a proxy. You should be able to browse the internet.
Q:
2.2.2. Restrictions
Q: A: Prevent users from browsing linuxfr.org and slashdot.com during work days (Monday to Friday) between 9:00 and 18:00. Edit your configuration file as follows (and restart the service):
Page 17
Labs
[...] acl myNetwork src 192.168.82.0/24 acl worktime time MTWHF 09:00-18:00 acl timewaste dstdomain .linuxfr.org .slashdot.org http_access http_access http_access http_access http_access allow cacheProto localhost deny cacheProto deny !authorizedPorts deny connect !SSLPort deny !myNetwork
http_access allow timewaste !worktime http_access deny timewaste http_access allow myNetwork http_access deny all Q: A: You need to block more websites, like bash.org and xkcd.com. Configure squid to read domains to block from /etc/squid/bad-domains (create that file and fill it with blocked domains). Edit your configuration file to: acl timewaste dstdomain "/etc/squid/bad-domains" And fill /etc/squid/bad-domains with websites to block: .linuxfr.org .slashdot.org .bash.org .xkcd.com Q: A: Add an exception ACL to allow these otherwise blocked websites on friday afternoons (14h-18h). Edit your configuration as follows: acl funkytime time F 14:00-18:00 http_access allow timewaste !worktime http_access allow timewaste funkytime http_access deny timewaste
2.2.3. Authentication
Q: A: Configure Squid to require users authentication through PAM. Users shouldn't have to enter their credentials more than once during their typical work day (8 hours). Edit the configuration as follows(and restart the service):
Page 18
Labs
acl timewaste dstdomain "/etc/squid/bad-domains" auth_param basic program /usr/lib/squid/pam_auth auth_param basic realm "Squid Authentication" auth_param basic credentialsttl 8 hour acl knownUsers proxy_auth REQUIRED http_access deny !knownUsers http_access allow cacheProto localhost Q: A: Q: A: What's the main issue with such a setup? Security: System passwords are sent unencrypted through the HTTP communication. Configure Squid to use a NCSA-like password file instead of PAM. Create the /etc/squid/users.passwd using htpasswd(from the apache2 package) and fill it with some users in it for your tests. Create the password file: root@squid-server:~# htpasswd -c /etc/squid/users.passwd supinfo New password: Re-type new password: Adding password for user supinfo root@squid-server:~# htpasswd /etc/squid/users.passwd rick Configure and restart squid:
auth_param basic program /usr/lib/squid/ncsa_auth /etc/squid/users.passwd
2.3. Squid as a surrogate

In this section, you're going to make squid accelerate a single webserver. Clients will connect to squid thinking they are talking to the real webserver. In turn, Squid will either use its cache or query the real webserver to answer the client's request. Before going any further, be sure to stop any running instances of Squid and to empty the configuration you've done so far. Start with an empty /etc/squid/squid.conf file. Remove the proxy configuration you've done before from the web browser you've been using for tests. On your host operating system, create a static resolution entry to make it resolve www.supinfo.com to your virtual machine IP address. Q: Configure Squid to meet the following requirements: Listen on port 80 Forward all requests to www.supinfo.com, regardless of supplied URI or Host header. Use ACL's to prevent squid from talking to other servers than www.supinfo.com. A: Edit /etc/squid/squid.conf as follows(and restart the service):
Page 19
Labs
http_port 80 accel defaultsite=www.supinfo.com cache_peer www.supinfo.com parent 80 0 no-query originserver name=accelPeer acl all src 0/0 acl accelerated dstdomain www.supinfo.com http_access allow accelerated http_access deny all :q cache_peer_access accelPeer allow accelerated cache_peer_access accelPeer deny all
Q: A:
Test your setup from your host operating system (Be sure to resolve www.supinfo.com to the virtual machine).
samuel@chickamauga ~ $ ping www.supinfo.com PING www.supinfo.com (192.168.82.156) 56(84) bytes of data. 64 bytes from www.supinfo.com (192.168.82.156): icmp_req=1 ttl=64 time=0.354 ms 64 bytes from www.supinfo.com (192.168.82.156): icmp_req=2 ttl=64 time=0.283 ms samuel@chickamauga ~ $ curl http://www.supinfo.com/ <script src="http://www.google-analytics.com/urchin.js" type="text/javascript"> </script> <script type="text/javascript"> [...]
Page 20
Labs
3. Mail Services
3.1. Local only mail delivery
In this section, you're going to configure postfix to deliver mail directly to the local users. You'll need a stand alone virtual machine for this setup. Before going any further, make sure postfix is installed on your system (postfix package on Debian-based systems) and that no other smtp daemon is running on port 25. Q: A: Set the hostname to mangus. Ensure this setting will presist accross reboots. root@debian-master:~# echo "mangus" > /etc/hostname root@debian-master:~# /etc/init.d/hostname.sh Create the following users, make sure their hone directory is also created on the way. Give them random passwords as needed. bob sophie sarah john If your distro doesn't set bash as new users default shell, you might want to ensure this using -s switch (or enjoy sh fancy features). A: root@magnus:~# root@magnus:~# root@magnus:~# root@magnus:~# useradd useradd useradd useradd -m -m -m -m bob -s /bin/bash sophie -s /bin/bash sarah -s /bin/bash john -s /bin/bash
Q:
Q: A:
Configure postifx to deliver local mail only. It should allow bob to mail sophie from their respective terminals. Postfix should not listen on the network, only on the loopback interface. The /etc/postfix/main.cf should look like this:
smtpd_banner = $myhostname ESMTP $mail_name biff = no # appending .domain is the MUA's job. append_dot_mydomain = no # Uncomment the next line to generate "delayed mail" warnings #delay_warning_time = 4h inet_interfaces = loopback-only mynetworks_style = host default_transport = error: Local delivery only! alias_maps = hash:/etc/aliases mydestination = magnus, localhost, localhost.localdomain, magnus.localdomain
Page 21
Labs
Q: A: As supinfo, try to send john a mail. supinfo@magnus:~$ mail john@localhost Subject: Hi Buddy This is a sample mail . Cc: As john, check your mailbox to see if something has arrived for you. john@mangus:~$ mail Mail version 8.1.2 01/15/2001. Type ? for help. "/var/mail/john": 1 message 1 new >N 1 supinfo@mangus.lo Thu May 24 12:48 14/482 Hi Buddy & t 1 Message 1: From supinfo@mangus.localdomain Thu May 24 12:48:38 2012 X-Original-To: john@localhost To: john@localhost Subject: Hi Buddy Date: Thu, 24 May 2012 12:48:37 +0200 (CEST) From: supinfo@mangus.localdomain (supinfo) This is a sample mail & Or alternatively: john@mangus:~$ cat /var/mail/john From supinfo@mangus.localdomain Thu May 24 12:48:38 2012 Return-Path: <supinfo@mangus.localdomain> X-Original-To: john@localhost Delivered-To: john@localhost Received: by magnus.localdomain (Postfix, from userid 1000) id B0C6C53041; Thu, 24 May 2012 12:54:49 +0200 (CEST) To: john@localhost Subject: Hi Buddy Message-Id: <20120524105449.B0C6C53041@mangus.localdomain> Date: Thu, 24 May 2012 12:48:38 +0200 (CEST) From: supinfo@mangus.localdomain (supinfo) This is a sample mail
Q: A:
3.1.1. Aliases
Q: A: Add a new alias that resolves bobby to bob@localhost. Add the following to /etc/aliases: bobby: bob@localhost
Page 22
Labs
Q: A: Q: A: Rebuild the alias table root@magnus:~# newaliases Try to send a mail to bobby@localhost. root@magnus:~# mail bobby@localhost Subject: Test . Cc: Null message body; hope that's ok Check your mails as bob bob@mangus:~$ mail Mail version 8.1.2 01/15/2001. Type ? for help. "/var/mail/bob": 1 message 1 new >N 1 root@mangus Thu May 24 16:42 13/384 & t 1 Message 1: From root@mangus Thu May 24 16:42:39 2012 X-Original-To: bobby@localhost To: bobby@localhost Subject: Test Date: Thu, 24 May 2012 16:42:39 +0200 (CEST) From: root@mangus (root)
Q: A:
Test
3.2. Mail domains

In this section, you'll setup a mail server that receive (and sends) mail for a given domain. You'll need two virtual machines: Mail Server + DNS Workstation You can keep the "magnus" host you've setup in the previous part as the first virtual machine.
3.2.1. Setting up DNS

Q: Install a DNS server on magnus and create a utopia.net zone with the following records: A record the resolves to magnus MX record that annonces magnus as the mail host of utopia.net Three smtp, imap and pop3 CNAME records resolving to magnus. A: Add a zone declaration in Bind configuration: zone "utopia.net" { type master; file "/etc/bind/db.utopia.net"; };
Page 23
Labs
And fill the zone file: @ IN SOA utopia.net. root.utopia.net. ( 20120524 ; Serial 604800 ; Refresh 86400 ; Retry 2419200 ; Expire 604800 ) ; Negative Cache TTL ns mail.utopia.net. 192.168.82.156 192.168.82.156 192.168.82.156 magnus magnus magnus
; @ @ ns mail magnus imap pop3 smtp Q: A:
IN IN IN IN IN IN IN IN
NS MX 10 A A A CNAME CNAME CNAME
On the second virtual machine, set the first virtual machine as the DNS.
root@debian-master:~# echo "nameserver 192.168.82.156" > /etc/resolv.conf
3.2.2. Mail server configuration

Q: A: Create a "Maildir" directory in each of your users' home directory. This directory should belong to the user. Either create it as $USER or change ownership afterwards.
root@mangus:~# for u in bob sophie sarah john; do mkdir "/home/$u/Maildir"; \ >chown $u:$u "/home/$u/Maildir"; done;
Q: A:
Configure mangus to accept mail for utopia.net. Mail should be stored in the maildir format, in each user's $HOME/Maildir. Edit /etc/postfix/main.cf as follows:
Page 24
Labs
mydomain = utopia.net smtpd_banner = $myhostname ESMTP $mail_name biff = no # appending .domain is the MUA's job. append_dot_mydomain = no # Uncomment the next line to generate "delayed mail" warnings #delay_warning_time = 4h mydestination = magnus localhost $mydomain inet_interfaces = all mynetworks_style = host alias_maps = hash:/etc/aliases home_mailbox = Maildir/ Q: A: Enforce your modifications root@magnus:~# postfix reload Or alternatively: root@magnus:~# /etc/init.d/postfix restart Q: A: Install a maildir-aware mail client such as nail or heirloom-mailx on Debian-based systems. root@mangus:~# apt-get install heirloom-mailx
Note
This is Debian-specific.
Q: A:
As supinfo, point the mail environment variable to your maildir. supinfo@mangus:~$ export MAIL=/home/supinfo/Maildir
3.2.3. Configuring a client 3.2.3.1. Network setup

Q: You don't want anyone to modify /etc/resolv.conf: Kill any running dhcp client.
Page 25
Labs
A:
root@debian-master:~# pkill -9 dhc Configure the resolver to use the first virtual machine as the main (and single) DNS server.
root@debian-master:~# echo "nameserver 192.168.82.56" > /etc/resolv.conf
Q: A:
3.2.3.2. Configuring the local smtpd

When sending mail using the mail command, you need a local smtpd. Even if you don't want to receive anything and only send outgoing messages, you'll need that daemon. The Debian default install provides a running exim4 smtpd, but configured for local delivry only. You need to reconfigure it to be able to send mail to remote (not the local machine) domains. If you're using a Debian-based system, use the following command to enable external mail delivery: root@debian-master:~# dpkg-reconfigure exim4-config Just select internet site and accept all default values. The default configuration also prevents exim4 from sending mail to local addresses. Execute the following commands to remove this limitation:
root@debian-master:~# sed -i 36,38d /etc/exim4/conf.d/router/200_exim4-config_primary root@debian-master:~# sed -i 1040,1042d /etc/exim4/exim4.conf.template root@debian-master:~# /etc/init.d/exim4 restart
3.2.3.3. Sending mail to utopia.net

Q: A: Use the mail command to send a random mail to supinfo@utopia.net. root@debian-master:~# mail supinfo@utopia.net Subject: Hi from Workstation Everything is up and running . EOT Check your mails as supinfo (on the server)
Q:
Page 26
Labs
A:
supinfo@mangus:~$ mail Heirloom mailx version 12.4 7/29/08. Type ? for help. "/home/supinfo/Maildir": 1 message 1 new >N 1 supinfo Tue May 29 17:51 22/875 Hi from Workstation ? t 1 Message 1: From supinfo@debian-master.localdomain Tue May 29 17:51:29 2012 Return-Path: <supinfo@debian-master.localdomain> X-Original-To: supinfo@utopia.net Delivered-To: supinfo@utopia.net Date: Tue, 29 May 2012 17:51:37 +0200 To: supinfo@utopia.net Subject: Hi from Workstation User-Agent: Heirloom mailx 12.4 7/29/08 Content-Type: text/plain; charset=us-ascii From: supinfo <supinfo@debian-master.localdomain> Status: R Everything is up and running
3.2.4. Remote access 3.2.4.1. IMAP

Q: A: Install courier-imap on the server. root@mangus:~# apt-get install courier-imap
Note
This is Debian-specific
Q: A:
If you're running Debian, also install the libgamin0 package to avoid error messages from the imap server. root@mangus:~# apt-get install libgamin0 Make sure courier-imap is running. root@mangus:~# /etc/init.d/courier-imap restart Use telnet to test IMAP access from a virtual machine or directly from the host operating system: Just connect to the IMAP server (on port 143) and type-in IMAP commands.
Q: A:
Q:
Page 27
Labs
A:
samuel@chickamauga ~ $ telnet 192.168.82.156 143 Trying 192.168.82.156... Connected to 192.168.82.156. Escape character is '^]'. * OK [CAPABILITY IMAP4rev1 UIDPLUS CHILDREN NAMESPACE THREAD=ORDEREDSUBJECT THREAD=REFERENCES SORT QUOTA IDLE ACL ACL2=UNION] Courier-IMAP ready. Copyright 1998-2010 Double Precision, Inc. See COPYING for distribution information. 01 LOGIN supinfo supinfo * OK [ALERT] Filesystem notification initialization error -- contact your mail administrator (check for configuration errors with the FAM/Gamin library) 01 OK LOGIN Ok. 02 LIST "" * * LIST (\Unmarked \HasNoChildren) "." "INBOX" 02 OK LIST completed 03 SELECT INBOX * FLAGS (\Draft \Answered \Flagged \Deleted \Seen \Recent) * OK [PERMANENTFLAGS (\* \Draft \Answered \Flagged \Deleted \Seen)] Limited * 1 EXISTS * 1 RECENT * OK [UIDVALIDITY 1338302965] Ok * OK [MYRIGHTS "acdilrsw"] ACL 03 OK [READ-WRITE] Ok 04 FETCH 1 ALL * 1 FETCH (FLAGS (\Seen \Recent) INTERNALDATE "29-May-2012 17:51:29 +0200" RFC822.SIZE 895 ENVELOPE ("Tue, 29 May 2012 17:51:37 +0200" "Hi from Workstation" (("supinfo" NIL "supinfo" "debian-master.localdomain")) (("supinfo" NIL "supinfo" "debian-master.localdomain")) (("supinfo" NIL "supinfo" "debian-master.localdomain")) ((NIL NIL "supinfo" "utopia.net")) NIL NIL NIL "<E1SZOht-0003PK-Kl@debian-master.localdomain>")) 04 OK FETCH completed.
3.2.4.2. POP3
Q: A: Install courier-pop on the server. root@mangus:~# apt-get install courier-pop Make sure the pop3 service is running. root@mangus:~# /etc/init.d/courier-pop restart Use telnet (or any interactive multi-purpose clilent such as netcat) to test the pop3 service running on the server.
Q: A:
Q:
Page 28
Labs
A:
supinfo@mangus:~$ telnet localhost pop3 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. +OK Hello there. USER supinfo +OK Password required. PASS supinfo +OK logged in. LIST +OK POP3 clients that break here, they violate STD53. 1 895 . RETR 1 +OK 895 octets follow. Return-Path: <supinfo@debian-master.localdomain> X-Original-To: supinfo@utopia.net Delivered-To: supinfo@utopia.net Received: from debian-master.localdomain (unknown [192.168.82.142]) by mangus.utopia.net (Postfix) with ESMTP id 313BF530A8 for <supinfo@utopia.net>; Tue, 29 May 2012 17:51:29 +0200 (CEST) Received: from supinfo by debian-master.localdomain with local (Exim 4.72) (envelope-from <supinfo@debian-master.localdomain>) id 1SZOht-0003PK-Kl for supinfo@utopia.net; Tue, 29 May 2012 17:51:37 +0200 Message-Id: <E1SZOht-0003PK-Kl@debian-master.localdomain> Date: Tue, 29 May 2012 17:51:37 +0200 To: supinfo@utopia.net Subject: Hi from Workstation User-Agent: Heirloom mailx 12.4 7/29/08 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit From: supinfo <supinfo@debian-master.localdomain> Everything is up and running . ^] telnet> quit Connection closed.
3.2.4.3. SMTP Authentication

With the current mail server configuration, the mail server will accept mail for any user in the utopia.net domain. However, it will only relay mail from users on the local machine. It means that if you want to send a mail to foo@domain.com, you need to be "physically" logged on the server. The server will refuse to forward mail is the client isn't connected from the same machine. You can try this out from your second virtual machine:
Page 29
Labs
supinfo@debian-master:~$ telnet 192.168.82.156 smtp Trying 192.168.82.156... Connected to 192.168.82.156. Escape character is '^]'. 220 mangus.utopia.net ESMTP Postfix HELO workstation 250 mangus.utopia.net MAIL FROM: supinfo@utopia.net 250 2.1.0 Ok RCPT TO: 40793@supinfo.com 554 5.7.1 <40793@supinfo.com>: Relay access denied ^] telnet> quit Connection closed. This is the expected and desired behavior. If you want to allow "remote" (not logged on the server) users to send mail to other domains, you can: Accept to relay mail from a subset of network hosts Use user-based authentication Of course the second solution is far more flexible than the first one which is not really user-aware as it only deals with IP addresses. The first solution is implemented through the mynetworks directive. In this section, you'll setup a user-based authentication using cyrus-sasl. Q: A: Q: A: Install cyrus-sasl packages. On a standard Debian install you only need to install sasl2-bin root@mangus:/home/supinfo# apt-get install sasl2-bin Configure Cyrus sasl to use the saslauthd deamon as the password checking method for the smtpd application.
root@mangus:~# echo "pwcheck_method: saslauthd" > /etc/postfix/sasl/smtpd.conf
Note
This is Debian-specific. Debian's postfix has been patched to have this configuration file under / etc/postfix. Vanilla versions of Cyrus SASL search for application config files under the /etc/ sasl2/ directory. Q: Now you need to set the backend that cyrus-sasl's saslauthd is going to check credentials against. There are many popular options, such as pam or shadow. There is another interesting option you can use as a backend: rimap. This backend takes user-supplied credentials and try them against a given imap server. If the imap server grants the connection, the backend does so. In other words, the SMTP authentication is tied to the IMAP authentication mechanism. It provides a reliable way to manage authentication in only one place. Setup cyrus-sasl to use the rimap backend and to authenticate against the IMAP server you've configured earlier. You'll find all necessary information in the saslauthd man page.
Page 30
Labs
A: On a Debian system, modify the /etc/default/saslauthd file as follows:
# Should saslauthd run automatically on startup? (default: no) START=yes [...] # Example: MECHANISMS="pam" MECHANISMS="rimap" # Additional options for this mechanism. (default: none) # See the saslauthd man page for information about mech-specific options. MECH_OPTIONS="127.0.0.1" [...]
Q:
Postfix and saslauthd will communicate through a unix-domain socket. This socket defaultly lives in the / var/run/saslauthd directory. However if postfix runs in a chrooted environement, it won't find this directory. Make sure postfix is running and find out if it's running chrooted. First, check if the smtp daemon is configured to run chrooted: root@mangus:~# cat /etc/postfix/master.cf | grep ^smtp smtp inet n -
A:
smtpd
The fifth field controles whether the daemon will run chrooted or not. It shows a dash. It means to use the default that is 'yes', therefore postfix runs chrooted in the queue_directory directory. Get the queue_directory using postconf: root@mangus:~# postconf | grep queue_directory queue_directory = /var/spool/postfix The smtpd deamon runs chrooted in the /var/spool/postfix directory. Q: A: If postfix runs chrooted, create a var/run/saslauthd directory within the chroot. Ensure that the sasl group is the group owner of this directory. root@mangus:~# mkdir -p /var/spool/postfix/var/run/saslauthd root@mangus:~# chgrp sasl /var/spool/postfix/var/run/saslauthd The postfix daemons, which runs as postfix needs to go through this directory to open the socket. Make the postfix user a member of the sasl group. root@mangus:~# gpasswd -a postfix sasl Configure postfix to: Enable SMTP authentication Support pre-RFC or not RFC-abiding clients Allow mail relay from authenticated clients A: Add the following directives to /etc/postfix/main.cf:
Q: A:
Q:
Page 31
Labs
smtpd_sasl_auth_enable = yes broken_sasl_auth_clients = yes smtpd_recipient_restrictions = permit_mynetworks permit_sasl_authenticated reject_unauth_destination Q: A: Restart postfix and saslauthd services. root@mangus:~# /etc/init.d/postfix restart root@mangus:/home/supinfo# /etc/init.d/saslauthd restart Try to telnet to your mail server (from either the workstation or the server). Say Hi with a EHLO. What do you notice? supinfo@mangus:~$ telnet localhost smtp Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 220 mangus.utopia.net ESMTP Postfix EHLO workstation 250-mangus.utopia.net [...] 250-AUTH LOGIN CRAM-MD5 DIGEST-MD5 PLAIN NTLM 250-AUTH=LOGIN CRAM-MD5 DIGEST-MD5 PLAIN NTLM The server now announces that it supports authentication, and lists supported methods. It also announces it in both syntaxes, RFC and pre-RFC (equal sign). Q: Use telnet from the workstation to test the authentication process. You'll need to issue a AUTH command. This command takes a base64 encoded string holding the login and password in the following format: \0login\0password Use the base64 command to generate the encoded string. If you're on a network with restrictive policies on SMTP your mail might not reach the destination server. However the "Authentication successful" message will demonstrate that the feature is working and the mail been sent.
Q: A:
Page 32
Labs
A:
supinfo@debian-master:~$ printf "\0supinfo\0supinfo" | base64 AHN1cGluZm8Ac3VwaW5mbw== supinfo@debian-master:~$ telnet 192.168.82.156 smtp Trying 192.168.82.156... Connected to 192.168.82.156. Escape character is '^]'. 220 mangus.utopia.net ESMTP Postfix EHLO workstation 250-mangus.utopia.net 250-PIPELINING 250-SIZE 10240000 250-VRFY 250-ETRN 250-AUTH LOGIN CRAM-MD5 DIGEST-MD5 PLAIN NTLM 250-AUTH=LOGIN CRAM-MD5 DIGEST-MD5 PLAIN NTLM 250-ENHANCEDSTATUSCODES 250-8BITMIME 250 DSN AUTH PLAIN AHN1cGluZm8Ac3VwaW5mbw== 235 2.7.0 Authentication successful MAIL FROM: supinfo@utopia.net 250 2.1.0 Ok RCPT TO: 40793@supinfo.com 250 2.1.5 Ok DATA 354 End data with <CR><LF>.<CR><LF> Subject: Test mail Authenticated . 250 2.0.0 Ok: queued as 22603530DA ^] telnet> quit
3.2.5. Security
In the previous section, you've setup SMTP authentication to only allow authenticated remote users to have their mail relayed to other domains. This is mandatory if you don't want your server to be an open relay for spammers. However the authentication method sends clear-text passwords over the network: Base64 is an encoding system, not a cipher algorithm. In this section, you'll enable SSL/TLS support in postfix to provide secure SMTPS, IMAPS (respectively SMTP and IMAP over SSL) and POP3 over SSL communication channels.
3.2.5.1. Creating certificates

The first step is to create certificates. To create certificates, you need a certifiacation authority. in this section, you're going to create a new certificate authority and self-signed certificates. You can either use the CA.pl script that comes with openssl or directly the openssl command as you did in the "OpenSSL: Keys and certificates" chapter of the Edge Computing course. However the solution of all questions will use the warpper script only. Stick with what you're used to use. Q: Use the CA.pl script to create a new certification authority. If you don't know how to use CA.pl, issue a man CA.pl statement. If CA.pl is not in the PATH, use your distro's package management system to locate it.
Page 33
Labs
A:
root@mangus:~# /usr/lib/ssl/misc/CA.pl -newca CA certificate filename (or enter to create) Making CA certificate ... Generating a 1024 bit RSA private key .......................................................++++++ ........++++++ writing new private key to './demoCA/private/cakey.pem' Enter PEM pass phrase: Verifying - Enter PEM pass phrase: ----You are about to be asked to enter information that will be incorporated into your certificate request. What you are about to enter is what is called a Distinguished Name or a DN. There are quite a few fields but you can leave some blank For some fields there will be a default value, If you enter '.', the field will be left blank. ----Country Name (2 letter code) [AU]:US State or Province Name (full name) [Some-State]:Utah Locality Name (eg, city) []:Provo Organization Name (eg, company) [Internet Widgits Pty Ltd]:Utopia Organizational Unit Name (eg, section) []: Common Name (eg, YOUR name) []:ca.utopia.net Email Address []:
Q:
Create a certificate and key for smtp.utopia.net and sign it using the CA. Don't protect the key with a password. Remember that the only really important field is the CN: It should match the server's hostname, here smtp.utopia.net.
Page 34
Labs
A:
root@mangus:~# /usr/lib/ssl/misc/CA.pl -newreq-nodes Generating a 1024 bit RSA private key ....................++++++ .++++++ writing new private key to 'newkey.pem' ----You are about to be asked to enter information that will be incorporated into your certificate request. What you are about to enter is what is called a Distinguished Name or a DN. There are quite a few fields but you can leave some blank For some fields there will be a default value, If you enter '.', the field will be left blank. ----Country Name (2 letter code) [AU]:US State or Province Name (full name) [Some-State]:Utah Locality Name (eg, city) []:Provo Organization Name (eg, company) [Internet Widgits Pty Ltd]:Utopia Organizational Unit Name (eg, section) []: Common Name (eg, YOUR name) []:smtp.utopia.net Email Address []: Please enter the following 'extra' attributes to be sent with your certificate request A challenge password []: An optional company name []: Request is in newreq.pem, private key is in newkey.pem root@mangus:~# /usr/lib/ssl/misc/CA.pl -signreq Using configuration from /usr/lib/ssl/openssl.cnf Enter pass phrase for ./demoCA/private/cakey.pem: Check that the request matches the signature Signature ok Certificate Details: Serial Number: e1:d8:25:12:be:99:bc:ab Validity Not Before: May 31 15:09:17 2012 GMT Not After : May 31 15:09:17 2013 GMT Subject: countryName = US stateOrProvinceName = Utah localityName = Provo organizationName = Utopia commonName = smtp.utopia.net X509v3 extensions: X509v3 Basic Constraints: CA:FALSE Netscape Comment: OpenSSL Generated Certificate X509v3 Subject Key Identifier: B0:86:C3:6F:B5:28:C8:AF:E9:BD:2A:08:59:76:99:6B:B4:DB:67:1B X509v3 Authority Key Identifier: keyid:01:A9:62:BC:2B:5C:A9:8D:7C:80:9C:55:B9:1B:1C:B5:2E:92:27:16 Certificate is to be certified until May 31 15:09:17 2013 GMT (365 days) Sign the certificate? [y/n]:y
1 out of 1 certificate requests certified, commit? [y/n]y Write out database with 1 new entries Data Base Updated Signed certificate is in newcert.pem
Q:
Keep the newly created files newkey.pem and newcert.pem as smtpd-key.pem and smtpdcert.pem in your home directory (user's or root's, prefeably the latter). You'll use these files later.
Page 35
Labs
A:
root@mangus:~# mv newkey.pem smtpd-key.pem root@mangus:~# mv newcert.pem smtpd-cert.pem Go through the same steps to create a certificate (and key) for imap.utopia.net. Rename the resulting files to imapd-cert.pem and imapd-key.pem.
Q:
Page 36
Labs
A:
root@mangus:~# /usr/lib/ssl/misc/CA.pl -newreq-nodes Generating a 1024 bit RSA private key .......................++++++ ...................................++++++ writing new private key to 'newkey.pem' ----You are about to be asked to enter information that will be incorporated into your certificate request. What you are about to enter is what is called a Distinguished Name or a DN. There are quite a few fields but you can leave some blank For some fields there will be a default value, If you enter '.', the field will be left blank. ----Country Name (2 letter code) [AU]:US State or Province Name (full name) [Some-State]:Utah Locality Name (eg, city) []:Provo Organization Name (eg, company) [Internet Widgits Pty Ltd]:Utopia Organizational Unit Name (eg, section) []: Common Name (eg, YOUR name) []:imap.utopia.net Email Address []: Please enter the following 'extra' attributes to be sent with your certificate request A challenge password []: An optional company name []: Request is in newreq.pem, private key is in newkey.pem root@mangus:~# /usr/lib/ssl/misc/CA.pl -signreq Using configuration from /usr/lib/ssl/openssl.cnf Enter pass phrase for ./demoCA/private/cakey.pem: Check that the request matches the signature Signature ok Certificate Details: Serial Number: e1:d8:25:12:be:99:bc:ac Validity Not Before: May 31 15:19:55 2012 GMT Not After : May 31 15:19:55 2013 GMT Subject: countryName = US stateOrProvinceName = Utah localityName = Provo organizationName = Utopia commonName = imap.utopia.net X509v3 extensions: X509v3 Basic Constraints: CA:FALSE Netscape Comment: OpenSSL Generated Certificate X509v3 Subject Key Identifier: AB:D4:28:4A:D1:44:79:FE:91:94:29:0E:DB:D0:50:57:A4:CA:DC:E0 X509v3 Authority Key Identifier: keyid:01:A9:62:BC:2B:5C:A9:8D:7C:80:9C:55:B9:1B:1C:B5:2E:92:27:16 Certificate is to be certified until May 31 15:19:55 2013 GMT (365 days) Sign the certificate? [y/n]:y
1 out of 1 certificate requests certified, commit? [y/n]y Write out database with 1 new entries Data Base Updated Signed certificate is in newcert.pem root@mangus:~# mv newcert.pem imapd-cert.pem root@mangus:~# mv newkey.pem imapd-key.pem
Page 37
Labs
Q: Go through the same steps to create a certificate (and key) for pop3.utopia.net. Rename the resulting files to pop3d-cert.pem and pop3d-key.pem.
Page 38
Labs
A:
root@mangus:~# /usr/lib/ssl/misc/CA.pl -newreq-nodes Generating a 1024 bit RSA private key ..........++++++ .....................++++++ writing new private key to 'newkey.pem' ----You are about to be asked to enter information that will be incorporated into your certificate request. What you are about to enter is what is called a Distinguished Name or a DN. There are quite a few fields but you can leave some blank For some fields there will be a default value, If you enter '.', the field will be left blank. ----Country Name (2 letter code) [AU]:US State or Province Name (full name) [Some-State]:Utah Locality Name (eg, city) []:Provo Organization Name (eg, company) [Internet Widgits Pty Ltd]:Utopia Organizational Unit Name (eg, section) []: Common Name (eg, YOUR name) []:pop3.utopia.net Email Address []: Please enter the following 'extra' attributes to be sent with your certificate request A challenge password []: An optional company name []: Request is in newreq.pem, private key is in newkey.pem root@mangus:~# /usr/lib/ssl/misc/CA.pl -signreq Using configuration from /usr/lib/ssl/openssl.cnf Enter pass phrase for ./demoCA/private/cakey.pem: Check that the request matches the signature Signature ok Certificate Details: Serial Number: e1:d8:25:12:be:99:bc:ae Validity Not Before: May 31 18:37:31 2012 GMT Not After : May 31 18:37:31 2013 GMT Subject: countryName = US stateOrProvinceName = Utah localityName = Provo organizationName = Utopia commonName = pop3.utopia.net X509v3 extensions: X509v3 Basic Constraints: CA:FALSE Netscape Comment: OpenSSL Generated Certificate X509v3 Subject Key Identifier: 89:46:BF:81:EC:CF:08:AC:3A:7E:95:C2:39:53:41:E1:F3:07:17:C1 X509v3 Authority Key Identifier: keyid:01:A9:62:BC:2B:5C:A9:8D:7C:80:9C:55:B9:1B:1C:B5:2E:92:27:16 Certificate is to be certified until May 31 18:37:31 2013 GMT (365 days) Sign the certificate? [y/n]:y
1 out of 1 certificate requests certified, commit? [y/n]y Write out database with 1 new entries Data Base Updated Signed certificate is in newcert.pem root@mangus:~# mv newcert.pem pop3d-cert.pem root@mangus:~# mv newkey.pem pop3d-key.pem
Page 39
Labs
3.2.5.2. Setting up SMTPS
Q: A: Move the smtpd-key.pem file into the /etc/postfix directory. This key file contains sensitive information. It should only accessible by root. Adjust permissions accordingly.
root@mangus:~# mv newkey.pem /etc/postfix/smtpd-key.pem root@mangus:~# chmod 600 /etc/postfix/smtpd-key.pem
Q:
Postfix needs to have two certificates handy: The certificate holding the public key matching the private key to make ciphered communications and the certificate from the CA that has signed the server's certificate, especially if the CA is a private one (self-signed certificates). It prefers to have both in one file. Concatenate (in that order) the smtpd-cert.pem certificate and the CA certificate (./demoCA/cacert.pem with default settings) to create /etc/postfix/smtpd-cert.pem. The certificate file holds public information. However it should not be world-writable. Adjust permissions accordingly.
root@mangus:~# cat smtpd-cert.pem demoCA/cacert.pem > /etc/postfix/smtpd-cert.pem root@mangus:~# chmod 644 /etc/postfix/smtpd-cert.pem
A:
Q: A:
Configure Postfix to use the key and certificate you've just created. Enable SMTPS as an optional feature: The server should provide SMTPS upon client request, not force it. Add the following lines to /etc/postfix/main.cf: smtpd_tls_cert_file = /etc/postfix/smtpd-cert.pem smtpd_tls_key_file = /etc/postfix/smtpd-key.pem smtpd_tls_security_level = may
Q: A: Q: A:
Restart the service. root@mangus:~# /etc/init.d/postfix restart Use telnet to check the STARTTLS capability of your server. root@mangus:~# telnet localhost smtp Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 220 mangus.utopia.net ESMTP Postfix EHLO wrk 250-mangus.utopia.net 250-PIPELINING 250-SIZE 10240000 250-VRFY 250-ETRN 250-STARTTLS 250-AUTH LOGIN CRAM-MD5 DIGEST-MD5 PLAIN NTLM 250-AUTH=LOGIN CRAM-MD5 DIGEST-MD5 PLAIN NTLM 250-ENHANCEDSTATUSCODES 250-8BITMIME 250 DSN STARTTLS 220 2.0.0 Ready to start TLS
Page 40
Labs
3.2.5.3. Setting up IMAPS
If you're running a Debian-based system, you need to install the courier-imap-ssl package before going any further. Q: Courier's imapd supports SSL/TLS. However, it expects the key and the certificate to be in the same file. Concatenate imapd-key.pem and imapd-cert.pem to /etc/courier/imapd.pem. This file holds sensitive information. It should only accessible by root. Adjust permissions accordingly.
root@mangus:~# cat imapd-key.pem imapd-cert.pem > /etc/courier/imapd.pem root@mangus:~# chmod 600 /etc/courier/imapd.pem
A:
Q: A:
Edit courier-imap-ssl configuration file to ensure it uses that the right certificate file. root@mangus:~# grep ^TLS_CERTFILE /etc/courier/imapd-ssl TLS_CERTFILE=/etc/courier/imapd.pem Restart the courier-imap-ssl service. root@mangus:~# /etc/init.d/courier-imap-ssl restart Use the openssl s_client command to test the SSL connection to the imapd running on port 993. You use the -CAfile openssl switch to provide your CA certificate file.
Q: A:
Q:
Page 41
Labs
A:
root@mangus:~# openssl s_client -connect imap.utopia.net:993 \ >-CAfile /root/demoCA/cacert.pem CONNECTED(00000003) depth=1 /C=US/ST=Utah/O=Utopia/CN=ca.utopia.net verify return:1 depth=0 /C=US/ST=Utah/L=Provo/O=Utopia/CN=imap.utopia.net verify return:1 --Certificate chain 0 s:/C=US/ST=Utah/L=Provo/O=Utopia/CN=imap.utopia.net i:/C=US/ST=Utah/O=Utopia/CN=ca.utopia.net --Server certificate -----BEGIN CERTIFICATE----MIIClTCCAf6gAwIBAgIJAOHYJRK+mbysMA0GCSqGSIb3DQEBBQUAMEUxCzAJBgNV BAYTAlVTMQ0wCwYDVQQIEwRVdGFoMQ8wDQYDVQQKEwZVdG9waWExFjAUBgNVBAMT DWNhLnV0b3BpYS5uZXQwHhcNMTIwNTMxMTUxOTU1WhcNMTMwNTMxMTUxOTU1WjBX MQswCQYDVQQGEwJVUzENMAsGA1UECBMEVXRhaDEOMAwGA1UEBxMFUHJvdm8xDzAN BgNVBAoTBlV0b3BpYTEYMBYGA1UEAxMPaW1hcC51dG9waWEubmV0MIGfMA0GCSqG SIb3DQEBAQUAA4GNADCBiQKBgQCyG9VZCGyL5+5TYpPOYCm6dgbYVCufCOACrTUC au2QDXuQVN201qniiNI4tX+Mvo7fkgtJtUv6wbpdNlK++Okk0z12Fa2AiZfqiEVm 2kRiMt7/rhYQY/CRnn6uC8Z7sOBshclr4c3d41gE5LzbYErhsZD6K2pfnGWvRiby eOSS+wIDAQABo3sweTAJBgNVHRMEAjAAMCwGCWCGSAGG+EIBDQQfFh1PcGVuU1NM IEdlbmVyYXRlZCBDZXJ0aWZpY2F0ZTAdBgNVHQ4EFgQUq9QoStFEef6RlCkO29BQ V6TK3OAwHwYDVR0jBBgwFoAUAalivCtcqY18gJxVuRsctS6SJxYwDQYJKoZIhvcN AQEFBQADgYEAJgtF9XHVcbFS+QJ2Rm9oxSdE1yXg4vjYPWE0EvjkHFLZdmPK7V1D MbKhP84rNDpu+dxG+xgjP2fCPNhWgZwnaVIFYbBjrar743uX8QVeYtMfF99KxnKU lBk8hL/aMov4Rr/grBvzIZ5AyTVFTh6syeZKN/r70DDly1fvErtjWbA= -----END CERTIFICATE----subject=/C=US/ST=Utah/L=Provo/O=Utopia/CN=imap.utopia.net issuer=/C=US/ST=Utah/O=Utopia/CN=ca.utopia.net --No client certificate CA names sent --SSL handshake has read 834 bytes and written 319 bytes --New, TLSv1/SSLv3, Cipher is AES256-SHA Server public key is 1024 bit Secure Renegotiation IS supported Compression: NONE Expansion: NONE SSL-Session: Protocol : TLSv1 Cipher : AES256-SHA Session-ID: D5295F6E30A195A52F56FBB1F381A472251B4D336755967509214F5BEE85ACB8 Session-ID-ctx: Master-Key: C6FF20D5A2A212D9C1134A40617F20E91D675F2D9E2D290ADEE6C1F77B6C4869C53E882E7770D2102CB1FCAFA0253BDA Key-Arg : None Start Time: 1338490087 Timeout : 300 (sec) Verify return code: 0 (ok) --* OK [CAPABILITY IMAP4rev1 UIDPLUS CHILDREN NAMESPACE THREAD=ORDEREDSUBJECT THREAD=REFERENCES [...] 01 LOGIN supinfo supinfo 01 OK LOGIN Ok. 02 LIST "" * * LIST (\HasNoChildren) "." "INBOX.Trash" * LIST (\HasNoChildren) "." "INBOX.Drafts" * LIST (\Marked \HasChildren) "." "INBOX" 02 OK LIST completed 03 SELECT INBOX * FLAGS (\Draft \Answered \Flagged \Deleted \Seen \Recent) * OK [PERMANENTFLAGS (\* \Draft \Answered \Flagged \Deleted \Seen)] Limited * 2 EXISTS * 0 RECENT * OK [UIDVALIDITY 1338302965] Ok * OK [MYRIGHTS "acdilrsw"] ACL 03 OK [READ-WRITE] Ok 04 FETCH 1 ALL * 1 FETCH (FLAGS (\Seen) INTERNALDATE "29-May-2012 17:51:29 +0200" RFC822.SIZE 895 ENVELOPE ("Tue, 29 May 2012 17:51:37 +0200" "Hi from Workstation" (("supinfo" NIL "supinfo" "debian-master.localdomain")) (("supinfo" NIL "supinfo" "debian-master.localdomain")) (("supinfo" NIL "supinfo" "debian-master.localdomain")) ((NIL NIL "supinfo" "utopia.net")) NIL NIL NIL "<E1SZOht-0003PK-Kl@debian-master.localdomain>")) 04 OK FETCH completed. 05 LOGOUT * BYE Courier-IMAP server shutting down 05 OK LOGOUT completed closed
3.2.5.4. Setting up POP3 over SSL

If you're running a Debian-based system, you need to install the courier-pop-ssl package before going any further.
Page 42
Labs
Q: Courier's pop3d supports SSL/TLS. However, it expects the key and the certificate to be in the same file. Concatenate pop3d-key.pem and pop3d-cert.pem to /etc/courier/pop3d.pem. This file holds sensitive information. It should only accessible by root. Adjust permissions accordingly.
root@mangus:~# cat pop3d-key.pem pop3d-cert.pem > /etc/courier/pop3d.pem root@mangus:~# chmod 600 /etc/courier/pop3d.pem
A:
Q: A:
Edit courier-pop-ssl configuration file to ensure it uses that the right certificate file. root@mangus:~# grep ^TLS_CERTFILE /etc/courier/pop3d-ssl TLS_CERTFILE=/etc/courier/pop3d.pem Restart the courier-imap-ssl service. root@mangus:~# /etc/init.d/courier-pop-ssl restart Use the openssl s_client command to test the SSL connection to the popd running on port 995. You use the -CAfile openssl switch to provide your CA certificate file.
Q: A:
Q:
Note
Don't use the RETR POP3 command in capitals: openssl s_client will understand this first capital R as a renogiate command. Either use retr in lowercase or put a space before RETR. For more information, look at the "CONNECTED COMMANDS" section of s_client man page.
Page 43
Labs
A:
root@mangus:~# openssl s_client -connect pop3.utopia.net:995 \ >-CAfile /root/demoCA/cacert.pem CONNECTED(00000003) depth=1 /C=US/ST=Utah/O=Utopia/CN=ca.utopia.net verify return:1 depth=0 /C=US/ST=Utah/L=Provo/O=Utopia/CN=pop3.utopia.net verify return:1 --Certificate chain 0 s:/C=US/ST=Utah/L=Provo/O=Utopia/CN=pop3.utopia.net i:/C=US/ST=Utah/O=Utopia/CN=ca.utopia.net --Server certificate -----BEGIN CERTIFICATE----MIIClTCCAf6gAwIBAgIJAOHYJRK+mbyuMA0GCSqGSIb3DQEBBQUAMEUxCzAJBgNV BAYTAlVTMQ0wCwYDVQQIEwRVdGFoMQ8wDQYDVQQKEwZVdG9waWExFjAUBgNVBAMT DWNhLnV0b3BpYS5uZXQwHhcNMTIwNTMxMTgzNzMxWhcNMTMwNTMxMTgzNzMxWjBX MQswCQYDVQQGEwJVUzENMAsGA1UECBMEVXRhaDEOMAwGA1UEBxMFUHJvdm8xDzAN BgNVBAoTBlV0b3BpYTEYMBYGA1UEAxMPcG9wMy51dG9waWEubmV0MIGfMA0GCSqG SIb3DQEBAQUAA4GNADCBiQKBgQCtERDr0IKAnnvmO8cm4lH2MYkFipe3h0QTDfyj dXyatuDYGCi0LYAZTpSn8WwtOQnldH8XT0hhjE2uHND1xznGK6hNMoNPA/ysrPIe TNkvNfzvT2t0iCqteLftKaozvxJ1bCEJGrIjyTbw2X7nX452kCL0S51N1iVKRN9O +x73aQIDAQABo3sweTAJBgNVHRMEAjAAMCwGCWCGSAGG+EIBDQQfFh1PcGVuU1NM IEdlbmVyYXRlZCBDZXJ0aWZpY2F0ZTAdBgNVHQ4EFgQUiUa/gezPCKw6fpXCOVNB 4fMHF8EwHwYDVR0jBBgwFoAUAalivCtcqY18gJxVuRsctS6SJxYwDQYJKoZIhvcN AQEFBQADgYEADDCJFnpv0dQvjUqzu8mhXJ+Rpw4EG6qtgHjy+RAs+r6f7gi+rOMp 8rB2+6VkZbyN1DwGIhlisGvA0qVx00kt+su834I1085jpyTknSFqYv6mdxoQtvKm R3BPF7gpKk0YDqsIlxLDOn1Vo0i0SwMNMA+8Rgubmj3RyZSy0tauAzk= -----END CERTIFICATE----subject=/C=US/ST=Utah/L=Provo/O=Utopia/CN=pop3.utopia.net issuer=/C=US/ST=Utah/O=Utopia/CN=ca.utopia.net --No client certificate CA names sent --SSL handshake has read 834 bytes and written 319 bytes --New, TLSv1/SSLv3, Cipher is AES256-SHA Server public key is 1024 bit Secure Renegotiation IS supported Compression: NONE Expansion: NONE SSL-Session: Protocol : TLSv1 Cipher : AES256-SHA Session-ID: 7F0CB5AFC7977A25057C6E3D0E1978D5E074500D289D37CCCCDF02A9524A124E Session-ID-ctx: Master-Key: C34D4789EA167F63D425C19365B180DD3568D48DE17A9443CDBBFE545B72E382AEBD08979887D412DC68CC617628F953 Key-Arg : None Start Time: 1338489795 Timeout : 300 (sec) Verify return code: 0 (ok) --+OK Hello there. USER supinfo +OK Password required. PASS supinfo +OK logged in. LIST +OK POP3 clients that break here, they violate STD53. 1 895 2 673 . retr 1 +OK 895 octets follow. Return-Path: <supinfo@debian-master.localdomain> X-Original-To: supinfo@utopia.net Delivered-To: supinfo@utopia.net Received: from debian-master.localdomain (unknown [192.168.82.142]) by mangus.utopia.net (Postfix) with ESMTP id 313BF530A8 for <supinfo@utopia.net>; Tue, 29 May 2012 17:51:29 +0200 (CEST) Received: from supinfo by debian-master.localdomain with local (Exim 4.72) (envelope-from <supinfo@debian-master.localdomain>) id 1SZOht-0003PK-Kl for supinfo@utopia.net; Tue, 29 May 2012 17:51:37 +0200 Message-Id: <E1SZOht-0003PK-Kl@debian-master.localdomain> Date: Tue, 29 May 2012 17:51:37 +0200 To: supinfo@utopia.net Subject: Hi from Workstation User-Agent: Heirloom mailx 12.4 7/29/08 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit From: supinfo <supinfo@debian-master.localdomain> Everything is up and running . quit +OK Bye-bye. closed
Page 44
Labs
3.2.6. Webmail
In the previous sections, you've configured a SMTP server with standard remote access protocols such as IMAP and POP3. You've lately added security on top of these protocols by enableing their "over SSL" counterparts. In this section, you're going to install a Webmail to let your users work with their email directly from the web, without a dedicated email client. The Webmail is no more than a "regular" mail client: It makes uses of the standard POP3 and IMAP protocols. Therefore, it can run on any machine that can communicate with the mail server. In this lab, you're going to install it on the same machine as the mail server to minimize the need of additional virtual machines. However, if you really want to test operations on a separate machine, feel free to do so. Just adapt IP adresses to your specific scenario. Q: A: Q: Install the squirrelmail package. root@mangus:~# apt-get install squirrelmail Configure Apache2 to serve content from /usr/share/squirrelmail. You can either directly include the /etc/squirrelmail/apache.conf file shipped with squirrelmail in apache configuration or create a virtual host on your own. Don't forget to restart the service to apply your modifications.
root@mangus:~# echo "Include /etc/squirrelmail/apache.conf" >> /etc/apache2/apache2.conf root@mangus:~# /etc/init.d/apache2 restart
A:
Q: A:
Run the squirrelmail-configure command and adjust the settings to your setup. Be sure to set the domain to utopia.net in the "Server Settings" section. root@mangus:~# squirrelmail-configure SquirrelMail Configuration : Read: config.php (1.4.0) --------------------------------------------------------Main Menu -1. Organization Preferences 2. Server Settings 3. Folder Defaults 4. General Options 5. Themes 6. Address Books 7. Message of the Day (MOTD) 8. Plugins 9. Database 10. Languages D. C S Q Set pre-defined settings for specific IMAP servers Turn color on Save data Quit
Command >> D SquirrelMail Configuration : Read: config.php --------------------------------------------------------While we have been building SquirrelMail, we have discovered some preferences that work better with some servers that don't work so well with others. If you select your IMAP server, this option will set some pre-defined settings for that server.
Page 45
Labs
Please note that you will still need to go through and make sure everything is correct. This does not change everything. There are only a few settings that this will change. Please select your IMAP server: bincimap = Binc IMAP server courier = Courier IMAP server cyrus = Cyrus IMAP server dovecot = Dovecot Secure IMAP server exchange = Microsoft Exchange IMAP server hmailserver = hMailServer macosx = Mac OS X Mailserver mercury32 = Mercury/32 uw = University of Washington's IMAP server gmail = IMAP access to Google mail (Gmail) accounts quit = Do not change anything Command >> courier imap_server_type default_folder_prefix trash_folder sent_folder draft_folder show_prefix_option default_sub_of_inbox show_contain_subfolders_option optional_delimiter delete_folder Press any key to continue... SquirrelMail Configuration : Read: config.php (1.4.0) --------------------------------------------------------Main Menu -1. Organization Preferences 2. Server Settings 3. Folder Defaults 4. General Options 5. Themes 6. Address Books 7. Message of the Day (MOTD) 8. Plugins 9. Database 10. Languages D. C S Q Set pre-defined settings for specific IMAP servers Turn color on Save data Quit = = = = = = = = = = courier INBOX. Trash Sent Drafts false false false . true
Command >> 2 SquirrelMail Configuration : Read: config.php (1.4.0)
Page 46
Labs
--------------------------------------------------------Server Settings General ------1. Domain 2. Invert Time 3. Sendmail or SMTP A. B. R C S Q Update IMAP Settings Update SMTP Settings Return to Main Menu Turn color on Save data Quit
: trim(implode('', file('/etc/'.(file_exists('/e : false : SMTP : localhost:143 (courier) : localhost:25
Command >> 1 The domain name is the suffix at the end of all email addresses. If for example, your email address is jdoe@example.com, then your domain would be example.com.
[trim(implode('', file('/etc/'.(file_exists('/etc/mailname')?'mail':'host')
SquirrelMail Configuration : Read: config.php (1.4.0) --------------------------------------------------------Server Settings General ------1. Domain 2. Invert Time 3. Sendmail or SMTP A. B. R C S Q Update IMAP Settings Update SMTP Settings Return to Main Menu Turn color on Save data Quit
: utopia.net : false : SMTP : localhost:143 (courier) : localhost:25
Command >> Q You have not saved your data. Save? [Y/n]: Y Data saved in config.php Exiting conf.pl. You might want to test your configuration by browsing to http://your-squirrelmail-location/src/configtest.php Happy SquirrelMailing!
Page 47
Labs
Q: Point a browser or any http client to http://127.0.0.1/squirrelmail/src/ configtest.php to check the configuration. This URL is valid if you've used the apache.conf file that comes with squirrelmail. If you've created a virtual host or a different configuration, your milage may vary.
root@mangus:~# apt-get install elinks supinfo@mangus:~$ elinks http://127.0.0.1/squirrelmail/src/configtest.php
A:
Q: A:
From your host operating system, point your browser at http://vm-ip/squirrelmail/ (adjust the URL as needed) and try to login as supinfo/supinfo.
3.3. Virtual domains

In the previous section, you've setup a complete e-mail system for the utopia.net domain. This setup was a "canonical" setup: Mail addresses were bound to system users. Although it's possible to have more than one canonical domain attached to the mail server, it's not the most flexible setup: If you have more than one canonical domain, you've still the same user set: supinfo@domain1.tld and supinfo@domain2.tld are indeed the same user and therefore mailbox. In this section, you're going to alter the configuration to host virtual domains. Each virtual domain has its own set of users and mailboxes.
Page 48
Labs
3.3.1. Utopia.net goes virtual
In this section, you're going to tune Postfix configuration to make utopia.net a virtual domain.
3.3.1.1. Filesystem: Prepare virtual mailboxes

In this section, you're going to create necessary directory adjust permissions for virtual domains to operate. Q: Each virtual domain user mailboxes must go in a dedicated directory under the virtual "mail root". This directory is usually named after the domain itself. Provided that the virtual mail root is /var/spool/ mail, create a directory for the utopia.net domain. root@mangus:~# mkdir /var/spool/mail/utopia.net Each "virtual" user (domain mailbox) will automatically have his mailbox directory created under the domain directory where it belongs. as you knoz Unix files and directories are always owned by a user and a group. When mail addresses matches system users, their mailboxes and files are just owned by these system users uid and gid. However, with virtual domains enabled there is no system account associated with each mailbox and therefore no "natural" ownership policy. Moreover, the imap and pop3 deamons expects mailboxes to bo owned by the matching user and its primary group. When an user authenticates on the imap or pop3 deamon the undertakes this very user identity, that is switching to this user effective uid and primary gid. As an example when you issue a AUTH supinfo supinfo when talking with the imapd, the imapd switches its effective uid to supinfo and effective gid to supinfo's primary group. The only satisfactory solution to these issues is to create a special user and a special group associated with virtual users mail delivery. All virtual users will be bound to this uid/gid couple to solve the ownership problem. Create a vmail system group and a vmail system user for this task. As these user and group are not "real" users, they should get a uid/gid in the appropriate range. Additionaly, the user should get /bin/false as its shell, the virtual mail root as its home and of course the epony;ous group as primary group. A:
root@mangus:~# groupadd -g 111 vmail root@mangus:~# useradd -u 107 -d /var/spool/mail/ -s /bin/false -g vmail vmail
A:
Q:
Q: Q:
Change ownership of the utopia.net virtual mail directory to vmail:vmail. root@mangus:~# chown vmail:vmail /var/spool/mail/utopia.net
3.3.1.2. Postfix
In this section, you're going to alter postfix configuration to move utopia.net from canonical to virtual.
3.3.1.2.1. Virtual mailboxes mapping

In this section you're going to map mail addresses to mailbox directories. Q: Create a /etc/postfix/vmailboxes file that maps supinfo, sarah and bob accounts from the utopia.net domain to their mailboxes stored under the domain directory. All relative path are relative to the virtual mail root that will be /var/spool/mail. You want mail to be delivered in the maildir format: Use the appropriate syntax! Create /etc/postfix/vmailboxes as follows:
A:
Page 49
Labs
# utopia.net domain supinfo@utopia.net bob@utopia.net sarah@utopia.net
utopia.net/supinfo/ utopia.net/bob/ utopia.net/sarah/
Warning
Be sure to add the trailing / to get mail delivered into maildir format.
Q: A:
Generate the binary hash table file from your plain-text mapping file. root@mangus:~# postmap /etc/postfix/vmailboxes
3.3.1.2.2. Server configuration

Q: Alter the server configuration to have: utopia.net as a virtual domain /var/spool/mail as the virtual mail domains root. /etc/postfix/vmailboxes as the virtual mail addresses mapping file vmail as the uid and gid used for virtual domains delivery A: Your configuration should look like the following:
Page 50
Labs
smtpd_banner = $myhostname ESMTP biff = no # appending .domain is the MUA's job. append_dot_mydomain = no mydestination = magnus localhost mangus.localdomain localhost.localdomain home_mailbox = Maildir/ inet_interfaces = all mynetworks_style = host
virtual_mailbox_domains = utopia.net virtual_mailbox_base = /var/spool/mail virtual_mailbox_maps = hash:/etc/postfix/vmailboxes virtual_uid_maps = static:107 virtual_gid_maps = static:111 smtpd_sasl_auth_enable = yes broken_sasl_auth_clients = yes smtpd_recipient_restrictions = permit_mynetworks permit_sasl_authenticated reject_unauth_destination smtpd_tls_cert_file = /etc/postfix/smtpd-cert.pem smtpd_tls_key_file = /etc/postfix/smtpd-key.pem smtpd_tls_security_level = may
3.3.1.2.3. Testing the virtual domain

Q: A: Try to send mails to supinfo, bob and/or sarah. root@mangus:~# mail supinfo@utopia.net Subject: Test virtual Some content . EOT root@mangus:~# mail sarah@utopia.net Subject: Virtual test Mail received ? . EOT Check if your mails have reached the new mailboxes.
Q:
Page 51
Labs
A:
root@mangus:~# find /var/spool/mail/ /var/spool/mail/ /var/spool/mail/utopia.net /var/spool/mail/utopia.net/sarah /var/spool/mail/utopia.net/sarah/tmp /var/spool/mail/utopia.net/sarah/new /var/spool/mail/utopia.net/sarah/new/1338567473.V801I531b3M813160.mangus /var/spool/mail/utopia.net/sarah/cur /var/spool/mail/utopia.net/supinfo /var/spool/mail/utopia.net/supinfo/tmp /var/spool/mail/utopia.net/supinfo/new /var/spool/mail/utopia.net/supinfo/new/1338567341.V801I531a8M199840.mangus /var/spool/mail/utopia.net/supinfo/cur
3.3.1.3. Courier
In the previous section, you've tuned postfix configuration to host virtual domains. The server is now accepting incoming mail for virtual users mapped in the /etc/postfix/vmailboxes file. However the SMTPd cannot authenticate you anymore, because it relies on the IMAP authentication that relies in turn on PAM that only knows about system users. In this section, you'll configure Courier authentication layer to use virtual users.
3.3.1.3.1. Courier authentication daemon

Q: A: Edit the courier authentication daemon configuration to use the authuserdb authentication module (only). Alter /etc/courier/authdaemonrc as follows: authmodulelist="authuserdb" Q: A: Restart courier authentication daemon root@mangus:~# /etc/init.d/courier-authdaemon restart
3.3.1.3.2. Virtual user accounts

Virtual users are "created" by two complementary methods: Email address to mailbox mapping in postfix configuration Credentials to mailbox mapping in courier configuration The first one tells postfix where to deliver messages whereas the second tells how to let people access these mailboxes. Q: Use the userdb command to create virtual users matching bob, supinfo and sarah. Be sure to Use full mail addresses as login names Bind them to the vmail uid and gid Set both their home and mail properties to their mailbox
Page 52
Labs
It doesn't matter if the mailbox directory hasn't been created yet (if the user hasn't received any message). The mailbox is VIRTUAL_MAIL_ROOT/DOMAIN_NAME/USERNAME, as typed in the postfix mapping file. A:
root@mangus:~# userdb supinfo@utopia.net set uid=107 gid=111 \ >home=/var/spool/mail/utopia.net/supinfo mail=/var/spool/mail/utopia.net/supinfo root@mangus:~# userdb bob@utopia.net set uid=107 gid=111 \ >home=/var/spool/mail/utopia.net/bob mail=/var/spool/mail/utopia.net/bob root@mangus:~# userdb sarah@utopia.net set uid=107 gid=111 \ >home=/var/spool/mail/utopia.net/bob mail=/var/spool/mail/utopia.net/sarah
Q: A:
Set a random password for all these users. root@mangus:~# userdbpw -md5 | userdb supinfo@utopia.net set systempw Password: Reenter password: root@mangus:~# userdbpw -md5 | userdb bob@utopia.net set systempw Password: Reenter password: root@mangus:~# userdbpw -md5 | userdb sarah@utopia.net set systempw Password: Reenter password: Rebuild the user database root@mangus:~# makeuserdb
Q: A:
3.3.1.3.3. Adjusting Cyrus-SASL options

Q: Usernames are now in the "user@domain.tld" form. In the default configuration, Cyrus-sasl's saslauthd splits the username in two parts and passes the login as "user" to the authentication backend. In our case, the authentication backup is the IMAP server, which expects logins in the "user@domain.tld" form. Use the saslauthd man page to find which option tells the authentication deamon not to split the login(search for the realm keyword). Adjusts saslauthd startup options and restart the daemon. The switch is -r. Edit /etc/default/saslauthd as follows: OPTIONS="-r c -m /var/spool/postfix/var/run/saslauthd" And restart the service: root@mangus:~# /etc/init.d/saslauthd restart
A:
3.3.1.3.4. Tests
Q: Try to establish an IMAP session as supinfo@utopia.net, using SSL or not, at your option.
Page 53
Labs
A:
supinfo@mangus:~$ telnet localhost imap Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. * OK [CAPABILITY IMAP4rev1 UIDPLUS CHILDREN NAMESPACE THREAD=ORDEREDSUBJECT THREAD=REFERENCES SORT QUOTA IDLE ACL ACL2=UNION STARTTLS] Courier-IMAP ready. Copyright 1998-2010 Double Precision See COPYING for distribution information. 01 LOGIN supinfo@utopia.net supinfo * BYE [ALERT] Fatal error: Account's mailbox directory is not owned by the correct uid or gid: Connection closed by foreign host. supinfo@mangus:~$ telnet localhost imap Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. * OK [CAPABILITY IMAP4rev1 UIDPLUS CHILDREN NAMESPACE THREAD=ORDEREDSUBJECT THREAD=REFERENCES SORT QUOTA IDLE ACL ACL2=UNION STARTTLS] Courier-IMAP ready. Copyright 1998-2010 Double Precision See COPYING for distribution information. 01 LOGIN supinfo@utopia.net supinfo * BYE [ALERT] Fatal error: Account's mailbox directory is not owned by the correct uid or gid: Connection closed by foreign host. supinfo@mangus:~$ telnet localhost imap Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. * OK [CAPABILITY IMAP4rev1 UIDPLUS CHILDREN NAMESPACE THREAD=ORDEREDSUBJECT THREAD=REFERENCES SORT QUOTA IDLE ACL ACL2=UNION STARTTLS] Courier-IMAP ready. Copyright 1998-2010 Double Precision See COPYING for distribution information. 01 LOGIN supinfo@utopia.net supinfo 01 OK LOGIN Ok. 02 LIST "" * * LIST (\Marked \HasNoChildren) "." "INBOX" 02 OK LIST completed 03 SELECT INBOX * FLAGS (\Draft \Answered \Flagged \Deleted \Seen \Recent) * OK [PERMANENTFLAGS (\* \Draft \Answered \Flagged \Deleted \Seen)] Limited * 1 EXISTS * 1 RECENT * OK [UIDVALIDITY 1338813376] Ok * OK [MYRIGHTS "acdilrsw"] ACL 03 OK [READ-WRITE] Ok 04 FETCH 1 ALL * 1 FETCH (FLAGS (\Recent) INTERNALDATE "01-Jun-2012 18:15:41 +0200" RFC822.SIZE 560 ENVELOPE ("Fri, 01 Jun 2012 18:15:41 +0200" "Test virtual" (("root" NIL "root" "mangus.localdomain")) (("root" NIL "root" "mangus.localdomain")) (("root" NIL "root" "mangus.localdomain")) ((NIL NIL "supinfo" "utopia.net")) NIL NIL NIL "<20120601161541.2BB6B5310F@mangus.localdomain>")) 04 OK FETCH completed.
Q:
Try to authenticate against the SMTPD as supinfo@utopia.net.
Page 54
Labs
A:
supinfo@mangus:~$ printf '\0supinfo@utopia.net\0supinfo' | base64 AHN1cGluZm9AdXRvcGlhLm5ldABzdXBpbmZv supinfo@mangus:~$ telnet localhost smtp Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 220 mangus.localdomain ESMTP EHLO ts 250-mangus.localdomain 250-PIPELINING 250-SIZE 10240000 250-VRFY 250-ETRN 250-STARTTLS 250-AUTH LOGIN CRAM-MD5 DIGEST-MD5 PLAIN NTLM 250-AUTH=LOGIN CRAM-MD5 DIGEST-MD5 PLAIN NTLM 250-ENHANCEDSTATUSCODES 250-8BITMIME 250 DSN AUTH PLAIN AHN1cGluZm9AdXRvcGlhLm5ldABzdXBpbmZv 235 2.7.0 Authentication successful QUIT 221 2.0.0 Bye Connection closed by foreign host. Try to authenticate as supinfo@utopia.net using the webmail.
Q: A:
3.3.2. Adding virtual domains

In the previous section, you've configured your mail system to host utopia.net as a virtual domain. In this section, you'll see how easy it is to host a new domain with the same system.
3.3.2.1. DNS
The first step is to make the domain's MX records point to our mail server. In this lab, we are going to setup a new zone in our DNS for newdomain.com which will figure the domain to add. Q: Add a newdomain.com zone in the DNS with the MX record resolving to magnus ip address. Also add smtp, pop and imap A records to the domain. You can just copy the utopia.net zone file and do appropriate modifications. Add the zone declaration in the DNS configuration: zone "newdomain.com" { type master; file "/etc/bind/db.newdomain.com"; };
A:
Page 55
Labs
And the zone file itself: $TTL @ 604800 IN
SOA
newdomain.com. root.newdomain.com. ( 20120604 ; Serial 604800 ; Refresh 86400 ; Retry 2419200 ; Expire 604800 ) ; Negative Cache TTL ns mail.newdomain.com. 192.168.82.156 192.168.82.156 mail mail mail
; @ @ ns mail imap pop3 smtp Q: A:
IN IN IN IN IN IN IN
NS MX 10 A A CNAME CNAME CNAME
Restart the service and check that the MX record is resolved to the local machine.
root@mangus:~# /etc/init.d/bind9 restart root@mangus:~# dig MX newdomain.com ; <<>> DiG 9.7.3 <<>> MX newdomain.com ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 50546 ;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 1, ADDITIONAL: 2 ;; QUESTION SECTION: ;newdomain.com. IN MX ;; ANSWER SECTION: newdomain.com. 604800 IN MX 10 mail.newdomain.com. ;; AUTHORITY SECTION: newdomain.com. 604800 IN NS ns.newdomain.com. ;; ADDITIONAL SECTION: mail.newdomain.com. 604800 IN A 192.168.82.156 ns.newdomain.com. 604800 IN A 192.168.82.156 ;; ;; ;; ;; Query time: 0 msec SERVER: 192.168.82.156#53(192.168.82.156) WHEN: Mon Jun 4 15:50:57 2012 MSG SIZE rcvd: 101
3.3.2.2. Filesystem:
Q: Create a directory to store newdomain.com users mailboxes, just like you did for utopia.net. Remember that the virtual mail domains root is /var/spool/mail. Also make sure this directory is owned by the virtual mail user and group.
Page 56
Labs
A:
root@mangus:~# mkdir /var/spool/mail/newdomain.com root@mangus:~# chown vmail:vmail /var/mail/newdomain.com
3.3.2.3. Postfix configuration

Q: A: Add newdomain.com to the list of virtual hosted domains. Edit postfix configuration as follows: virtual_mailbox_domains = utopia.net newdomain.com Q: Add virtual mailbox mappings for: john@newdomain.com glenn@newdomain.com brian@newdomain.com Don't forget to rebuild the mapping file afterwards A: Your /etc/postfix/vmailboxes file should look like the following: # utopia.net domain supinfo@utopia.net bob@utopia.net sarah@utopia.net # newdomain.com domain john@newdomain.com glenn@newdomain.com brian@newdomain.com
utopia.net/supinfo/ utopia.net/bob/ utopia.net/sarah/
newdomain.com/john/ newdomain.com/glenn/ newdomain.com/brian/
Warning
Don't forget to trailing slash to get the mail delivered in the maildir format that Courier's daemons expect.
Re-hash the mappings: root@mangus:~# postmap /etc/postfix/vmailboxes Q: A: Restart the service. root@mangus:~# /etc/init.d/postfix restart
Page 57
Labs
3.3.2.4. Registering users
In this section, you'll register virtual users into courier and bind them to the virtual addresses you've mapped in postfix configuration. Q: Use the userdb command to create virtual users matching john, glenn and brian. Be sure to Use full mail addresses as login names Bind them to the vmail uid and gid Set both their home and mail properties to their mailbox It doesn't matter if the mailbox directory hasn't been created yet (if the user hasn't received any message). The mailbox is VIRTUAL_MAIL_ROOT/DOMAIN_NAME/USERNAME, as typed in the postfix mapping file. A:
root@mangus:~# userdb john@newdomain.com set uid=107 gid=111 \ >home=/var/spool/mail/newdomain.com/john mail=/var/spool/mail/newdomain.com/john root@mangus:~# userdb glenn@newdomain.com set uid=107 gid=111 \ >home=/var/spool/mail/newdomain.com/glenn mail=/var/spool/mail/newdomain.com/glenn root@mangus:~# userdb brian@newdomain.com set uid=107 gid=111 \ >home=/var/spool/mail/newdomain.com/brian mail=/var/spool/mail/newdomain.com/brian
Q: A:
Set a random password for all these users.

root@mangus:~# echo "supinfo" | userdbpw -md5 | userdb john@newdomain.com set systempw root@mangus:~# echo "supinfo" | userdbpw -md5 | userdb glenn@newdomain.com set systempw root@mangus:~# echo "supinfo" | userdbpw -md5 | userdb brian@newdomain.com set systempw
Q: A:
Rebuild the user database root@mangus:~# makeuserdb
3.3.2.5. Tests
Q: A: Try to open an IMAP session as glenn@newdomain.com. Does it works? Why?
supinfo@mangus:~$ telnet imap.newdomain.com imap Trying 192.168.82.156... Connected to mail.newdomain.com. Escape character is '^]'. * OK [CAPABILITY IMAP4rev1 UIDPLUS CHILDREN NAMESPACE THREAD=ORDEREDSUBJECT THREAD=REFERENCES SORT QUOTA IDLE ACL ACL2=UNION STARTTLS] Courier-IMAP ready. Copyright 1998-2010 Double Precision See COPYING for distribution information. 01 LOGIN glenn@newdomain.com supinfo * BYE Temporary problem, please try again later Connection closed by foreign host.
It doesn't works, because glenn mailbox hasn't been created yet. It will be automatically created by postfix if he gets mail. Q: Fix the problem, and try again.
Page 58
Labs
A:
supinfo@mangus:~$ telnet imap.newdomain.com imap Trying 192.168.82.156... Connected to mail.newdomain.com. Escape character is '^]'. * OK [CAPABILITY IMAP4rev1 UIDPLUS CHILDREN NAMESPACE THREAD=ORDEREDSUBJECT THREAD=REFERENCES SORT QUOTA IDLE ACL ACL2=UNION STARTTLS] Courier-IMAP ready. Copyright 1998-2010 Double Precision See COPYING for distribution information. 01 LOGIN glenn@newdomain.com supinfo * BYE Temporary problem, please try again later Connection closed by foreign host. supinfo@mangus:~$ mail glenn@newdomain.com Subject: Welcome to our greate mail system This welcome mail will automatically create your mailbox . EOT supinfo@mangus:~$ telnet imap.newdomain.com imap Trying 192.168.82.156... Connected to mail.newdomain.com. Escape character is '^]'. * OK [CAPABILITY IMAP4rev1 UIDPLUS CHILDREN NAMESPACE THREAD=ORDEREDSUBJECT THREAD=REFERENCES SORT QUOTA IDLE ACL ACL2=UNION STARTTLS] Courier-IMAP ready. Copyright 1998-2010 Double Precision See COPYING for distribution information. 01 LOGIN glenn@newdomain.com supinfo 01 OK LOGIN Ok.
Q: A:
Try to read your mail as glenn using the webmail.
Page 59
Labs
4. iSCSI
In this section, you're going to experiment with iSCSI. Working with iSCSI takes at least two virtual machines: A machine that exports devices, the target, and a machine that uses these exported devices, the initiator. Prepare two virtual machines, with two network interfaces: One on the NAT'ed virtual network and the second on the host-only network. Be sure to add two more hard drives to the machine you're going to use as the iSCSI target.
4.1. Simple iSCSI

In this section, you're going to export a pair of devices through iSCSI. I'm going to assume the you've prepared the virtual machines with appropriate devices as stated above.
4.1.1. Target setup

In this section, you're going to configure the virtual machine that will export devices. You'll export the additional two disks you've added to the virtual machine. Be sure to pick the right one. Q: A: Set the machine hostname to "san". root@debian-master:~# echo "san" > /etc/hostname root@debian-master:~# /etc/init.d/hostname.sh Initialize your two additional disks: Create a single partition on each one and format it as ext3. root@san:~# fdisk /dev/sdb Command (m for help): n Command action e extended p primary partition (1-4) p Partition number (1-4): 1 First cylinder (1-1044, default 1): Using default value 1 Last cylinder, +cylinders or +size{K,M,G} (1-1044, default 1044): Using default value 1044 Command (m for help): w root@san:~# fdisk /dev/sdc Command (m for help): n Command action e extended p primary partition (1-4) p Partition number (1-4): 1 First cylinder (1-1044, default 1): Using default value 1 Last cylinder, +cylinders or +size{K,M,G} (1-1044, default 1044): Using default value 1044 Command (m for help): w root@san:~# mkfs.ext3 /dev/sdb1 root@san:~# mkfs.ext3 /dev/sdc1
Q: A:
Page 60
Labs
Q: A: Install appropriate packages to act as a target. root@san:~# apt-get install iscsitarget root@san:~# apt-get install iscsitarget-dkms
Note
Q:
Export your devices as disk1 and disk2 in the utopia.net domain. Use the UUID's rather than devices names: Devices names are connection-dependant whereas the UUID's will never change. Protect your devices with sanuser/sanuser credentials.
root@san:~# ls /dev/disk/by-uuid/ -l total 0 lrwxrwxrwx 1 root root 10 Jun 12 13:41 lrwxrwxrwx 1 root root 10 Jun 12 13:41 lrwxrwxrwx 1 root root 10 Jun 12 13:47 lrwxrwxrwx 1 root root 10 Jun 12 13:47
A:
0d249d58-fa1e-4e09-ae98-ea777460224e 44d22a03-f7ea-44c0-ae72-9eb1465f51f5 5ee38f27-1d89-4ca6-94f6-c2e10cab88b3 d7a134e3-bf82-4702-9ae1-bec7513baa7b
-> -> -> ->
../../sda2 ../../sda1 ../../sdc1 ../../sdb1
Edit your /etc/iet/ietd.conf as follow:

Target iqn.2012.06.net.utopia:disk1 IncomingUser sanuser sanuser Lun 0 Path=/dev/disk/by-uuid/d7a134e3-bf82-4702-9ae1-bec7513baa7b,Type=fileio Target iqn.2012.06.net.utopia:disk2 IncomingUser sanuser sanuser Lun 0 Path=/dev/disk/by-uuid/5ee38f27-1d89-4ca6-94f6-c2e10cab88b3,Type=fileio
Q: A:
(Re)start the service. If you're running Debian, enable the target first: root@san:~# echo "ISCSITARGET_ENABLE=true" > /etc/default/iscsitarget And then restart the service: root@san:~# /etc/init.d/iscsitarget restart
4.1.2. Initiator setup

In this section, you're going to configure the second virtual machine as an iSCSI initiator that connects the target and mount devices. Q: A: Set the machine hostname to "initiator". root@debian-master:~# echo "initiator" > /etc/hostname root@debian-master:~# /etc/init.d/hostname.sh Install the iscsi initiator userland tools.
Q:
Page 61
Labs
A:
root@initiator:~# apt-get install open-iscsi Start the iSCSI client deamon. root@initiator:~# /etc/init.d/open-iscsi start Try to discover targets exported from the first virtual machine. root@initiator:~# iscsiadm -m discovery -t st -p 192.168.82.164 192.168.82.164:3260,1 iqn.2012.06.net.utopia:disk1 192.168.82.164:3260,1 iqn.2012.06.net.utopia:disk2 Try to have the local iSCSI node to login to the first exported target. Does it works? Why?
root@initiator:~# iscsiadm -m node --targetname "iqn.2012.06.net.utopia:disk1" \ >--portal "192.168.82.164:3260" --login Logging in to [iface: default, target: iqn.2012.06.net.utopia:disk1, portal: 192.168.82.164,3260] iscsiadm: Could not login to [iface: default, target: iqn.2012.06.net.utopia:disk1, portal: 192.168.82.164,3260]:
Q: A:
Q: A:
Q: A:
It doesn't work because you need to authenticate to the target as sanuser/sanuser to access this resource. Q: A: Try again while supplying the right credentials for this target.
root@initiator:~# iscsiadm -m node --targetname "iqn.2012.06.net.utopia:disk1" \ >-p "192.168.82.164:3260" --op=update -n node.session.auth.username -v sanuser root@initiator:~# iscsiadm -m node --targetname "iqn.2012.06.net.utopia:disk1" \ >-p "192.168.82.164:3260" --op=update -n node.session.auth.password -v sanuser root@initiator:~# iscsiadm -m node --targetname "iqn.2012.06.net.utopia:disk1" \ >--portal "192.168.82.164:3260" --login Logging in to [iface: default, target: iqn.2012.06.net.utopia:disk1, portal: 192.168.82.164,3260] Login to [iface: default, target: iqn.2012.06.net.utopia:disk1, portal: 192.168.82.164,3260]: successful
Q: A:
Edit the local node configuration to supply sanuser/sanuser credentials by default. Also enable CHAP authentication. Edit /etc/iscsi/iscsid.conf as follows: node.session.auth.authmethod = CHAP node.session.auth.username = sanuser node.session.auth.password = sanuser
Q: A:
Try to connect to the second target. Does it works? Why?

root@initiator:~# iscsiadm -m node --targetname "iqn.2012.06.net.utopia:disk2" \ >--portal "192.168.82.164:3260" --login Logging in to [iface: default, target: iqn.2012.06.net.utopia:disk2, portal: 192.168.82.164,3260] Login to [iface: default, target: iqn.2012.06.net.utopia:disk2, portal: 192.168.82.164,3260]: successful
It works because the expected credentials have been sent using the default values. Q: Mount the first disk, try to create a file on it and then unmount the remote device.
Page 62
Labs
A:
root@initiator:~# mount \ >/dev/disk/by-path/ip-192.168.82.164\:3260-iscsi-iqn.2012.06.net.utopia\:disk1-lun-0 /mnt/ root@initiator:~# echo "Hello SAN" > /mnt/test root@initiator:~# umount /mnt/
Q: A:
Restart the daemon. Are the devices still present? Why?

root@initiator:~# /etc/init.d/open-iscsi restart root@initiator:~# ls /dev/disk/by-path/ pci-0000:00:07.1-scsi-1:0:0:0 pci-0000:00:10.0-scsi-0:0:0:0 pci-0000:00:10.0-scsi-0:0:0:0-part1 pci-0000:00:10.0-scsi-0:0:0:0-part2
The remote devices didn't survived the deamon restart, because the node.startup value is set to its default, manual. Therefore devices are never automatically attached are created. Q: A: Adjust the node configuration to have the devices mounted at deamon startup.
root@initiator:~# iscsiadm -m node --targetname "iqn.2012.06.net.utopia:disk1" \ >-p "192.168.82.164:3260" --op=update -n node.startup -v automatic root@initiator:~# iscsiadm -m node --targetname "iqn.2012.06.net.utopia:disk2" \ >-p "192.168.82.164:3260" --op=update -n node.startup -v automatic root@initiator:~# /etc/init.d/open-iscsi restart Login to [iface: default, target: iqn.2012.06.net.utopia:disk1, portal: 192.168.82.164,3260]: successful Login to [iface: default, target: iqn.2012.06.net.utopia:disk2, portal: 192.168.82.164,3260]: successful
Q: A:
Modify the default settings to have any newly created device automatically attached as well. Edit the /etc/iscsi/iscsid.conf file as well: node.startup = automatic
4.2. Multipath iSCSI

In this section, you're going to configure multipath iSCSI. Multipath iSCSI provides network link redondancy and let your devices survive a link failure. In real-world setups, the complete link should be redundant, including different network cards, but also different switches, like in the follwing schema:
Page 63
Labs
4.2.1. Target setup

Q: Enable and configure the second NIC. You can either use DHCP or set a fixed IP addresses. Of course a reallife SAN server should rather have fixed IP addresses. However for a lab environment there is no practical difference. Edit /etc/network/interfaces as follows: allow-hotplug eth1 iface eth1 inet static address 172.17.2.11 netmask 255.255.255.0 And apply your modifications: root@san:~# ifup eth1
A:
Note
This is Debian-specific. This is also a fixed-address configuration. You can perfectly use a DHCPbased setup instead.
4.2.2. Initiator setup

Q: A: Configure your second network interface to be on the same network as the first virtual machine second virtual interface you've just configured. Edit /etc/network/interfaces as follows:
Page 64
Labs
allow-hotplug eth1 iface eth1 inet static address 172.17.2.22 netmask 255.255.255.0 And apply your modifications: root@initiator:~# ifup eth1 Q: A: List exported devices from the first virtual machine. What do you notice? root@initiator:~# iscsiadm -m discovery -t st -p 192.168.82.164 192.168.82.164:3260,1 iqn.2012.06.net.utopia:disk1 172.17.2.11:3260,1 iqn.2012.06.net.utopia:disk1 192.168.82.164:3260,1 iqn.2012.06.net.utopia:disk2 172.17.2.11:3260,1 iqn.2012.06.net.utopia:disk2 Each device appears twice, once per IP. Q: A: Login to the targets on the second IP address.
root@initiator:~# iscsiadm -m node -T "iqn.2012.06.net.utopia:disk1" \ >-p "172.17.2.11:3260" --login root@initiator:~# iscsiadm -m node -T "iqn.2012.06.net.utopia:disk2" \ >-p "172.17.2.11:3260" --login
Q: A:
Lis all devices present in /dev/disk/by-path/ using long listing mode. What do you notice?
root@initiator:~# total 0 lrwxrwxrwx 1 root lrwxrwxrwx 1 root lrwxrwxrwx 1 root lrwxrwxrwx 1 root lrwxrwxrwx 1 root lrwxrwxrwx 1 root lrwxrwxrwx 1 root lrwxrwxrwx 1 root ls /dev/disk/by-path/ -l root 9 Jun root 9 Jun root 9 Jun root 9 Jun root 9 Jun root 9 Jun root 10 Jun root 10 Jun 12 12 12 12 12 12 12 12 18:00 18:01 17:09 17:09 13:41 13:41 13:41 13:41 ip-172.17.2.11:3260-iscsi-iqn.2012.06.net.utopia:disk1-lun-0 -> ip-172.17.2.11:3260-iscsi-iqn.2012.06.net.utopia:disk2-lun-0 -> ip-192.168.82.164:3260-iscsi-iqn.2012.06.net.utopia:disk1-lun-0 ip-192.168.82.164:3260-iscsi-iqn.2012.06.net.utopia:disk2-lun-0 pci-0000:00:07.1-scsi-1:0:0:0 -> ../../sr0 pci-0000:00:10.0-scsi-0:0:0:0 -> ../../sda pci-0000:00:10.0-scsi-0:0:0:0-part1 -> ../../sda1 pci-0000:00:10.0-scsi-0:0:0:0-part2 -> ../../sda2 ../../sdd ../../sde -> ../../sdc -> ../../sdb
You know have two devices per target, one per path.
4.2.2.1. Using multipath

You have two distinct network paths to reach each of your targets. Each network path is represented by a device. It means that from the system perspective, each path is something different. If you want to use both at the same time, one solution could be to mount these devices in separate mount points. However, only a few filesystem designed on purpose actually supports multi-mounting. At this point having two path but two devices seems pretty useless. What you want is to have only one device and the system using the two path indinstinctively. The achieve this goal, you need an additional piece of software that will tell the system that these two devices are actually the same. Q: A: Install the multipath tools package. root@initiator:~# apt-get install multipath-tools
Page 65
Labs
Q: A: Q: A: Ensure that the device mapper kernel module is loaded. root@initiator:~# modprobe dm_mod Restart the multipath deamon and check the multipath status.
root@initiator:~# /etc/init.d/multipath-tools restart root@initiator:~# multipath -l mpath1 (14945540000000000745c758a8fcf684b36429b04a630ef63) dm-1 IET,VIRTUAL-DISK size=8.0G features='0' hwhandler='0' wp=rw |-+- policy='round-robin 0' prio=-1 status=active | `- 15:0:0:0 sdc 8:32 active undef running `-+- policy='round-robin 0' prio=-1 status=enabled `- 17:0:0:0 sdd 8:48 active undef running mpath0 (14945540000000000a27ba0f0a1070b8e757e06570954193d) dm-0 IET,VIRTUAL-DISK size=8.0G features='0' hwhandler='0' wp=rw |-+- policy='round-robin 0' prio=-1 status=active | `- 16:0:0:0 sdb 8:16 active undef running `-+- policy='round-robin 0' prio=-1 status=enabled `- 18:0:0:0 sde 8:64 active undef running
Q: A:
Mount the mpath0 device and create a file on the filesystem. root@initiator:~# mount /dev/mapper/mpath0 /mnt/ root@initiator:~# echo "Hello multipath" > /mnt/dummy Bring one of your interfaces down and try to read the file. root@initiator:~# ifconfig eth1 down root@initiator:~# cat /mnt/dummy Hello multipath
Q: A:
Page 66
Labs
5. DRBD: Distributed Replicated Block Device

In this Lab, you're going to experiment with the distributed replicated block device. This in-kernel subsystem allows administrator to build what one could call networked RAID-1. You'll need a total of three virtual machines, each one with an additional disk that will represent the volume being replicated. The first section will only use two virtual machines whereas the second will demonstrate resource stacking with three nodes. Two of these nodes will also need additional network interfaces. However as these interfaces would be useless for this first part of this lab, there is no need to add them now. You'll be given appropriate instructions in the begining of the second section. Prepare two virtual machines with an additional disk drive. The disk drive must be of the same size on both nodes. A size of 200MB should be sufficient. This lab is about DRBD. DRBD sits at the block layer, therefore it has now knowledge of the above layers, i.e filesystems. This lab will not feature any filesystem-related operations.s
5.1. Two-nodes
In this section, you're going to setup a two-node replicated block device, like in the following picture:
5.1.1. Preparing the nodes.

Q: A: Set hostnames on both nodes: drbd-node1 and drbd-node2. root@debian-master:~# echo "drbd-node1" > /etc/hostname root@debian-master:~# /etc/init.d/hostname.sh And: root@debian-master:~# echo "drbd-node2" > /etc/hostname root@debian-master:~# /etc/init.d/hostname.sh Q: If you're running a Debian-based system, check you've the following source and update your package database.
Page 67
Labs
deb http://repo.labo-linux.fr/debian/ binary/ A:
root@drbd-node1:~# echo \ >"deb http://repo.labo-linux.fr/debian/ binary/" >> /etc/apt/sources.list root@drbd-node1:~# apt-get update
And:
root@drbd-node2:~# echo \ >"deb http://repo.labo-linux.fr/debian/ binary/" >> /etc/apt/sources.list root@drbd-node2:~# apt-get update
Q: A:
Install the drbd 8.4 userland tools package and kernel module on the two nodes. If you're running a Debianbased distro, the package name are drbd8-utils and drbd8-module-2.6.32-5-686.
root@drbd-node1:~# root@drbd-node1:~# apt-get install drbd8-module-2.6.32-5-686 drbd8-utils
And:
root@drbd-node2:~# apt-get install drbd8-module-2.6.32-5-686 drbd8-utils
Q: A:
Be sure to have DRBD >= 8.4 for either the kernel module and the userland tools. Double-check the version.
root@drbd-node2:~# modinfo drbd filename: /lib/modules/2.6.32-5-686/updates/drbd.ko alias: block-major-147-* license: GPL version: 8.4.1 root@drbd-node2:~# drbdadm --version DRBDADM_BUILDTAG=GIT-hash:\ 91b4c048c1a0e06777b5f65d312b38d47abaea80\ build\ by\ supinfo@debian-dev\,\ 2012-06-14\ 12:35:11i DRBDADM_API_VERSION=1 DRBD_KERNEL_VERSION_CODE=0x000000 DRBDADM_VERSION_CODE=0x080401 DRBDADM_VERSION=8.4.1
Q: A:
Load the drbd kernel module (on both nodes). root@drbd-node1:~# modprobe drbd And root@drbd-node2:~# modprobe drbd
Q:
On both nodes, create a single partition from the additional disk. This partition will be the replicated device.
Page 68
Labs
A:
root@drbd-node1:~# fdisk /dev/sdb Command (m for help): n Command action e extended p primary partition (1-4) p Partition number (1-4): 1 First cylinder (1-204, default 1): Using default value 1 Last cylinder, +cylinders or +size{K,M,G} (1-204, default 204): Using default value 204 Command (m for help): Command (m for help): w The partition table has been altered! Calling ioctl() to re-read partition table. Syncing disks. And: root@drbd-node2:~# fdisk /dev/sdb Command (m for help): n Command action e extended p primary partition (1-4) p Partition number (1-4): 1 First cylinder (1-1044, default 1): Using default value 1 Last cylinder, +cylinders or +size{K,M,G} (1-1044, default 1044): Using default value 1044 Command (m for help): Command (m for help): w The partition table has been altered! Calling ioctl() to re-read partition table. Syncing disks.
5.1.2. Configuring replication

The drbd configuration files must be the same on both nodes. It means you can write your initial configuration on any node and then copy it to the other over SSH.
5.1.2.1. Single-Primary setup

Q: Create a "storage" replication resource configuration with the following properties: Exposed through the /dev/drbd1 device Replicates the /dev/sdb1 device Synchronous replication
Page 69
Labs
metadata stored on the same device as the actual data. A: Edit /etc/drbd.d/storage.res as follows: resource storage { device /dev/drbd1; disk /dev/sdb1; meta-disk internal; on drbd-node1 { address 192.168.82.168:7788; } on drbd-node2 { address 192.168.82.156:7788; } }
Note
As the default settings in /etc/drbd.d/global_common.conf set Protocol C globally, there is no need to state it again at the resource level. However you can still explicitly mention it. It will do nothing more but won't harm your configuration either.
Q: A: Q: A:
Copy your configuration to the other node.

root@drbd-node1:~# scp /etc/drbd.d/storage.res root@192.168.82.156:/etc/drbd.d/
Initialize the metadata and bring the resource up on both nodes. root@drbd-node1:~# drbdadm create-md storage --== Thank you for participating in the global usage survey The server's response is: you are the 5015th user to install this version Writing meta data... initializing activity log NOT initializing bitmap New drbd meta data block successfully created. success root@drbd-node1:~# drbdadm up storage and on the second node:
==--
Page 70
Labs
root@drbd-node2:~# drbdadm create-md storage --== Thank you for participating in the global usage survey The server's response is: you are the 16339th user to install this version Writing meta data... initializing activity log NOT initialized bitmap New drbd meta data block successfully created. success root@drbd-node2:~# drbdadm up storage Q: A: What's the current DRBD state? Why?
root@drbd-node2:~# cat /proc/drbd version: 8.3.7 (api:88/proto:86-91) srcversion: EE47D8BF18AC166BE219757
==--
1: cs:Connected ro:Secondary/Secondary ds:Inconsistent/Inconsistent C r---ns:0 nr:0 dw:0 dr:0 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:b oos:8385604
And/Or
root@drbd-node2:~# drbd-overview 1:storage/0 Connected Secondary/Secondary Inconsistent/Inconsistent C r-----
Both nodes are in the secondary state, and the replicated device is considered to be in the inconsistent state. As we've just created the volume this is perfectly normal: We've still have to tell DRBD which device is the original master and enable initial synchronization. Q: A: Q: A: Set drbd-node1 as primary and force a synchronization. root@drbd-node1:~# drbdadm primary --force storage Check DRBD status. What do you notice? root@drbd-node1:~# drbd-overview 1:storage/0 Connected Primary/Secondary UpToDate/UpToDate C r----The DRBD volume is now up to date.
5.1.2.1.1. Tests
Q: A: Q: On the first node, write a FOO string directly to the DRBD device. This string will be used as a synchronisation reference. root@drbd-node1:~# echo "FOO" > /dev/drbd/by-res/storage/0 On the second node, try to read the first 4 bytes in ASCII mode (If you don't know how to do that, read the od man page) from the DRBD device. Does it works? Why?
Page 71
Labs
A:
root@drbd-node2:~# od -c -N 4 /dev/drbd/by-res/storage/0 od: /dev/drbd/by-res/storage/0: Wrong medium type It doesn't work because the DRBD driver doesn't expose data on the two nodes while running in primary/ secondary mode.
Q: A:
Switch nodes status (first node to secondary and second node to primary) and try again. root@drbd-node1:~# drbdadm secondary storage And: root@drbd-node2:/home/supinfo# drbdadm primary storage root@drbd-node2:/home/supinfo# od -c -N 4 /dev/drbd/by-res/storage 0000000 F O O \n 0000004
5.1.2.2. Dual-primary setup

Q: A: Temporary enable the dual-primary mode on for the "storage" resource.
root@drbd-node1:~# drbdadm net-options --protocol=C --allow-two-primaries storage
Note
Dual-primary is only available using protocol C and thus is a C-protocol option. That's why you need to explicitly set C protocol, even if you're already using it.
Q: A:
Make the two nodes primary. root@drbd-node1:~# drbdadm primary storage And root@drbd-node2:~# drbdadm primary storage
Q: A:
Check the resource status. root@drbd-node2:~# drbd-overview 1:storage/0 Connected Primary/Primary UpToDate/UpToDate C r----Write a random string on the device from a node and check the replication on the other. root@drbd-node1:~# echo "BAR" > /dev/drbd/by-res/storage/0
Q: A:
Page 72
Labs
And root@drbd-node2:~# od -c -N 4 /dev/drbd/by-res/storage/0 0000000 B A R \n 0000004 Q: A: Put one of the nodes in secondary mode and restore the original configuration by re-reading the config file on both nodes. root@drbd-node1:~# drbdadm secondary storage root@drbd-node1:~# drbdadm adjust storage and root@drbd-node2:~# drbdadm adjust storage Q: A: Check resource status. root@drbd-node2:~# drbd-overview 1:storage/0 Connected Primary/Secondary UpToDate/UpToDate C r-----
5.1.3. Dealing with split-brain

In this section, you're going to manually resolve a split brain case. Split-brain occurs when the link between the two nodes fails and when for some reason (manual, cluster-driven, ...) the two nodes become primary in the mean time. The following picture illustrate such a situation:
Q: Q: A:
Break the link between the two nodes: Unplug the virtual interace. Write a different random string on the device on each node. root@drbd-node1:~# echo "NEW1" > /dev/drbd/by-res/storage/0 And root@drbd-node2:~# echo "NEW2" > /dev/drbd/by-res/storage/0
Q:
Re-plug the virtual network interface and check the device status.
Page 73
Labs
A:
root@drbd-node1:~# drbd-overview 1:storage/0 StandAlone Primary/Unknown UpToDate/DUnknown r----And root@drbd-node2:~# drbd-overview 1:storage/0 StandAlone Primary/Unknown UpToDate/DUnknown r-----
Q: A:
The resource is now in split-brain state. You must decide which copy of the data you want to keep (survivor) and which one to discard (victim). Select a node and discard its data. root@drbd-node2:~# drbdadm secondary storage root@drbd-node2:~# drbdadm connect --discard-my-data storage I've choosen to discard data from node2. You can perfectly discard data from node1 if you wish.
Q: A:
Connect the other node back, and check resource status. root@drbd-node1:~# drbdadm connect storage root@drbd-node1:~# drbd-overview 1:storage/0 Connected Primary/Secondary UpToDate/UpToDate C r----Check the device content: It should now contain the data from the "surviving" node. root@drbd-node1:~# od -c -N 4 /dev/drbd/by-res/storage/0 0000000 N E W 1 0000004
Q: A:
5.2. Three-nodes
In this section, you're going to add a supplementary replica to the existing DRBD array. However this third node will not behave like the first two. The first two nodes are using protocol C: A write is considered complete only when the other node has actually written the data to the disk. The third node will "emulate" an off-site replica connected through a relatively low bandwith WAN. This node will use protocol A: A write is considered complete as soon as the sync data has been pushed in the socket buffer (and of course written to the local disk). The following picture illustrate the network setup:
Page 74
Labs
The WAN will be simulated by an additional host-only network link. Add a NIC to drbd-node1 and connect it to the host-only network. You'll also need to prepare your third virtual machine as follows: Hostname: drbd-node3 Additional single-partitioned 200 MB hard drive. Virtual NIC connected to the host-only network. Also make sure drbd-node2 and drbd-node3 can communicate through the host-only network: Configure the interfaces to either get addresses via DHCP or set fixed IP addresses. Also make sure to add the updated drbd packages source as you've done before. Install appropriate packages as well. You might need to temporary switch your VNIC from host-only to NAT to perform this step. The rest of this lab assumes the above elements have been correctly configured. Q: Add a new stacked resource to your configuration. This resource is stacked on top of storage and will be bound to /dev/drbd5 and listen to the host-only NIC of drbd-node2. On drbd-node3 this resource is also bound to /dev/drbd5 and uses the /dev/sdb1 device. As this is a stacked resources, it also have internal metadata. Also add the follwing disk parameters to your stacked resource: disk { disk-barrier no; disk-flushes no; md-flushes no; } These parameters will prevent a deadlock issue you can encounter on VMware virtual machines. What can you say about this configuration? A: Edit /etc/drbd.d/storage.res as follows:
Page 75
Labs
resource storage { device /dev/drbd1; disk /dev/sdb1; meta-disk internal; on drbd-node1 { address 192.168.82.168:7788; } on drbd-node2 { address 192.168.82.156:7788; } } resource storage-m { net { protocol A; } disk { disk-barrier no; disk-flushes no; md-flushes no; } stacked-on-top-of storage { device /dev/drbd5; address 172.17.2.129:7788; } on drbd-node3 { device /dev/drbd5; disk /dev/sdb1; address 172.17.2.128:7788; meta-disk internal; } } The stacked resource is bound to drbd-node2 NIC. Therefore it will only work when the node2 is the primary node. It would be better to have the stacked resource bound to an IP address that always "moves" to the primary node. That's what cluster software (like pacemake) are used for. Q: A: Replicate the configuration on all the three nodes.
root@drbd-node1:~# scp /etc/drbd.d/storage.res root@192.168.82.156:/etc/drbd.d/ root@drbd-node2:~# scp /etc/drbd.d/storage.res root@172.17.2.128:/etc/drbd.d/
Q:
Ensure that drbd-node2 has primary role. Set it as needed.
Page 76
Labs
A:
root@drbd-node2:~# Secondary/Primary root@drbd-node1:~# root@drbd-node2:~# root@drbd-node2:~# Primary/Secondary
drbdadm role storage drbdadm secondary storage drbdadm primary storage drbdadm role storage
Q: A:
From drbd-node2, initialize the meta-data on the stacked resource, and bring it up. root@drbd-node2:~# drbdadm create-md --stacked storage-m md_offset 213843968 al_offset 213811200 bm_offset 213803008 Found some data ==> This might destroy existing data! <== Do you want to proceed? [need to type 'yes' to confirm] yes Writing meta data... initializing activity log NOT initializing bitmap New drbd meta data block successfully created. success root@drbd-node2:~# drbdadm up --stacked storage-m
Q: A:
Set drdb-node2 as primary for the stacked resource. root@drbd-node2:~# drbdadm primary --stacked --force storage-m On drbd-node3, initialize the resource by creating the metadata and bring the resource up.
root@drbd-node3:/home/supinfo# drbdadm create-md storage-m root@drbd-node3:/home/supinfo# drbdadm up storage-m root@drbd-node3:/home/supinfo# drbd-overview 5:storage-m/0 Connected Secondary/Primary UpToDate/UpToDate A r-----
Q: A:
5.2.1. Tests
Q: A: On drbd-node2 write the "STK" string to the drbd device (not the stacked one). root@drbd-node2:~# echo "STK" > /dev/drbd/by-res/storage/0 On drbd-node2, check the replication on the stacked device. Due to the very low amount of data, you may have to wait a couple of seconds to get it replicated.
Q:
Page 77
Labs
A:
root@drbd-node2:~# od -c 0000000 N E W 1 0000004 root@drbd-node2:~# od -c 0000000 N E W 1 0000004 root@drbd-node2:~# od -c 0000000 N E W 1 0000004 root@drbd-node2:~# od -c 0000000 N E W 1 0000004 root@drbd-node2:~# od -c 0000000 S T K \n 0000004
-N 4 /dev/drbd/by-res/storage-m
Q:
Speed up the replication process by remotely invalidate the stacked resource on drdb-node3 from drdbnode2, and then switch the stacked resource from primary to secondary (You want to read data on the resource on drbd-node3, so you'll have to set the resource as primary there, therefore you must demote it on drbd-node2 before). root@drbd-node2:~# drbdadm invalidate-remote --stacked storage-m root@drbd-node2:~# drbdadm secondary --stacked storage-m On drdb-node3, promote the resource and read the first bytes to see if the "STK" string you've originaly written to the underlying resource has been replicated over here. root@drbd-node3:~# drbdadm primary storage-m root@drbd-node3:~# od -c -N 4 /dev/drbd/by-res/storage-m 0000000 S T K \n 0000004 Demote the resource on drbd-node3 and promote it again on drbd-node2 to ensure proper replication operations. root@drbd-node3:~# drbdadm secondary storage-m root@drbd-node2:~# drbdadm primary --stacked storage-m
A:
Q: A:
Q: A:
Page 78
Labs
6. High Availability
6.1. High available website
In this section, you're going to build a two-nodes cluster solution that provides high-availability for a website. Of course the two-node is just an arbitrary number choosed to minimize the amount of resources needed to run this lab. You can expand this number at will to scale the cluster up. The website data will be stored on an NFS volume mounted from the active node, like in the following schema:
Note
This is a functional schema. All hosts are connected to the same network.
You'll need at least three virtual machines: Two cluster nodes and an NFS server. The client machine can be your host operating system, or an additional virtual machine, at your option. Prepare your virtual machines and ensure that they are connected to the same network and see each other. Here is a list of hostnames for these machines: nfs-server cluster-node1 cluster-node2
6.1.1. The NFS server and web content

Q: Create a /exports/docroot/index.php file with some php code that shows the following message (hostname being the hostname of the machine where the code runs): Hello from <hostname> A:
root@nfs-server:~# mkdir -p /exports/www root@nfs-server:~# echo '<?php echo "Hello from ".gethostname()."\n" ?>' > /exports/www/index.php
Page 79
Labs
Q: A: Install appropriate packages to serve content over NFS (as needed). root@nfs-server:~# apt-get install nfs-kernel-server
Note
Q: A:
Export /exports/www in read-only for all hosts.

root@nfs-server:~# echo "/exports/www *(ro,no_subtree_check)" >> /etc/exports root@nfs-server:~# exportfs -a
6.1.2. Cluster nodes 6.1.2.1. Installing services

Before actually setting up the cluster engine that will manage nodes and fail nodes over, you need to configure actual the services that needs to be managed. In this section, you'll configure Apache to serve content and then copy over the configuration to the other node. You can do the inital configuration on whichever node you want. Q: A: Install the Apache webserver and its php module. Restart the service to ensure proper php module loading. root@cluster-node1:~# apt-get install apache2 libapache2-mod-php5 root@cluster-node2:~# /etc/init.d/apache2 restart
Note
Q: A:
Add line to your fstab to mount the NFS exported directory from nfs-server to /var/www, and perform the mount. In /etc/fstab:
192.168.82.156:/exports/www /var/www/ nfs defaults 0 0
root@cluster-node1:~# mount -a Q: Check if the website is working.
Page 80
Labs
A:
Hello from cluster-node1root@cluster-node1:~# curl http://localhost Hello from cluster-node1 root@cluster-node1:~# umount /var/www/ Mirror the configuration on the second node.
root@cluster-node2:~# apt-get install apache2 libapache2-mod-php5 root@cluster-node2:~# /etc/init.d/apache2 restart root@cluster-node2:~# echo \ >"192.168.82.156:/exports/www /var/www/ nfs defaults 0 0" > /etc/fstab root@cluster-node2:~# mount -a root@cluster-node2:~# curl http://localhost/ Hello from cluster-node2
Q: A:
Q: A:
The service will be managed by the cluster. It must not be started during the init process. Remove the apache2 service from your runlevels (on both nodes). root@cluster-node1:~# insserv -r apache2 root@cluster-node2:~# insserv -r apache2
6.1.2.2. Configuring the cluster

In this section you're going to install and configure the cluster engine itself. Q: A: Install pacemaker on both nodes. root@cluster-node1:~# apt-get install pacemaker root@cluster-node2:~# apt-get install pacemaker If you're running a Debian-based system ,enable the service in /etc/default/corosync. root@cluster-node1:~# echo "START=yes" > /etc/default/corosync root@cluster-node2:~# echo "START=yes" > /etc/default/corosync Choose a private multicast address (begins with 238 or 239) and configure your two nodes to use that address for internal cluster communication. Also configure the cluster engine on both nodes to bind to the right network interface. Edit /etc/corosync/corosync.conf interface section on node1: interface { # The following values need to be set based on your environment ringnumber: 0 bindnetaddr: 192.168.82.0 mcastaddr: 239.12.13.14 mcastport: 5405 } and on node2 (or copy it over using ssh):
Q: A:
Q:
A:
Page 81
Labs
interface { # The following values need to be set based on your environment ringnumber: 0 bindnetaddr: 192.168.82.0 mcastaddr: 239.12.13.14 mcastport: 5405 } Q: Use corosync-keygen to create a private key for your cluster. Remember that corosync-keygen needs entropy. Run a background task that writes the content of your hard drive to /dev/null to generate entropy. Don't forget to kill the task when corosync-keygen is done. Copy the key over to the second node.
root@cluster-node1:~# cat /dev/sda > /dev/null& corosync-keygen [1] 3594 Corosync Cluster Engine Authentication key generator. Gathering 1024 bits for key from /dev/random. Press keys on your keyboard to generate entropy. Press keys on your keyboard to generate entropy (bits = 184). Press keys on your keyboard to generate entropy (bits = 248). Press keys on your keyboard to generate entropy (bits = 312). Press keys on your keyboard to generate entropy (bits = 376). Press keys on your keyboard to generate entropy (bits = 440). Press keys on your keyboard to generate entropy (bits = 504). Press keys on your keyboard to generate entropy (bits = 568). Press keys on your keyboard to generate entropy (bits = 632). Press keys on your keyboard to generate entropy (bits = 696). Press keys on your keyboard to generate entropy (bits = 760). Press keys on your keyboard to generate entropy (bits = 824). Press keys on your keyboard to generate entropy (bits = 888). Press keys on your keyboard to generate entropy (bits = 952). Press keys on your keyboard to generate entropy (bits = 1016). Writing corosync key to /etc/corosync/authkey. root@cluster-node1:~# kill %1 [1]+ Terminated cat /dev/sda > /dev/null root@cluster-node1:~# scp /etc/corosync/authkey root@192.168.82.170:/etc/corosync
A:
Q: A:
On both nodes, (re) start the service. root@cluster-node1:~# /etc/init.d/corosync restart root@cluster-node2:~# /etc/init.d/corosync restart From any node, check the cluster status. You might have to wait a couple of seconds before the cluster cames up. root@cluster-node1:~# crm_mon --one-shot ============ Last updated: Tue Jun 26 12:52:37 2012 Stack: openais Current DC: cluster-node1 - partition with quorum Version: 1.0.9-74392a28b7f31d7ddc86689598bd23114f58978b 2 Nodes configured, 2 expected votes 0 Resources configured. ============ Online: [ cluster-node1 cluster-node2 ]
Q:
Page 82
Labs
6.1.2.3. Configuring resources
In this section, you're going to configure cluster-managed resources and cluster internal settings. Q: A: As your cluster only has two nodes, the quorum property doesn't make sense (quroum will be lost if one out of the two nodes if lost). Configure the cluster to ignore quorum. Also disable the STONITH feature.
root@cluster-node1:~# crm crm(live)# configure INFO: building help index crm(live)configure# property stonith-enabled="false" no-quorum-policy="ignore" crm(live)configure# commit crm(live)configure# bye
Q: A:
Choose an unallocated IP address within the virtual network pool. Create a primitive web-ip IPaddr resource to handle it.
root@cluster-node1:~# crm crm(live)# configure crm(live)configure# primitive web-ip ocf:heartbeat:IPaddr params ip=192.168.82.200 crm(live)configure# commit
Q: A:
Add a lsb:apache2 resource named web-server. This resource will handle the start/stop of the apache web server on the node its active/not active. crm(live)configure# primitive web-server lsb:apache2 crm(live)configure# commit Group the two previous resources in the website group. crm(live)configure# group website web-ip web-server crm(live)configure# commit Check the cluster status and the resources location crm(live)# status ============ Last updated: Tue Jun 26 13:23:11 2012 Stack: openais Current DC: cluster-node1 - partition with quorum Version: 1.0.9-74392a28b7f31d7ddc86689598bd23114f58978b 2 Nodes configured, 2 expected votes 1 Resources configured. ============ Online: [ cluster-node1 cluster-node2 ] Resource Group: website web-ip (ocf::heartbeat:IPaddr): Started cluster-node1 web-server (lsb:apache2): Started cluster-node1
Q: A:
Q: A:
Q:
From another machine, try to access the website using the "web-ip" address.
Page 83
Labs
A:
samuel@chickamauga ~ $ curl http://192.168.82.200/ Hello from cluster-node1 Manually migrate the resource group website to the other node and try to access the website again. crm(live)# resource crm(live)resource# migrate website cluster-node2 crm(live)resource# cd .. crm(live)# status ============ Last updated: Tue Jun 26 13:26:01 2012 Stack: openais Current DC: cluster-node1 - partition with quorum Version: 1.0.9-74392a28b7f31d7ddc86689598bd23114f58978b 2 Nodes configured, 2 expected votes 1 Resources configured. ============ Online: [ cluster-node1 cluster-node2 ] Resource Group: website web-ip (ocf::heartbeat:IPaddr): Started cluster-node2 web-server (lsb:apache2): Started cluster-node2 From another computer in the network: samuel@chickamauga ~ $ curl http://192.168.82.200/ Hello from cluster-node2
Q: A:
Q:
Still from another computer in the network (a virtual machine or your host operating system), run a task that regulary queries the web server (the watch command can help). While the monitoring is going, unplug the virtual interface of the cluster node where the group is currently running and see what happens. Plug the virtual interface back while still monitoring the web site. What do you notice ? samuel@chickamauga ~ $ watch -n1 curl -s http://192.168.82.200/ Shows Every 1.0s: curl -s http://192.168.82.200/ Hello from cluster-node2 Unplug cluster-node2 virtual iface: Every 1.0s: curl -s http://192.168.82.200/ Hello from cluster-node1 Plug the interface back:
A:
Page 84
Labs
Every 1.0s: curl -s http://192.168.82.200/ Hello from cluster-node2 The resource comes back on the node where it was as soon as the failed node comes back online again. Q: Configure the cluster to prevent it to migrate resources back automatically by setting the resource-stickness default to 1000. Also make sure that no "location" constraints tries to put the website group to a particular node. crm(live)configure# delete cli-prefer-website crm(live)configure# rsc_defaults resource-stickiness="1000" crm(live)configure# commit Do the same test again (unplug and plug back the "active" node while monitoring the website) and see what happens. samuel@chickamauga ~ $ watch -n1 curl -s http://192.168.82.200/ Shows Every 1.0s: curl -s http://192.168.82.200/ Hello from cluster-node2 Unplug cluster-node2 virtual iface: Every 1.0s: curl -s http://192.168.82.200/ Hello from cluster-node1 Plug the interface back: Every 1.0s: curl -s http://192.168.82.200/ Hello from cluster-node1 The resource now stays where it is.
A:
Q: A:
6.2. High available iSCSI-based SAN

In this section, you're going to make DRBD and Pacemaker work together to provide a high-available iSCSI SAN. DRBD will ensure data replication between the two nodes where as Pacemaker will manage Primary/Secondary states and the iSCSI service (and IP) when a node becomes unavailable. The final setup will look like the following:
Page 85
Labs
You'll need three virtual machines: Two acting as cluster/drbd nodes and the third being the iSCSI initiator that mounts the exported device. Prepare three virtual machines with the following hostnames: node1 node2 initiator The DRBD nodes, node1 and node2 will need an additional hard drive to create the replicated volume. Add a 1GB hard drive to these virtual machines. Also add an additional virtual NIC (host-only virtual network) on node1 and node2. Configure these NIC with fixed IP addresses: 172.17.2.11 for node1 and 172.17.2.22 for node2.
6.2.1. Setting up DRBD

Q: A: Add the linux lab repository and install appropriate packages on both nodes (Refer to the DRBD lab). Once installed, probe the module.
root@node1:~# root@node1:~# root@node1:~# root@node1:~# root@node2:~# root@node2:~# root@node2:~# root@node2:~# echo "deb http://repo.labo-linux.fr/debian/ binary/" >> /etc/apt/sources.list apt-get update apt-get install drbd8-module-2.6.32-5-686 drbd8-utils modprobe drbd echo "deb http://repo.labo-linux.fr/debian/ binary/" >> /etc/apt/sources.list apt-get update apt-get install drbd8-module-2.6.32-5-686 drbd8-utils modprobe drbd
Q:
Prepare the additional disk (on both nodes): Create a single partition on it.
Page 86
Labs
A:
root@node1:~# fdisk /dev/sdb Command (m for help): n Command action e extended p primary partition (1-4) p Partition number (1-4): 1 First cylinder (1-130, default 1): Using default value 1 Last cylinder, +cylinders or +size{K,M,G} (1-130, default 130): Using default value 130 Command (m for help): Command (m for help): w The partition table has been altered! Calling ioctl() to re-read partition table. Syncing disks. root@node2:~# fdisk /dev/sdb Command (m for help): n Command action e extended p primary partition (1-4) p Partition number (1-4): 1 First cylinder (1-130, default 1): Using default value 1 Last cylinder, +cylinders or +size{K,M,G} (1-130, default 130): Using default value 130 Command (m for help): w The partition table has been altered! Calling ioctl() to re-read partition table. Syncing disks.
Q: A:
Configure DRBD: Create a storage resource that replicates over the 172.17.2.0/24 network. Create the /etc/drbd.d/storage.res file on both nodes with the following content: resource storage { device /dev/drbd1; disk /dev/sdb1; meta-disk internal; on node1 { address 172.17.2.11:7788; } on node2 { address 172.17.2.22:7788; } }
Page 87
Labs
Q: A: Initialze the device and bring it up (on both nodes). root@node1:~# drbdadm create-md storage Writing meta data... initializing activity log NOT initializing bitmap New drbd meta data block successfully created. success root@node1:~# drbdadm up storage root@node2:~# drbdadm create-md storage Writing meta data... initializing activity log NOT initializing bitmap New drbd meta data block successfully created. success root@node2:~# drbdadm up storage Check the current drbd status (on any node).
root@node2:~# drbd-overview 1:storage/0 Connected Secondary/Secondary Inconsistent/Inconsistent C r-----
Q: A:
Q: A: Q: A:
Do the first forced replication by setting node1 as the primary node. root@node1:~# drbdadm primary --force storage Check the current DRBD status from node1. What do you notice?
root@node1:~# drbd-overview 1:storage/0 SyncSource Primary/Secondary UpToDate/Inconsistent C r----[====>...............] sync'ed: 27.1% (762524/1044124)K
Note
The volume is sync'ing.
Q: A:
Create an ext3 filesystem on the DRBD device, from the node that as the primary role. root@node1:~# mkfs.ext3 /dev/drbd/by-res/storage/0
6.2.2. Setting up iSCSI

Q: A: Install appropriate packages to run a iSCSI target on the two nodes (Refer to the iSCSI lab as needed). If you're running a Debian-based system, don't forget to enable the service in /etc/default/iscsitarget.
root@node1:~# root@node1:~# root@node2:~# root@node2:~# apt-get install iscsitarget iscsitarget-dkms echo "ISCSITARGET_ENABLE=true" > /etc/default/iscsitarget apt-get install iscsitarget iscsitarget-dkms echo "ISCSITARGET_ENABLE=true" > /etc/default/iscsitarget
Page 88
Labs
Note
Q:
Configure the iscsi target to export the drbd device (on both node) as iqn.2012.06.net.utopia:storage. Better use the per-resource special file from /dev/drbd/by-res rather than the numbered ones such as / dev/drbd1. Edit /etc/iet/ietd.conf as follows(on both nodes): Target iqn.2012.06.net.utopia:storage Lun 0 Path=/dev/drbd/by-res/storage/0,Type=fileio
A:
Q: A:
The service will be managed by the cluster: Remove it from the nodes' runlevels. root@node1:~# insserv -r iscsitarget root@node2:~# insserv -r iscsitarget
6.2.3. Setting up the cluster 6.2.3.1. Configuring the cluster

Q: A: Install pacemaker on both nodes. root@node1:~# apt-get install pacemaker root@node2:~# apt-get install pacemaker If you're running a Debian-based system ,enable the service in /etc/default/corosync. root@node1:~# echo "START=yes" > /etc/default/corosync root@node2:~# echo "START=yes" > /etc/default/corosync Choose a private multicast address (begins with 238 or 239) and configure your two nodes to use that address for internal cluster communication. Also configure the cluster engine on both nodes to bind to the right network interface. Edit /etc/corosync/corosync.conf interface section on node1: interface { # The following values need to be set based on your environment ringnumber: 0 bindnetaddr: 192.168.82.0 mcastaddr: 239.12.13.14 mcastport: 5405 } and on node2 (or copy it over using ssh):
Q: A:
Q:
A:
Page 89
Labs
interface { # The following values need to be set based on your environment ringnumber: 0 bindnetaddr: 192.168.82.0 mcastaddr: 239.12.13.14 mcastport: 5405 } Q: Use corosync-keygen to create a private key for your cluster. Remember that corosync-keygen needs entropy. Run a background task that writes the content of your hard drive to /dev/null to generate entropy. Don't forget to kill the task when corosync-keygen is done. Copy the key over to the second node.
root@node1:~# cat /dev/sda > /dev/null& corosync-keygen [1] 3594 Corosync Cluster Engine Authentication key generator. Gathering 1024 bits for key from /dev/random. Press keys on your keyboard to generate entropy. Press keys on your keyboard to generate entropy (bits = 184). Press keys on your keyboard to generate entropy (bits = 248). Press keys on your keyboard to generate entropy (bits = 312). Press keys on your keyboard to generate entropy (bits = 376). Press keys on your keyboard to generate entropy (bits = 440). Press keys on your keyboard to generate entropy (bits = 504). Press keys on your keyboard to generate entropy (bits = 568). Press keys on your keyboard to generate entropy (bits = 632). Press keys on your keyboard to generate entropy (bits = 696). Press keys on your keyboard to generate entropy (bits = 760). Press keys on your keyboard to generate entropy (bits = 824). Press keys on your keyboard to generate entropy (bits = 888). Press keys on your keyboard to generate entropy (bits = 952). Press keys on your keyboard to generate entropy (bits = 1016). Writing corosync key to /etc/corosync/authkey. root@node1:~# kill %1 [1]+ Terminated cat /dev/sda > /dev/null root@node1:~# scp /etc/corosync/authkey root@192.168.82.173:/etc/corosync
A:
Q: A:
On both nodes, (re) start the service. root@node1:~# /etc/init.d/corosync restart root@node2:~# /etc/init.d/corosync restart From any node, check the cluster status. You might have to wait a couple of seconds before the cluster cames up. root@node1:~# crm_mon --one-shot ============ Last updated: Tue Jun 26 19:16:45 2012 Stack: openais Current DC: node1 - partition with quorum Version: 1.0.9-74392a28b7f31d7ddc86689598bd23114f58978b 2 Nodes configured, 2 expected votes 0 Resources configured. ============ Online: [ node2 node1 ]
Q:
Page 90
Labs
6.2.3.2. Creating resources
Q: As your cluster only has two nodes, the quorum property doesn't make sense (quroum will be lost if one out of the two nodes if lost). Configure the cluster to ignore quorum. Also disable the STONITH feature. Set the resource stickiness to 1000.
root@node1:~# crm crm(live)# configure INFO: building help index crm(live)configure# property stonith-enabled="false" no-quorum-policy="ignore" crm(live)configure# rsc_defaults resource-stickiness="1000" crm(live)configure# commit crm(live)configure# bye
A:
Q: A:
Choose an unallocated IP address within the virtual network pool. Create a primitive "san-ip" IPaddr resource to handle it.
root@node1:~# crm crm(live)# configure crm(live)configure# primitive san-ip ocf:heartbeat:IPaddr params ip=192.168.82.200 crm(live)configure# commit
Q: A:
Add a drbddisk "drbd-service" resource that will manage the storage volume. crm(live)configure# primitive drdb-service heartbeat:drbddisk crm(live)configure# commit Add a iscsitarget "iscsi-service" resource to manage the iscsitarget service. crm(live)configure# primitive iscsi-service lsb:iscsitarget crm(live)configure# commit Group all these resource in a "san" group. crm(live)configure# group san san-ip drbd-service iscsi-service crm(live)configure# commit Check the cluster status
Q: A:
Q: A:
Q:
Page 91
Labs
A:
crm(live)# status ============ Last updated: Wed Jun 27 12:31:31 2012 Stack: openais Current DC: node1 - partition with quorum Version: 1.0.9-74392a28b7f31d7ddc86689598bd23114f58978b 2 Nodes configured, 2 expected votes 1 Resources configured. ============ Online: [ node2 node1 ] Resource Group: san san-ip (ocf::heartbeat:IPaddr): Started node2 drdb-service (heartbeat:drbddisk): Started node2 iscsi-service (lsb:iscsitarget): Started node2
6.2.3.3. Testing the High availability

Q: A: On the "initiator" virual machine, install all appropriate packages to connect to iscsi devices. root@initiator:~# apt-get install open-iscsi
Note
Q: A:
Do a iSCSI exported devices discovery on the san-ip you've chosen. root@initiator:~# iscsiadm -m discovery -t st -p 192.168.82.200 192.168.82.156:3260,1 iqn.2012.06.net.utopia:storage 172.17.2.22:3260,1 iqn.2012.06.net.utopia:storage 192.168.82.200:3260,1 iqn.2012.06.net.utopia:storage Connect to the exported device using the san-ip portal.
root@initiator:~# iscsiadm -m node --targetname "iqn.2012.06.net.utopia:storage" \ >--portal "192.168.82.200:3260" --login Logging in to [iface: default, target: iqn.2012.06.net.utopia:storage, portal: 192.168.82.200,3260] Login to [iface: default, target: iqn.2012.06.net.utopia:storage, portal: 192.168.82.200,3260]: successful
Q: A:
Q: A:
Mount the filesystem on /mnt.

root@initiator:~# mount \ >/dev/disk/by-path/ip-192.168.82.200\:3260-iscsi-iqn.2012.06.net.utopia\:storage-lun-0 /mnt/
Q:
Create a file on that filesystem, with some random content in it.
Page 92
Labs
A: Q: A:
root@initiator:~# echo "highly available content" > /mnt/highly-available-file
Disconnect the node where the SAN is running from the network it uses to communictate with the client. root@node1:~# crm_mon --one-shot ============ Last updated: Wed Jun 27 12:45:09 2012 Stack: openais Current DC: node1 - partition with quorum Version: 1.0.9-74392a28b7f31d7ddc86689598bd23114f58978b 2 Nodes configured, 2 expected votes 1 Resources configured. ============ Online: [ node2 node1 ] Resource Group: san san-ip (ocf::heartbeat:IPaddr): Started node2 drdb-service (heartbeat:drbddisk): Started node2 iscsi-service (lsb:iscsitarget): Started node2 The SAN services are running on node2. Just disconnect this node.
Q: A:
Try to read your file from the initiator. root@initiator:~# cat /mnt/highly-available-file highly available content
Page 93
Labs
7. SELinux
In this lab, you're going to work with SELinux. You only need a single virtual machine, in its standard configuration.
7.1. Enabling SELinux

Q: A: Install SELinux packages on your system as well as the auditing daemon.
root@debian-selinux:~# apt-get install selinux-policy-default selinux-basics selinux-utils auditd
Note
Q: A:
Check current SELinux status root@debian-selinux:~# sestatus SELinux status: disabled Activate SELinux using the selinux-activate script and proceed to reboot. Note this script is specific to Debian-based systems. root@debian-selinux:~# selinux-activate Activating SE Linux Generating grub.cfg ... Found linux image: /boot/vmlinuz-2.6.32-5-686 Found initrd image: /boot/initrd.img-2.6.32-5-686 done SE Linux is activated. You may need to reboot now.
Q: A:
Note
This is Debian specific
Q: A:
Check current SELinux status. What's the current mode? What does it means? root@debian-selinux:~# sestatus SELinux status: SELinuxfs mount: Current mode: Mode from config file: Policy version: Policy from config file:
enabled /selinux permissive permissive 24 default
Page 94
Labs
SELinux is in permissive mode. It means that all denials will be logged but not applied. e.i your system is running as it would without SELinux being enabled. Q: A: Show your current SELinux context. What does it means ? root@debian-selinux:~# id -Z unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 You currently have unconfined role and domain. It means that SELinux "rules" won't apply to you and to the process you'll run. Q: A: List all running processes and their security context. What do you notice ?
root@debian-selinux:~# ps aux -Z LABEL USER PID %CPU %MEM VSZ RSS TTY STAT START system_u:system_r:init_t:s0 root 1 0.1 0.2 2032 680 ? Ss 16:36 system_u:system_r:kernel_t:s0 root 2 0.0 0.0 0 0 ? S 16:36 system_u:system_r:kernel_t:s0 root 3 0.0 0.0 0 0 ? S 16:36 system_u:system_r:portmap_t:s0 daemon 780 0.0 0.1 1808 496 ? Ss 16:36 system_u:system_r:rpcd_t:s0 statd 792 0.0 0.3 1936 768 ? Ss 16:36 system_u:system_r:auditd_t:s0 root 949 0.0 0.3 11732 848 ? S<sl 16:36 system_u:system_r:audisp_t:s0 root 951 0.0 0.2 10052 736 ? S<sl 16:36 system_u:system_r:kernel_t:s0 root 953 0.0 0.0 0 0 ? S 16:36 system_u:system_r:syslogd_t:s0 root 955 0.0 0.6 27684 1756 ? Sl 16:36 system_u:system_r:apmd_t:s0 root 992 0.0 0.2 1704 592 ? Ss 16:36 system_u:system_r:crond_t:s0-s0:c0.c1023 daemon 1004 0.0 0.1 2160 416 ? Ss 16:36 system_u:system_r:sshd_t:s0-s0:c0.c1023 root 1433 0.0 0.3 5496 976 ? Ss 16:37 unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 root 1504 0.0 0.4 3872 1044 pts/0 R+
TIME COMMAND 0:01 init [2] 0:00 [kthreadd] 0:00 [migration/0] 0:00 /sbin/portmap 0:00 /sbin/rpc.statd 0:00 /sbin/auditd 0:00 /sbin/audispd 0:00 [kauditd] 0:00 /usr/sbin/rsyslogd -c4 0:00 /usr/sbin/acpid 0:00 /usr/sbin/atd 0:00 /usr/sbin/sshd 16:55 0:00 ps aux -Z
Note
Reduced output
Processes running in kernel space runs within the kernel_t SElinux domain whereas daemons runs with a dedicated domain. User (incl. root) processes runs in the unconfined domain. Q: A: Switch to the enforcing mode. root@debian-selinux:~# setenforce 1
Note
This won't survive a reboot.
7.2. Working with booleans

Q: List all available booleans.
Page 95
Labs
A:
root@debian-selinux:~# getsebool -a [...] user_direct_mouse --> off user_dmesg --> off user_manage_dos_files --> on user_ping --> off user_rw_noexattrfile --> off user_tcp_server --> off user_ttyfile_stat --> off The user_dmesg boolean is set to off/false. User should't be able to use the dmesg command. Try to run it as supinfo. Do you get the expected result? Why?
supinfo@debian-selinux:~$ dmesg | tail -n 4 [ 5.296247] type=1400 audit(1341585414.507:4): avc: denied { append } for pid=618 comm="mount" name="mtab" dev=sda1 ino=268024 scontext=system_u:system_r:mount_t:s0 tcontext=system_u:object_r:etc_t:s0 tclass=file [ 5.341905] loop: module loaded [ 6.352018] eth0: link up [ 17.482012] eth0: no IPv6 routers present
Q: A:
Although the user_dmesg boolean has been set to prevent users from using the dmesg command, you've no problem to run it as supinfo. Maybe the Unix user -> SELinux user mapping as something to do with that. Q: A: Check supinfo's current security context and existing Unix -> SELinux user mappings. What do you notice. supinfo@debian-selinux:~$ id -Z unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 root@debian-selinux:~# semanage login -l Login Name __default__ root system_u SELinux User unconfined_u unconfined_u system_u MLS/MCS Range s0-s0:c0.c1023 s0-s0:c0.c1023 s0-s0:c0.c1023
Both root and regular users (__default__) are mapped to the unconfined_u SELinux user. Therefore, from a SELinux perspective, regular users cannot be more restricted than root. Q: A: Q: A: Change the default user mapping from unconfined_u to user_u for the default policy.
root@debian-selinux:~# semanage login -m -S default -r s0 -s "user_u" __default__
Check your security context and try to dmesg again as supinfo (login/logout as needed). supinfo@debian-selinux:~$ id -Z user_u:user_r:user_t:s0 supinfo@debian-selinux:~$ dmesg klogctl: Permission denied The restriction now works as expected.
Q:
Turn the user_dmesg boolean on and try to dmesg again as supinfo.
Page 96
Labs
A:
root@debian-selinux:~# setsebool user_dmesg 1 supinfo@debian-selinux:~$ dmesg | tail -n 2 [ 3904.943279] SELinux: 6 users, 6 roles, 1209 types, 46 bools, 1 sens, 1024 cats [ 3904.943285] SELinux: 77 classes, 27184 rules
Q: A:
Do the same with the user_ping boolean.

supinfo@debian-selinux:~$ ping google.fr ping: icmp open socket: Permission denied supinfo@debian-selinux:~$ ping google.fr PING google.fr (173.194.67.94) 56(84) bytes of data. 64 bytes from wi-in-f94.1e100.net (173.194.67.94): icmp_req=1 ttl=128 time=36.0 ms
Q: A:
Try to su to root and to halt the system. What happens? Why? supinfo@debian-selinux:~$ su Password: root@debian-selinux:/home/supinfo# halt shutdown: warning: cannot open /var/run/shutdown.pid shutdown: /dev/initctl: Permission denied shutdown: cannot execute /sbin/init shutdown: /dev/initctl: Permission denied root@debian-selinux:/home/supinfo# id -Z user_u:user_r:user_t:s0 Even if the su command successuflly gives you id 0, you're still in the user_t context. This context cannot do much things, and especially not halting the system. You can still get the unconfined_t context by opening a session as root from the login prompt.
Q: A:
Change back the default mapping to unconfined_u.

root@debian-selinux:~# id -Z unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 root@debian-selinux:~# semanage login -m -S default -r s0 -s "unconfined_u" __default__
7.3. Using audit2allow

Be sure your system is in enforcing mode before going any further. Q: A: Q: A: Set the default user mapping to user_u.
root@debian-selinux:~# semanage login -m -S default -r s0 -s "user_u" __default__
On a second console, open a session as supinfo and su to root. Check you've user_r role. supinfo@debian-selinux:~$ su Password: root@debian-selinux:/home/supinfo# id -Z user_u:user_r:user_t:s0 On the first console erase blank off the /var/log/audit/audit.log file.
root@debian-selinux:/home/supinfo# cat /dev/null > /var/log/audit/audit.log
Q: A:
Page 97
Labs
Q: A: Disable all active dontaudit rules. This will make your upcoming task a lot easier. root@debian-selinux:~/local# semodule -DB Setup One: On the console were you're logged as root with the user_r role, try to halt the system using the /sbint/init 0 command. This will try to perform the appropriate syscalls and fail generating 'denied' avc log messages for any syscall that have not been allowed yet.
Q:
Note
You'll have to do this step more than once.
A:
root@debian-selinux:~# /sbin/init 0 bash: /sbin/init: Permission denied On the first console, use audit2allow to generate a set of rules allowing what has just been denied (halting the system). The rules belong to a module named local. Save the rules in a local.te file.
root@debian-selinux:~# audit2allow -m local -i /var/log/audit/audit.log > local.te
Q: A: Q: A:
Ensure the generated rules only refer to user_t domain processes halting the system. The local.te file should have the following content:
module local 1.0; require { type init_exec_t; type var_run_t; type syslogd_t; type initctl_t; type devlog_t; type user_t; class file { write getattr read unlink open execute execute_no_trans }; class capability { setuid dac_override }; class fifo_file { write open }; class sock_file write; class unix_dgram_socket sendto; class dir { write remove_name }; }
#============= user_t ============== allow user_t devlog_t:sock_file write; allow user_t init_exec_t:file { read execute open getattr execute_no_trans }; allow user_t initctl_t:fifo_file { write open }; allow user_t self:capability { setuid dac_override }; allow user_t syslogd_t:unix_dgram_socket sendto; allow user_t var_run_t:dir { write remove_name }; allow user_t var_run_t:file { read write getattr unlink open };
Page 98
Labs
Note
You won't get this result in the very first step. This is perfectly normal. New syscalls will become "available" as you allow them. If syscall B depends on syscall A success' it won't be tried until syscall A got allowed. You'll allow syscall A in the first run and syscall B in the second try, and so on. Q: A: Build the module into local.mod. root@debian-selinux:~# checkmodule -M -m -o local.mod local.te checkmodule: loading policy configuration from local.te checkmodule: policy configuration loaded checkmodule: writing binary representation (version 10) to local.mod Package the module into local.pp. root@debian-selinux:~# semodule_package -o local.pp -m local.mod Load the module package. root@debian-selinux:~# semodule -i local.pp Try to halt the system again from the console where you're running under the user_t domain. If it doesn't work, start again from step one. root@debian-selinux:/home/supinfo# id -Z user_u:user_r:user_t:s0 root@debian-selinux:/home/supinfo# /sbin/init 0
Q: A: Q: A: Q: A:
7.4. Creating a policy

In this section, you're going to create a new policy and set appropriate labels on executables and files to confine a process: The apache web server. Ensure that users are mapped to the unconfined domain before going any further. Refer to previous questions to change the default user mapping if necessary. Q: A: Install SELinux policy developpement packages. root@debian-selinux:~# apt-get install selinux-policy-dev
Note
This is Debian-Specific
Page 99
Labs
Q: A: Install Apache2 on your system. root@debian-selinux:~# apt-get install apache2
Note
This is Debian specific.
Q: A:
Ensure the service is running and check its process security context. What do you notice? Stop the service.
root@debian-selinux:~# /etc/init.d/apache2 restart root@debian-selinux:~# ps aux -Z | grep apache2 unconfined_u:unconfined_r:unconfined_t:s0 root 2808 0.0 1.0 unconfined_u:unconfined_r:unconfined_t:s0 www-data 2814 0.0 unconfined_u:unconfined_r:unconfined_t:s0 www-data 2815 0.0 unconfined_u:unconfined_r:unconfined_t:s0 www-data 2816 0.0 unconfined_u:unconfined_r:unconfined_t:s0 root 2873 0.0 0.2 root@debian-selinux:~# /etc/init.d/apache2 stop
5428 2708 ? Ss 0.7 5200 1832 ? S 0.8 226840 2284 ? Sl 0.9 226840 2304 ? Sl 3300 748 pts/0 S+
14:50 14:50 14:50 14:50 14:50
0:00 0:00 0:00 0:00 0:00
/usr/sbin/apache2 /usr/sbin/apache2 /usr/sbin/apache2 /usr/sbin/apache2 grep apache2
-k -k -k -k
start start start start
The service is running unconfined. Q: A: Use the policygentool command to generate a policy module called apache2 for the /usr/bin/apache2 binary.
root@debian-selinux:~/apache2# policygentool apache2 /usr/sbin/apache2
This tool generate three files for policy development, A Type Enforcement (te) file, a File Context (fc), and a Interface File(if). Most of the policy rules will be written in the te file. Use the File Context file to associate file paths with security context. Use the interface rules to allow other protected domains to interact with the newly defined domains. After generating these files use the /usr/share/selinux/POLICY-NAME/include/Makefile to compile your policy package. Then use the semodule tool to load it. # # # # # # /usr/bin/policygentool myapp /usr/bin/myapp echo 'HEADERDIR:=/usr/share/selinux/refpolicy-targeted/include' > Makefile echo 'include $(HEADERDIR)/Makefile' >> Makefile make semodule -l myapp.pp restorecon -R -v /usr/bin/myapp "all files defined in myapp.fc"
Now you can turn on permissive mode, start your application and avc messages will be generated. You can use audit2allow to help translate the avc messages into policy. # setenforce 0 # /etc/init.d/myapp start # audit2allow -R -i /var/log/audit/audit.log Return to continue: If the module uses pidfiles, what is the pidfile called? /var/run/apache2.pid If the module uses logfiles, where are they stored? /var/log/apache2(/.*)? If the module has var/lib files, where are they stored? Does the module have a init script? [yN] y Does the module use the network? [yN] y
Q:
Create a link (or copy) the Makefile supplied by the SELinux policy developpment package into the current directory. Use your package management system to find where is that Makefile.
Page 100
Labs
A: Q: A:
root@debian-selinux:~/apache2# ln -s /usr/share/doc/selinux-policy-dev/examples/Makefile
Compile and load the apache2 policy module you've created with policygentool. root@debian-selinux:~/apache2# make root@debian-selinux:~/apache2# semodule -i apache2.pp Relabel the actual apache2 service binary to apache2_exect_t.
root@debian-selinux:~# ls -lZ /usr/sbin/apache2 lrwxrwxrwx. 1 root root system_u:object_r:bin_t:s0 33 Jun 28 14:14 /usr/sbin/apache2 -> ../lib/apache2/mpm-worker/apache2 root@debian-selinux:~# chcon -t apache2_exec_t /usr/lib/apache2/mpm-worker/apache2 root@debian-selinux:~# ls -lZ /usr/lib/apache2/mpm-worker/apache2 -rwxr-xr-x. 1 root root system_u:object_r:apache2_exec_t:s0 406784 Apr 1 08:40 /usr/lib/apache2/mpm-worker/apache2
Q: A:
Or:
root@debian-selinux:/home/supinfo# chcon -t apache2_exec_t /usr/sbin/apache2 root@debian-selinux:/home/supinfo# ls -lZ /usr/lib/apache2/mpm-worker/apache2 -rwxr-xr-x. 1 root root system_u:object_r:apache2_exec_t:s0 406784 Apr 1 08:40 /usr/lib/apache2/mpm-worker/apache2 root@debian-selinux:/home/supinfo# ls -lZ /usr/sbin/apache2 lrwxrwxrwx. 1 root root system_u:object_r:bin_t:s0 33 Jun 28 14:14 /usr/sbin/apache2 -> ../lib/apache2/mpm-worker/apache2
Q: A:
Relabel anything below /var, in verbose mode. What do you notice?

root@debian-selinux:~/apache2# restorecon -vR /var/ restorecon reset /var/log/apache2 context system_u:object_r:var_log_t:s0->system_u:object_r:apache2_var_log_t:s0 restorecon reset /var/log/apache2/other_vhosts_access.log context user_u:object_r:var_log_t:s0->system_u:object_r:apache2_var_log_t:s0 restorecon reset /var/log/apache2/error.log context user_u:object_r:var_log_t:s0->system_u:object_r:apache2_var_log_t:s0 restorecon reset /var/log/apache2/access.log context user_u:object_r:var_log_t:s0->system_u:object_r:apache2_var_log_t:s0
Files got relabled according to context definitions from apache2.fc. Q: A: Restart the service and check the process context. What do you notice?
root@debian-selinux:~/apache2# /etc/init.d/apache2 restart root@debian-selinux:~/apache2# ps aux -Z | grep apache2 unconfined_u:system_r:apache2_t:s0 root 3071 0.0 1.0 5428 unconfined_u:system_r:apache2_t:s0 www-data 3076 0.0 0.7 5200 unconfined_u:system_r:apache2_t:s0 www-data 3077 0.0 0.8 226840 unconfined_u:system_r:apache2_t:s0 www-data 3078 0.0 0.9 226840
2708 1832 2284 2304
? ? ? ?
Ss S Sl Sl
14:56 14:56 14:56 14:56
0:00 0:00 0:00 0:00
/usr/sbin/apache2 /usr/sbin/apache2 /usr/sbin/apache2 /usr/sbin/apache2
-k -k -k -k
start start start start
The service is now running in the apache2_t confined domain. Q: A: Q: A: Set SELinux to enforcing mode. root@debian-selinux:~/apache2# setenforce 1 Try to restart apache2. Does it works? Why? root@debian-selinux:~/apache2# /etc/init.d/apache2 restart It'll fail, because some apache2 syscalls will got denied by SELinux. In the permissive mode, SELinux only logs denied action while in the enforcing mode it logs denied actions and actually denies them. Q: Use the avc log messages to add appropriate allow directives in the apache2.te file. This involves trying to start the service check denied actions from the logs, adding appropriate directives in the apache2.te file, rebuilding the policy, reloading it and trying to start the service again. Repeat until you've allowed all needed actions. audit2allow can also help you in this task. Here is a working apache2.te:
A:
Page 101
Labs
policy_module(apache2,1.0.0) require { type sysctl_t; type sysctl_kernel_t; } ######################################## # # Declarations # type apache2_t; type apache2_exec_t; domain_type(apache2_t) init_daemon_domain(apache2_t, apache2_exec_t) # pid files type apache2_var_run_t; files_pid_file(apache2_var_run_t) # log files type apache2_var_log_t; logging_log_file(apache2_var_log_t) ######################################## # # apache2 local policy # # Check in /etc/selinux/refpolicy/include for macros to use instead of allow rules. # Some common macros (you might be able to remove some) files_read_etc_files(apache2_t) libs_use_ld_so(apache2_t) libs_use_shared_libs(apache2_t) miscfiles_read_localization(apache2_t) ## internal communication is often done using fifo and unix sockets. allow apache2_t self:fifo_file { read write }; allow apache2_t self:unix_stream_socket create_stream_socket_perms; # pid file allow apache2_t apache2_var_run_t:file manage_file_perms; allow apache2_t apache2_var_run_t:sock_file manage_file_perms; allow apache2_t apache2_var_run_t:dir rw_dir_perms; files_pid_filetrans(apache2_t,apache2_var_run_t, { file sock_file }) # log files allow apache2_t apache2_var_log_t:file { create_file_perms append }; allow apache2_t apache2_var_log_t:sock_file create_file_perms; allow apache2_t apache2_var_log_t:dir { rw_dir_perms setattr }; logging_log_filetrans(apache2_t,apache2_var_log_t,{ sock_file file dir }) ## Networking basics (adjust to your needs!) sysnet_dns_name_resolve(apache2_t) corenet_tcp_sendrecv_all_if(apache2_t) corenet_tcp_sendrecv_all_nodes(apache2_t) corenet_tcp_sendrecv_all_ports(apache2_t) corenet_non_ipsec_sendrecv(apache2_t) corenet_tcp_connect_http_port(apache2_t) #corenet_tcp_connect_all_ports(apache2_t) ## if it is a network daemon, consider these: #corenet_tcp_bind_all_ports(apache2_t) #corenet_tcp_bind_all_nodes(apache2_t) allow apache2_t self:tcp_socket { listen accept }; #Manual entries allow apache2_t http_port_t:tcp_socket name_bind; allow apache2_t node_t:tcp_socket node_bind;
Page 102
Labs
allow allow allow allow allow allow apache2_t apache2_t apache2_t apache2_t apache2_t apache2_t self:capability { kill net_bind_service setgid setuid chown }; self:process { sigchld sigkill sigstop signull signal fork}; self:sem { create setattr read destroy write unix_write }; sysctl_t:dir {read search}; sysctl_kernel_t:file {read}; sysctl_kernel_t:dir {read search};
# Init script handling init_use_fds(apache2_t) init_use_script_ptys(apache2_t) domain_use_interactive_fds(apache2_t)
Q: A:
Try to get the default "It Works" content from the web server. Does it works? Why? root@debian-selinux:~/apache2# curl http://localhost/ <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN"> <html><head> <title>403 Forbidden</title> </head><body> <h1>Forbidden</h1> <p>You don't have permission to access / on this server.</p> <hr> <address>Apache/2.2.16 (Debian) Server at localhost Port 80</address> </body></html> It doesn't works. If you look at the avc log, you'll see that the apache2 process got denied access to the directories and files representing web content. This is perfectly normal: You didn't have authorized that yet.
Q: A:
Add a line in apache2.fc that assigns the apache2_sys_content_t context to all files in /var/www and below. Your apache2.fc file should look like this one:
# # # # apache2 executable will have: label: system_u:object_r:apache2_exec_t MLS sensitivity: s0 MCS categories: <none> -gen_context(system_u:object_r:apache2_exec_t,s0) gen_context(system_u:object_r:apache2_var_run_t,s0) gen_context(system_u:object_r:apache2_var_log_t,s0) gen_context(system_u:object_r:apache2_sys_content_t,s0)
/usr/sbin/apache2 /var/run/apache2.pid /var/log/apache2(/.*)? /var/www(/.*)?
Q: A:
Also edit apache2.te to allow it to browse and read apache2_sys_content_t tagged directories/files. Add the following to apache2.te: type apache2_sys_content_t; #content to serve allow apache2_t apache2_sys_content_t:dir {getattr search}; allow apache2_t apache2_sys_content_t:file {open read getattr};
Q:
Compile and load the policy. Relabel /var. Switch enforcing mode off and on before/after relabeling. You have to do that (mode switching) because you've only authorized the apache2_sys_contetn_t domain to manipulate /var/www. Even unconfined_t can't do anything on these files due to the lack of authorizations.
Page 103
Labs
The alternative to switching SELinux off/on before/after relabeling is to add explicit authorizations for contexts like unconfined_t and setfiles_t. A: root@debian-selinux:~/apache2# root@debian-selinux:~/apache2# root@debian-selinux:~/apache2# root@debian-selinux:~/apache2# Try to query to local web server again.
root@debian-selinux:~/apache2# curl http://localhost/ <html><body><h1>It works!</h1> <p>This is the default web page for this server.</p> <p>The web server software is running but no content has been added, yet.</p> </body></html>
make && semodule -i apache2.pp setenforce 0 restorecon -Rv /var/ setenforce 1
Q: A:
Page 104
Labs
8. Quality Of Service
8.1. Bandwidth fairness
In this section, you're in a scenario where your router provide internet access for clients in the private network. However, with the default configuration on client can get all the bandwith and starve others. For this scenario, you'll need three virtual machines, connected like in the following schema:
Prepare these three virtual machines with appropriate hostnames and network configuration. Note that the router will need two NIC: The first one plugged to the NAT'ed virtual network and the second one plugged to the hostonly virtual network.
8.1.1. Routing/NAT
Q: A: Enable routing on debian-router. root@router:~# echo 1 > /proc/sys/net/ipv4/ip_forward
Note
This won't survive a reboot. You can also edit the sysctl.conf file to ensure persistance.
Q: A: Q: A:
On Debian-router, create a rule that NAT all incoming traffic from the host-only network to the VMware NAT'ed virtual network.
root@router:~# iptables -t nat -A POSTROUTING -s 172.17.2.0/24 -j MASQUERADE
From the client machines, try to reach something on the internet (You might need to adjust DNS settings before doing so).
root@client1:~# echo "nameserver 8.8.8.8" > /etc/resolv.conf supinfo@client1:~$ ping google.com PING google.com (173.194.34.38) 56(84) bytes of data. 64 bytes from par03s03-in-f6.1e100.net (173.194.34.38): icmp_req=1 ttl=127 time=23.1 ms root@client2:~# echo "nameserver 8.8.8.8" > /etc/resolv.conf supinfo@client2:~$ ping google.com PING google.com (173.194.34.38) 56(84) bytes of data. 64 bytes from par03s03-in-f6.1e100.net (173.194.34.38): icmp_req=1 ttl=127 time=25.4 ms
Page 105
Labs
Q: A: On the router, add a traffic control rule that limits the bandwidth available to clients to 100 kbits. This will help simulate a low performance link and enlighten how a client can eat all the bandwidth.
root@router:~# tc qdisc add dev eth1 root handle 1: htb default 10 root@router:~# tc class add dev eth1 parent 1:0 classid 1:10 htb rate 100kbit ceil 100kbit
8.1.2. Natural bandwidth distribution

Q: A: On both clients, start a big HTTP download, like the full source code of the latest Linux kernel. What do you notice ?
supinfo@client1:~$ wget http://www.kernel.org/pub/linux/kernel/v3.0/testing/linux-3.5-rc6.tar.bz2 --2012-07-11 17:21:20-- http://www.kernel.org/pub/linux/kernel/v3.0/testing/linux-3.5-rc6.tar.bz2 Resolving www.kernel.org... 149.20.4.69, 149.20.20.133 Connecting to www.kernel.org|149.20.4.69|:80... connected. HTTP request sent, awaiting response... 200 OK Length: 80960577 (77M) [application/x-bzip2] Saving to: linux-3.5-rc6.tar.bz2.11 0% [
And:
supinfo@client2:~$ wget http://www.kernel.org/pub/linux/kernel/v3.0/testing/linux-3.5-rc6.tar.bz2 --2012-07-11 17:21:21-- http://www.kernel.org/pub/linux/kernel/v3.0/testing/linux-3.5-rc6.tar.bz2 Resolving www.kernel.org...
The first download uses all the available bandwidth: The second client can't even perform a responsive DNS lookup.
8.1.3. Fair bandwidth distribution

Q: A: Q: A: On the router, add a traffic control rule that adds fairness queuing with a hash function updated every 10 seconds.
root@router:~# tc qdisc add dev eth1 parent 1:10 handle 110: sfq perturb 10
Try to start a big download on both clients again and see what happens.
supinfo@client1:~$ wget http://www.kernel.org/pub/linux/kernel/v3.0/testing/linux-3.5-rc6.tar.bz2 --2012-07-11 17:29:09-- http://www.kernel.org/pub/linux/kernel/v3.0/testing/linux-3.5-rc6.tar.bz2 Resolving www.kernel.org... 149.20.20.133, 149.20.4.69 Connecting to www.kernel.org|149.20.20.133|:80... connected. HTTP request sent, awaiting response... 200 OK Length: 80960577 (77M) [application/x-bzip2] Saving to: ###linux-3.5-rc6.tar.bz2.12### 0% [
And:
supinfo@client2:~$ wget http://www.kernel.org/pub/linux/kernel/v3.0/testing/linux-3.5-rc6.tar.bz2 --2012-07-11 17:29:07-- http://www.kernel.org/pub/linux/kernel/v3.0/testing/linux-3.5-rc6.tar.bz2 Resolving www.kernel.org... 149.20.20.133, 149.20.4.69 Connecting to www.kernel.org|149.20.20.133|:80... connected. HTTP request sent, awaiting response... 200 OK Length: 80960577 (77M) [application/x-bzip2] Saving to: ###linux-3.5-rc6.tar.bz2.7### 0% [
Page 106
Labs
The bandwidth is now fairly shared by both clients.
8.2. Traffic Shaping

In this section, you're going to configure the router/gateway to shape traffic according to the following schema:
Remember that you're shapping outgoing traffic. For example the HTTP traffic shaping rule will by applied to requests made by clients, not incoming server response. Your router has two interfaces: One is connected to the internet and the other one is connected to the private client's network. By applying traffic control rules to the private network interface, you're limiting outgoing rate to that network and thus its downloading capacity. In this scenario, you want to throttle and control outgoing traffic to the internet. Therefore, you'll apply these rules to the interface connected to the internet.
8.2.1. Creating the tree

Q: A: Q: A: Q: A: Q: Delete any pre-existing root qdisc on eth0 iface. root@router:~# tc qdisc del dev eth0 root Apply a root HTB qdisc on eth0. This qdisc should have handle 1:0 and redirect traffic to the class having the 1:40 handle. root@router:~# tc qdisc add dev eth0 root handle 1:0 htb default 40 Add the 1:1 htb class to the tree, using values from the schema. This class represent the "full power" of the virtual uplink you're working with. Note that this link is symmetrical.
root@router:~# tc class add dev eth0 parent 1:0 classid 1:1 htb rate 4096 ceil 4096
Add the "interactive" class as well, with a priority of 1.
Page 107
Labs
A:
root@router:~# tc class add dev eth0 parent 1:1 classid 1:10 \ >htb rate 1024 ceil 2048 burst 32k prio 1
Q: A: Q: A:
Finally add the leaf of the tree branch: A SFQ queue discipline that will ensure fair bandwidth distribution between connections. root@router:~# tc qdisc add dev eth0 parent 1:10 handle 110 sfq Perform analog steps for other branches.Don't forget to increment priority when going from leftmost to rightmost tree branches.
root@router:~# tc class add dev eth0 parent >htb rate 256 ceil 4096 burst 512 prio 2 root@router:~# tc qdisc add dev eth0 parent root@router:~# tc class add dev eth0 parent >htb rate 2048 ceil 4096 burst 1024 prio 3 root@router:~# tc qdisc add dev eth0 parent root@router:~# tc class add dev eth0 parent >htb rate 768 ceil 4096 burst 1024 prio 4 root@router:~# tc qdisc add dev eth0 parent 1:1 classid 1:20 \ 1:20 handle 120 sfq 1:1 classid 1:30 \ 1:30 handle 130 sfq 1:1 classid 1:40 \ 1:40 handle 140 sfq
Q: A:
Use tc to show classes and qdiscs in your tree.

root@router:~# tc class show dev eth0 class htb 1:1 root rate 4096bit ceil 4096bit burst 1600b cburst 1600b class htb 1:10 parent 1:1 leaf 110: prio 1 rate 1024bit ceil 2048bit burst 32Kb cburst 1600b class htb 1:20 parent 1:1 leaf 120: prio 2 rate 256bit ceil 4096bit burst 512b cburst 1600b class htb 1:30 parent 1:1 leaf 130: prio 3 rate 2048bit ceil 4096bit burst 1Kb cburst 1600b class htb 1:40 parent 1:1 leaf 140: prio 4 rate 768bit ceil 4096bit burst 1023b cburst 1600b root@router:~# tc qdisc show dev eth0 qdisc htb 1: root refcnt 2 r2q 10 default 40 direct_packets_stat 467 qdisc sfq 110: parent 1:10 limit 127p quantum 1514b qdisc sfq 120: parent 1:20 limit 127p quantum 1514b qdisc sfq 130: parent 1:30 limit 127p quantum 1514b qdisc sfq 140: parent 1:40 limit 127p quantum 1514b
8.2.2. Marking packets

In the previous section, you've set a traffic control tree. However at the moment all packets are going to the default class (1:40). You know need to tell the system how to put packets into classes. This will be done using the firewall to mark packets and then tc rules to put packets having a given mark to a given class. Packet marks are integers. You're going to mark packets with the minor number of the class they belong. Q: A: Use iptable's connection tracking module to mark packets going to the DNS and NTP udp ports as "10".
root@router:~# iptables -t mangle -A POSTROUTING -o eth0 -p udp -m multiport \ >--dports 53,123 -j CONNMARK --set-mark 10
Q: A:
Do the same with packets going to the port 80: They must be marked 30.
root@router:~# iptables -t mangle -A POSTROUTING -o eth0 -p tcp \ >--dport 80 -j CONNMARK --set-mark 30
Q: A:
Add a rule that propagate connection marks to actual packets.

root@router:~# iptables -t mangle -A POSTROUTING -j CONNMARK --restore-mark
Page 108
Labs
Q: Now add a last-in-order rule that will mark (actual packet mark, not connection tracking mark) TCP packets having the ACK flag (and only this flag) set to 20. Only select "empty" (no data) packets: Their size is between 40 and 64 bytes.
root@router:~# iptables -t mangle -A POSTROUTING -p tcp \ >--tcp-flags URG,ACK,PSH,RST,SYN,FIN ACK -m length --length 40:64 -j MARK --set-mark 20
A:
8.2.3. Glue
You know have a traffic control tree with classes in one hand and packets marked with the minor of these classes in the other hand. The last step is to glue this together by add filters to the traffic control tree to redirect packets with a certain mark to the matching class. Attach all filters to the root (1:0) qdisc. Q: A: Q: A: Add a filter that uses the fw module to selects packets having the 10 mark and sends them to the 1:10 class. This filter has a priority of one.
root@router:~# tc filter add dev eth0 parent 1:0 protocol ip prio 1 handle 10 fw flowid 1:10
Add filters for the 2 remaining mark/classes. Don't forget to increment to filter priority for each new filter.
root@router:~# tc filter add dev eth0 parent 1:0 protocol ip prio 2 handle 20 fw flowid 1:20 root@router:~# tc filter add dev eth0 parent 1:0 protocol ip prio 3 handle 30 fw flowid 1:30
Page 109
Labs
9. IPSec
9.1. Host to host
In this chapter, you're going to setup a IPSec connection between two network hosts and experiment with various keying mecanisms. Your network setup will look like the following schema:
Prepare two virtual machines with appropriate netwok configuration (these hosts should be able to talk to each other) and hostnames.
9.1.1. Static Keying

Q: A: Install ipsec-tools on both virtual machines. root@debian-ipsec1:~# apt-get install ipsec-tools root@debian-ipsec2:~# apt-get install ipsec-tools
Note
This is Debian-specific.
Q: A:
Generate two 128 bit keys for AH hmac-md5 encryption.

root@debian-ipsec1:~# dd if=/dev/urandom count=1 bs=16 2>/dev/null | xxd -p 4e331c97f663cd11e4f9885995020588 root@debian-ipsec1:~# dd if=/dev/urandom count=1 bs=16 2>/dev/null | xxd -p a9e6d3bf3bc69d7ed89dc88b97b0e669
Q: A:
Generate two 192 bit keys for ESP 3des-cbc encryption.

root@debian-ipsec1:~# dd if=/dev/urandom count=1 bs=24 2>/dev/null | xxd -p cca621f764a640fb8efaa7527df2fdb6ad7163eb99dd299f root@debian-ipsec1:~# dd if=/dev/urandom count=1 bs=24 2>/dev/null | xxd -p 5f6cf7dabd5a5f7704a98979ee9511f8af3e02436960cb06
Q: A:
Configure IPSec to use these keys for communications between your two hosts. Don't write security policies yet, just declare these keys as back and forth communication keys between these machines. Add the following lines to /etc/ipsec-tools.conf, on both machines (replace IP addresses by those of your hosts):
Page 110
Labs
# AH SA's add 192.168.82.156 192.168.82.180 ah 0x200 -A hmac-md5 0x4e331c97f663cd11e4f9885995020588; add 192.168.82.180 192.168.82.156 ah 0x300 -A hmac-md5 0xa9e6d3bf3bc69d7ed89dc88b97b0e669; # ESP SA's add 192.168.82.156 192.168.82.180 esp 0x201 -E 3des-cbc 0xcca621f764a640fb8efaa7527df2fdb6ad7163eb99dd299f; add 192.168.82.180 192.168.82.156 esp 0x301 -E 3des-cbc 0x5f6cf7dabd5a5f7704a98979ee9511f8af3e02436960cb06; Q: A: Add security policies requiring all traffic (incoming and outgoing) to be encapsulated by IPSec in transport mode, using ESP+AH flavor. Edit /etc/ipsec-tools.conf as follows. On debian-ipsec1 (192.168.82.156): spdadd 192.168.82.156 192.168.82.180 any -P out ipsec esp/transport//require ah/transport//require; spdadd 192.168.82.180 192.168.82.156 any -P in ipsec esp/transport//require ah/transport//require; On debian-ipsec2 (192.168.82.180): spdadd 192.168.82.156 192.168.82.180 any -P in ipsec esp/transport//require ah/transport//require; spdadd 192.168.82.180 192.168.82.156 any -P out ipsec esp/transport//require ah/transport//require; Q: A: Ensure IPSec is not active on any host. root@debian-ipsec1:~# /etc/init.d/setkey stop Flushing IPsec SA/SP database: done. root@debian-ipsec2:~# /etc/init.d/setkey stop Flushing IPsec SA/SP database: done. Try to ping from one host to another while tcpdump'ing on the other. What do you notice? root@debian-ipsec1:~# ping 192.168.82.180 PING 192.168.82.180 (192.168.82.180) 56(84) bytes of data. 64 bytes from 192.168.82.180: icmp_req=1 ttl=64 time=3.64 ms 64 bytes from 192.168.82.180: icmp_req=2 ttl=64 time=0.430 ms 64 bytes from 192.168.82.180: icmp_req=3 ttl=64 time=0.292 ms 64 bytes from 192.168.82.180: icmp_req=4 ttl=64 time=0.314 ms
Q: A:
Page 111
Labs
On debian-ipsec2:
root@debian-ipsec2:~# tcpdump -i eth0 -n 'host 192.168.82.156' tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes 17:45:06.993143 ARP, Request who-has 192.168.82.180 tell 192.168.82.156, length 46 17:45:06.993247 ARP, Reply 192.168.82.180 is-at 00:0c:29:e9:6e:d7, length 28 17:45:06.993802 IP 192.168.82.156 > 192.168.82.180: ICMP echo request, id 4215, seq 1, length 64 17:45:06.993893 IP 192.168.82.180 > 192.168.82.156: ICMP echo reply, id 4215, seq 1, length 64 17:45:07.991638 IP 192.168.82.156 > 192.168.82.180: ICMP echo request, id 4215, seq 2, length 64 17:45:07.991696 IP 192.168.82.180 > 192.168.82.156: ICMP echo reply, id 4215, seq 2, length 64 17:45:08.990532 IP 192.168.82.156 > 192.168.82.180: ICMP echo request, id 4215, seq 3, length 64 17:45:08.990576 IP 192.168.82.180 > 192.168.82.156: ICMP echo reply, id 4215, seq 3, length 64 17:45:09.989538 IP 192.168.82.156 > 192.168.82.180: ICMP echo request, id 4215, seq 4, length 64 17:45:09.989591 IP 192.168.82.180 > 192.168.82.156: ICMP echo reply, id 4215, seq 4, length 64
The ICMP protocol can be read clearly over the wire. Q: A: Enable IPSec on both machines. root@debian-ipsec1:~# /etc/init.d/setkey start root@debian-ipsec2:~# /etc/init.d/setkey start Try to ping from one host to another while tcpdump'ing on the other. What do you notice? root@debian-ipsec1:~# ping 192.168.82.180 PING 192.168.82.180 (192.168.82.180) 56(84) bytes of data. 64 bytes from 192.168.82.180: icmp_req=1 ttl=64 time=0.403 64 bytes from 192.168.82.180: icmp_req=2 ttl=64 time=0.417 64 bytes from 192.168.82.180: icmp_req=3 ttl=64 time=0.877 64 bytes from 192.168.82.180: icmp_req=4 ttl=64 time=0.432 On debian-ipsec2:
root@debian-ipsec2:~# tcpdump -i eth0 -n 'host 192.168.82.156' tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes 17:48:52.358512 IP 192.168.82.156 > 192.168.82.180: AH(spi=0x00000200,seq=0x4): 17:48:52.358598 IP 192.168.82.180 > 192.168.82.156: AH(spi=0x00000300,seq=0x4): 17:48:53.357539 IP 192.168.82.156 > 192.168.82.180: AH(spi=0x00000200,seq=0x5): 17:48:53.357608 IP 192.168.82.180 > 192.168.82.156: AH(spi=0x00000300,seq=0x5): 17:48:54.356752 IP 192.168.82.156 > 192.168.82.180: AH(spi=0x00000200,seq=0x6): 17:48:54.356901 IP 192.168.82.180 > 192.168.82.156: AH(spi=0x00000300,seq=0x6): 17:48:55.357142 IP 192.168.82.156 > 192.168.82.180: AH(spi=0x00000200,seq=0x7): 17:48:55.357211 IP 192.168.82.180 > 192.168.82.156: AH(spi=0x00000300,seq=0x7):
Q: A:
ms ms ms ms
ESP(spi=0x00000201,seq=0x4), ESP(spi=0x00000301,seq=0x4), ESP(spi=0x00000201,seq=0x5), ESP(spi=0x00000301,seq=0x5), ESP(spi=0x00000201,seq=0x6), ESP(spi=0x00000301,seq=0x6), ESP(spi=0x00000201,seq=0x7), ESP(spi=0x00000301,seq=0x7),
length length length length length length length length
88 88 88 88 88 88 88 88
Packets are now encrypted.
9.1.2. Dynamic keying with racoon 9.1.2.1. Using a pre-shared key

Q: A: Remove the static keys you've added previously from IPSec configuration (on both machines). Don't remove security policies: Only remove security associations. Remove (or comment out) this part of /etc/ipsec-tools.conf:
Page 112
Labs
# AH SA's add 192.168.82.156 192.168.82.180 ah 0x200 -A hmac-md5 0x4e331c97f663cd11e4f9885995020588; add 192.168.82.180 192.168.82.156 ah 0x300 -A hmac-md5 0xa9e6d3bf3bc69d7ed89dc88b97b0e669; # ESP SA's add 192.168.82.156 192.168.82.180 esp 0x201 -E 3des-cbc 0xcca621f764a640fb8efaa7527df2fdb6ad7163eb99dd299f; add 192.168.82.180 192.168.82.156 esp 0x301 -E 3des-cbc 0x5f6cf7dabd5a5f7704a98979ee9511f8af3e02436960cb06; Q: A: Install racoon on both machines. root@debian-ipsec1:~# apt-get install racoon root@debian-ipsec1:~# apt-get install racoon On both machines, configure racoon with the following settings: IKE Settings (for all potential communication partners) : Main mode Use 3des for encryption and sha1 for hashing. Use pre shared keys found in /etc/racoon/psk.txt IPSec SA's settings (for all hosts). Same algorithms as for IKE. Enable compression A: Edit /etc/racoon/racoon.conf as follows (on both nodes): path pre_shared_key "/etc/racoon/psk.txt"; remote anonymous { exchange_mode main; proposal { encryption_algorithm 3des; hash_algorithm sha1; authentication_method pre_shared_key; dh_group 2; } } sainfo anonymous { encryption_algorithm 3des; authentication_algorithm hmac_sha1; compression_algorithm deflate ; }
Q:
Page 113
Labs
Q: A: Set the psk to use for communications between these hosts to "supinfoipsecaccess" (without quotes).
root@debian-ipsec1:~# echo "192.168.82.180 supinfoipsecaccess" >> /etc/racoon/psk.txt root@debian-ipsec2:~# echo "192.168.82.156 supinfoipsecaccess" >> /etc/racoon/psk.txt
Q: A:
Restart the whole IPSec userland stack on both machines.

root@debian-ipsec1:~# /etc/init.d/setkey restart && /etc/init.d/racoon restart root@debian-ipsec2:~# /etc/init.d/setkey restart && /etc/init.d/racoon restart
Q: A:
Try to ping from one host to another while tcpdump'ing on the other. root@debian-ipsec1:~# ping 192.168.82.180 PING 192.168.82.180 (192.168.82.180) 56(84) bytes of data. 64 bytes from 192.168.82.180: icmp_req=2 ttl=64 time=0.602 ms 64 bytes from 192.168.82.180: icmp_req=3 ttl=64 time=0.405 ms 64 bytes from 192.168.82.180: icmp_req=4 ttl=64 time=0.437 ms On debian-ipsec2:
root@debian-ipsec2:~# tcpdump -i eth0 -n 'host 192.168.82.156' tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes 18:32:01.578661 IP 192.168.82.156 > 192.168.82.180: AH(spi=0x0c401987,seq=0x4): 18:32:01.578770 IP 192.168.82.180 > 192.168.82.156: AH(spi=0x0c244255,seq=0x4): 18:32:02.577657 IP 192.168.82.156 > 192.168.82.180: AH(spi=0x0c401987,seq=0x5): 18:32:02.577733 IP 192.168.82.180 > 192.168.82.156: AH(spi=0x0c244255,seq=0x5): 18:32:03.576624 IP 192.168.82.156 > 192.168.82.180: AH(spi=0x0c401987,seq=0x6):
ESP(spi=0x051bcd76,seq=0x4), ESP(spi=0x0da4c8b8,seq=0x4), ESP(spi=0x051bcd76,seq=0x5), ESP(spi=0x0da4c8b8,seq=0x5), ESP(spi=0x051bcd76,seq=0x6),
length length length length length
100 100 100 100 100
9.1.2.2. Using X.509 certificates 9.1.2.2.1. Creating certificates

In this section, you're going to create two certificates/key pair, one for each host. You can directly use the openssl command or make use of the CA.pl script that comes with the openssl package, at your option. This document will illustrate the second option. Q: On one of the machines, create a /root/ca directory and call CA.pl -newca from that directory. You can get the full path to CA.pl by using your distro package management system. Type-in appropriate values when requested. You can refer to the OpenSSL lab of the "System Fundamentals" course.
Page 114
Labs
A:
root@debian-ipsec1:~# mkdir ca root@debian-ipsec1:~# cd ca root@debian-ipsec1:~/ca# /usr/lib/ssl/misc/CA.pl -newca CA certificate filename (or enter to create) Making CA certificate ... Generating a 1024 bit RSA private key ...++++++ ..++++++ writing new private key to './demoCA/private/cakey.pem' Enter PEM pass phrase: supinfo Verifying - Enter PEM pass phrase: supinfo ----You are about to be asked to enter information that will be incorporated into your certificate request. What you are about to enter is what is called a Distinguished Name or a DN. There are quite a few fields but you can leave some blank For some fields there will be a default value, If you enter '.', the field will be left blank. ----Country Name (2 letter code) [AU]:US State or Province Name (full name) [Some-State]:Utah Locality Name (eg, city) []:Provo Organization Name (eg, company) [Internet Widgits Pty Ltd]:Utopia Organizational Unit Name (eg, section) []: Common Name (eg, YOUR name) []:ca.utopia.net Email Address []:certmaster@utopia.net Please enter the following 'extra' attributes to be sent with your certificate request A challenge password []: An optional company name []: Using configuration from /usr/lib/ssl/openssl.cnf Enter pass phrase for ./demoCA/private/cakey.pem: supinfo Check that the request matches the signature Signature ok Certificate Details: Serial Number: c8:d9:f1:ca:a1:6a:bb:bd Validity Not Before: Jul 23 14:41:22 2012 GMT Not After : Jul 23 14:41:22 2015 GMT Subject: countryName = US stateOrProvinceName = Utah organizationName = Utopia commonName = ca.utopia.net emailAddress = certmaster@utopia.net X509v3 extensions: X509v3 Subject Key Identifier: 55:59:38:CB:31:43:EB:CA:56:01:6B:D1:D4:C8:02:4A:14:E1:34:94 X509v3 Authority Key Identifier: keyid:55:59:38:CB:31:43:EB:CA:56:01:6B:D1:D4:C8:02:4A:14:E1:34:94 DirName:/C=US/ST=Utah/O=Utopia/CN=ca.utopia.net/emailAddress=certmaster@utopia.net serial:C8:D9:F1:CA:A1:6A:BB:BD X509v3 Basic Constraints: CA:TRUE Certificate is to be certified until Jul 23 14:41:22 2015 GMT (1095 days) Write out database with 1 new entries Data Base Updated
Q:
Create a new signing request for debian-ipsec1 using CA.pl -newreq.
Page 115
Labs
A:
root@debian-ipsec1:~/ca# /usr/lib/ssl/misc/CA.pl -newreq Generating a 1024 bit RSA private key ........++++++ .....++++++ writing new private key to 'newkey.pem' Enter PEM pass phrase: supinfo Verifying - Enter PEM pass phrase: supinfo ----You are about to be asked to enter information that will be incorporated into your certificate request. What you are about to enter is what is called a Distinguished Name or a DN. There are quite a few fields but you can leave some blank For some fields there will be a default value, If you enter '.', the field will be left blank. ----Country Name (2 letter code) [AU]:US State or Province Name (full name) [Some-State]:Utah Locality Name (eg, city) []:Provo Organization Name (eg, company) [Internet Widgits Pty Ltd]:Utopia Organizational Unit Name (eg, section) []: Common Name (eg, YOUR name) []:debian-ipsec1 Email Address []: Please enter the following 'extra' attributes to be sent with your certificate request A challenge password []: An optional company name []: Request is in newreq.pem, private key is in newkey.pem
Q: A:
Sign the request using CA.pl -sign.

root@debian-ipsec1:~/ca# /usr/lib/ssl/misc/CA.pl -sign Using configuration from /usr/lib/ssl/openssl.cnf Enter pass phrase for ./demoCA/private/cakey.pem: Check that the request matches the signature Signature ok Certificate Details: Serial Number: c8:d9:f1:ca:a1:6a:bb:be Validity Not Before: Jul 23 14:45:58 2012 GMT Not After : Jul 23 14:45:58 2013 GMT Subject: countryName = US stateOrProvinceName = Utah localityName = Provo organizationName = Utopia commonName = debian-ipsec1 X509v3 extensions: X509v3 Basic Constraints: CA:FALSE Netscape Comment: OpenSSL Generated Certificate X509v3 Subject Key Identifier: 1E:5B:8F:12:22:E1:61:EA:59:26:26:BD:D3:A3:2B:5F:65:05:F8:40 X509v3 Authority Key Identifier: keyid:55:59:38:CB:31:43:EB:CA:56:01:6B:D1:D4:C8:02:4A:14:E1:34:94 Certificate is to be certified until Jul 23 14:45:58 2013 GMT (365 days) Sign the certificate? [y/n]:y
1 out of 1 certificate requests certified, commit? [y/n]y Write out database with 1 new entries Data Base Updated Signed certificate is in newcert.pem
Q:
Use the openssl command to read RSA-encrypted key from newkey.pem and output it in clear form to debian-ipsec1.key.
Page 116
Labs
A:
root@debian-ipsec1:~/ca# openssl rsa -in newkey.pem -out debian-ipsec1.key Enter pass phrase for newkey.pem: supinfo writing RSA key
Q: A:
Rename the certificate to debian-ipsec1.crt, and copy both key and certificate to the /etc/racoon/ certs directory (create it as needed). root@debian-ipsec1:~/ca# mv newcert.pem debian-ipsec1.crt root@debian-ipsec1:~/ca# mkdir /etc/racoon/certs root@debian-ipsec1:~/ca# cp debian-ipsec1.* /etc/racoon/certs Follow the same steps to create a certificate/key pair for debian-ipsec2.
Q:
Page 117
Labs
A:
root@debian-ipsec1:~/ca# /usr/lib/ssl/misc/CA.pl -newreq Generating a 1024 bit RSA private key .++++++ ...............................++++++ writing new private key to 'newkey.pem' Enter PEM pass phrase: supinfo Verifying - Enter PEM pass phrase: supinfo ----You are about to be asked to enter information that will be incorporated into your certificate request. What you are about to enter is what is called a Distinguished Name or a DN. There are quite a few fields but you can leave some blank For some fields there will be a default value, If you enter '.', the field will be left blank. ----Country Name (2 letter code) [AU]:US State or Province Name (full name) [Some-State]:Utah Locality Name (eg, city) []:Provo Organization Name (eg, company) [Internet Widgits Pty Ltd]:Utopia Organizational Unit Name (eg, section) []: Common Name (eg, YOUR name) []:debian-ipsec2 Email Address []: Please enter the following 'extra' attributes to be sent with your certificate request A challenge password []: An optional company name []: Request is in newreq.pem, private key is in newkey.pem root@debian-ipsec1:~/ca# /usr/lib/ssl/misc/CA.pl -sign Using configuration from /usr/lib/ssl/openssl.cnf Enter pass phrase for ./demoCA/private/cakey.pem: supinfo Check that the request matches the signature Signature ok Certificate Details: Serial Number: c8:d9:f1:ca:a1:6a:bb:bf Validity Not Before: Jul 23 14:54:41 2012 GMT Not After : Jul 23 14:54:41 2013 GMT Subject: countryName = US stateOrProvinceName = Utah localityName = Provo organizationName = Utopia commonName = debian-ipsec2 X509v3 extensions: X509v3 Basic Constraints: CA:FALSE Netscape Comment: OpenSSL Generated Certificate X509v3 Subject Key Identifier: 0D:3D:47:4D:9F:CA:BE:A4:52:EE:4F:FD:EF:7B:21:CD:BC:2A:89:CE X509v3 Authority Key Identifier: keyid:55:59:38:CB:31:43:EB:CA:56:01:6B:D1:D4:C8:02:4A:14:E1:34:94 Certificate is to be certified until Jul 23 14:54:41 2013 GMT (365 days) Sign the certificate? [y/n]:y
1 out of 1 certificate requests certified, commit? [y/n]y Write out database with 1 new entries Data Base Updated Signed certificate is in newcert.pem root@debian-ipsec1:~/ca# openssl rsa -in newkey.pem -out debian-ipsec2.key Enter pass phrase for newkey.pem: writing RSA key root@debian-ipsec1:~/ca# mv newcert.pem debian-ipsec2.crt
Page 118
Labs
Q: A: Copy the certificate/key pair to debian-ipsec2 /etc/racoon/certs.
root@debian-ipsec2:~# mkdir /etc/racoon/certs root@debian-ipsec1:~/ca# scp debian-ipsec2.* root@192.168.82.180:/etc/racoon/certs/ root@192.168.82.180's password: debian-ipsec2.crt debian-ipsec2.key
Q: A:
Copy the certification authority certificate to /etc/racoon/certs as well (on both machines).
root@debian-ipsec1:~/ca# cp demoCA/cacert.pem /etc/racoon/certs/CA.crt root@debian-ipsec1:~/ca# scp demoCA/cacert.pem root@192.168.82.180:/etc/racoon/certs/CA.crt root@192.168.82.180's password: cacert.pem
Q: A:
In the /etc/racoon/certs (on both machines), create a symlink key-hash.0 that links to the actual certificate. You can get the key hash by using the openssl x509 -noout -hash -in <filename> command.
root@debian-ipsec1:/etc/racoon/certs# ln -s CA.crt `openssl x509 -noout -hash -in CA.crt`.0 root@debian-ipsec2:/etc/racoon/certs# ln -s CA.crt `openssl x509 -noout -hash -in CA.crt`.0
9.1.2.2.2. Configuring racoon

Q: Configure racoon IKE to: Look for certificates in /etc/racoon/certs. Use certificate and keys you've created. Use the certificate common name as the identifier for both local and remote machines. Use a RSA signature as the authentication method. A: Edit /etc/racoon/racoon.conf as follows. On debian-ipsec1:
Page 119
Labs
path pre_shared_key "/etc/racoon/psk.txt"; path certificate "/etc/racoon/certs"; remote anonymous { exchange_mode main; certificate_type x509 "debian-ipsec1.crt" my_identifier asn1dn; peers_identifier asn1dn; proposal { encryption_algorithm 3des; hash_algorithm sha1; authentication_method rsasig; dh_group 2; } } sainfo anonymous { encryption_algorithm 3des; authentication_algorithm hmac_sha1; compression_algorithm deflate ; } Same configuration on debian-ipsec2 with debian-ipsec2.crt and debian-ipsec2.key instead of the above values. Q: A: Restart the whole IPSec userland stack on both machines.
"debian-ipsec1.key";
Q: A:
Try to ping from one host to another while tcpdump'ing on the other. root@debian-ipsec1:/etc/racoon# ping 192.168.82.180 PING 192.168.82.180 (192.168.82.180) 56(84) bytes of data. 64 bytes from 192.168.82.180: icmp_req=1 ttl=64 time=0.469 ms 64 bytes from 192.168.82.180: icmp_req=2 ttl=64 time=0.376 ms 64 bytes from 192.168.82.180: icmp_req=3 ttl=64 time=0.463 ms On debian-ipsec2:
root@debian-ipsec2:/etc/racoon# tcpdump -i eth0 -n 'host 192.168.82.156' tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes 19:16:02.587220 IP 192.168.82.156 > 192.168.82.180: AH(spi=0x036acca3,seq=0xc): 19:16:02.587335 IP 192.168.82.180 > 192.168.82.156: AH(spi=0x05900922,seq=0xc): 19:16:03.586149 IP 192.168.82.156 > 192.168.82.180: AH(spi=0x036acca3,seq=0xd): 19:16:03.586237 IP 192.168.82.180 > 192.168.82.156: AH(spi=0x05900922,seq=0xd):
ESP(spi=0x0d19434d,seq=0xc), ESP(spi=0x027aa409,seq=0xc), ESP(spi=0x0d19434d,seq=0xd), ESP(spi=0x027aa409,seq=0xd),
length length length length
100 100 100 100
9.2. Network to network

In this section, you're going to leverage your existing setup to make the host-to-host setup into a tunneled networkto-network configuration, as in the following schema:
Page 120
Labs
You'll need two more virtual machines and to alter you existing virtual machines configuration. Add a NIC to each one. This NIC must be connected to a host-only virtual network. The 172.17.1.0/24 and 172.17.2.0/24 networks must be separate virtual networks. Use the vmware virtual network editor to create enough dedicated host-only virtual networks. Make sure all virtual machines have the correct hostname and that debian-ipsec2 and debian-net2 can communicate over the 172.17.2.0/24 network. Do the same verifications for the 172.17.1.0/24 network. debian-net1 and debian-net2 should also have respectively debian-ipsec1 and debian-ipsec2 as their default gateway. The following questions assumes that all these networking settings have been correctly configured. Q: A: Enable packet routing on debian-ipsec1 and debian-ipsec2. root@debian-ipsec1:~# echo 1 > /proc/sys/net/ipv4/ip_forward root@debian-ipsec2:~# echo 1 > /proc/sys/net/ipv4/ip_forward On both IPSec endpoints, add SA's to specify that all traffic going from or to the 172.17.X.0/24 network behind the local or remote endpoint should go through the IPSec tunnel working in ESP mode. Edit /etc/ipsec-tools.conf as follows. On debian-ipsec1: spdadd 172.17.1.0/24 172.17.2.0/24 any -P out ipsec esp/tunnel/192.168.82.156-192.168.82.180/require; spdadd 172.17.2.0/24 172.17.1.0/24 any -P in ipsec esp/tunnel/192.168.82.180-192.168.82.156/require; On debian-ipsec2: spdadd 172.17.2.0/24 172.17.1.0/24 any -P out ipsec esp/tunnel/192.168.82.180-192.168.82.156/require; spdadd 172.17.1.0/24 172.17.2.0/24 any -P in ipsec esp/tunnel/192.168.82.156-192.168.82.180/require; Q: A: On both endpoints, stop the whole IPSec userland stack.
root@debian-ipsec1:~# /etc/init.d/setkey stop && /etc/init.d/racoon stop root@debian-ipsec2:~# /etc/init.d/setkey stop && /etc/init.d/racoon stop
Q: A:
Q:
From debian-net1, try to ping debian-net2. Does it works? Why?
Page 121
Labs
A:
supinfo@debian-net1:~$ ping 172.17.2.128 PING 172.17.2.128 (172.17.2.128) 56(84) bytes of data. It doesn't work, because even if debian-net1 as debian-ipsec1 as the default router, there is no route between debian-ipsec1 and debian-ipsec2 saying that debian-ipsec2 is a router for 172.17.2.0/24. Therefore debian-ipsec1 will send the packet to its default gateway which is not debian-ipsec2.
Q: A:
Restart the whole IPSec userland stack on both machines.

Q: A:
Try to ping again between debian-net1 and debian-net2. Does it works? Why? supinfo@debian-net1:~$ ping 172.17.2.128 PING 172.17.2.128 (172.17.2.128) 56(84) bytes of data. 64 bytes from 172.17.2.128: icmp_req=3 ttl=62 time=7.24 ms 64 bytes from 172.17.2.128: icmp_req=4 ttl=62 time=1.98 ms It works, because the packets are now forwarded to the other endpoint through the IPSec tunnel, despite the lack of routing table entry.
Page 122
Labs
10. RADIUS
In this Lab, you're going to setup a RADIUS authentication server and an AS to demonstrate its features. The AS will be a dd-wrt based router that will run a Chillispot captive portal. Users plugged to this router will automatically land on a connection page the first time they try to access a web page. They will enter their credentials and the router will try these credentials against the RADIUS server which will in turn grant or deny access depending on the credentials validity. The network topology will look like the following schema:
You'll need two "regular" (based on debian-master or your favorite Linux distro) virtual machines for debianradius and debian-netclient plus a special dd-wrt vmware image. You'll find this image on courses (http:// courses.supinfo.com/), along with other Linux resources. Prepare and configure these virtual machines as in the topology schema. As dd-wrt will distribute IP leases to clients and get its "WAN" ip from dhcp, disable VMware built-in dhcp for the host-only network and make sure it's enabled for the NAT'ed network. The dd-wrt virtual machine as two network interfaces already configured to be on the right network: The first interface is to be connected on the "WAN" (VMware NAT'ed vnet) and the second to the host-only network. Make sure these interfaces as connected as expected.
10.1. Setting up the RADIUS Server

10.1.1. RADIUS
Q: A: Install the freeradius package. root@debian-radius:~# apt-get install freeradius
Note
Q:
Configure freeradius to accpet clients from the VMware shared network with the "supinfo-radius" secret. Also set the same secret for the "localhost" client.
Page 123
Labs
A: Edit (add) /etc/freeradius/clients.conf as follows: client localhost { [...] secret = supinfo-radius [...] } client 192.168.82.0/24 { secret = supinfo-radius shortname = vmware-nated } Q: A: Add a user record to your radius flat file user database: The username is your Open Campus ID and the password is "supinfo". Edit (add) /etc/freeradius/users as follows: 40793 Q: A: Cleartext-Password := "supinfo"
Make sure the radius service is not running. Using two shells, run the service binary in the foreground in debug mode and do a test run using radtest. In a first shell: root@debian-radius:~# /etc/init.d/freeradius stop root@debian-radius:~# freeradius -X [...] Listening on proxy address * port 1814 Ready to process requests. In a second shell:
supinfo@debian-radius:~$ radtest 40793 supinfo localhost 0 supinfo-radius Sending Access-Request of id 228 to 127.0.0.1 port 1812 User-Name = "40793" User-Password = "supinfo" NAS-IP-Address = 127.0.1.1 NAS-Port = 0 rad_recv: Access-Accept packet from host 127.0.0.1 port 1812, id=228, length=20
Q: A:
Start the service. root@debian-radius:~# /etc/init.d/freeradius start
10.1.2. Chillispot CGI

Unfortunately, the chillispot version that comes with dd-wrt doesn'r fully run on the router: Only the deamon runs there. The other part, the login cgi, must be installed (along with a webserver) on another machine. For this Lab, I've choosen to run minimize the number of virtual machines and run the CGI on the same server as Radius. You'll now configure debian-radius to run that CGI. Q: Install the apache2 webserver.
Page 124
Labs
A:
root@debian-radius:~# apt-get install apache2
Note
Q: A:
Download and install the chillispot debian package from http://www.chillispot.info/ download/chillispot_1.0_i386.deb.
root@debian-radius:~# wget http://www.chillispot.info/download/chillispot_1.0_i386.deb root@debian-radius:~# dpkg -i chillispot_1.0_i386.deb
Q: A:
Use the hotspotlogin.cgi.gz file from the chillispot package (use your package system tools to locate the file) to create /var/www/cgi-bin/hotspotlogin.cgi (create directories as needed).
root@debian-radius:~# mkdir /var/www/cgi-bin root@debian-radius:~# zcat \ >/usr/share/doc/chillispot/hotspotlogin.cgi.gz > /var/www/cgi-bin/hotspotlogin.cgi
Q:
Edit the CGI script to set the secret that will be shared between the CGI and the remote chillispot deamon. This secret will be supinfo-uam. Also enable user-password RADIUS authentication. Just open the script with a text editor and read the 30+ first lines to know how to perform this task. Everything is in the comments. Edit(alter) the CGI as follows:
# Shared secret used to encrypt challenge with. Prevents dictionary attacks. # You should change this to your own shared secret. $uamsecret = "supinfo-uam"; # Uncomment the following line if you want to use ordinary user-password # for radius authentication. Must be used together with $uamsecret. $userpassword=1;
A:
Q: A:
Configure the default SSL vhost that come with the apache2 package (or create a new one, along with certificates) to serve CGI's from /var/www/cgi-bin. Edit(Alter) /etc/apache2/sites-available/default-ssl as follows: [...] ScriptAlias /cgi-bin/ /var/www/cgi-bin/ <Directory "/var/www/cgi-bin"> [...] </Directory> [...]
Q: A:
Enable appropriates modules and virtual hosts as needed. root@debian-radius:~# a2enmod ssl root@debian-radius:~# a2ensite default-ssl
Page 125
Labs
Note
Q: A: Q: A:
Restart the Apache service. root@debian-radius:~# /etc/init.d/apache2 restart Point a browser(curl works well for that) at the CGI, using HTTPS. You should get a message that says you can only login through the chillispot deamon.
root@debian-radius:~# curl -k https://192.168.82.156/cgi-bin/hotspotlogin.cgi <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> <html> <head> <title>ChilliSpot Login Failed</title> <meta http-equiv="Cache-control" content="no-cache"> <meta http-equiv="Pragma" content="no-cache"> </head> <body bgColor = '#c0d8f4'> <h1 style="text-align: center;">ChilliSpot Login Failed</h1> <center> Login must be performed through ChilliSpot daemon. </center> </body> </html>
10.2. Setting up dd-wrt

Start the dd-wrt virtual machine. If you've followed the previous instructions, dd-wrt has its second interface connected to the host-only virtual network. In its default configuration, dd-wrt will distribute IP addresses on that network. Either use your host operating system to get an IP address via DHCP on this interface (or manually assign the host-only virtual host iface an IP address within the 192.168.1.0/24 network), or use another virtual machine plugged on the host-only network. Once you've got an IP address, you can point your browser to http://192.168.1.1/. Q: A: Go to the dd-wrt web admin interface and follow Setup -> Basic Setup and set the WAN connection type to DHCP. Don't forget to Save & Apply Settings afterwise. Edit settings as follows:
Q:
Still from the dd-wrt web admin interface, go through Services -> Hotspot. In this page, enable Chillispot and fill the parameters as follows:
Page 126
Labs
Primary Radius Server: IP of debian-radius. DNS IP: IP of the VMware NAT'ed network DNS. You can find it on /etc/resolv.conf of any machine connected to the VMware NAT'ed network, such as debian-radius. Remote network: Network were the clients are going to be. This setting will alter dd-wrt configuration. It'll then provide DHCP leases for that network instead of the default 192.168.82.1.0/24. You can leave the default. Redirect URL: URL of the CGI login script you've configured in the previous part. Shared Key: Radius secret you've configured in section 1.1. UAM Secret: Shared secret between chillispot deamon and authentication CGI you've configured earlier. Don't forget to Save & Apply Settings afterwise. A: Edit settings as follows:
10.3. Tests with the client

Before going any further, make sure the client network interface is connected to the host-only network were ddwrt also is. Please note and keep in mind that you'll have to temporarily switch this interface connection to NAT if you need to access the internet before being able to authenticiate against chillispot (i.e install packages). Don't forget to switch the interface back to host-only afterwise. Also remember that you'll have to bring the iface up and down (or run the DHCP client manually) each time you switch the connection. Q: To test proper operation of this setup, you'll need a browser on the client. You can choose to install firefox (package name iceweasel in Debian) and run it through a SSH connection if you're running an X-capable host operating system, install a complete GUI environment on debian-netclient, or install a text-mode web broswser such as elinks. Make your choice and install appropriate packages. Be sure to have read the above instructions about network connectivity. Connect virtual network iface to NAT'ed network.
A:
Page 127
Labs
root@debian-netclient:~# ifdown eth0 && ifup eth0 root@debian-netclient:~# apt-get install iceweasel Connect virtual NIC back to host-only network. root@debian-netclient:~# ifdown eth0 && ifup eth0 Run firefox through ssh from host operating system. Q: A: Using a browser running on debian-netclient, try to reach www.google.com. What happens? Why? AS you're not identified yet, you are redirected to the chillispot login page:
Q: A:
Use your RADIUS credentials to open a session. What happens? You get a "session control window" that allows you do end it and you're redirected to the website you were trying to reach (in this case www.google.com).
Page 128
Labs
11. Puppet
In this lab, you're going to puppetize various services. You'll need two virtual machines. The first one will be your DNS server and your Puppet Master, the second one will be your Puppet-managed node. You'll be working with Bind9 and Puppet. These tools are not installed by default on all distros. Use your package management system to install them as needed. On Debian-based systems, package names are bind9, puppetmaster and puppet. Also set virtual machines hostnames to "puppetmaster" (DNS/Puppet Master) for the first one and "workstation" (Puppet-managed node) for the second one.
11.1. Preparing the DNS Server

Q: Configure your first virtual machine to act as a DNS server. This DNS server will host the utopia.net with two hosts: puppetmaster.utopia.net workstation.utopia.net Resolve these names to the virtual machine's IP addresses. A: Edit your DNS configuration as follows: zone "utopia.net" { type master; file "/etc/bind/utopia.net"; }; Create the /etc/bind/utopia.net zone file with the following content: $TTL @ 604800 IN
SOA
ns.utopia.net. root.utopia.net. ( 1 ; Serial 604800 ; Refresh 86400 ; Retry 2419200 ; Expire 604800 ) ; Negative Cache TTL NS A A A ns.utopia.net. 192.168.0.25 192.168.0.25 192.168.0.50
; @ ns puppetmaster workstation
IN IN IN IN
11.2. Working with Puppetmaster

Q: On the client, enter the puppet master FQDN into the adequate file.
Page 129
Labs
A:
root@workstation:~# vim /etc/puppet/puppet.conf [main] server=puppetmaster.utopia.net Generate (and sign) the SSL certificate needed by Puppet for the workstation node. root@workstation:~# puppetd --test --waitforcert 60
Q: A:
root@puppetmaster:~# puppetca --list root@puppetmaster:~# puppetca --sign workstation.utopia.net Q: A: Start the Puppet daemon on the workstation. On Debian-based systems, you might need to edit the /etc/ default/puppet file.
root@workstation:~# /etc/init.d/puppet start Starting puppet agent puppet not configured to start, please edit /etc/default/puppet to enable. root@workstation:~# echo "START=yes" > /etc/default/puppet root@workstation:~# /etc/init.d/puppet start
Enable Puppet files transfer by adding these two lines into /etc/puppet/manifests/site.pp (on the server): filebucket { main: server => puppetmaster.utopia.net } File { backup => main } Q: A: Create the directory structure for a module called editor, a module called ssh and another called apache.
root@puppetmaster:~# cd /etc/puppet/modules root@puppetmaster:~# mkdir -p {editor,ssh,apache}/{manifests,files,templates}
Q: A:
Write a Puppet program that makes sure the vim editor is installed, whereas nano has to be absent. This program is part of the editor module. root@puppetmaster:~# vim /etc/puppet/modules/editor/init.pp class ssh { package { "vim": ensure => installed } package { "nano": ensure => absent } }
Page 130
Labs
Q: A: Create a "basenode" node configuration spec in your Puppetmaster configuration. This node spec includes your editor module. root@puppetmaster:~# vim /etc/puppet/manifests/nodes.pp node basenode { include editor } Q: A: Alter your configuration to add a "workstation" node specification. This node spec inherits from the "basenode" spec. root@puppetmaster:~# vim /etc/puppet/manifests/nodes.pp node basenode { include editor } node workstation.utopia.net inherits basenode { } Q: Write a "ssh" Puppet class to manage SSH. This class must ensure SSH is installed in the latest version. SSH must be running and launched during system boot. The SSH package is required for the service : it must be explicitily specified. root@puppetmaster:~# vim /etc/puppet/modules/ssh/init.pp class ssh { package { "openssh-server": ensure => latest } service { "ssh": ensure => running, enale => true, hasstatus => true, hasrestart => true, require => Package[openssh-server] } } Q: Alter your configuration to manage SSH configuration using Puppet: Distribute a fixed default configuration file with the following items: X Forwarding enabled SSH v2 only Direct root login forbidden sshd listens on 2222
A:
Page 131
Labs
Be sure that any future change in this SSH configuration will result in a service restart. Of course, the SSH package is also required. A: Edit /etc/puppet/modules/ssh/init.pp as follows: class ssh { package { "openssh-server": ensure => latest } service { "ssh": ensure => running, enable => true, hasstatus => true, hasrestart => true, require => Package[openssh-server] file } { "/etc/ssh/sshd_config": owner => root, group => root, mode => 644, source => "puppet:///ssh/sshd_cfg", notify => Service['ssh'], require => Package['openssh-server'] }
} Create /etc/puppet/modules/ssh/files/sshd_config from a default sshd configuration file and alter it as follows: [...] Port 2222 Protocol 2 PermitRootLogin yes X11Forwarding yes [...] Q: A: Alter the configuration to include the ssh module in the workstation node spec. Restart the Puppetmaster and test your module on the node with puppetd --test. Edit /etc/puppet/manifests/nodes.pp as follows:
Page 132
Labs
node basenode { include editor } node workstation.utopia.net inherits basenode { include ssh }
root@puppetmaster:~# service puppetmaster restart
root@workstation:~# puppetd --test Q: Create a "apache" Puppet class to manage tha Apache webserver. It will ensure that apache is installed in the latest version and the daemon is launched. Imagine that you have also some servers using RedHat-based disitributions. You have to make sure that the package is installed and the daemon is launched, regardless of the script name. Use only one class. Edit /etc/puppet/modules/apache/init.pp as follows:
class apache { case $operatingsystem { debian, ubuntu: { $apache_name = 'apache2' } fedora, redhat: { $apache_name = 'httpd' } } package { "apache": name => $apache_name ensure => latest } service { "apache": name => $apache_name ensure => running, enable => true, hasstatus => true, hasrestart => true, require => Package[$apache_name] } }
A:
Q:
Imagine that your server has more than one network interface. You want your webserver to only honor requests from the eth0 interface. Use a Puppet template to distribute an Apache IP-based virtual host that "listens" on eth0 only.
Note
There is no need to cover all and every possible Apache install / configuration cases. Create a configuration that works on the distribution you're working on.
Page 133
Labs
A: Edit /etc/puppet/modules/apache/init.pp as follows:
class apache { case $operatingsystem { debian, ubuntu: { $apache_name = 'apache2' } fedora, redhat: { $apache_name = 'httpd' } } package { "apache": name => $apache_name ensure => latest } service { "apache": name => $apache_name ensure => running, enable => true, hasstatus => true, hasrestart => true, require => Package[$apache_name] } file { "/etc/apache2/sites-available/default": owner => root, group => root, mode => 644, content => template("apache/default.erb"), notify => Service['$apache_name'], require => Package['$apache_name'] } { "/etc/apache2/sites-enabled/default": ensure => link, target => '/etc/apache2/sites-available/default' }
file
Create the template in /etc/puppet/modules/apache/templates: <VirtualHost <%= ipaddress_eth0 %>:80> ServerAdmin webmaster@localhost Servername www.utopia.net DocumentRoot /var/www/ ErrorLog /var/log/apache2/error.log LogLevel warn CustomLog /var/log/apache2/access.log combined </VirtualHost> Q: A: Include the Apache module into the workstatio node spec. Restart the Puppetmaster and test your module on the node with puppetd --test. Edit /etc/puppet/manifests/nodes.pp as follows:
Page 134
Labs
node basenode { include editor } node workstation.utopia.net inherits basenode { include ssh include apache }
root@puppetmaster:~# service puppetmaster restart
root@workstation:~# puppetd --test
Page 135

02 - Workbook - Teachers Edition

Încărcat de

Informații document

Drepturi de autor

Formate disponibile

Partajați acest document

Partajați sau inserați document

Opțiuni de partajare

Vi se pare util acest document?

Este necorespunzător acest conținut?

Drepturi de autor:

Formate disponibile

02 - Workbook - Teachers Edition

Încărcat de

Drepturi de autor:

Formate disponibile

! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! !

)#'(*+,!-./! 0$(1!234$1#5!/67-878/-8! 9(#5!"#$%&#'(!:!;124#,1(! <21&+'5!;$=2#>!?9@00<! ! ! !

1.1. Common configuration

1.1.1.2. PKI Setup

Configure easy-rsa with the following values: Item Country Value US

Create a key/certificate for your client, vpn-client.

1.1.2.1. Host setup

Remove (or unplug) eth1 from your virtual machine.

1.2. Bridged VPN

1.2.1.1. Creating the bridge

1.2.2. Client configuration 1.2.2.1. Service configuration

root@vpn-client:~# cp /usr/share/doc/openvpn/examples/sample-config-files/client.conf /etc/openvpn/

Use Iface 0 tap0 0 eth0

1.2.4. Disabling automatic IP distribution

Use Iface 0 eth0

1.3. Routing VPN

1.3.1. Server configuration 1.3.1.1. Service setup

Edit /etc/openvpn/server.conf as follows:

1.3.2. Client configuration 1.3.2.1. Service configuration

Show the current routing table (no dns resolution).

Genmask 255.255.255.255 255.255.255.255 255.255.255.0 255.255.255.0

Iface tun0 tun0 tun0 eth0

2.1. Preliminary Setup

2.2. Squid as a caching proxy

http_access http_access http_access http_access http_access

2.3. Squid as a surrogate

3.2. Mail domains

3.2.1. Setting up DNS

; @ @ ns mail magnus imap pop3 smtp Q: A:

NS MX 10 A A A CNAME CNAME CNAME

3.2.2. Mail server configuration

3.2.3. Configuring a client 3.2.3.1. Network setup

3.2.3.2. Configuring the local smtpd

3.2.3.3. Sending mail to utopia.net

3.2.4. Remote access 3.2.4.1. IMAP

3.2.4.3. SMTP Authentication

3.2.5.1. Creating certificates

3.2.5.4. Setting up POP3 over SSL

Command >> 2 SquirrelMail Configuration : Read: config.php (1.4.0)

: trim(implode('', file('/etc/'.(file_exists('/e : false : SMTP : localhost:143 (courier) : localhost:25

: utopia.net : false : SMTP : localhost:143 (courier) : localhost:25

3.3. Virtual domains

3.3.1.1. Filesystem: Prepare virtual mailboxes

3.3.1.2.1. Virtual mailboxes mapping

# utopia.net domain supinfo@utopia.net bob@utopia.net sarah@utopia.net

utopia.net/supinfo/ utopia.net/bob/ utopia.net/sarah/

3.3.1.2.2. Server configuration

3.3.1.2.3. Testing the virtual domain

3.3.1.3.1. Courier authentication daemon

3.3.1.3.2. Virtual user accounts

3.3.1.3.3. Adjusting Cyrus-SASL options

Try to authenticate against the SMTPD as supinfo@utopia.net.

3.3.2. Adding virtual domains

; @ @ ns mail imap pop3 smtp Q: A:

NS MX 10 A A CNAME CNAME CNAME

root@mangus:~# mkdir /var/spool/mail/newdomain.com root@mangus:~# chown vmail:vmail /var/mail/newdomain.com

3.3.2.3. Postfix configuration

utopia.net/supinfo/ utopia.net/bob/ utopia.net/sarah/

newdomain.com/john/ newdomain.com/glenn/ newdomain.com/brian/

Set a random password for all these users.

Rebuild the user database root@mangus:~# makeuserdb

Try to read your mail as glenn using the webmail.

4.1. Simple iSCSI

4.1.1. Target setup

/usr/sbin/apache2 /var/run/apache2.pid /var/log/apache2(/.)? /var/www(/.)?