Lastest version of this article available here | Read other articles
Vserver is a kernel patch and a set of tools that allow virtualisation. Each process on a vserver system can operate in independent "contexts". Each context is a virtual server. The resources of the "host" kernel are used by each of the "guest" vservers, though they have no access to it or to each other.
Generally vserver applications run 1%-2% slower than on an unpatched kernel, though sometimes faster. This is a very low overhead for a virtualisation environment.
Any OS that can run under the host's kernel may be a guest, though the process here describes how to create Debian guests only. It is particularly important to follow the section on adding a vserver as described.
The configuration files for a vserver live in the directory /etc/vservers/<servername>/. The files that make up the server live in /var/lib/vservers/<servername>/. These directories contain the same file structure as the root directory tree (/) of any Unix server.
Debian ships with two packages: util-vserver and vserver-debiantools. The former is an official package from the vserver project and the latter packages up tools used to build, move and copy vservers that are specific to Debian. Some of the tools that can be found on a system once these two packages have been installed are not for use on Debian systems, but are part of the standard util-vserver package (a guest system's OS may differ from that of the host of course).
A hosted vserver is known as a guest and it may be built with one single command; for example:
newvserver --hostname example --domain example.net --ip 192.168.0.2 --dist etch --interface eth0 --context 2001
This essentially builds a basic Debian system inside /var/lib/vservers/example/ and removes the hardware-dependent parts. Use the -vsroot switch to set another place to install. You will be asked some of the usual Debian questions about the install via the installer's familiar ncurses interface.
Make sure you use a unique context for the -context switch; see Make the context of each server static below.
You will need to give your new server a unique IP. Note that this cannot be changed from inside the server once set. To find which addresses are already in use by the vservers already on the system use:
cat /etc/vservers/*/interfaces/*/ip | sort
Once the guest vserver has been set up, you may use the following commands:
Dynamic contexts are deprecated in vserver 2.2 (the version in Debian is 2.0.3) and static contexts are essential for disc quotas. Given that the servers will be updated at some point and that giving static context to each vserver makes for easier maintenance, we assign an explicit context to each guest. If we do not do this with the -context switch when the guest is built, or if the guest is being migrated from another system, you can set the context manually with the following command:
echo "<XID>" > /etc/vservers/<servername>/context
Valid static contexts are numbers between 2 and 49151. To find out what context numbers already exist use:
cat /etc/vservers/*/context | sort
You'll probably want to SSH in to your guest server. There are a number of methods of doing this. The most familiar way would be to enter the server then use apt-get install ssh. This will work fine. Configure as normal after that.
/etc/default/util-vserver controls which vservers are started at boot time. By default all servers marked 'default' are run. To mark a server default, put 'default' in the appropriate file:
echo "default" > cat /etc/vservers/<servername>/apps/init/mark
The server will now be started automatically at boot.
To delete a vserver, stop it and then remove the files:
rm -rf /var/lib/vservers/<servername>
rm -rf /etc/vservers/<servername>
The command vserver-stat gives an overview of the state of play for all running vservers. Here is some sample output (the CTX shows now deprecated dynamically allocated XIDs):
server:~# vserver-stat
CTX PROC VSZ RSS userTIME sysTIME UPTIME NAME
0 53 97.5M 36.1M 1d03h39 3h31m59 142d07h01 root server
49152 17 226.9M 65.1M 19m00s65 6m34s44 142d07h01 pod
49153 14 246.5M 61.5M 2m11s77 0m58s80 142d07h01 shoot
49154 36 553.1M 276.9M 2h05m52 12m52s86 142d07h01 root
49155 12 140.5M 33.7M 0m05s41 0m17s94 142d07h01 leaf
49156 7 73.9M 16.8M 14m33s93 3m41s74 142d07h01 bark
49159 21 317.4M 74.7M 12m31s22 2m49s37 28d06h41 bud
49160 3 8.5M 2.4M 0m00s12 0m00s20 1d02h08 anther
Moreover, some information is also gained using the vserver command:
server:~# vserver pod status
Vserver 'pod' is running at context '49152'
Number of processes: 23
Uptime: 142 days, 07:10
There are other tools that can be used to find out about a guest's state in more details. The first column in the output above labelled CTX is the context ID, often referred to as the XID (and referred to as such in this document). A space in /proc gives details of each context by its XID:
server:~# ls /proc/virtual/49152/
cacct cvirt info limit sched status
Inside each of these context directories is a set of files:
server:~# ls /proc/virtual/
49152 49153 49154 49155 49156 49159 49160 info
These contain a huge amount of information about the state of the guest, for example:
server:~# cat /proc/virtual/49152/cvirt
BiasUptime: 19.01
SysName: Linux
NodeName: pod
Release: 2.6.17-2-vserver-686
Version: #1 SMP Wed Sep 13 18:41:34 UTC 2006
Machine: i686
DomainName: (none)
nr_threads: 23
nr_running: 0
nr_unintr: 0
nr_onhold: 0
load_updates: 346952845
loadavg: 0.00 0.00 0.00
total_forks: 331952
The util-vserver package also provides a set of tools mirroring the basic system tools, but have a v in front. For example vps -ef will list all the processes in all contexts on the system, and which guest is running them. vkill allows cross-context killing of processes. vtop gives a table of processes across all contexts. There are others. We'll demonstrate some handy ones later.
This section contains some tips about maintaining versevers.
Make the following considerations when maintaining the hosts:
To upgrade guests without needing to enter them vapt-get is used. For example, to install the ssh package on a guest vserver use:
vapt-get <servername> -- install ssh
To update every running guest on the host use:
vapt-get --all -- update vapt-get --all -- -y dist-upgrade
The default size for a vserver's /tmp is 16MB. This is often not enough. To change its size without stopping a vserver, use a command like this:
vnamespace -e <XID> mount -t tmpfs -o remount,size=256m,mode=1777 none /var/lib/vservers/<servername>/tmp/
To set this permanently alter the entry in /etc/vservers/<servername>/fstab.
In order to move a vserver guest from one host to another do the following:
This translates to the following set of actions for an example of moving a vserver called 'exampleguest' from host1.example.com to host2.example.com. The guests are stored in /vserver/ on one server and /home/vservers/ on the other. A temporary secure key was set up to allow the rsync to take place. All the following commands were entered on examplehost2.example.com:
Stop the vserver:
vserver exampleguest stop
Copy the guest's files then the guest's configuration files:
rsync --numeric-ids -ac host1.example.com:/vserver/exampleguest/ /var/lib/vservers/examplehost/
rsync -acv host1.example.com:/etc/vservers/exampleguest/ /etc/vservers/exampleguest/
Make any changes necessary to the configuration files in keeping with the setup on the new host. In this case the vdir link needs to be changed:
cd /etc/vservers/exampleguest
rm vdir
ln -s /var/lib/vservers/exampleguest vdir
and the interface needed to be changed from eth1 to eth0:
echo "eth0" > /etc/vservers/exampleguest/interfaces/0/dev
and a context ID assigned:
echo 2002 > /etc/vservers/exampleguest/context
Start the server, enter the server and ping something (ARP):
vserver exampleguest start
vserver exampleguest enter
ping google.co.uk
And we're done! :)
Sample result from moving the configuration files:
server:~# rsync -acv host1.example.com:/etc/vservers/exampleguest/ /etc/vservers/exampleguest/
Debian GNU/Linux 4.0
receiving file list ... done
created directory /etc/vservers/exampleguest
./
cache -> /etc/vservers/.defaults/cachebase/exampleguest
fstab
name
run -> /var/run/vservers/exampleguest
vdir -> /vserver/exampleguest
apps/
apps/init/
apps/init/mark
apps/pkgmgmt/
apps/pkgmgmt/internal
cpuset/
interfaces/
interfaces/0/
interfaces/0/dev
interfaces/0/ip
interfaces/0/prefix
uts/
uts/nodename
This is possible, and in fact the vserver 'exampleguest' in our example was moved in from a 32-bit server to a 64-bit one with no difficultly. The guest will continue to be a 32-bit system.
32-bit guests run faster on a 64-bit host than they would on a system with a 32-bit kernel.
Useful per context quotas can be set with 'rlimits'. They can be set permanently in /etc/vservers/<servername>/rlimits/ or immediately with the vlimit command. Current active limits can be viewed in the guests /proc/virtual/<XID>/limit file. The default is to have no limits, as shown below, -1 being equivalent to no limit:
server:~# cat /proc/virtual/2001/limit
PROC: 9 16 -1 0
VM: 127314 140118 -1 0
VML: 0 0 -1 0
RSS: 2894 4104 -1 0
ANON: 6797 6797 -1 0
FILES: 139 139 -1 0
OFD: 90 90 -1 0
LOCKS: 1 1 -1 0
SOCK: 3 3 -1 0
MSGQ: 0 0 -1 0
SHM: 0 0 -1 0
Memory quotas on guest servers are set in terms of a maximum number of pages. The page size for the AMD64/EM64T architectures is 4096 bytes (4K). Thus setting a limit of 20,000 pages would limit memory usage to 80MB. The most useful size to set is the Resident Set Size (RSS) which limits the total RAM (in or out of swap) a guest can use. The following sets this the RSS limit for context 2001:
server:/# vlimit -c 2001 --rss 20000
Let's see if that's worked:
server:/# cat /proc/virtual/2001/limit | grep RSS
RSS: 2894 4104 20000 0
This shows that this context has a current usage of 2,894 pages, a maximum observed usage of 4,104 pages, and a hard limit of 20,000 pages. The final column shows that this limit has not yet been reached. Cool. To set this permanently (i.e. when the guest starts up) do:
echo 20000 > /etc/vservers/<servername>/rlimits/rss
Setting the 'nproc' limit for a guest means it cannot create more than that many processes. The following sets the PROC limit for context 2001 to 600 processses:
server:/# vlimit -c 2001 --nproc 600
Let's see if that's worked:
server:~# cat /proc/virtual/2001/limit | grep PROC PROC: 9 16 600 0
To set this same limit when a vserver is started:
echo 600 > /etc/vservers/2001/rlimits/nproc
600 is a lot of processes and a reasonable limit. This will stop fork bombs from trashing a system. If a guest experiences a fatal error that causes it to hang it may be internally inoperable, but can still be shut down. If an application with lots of memory is called recursively, the system may still break. So, combined with an RSS limit the PROC limit should ensure the host system is armoured against fork bombs and run away applications.
Of 2 servers test under heavy load, our straw poll found one server with 87 processes and one with 156.
In addition to 'rss' and 'nproc' other limits can be set; here is the whole set:
| Name in /proc/virtual/<XID>/limit | Name in /etc/vserver/<servername>/rlimits/ | Controls maximum |
|---|---|---|
| PROC | nproc | number of processes |
| VM | as | virtual memory |
| VML | memlock | locked virtual memory |
| RSS | rss | resident memory |
| FILES | nofile | open files |
| LOCKS | locks | file locks |
Disc limits are set in a completely different way. Each file in a guests tree is tagged with the XID. The files with this XID tag can then be assigned a quota. For this to work the partition where the vserver guest live must be mounted with the option 'tagxid' to allow this. Remounting this partition live does not work, so unless you can completely disconnect the partition by umounting then remount, you will need to reboot.
There is a handy script to allow guests to be assigned quotas in a simple way available at http://people.linux.org.tw/~andrew/DiskLimit.sh:
DiskLimit <servername>
Gives the current disc usage of the guest, and:
DiskLimit <servername> <quota>
Limits that guest to quota Kilobytes of disc space.
The 'DiskLimit' script does not set the inode quota of the guest in any useful way, just the maximum total space.
This quota will not survive a reboot. In order to do that some settings must be added to /etc/vserver/<servername>/dlimits/. This example set of command below sets a limit of 5Gb space, 100000 inodes and 5 reserved inodes:
mkdir -p /etc/vservers/<servername>/dlimits/0
echo /var/lib/vservers/<servername> > /etc/vservers/<servername>/dlimits/0/directory
echo $(( 5 * 1024 * 1024 )) > /etc/vservers/<servername>/dlimits/0/space_total
echo 100000 > /etc/vservers/<servername>/dlimits/0/inodes_total
echo 5 > /etc/vservers/<servername>/dlimits/0/reserved
Guests cannot access 127.0.0.1, so some programs need to be made to bind to the guests IP address. Most programs just work. If you really need a 127.0.0.1 see the link below:
http://linux-vserver.org/Problematic_Programs#127.0.0.1_issues
Author: Ben Green (ben at bristolwireless dot net) with editing by Mike Harris (mike at psand dot net) and corrections by Steve Woods (woodsy at bristolwireless dot net)
Copyright (c) 2007 Psand Limited and the respective authors. Permission is granted to copy, distributed and/or modify this document under the terms of the GNU Free Documentation License, Version 1.1 or any later version published by the Free Software Foundation; with no Invariant Sections, with no Front-Cover Texts, and with no Back-Cover Texts. A copy of the license is included in the section entitled "GNU Free Documentation License".
All trademarks are those of their respective trademark holders.