Installing, configuring and running Vservers on Debian GNU/Linux (Etch)

Lastest version of this article available here | Read other articles

A basic introduction to vservers

Vserver is a kernel patch and a set of tools that allow virtualisation. Each process on a vserver system can operate in independent "contexts". Each context is a virtual server. The resources of the "host" kernel are used by each of the "guest" vservers, though they have no access to it or to each other.

Generally vserver applications run 1%-2% slower than on an unpatched kernel, though sometimes faster. This is a very low overhead for a virtualisation environment.

Any OS that can run under the host's kernel may be a guest, though the process here describes how to create Debian guests only. It is particularly important to follow the section on adding a vserver as described.

Where do the vservers live?

The configuration files for a vserver live in the directory /etc/vservers/<servername>/. The files that make up the server live in /var/lib/vservers/<servername>/. These directories contain the same file structure as the root directory tree (/) of any Unix server.

Adding a new vserver "guest" virtual server

Debian ships with two packages: util-vserver and vserver-debiantools. The former is an official package from the vserver project and the latter packages up tools used to build, move and copy vservers that are specific to Debian. Some of the tools that can be found on a system once these two packages have been installed are not for use on Debian systems, but are part of the standard util-vserver package (a guest system's OS may differ from that of the host of course).

A hosted vserver is known as a guest and it may be built with one single command; for example:

newvserver  --hostname example --domain --ip --dist etch --interface eth0 --context 2001

This essentially builds a basic Debian system inside /var/lib/vservers/example/ and removes the hardware-dependent parts. Use the -vsroot switch to set another place to install. You will be asked some of the usual Debian questions about the install via the installer's familiar ncurses interface.

Make sure you use a unique context for the -context switch; see Make the context of each server static below.

You will need to give your new server a unique IP. Note that this cannot be changed from inside the server once set. To find which addresses are already in use by the vservers already on the system use:

cat /etc/vservers/*/interfaces/*/ip | sort

Once the guest vserver has been set up, you may use the following commands:

Make the context of each vserver static

Dynamic contexts are deprecated in vserver 2.2 (the version in Debian is 2.0.3) and static contexts are essential for disc quotas. Given that the servers will be updated at some point and that giving static context to each vserver makes for easier maintenance, we assign an explicit context to each guest. If we do not do this with the -context switch when the guest is built, or if the guest is being migrated from another system, you can set the context manually with the following command:

echo "<XID>" > /etc/vservers/<servername>/context

Valid static contexts are numbers between 2 and 49151. To find out what context numbers already exist use:

cat /etc/vservers/*/context | sort

Adding the SSH service

You'll probably want to SSH in to your guest server. There are a number of methods of doing this. The most familiar way would be to enter the server then use apt-get install ssh. This will work fine. Configure as normal after that.

Making vservers start at boot time

/etc/default/util-vserver controls which vservers are started at boot time. By default all servers marked 'default' are run. To mark a server default, put 'default' in the appropriate file:

echo "default" > cat /etc/vservers/<servername>/apps/init/mark

The server will now be started automatically at boot.

Deleting a vserver

To delete a vserver, stop it and then remove the files:

rm -rf /var/lib/vservers/<servername>
rm -rf /etc/vservers/<servername>

Obtaining information on running vservers

The command vserver-stat gives an overview of the state of play for all running vservers. Here is some sample output (the CTX shows now deprecated dynamically allocated XIDs):

	server:~# vserver-stat
	0       53  97.5M  36.1M   1d03h39   3h31m59 142d07h01 root server
	49152   17 226.9M  65.1M  19m00s65   6m34s44 142d07h01 pod
	49153   14 246.5M  61.5M   2m11s77   0m58s80 142d07h01 shoot
	49154   36 553.1M 276.9M   2h05m52  12m52s86 142d07h01 root
	49155   12 140.5M  33.7M   0m05s41   0m17s94 142d07h01 leaf
	49156    7  73.9M  16.8M  14m33s93   3m41s74 142d07h01 bark
	49159   21 317.4M  74.7M  12m31s22   2m49s37  28d06h41 bud
	49160    3   8.5M   2.4M   0m00s12   0m00s20   1d02h08 anther

Moreover, some information is also gained using the vserver command:

server:~# vserver pod status
Vserver 'pod' is running at context '49152'
Number of processes:  23
Uptime:               142 days, 07:10

There are other tools that can be used to find out about a guest's state in more details. The first column in the output above labelled CTX is the context ID, often referred to as the XID (and referred to as such in this document). A space in /proc gives details of each context by its XID:

server:~# ls /proc/virtual/49152/
cacct  cvirt  info  limit  sched  status

Inside each of these context directories is a set of files:

server:~# ls /proc/virtual/
49152  49153  49154  49155  49156  49159  49160  info

These contain a huge amount of information about the state of the guest, for example:

	server:~# cat /proc/virtual/49152/cvirt
	BiasUptime:     19.01
	SysName:        Linux
	NodeName:       pod
	Release:        2.6.17-2-vserver-686
	Version:        #1 SMP Wed Sep 13 18:41:34 UTC 2006
	Machine:        i686
	DomainName:     (none)
	nr_threads:     23
	nr_running:     0
	nr_unintr:      0
	nr_onhold:      0
	load_updates:   346952845
	loadavg:        0.00 0.00 0.00
	total_forks:    331952

The util-vserver package also provides a set of tools mirroring the basic system tools, but have a v in front. For example vps -ef will list all the processes in all contexts on the system, and which guest is running them. vkill allows cross-context killing of processes. vtop gives a table of processes across all contexts. There are others. We'll demonstrate some handy ones later.

Vserver maintenance

This section contains some tips about maintaining versevers.

Maintaining the host

Make the following considerations when maintaining the hosts:

Maintaining the guest

Updating and upgrading guest vservers

To upgrade guests without needing to enter them vapt-get is used. For example, to install the ssh package on a guest vserver use:

vapt-get <servername> -- install ssh

To update every running guest on the host use:

vapt-get --all -- update
vapt-get --all -- -y dist-upgrade

Increasing /tmp size

The default size for a vserver's /tmp is 16MB. This is often not enough. To change its size without stopping a vserver, use a command like this:

vnamespace -e <XID> mount -t tmpfs -o remount,size=256m,mode=1777 none /var/lib/vservers/<servername>/tmp/

To set this permanently alter the entry in /etc/vservers/<servername>/fstab.

Migrating vservers

In order to move a vserver guest from one host to another do the following:

  1. Stop the vserver
  2. Copy the files across exactly
  3. Copy the configuration files
  4. Make any necessary changes to the configuration files in keeping with the set-up on the new host
  5. Start the server
  6. Enter the server
  7. Ping something on the WWW to force all the relevant router to update their ARP tables.

This translates to the following set of actions for an example of moving a vserver called 'exampleguest' from to The guests are stored in /vserver/ on one server and /home/vservers/ on the other. A temporary secure key was set up to allow the rsync to take place. All the following commands were entered on

Stop the vserver:

vserver exampleguest stop

Copy the guest's files then the guest's configuration files:

rsync --numeric-ids -ac /var/lib/vservers/examplehost/
rsync -acv /etc/vservers/exampleguest/

Make any changes necessary to the configuration files in keeping with the setup on the new host. In this case the vdir link needs to be changed:

cd /etc/vservers/exampleguest
rm vdir
ln -s /var/lib/vservers/exampleguest vdir

and the interface needed to be changed from eth1 to eth0:

echo "eth0" > /etc/vservers/exampleguest/interfaces/0/dev

and a context ID assigned:

echo 2002 > /etc/vservers/exampleguest/context

Start the server, enter the server and ping something (ARP):

vserver exampleguest start
vserver exampleguest enter

And we're done! :)

Sample result from moving the configuration files:

	server:~# rsync -acv /etc/vservers/exampleguest/
	Debian GNU/Linux 4.0
	receiving file list ... done
	created directory /etc/vservers/exampleguest
	cache -> /etc/vservers/.defaults/cachebase/exampleguest
	run -> /var/run/vservers/exampleguest
	vdir -> /vserver/exampleguest

32-bit (x86) guests files on 64-bit (x86_64) hosts

This is possible, and in fact the vserver 'exampleguest' in our example was moved in from a 32-bit server to a 64-bit one with no difficultly. The guest will continue to be a 32-bit system.

32-bit guests run faster on a 64-bit host than they would on a system with a 32-bit kernel.

Setting quotas on vservers

Useful per context quotas can be set with 'rlimits'. They can be set permanently in /etc/vservers/<servername>/rlimits/ or immediately with the vlimit command. Current active limits can be viewed in the guests /proc/virtual/<XID>/limit file. The default is to have no limits, as shown below, -1 being equivalent to no limit:

server:~# cat /proc/virtual/2001/limit
PROC:            9              16              -1           0
VM:         127314          140118              -1           0
VML:             0               0              -1           0
RSS:          2894            4104              -1           0
ANON:         6797            6797              -1           0
FILES:         139             139              -1           0
OFD:            90              90              -1           0
LOCKS:           1               1              -1           0
SOCK:            3               3              -1           0
MSGQ:            0               0              -1           0
SHM:             0               0              -1           0

Memory quotas

Memory quotas on guest servers are set in terms of a maximum number of pages. The page size for the AMD64/EM64T architectures is 4096 bytes (4K). Thus setting a limit of 20,000 pages would limit memory usage to 80MB. The most useful size to set is the Resident Set Size (RSS) which limits the total RAM (in or out of swap) a guest can use. The following sets this the RSS limit for context 2001:

server:/# vlimit -c 2001 --rss 20000

Let's see if that's worked:

server:/# cat /proc/virtual/2001/limit | grep RSS
RSS:          2894            4104           20000           0

This shows that this context has a current usage of 2,894 pages, a maximum observed usage of 4,104 pages, and a hard limit of 20,000 pages. The final column shows that this limit has not yet been reached. Cool. To set this permanently (i.e. when the guest starts up) do:

echo 20000 > /etc/vservers/<servername>/rlimits/rss

Process quotas

Setting the 'nproc' limit for a guest means it cannot create more than that many processes. The following sets the PROC limit for context 2001 to 600 processses:

server:/# vlimit -c 2001 --nproc 600

Let's see if that's worked:

server:~# cat /proc/virtual/2001/limit | grep PROC
PROC:            9              16             600           0

To set this same limit when a vserver is started:

echo 600 > /etc/vservers/2001/rlimits/nproc

600 is a lot of processes and a reasonable limit. This will stop fork bombs from trashing a system. If a guest experiences a fatal error that causes it to hang it may be internally inoperable, but can still be shut down. If an application with lots of memory is called recursively, the system may still break. So, combined with an RSS limit the PROC limit should ensure the host system is armoured against fork bombs and run away applications.

Of 2 servers test under heavy load, our straw poll found one server with 87 processes and one with 156.

Other quotas

In addition to 'rss' and 'nproc' other limits can be set; here is the whole set:

Name in /proc/virtual/<XID>/limit Name in /etc/vserver/<servername>/rlimits/ Controls maximum
PROCnprocnumber of processes
VMasvirtual memory
VMLmemlocklocked virtual memory
RSSrssresident memory
FILESnofileopen files
LOCKSlocksfile locks

File quotas (disc limits)

Disc limits are set in a completely different way. Each file in a guests tree is tagged with the XID. The files with this XID tag can then be assigned a quota. For this to work the partition where the vserver guest live must be mounted with the option 'tagxid' to allow this. Remounting this partition live does not work, so unless you can completely disconnect the partition by umounting then remount, you will need to reboot.

There is a handy script to allow guests to be assigned quotas in a simple way available at

DiskLimit <servername>

Gives the current disc usage of the guest, and:

DiskLimit <servername> <quota>

Limits that guest to quota Kilobytes of disc space.

The 'DiskLimit' script does not set the inode quota of the guest in any useful way, just the maximum total space.

This quota will not survive a reboot. In order to do that some settings must be added to /etc/vserver/<servername>/dlimits/. This example set of command below sets a limit of 5Gb space, 100000 inodes and 5 reserved inodes:

mkdir -p /etc/vservers/<servername>/dlimits/0
echo /var/lib/vservers/<servername> > /etc/vservers/<servername>/dlimits/0/directory
echo $(( 5 * 1024 * 1024 )) > /etc/vservers/<servername>/dlimits/0/space_total
echo 100000 > /etc/vservers/<servername>/dlimits/0/inodes_total
echo 5 > /etc/vservers/<servername>/dlimits/0/reserved

Other issues

Guests cannot access, so some programs need to be made to bind to the guests IP address. Most programs just work. If you really need a see the link below:

Lessons learned


Author: Ben Green (ben at bristolwireless dot net) with editing by Mike Harris (mike at psand dot net) and corrections by Steve Woods (woodsy at bristolwireless dot net)

Copyright (c) 2007 Psand Limited and the respective authors. Permission is granted to copy, distributed and/or modify this document under the terms of the GNU Free Documentation License, Version 1.1 or any later version published by the Free Software Foundation; with no Invariant Sections, with no Front-Cover Texts, and with no Back-Cover Texts. A copy of the license is included in the section entitled "GNU Free Documentation License".

All trademarks are those of their respective trademark holders.

Valid XHTML 1.1!