Getting Started with Vesta

Tim Mann <tim at tim dash mann dot org>, Ken Schalk <ken at xorian dot net>
Last modified: Tue Jun 26 23:37:48 EDT 2007

I. Requirements

  1. Platform: Vesta currently runs on the following platforms:

    • Linux on IA-32 compatible processors. The kits (one for i386 and one for i686) have been built in a RedHat 7.1 environment, but the binaries are statically linked and should work on a variety of distributions. A 2.4 kernel is a necessity, due to deficiencies in the 2.2 NFS client.

    • Linux on Alpha. As with the kit for IA-32 Linux, the binaries were statically linked in a RedHat 7.1 environment, and a 2.4 kernel is required.

    • Linux on PowerPC. The kit for PowerPC Linux was built in an environment derived from Debian stable (woody). A 2.4 kernel is required.

    • Linux on Sparc. The kit for Sparc Linux was also built in an environment derived from Debian stable (woody). A 2.4 kernel is required.

    • Compaq Tru64 UNIX on Alpha, versions 4.0D through 4.0G. (It has not been tested on 5.0 or 5.1.)

    These requirements apply to both servers and clients, but a mix of platforms between clients and servers is fully supported. (Note: setting up a multi-platform Vesta installation requires additional setup not covered in this document. We recommend trying it on a single platform until you've gained familiarity with it.)

    Vesta is designed to be portable, and we expect it to be ported to other platforms in the future.

  2. Server machine: The Vesta kit consists of several servers, a collection of user programs, and documentation. Two of the servers, the repository server and the cache server, are shared by all users at a Vesta installation site, so they need to run on a suitable server machine (or two machines). If you are installing the kit just to get a quick test drive, you can choose any workstation as the server machine. However, the repository and cache servers need to run on a machine with locally attached disks.

  3. Disk space: To make any use of Vesta, you need not just the core system, but also a set of models that describe a standard environment for building -- that is, a set of compilers, libraries, etc. We have made such an environment available for replication over the Internet, as well as the complete source code for Vesta itself and some other software. Part of these instructions includes using the Vesta replicator to obtain a copy of this environment in your local Vesta repository server. Basically, what you will get is a cut-down snapshot of the Vesta repository that the Vesta developers use.

    You will need to allocate disk space for storing the build environment, source revisions, plus a cache of previously built derived files (object files, executables, or whatever else you build with Vesta).  The more space you allocate, the less often you'll need to run the weeder to remove old builds.  Just how much you need will depend on the size of derived data produced by your builds and how often you perform builds.

    This storage must be local to the machine which will run the two Vesta servers, not served over NFS from another system.  In most installations, this space is mounted as "/vesta-srv" (but you can give it any path you like).  You'll need to NFS export it to any Vesta client machines, and have them mount it at the same path as the server uses.

    For a single user installation building Vesta itself, you can get by with a few hundred megabytes.  If you have Linux with a root partition that has a gigabyte or more free, you can "mkdir /vesta-srv" and move on.  At the largest Vesta installation we know of, with over 100 active users performing builds each with over 1 GB of derived data, a 100 GB RAID array was used and the weeder was run a little less than once a week.

II. Installation

This is the first publicly available installation kit for Vesta, so unfortunately there are a lot of manual steps to follow. We hope to automate much of this in the future.

  1. Download the kit for your platform. Visit the Vesta project page at SourceForge, and download one of the binary kits available there.

    [New] Alternatively, you can try install-able RedHat/Debian packages for x86 Linux. These automate all required steps outlined in this guide, so they're a lot easier.

  2. Create accounts and groups. There are various ways you can do this, but rather than explaining the choices here, we recommend using the following user and group names:

    On RedHat Linux, you can create the above users and groups with the following commands:

    su root
    useradd -c 'Vesta administrator' -m vadmin
    useradd -c 'Vesta wizard' -g vadmin -m -n vwizard
    useradd -c 'Vesta runtool user' -g vadmin -m -n vruntool
    useradd -c 'Vesta foreign user' -m vforeign

    On Debian Linux, you can create the above users and groups with:

    su root
    groupadd vadmin
    groupadd vforeign
    useradd -c 'Vesta administrator' -g vadmin -m vadmin
    useradd -c 'Vesta wizard' -g vadmin -m vwizard
    useradd -c 'Vesta runtool user' -g vadmin -m vruntool
    useradd -c 'Vesta foreign user' -g vforeign -m vforeign

    If you're not sure whether you can use one of the above sets of commands, consult your useradd(8) and groupadd(8) man pages. If you do use either of the above methods, you will probably want to set a password for at least the vadmin user:

    su root
    passwd vadmin
    <enter new password for vadmin>
  3. Create installation directory. You'll need a new, empty directory on a local disk on your server machine, with enough space available (see Disk Space under Requirements above). It should have owner and group vadmin and mode 0775. This directory must be mounted on all client machines that will be using Vesta. Make sure all clients mount it under the same path, and that it's also available under this same path on the server, if necessary by making extra symbolic links. The text below assumes you will call the directory "/vesta-srv".

  4. a
  5. Untar the kit. Sample commands:

    su vadmin
    umask 022
    cd /vesta-srv
    tar -zxf vestakit.linux_i686.tar.gz

    (Of course, be sure to download and unpack the right kit for your platform.)

  6. Make links and apply setuid bits. Sample commands:

    ln -s vestaeval /vesta-srv/bin/vesta
    su root
    chown vadmin /vesta-srv/bin/vestaeval
    chmod u+s /vesta-srv/bin/vestaeval
    chown root /vesta-srv/bin/tool_launcher
    chmod u+s /vesta-srv/bin/tool_launcher

    (tool_launcher is a very simple program which performs a chroot system call, which requires root privileges, then drops its root privileges and exec's another program within the chrooted environment. If you're security conscious and don't want to make a binary setuid root without looking at the source, you can put this off until after you've replicated the Vesta sources to your repository, which is described below. Once you've done that, you can find the source for the tool_launcher in /vesta/ You can also get the source for tool_launcher ahead of time by using the web interface to the public repository. It's even fairly simple to compile it from source yourself without using Vesta's builder, as it's just one C source file and an asscoaited header.)

  7. Make mount points. On each client machine that will use Vesta, create /vesta and /vesta-work as empty directories. These will be used as mount points for the repository's NFS interface. (The permissions of these directories don't matter, because once the repository is mounted their permissions will be determined by the repository server.)

    On some platforms (Tru64), having the mount points owned by a non-root user (such as vadmin) will allow the repository to be mounted and unmounted by that user (with the mountrepos and umountrepos commands). On others (Linux), root privileges are always required to mount and unmount the repository.

  8. Choose a realm name for your site. Your realm name is used to form global user names of users at your site. Each user's global user name is formed by appending "@realm" to their UNIX login name. Global user names are used when interacting with other Vesta installations for replication and remote checkout/checkin. Your realm name must be unique across all Vesta installations, so it's best to use a domain name you own or a hostname within it. (If you chose the maildrop host for your system's users, this can have the added benefit of making Vesta global user names work as e-mail addresses.)

    If you do not have a registered domain name of your own, a good alternative is to use a working email address that you intend to keep permanently. Simply change the "@" to an "_", and you have a globally unique string which you can use as your realm name. (If your e-mail address is "", you can use "" for your realm name.)

    See the repository man page section on access control for more information on realm names, global user names, and how they are used.

  9. Edit vesta.cfg. Vesta uses a configuration file to set various (normally site-wide) values such as filenames and user and group ids. The configuration file is normally /vesta-srv/vesta.cfg. The kit you unpacked earlier included a sample vesta.cfg, that you'll need to specialize for your installation. The format of this file should be fairly clear, but see the vesta-intro man page if you want documentation.

    1. Set your realm name. Enter your chosen realm name (see the discussion above) as the value of the "realm" setting in the [Repository] sections of the configuration file. [Before Vesta release/8, it was neccessary to also set [UserInterface]realm to your realm. That setting is now optional and in most configurations can be omitted.]

    2. Enter user/group information. In the [Repository] section, edit the following settings:

      • vforeign_uid = <numeric user ID of vforeign>

        (You can find this value in the /etc/passwd file, or by using the command "id vforeign".)

      • vforeign_gid = <numeric group ID of vforeign>

        (Consult the /etc/group file to find this value.)

      If you're not using the suggested special user and group names, you may also need to edit the following settings.

      • vadmin_user = vadmin

        [Before Vesta release/8, it was neccessary to append "@<your realm>" to the _user and _group settings in the [Repository] section. You can specify a realm explicitly, but if these settings have no realm they are taken as being in [Repository]realm. In most cases this exactly what you want.]

      • root_user = root

      • vwizard_user = vwizard

      • vadmin_group = ^vadmin

      • runtool_user = vruntool

    3. Set the common name of the installation directory (created in an earlier step) into the following three places: [CacheServer]MetaDataRoot, [Weeder]CacheMDRoot, and [Repository]metadata_root. In the [Repository] section, be sure to terminate the name with a trailing slash (e.g. "/vesta-srv/").

    4. Set the name of your server machine into the following places: [CacheServer]Host, [Weeder]CacheHost, [Repository]NFS_host, [Repository]ShortIdSRPC_host, [Repository]VestaSourceSRPC_host.

    5. Set [Repository]ShortId_sorter to point into /vesta-srv/bin; that is, set it to /vesta-srv/bin/sidsort. Set [Run_Tool]helper to /vesta-srv/bin/tool_launcher. Set [UserInterface]Interpreter to /vesta-srv/bin/vesta.

    6. Set [UserInterface]DefaultPackageParent to a new, unique name that you will use for creating packages. The name must begin with /vesta/. You must make the next component a globally unique name, for compatibility with Vesta's wide-area replication feature. The established convention is to use an Internet domain name or host name that belongs to you. You may use the same value as your realm name. (If your realm name is "", you can use /vesta/ as the DefaultPackageParent.)

      As with your realm name, if you don't own a domain you can use your e-mail address with "@" replaced by "_". (If your e-mail address is "", you can set your DefaultPackageParent to /vesta/

    7. Set [Evaluator]DefaultMain to an appropriate value for your platform. This setting controls the default model file name evaluated when no path is specified or a directory is specified. (See the evaluator man page.) The table below shows appropriate values for the different platforms.

      Platform [Evaluator]DefaultMain
      Linux i686
      linux_i686.main.ves linux_i386.main.ves .main.ves
      Linux i386
      linux_i386.main.ves .main.ves
      Linux Alpha
      linux_alpha.main.ves .main.ves
      Linux PowerPC
      linux_ppc.main.ves .main.ves
      Linux Sparc
      linux_sparc.main.ves .main.ves
      Tru64 Alpha
      tru64_alpha.main.ves .main.ves

      (The .main.ves used to be used for top-level models when Vesta ran on just one platform. With the port to Linux, we needed one top-level model per target platform.)

    8. (Optional)  If you're going to have multiple client machines, you may want to add them to the list of hosts available to run tools.  The sample configuration file distributed with Vesta only lists localhost as available for running tools, so whichever machine you're running the evaluator on will handle all tool invocations.  Listing more hosts in your configuration file will spread out the work of performing a build and increase parallelism.  (Most builds can take advantage of the evaluator's multi-threading capabilities.)

      List the hostnames you plan to make available for tool invocations separated by whitespace in the value for [platform]hosts, where platform is the platform name you'll be using (usually one of "Linux2.4-i686", "Linux2.4-i386", "Linux2.4-alpha", "Linux2.4-ppc", "Linux2.4-sparc", or "DU4.0").

      (See the evaluator's man page for more information on host selection for tool invocations.)

    9. (Optional)  If you would like remote clients (e.g. users at peer repositories performing a remote checkout from your repository) to contact your repository using a different host or port from local clients, you may wish to set [Repository]master_hint. If set, this should be a host name and TCP port number in the format host:port. The value of this setting goes into "master-repository" attributes which the repository tools later use to contact the repository with the master copy of an object. You may need to use this if your repository has a different "public address" accessible by hosts outside your LAN (e.g. a port forwarded through by a firewall with a different hostname). It can also be useful for using Vesta installed on a laptop, when the laptop has a fixed hostname on some "home" network.

  10. Make Vesta commands available. To user Vesta, /vesta-srv/bin should be included in the user's shell search path (the PATH environment variable). You can either do this in the individual shell start-up files of specific users (e.g. ~/.bash_profile, ~/.cshrc) or in one of the central shell start-up files (e.g. /etc/profile, /etc/csh.login). (If you don't do it centrally, you should at least make sure vadmin, vwizard, and vruntool have /vesta-srv/bin on their PATH.)

    For users with a System V style shell (e.g. bash, zsh), you might add these commands to the shell start-up:

    if [ ! -z "$PATH" ]; then
    export PATH

    For users with a csh style shell (e.g. csh, tcsh), you might add these commands to the shell start-up:

    if ($?PATH) then
        setenv PATH "/vesta-srv/bin:${PATH}"
        setenv PATH "/vesta-srv/bin:/bin:/usr/bin:/usr/local/bin"

    Also, to make the Vesta man pages available, /vesta-srv/man can be added to the MANPATH environment variable (the man page search path). For a System V style shell, you might add this to the shell start-up:

    if [ ! -z "$MANPATH" ]; then
    export MANPATH

    For a csh style shell, you might add this to the shell start-up:

    if ($?MANPATH) then
        setenv MANPATH "/vesta-srv/man:${MANPATH}"
        setenv MANPATH "/vesta-srv/man:/usr/man://usr/share/man"

    Note that on most Linux systems, an alternative to modifying or setting MANPATH is adding a line to the man configuration file (normally /etc/man.config or /etc/manpath.config) such as:

    MANPATH_MAP /vesta-srv/bin /vesta-srv/man
  11. Create /etc/vesta.cfg. On the server and any client machines, Make a symbolic link from /etc/vesta.cfg to /vesta-srv/vesta.cfg:

    su root
    ln -s /vesta-srv/vesta.cfg /etc/vesta.cfg

    This makes it possible for the Vesta tools to find the configuration file without requiring users to create a personal vesta.cfg in their home directory or setting the VESTACONFIG environment variable.

    After doing this you can test that the Vesta tools can find your config file by typing:

    vgetconfig -L
  12. Create the files and directories used by the repository. Before you start up the Vesta repository, there are a few directories and files it uses which must be created.

    1. First are the directories in which the repository stores individual files (sources and derived files) and metadata about them:

      su vadmin
      mkdir -p /vesta-srv/sid
      mkdir -p /vesta-srv/repos/log
      chmod 0755 /vesta-srv/sid /vesta-srv/repos /vesta-srv/repos/log
    2. Next you need a repository export file. This, which is normally /vesta-srv/repos/export. This file is used to specify which hosts may connect to the repository. You can allow access based on either a range of IP addresses or by hostname matching. For example, if your realm name is "" and you want to allow all the hosts within the domain to access your repository, your export file would look like this:

      * allow unix, allow global

      On the other hand, if your realm name is "" and your home LAN uses IP addresses starting with 192.168.1, then you could use this as your export file: allow unix, allow global

      You can also allow access from a single host or IP, like this: allow unix, allow global
    allow unix, allow global

      You may also want to allow the repository server host access through its loop-back interface: allow unix, allow global

      Note: If you do allow NFS access to your repository (turned on by "allow unix") from multiple machines, it's important that they all share user and group tables with the repository server machine (e.g. via NIS).

      See the repository man page for more information on the format of the export file.

    3. Lastly, you should create the alias and group files:

      su vadmin
      touch /vesta-srv/repos/alias
      touch /vesta-srv/repos/group
      chmod 0755 /vesta-srv/repos/alias /vesta-srv/repos/group

      These access control files can be used to grant additional privileges for local and remote users.  (For example, you can make a remote user a member of a local group, or allow one user to act on behalf of another.)  They should be present, but can be empty until you have a need to add something to them.  See the repository man page for information on their format and function.

  13. Create the directories used by the cache server. This requires just one command:

    su vadmin

III. Recurring Initialization

Step 1 below needs to be run every time your server machine reboots. Step 2 needs to be run on each client machine whenever it reboots; this includes the server machine if you're also using it as a client.

  1. Server initialization. Run /vesta-srv/bin/repository and /vesta-srv/bin/VCache on the server machine. Both must be started as user vadmin. Both servers print out a few progress and error messages at times, which should be saved to a file. For example:

    su vadmin
    cd /vesta-srv
    ./bin/repository >& repository.messages &
    ./bin/VCache -debug StatusMsgs >& VCache.messages &

    Both servers use logging for crash recovery. If either one crashes, just run it again, and it should recover.

  2. Client initialization. On each client host perform the following steps. (Note that the server host can also be a client, and that you need not set up multiple clients.)

    1. Mount the repository on the client machine. Assuming you have created the mount points as in II.6 above, just run mountrepos.

      On Linux systems, only root can do this. On Tru64 systems, the owner of the mount point directories (typically vadmin) can also mount and umount the repository.

    2. Check that the repository is mounted. Type "df /vesta". If it shows that it's mounted from your repository server, it's working. (Once you have populated your repository, which we'll talk about shortly, you could also "ls /vesta" and see if it's non-empty.)

    3. Start the RunToolServer as vruntool. The server may print some debug information from time to time, so you might want to redirect its standard output and error to a file. For example:

      su vruntool
      cd /vesta-srv
      ./bin/RunToolServer >& runtool.messages &

      (If you set up a list of hostnames in [platform]hosts as described above, you should start a RunToolServer on each machine you listed.)

IV. Repository Initialization

These steps are really part of installation (II), but you can't do them until the servers are initialized and the repository is mounted (III), so we've put them in this small section of their own.

  1. Replicate a build environment. As mentioned earlier, in order to perform builds under Vesta you need a build environment (a set of compilers, libraries, etc.) Builds performed under Vesta take place in an encapsulated environment. Even if you already have compilers and libraries installed on your system, you need immutable copies of the same (or similar) components within your Vesta repository. The easiest way to get them is to replicate a copy over the network from another repository.

    1. First, tell your repository that its OK to replicate from the public Vesta repository:

      su vadmin
      vattrib -a "#replicate-from" /vesta
    2. Now use the replicator to get a copy of a build environment for your platform. If you're using Linux on i686 or higher (run "uname -m" if you're not sure), do this:

      vrepl -I -s -e@ /vesta/

      If you're using Linux on an older IA-32 machine (i386 or higher), do this:

      vrepl -I -s -e@ /vesta/

      If you're using Alpha Linux, do this:

      vrepl -I -s -e@ /vesta/

      If you encounter problems during the replication process, see the troubleshooting section.

      (Unfortunately, we can't make a build environment for Tru64 available for replication, due to licensing restrictions on binary redistribution. We have some methods for constructing one on an installed Tru64 system, but they haven't been cleaned up or documented. If you need a Vesta build environment for Tru64, send e-mail to the vesta-users mailing list (<vesta dash users at lists dot sourceforge dot net>), and we'll help you with building one.)

  2. Replicate the Vesta sources. The sources for Vesta and associated system models which build it form a fairly sophisticated example of what you can do with the system. These are also available for replication from the public Vesta repository. Assuming you already added the #replicate-from attribute as described above, you can get a copy of the full Vesta release for i686 Linux with:

    vrepl -I -s -e@ /vesta/

    Or for i386 Linux with:

    vrepl -I -s -e@ /vesta/

    For Alpha Linux with:

    vrepl -I -s -e@ /vesta/

    For PowerPC Linux with:

    vrepl -I -s -e@ /vesta/

    For Sparc Linux with:

    vrepl -I -s -e@ /vesta/
  3. Replicate the examples referenced in the tutorial. There are a couple simple examples of how to use Vesta which you'll probably want in your repository. You can get a copy of them with:

    vrepl -I -s -e+ "/vesta/*/LAST"
  4. Make a directory tree for new packages created at your site. The root of this tree will be the name that you chose in step II.8.F. above. You can type "vgetconfig UserInterface DefaultPackageParent" to remind yourself what it was. Suppose the directory name was /vesta/ Then you'd type these commands:

    su vwizard
    cd /vesta
    vmkdir -p
    chmod 777

    These commands create the DefaultPackageParent directory, tell the Vesta tools that it's a legitimate place to put packages, and set its mode to 777 so that everyone can create packages in it.

    You must use the special vwizard user for this step, because you are creating a new directory immediately below /vesta. The convention of deriving the name of such directories from domain names or e-mail addresses is the only protection against two people creating the same top-level directory in two different repositories. If you fail to make these names globally unique when you first create them, you will cause serious problems later if you ever try to have your Vesta installation cooperate with anyone else's. For these reasons, only vwizard is allowed to perform operations which have the potential to violate the replication invariant. (See the vrepl man page for more information on this subject.)

    (vmkdir does essentially the same thing as the normal mkdir command, but it uses the repository-specific SRPC interface rather than the NFS filesystem interface. In some cases, vwizard can create a top-level directory with mkdir. However, some NFS client implementations deny permission without asking the repository. vmkdir avoids the quirks of different NFS client implementations, and should always work.)

    Now that you've created the top-level directory for your site, you will probably want to create a subdirectory which users can feel free to fill with experiments as they learn how to use Vesta (so as not to clutter up your site's package namespace). Since you made the top-level directory world-writable, you can do the following as any user:

    cd /vesta/
    mkdir play
    cd play
    vattrib -s type package-parent .
    chmod 777 .

    You can create more subdirectories in the same way, as many levels deep as you like. You can forbid people from creating packages in the higher-level directories simply by not running "vattrib -s type package-parent" on those directories. You also can restrict who can create packages in a directory by using chown, chgrp, and chmod to set its access permissions. (Remember, vadmin can chown, chgrp, and chmod anything inside the repository.)

  5. Make sure that users have permission to create a directory tree for their working copies of checked-out packages. Each user's working copies will be found under a subdirectory of /vesta-work. (For example, user jsmith's working copies would be found under /vesta-work/jsmith.) The checkout tool (vcheckout) will create a user's directory if it doesn't exist, but the user must have permission to do so. Therefore you should do something like this:

    su vadmin
    chown vadmin /vesta-work
    chmod 777 /vesta-work

    Alternatively, you could restrict write permission to a group whose members include everyone who will be using Vesta:

    su vadmin
    chgrp vadmin.staff /vesta-work
    chmod 775 /vesta-work
  6. Create the temporary directory used by vupdate. This is just a temporary directory that exists within the mutable portion of the repository. Like /tmp, it should be world-writable. Creating it is simple:

    su vadmin
    mkdir /vesta-work/.tmp
    chmod 777 /vesta-work/.tmp

    (Actually, you can change the location of this directory with the configuration setting [UserInterface]TempDir, but normally there's no reason to do so.)

V. Where to go Next

Now you'll probably find it useful to go through the tutorial.  Now that you've installed and set up Vesta, other users at your site can also use that document to get an introduction to the system.

The vesta-intro man page is a useful guide to Vesta concepts and terminology. The command summary is a helpful reference to the most common Vesta user tasks.

For information on the Vesta System Description Language (aka Vesta SDL or just SDL), which is used to describe how to build derived files (e.g. compiling sources into objects linking the resulting objects into an executable), refer to the language reference guide. (Or, if you find it lacking, the more precise language specification.)

[We have a small related tool for general management of server processes named smgr which we plan to release soon.  It simplifies starting and stopping the Vesta servers, as it handles redirecting output, sends e-mail if a process it's managing stops unexpectedly, and can be used to start them at boot time.  In ouir experience it's a little better than some of the other similar tools that are freely available.]

VI. Troubleshooting

If you have a problem not covered below, please send e-mail to the vesta-users mailing list (<vesta dash users at lists dot sourceforge dot net>).

SRPC failures: Vesta releases 5 and below have a limitation in their remote procedure call library (SRPC) which causes the source address of all outgoing packets to be the IP address which a machines hostname resolves to. (The output of "ping `hostname`" should indicate what this IP address is.) You can either upgrade to relases 6 or higher, or make sure that this is the correct IP address of the outbound interface. (We realize this is difficult for dial-up users.) On recent RedHat systems, the installer usually makes the hostname resolve to the loopback address (, which won't work.

Failures during replication: There appears to be a problem with some versions of the Linux TCP implementation. Under certain packet loss conditions (which are not uncommon on the Internet), re-transmitted packets get an incorrect TCP checksum, causing the public repository server to simply ignore them. Linux clients can work around this bug by turning off TCP retransmission collapsing with:

echo 0 > /proc/sys/net/ipv4/tcp_retrans_collapse

Alternatively, an upgrade to a newer 2.4 kernel should also fix the problem.

Back to the Vesta home page