What Gets Put on the Compute Nodes at Boot Time

• Generally speaking, the /dev directory contains a subset of devices present in the /dev directory on the master node. The /usr/lib/beoboot/bin/mknoderootfs script creates most of the /dev/ entries (e.g., zero, null, and random). /etc/beowulf/init.d/20ipmi creates ipmi0. /usr/lib/beoboot/bin/setup_fs creates shm and pts (as directed by /etc/beowulf/fstab). The harddrive devices (e.g., sda) are created at compute node bootup time, if local drives are discovered. If Infiniband hardware is present on the compute node, /etc/beowulf/init.d/15openib creates various device entries in /dev/infiniband/.

• The /etc directory contains the ld.so.cache, localtime, mtab, and nsswitch.conf files. The node_up script creates a simple hosts file.

• The /home directory exists as a read-write NFS mount of the /home directory from the master node. Thus, all the home directories can be accessed by jobs running on the compute nodes.

• Additionally, other read-only NFS mounts exist by default, to better assist out-of-the-box application and script execution: /bin, /usr/bin, /opt, /usr/lib64/python2.3, /usr/lib/perl5, and /usr/lib64/perl5.

• The node_up script mounts pseudo-filesystems as directed by /etc/beowulf/fstab: /proc, /sys, and /bpfs.

• /etc/beowulf/config names various libraries directories that are managed by the compute node's library cache. Run beoconfig libraries to see the current list of library directories. Caching shared libraries, done automatically as needed on a compute node, speeds up the transfer process when you are trying to run jobs.

• Typically, when the loader starts up an application, it opens the needed shared libraries. This open causes the compute node to pull the shared library from the master node and save it in the library cache. However, some applications and scripts reference a shared library or other file that, although it resides in one of those libraries directories, the reference does not open the file, and so the file does not get automatically pulled into the library cache. The application or script thus fails to execute, and the missing library or file name is typically displayed as an error. To remedy this type of failure, you should use a prestage entry in /etc/beowulf/config to explicitly name files that should be pulled to each compute node at node startup time. Run beoconfig prestage for the current list of prestaged files.

• mknoderootfs creates /var and several of its subdirectories.

• The /tmp directory is world-writeable and can be used as temporary space for compute jobs.