The Kernel Command Line
Scyld ClusterWare supports a variety of options for configuring and controlling your cluster from the kernel command line. This section discusses the available command line tools, useful kernel command line options, and methods for adding new kernel modules.
Useful Command Line Options
There are a large number of different command line options that you can pass to the kernel. This section covers some of them. For information on how to feed these kernel command line options to the compute node kernels, see the section on /etc/beowulf/config earlier in this chapter, as well as the Chapter called Compute Node Boot Options.
- selinux= and capabilities.disable=
The BProc unified process space makes use of security hooks claimed by the Red Hat kernel for the Security Enhanced Linux (SELinux) and Capabilities security modules. These security modules are built into the Red Hat kernel, rather than loaded into an already running kernel.
To allow BProc to gain control of these hooks when the kernel is started, disable selinux and capabilities in the /etc/beowulf/config file. The lines should read as follows:
selinux=0 capabilities.disable=1
The parameters selinux=0 and capabilities.disable=1 are essential for the BProc unified process space to work properly. The section on /etc/beowulf/config provides important information regarding these parameters.
- mem=
One of the most frequently used command line options, "mem=" is used to set the memory size. Some older BIOS do not correctly report the amount of memory, so Linux ends up seeing only 64 MB of RAM. If your compute nodes come up and BeoStatus shows them with only 64 MB of RAM, but you know the compute nodes have more, you will want to use this option.
Set the parameter as "mem=XXXM", where "XXX" is the correct amount of RAM. For example, you might use "mem=128M" if your compute nodes have 128 MB of RAM.
- apic
This option turns on APIC support on the compute node. APIC is the newer of two different mechanisms Intel provides for invoking interrupts. It works better with SMP systems than the older mechanism, called XT-PIC. However, not every motherboard and chipset works correctly with APIC, so this option is disabled by default to avoid problems for those machines that do not support it.
If you find that your cluster nodes kernel panic or crash immediately upon boot, you probably want to turn off APIC by specifying "noapic" in the command line options. If you have many devices that generate interrupts (such as hard disk controllers, network adapters, etc.) you may want to try turning on APIC to see if there is any performance advantage for your cluster.
- panic=
This option allows you to specify how long the kernel should wait to reboot after a kernel panic. Set the parameter as "panic=num", where num indicates the number of seconds to wait before rebooting. For example, if you specify "panic=30", the kernel will wait 30 seconds after a panic, and then reboot.
Note that BeoBoot automatically adds "panic=30" to final boot images.
- apm=
This option allows you to specify APM options on the compute node. After the equals sign, you can put "on" to turn APM completely on, "off" to turn it completely off, "debug" to turn on debugging, and "power-off" to turn on only the power-off part of APM.
APM is not SMP-safe in the kernel; it will auto-disable itself if turned completely on for an SMP box. However, the power-off part of APM is SMP safe; thus, if you want to be able to power-off SMP boxes, you can do so by specifying "apm=power-off". Note that "apm=power-off" is specified in the default kernel command line in /etc/beowulf/config.
- console=
This option is used to select which device(s) to use for console output. The format of this option is "console=device,options". Device is "tty0" for the foreground virtual console, "ttyX" for any other virtual console, and "ttSx" for a serial port.
For the serial port, "options" defines the baud rate/parity/bits of the port in the format "BBBBPN", where "BBBB" is the speed, "P" is parity (n/o/e), and "N" is bits. The default setting is 9600n8, and the maximum baud rate is 115200. For example, to use the serial port at the maximum baud rate, specify "console=ttyS0,115200"
Adding New Kernel Modules
Many device drivers are included with Scyld ClusterWare and are supported out-of-the-box for both the master and the compute nodes. If you find that a device, such as your Ethernet adapter, is not supported and a Linux source code driver exists for it, then you will need to build the driver modules for the master.
To do this, you will need to install the RPM of kernel source code (if you haven't already done so). Next, compile the source code using the following extra GNU C Compiler (gcc) options.
-D__BOOT_KERNEL_SMP=1 -D__BOOT_KERNEL_UP=0 |
The compiled modules must be installed in the appropriate directories under /lib/modules. For example, if you are currently running under the 2.6.9-67.0.4.ELsmp kernel version, the compiled module for an Ethernet driver would be put in the following directory:
/lib/modules/2.6.9-67.0.4.ELsmp/kernel/drivers/net
Edit the config file /etc/beowulf/config to add the name of the driver to the "bootmodule" list; you can add more bootmodule lines if needed.
Next, you need to configure how the device driver gets loaded. You can set it up so that the device driver only loads if the specific device is found on the compute node. To do this, you need to add the PCI vendor/device ID pair to the PCI table information in the /usr/share/hwdata/pcitable file. You can figure out what these values are by using a combination of /sbin/lspci and /sbin/lspci -n.
So that your new kernel module is always loaded on the compute nodes, include the module in the initial RAM disk by adding a "modprobe" line to /etc/beowulf/config. The line should look like the following:
modprobe <module> |
where <module> is the kernel module in question.
To include the module in the root file system (rootfs) image for compute nodes, copy the module to /etc/beowulf/node, creating the directory if needed. You may need to copy the module to both the initrd and rootfs if the module is storage drivers required for local disks.
Finally, you can regenerate the BeoBoot images by running beoboot -2 -n. For more details, see the Section called Changing Boot Settings in the Chapter called Compute Node Boot Options.
Accessing External License Servers
To configure the firewall for accessing external license servers, enable ipforward in the /etc/beowulf/config file. The line should read as follows:
ipforward yes |
You must then reboot the compute nodes and restart the cluster services. To do so, run the following two commands as root in quick succession:
[root@cluster ~] # bpctl -S all -R [root@cluster ~] # service beowulf restart |
 | If IP forwarding is enabled in /etc/beowulf/config but is still not working, then check /etc/sysctl.conf to see if it is disabled. Check for the line "net.ipv4.ip_forward = 1". If the value is set to 0 (zero) instead of 1, then IP forwarding will be disabled, even if it is enabled in /etc/beowulf/config. |