Working with Hardware and Device Management

A SA must be able to safely add RAM, install or replace NICs and hard disks.  Keep an inventory of all workstations, servers, network components, printers, etc., (including location, installed software (and versions/patches), and configuration information).  (Demo installing RAM, drives, add-on boards to visible computer.  Have students practice with RAM.)

Discuss ESD (electro-static discharge) and how it can damage (cumulatively!) components: handle carefully, ground yourself using strap or at least touch ground first.  Enough ESD to feel is far more than enough to damage!

Discuss severe shock hazard of capacitors, even after unit unplugged.  Modern devices have “bleeder resisters” to drain the charge, takes 10-30 seconds.

Be careful when running cabling.  Cat5 cables are very fragile and will not operate at full capacity if ever bent sharply or kinked.  Get a professional to run cables or take courses!  Cable types include crossover/null-modem, rollover, and straight-patch.  Cables mostly have cheap and effective PVC insulation, but in a fire this give off deadly chlorine gas, so use plenum cables in human spaces.  Handling cables can be tricky as a tug or kink can dramatically affect performance (Demo).  There are “structured cabling” or EIA/TIA 568B standards: 100m overall, wiring closet on each floor, no more than 3m from host to outlet, 90m from outlet to wiring closet, and 9m of patch cables inside the closet.  Discuss UPS (uninterruptable power supply).

Working with Hardware

Traditional Unix systems ran multi-user non-GUI applications (DB, Telnet, email, etc.).  Hardware on such systems involves drives (including disk and tape), serial (terminal, modem), parallel (printer), and networking hardware only.  Modern systems are used as workstations too and include video, sound, and other hardware that must be configured.  Depending on the platform you may also have to manage ISA, ISA-PnP, and PCI busses, and SCSI, or IDE (or both) controllers.

Hardware is controlled in the Unix kernel by device drivers.  These are small bits of code that have a common API (software interface) and are integrated into the kernel.  (Some device drivers manage virtual devices, not physical ones.)

Modern systems auto-detect new hardware at boot time, if that hardware is powered up before the OS boots.  New hardware added once the system boots may or may not be auto-detected.  With Linux you can run a program to force the kernel to scan for new hardware, kudzu.  This will rebuilt /sys and then /dev.  (With Solaris use devfsadm to rebuild the /device and /dev entries with a reboot, or touch /reconfigure and reboot.  See also cfgadm.)

Some hardware is considered hot-pluggable and will be auto-detected.  This includes USB devices and some others.  (Such devices are normally handled by a separate kernel subsystem and managed by separate commands, vol* on Solaris.)

Major and Minor Device Numbers

Each device driver has a number known as the major device number.  All hardware that uses the same driver uses the same major device number.  Each individual device is also assigned a unique minor number.  So if you have two identical hard drives they will be assigned the same major but different minor numbers.

Some devices use a range of minor nums to select features of a particular type of hardware rather than to distinguish among several identical devices (e.g., 1-16 = dev 1, 17-32=dev2, ...).

Each hardware device used on your system is represented by an entry in /dev known as a special file (“special files” can receive or generate data, and usually correspond to specific hardware but not always).  These are not real files; the inodes don’t contain location information, but instead the major and minor numbers.  (“ls -l” shows these numbers where the file size is normally shown.)

Special files are created with the mknod command.  (There is usually a shell script in /dev called MAKEDEV that will create them automatically with correct names.)  Note these are not files stored on your disk and can’t be backed-up with a tool such as tar.

Special files come in two types: block and character.  The block devices are any hardware that behaves like a hard disk, all the rest are character special files which are accessed a byte (or character) at a time.  Disk and tape drives usually have two entries in /dev.  The first is the regular block special file and the second is the character special file, known as the raw interface to the device.  (The raw device is sometimes used for backups or debugging disks.)

In this directory are also found subdirectories and aliases; this makes it easier to work with the special files.  A lot of software uses these standard aliases rather than prompt the user for the actual device, so make sure you set these.  (MAKEDEV usually does this too.)  Note NICs are an exception and usually don’t have entries in /dev on most *nix systems (neither do PPP connections).

The list of major device numbers and what hardware they refer to is kept in files that vary from system to system.  On Linux that list is in the file (available on-one too) /usr/src/linux/Documentation/devices.txtAlso see the numbers for currently used devices by /proc/devices on some systems.

 [Skip to the next section; let folks take A+ if curious.]

Bus Details                 A bus connects components in a computer system.  There are several buses in most computers: PCI (today = PCIe or PCI express; PCI-X is not “PCI express, but an older PCI standard; ISA is an even older standard at 8Mhz), AGP (advanced graphics port; used for graphics cards before PCIe became common), and the FSB (front side bus).

The FSB is the most important bus to consider when you are talking about the performance of a computer.  It connects the processors (CPUs) to the system memory.  When people talk about the speed of a computer, they mean the clock speed, which is used for the CPU and also determines the FSB speed (which is some multiplier value of the CPU speed: 1x, .5x, or .25x, the clock speed depending on the hardware).  Note many systems default the FSB to the lowest supported speed, so you may be underclocking your system!

Overclocking is done through manipulating the CPU multiplier and the motherboard's front side bus (FSB) speed until a maximum stable operating frequency is reached. While the idea is simple, variations in the electrical and physical characteristics of computing systems can complicate the process.  Several factors limit how much overclocking is possible: bus dividers, voltages, thermal loads, cooling techniques, and other factors can limit overclocking to 1.1x or 1.2x.  Even then overclocking is likely to reduce the lifespan of the hardware.

The PCI bus connects expansion slots for peripherals as well as meda drives (disk, DVD) to the rest of the system.  Depending on the version the top speed is only up to 144 MB/sec.  (Many non-standard systems ran PCI up to 800Mhz.)  This isn’t fast enough for many video applications, so an AGP bus was added about 8 times faster (more throughput) for video card use.

In PCI, each device is identified by a bus number, a device number, and a device function. A given computer might have several PCI buses which might be linked (one bus used to extend another bus, joined through a PCI bridge) or independent (several buses all attached to the CPU), or some combination of the two. Generally, large high-end machines with lots of I/O expansion have more complicated PCI topologies than smaller or cheaper systems. Each device on a bus is assigned a device number by the PCI controller, and each device exposes one or more numbered functions. For example, many graphics cards offer integrated sound hardware for use with HDMI; typically the graphics capability will be function zero, the sound will be function 1. Only one device can use the bus at any given moment, which is why high-end machines often have multiple independent buses—this allows multiple devices to be active simultaneously.

The newer PCIe (PCI Express) systems are much faster (up to 16 GB/sec) and PCIe has replaced the older PCI+AGP motherboards.  PCIe is full-duplex serial (old PCI and ABP were half-duplex, parallel) and doesn’t share the bandwidth.  This is not a bus but a switch.  (Qu: why is that good?)

PCIe operates similarly. PCIe is a point-to-point architecture rather than a bus architecture; rather than all devices (and all hardware slots) on the same bus being electrically connected, in PCIe there are no connections between devices. Instead, each device is connected solely to the controller. Each connection between device and controller is regarded as its own bus; devices are still assigned numbers, but because there can only be one device on each “bus,” this number will always be zero. This approach allows software to treat PCIe as if it were PCI, allowing for easier migration from PCI to PCIe. This point-to-point topology alleviates the bus contention problem in PCI—since there is no bus sharing, there are fewer restrictions on concurrent device activity.

Connecting all the buses and other devices attached to the motherboard together is the job of a host bridge.  A host may have several (PCI) busses connected to the same host bridge.  Most hosts have several host bridges.  In this case they may either appear as separate domains (group of buses/devices) or as a single domain; it depends on the motherboard.

All the host bridges and the buses attached to them, and the devices attached to those buses, are represented as a device hierarchy; devices thus have a kind of pathname.

The devices and buses are usually grouped into two sub-systems (each with its own bridge) on the motherboard.  The first is the northbridge, also known as the memory controller hub (MCH) in Intel systems.  The northbridge typically handles communications between the CPU, RAM, AGP (or PCI Express video card), and the southbridge.  The Southbridge, also known as the I/O Controller Hub (ICH) in Intel systems implements the “slower” capabilities of the motherboard including the PCI expansion slots, serial and parallel ports, keyboard, mouse, “on-board” video chip, USB, NICs, and so on.  (See IntelArch.gif.)  AMD improved performance in 2003 by connecting memory directly to the CPU.  Now (2009) Intel has eliminated the Northbridge (MCH) completely (see IntelP55.gif).

To save space a single physical device in a PCI bus slot may have several un-related functions.  The fully qualified name of a PCI device is thus:

       domain:bus:slot.function

(The “domain:” and “.function” are often omitted.)

·       IRQs:  Only 15 of these (0-15, 2=9), all but a few are reserved.  ( 4=ttyS0, 3=ttyS1, 7=lp0, 5=lp1, 6=fd0)  Some devices can share a single IRQ but this is rare.  Some systems allow soft IRQs so there is no limit on the number of them.

IRQ are supposed to be assigned automatically on PCI  and ISA-PnP  but manually on ISA devices (using jumpers, DIP switches, or special software).  Many PnP devices don’t work according to the standard and will only work if a particular IRQ is assigned, so manually setting these may be needed.

For PCI use lspci [-t] and /proc/bus/pci.  For jumperless ISA and ISA-PnP use isadump > /etc/isapnp.conf > isapnp, pnpdump and pnpprobe.  See also /proc/interrupts and /proc/irq/*.

Originally each device had its own private IRQ.  Today PCI devices share IRQs.  When a device interrupts the CPU an interrupt handler checks to see which device actually caused the interrupt, and then runs the proper code.  Note /proc/interrupts doesn’t know which devices share a common IRQ; use the command “dmesg |grep ‑i irq” to learn which IRQ some device uses.

·       I/O Base address  (sometimes called a port, but not related to the network term!)  Memory-mapped I/O is a common technique: the CPU reads or writes a certain address but instead of the data coming from RAM it comes from or goes to some RAM (or registers) on the device.  Each device that communicates this way will respond to a certain address or (small) range of addresses only.  If PnP doesn’t set this correctly you may have to set it manually or the kernel won’t be able to talk to the device (or two devices may interfere with each other.)  (Some std. address ranges are: 0x3F8=ttyS0, 0x378–37F=lp0, 0x3F0–0F7=fd0)

·       DMADirect Memory Access is a high speed transfer of data directly to RAM from some device.  Normally software running on the CPU reads a few bytes from the device (to the CPU) and then writes it to RAM, using the I/O port.  Going direct without involving the CPU is faster.  Like IRQs and I/O addresses, this should normally be configured automatically for those devices that use it.

NDISwrapper is special software that can translate between Windows32 driver API and the Linux API.  It was designed for NICs and Wi-Fi cards that lack Linux drivers.  You simply install a (legal copy of a) Windows driver (a pair of files, foo.sys and foo.inf) and use ndiswrapper build a kernel module from it that you can load.  Since Windows drivers are notorious for bypassing the Win32 API, this won’t always work, but is worth a try if you can’t find a Linux/Unix driver.

To use:
yum install --enablerepo=livna kmod-ndiswrapper
ndiswrapper ‑i foo.inf   # creates module
ndiswrapper -m  # adds conf info to /etc/modprobe.d/ndiswrapper

Hardware changes may be auto detected at every boot.  On Linux this is done with kudzu.  Solaris doesn’t run auto detection at each boot, and different commands are used to scan for different types of new hardware.  For disks that have been added, use devfsadm(1M).  For any new hardware you need to perform a reconfiguration boot.  At the “ok” prompt use “boot ‑r”, or you can touch /reconfigure and reboot normally.

On Linux you can use lsdev to view a hardware inventory.  On Solaris use the  various prt* commands (e.g. prtconfig), the sysdef command, and the various *adm commands (e.g. cfgadm).

Summary

Safety, ESD (note this is cumulative!), grounding.  (cabling: type (crossover-null-modem/rollover/straight-patch), plenum, handling, placement, and length; EIA/TIA 568B standards, shock hazard.  Bottom line: Don’t be afraid to hire an expert!  Have spare parts: NICs, drives, cables, RAM, printer supplies too.  UPS.  Maintenance: BIOS updates, clean drives.  Keep inventory (including installed software standard configuration).

RAM

Use top, cat /proc/meminfo, vmstat to monitor RAM and swap space usage.

Disks and Tape controllers:

Controller information:  IDE (hdparm /dev/hda), SCSI which is used mostly on servers (scsi_info /dev/sda) due to better performance with multiple devices.  lsdev (if available), procinfo -a (demo).  smartctl -{iH} and smartd can be used to check, control and monitor disks (demo).  May be incompatible with hardware RAID.  If /dev/sd? doesn’t work, try the more generic driver /dev/sg#.  See also iostat for monitoring.

Temperature Monitoring

Many motherboards, CPUs, disk drives, etc., today include temperature sensors you can monitor.  For hard disks that support SMART you can get this info from hddtemp on Linux, and prtpicl -v -c temperature-sensor for
Solaris on Sparc, and
/usr/sfw/bin/ipmitool sdr list |grep temp for Solaris x86.

Write Cache

Most modern disks now support an on-disk write cache, usually enabled by default.  This cache accepts data from the OS and immediately tells the OS the data has been written.  This allows the drive manufacturer to quote more impressive disk speed numbers, but defeats the filesystem journaling mechanism.  While this might be acceptable for home computers, workstations, or laptops, the risk isn’t generally acceptable for a server or disks in a data center, unless redundant UPS and generators make the risk of power loss acceptable.

This can be disabled using hdparm ‑W on Linux, /boot/loader.conf setting of hw.ata.wc=0 on FreeBSD.

[Skip to the next section.]

Modems:

Generally speaking PCI (internal) modems have problems and external ones don’t.  To set up a modem I generally like to use the kppp GUI program (part of KDE).  It has several features that allow you to troubleshoot your modem setup, such as a log and a built in terminal window you can pop-up.

The setserial command allows you to query and set very low level options on serial ports.  The only real option you might need to set would be the speed.  To troubleshoot a modem/serial port, try a higher level tool to start with such as kppp, or use a terminal emulator such as minicomm or seyon to see if you can even talk to your modem.

Some software attempts to use /dev/modem to talk to the modem.  That should be a symlink to your actual serial port where the modem is connected.  Usually that would be /dev/ttyS0 (that is the same as MS Windows “com port 1”).  On some Unixes the device name is /dev/cua0 instead.

Solaris Disk Commands and /dev entries

uses /devices but has links in /dev (use those)
SCSI: /dev/[r]dsk/cAtBdCsD (A=controller#, B=SCSI-ID, C=unit#, D=partition#).  IDE: /dev/[r]dsk/cAdBsC (A=controller#, B=disk#, s=partition#).
/etc/vfstab
probe-scsi,
boot -r, dmesg
drvconfig
disks   # create /dev entries
format   #partitions disk
newfs  /dev/rdisk/c0t3d0s0   # adds a filesystem to a disk
fsck    #check disk for errors
tunefs, fstyp -v /dev/dsk/c0t0d0s0 # show/change disk settings

Other Hardware:

Serial: stty, setserial, getty (and pals), UART (USART) #s

Parallel:          printers, /dev/lp0, parport, parportpc, ..., lp.o  (just modprobe lp.o).  tunelp

USB, IR

USB (universal serial bus) is a replacement for older serial ports.  USB supports hot-plugging, which allows devices to be physically attached and removed at any time.  USB supports much higher speeds than RS-232 serial ports (up to 480 Mbps, that’s bits/second).

There is almost no limit on the types of devices that can be attached using USB: flash disks, portable disks and other media drives, mice and keyboards (USB has replaced older PS/2 ports for these), cameras, microphones and speakers, NICs, etc.  USB connected devices often include their own drivers in firmware, so no special OS drivers are needed.  (Some devices do require additional drivers.)  Most USB devices will appear as SCSI devices on your system.  In addition to /dev/sg# you will often get /dev/usb/* devices as well, .../lp0 for a USB printer for instance.

Unlike older serial ports USB connections are made to a hub (or USB controller).  The socket is referred to as a USB port.  A hub can connect up to 7 devices, including other hubs in a star-like hierarchical topology.  The top-level hub (the USB controller in the computer) is called the root hub (what else?).  A single root hub can support a total of up to 127 USB devices (counting the hubs too).

USB is a layered subsystem and may use several device drivers.  USB 2.0 for Linux requires the EHCI driver, plus a USB 1.1 driver, usually either UHCI (Intel and VIA chipsets) or OHCI (Compaq, most PowerMacs, iMacs).

Use lspci |grep ‑i usb to see if your host has any root hubs installed, and what drivers are needed for each.  You can see this info with dmesg too.

To see which modules are installed try:

  lsmod | egrep 'usb|hci|hid|mouse|Module'

(Some of the modules shown (hid, mouse) are related to user input.)

To see what USB devices are connected look at /proc/bus/usb/devices.  Like a lot of the /proc data there is a command to display the information nicely: lsusb.  The “-t” option will show a tree diagram.  the “-v” (verbose) option can be useful to find identifying data to use for udev configuration.

To see what LKMs might be needed for some USB attached device you can use a command such as: “usbmodules ‑‑device /proc/bus/usb/004/003”.

Linux and Unix (at least Solaris) have a separate hot-plug system just for USB.  On Linux this is done by bewildering set of files under /etc/hotplug.  Hot plugging for USB (and also PC cards) involves users plugging in devices while a system is running.  The system then has to:

·       Determine the device type and find and load a driver to run it

·       Bind the driver to the device  (e.g., have udev set up /dev entries)

·       Notify other subsystems about the device.  (This allows disks to be mounted, Networks configured, or print queues to be added, etc.)

Network

ifconfig, route, hostname, ethtool or mii-tool -v (Linux only; Solaris use ndd), lsof -Pi (lists open ports)

Sound And Audio Hardware

Processing and mixing audio signals has always been a CPU intensive job.  So sound cards became popular.  These contain a special CPU for the job called a DSP (a Digital Signal Processor).  (A common one is Intel’s AC’97.)  To work with sound software must send DSP processing instructions to the sound card.  Since each brand uses a different API, a device driver in the kernel is used to translate (“raw”) sound to the DSP’s API.  To make working with sound easier the kernel includes a platform-specific API for sound, and additional device4 drivers to translate that API to the DSP’s API.  In fact several drivers must be loaded, for the speaker ports, the DSP, the API, and sometimes “virtual” sound devices (that can accept “raw”sound).  Use “dmesg |grep ‑i aud” to see what type of audio hardware is installed.  In addition various utilities are present to convert between the standard sound formats, and to/from raw sound.

Raw sound (audio) consists of a series of integers that represent the sound at a given instant.  These are called samples.  The sample size can be 8, 16, or larger number of bits per sample.  The samples are recorded (and later played back) at a specific sample rate (number of samples per second).  Sound utilities allow raw sound to be resampled to change the size of the audio file by reducing the sample rate or the bit size of samples; naturally this loses quality.

To play sound in any of the dozens of popular formats requires conversion by software (usually a DLL you must install) called an audio codec (“codec” is an abbreviation for “encoder/decoder”).  A big problem today is that most audio formats are proprietary and require special licenses, and so don’t ship with your distro  (“Mint” Linux is an exception).  You must find and install codecs on your own.  Very few formats are public domain with free codecs, except the “Ogg” family of formats.  Others are usually available for free for playback.  (The vendor’s dream of market dominance and making money for the license fees for the encoder part of the codec.)  Note you have similar issues with video codecs.

The Creative Labs “Sound Blaster” series of sound cards have set de facto industry standards for sound card API and feature set.  Most sound cards provide a compatibility mode for one or more of the Sound Blaster series.  Many motherboards provide a sound chip with Sound Blaster compatibility on board.  The original Sound Blaster card was an 8-bit card. Today, most of these cards use the 32-bit PCI bus.

Many sound cards also have in interface to attach a MIDI (from Musical Instrument Digital Interface) device.  Commonly, this interface emulates the Roland MPU-401.  MIDI can play (and record) music as notes, played via some “voice” (instrument such as a piano or flugle horn) over several channels (2 for stereo).

·                 OSS (open source sound) is an early Unix/Linux platform sound API developed by Hannu Savolainen of Finland for Creative Labs SoundBlaster.  OSS is still maintained and used on Unix systems, however audio features were never a priority for Unix (essentially server) systems.  So for Linux a new API called ALSA (Advanced Linux Sound Architecture) was developed and is now widely used.  ALSA does support an OSS compatibility mode for old, legacy applications.

·                 ALSA is very powerful but very hard to program correctly.  In addition certain tasks are impossible by just using a DLL; a daemon is needed to mix and control sound from different processes.  Such a daemon is called a sound server or framework and includes its own API.  Initially there were many of these, including ones built into KDE and Gnome.

So, to play sound you need to play raw sound through an audio (DSP) device with a low-level device driver, or using the audio card’s API (some applications do this and ship with multiple drivers), or the platform’s API, or some sound server’s API.  Multiple device drivers and processes each think they control the sound hardware, leading to predictable results: lousy, intermittent, or no sound.  Fortunately most servers don’t need sound/audio!

(Windows has had this same architecture, with raw sound drivers, a platform API (“Windows Core Audio”), and various sound servers (“Direct Sound” and “Direct Music”, ..., and as of Vista, “Media Foundation”).

·                 Today Linux has two popular sound frameworks: JACK and PulseAudio.  These are built on top of ALSA and are interoperable with it.  Modern applications use one or the other to play audio, not ALSA directly (which would work but interfere with any other audio playing).  In addition there are many GUI and command line tools for working with audio.

Many professional-grade audio applications use JACK while most others use PulseAudio.  Some applications will use one and some the other, so it pays to install both.  PulseAudio has plug-in modules to send all output to or from JACK, but to make all the various sound servers and DLLs work together often requires some tweaking.

To find sound related utilities on your system, try this shell command:

 ( man -k sound; man -k audio) | egrep '\((1|8)' |sort -iu

Command-line utilities include play (see /usr/share/sounds/), rec, and sox which provides various sound conversions and was written to use the OSS API; aumix, xmix, aplay, amixer, alsa{mixer,ctl}, all written to use the ALSA API; tracker (MOD player), and playmidi, used for MIDI sound; and paplay, parec, and pavucontrol , which use the PulseAudio API and are the preferred CLI player, recorder, and volume control/mixer app.  You can use ffmpeg, sox, sound-convert, mplayer/mencoder,and others to convert sound files to many formats.

There are tons of open and proprietary audio tools, including media players, converters, and editors, such as Amarok, Audacious, Audacity, xmms2, BMPx, mpg123, Mplayer, Rythmbox, and VLC (available for Solaris too).  Both KDE and Gnome have their own GUI tools and frameworks as well.

There are also some text-to-speech utilities such as recite, festival, and espeak.

At the lowest level you can control all audio streams (input and output) that use the ALSA API with “alsamixer ‑c0”.

/dev files: audio, dsp (OSS), mixer, sequencer, sndstat.  Solaris commands that report hardware information include psrinfo and isainfo.

If the auto-detected sound loadable kernel modules don’t work, make sure that is the problem.  The ALSA system boots with sound muted.  The ALSA libraries must be configured initially to set volume levels, and to un-mute at boot time.

If  the audio hardware isn’t detected correctly use sndconfig to probe for PCI audio devices.  If your card isn’t found by the probe, you must set the parameters (IRQ, I/O and base addresses) manually.  However, sndconfig won’t detect conflicts if used to manually set parameters, so be careful!  On Solaris, see the man pages for audio and also audioplay.

With very old hardware (ISA bus), install the “isapnptools” software, and use isapnpdump to create /etc/isapnp.conf.  Edit that file, then use isapnp to configure the devices from the parameters in that file.

Video Cards:

Video cards (controllers) today have become very sophisticated.  To allow high quality, high-speed graphics software, the bulk of the work of processing video has been off-loaded from the CPU to the video card.  These now contain lots of RAM and even a CPU.  In fact the graphics card/controller is often referred to as a GPU -  Graphic processing Unit.  Of course there are many different video formats, a few free and open and many proprietary ones.  To use video (record, play, or convert to or from) you must have the correct video codecs installed.

GPUs are so powerful that today they can be used for other purposes, such as code breaking.  In face some computers contain multiple GPUs for intensive processing that multi-core CPUs can’t do well.

Graphic processing can be more time-consuming than even sound processing.  To speed things up the modern video cards support a standard API in hardware.  The two most popular such APIs are DirectX and OpenGL.  Since Microsoft embraces DirectX, many games use DirectX and most video cards support that.  Of course they may not support the most current version of DirectX!  If the software uses OpenGL or an old version of DirectX, a kernel device driver can be used to translate.  But also note many modern video cards support updatable firmware.

Today it is not uncommon to use your computer both as a workstation and also to listen to radio or watch TV.  These require special hardware (capture cards and tuners) and of course special device drivers.  These cards often are bundled with application software that uses the (custom) API of your hardware/device drivers, and that software may not work with another vendor’s hardware.

There are many free/open and costly/proprietary video players, recorders, and editors you can use to create your own videos, record, edit, and play video to/from your TV (a personal digital video recorder, or DVR), including the popular MythTV.  Fortunately, as with audio servers rarely if ever need to include GPUs or play media.  (However they often serve streaming media, but it is the clients that need the codecs and players, not the servers.)

Modern Device Management for Linux: udev

On older *nix systems you needed to create a /dev entry manually for each device added to the system using mknod.  This is such a pain that systems would ship with thousands of entries in /dev already, so if you added (standard) hardware the /dev entry for it was likely already there.

The big picture: What happens when a new bit of hardware is detected?

·       The hardware bus detects the hardware electrically.  It sends a signal to the kernel.

·       The kernel update /sys (sysfs) (or /devices on Solaris) with the information, then sends a hardware detected message to udev.  (A uevent is sent to the udevd daemon through a netlink socket—used to send messages between the kernel and applications.)

·       Udev consults sysfs for the device particulars, consults its rules files, and then creates one or more entries in /dev for the devices.  (Sysfs gets updated with the device name too.)  Next udev runs modprobe to load in any required drivers.  Finally udev sends an event to HAL over D-BUS.

·       HAL (discussed below) examines the device particulars from sysfs, the device drivers, etc. and creates an abstract representation of the device, with pre-defined properties any application can query, and with a standard interface so any application can configure the device.  HAL makes this representation available as a tree (much like /sys; use lshal), then publishes a new device present event over D-BUS.  Although not required by HAL, some systems (Fedora) have the hald automatically mount storage volumes under /media, provided there is no entry for them in fstab (which presumably would mean some automounter would do it, after getting the HAL message over D-BUS).

·       When running a GUI, the desktop manager is listening on D-BUS for new hardware (and other) events from HAL.  It consults a policy manager to decide what to do and what the current user(s) can do.  Typically it will draw new icons on the desktop for the new media and allow the user to mount/umount it as needed.

·       On Linux PolicyKit is used to decide what HAL operations are permitted by the user.  This may request the user authenticate again before allowing the operation.  This can be configured in several ways: on Debian-based systems the user needs to be a member of special groups (volume, power, ...) while Red Hat systems use various configuration files.

·       Per-user PolicyKit configuration and settings are managed by gconf (see ~/.gconf*).  Originally part of Gnome it is used by KDE too.  See gconftool-2 and gconf-editor tools to customize your policy.

Notice how confusing this is!  Also notice that the kernel, udev, and HAL all keep independent views of hardware, and that both udev and HAL send events on new hardware detected (and each can run a script in response to hardware changes).

Static Major Number Assignment

Some major device numbers are statically assigned for the most common devices.  A list of the static device numbers is in Documentation/devices.txt for Linux.  When all numbers are statically assigned, you need these “official” major device number for any device and that can be difficult to learn.  Some devices don’t have an official major number in the kernel you are using.

The first Linux attempt to fix these issues was devfs.  This was a system that watched as the kernel detected devices (a.k.a. a hotplug system).  As each device was detected an entry in /dev was created for it.  With devfs you could simply “ls /dev” to see what hardware was on your system.

The devfs has a few problems.  A newer version of this idea is now used for Linux: udev.  (Solaris uses devfs but it is a better system than the Linux devfs was.)  This too has some problems but is standard on many Linux distros including Fedora.  The udev system learns about hardware by examining /sys (sysfs).  Like the earlier /proc filesystem, /sys exposes kernel data.  Udev examines that, consults its configuration files, and creates entries in /dev and performs other tasks.  Solaris has a similar system in place to manage /dev entries called devfsadm(1M), that gets information from /devices (similar to Linux /sys).  (Solaris calls this devfs but it has nothing to do with the Linux system of that name.)

Explore /sys with tree:  All information is under devices.  To help find some devices a forest of symlinks is also available so you can find devices by bus, type (e.g. block), etc.  See ls -l /sys/devices/pci0000:00/ and note the numbers and virtual devices (same slot but different “function”).  “lspci” is a nicer way to see this information.  (Experts use setpci too.)

One great advantage to this system is that you can have the udev configuration assign device names based on vendor IDs and model numbers, MAC addresses (for NICs), etc.  So the same device gets the same name, always.  (Else a system with two NICs or several USB devices may have each device assigned a different name in /dev on each boot!)  So you can count on eth0 being the NIC you think it is, and /dev/sdc being the flash device you think it is.

Dynamic Major Number Assignment

In a modern kernel many devices can be assigned a major number dynamically, when the kernel module (the device driver) is determined for some attached hardware (at boot time).  Using dynamic assignment is easier (no need to register the device number with the official kernel source) and encouraged.

The disadvantage of dynamic assignment is that you can’t create the device nodes in /dev in advance, because the major number assigned to some driver won’t always be the same.  However the major number assigned to a device can be read from /proc/devices, and udev uses that information (as well as information from /sys) to create the device node files (/dev entries).  Even when using dynamic assignment, some major (and minor) numbers are assigned statically.

Linux new (2010) inlcludes a ramdisk (using tmpfs) called devtmpfs that is mounted at /dev and populated using dynamic major numbers.  This means you have a functional system long before init runs, so “init=/bin/sh” works well, and so initramfs and init scripts can be made simpler.  Note you don’t get the power of udev at this point, just a basic system and no persistent device naming.

[Skip to the next section.]

Udev Configuration:

/etc/udev/udev.conf in the main config file, used to say where the rule files are, and some default values (say for the permissions of new entries under /dev).

/etc/udev/rules.d/ is a directory of rules files.  These files tell udev what names to create in /dev for different devices, what permissions, owner, group it should have, any additional symlinks to setup, and what additional steps should be done when the device is added or removed (if any).

These rule files are edited when you need the system to do something special when certain hardware is attached or removed, when the default is wrong, or when you want some special action (e.g., a shell script) to run.  However, some rules files are also edited automatically, for example when a NIC is detected, udev will record its MAC address in a rule file so that NIC gets the same name (e.g., “eth0”) every time.

Why the system sometimes renames eth0 as eth1:

Your system may rename the NIC.  This happens because udev stores the MAC address of the NIC, so it always gets the same name.  It puts that info in /etc/udev/rules.d/70-persistent-net.rules and uses that data to name the NIC.  But if you use your removable disk in the open lab, a new MAC address is found for the single NIC the kernel will call eth0.  The rule for the old NIC’s MAC address is changed to name it eth1.  When you get back to the classroom, udev gets sees the old MAC address again and creates eth1.  (And if you used a third host in the lab you might get eth2!)

This is a good thing for Flash drives but a pain with NICs.  To fix, just stop udev and then edit or delete the file 70-persistent-net.rules.  The NIC will be named eth0 after a reboot.


Solaris has a bunch of files in /etc with that information such as /etc/system, /etc/name_to_major, /etc/minor_perm, /etc/security/device_policy, and others.  Solaris manages device names in /dev using “devfs” and the devfsadm and cfgadm commands.  Usually the names in /dev are just symlinks to names in /devices.

The “*.rules” files in the rules directory are run in alphabetical order, looking for rules that match the detected hardware.  Generally udev stops looking after the first match, so to over-ride the default behavior you need to create a new rules file that gets run before the standard ones, say  “00-local.rules”.

Specifying rules is complicated because there is no usable standard for what a device reports when detected — every one may report differently.  Also there is no standard for kernel module developers as to what to name devices.  These things just grew over time as different developers had different ideas.  (It’s also complex because the udev syntax is just plain ugly!)

An SA should be able to get new hardware working; this may mean editing udev rules.

A udev rule contains a list of “match” keys and “action” keys.  Match keys are of the form name==value and are used to identify a device depending on what is reported for it.  All the match keys of a rule must be true for the rule to match.  The match keys try to match the rule to the device, according to the information in the kernel event message and the sysfs data for the device (identified by the event message).

Examples of match keys include:  KERNEL==subsystem (the subsystem of the kernel that controls the device, such as block), DRIVER==type, and ATTRS==value (match one or more sysfs listed device attributes).

The match rules can use pattern matching, and the matching values are put into variables you can refer to in the action keys.

Action keys are of the form name=value (or name+=value if name is a list) and are used: to name the /dev entry, to set the owner, group, and permission, to define additional symlinks, and to define a list of programs (scripts) to run.  A common action is to send a message to HAL about the new hardware, but could be any command or program (such as a shell script that mounts a new USB disk someplace).  Note several environment variables are set by udev before calling any program, to provide additional information to the program.  See udev(7) for a complete list of match and action keys.

Here’s an example rule:

KERNEL=="tty[0-9]", GROUP="tty", MODE="0660", OPTIONS="last_rule"

(Notice the OPTIONS key that says last_rule.  This means if this rule matches, don’t look for any other rules.)

Writing rules for udev is an usually not needed; the default rules work well.  You should be able to plug in a device and the a relevant node (e.g. /dev/sda for a mass-storage device) will be there, just like in previous /dev implementations.  [ The following is from: reactivated.net/writing_udev_rules.html ]

But it may be useful to create new rules.  Suppose you have two USB printers.  When one is plugged in it gets named as /dev/lp0.  A more convenient name might be /dev/printer.  By editing the correct rule you can pick your device names (or let it name the device normally, but add a symlink with a nicer name).

Case Study:  Sierra wireless 3G PCMCIA card

To get a cellular Internet connection you have to run “ifup ppp0” each time the card is inserted.  You can configure udev to automate the process. by default when the card was detected, it created a series of /dev/ttyUSB devices from 0 through 5.  To have a udev rule run the ifup command, you need to determine a unique product ID for the device.

You can determine the information if you know the device path (under /sys).  this can be done with:

       # udevinfo -q path -n /dev/ttyUSB0

You can then use udefinfo to display the information for that device:

# udevinfo -a -p $(udevinfo -q path -n /dev/ttyUSB0)

If you examine the output you’ll discover an idProduct key that was set to 6880 and seemed unique for this type of card.  You can use this to write the new udev rules, in the file .../rules.d/95-sierra.rules:

KERNELS=="5‑1", ATTRS{idProduct}=="6880", ENV{ACTION}=="add",
 RUN+="/sbin/ifup ppp0"

KERNELS=="5‑1", ATTRS{idProduct}=="6880", ENV{ACTION}=="remove",
 RUN+="/sbin/ifdown ppp0"

Persistent Naming

It’s not only a convenience, it’s a solution for persistent naming.  Suppose you have two printers: a HP laser printer and an Epson inkjet. When they are both plugged in and on, I have /dev/lp0 and /dev/lp1.  Which node refers to which printer?  There is no easy way to tell.  The first printer that got connected was assigned name “lp0” and the second “lp1”, but they might have been plugged in for days (or plugged in at boot time).  Not having consistent names would mess up scripts that always expect the HP laser printer to be lp1.

However, if the HP laser printer got named lp_hp (as well as lpX) and the other printer got named lp_epson (as well as lpY), then scripts could just refer to those names.  Udev can control this and ensure that these persistent names always point to the device that you intended.

For external mass-storage devices (e.g. USB hard disks), persistent naming is very helpful in that it allows you to hardcode accurate device paths into /etc/fstab, or have the desktop icon say “Sans mini 8GB” instead of “Flash” or “sdb1”.

If you need to write a rule (e.g., to automate mounting of a USB device) there are some decent tutorials on the Internet such as at www.reactivated.net/udevrules.php.

Step 1:  Gather information to write your udev rule.  Start udevmonitor (to see the uevent details), then plug in your USB drive.  You can also look at tail ‑f /var/log/messages or at dmesg output, or lshal or hal‑device-manager (GUI).

You can observe the uevents passed between the kernel and udevd with udevmonitor.

See what information is reported that you can use to identify this device.  If you want to be real specific use the manufacturer’s name and the device’s serial number.  For example using dmesg after inserting my USB stick/flash/pen drive (interesting information is in boldface):

# dmesg
usb 1-6: new high speed USB device using ehci_hcd and address 3
usb 1-6: configuration #1 chosen from 1 choice
Initializing USB Mass Storage driver...
scsi5 : SCSI emulation for USB Mass Storage devices
usbcore: registered new driver usb-storage
USB Mass Storage support registered.
usb-storage: device found at 3
usb-storage: waiting for device to settle before scanning
  Vendor: SanDisk   Model: Cruzer Titanium   Rev: 2000
  Type:   Direct-Access                      ANSI SCSI revision: 00
SCSI device sdb: 1014784 512-byte hdwr sectors (520 MB)
sdb: Write Protect is off
sdb: Mode Sense: 43 00 00 00
sdb: assuming drive cache: write through
SCSI device sdb: 1014784 512-byte hdwr sectors (520 MB)
sdb: Write Protect is off
sdb: Mode Sense: 43 00 00 00
sdb: assuming drive cache: write through
 sdb: sdb1
sd 5:0:0:0: Attached scsi removable disk sdb
usb-storage: device scan complete

Note the USB bus id shown first (1-6). Use that to see the device information:

cd /sys/bus/usb/devices/1-6; ls
cat manufacturer serial

Another way is to look there for the file usb_device:name (in my case it is usbdev1.3).  Use this to get all sysfs information that may be used with udev:

     udevinfo -ap /class/usb_device/usbdev1.3

many blocks of data will be reported, for each sub-device (the USB bus device, the PCI bus device, etc.)  One block will obviously be for your specific device.  You can use any of that information to identify the device in your rule.

Here’s the abbreviated output for my flash drive with the most useful data in boldface:

follow the "device"-link to the physical device:
  looking at the device chain at '/sys/devices/pci0000:00/0000:00:1d.7/usb1/1-6':
    BUS=="usb"
    ID=="1-6"
    DRIVER=="usb"
    SYSFS{bConfigurationValue}=="1"
    SYSFS{bDeviceClass}=="00"
    SYSFS{bDeviceProtocol}=="00"
    SYSFS{bDeviceSubClass}=="00"
    SYSFS{bMaxPacketSize0}=="64"
    SYSFS{bMaxPower}=="100mA"
    SYSFS{bNumConfigurations}=="1"
    SYSFS{bNumInterfaces}==" 1"
    SYSFS{bcdDevice}=="2000"
    SYSFS{bmAttributes}=="80"
    SYSFS{configuration}==""
    SYSFS{devnum}=="3"
    SYSFS{idProduct}=="7108"
    SYSFS{idVendor}=="0781"
    SYSFS{manufacturer}=="SanDisk Corporation"
    SYSFS{maxchild}=="0"
    SYSFS{product}=="Cruzer Titanium"
    SYSFS{serial}=="00000000000000217115"
    SYSFS{speed}=="480"
    SYSFS{version}==" 2.00"

Step 2:  Decide what /dev name you would like to use.  /dev/flash0, 1, ... or /dev/cruzer, /dev/SanDisk-titanium, ...  You can have additional symlink names.  Note by default interesting symlinks are created in /dev/disk.

Step 3:  Create an appropriate fstab entry, specifying some mount point and mount options, for the device name you picked.  Most flash drives are vfat and common options include user=, uid=, gid=, mode=, nodev, noatime, and others.  Be sure to actually create this mount point if you want the device to be automatically mounted when inserted.  Otherwise, chose a mount point under the control of an auto-mounter.

Step 4:  Create your rule in the file /etc/udev/rules.d/00-local.rules, which you may have to create.  This rule can include a shell script to run.  The script in turn adds/removes icons from your desktop, and possibly mounts/umounts the device too (if no auto-mounter is used).  Udev can but shouldn’t mount anything.  It is better to use HAL or an auto-mounter.

The script will be passed information in environment variables that say if the device has just been added or removed.  My rule used the default kernel name for the disk but adds a symlink for /dev/cruzer#:

BUS=="usb", SYSFS{manufacturer}=="SanDisk Corporation", SYSFS{product}=="Cruzer Titanium", SYMLINK+="cruzer%n", OPTIONS="all_partitions", RUN+="pathname_to_script_or_program"

(all one line).  The script or program is passed an environment variable saying if the ACTION was ADD or REMOVE, and other information.  You can test your new rule(s) with:

            udevtest /block/sda/sda1 usb

D-BUS

D-BUS is a simple IPC (interprocess communication) protocol that allows asynchronous messages be either directly sent between two applications (peer to peer model), or be “broadcast” (routed among multiple applications; this is the hub model).  In other words D-BUS manages a system-wide message bus.  Processes can listen for and/or send messages to other processes.  You can easily set up multiple busses so sets of processes can talk to each other.

Until D-BUS applications could only communicate and cooperate using primitive means of pipes, sockets, shared memory, and file systems (shared files or named pipes).  Usually only two applications could communicate with each other; no broadcast, and a lot of setup and programming support is needed.

The various hub buses use dbus-daemon processes as message routers to forward the messages back and forth (the peer to peer model buses don’t need a daemon.)

There are two standard hub buses that a developer can rely on always being around.  These are the system bus and the session bus.

The system bus can be used by any application running in any context to to post or receive messages.  It is a single point where applications can export services that anyone can use.  Only one system bus daemon can be run at a time.

The session bus is local to the current user’s X session.  It is used for communication between applications running within the same GUI session.  For every login to X one session bus daemon is started.

A modern *nix system uses the D-BUS session bus to “post” hardware change events, replacing older mechanisms such as CORBA (Used by old Gnome) and DCOP (used by old KDE).  Desktop applications can then refresh the desktop icons or “pop-up” messages.

Every bus has an address describing how to connect to it.  A bus address will typically be the filename of a Unix-domain socket such as “/tmp/.session” but it may also be a TCP port where a bus daemon is listening.  All D-BUS functionality is encapsulated in libdbus.so.  The library is used directly for peer-to-peer communications or to connect with some bus (hub).  Each application that connects to a bus must provide a unique (to that bus) service name so other applications can communicate with it.  (They will need to know that name before they can talk to that application.)

D-BUS messages have a header (the sender bus name, destination bus name, message type, etc.) and a body.

Each D-BUS hub bus has an XML config file (documented in dbus‑daemon(1)) that specifies the user the daemon should run as, the socket to listen on, various resource limits (e.g., the max size of a message), the location of other config files (e.g., each service can have a config file which would allow D-BUS to start an application automatically when a message is sent to its service name), and a security policy that controls the communications, and other information.  Since one config file can “include” others, it is common to put the security policy in a separate file.

D-BUS configuration is located in /etc/dbus directory and consists of several XML files, but D-BUS just works.  An SA shouldn’t need to mess with it.

You can use some command line tools (dbus-*) to send and receive messages.

More information can be found in the tutorial at: dbus.freedesktop.org/

HAL

HAL (Hardware Abstraction Layer) provides a view of the hardware attached to a system, updated dynamically as hardware configuration changes via udev (or other mechanisms).  HAL represents a piece of hardware as a device object.  A device object is identified by a unique device identifier (udi) and carries a set of key/value pairs referred to as device properties.  Some properties are derived from the actual hardware, some from device information files (.fdi files), some from sysfs (/sys), and some are related to the actual device configuration.  HAL allows applications to easily determine what hardware is attached to the system.  It is especially handy for GUI desktop systems to “pop-up” icons for attached disks or other USB (or other removable) devices, start up networking (and alert the user) when the NIC detects a link is up, etc.  HAL should just work and not require any tinkering by an SA.

HAL provides any program an easy way to register for D-BUS event notification of added/removed hardware.  (This depends on running “hald”, the HAL daemon).  So to use HAL applications must listen for the standard HAL D-BUS messages.

On modern systems hardware busses such as USB, PCI, PC-card., ... notify the kernel as hardware insertion/removal is detected, via the bus’ device driver.  Then kernel loads required device drivers and/or notifies udev.

Once udev creates or removes /dev entries (and loads the correct drivers), it runs scripts located in /etc/dev.d, which makes udev send a message to the hald daemon.

For mountable devices hald will create a mount-point under /media (by default) and a new entry in /etc/fstab via a process called fstab‑sync.  Finally hald notifies GUIs or other interested processes by posting the “new hardware” message via D-BUS.  Any interested processes must extract the device path from the respective property and use OS-specific APIs to access the device.  (Actually, on FreeBSD at least, HAL won’t initially mount anything.  Instead it waits for the GUI or some other app to request the mount.  HAL then makes sure the current user has permissions to do so, via policykit, and then mounts the device.)

Modern desktop environments use HAL to automatically add desktop icons for newly added flash drives, which may or may not be automatically mounted.  (If not, then when you click the icon the drive gets mounted.)  Removing the drive causes HAL to remove the mount-point and the fstab entry, and to notify the GUI system so it can remove any icons.

Not all software has been updated to be “HAL-aware”.  So an older CD or DVD buring program may not run; this happens because both the app and HAL attempt to access the drive simultaneously.  You can use hal‑lock to run such apps.

See man pages for hald, lshal, hal-device-manager (GUI python version of lshal), hal-get-property and hal-set-property (used in scripts), other hal-* commands, libhal.so (check apps with ldd for this to see if they use HAL), /etc/dbus/system.d/hal.conf (D-BUS security policy for HAL), /usr/share/hal/fdi/ (directory for .fdi files), and /etc/hal (holds configuration and scripts).

If your system doesn’t support D-BUS/HAL, or you’re not running HAL and/or some GUI and would prefer to setup an auto-mounter instead when a USB flash drive is inserted, you can add a custom udev rule that will run a script or program whenever the device is inserted or removed.

First the script must test if the action was ADD or REMOVE.  On ADD the script should test if a GUI is running, and if so try to create a desktop icon depending on which GUI is running.  Next the disk may or may not be mounted by the script.  If not I suggest you chose a mount point managed by an auto-mounter.  This is the easiest setup: just add the device name (/dev/cruzer0) to fstab, and configure the auto-mounter for, say /media/cruzer.

In 2009 HAL started a major rewrite, the result has a new name: DeviceKit.  (There is also DeviceKit-disks and DeviceKit-power, each manages just those types of devices.)  DeviceKit should be simpler than HAL was and let udev take over the overlapping functionality.  The new system should enhance the GUI admin tools while not impacting system administrators.