Linux Training : e. Filesystem

This page last changed on Aug 13, 2013 by jkinney.

Home Page

  • filetree, differences, mountpoints and filesystem types

    Linux is organized around files in a tree arrangement. The top of the tree is called "the root" and is designated "/". Everything is attached to the tree at some point. Hard drives are "mounted" to specified points in the tree. So are CD drives, USB thumb drives. Even network process use special files called "sockets". By using tools like devfs, Linux systems can keep track of UUID numbers for hard drives and USB thumbdrives and always make them available in the same location.
    The / section of the RHEL tree looks like 
    /
|-- audit_backup
|-- bin
|-- boot
|-- dev
|-- etc
|-- home
|-- lib
|-- lib64
|-- lost+found
|-- media
|-- misc
|-- mnt
|-- net
|-- opt
|-- proc
|-- root
|-- sbin
|-- selinux
|-- srv
|-- sys
|-- tftpboot
|-- tmp
|-- users
|-- usr
`-- var

    The regularity of the Linux filetree is essential in the boot up process. There is a standard form of the tree described by the Filesystem Hierarchy Standard
    User data is typically stored in their home directory which is in /home for users with local accounts. Users with network accounts may have their home directories in /users or /usr/local/users or even in /opt.

    (lightbulb) Where is your home directory? From a shell window or console, type: 

     echo $HOME

    to find out. Remember, this is a command. It doesn't actually do anything until you tell it to by pressing Enter.


    What about the other directories?
    Some, such as /proc, /dev and /sys are not storing real "files". /proc is a way to look inside the kernel and see the processes that are running and what a lot of internal values are. /dev is where the system keeps track of all the physical and virtual devices. /sys is similar to /proc but provides information about the in-kernel status to userspace objects like devices, busses and loaded modules.
    /etc is the directory that should be looked at first for system configuration. Every application, process, device and account is configured here (unless it's non-standard and done in /usr/local/etc - more on /usr/local later). RedHat system will usually put configuration data for start up processes in the /etc/sysconfig directory and kernel level parameters in /etc/sysctl.conf file. Other configuration files in /etc include passwd, group, shadow and gshadow ( these control local usernames, passwords and group affiliations), skel (which provides the default startup new user environment), resolve.conf (for storing the DNS servers for network name resolving), and everything else from a2ps (an ascii to postscript converter)to zshrc (configuration for the z-shell environment). There are some special directories in /etc that are used to decide what runs when during system start up and shutdown. These are the rc* directories. rc1.d, rc2.d, rc3.d, through rc6.d . rc is for "runtime configuration". The numbers refer to different runlevels. Putting a specific process into the correct rc* directory can be done manually (they are just symlinks) but it's more efficient to use the chkconfig command

  • Difference between Linux and Microsoft filesystems

    Microsoft system were designed around the home user machine. So it's designed around drives. Add a new drive and your system may not boot again! Or your CD burner is no longer drive D: but it's now drive E: and your USB thumbdrive changes drive letters also depending on whether your external back up drive is plugged in as well.
    Linux systems don't automatically reconfigure what you already have just because you add a new part to your system.

  • Mount points

    OK. So you have to tell it where to IT:mount that new 4TB drive array you grabbed from craigslist for $50. (wink)
    Drives that are always mounted when the system boots up are configured in the /etc/fstab file (think:file system table). For example here's a typical desktop system here at GTRI: 
    /dev/VolGroup70/LvRoot  /                       ext3    defaults        1 1
/dev/VolGroup70/LvLogs  /var/log                ext3    defaults        1 2
LABEL=/boot             /boot                   ext3    defaults        1 2
tmpfs                   /dev/shm                tmpfs   defaults        0 0
devpts                  /dev/pts                devpts  gid=5,mode=620  0 0
sysfs                   /sys                    sysfs   defaults        0 0
proc                    /proc                   proc    defaults        0 0
/dev/VolGroup70/LvSwap  swap                    swap    defaults        0 0
    Yikes! What is this mess?! It's really pretty simple once you learn the meaning of the different columns. 
    source                  mount point             format  parameters      dump and check parameters
/dev/VolGroup70/LvRoot  /                       ext3    defaults        1 1
/dev/VolGroup70/LvLogs  /var/log                ext3    defaults        1 2
LABEL=/boot             /boot                   ext3    defaults        1 2
tmpfs                   /dev/shm                tmpfs   defaults        0 0
devpts                  /dev/pts                devpts  gid=5,mode=620  0 0
sysfs                   /sys                    sysfs   defaults        0 0
proc                    /proc                   proc    defaults        0 0
/dev/VolGroup70/LvSwap  swap                    swap    defaults        0 0
    The source is where in the filetree the device file is located for that source. /dev/VolGroup70/LvRoot is a volume group that hold the /, /var/logs and swap space. So clearly the mount point is the place in the filetree that device is found for use by something other than the mount command. The format is the filesystem type. In the first line example, it is ext3. The parameters column defines how to perform the mount. defaults means the following options are used:

    rw - Mount the file system read-write.
    suid - Allow set-user-identifier or set-group-identifier bits to take effect.
    dev - Interpret character or block special devices on the file system.
    exec - Permit execution of binaries.
    auto - Can be mounted with the -a option (i.e. automatically at boot time or manually with the -a)
    nouser - Forbid an ordinary (i.e., non-root) user to mount the file system. This is the default.
    async - All I/O to the file system should be done asynchronously.

    The remaining two fields are used by the dump process (a very basic back-up tool) and the fsck (file system check) process respectively. A dump field of 0 means no dump needed. An fsck field other than 0 directs the fsck process to the order to perform a filesystem check. 0 means no check.

    (lightbulb) What do you have mounted right now? 

    mount

    will show you something like

    /dev/mapper/VolGroup70-LvRoot on / type ext3 (rw)
proc on /proc type proc (rw)
sysfs on /sys type sysfs (rw)
devpts on /dev/pts type devpts (rw,gid=5,mode=620)
/dev/mapper/VolGroup70-LvLogs on /var/log type ext3 (rw)
/dev/sda1 on /boot type ext3 (rw)
tmpfs on /dev/shm type tmpfs (rw)
none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw)

    (question) Can you identify all the parts based on what you just read?

  • filesystem types

    • ext2 : older format that did not support a rapid filesystem recover process called journalling.
    • ext3 : modern format that supports journalling by tracking open files with a write log journal that is written to first, then the actual file is written to and the journal entry is deleted. Very useful if the system goes down without first performing a write buffer flush operation.
    • ext4 : new replacement for ext3 that is faster and solves some issues with ext3. Both ext3 and ext4 use journals but ext4 has greatly expanded filesystem size limits (exabytes) and file sizes (16 terabytes). See http://en.wikipedia.org/wiki/Ext4 for a good overview of ext4 improvements over ext3.
    • swap : swap is special filesystem space used to park memory that has not been used in a while when the kernel needs more memory than it has available. It is similar to "virtual RAM" in that is uses the much slower hard drive space. When a server is thrashing it is constantly swapping memory back and forth from swap space. The system gets slower and slower until an overly large process is killed off and the swap space is no longer in heavy use. NOTE: if there is a drive error in the swap space partition and the kernel tries to access that data, the system will typically crash.
    • nfs : There are 3 versions of the Network File System: nfs, nfsv3 and nfsv4. the ancient nfs should never be used as it's totally insecure and very buggy. nfsv3 addressed the security issues but left the instability issues. nfsv4 is both fairly securable and quite stable. With nfs, a file server can provide a common data storage area across a LAN (or WAN with v4 and vpn tunneling).

  • Filesystem quiz

    Filesystem exercise

    (question)

    1. What are all of the /dev/names for your CD drive pointing to?
    2. What are the different names for the hard drive(s) in your system?
    3. What is the hardware device and filesystem type of the /boot partition?
    LIN:page top

  • Answers

    Filesystem

    1. What are all of the /dev/names for your CD drive pointing to? 
      ll /dev/[cd][dv][rwd]*
lrwxrwxrwx 1 root root 4 Apr 26 09:07 /dev/cdrom -> scd0
lrwxrwxrwx 1 root root 4 Apr 26 09:07 /dev/cdrom-sr0 -> scd0
lrwxrwxrwx 1 root root 4 Apr 26 09:07 /dev/cdrw -> scd0
lrwxrwxrwx 1 root root 4 Apr 26 09:07 /dev/cdrw-sr0 -> scd0
lrwxrwxrwx 1 root root 4 Apr 26 09:07 /dev/cdwriter -> scd0
lrwxrwxrwx 1 root root 4 Apr 26 09:07 /dev/cdwriter-sr0 -> scd0
lrwxrwxrwx 1 root root 4 Apr 26 09:07 /dev/dvd -> scd0
lrwxrwxrwx 1 root root 4 Apr 26 09:07 /dev/dvdrw -> scd0
lrwxrwxrwx 1 root root 4 Apr 26 09:07 /dev/dvdrw-sr0 -> scd0
lrwxrwxrwx 1 root root 4 Apr 26 09:07 /dev/dvd-sr0 -> scd0
lrwxrwxrwx 1 root root 4 Apr 26 09:07 /dev/dvdwriter -> scd0
lrwxrwxrwx 1 root root 4 Apr 26 09:07 /dev/dvdwriter-sr0 -> scd0
      * ll is a RedHat alias for ls -l. The [LIN:cd] means choose one of the enclosed characters. The same process applies to the [LIN:dv] and [LIN:rwd]. By choosing to use regular expressions for the first three letters of the it was easy to grab all combinations at once of cdr, cdw and dvd.
      # What are the different names for the hard drive(s) in your system? 
      ll /dev/disk/*
/dev/disk/by-id:
total 0
lrwxrwxrwx 1 root root  9 Apr 26 09:07 scsi-SATA_ST3160811AS_6PT0VFZM -> ../../sda
lrwxrwxrwx 1 root root 10 Apr 26 09:07 scsi-SATA_ST3160811AS_6PT0VFZM-part1 -> ../../sda1
lrwxrwxrwx 1 root root 10 Apr 26 09:07 scsi-SATA_ST3160811AS_6PT0VFZM-part2 -> ../../sda2

/dev/disk/by-label:
total 0
lrwxrwxrwx 1 root root 10 Apr 26 09:07 boot -> ../../sda1

/dev/disk/by-path:
total 0
lrwxrwxrwx 1 root root  9 Apr 26 09:07 pci-0000:00:1f.2-scsi-0:0:0:0 -> ../../sda
lrwxrwxrwx 1 root root 10 Apr 26 09:07 pci-0000:00:1f.2-scsi-0:0:0:0-part1 -> ../../sda1
lrwxrwxrwx 1 root root 10 Apr 26 09:07 pci-0000:00:1f.2-scsi-0:0:0:0-part2 -> ../../sda2
lrwxrwxrwx 1 root root 10 Apr 26 09:07 pci-0000:00:1f.2-scsi-1:0:0:0 -> ../../scd0

/dev/disk/by-uuid:
total 0
lrwxrwxrwx 1 root root 10 Apr 26 09:07 5e36b847-df4a-40dd-ab8f-573286f780c9 -> ../../sda1
      * The /dev/disk directory is very useful for showing the different ways a device can be named. Most of the names are only used internally. However, it is possible to "see" how the drives are physically connected with the /dev/disk/by-path data coupled with the output of /sbin/lspci 
       /sbin/lspci
00:00.0 Host bridge: Intel Corporation 4 Series Chipset DRAM Controller (rev 03)
00:01.0 PCI bridge: Intel Corporation 4 Series Chipset PCI Express Root Port (rev 03)
00:1a.0 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #4
00:1a.1 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #5
00:1a.2 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #6
00:1a.7 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB2 EHCI Controller #2
00:1b.0 Audio device: Intel Corporation 82801JI (ICH10 Family) HD Audio Controller
00:1c.0 PCI bridge: Intel Corporation 82801JI (ICH10 Family) PCI Express Root Port 1
00:1c.3 PCI bridge: Intel Corporation 82801JI (ICH10 Family) PCI Express Root Port 4
00:1c.4 PCI bridge: Intel Corporation 82801JI (ICH10 Family) PCI Express Root Port 5
00:1c.5 PCI bridge: Intel Corporation 82801JI (ICH10 Family) PCI Express Root Port 6
00:1d.0 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #1
00:1d.1 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #2
00:1d.2 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #3
00:1d.7 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB2 EHCI Controller #1
00:1e.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev 90)
00:1f.0 ISA bridge: Intel Corporation 82801JIR (ICH10R) LPC Interface Controller
00:1f.2 SATA controller: Intel Corporation 82801JI (ICH10 Family) SATA AHCI Controller
00:1f.3 SMBus: Intel Corporation 82801JI (ICH10 Family) SMBus Controller
01:00.0 VGA compatible controller: nVidia Corporation G98 [GeForce 8400 GS] (rev a1)
03:00.0 SATA controller: JMicron Technology Corp. JMB362/JMB363 Serial ATA Controller (rev 02)
03:00.1 IDE interface: JMicron Technology Corp. JMB362/JMB363 Serial ATA Controller (rev 02)
04:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168B PCI Express Gigabit Ethernet controller (rev 02)
05:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168B PCI Express Gigabit Ethernet controller (rev 02)
06:07.0 FireWire (IEEE 1394): Texas Instruments TSB43AB23 IEEE-1394a-2000 Controller (PHY/Link)
      * Look at the first part of the names in by-path - pci-0000:00:1f.2. The section that begins after the first ":" 00:1f.2 identifies the pci controller in the lspci output as being 
      00:1f.2 SATA controller: Intel Corporation 82801JI (ICH10 Family) SATA AHCI Controller
      . So all of the drives on the system are connected to a single SATA controller.
    2. What is the hardware device and filesystem type of the /boot partition?
      */etc/fstab* tells use the /boot is an ext3 type filesystem as has the label "/boot". My using the mount command we can see the hardware device is /dev/sda1 (on the machine this was written from).

    LIN:page top