CTS-2301C (Unix/Linux Administration I) Project #6
Enabling Services

Due: by the start of class on the date shown on the syllabus

(This is a long project; allow extra time to complete.)

Description:

Services (file, print, remote access, database, etc.) are provided by server programs (daemons) on your host.  Some modern servers make a copy of themselves for each request to that service.  So at any given moment you may see several instances of a server daemon running, or none at all, depending on how many requests for that service are currently being handled by that server.  Running servers you don't need wastes memory and CPU cycles, enough to have a user-noticeable effect on system performance.

For this project you are to disable some services (enabled by default during the installation) and to enable the following services on your classroom computer:

There are two types of servers, stand-alone servers such as web, database, and mail servers, and on-demand servers such as telnet.  The difference between these two types of services is that a stand-alone server is “always on” once enabled.  This makes for much faster responses to web page or database requests, since it takes these servers several seconds to start up each time.  Some service take so long to start, that at boot time a number of server instances (processes or threads) are created in advance, so they are ready to go as soon as requests arrive.

A program (client or server) that is always on is called a daemon (pronounced either as “DAY-mən” or “DEE-mən”), after the Greek term for “a person's attendant spirit”.  (A demon would be an evil spirit.)

Daemons historically are enabled or disabled in sets called run-levels which are identified by a number.  Changing run-levels will turn off some servers and turn on others.  Your system has a default run-level “entered” when you boot up; for Fedora, this is run-level “5”.  (Modern systems are moving away from run-levels, but have equivalent concepts; Linux systemd uses targets in a similar way.)

On the other hand, once services such as telnet are enabled a server (still called a daemon) is started on-demand for each request received.  Services such as telnet and FTP can start up quickly so there is no need to have daemons running all the time.  Such services used to be managed by a single (always on) “super” server called “xinetd” (or “inetd” on older systems).  More recent Linux systems using systemd don't need any additional daemon to manage on-demand services.  With systemd, all services are controlled via unit files, typically installed in /lib/systemd/system instead of under /etc.  Note that on-demand servers cannot be manually started and stopped like stand-alone servers.  All you can do is enable/disable them, so they are ready to start (or not) when the next request arrives.  This is done by configuring the systemd socket, or for legacy services a “super” server.

How you list, start, stop, enable, or disable servers or other daemons depends entirely on your host's init system.  Since the 1980s, most systems have used the “System V” (or “SysV”) init system.  This uses shell scripts and directories of symlinks to manage daemons.  The commands available to manage services include chkconfig and service on most systems.

Fedora and most other Linux distros have switched to a new init system, “systemd”.  This system uses the command systemctl to manage daemons.  Most, but not all, services have migrated to systemd.  On your system, you may or may not have any legacy init services to manage.

For details on these commands and init systems, see the class notes, man pages, and resources provided on our class website.

Make sure you keep an accurate system journal of any and all changes you make to your system!  You will need to turn this in, along with the answers to the questions asked below.

It should be noted that some systems have firewalls running that would prevent access to some services.  The default firewall generally will allow access from localhost to any service running on that host, so you can ignore any firewall issues (for now).  Also, the default file and directory permissions should be fine.

However, other security sub-systems may block access that you will need to manage or turn off.  These include TCP Wrappers, SE Linux, and possibly PAM.

Requirements:

Answer the following questions and perform the following tasks:

  1. Identify and disable all unused services:
    1. Run the following commands as root, to create two service reports that show all running services:
         chkconfig --list > ~/service-report.chkconfig  # For legacy services
         systemctl --all --full --type service > ~/service-report.systemctl
      

      If you don't have any legacy "System V init" services installed, there will be no output from the chkconfig command, and the report you generated will be empty.  If that's the case, you can ignore legacy services for the rest of this assignment.

      In addition to creating the report files in your home directory, these commands may send some warning messages to stderr; ignore those.  (For systemd services, there are other reports you can generate.  Try “systemctl list-unit-files -t service” which is more readable but doesn't include the service descriptions.  I prefer a third report created by this command: “systemctl --all --full --type service |grep running |sed 's/ \{20\}//g'”.)

      Examine these reports (the two text files created above).  Note that with systemd, service names all end in “.service”.  However, you won't see that part of the name with other tools (such as ps or in log files.)  Which legacy services are running in your default run-level (usually “5”)?  The Systemd report doesn't include run-level info, since it doesn't use runlevels.  (They are supported as a legacy issue.)  To see systemd managed services running in the current run level (the “default.target”), use “grep running ~/service-report.systemctl”.  Are any of the services listed in the chkconfig output, not duplicated in the output of systemctl?  (Remember, some older services have not yet been ported to the systemd and only show when using the older command.  You may or may not have any of those running on your system.)

    2. Use the command “ps -ef” and compare this output to the service-report.systemctl output created in the previous step.  Determine which services that are enabled for the current run-level are actually running one or more processes.  Why are there services which are enabled that don't appear to be running now?  (Hint: a short shell script using a for loop, grep, and sed, can automate this task.  If you don't do that, you need not examine every single service; the idea is to get familiar with the tools, their output, and the concepts.)
    3. Now use a GUI tool to examine and manage services.  Previously with System V init, you could have used any of several tools such as system-config-services.  With Systemd, you have fewer choices.  Both Gnome and KDE have tools for this, but here we will use a Red Hat standard tool, “cockpit”.  First, you need to install it:

      $ su
      Password:
      # dnf install -y cockpit
      ... output omitted ...
      # systemctl enable cockpit.socket
      # systemctl start cockpit.socket
      

      Now that it's installed and enabled, open your web browser to this URL:

      https://localhost:9090/
      

      The user interface is simple to figure out.  After you log in as yourself or (more dangerously) as root, click on the “Services” tab to view “System Services”.  Click on a service to see the full name, status, and recent log entries.  This tool doesn't provide a description of each service when you click on its name, as the older tools did.  Instead, you need to use the systemctl command you used earlier to see a brief description of each service.  From this tool, you can also start/stop and enable/disable the service.

      Using either a GUI or command line tool, stop (but don't disable) each service you don't think you are using.  (Remember that on-demand services are disabled differently than stand-alone services.)  Which services did you stop?  Don't worry about breaking anything; if you turn off some necessary service, you can always turn it back on, or simply reboot.

      (Hint: if you are uncertain about some service, try finding a man page for it.)

    4. Stopping the service now won't prevent the system from restarting it at the next boot.  (That's a good thing when testing and learning, and you turn off some important service!)  For each service you have stopped, you should now disable the service, so that it won't start at the next boot.  Make sure to record each disabled service in your journal; you may need to re-enable some services if you disabled them mistakenly.
    5. What would be the commands to stop the pcscd (smart-card) service from the command line, using the systemctl command?
    6. What would be the exact command you would use to disable the bluetooth service from run-level 3, using the Linux command line tool chkconfig?  (Note, this question is theoretical, as you may not have that service running or it may not be a “SysV” service.)
    7. Generate a new systemctl service report, saving the result in a different file.  Now use the diff command to compare these two reports.  Whenever you make changes to services, you can generate a new report, and compare it to the previous one.  The diff output is useful to include in your system journal.  What changes are shown by diff?
  2. Enable Apache (httpd) web service:
    1. Manually start the web server and verify it started successfully.  How did you do this?  (Hint:  In Fedora, this is a “native” service, not a “SysV” one.  You must determine or guess the service name for the daemon.)
    2. You must check the Apache log files in /etc/httpd/logs/error_log and the system central log file in /var/log/messages, and examine the output of a ps -ef listing.  What Apache related message(s) did you see in the logs?  What Apache related processes are now running?
    3. Fire up a web browser on your system, and point to the URL http://localhost/What do you see from this URL?  (Make sure you have the correct URL, including the trailing slash!)
    4. Using the appropriate command line tool for this service (systemd or System V), enable Apache to start automatically at boot time.  Exactly what command did you use?
  3. Enable telnet network service:
    1. See if you have installed a telnet server.  You can check with “rpm -q telnet-server”.  If not, install it (the quick and easy way: “dnf install telnet-server”).
    2. Re-run cockpitWhat has changed now that there are on-demand services installed?  (Hint: you can use the diff command against the files created in step A.1, to locate newly added services.)
    3. The systemd way to enable/disable and to start/stop an on-demand service such as telnet is to run a command such as “systemctl enable telnet.socket”.  This adds new files to /lib/systemd/system/, not /etcWhat new files were created when you enabled telnet?  (Hint: use the find command to discover newly created files.)
    4. With SysV init, you only need to enable or disable on-demand services.  But with systemd, you need to both enable and start an on-demand service's socket, to make it available.  (Enabling the socket will cause systemd to create the .service file template; starting/stopping the socket causes systemd to enable/disable that service.)  Remember, on-demand services don't have something.service for their names; rather they are named something.socket.  Make the telnet service available.  What are the exact command(s) you used for this?
    5. Run the command ps -ef and examine the output.  What do you see related to telnet?  Compare this answer with your answer from question in part A.2 above.  Does the output agree with what you expected to see?  Explain.
    6. Verify telnet now works.  How did you do this?
  4. Enable anonymous FTP service:

    At one time FTP was a very vital service on the Internet.  Today a lot of files can be found and downloaded using HTTP from a web page.  But FTP hasn't gone away.  Indeed, modern (2019) web browser such as Chrome are considering blocking downloads of any “dangerous content”, such as executables and archive files.  So FTP may make a comeback!

    Plain FTP has low security, as everything is send across the network in plain text (unencrypted) form.  For upload configurations, web sites, and work documents, the secure variants sFTP (secure FTP) or FTPS are preferred.  However anonymous FTP is still useful and used.

    For this part we will enable vsftpd (the very secure FTP daemon) for read-only FTP access for an anonymous user, and normal read-write access for users.  This server can be configured to either run stand-alone or on-demand.  For this project we will enable vsftpd to run as a stand-alone service (which is the default).

    1. If necessary, install the vsftpd server on your system.
    2. Change the FTP user home directory from “/var/ftp” to “/var/ftp/pub”.  Verify that directory exists (and has appropriate permissions).  This user's home directory is (by default) the location of the anonymous FTP site (although you may have to set this in a config file instead).  What changes did you make?  What permissions should the directories /var/ftp and /var/ftp/pub have to allow anonymous FTP?
    3. Add some simple text files to /var/ftp/pub (so you can test your anonymous FTP site); for example:
         echo 'it works' > /var/ftp/pub/foo.txt
         chmod a+r /var/ftp/pub/foo.txt # make file readable by everyone
      
    4. If desired, you can edit /etc/vsftpd/vsftpd.conf to change the default behavior of the FTP server.  For example, you can add:
         ftpd_banner=Welcome to the GCAW.org FTP service!
      
      See the comments in that file, and the man page for vsftpd.conf to see what can be changed.  Make a backup copy of the original first!  What changes did you make?  (Use diff command to show your changes in your turn-in.)
    5. Determine if vsftpd is a systemd native or a SysV service, and start the vsftpd service using the appropriate command.  Exactly what did you do for this step?  Check logs for any errors.  What log messages were generated (and in which log files)?
    6. From your non-root account, and from your home directory, try to connect to the anonymous FTP server:
          [auser@localhost ~]$ ftp localhost
      

      Use username “anonymous” or “ftp” for the anonymous user.  By default, any password you enter will be accepted, including none at all.  This too is a config setting.)

      If this fails to work, what might be blocking access?  What must you do (the exact steps) to allow access to this service, if it is blocked?  Show a diff listing to indicate any changes you made to any files.

    7. Once you get connected, try various FTP commands: help, dir, ls, pwd, get foo.txt, and to exit, byeWhat directory does pwd show as the current directory?  What files can you see?
    8. Exit FTP and try it again, this time logging in using your normal user account with that account's password.  (Don't try to log in as root!)
      Did the login succeed?  If not, one possible cause is that SE Linux is running in enforcing mode, and is not configured to allow this.  If you have difficulty logging into FTP as a normal user (rather than the anonymous user), check if SE Linux is running in enforcing mode with getenforce command.  If so, there are some security settings you need to change to permit access.  Rather than bother with that for now, run (as root) the setenforce 0 command to switch to permissive mode.

      Now try to log in again.  (In the security course you will learn how to configure SE Linux for this, but for our class it is easier to switch it to permissive mode, and that should be safe enough from our classroom.)  To make the change permanent, edit the file /etc/selinux/config and set the mode from:

          SELINUX=enforcing
      to:
          SELINUX=permissive

      Note!  For security reasons, modern systems won't allow you to connect remotely as root, only as a regular user (or sometimes as an anonymous user such as for FTP).

    9. Once you are successfully logged in as a regular user, run some FTP commands again.  What is the result of the commands pwd and ls this time?
    10. To enable local users, to block others, or to set up chroot protection for (some) users, you may need to edit the vsftpd.conf file.  You should start by reading the comments in that file.  If that doesn't help try reading the man page.  Remember that to access services from other hosts have security issues and may involve configuring the firewall, TCP wrappers (hosts.allow/deny), or PAM.  (None of this should be necessary for this project.) What changes, if any, did you make?
  5. Creating and installing a new on-demand service:
    1. Create a shell script named “/usr/local/sbin/hellod” with the following contents:
      #!/bin/sh -
      echo 'Hello, World from the hellod server!'
    2. Make sure the script is readable and executable by everyone.  Run it (as non-root user) to test it.
    3. Steps for a Systemd on-demand service:

      Your new service should listen on port number TCP/9333.  (This is a currently unassigned port, according to the master list of port number assignments by the IANA.)  You need to create two unit files: a socket and a service template.  With systemd, on-demand services are started via a socket unit file.  Create a new socket unit file “/etc/systemd/system/hellod.socket” with the following contents:
      [Unit]
      Description=hellod Socket Per-Connection Server
      
      [Socket]
      ListenStream=9333
      Accept=yes
      
      [Install]
      WantedBy=sockets.target

      When a new request arrives, the socket will accept it and start (spawn) a new service, named for the connection and service name.  Systemd will actually create a new .service unit file for each connection.  The new service unit file is created from a template service unit file with the correct name (ending with “@” to identify the unit file as a template).  Create a new service temple unit file “/etc/systemd/system/hellod@.service” with the following contents:

      [Unit]
      Description=hellod Per-Connection Server
      
      [Service]
      ExecStart=-/usr/local/sbin/hellod
      StandardInput=socket
      
    4. The final step with systemd is to enable the new service and start the socket (make it start listening for incoming requests).  Run the commands (as root, of course):
      systemctl enable hellod.socket  # start the socket at each boot
      systemctl start hellod.socket   # enable the on-demand service
      systemctl status hellod.socket
      

      Systemd error messages can be viewed using journalctl command; consult the man page to see how to use it.  What messages resulted from starting hellod.socket?  If any errors were noted, fix them and repeat.  What errors (if any) were found and what did you do to resolve them?

      If you find errors preventing the socket from starting, after fixing the problem, you should run the command “systemctl daemon-reload”, before trying to start the socket again.

    5. Once there are no errors, your new service should be available.  Test it out by running this command (not as root):
      nc localhost 9333 </dev/null

      If you don't have nc installed, you can install the nc package with dnf.  “nc” is the Linux name of the “netcat” utility.  (You can use telnet instead of nc.)

      What was the result (output) of running this command?

      If this fails to work, what could be blocking access?  Keep in mind that on a modern system, the name “localhost” refers to the IPv6 address of “::1”, not “127.0.0.1”.  You may have better luck using the numeric address instead of the name.)  What did you do (the exact steps) to allow access to this service (if blocked)?  (You can show a diff listing to indicate the changes you made, if any.)  Once fixed, retry the command and show the result.

    6. This is an optional step, to help you learn how to use System V.  You should skip this step, unless you have the time.

      Steps for a System V on-demand service, using xinetd:

      Firstly, stop and disable the systemd service you setup previously.  Then remove any added unit files.  (You might want to just move them into your home directory for now, so you can easily put them back later.)  Verify your hellod service no longer is working.

      Next, make sure you have xinetd installed, enabled, and started.

      Now create a file /etc/xinetd.d/hello so your new “hello” service can be started on demand.  Your new service should listen on port number TCP/9333.  The file should have the following contents:

      # default: off
      # description: Demo service that runs a shell script to say hello, world!
      
      service hello
      {
        disable      = no
        socket_type  = stream
        type         = UNLISTED
        port         = 9333
        wait         = no
        user         = nobody
        server       = /usr/local/sbin/hellod
      }

      Now reload the xinetd service to make it read the new configuration.  Check the log file for any messages from xinetdWhat messages resulted from reloading xinetd?  If any errors were noted, fix them and repeat.  What errors (if any) were found and what did you do to resolve them?

  6. Listing Available Network Services:
    1. List all the network services currently available on your host.  There are a number of tools you can use for this purpose, including “lsof -i” and “netstat -l46”.  However, lsof lists each process listening, so ports such as TCP/80 will be listed multiple times, while netstat may not list on-demand services.  Perhaps the simplest way to see all available network services is to run (as root) the command “nmap -sSU -p 0-65535 localhost”.  (You may need to install nmap using dnf.)  This command may take about a minute to run.

      How many ports are open?  What are the meanings of the options to the nmap command used?  What would the option “-6” mean?

    2. What is the purpose of the “/etc/services” file?  What would happen if you commented out (or removed) the line(s) for some service, such as Telnet?  (You can always try it and see what happens, but make a backup copy of the file first.  Use the command “nc -t localhost 23” to test, or “telnet localhost 23”.  If you don't specify the port to use with telnet, it does use /etc/services to look up the port number, and so it will fail if you removed or changed that entry.)

To be turned in:

The answers to the questions above and the portion of your system journal describing the steps you have taken to enable these services.

You can submit your project as email to .  Please see your syllabus for more information about submitting projects.