(Very) Basic Linux Server Security

This post is meant for Linux newbies.

So you finally decide to set up a Linux machine to serve the glorious content you have created. Your Linux machine may seem invincible behind the safety of your home or corporate router’s firewall, but once it becomes a publicly accessible server, it becomes highly vulnerable to hackers, DoS bots, and spammers. If you are managing a VPS, or if have your own machine or VM exposed to the Internet through port forwarding, it becomes your responsibility to keep the machine secure and updated. A compromised machine is not only bad for your own business, but it can also become a device for hackers to launch other attacks.

Typical server uses include

  • publishing a website/app
  • ftp server
  • ssh server

Common ways to compromise a server are

  • exploiting a 0-day or other unpatched security vulnerability
  • crash it with a Denial of Service attack
  • brute force password cracking

I’ll just dump the basic practices I follow in hopes that it may some day help another uninformed soul. Since I’m no security expert, please take everything with a grain of salt.

Keep Everything Updated

This advice often falls on deaf ears. Outdated software is not only deficient in terms of features, it is also full of security holes that can be exploited by hackers.

On most Linux distros, updating software is as simple as

Debian style:

sudo apt-get update && sudo apt-get upgrade

or Red Hat style:

sudo yum update && sudo yum upgrade

Sometimes packages need you to a ‘dist-upgrade’ instead of ‘upgrade’, which should be fine most of the time.

Software updates always carry the risk of breaking things. I have run into servers running five year old versions of Apache, PHP, Perl, etc just to avoid the discomfort of dealing with broken dependencies. Most times these machines are located on private isolated networks so the threat is somewhat mitigated, but this is an absolute no-no on a publicly accessible server.

If you are extremely worried about updates breaking your server, you can choose to do the updates manually on a regular basis so that you can carefully examine what is being updated and what effects each update may have. But this is generally going overboard, as most software developers are careful about what they release and would never want to break millions of machines on the Internet. For everyone else, I recommend some automated way of patching your machines. The simplest way I know of is to create a cron job to do this regularly:

Edit the crontab file for the root user:

sudo crontab -e

(Choose your desired editor if prompted and) Add the following statement to update your system daily at midnight:

0 0 * * * apt-get update && apt-get -y dist-upgrade

(assumes a Debian type system).

Even with the most up-to-date software, expert hackers can still find holes that are not known publicly yet and get inside a system. But if those kind of guys are after you, then you shouldn’t be managing your server security yourself.


Detecting Power Outage From WiFi Networks and Stats

Many people use a Raspberry Pi as a server at home. Even though most uses are recreational and non-critical, power outages can occasionally lead to loss of data and corruption of the SD card. To avoid that, cautious users often connect their Pis to a backup uninterruptible power source (UPS). There are many dedicated UPS solutions for the Raspberry Pi. Some are sold as ready made products, others are DIY projects. If there is other IT equipment connected to a UPS, the Rasperry Pi can simply be plugged into one of those outlets instead of having its own separate UPS.

UPS systems have limited backup, and depending on the load and battery capacity, they can last anywhere from a few minutes to several hours. Many UPSs have a PC interface that alerts the system that a power outage is in effect and of the state of the batteries so that the system can gracefully shut itself down before the UPS dies. Dedicated Raspberry Pi UPSs also often come with circuitry that can send a message to the Pi via a GPIO pin. There are also cases where there is no such facility, for example, when sharing a UPS meant for other devices. People have come up with many DIY circuits that detect when there is no power in a non-UPS backed up outlet, but these involve spending money on components and building circuits.

Here, we will try to come up with a software only solution to detect the power outage.

During a power outage any WiFi routers not connected to a UPS will go down. Keeping an eye on how many traditionally stable WiFi networks are are down at the same time can be used to determine if there is a power outage in the area. This assumes the Raspberry Pi has WiFi capability (like an RPi3 or a WiFi USB dongle). It does not matter whether the Pi uses WiFi or Ethernet for Internet connectivity, WiFi is merely used to scan for available SSIDs.

Here is a simple script that counts the number of active SSIDs using the iwlist command, and if the number is less than a set threshold, it is flagged as an outage.




NUM_SSID=$(iwlist wlan0 scan | grep -c ESSID)
echo $NUM_SSID

if [ $NUM_SSID -gt $MIN_NUM_SSID ];
  echo "Power is good."
  echo "Power outage suspected."

In this example, the Raspberry Pi sees about 10 SSIDs on average with good strength at any given time. During a power cut, I expect at least half of them to be out of service as almost no one around me plugs their WiFi routers to a UPS but for a few mobile hotspots that spring up, so a threshold of 5 seems reasonable.

This script can be run as a root cron job, executing at say five minute intervals. When the script detects an outage, it can do several things, I am listing only two here:

  • Shut down the Raspberry Pi
    shutdown -h now
    Although this is the safest method, this is not recommended as the Raspberry Pi will not turn itself on unless it is actually power cycled – manually or by the UPS draining out. You could be stuck with a shut off Raspberry Pi.
  • Shut down all important services and sync the filesystem
    service vsftpd stop
    service nginx stop
    service mysql stop
    service php5-fpm stop
    To get a list of active services, run sudo service --status-all. The sync command flushes all buffers to the SD card. This is not a fool proof measure against data corruption, but it just might work. When all the networks come back up, you can restart all the services or simply reboot the Pi with the reboot command.

Better detection of the outage?

The script above only counts the number of SSIDs at the instant the cron job executes it. For a more reliable count, the script should average the number of SSIDs visible over a few minutes to prevent false triggering.

The script is also flawed because it assumes the number of active SSIDs in the area won’t change over time. New networks could come up and old ones could be taken down permanently. The script can keep track of online SSIDs over several days and use the moving average as the total. The threshold used to detect a power outage could be a fraction of this value.

There are many many ways to implement this more gracefully and I will leave it to the curious minds out there to do it in their own way.