Security isn’t merely a technical problem -- it's a people problem. There’s only so much technology you can throw at a network before dumb human mistakes trip you up.
But guess what? Those mistakes are often committed by the very people who should know better: system administrators and other IT staff.
Intermedia's 2015 Insider Risk Report found that IT professionals were the most likely group to engage in “dangerous” security practices, such as sharing passwords/logins, reusing personal passwords for business applications, or giving personal account credentials to others.
Such lapses tend to be far riskier than those of ordinary users, due to the godlike powers sys admins have over the network. IT pros can be as susceptible as users to phishing, malware, and other attacks -- and stolen, privileged sys admin credentials almost always result in far more serious security breaches.
Here are 10 common security blunders made by sys admins and other IT staff:
Blunder No. 1: Using sudo for everything
When you log in as root, you have full control over the box. This can be extremely dangerous because if your credentials get stolen, an attacker can do whatever he or she wants.
In Windows parlance, there’s no need to log in as Administrator if there are no administrator-level tasks to perform. Instead of directly logging into systems as root, log in through your personal account and use sudo for specific commands as needed.
It's easy to backslide if you're not careful. A script fails because one of the commands needed sudo -- and now everything must be restarted. If you fail to keep track of which commands require elevated privileges and which don’t, you may go back to running everything as sudo.
Blunder 2: Running scripts of unknown origin
Installing third-party Linux applications is another area where sudo can be abused. All you have to do is copy and paste the command -- which is already set up to use sudo -- directly into the terminal to kick off the install script. Every single command in that script will be executed with elevated privileges.
Here's an example, copied right off the Web (with the URL hidden):
sudo -v && wget -nv -O- https://xxx/xxx/linux-installer.py | sudo python -c "import sys; main=lambda:sys.stderr.write('Download failed\n'); exec(sys.stdin.read()); main()"
This gives sudo privileges to an item hosted elsewhere on the Web, as well as running Python locally. Not recommended! Windows admins face similar potential catastrophes running downloaded PowerShell scripts.
Even if you trust the source, never assume a script downloaded from the Internet is safe. Always vet the contents of the script first and verify that executing the commands will not result in nefarious actions.
Blunder No. 3: Running privileged services as root
Applications should never run as root. Create unique service accounts with very specific privileges for each application and service running on the machine.
Service accounts typically lack home directories and are restricted in what they can do on the file system if someone tries to log in using the account. If an attacker compromises a service account, he or she still has to get a local exploit working to get more privileges to execute code.
Each application should use a customized account to access the database instead of root or the administrator’s personal account. Web applications should be owned by the appropriate group and user. When assigning domain privileges to Windows applications, don’t give the application administrator-level access.
Major Linux distributions handle service accounts by default, but if the administrator manually configures third-party packages, it’s easy to make a mistake. Remember to switch permissions after all the installation and configuration is complete to make sure root or the administrator’s personal account is no longer the owner of the application.
Blunder 4: Reusing passwords
Go ahead, roll your eyes. We’ve all heard about the evils of reusing passwords across sites, systems, and applications. But the fact remains that it’s a big problem, and sys admins are not immune.
Recently, Mozilla said an unknown attacker broke into a privileged user’s account for its Bugzilla bug tracking database and stole information about 53 critical vulnerabilities. It turned out the “privileged user” had reused the Bugzilla password on another website, and the password had been exposed in that site’s breach.
Many times, servers are set up with weak administrator passwords or with the same password as other machines on the network. Brute-force attacks using common passwords and dictionary words work because enough people still make this elementary mistake. When multiple machines have the same password, the problem is compounded.
Instead of setting up the same root password on all machines, sys admins should opt to use a key file. Each server should have a public key file and the sys admin’s workstation would have the private key associated with the public key. This way, the sys admin can access all the machines that have been deployed on the network, but an attacker moving laterally through the network will not be able to log in without a valid key. And there is no password to intercept.
Blunder 5: Sharing admin accounts
Administrator accounts -- such as access to the database and administrator portals -- are often shared around the network. Instead of setting up the environment so that administrators request elevated privileges when needed, these admin accounts are shared willy-nilly. That's asking for trouble.
Ideally, there should be separate accounts: one for root and one for each administrator. The administrator accounts shouldn’t start off with the highest levels of access -- the administrator can ask for special access rights when working on specialized tasks. The Intermedia report found that 32 percent of IT professionals have given out their login and password credentials to other employees.
It’s bad enough not knowing exactly who is using the administrator accounts, but even worse, the passwords are rarely reset when an administrator leaves the company. Because the passwords aren’t regularly cycled, ex-colleagues can come right back in and cause damage with impunity. The Intermedia survey found that one in five IT professionals said they would access company information after they leave their current job. Password change policies aren’t for end-users only. Periodically change passwords, particularly administrator and service accounts. And whenever an administrator leaves, reset the passwords.
Blunder 6: Leaving troubleshooting tasks in place
When troubleshooting, you perform various tricks and experiments to find and fix the problem. As you make these attempts, you tend to bypass the usual processes. The problem comes when you fix the problem and move on to the next fire. Admins in a hurry may forget and leave things in disarray -- and open to potential abuse.
You may have opened up ports in the firewall, for example, as you tried to figure out why an application wasn’t responding. Once the fix is in place, you need to go back and close those ports before they can be used by attackers. By the same token, if you turned off SELinux because it was interfering with troubleshooting, remember to turn it back on again after you are done.
When troubleshooting, keep track of what you do as you go, so that afterward you can restore configurations to their original settings -- except for the changes you really needed to make.
Blunder 7: Failing to keep track of log files
Log files are handy, especially when troubleshooting, because they let you see what's happening at the most granular possible level. When you don’t need those files anymore, turn off the process generating them. The last thing you want to do is leave debugging on and generate log files containing information that may be useful to attackers.
As a best practice, always keep track of what logs are created and know what kind of information is in them.
Blunder 8: Storing passwords in plain-text files
When there are so many passwords to track, it’s tempting to write them down in a text file. That’s a gift for attackers snooping around as they get access to various systems. It sounds obvious, but everyone knows of at least one instance where someone saved all important passwords in a text file.
If the passwords must be saved in plain text in a file -- such as database credentials for an application -- set up file permissions to restrict who can view the contents of the file. Also, make sure the database account is a service account stripped to the bare privileges.
Blunder 9: Leaving unused accounts lying around
Old, unused accounts are liabilities. Perhaps software was installed for evaluation, then removed -- and the accounts that were added as part of the installation are still on the system. Don’t leave them there. Attackers can exploit forgotten accounts like these, especially if they retain their default passwords.
For accounts that need to remain on the system but will not being used going forward, disable the account by editing the password file and replacing the account password with a string of characters. Obviously, when employees leave your organization, a process should be in place to deprovision their accounts immediately.
Blunder 10: Being lax about patches
The golden rule: Install security updates as soon as they are available (backing up the affected systems first, of course). Too many servers are compromised not because of a zero-day exploit, but because a year-old patch was never installed.
Even if it's a critical server, a little downtime as part of a scheduled maintenance window is far better than losing hours and days because attackers successfully compromised the box. Promptly test patches as they are released and create a schedule for rolling out updates.
Unfortunately, you may be thwarted in your efforts to patch right away -- usually because the patch will break a legacy app. In that case, don’t simply shrug and say “too bad.” Highlight the problem to the appropriate stakeholders. Escalate the issue. Maybe there are ways to quarantine the servers to minimize risk or to adopt new technologies and reduce dependencies on legacy products.
In real life, patching can be a political quagmire. If a manager who outranks you prevents a system from being patched, make sure everyone understands the risks of failing to do so.
Don’t skimp on security tech
As a general rule, security technology helps keep known bad actors out and can help surface problems when things go wonky. There may be good reasons for not running antivirus or firewall on a particular workstation or server, for example, but those situations are rare.
Consider that several types of DDoS malware are currently making the rounds, infecting Linux Web servers because they don’t have tools to keep the badness out. Security tech should be deployed on every endpoint to keep all users -- senior management, workers in the trenches, sys admins, and other individuals with special privileges -- safe from attack.
Keep machines as clean as possible. Remove applications you are not using so that you don’t have forgotten accounts or tools on the machine. The goal is to run as clean a system as possible to minimize the attack surface. It takes only one mistake, one moment of inattentiveness, for it all to go south.
Security tools help you see what’s going on in the network. Use Nmap to scan for open ports that may have been opened during a troubleshooting session. Check which machines are missing which patches and come up with a plan.
The tools are there to tell you what’s wrong and give you the opportunity to fix them before the attackers swoop in. But all the security tech in the world won’t do any good if the sys admins don’t lead by example -- and follow the rules they set for everyone else.
Join the CIO Australia group on LinkedIn. The group is open to CIOs, IT Directors, COOs, CTOs and senior IT managers.