10 Tips for Hardening your Linux Servers

[Music].

Hello again, everyone,

and welcome back to LearnLinuxTV. Today,

I am launching a brand-new series on my channel:

Enterprise Linux Security. And in this series, I'm going to talk about,

well, Enterprise Linux Security. This is a

series that I've been wanting to launch for quite a while, and

today's the day. This is episode number one. And in this video,

I'm going to go over 10 tips for hardening your Linux servers.

Now, some of you out there that are more

seasoned when it comes to security than others

might feel that some of the tips

that I'm giving you in this video are a little,

well, entry-level. And that's not completely untrue.

This is episode number one, and we do have to start from somewhere.

But I really do feel that the tips that

I'm going to give you in this video are

the most important things to focus your attention

on when it comes to hardening your Linux

servers. Now, before we get into it, I want

to take a moment to mention the sponsor for this video: KernelCare.

Keeping servers safe, compliant, and ensuring constant uptime is a full-time job--

one that can't be left to chance and one that must be fully automated

and fully supported. To do that, you need a live patching tool that

integrates with automation tools and vulnerability scanners,

supported with the latest patches--and

one that lets you decide which patches

are rolled out across your organization and runs within your firewall.

And KernelCare Enterprise does this. It provides you with more integration, support, and control.

It works in your local infrastructure via ePortal,

a dedicated patch server that runs internally but outside your firewall.

It acts as a bridge between internal patch servers and the main KernelCare patch server.

This approach is ideal for staging and

production environments that need strict isolation from external networks

or require more stringent controls over the patches that are to be applied.

KernelCare Enterprise is available for all major Linux distributions

and includes priority support 24/7 via live chat,

email, or ticket system. Check out KernelCare Enterprise via the URL that's on the screen right now

or give the link that's in the description a click.

And thank you so much to KernelCare for sponsoring this video

as well as many other videos on this channel. I really appreciate it.

Now, let's get into my list of 10 things that you can do

to harden your Linux servers.

[Music].

Now, when it comes to my first tip, this is not actually a system tweak or a

system change or anything like that. It's all about your mindset. Now, for all

I know, you could be a system administrator, you could be a security

professional, or you might even be a CTO. Either way, it's very important to

understand what an appropriate mindset is

when it comes to the security of your servers. So, what do I mean by that?

The thing is, it's important to understand what is feasible

and infeasible when it comes to the security of your servers.

Namely, is it possible to have a completely unhackable server

that nobody can break into--that is completely bulletproof?

Well, yeah. Absolutely. You could definitely set up a server that is unhackable.

Basically, you just put that server under your desk, you don't power it on,

and you certainly don't connect a network cable to it. And I guarantee you

nobody's going to hack it.

But we need to be realistic.

A lot of companies out there--maybe even yours--

they make money by selling things to the public or

providing a service to the public, which requires a public-facing server.

And the thing is, there are all kinds of

vulnerabilities out there that are being leveraged every day,

and new ones are discovered every single day.

So, you could be the victim of a vulnerability that hasn't even been publicly disclosed yet.

If you follow every tip in this video, you should be relatively fine, but

you want to adjust your mindset. You don't want to have the mentality that

you are going to be creating perfect

servers that cannot be hacked, or that you've just hired this awesome security person

and now all your worries are just, you know, not worries anymore.

And you can't have that mindset. You have to have the mindset that anything is

possible, and you need to be ready for it at all times.

Now, I'm not trying to scare you. Well, actually, am I not trying to scare you?

Well, I kind of am. But the reality of the situation is, if you

follow everything in this video like I mentioned, you should be good--

but you should always be prepared for what could happen.

[Music].

For number two on my list, I really do

think that this is going to be one of those things that's going to be

painfully obvious to the majority of you guys that are watching this video.

But I don't think I can create a security series--especially not an introduction

to a security series--and not mention the importance of patching.

Now, the thing is, if patching is so obvious, then why do so many companies out there do a

terrible job of keeping their servers up to date? I mean,

it's almost appalling to me at this point. I've had so many companies out

there that I have worked with personally--when

I tell them, "You need to patch your servers. There's something critical that

is basically going around right now"--and the response I'll get

is, "Yeah, maybe next month. I don't think we can do that right now.

We have this really important release we've got to get out the door, but

I think things should slow down in a month or so, and maybe we'll have you patch our servers then."

And then a week later: "Oh my God. We got hacked. What do we do? How did this happen?"

It's obvious how this happened. You didn't take security patching seriously,

and now you've been owned by one of the vulnerabilities that one of those patches

would have protected you from. And I get it--rebooting your servers,

or patching your servers (which often does require a reboot),

is not easy to do. It's annoying. It's tedious.

And it's even harder to design your infrastructure in a way that you don't

need to reboot after patching. It causes service disruption. You have to

test the patches before you roll them out.

It's a big deal for a lot of people. And quite often,

some of these patches are created for very important reasons. I mean,

security researchers and people that write these security patches--they don’t do it

because they have nothing better to do. They do it because they’re actually patching

real vulnerabilities. So, you need to keep your servers up to date.

And if you don’t currently have a way to do

that, then I highly recommend you find a

way to do that, or at least work that into your workflow in some way.

Now, KernelCare--the sponsor of this video--

they actually offers a service called KernelCare.

And what that service does is it enables you, the administrator, to

live patch your servers. And if you can live patch your servers,

then that’s even easier because you won’t need a reboot.

A live patch is the process of injecting a patch right into the running kernel,

which means you could benefit from that security fix--if it is a kernel-related security fix--

right then and there, no reboot required.

But even if you don’t go with a service like KernelCare, at least

enable unattended upgrades. Various Linux distributions have

a similar solution like unattended upgrades.

It’s different per distribution, but you get the idea. Automatic updates are your friend because they’ll

keep your servers up to date--and that’s a very important thing.

[Music].

For number three, it’s probably even more obvious than number two,

and that is the importance of secure

passwords. And by secure, I mean randomly generated passwords.

The thing is, you would be surprised by how many hacks out there

were done solely because there were weak passwords involved.

So, definitely have randomly generated

secure passwords for all of your very important servers and services.

It’s critical. And that also implies good password management hygiene.

Something like Bitwarden or LastPass--

something like that--is very important to

keep your passwords, because if you forget your passwords, then

that’s even worse, right? Because you

can’t even get into your own servers.

But having really good password hygiene is extremely

important. Again, I’m not going to spend a

lot of time on this because I think it speaks for itself.

But if you, as the administrator for your company, notice some very easy or insecure passwords,

you really do need to change them on the spot.

Because if you don't, you could have a very long day or week ahead of you.

[Music].

Now, number four on my list is all about

not making things publicly available unless you absolutely have to.

Now, I get it. A lot of companies out there have a public-facing website

that's very important because you do want your customers to reach your website.

In that case, that server does truly need to be open to the public internet.

There's just no way around that. However, if a server or service does not

need to be public-facing, make sure that it's not.

Implement firewall rules that block its ability to be reached from the outside.

Now, don't just assume that a service on your company's network is not reachable from the outside

after you apply that firewall rule--actually check to make sure that it's not.

For example, you can use your phone--just make sure you're not on the company Wi-Fi--

and try to access that service. Make sure that you can't do that.

That's the only way to be sure that it's not publicly reachable from the outside.

If you are allowed to do so and you have permission to do so,

you could try a port scan from the outside. That'll really let you know

if a service is accessible from the outside. But either way, you do want to make sure of that.

Now, one particularly sore point for me is when people make database servers

accessible from the outside. And there is almost never

an excuse to make a database server accessible from the public internet,

unless your company actually offers managed database services.

Then, in that case, yeah, you do need to make that database server publicly available.

And I'm sure the majority of you guys are not in the business of providing

managed database services. So definitely make sure that your database servers are internal-only,

because they're probably the back end to your web server or something like that.

Just make sure they're not publicly available. It's very important.

Having a database server publicly available is one of the scariest things,

because there could be personally identifiable information on that server,

and your company could end up on the news for all the wrong reasons.

Long story made short: just make sure that your database servers,

as well as any other servers that don't need to be publicly available, are not publicly available.

[Music].

Now, number five on my list is closing down SSH.

OpenSSH, or simply SSH for short, is one of the greatest things in the

Linux community--at least one of the most convenient things in the Linux community--

because it allows you, the administrator, to manage your servers or your company's servers

from the comfort of your home office or your company's office.

Basically, you don't even have to get out of your chair to manage your servers.

And think about it--we used to have to walk into the data center to do

basically most of the things that we use SSH for nowadays.

SSH is awesome. But it's also a very, very, very large target.

Because if a remote attacker gets access to SSH,

especially as root, they will wreak havoc on your servers. You definitely want to lock down SSH.

And there are multiple things that you can do

in order to do that, and I have a dedicated video that talks about

how to lock down SSH. You should check out that video

because it'll tell you everything that you need to know.

But in summary, some of the things that you

want to do to lock down SSH include, but aren't limited to, ensuring

that root access is disabled. You don't want to allow

root authentication to SSH. In addition to that,

you should also disable password authentication as well

and only allow key-based authentication to your servers via SSH.

Going a step further, you can lock down SSH to

approved or whitelisted IP addresses to ensure that

IP addresses on the public internet cannot access

SSH on any of your servers. If you have a VPN endpoint,

then you can lock down SSH to be

accessible only from the IP address

of your VPN endpoint, and that would be another step in the right direction.

The more you lock down SSH, the better--

because it's usually the first target that hackers try to get access to when they want access to your servers.

[Music].

Now, item number six on my list is all about having multiple

layers of security. And what that means is that you should never rely on

just one thing. So like I mentioned, I recommended that you lock down

SSH, which is great. But if that's all you do, then maybe someone will get

access to your servers by another method. So the more layers of

security you have, the better. For example, you could consider Fail2Ban on your servers as another layer of protection.

Maybe you already have a firewall on that server as well

and you are locking down SSH. The more layers of security, the more hoops

you force hackers to try to get through in

order to get access to your servers, the

better--because you are making it that

much harder on them to access your server.

And after a while, maybe that person will

give up and then move on to another server, which is

exactly what you want. And only very targeted attacks would continue past that point.

But having multiple layers of security--for example, Fail2Ban

or a similar service that looks for intrusions in the logs

and then blocks IP addresses that basically try to bypass

the rules that you've set--that's a good

step to have. And other tools as well.

The more you have, the better. So try to

have multiple layers of security on your servers

and make it that much harder for outside intruders to break in.

[Music].

Now, number seven can be argued that it's not

really a security-specific thing, but I think it's important to include on

this list because it is very important--and that is the concept of backups.

And not just any backups--tested backups. Any backups

that you have not tested, and any backups that are not in at least

three different places, are not truly backups.

So you want to have your backups in, like I mentioned, three different places.

One of which should definitely be off-site. And you want to do

test restores on those backups to make sure that the backups are good.

Because trust me, if your servers go down and you need to restore from a backup,

you don't want to explain to your boss that you can't restore the servers

because the backups aren't working. And I have seen this happen.

It's horrifying, and it's not a good experience for

anyone involved. Definitely have backups

and have multiple layers of backups in multiple different locations--

but especially test those backups. And that ensures that if you are

actually facing a security incident and your servers are completely turned

inside out, you have backups. So you're probably

going to be good. Yes, it's going to be very inconvenient to have a security

incident, but you have backups. You can at least get up and running

quickly, and your company's data is not in jeopardy and not

lost forever--which is very important.

Especially if your company is housing

very important blueprints for products and things like that.

You definitely want to make sure that

those items are backed up, and they're backed up securely.

[Music].

Now, for number eight, it's very important to keep an eye on all of your servers

and the overall health of your servers.

And monitoring tools will help you do just that.

Nagios and Zabbix are two that come to mind immediately.

If there's any kind of issue and you have the appropriate checks configured,

then you will be notified that there's an issue. And if you know about the

problem before your customers know about it,

then you actually appear as a very competent IT professional--because

you are ahead of the game. You are aware of everything that's going on.

And it's not just, you know, a matter of

having these monitoring tools enabled--although that goes a long way.

You want to make sure that they're checking the right things. You don't want

to, for example, be checking for uptime only and then have the server

fall over because the disk is full. You should be checking disk space as well.

And obviously, website availability goes without saying if it's a web server.

And you could even have user checks on your monitoring tools.

If there's more than one user that is on that server, it should send you an alert. And you could even configure

it that if so much as one user logs into your

server, it sends you an alert. So if you're working on the server, for example,

and you're doing some administration work

and you get that alert that someone is logged into your server--oh yeah, that's fine.

That's me, actually. I'm on my server right now,

and I'm installing some updates. But if you get that alert and

there's no maintenance planned, that's a red flag. Someone got in.

So, there are all kinds of different security checks that you can configure.

It's very important to have monitoring tools in place.

[Music].

Now, for number nine. And I have to say, of all the things

on this list, number nine is definitely the hardest. It's the most expensive.

If you are working for a company and you have some very

important services that are running--and maybe you even store personally

identifiable information--you really should have a third-party security audit.

Now, it's one thing that, you know, you, the administrator,

you're checking everything all the time, and that's awesome.

But you're only one person. You need someone on the outside to check your

servers and make sure that there's nothing that you've missed.

But the problem with this, though, is that third-party security audits are

extremely expensive. So this is only for

those of you out there that work for enterprises that can afford such a thing.

But even if you can't afford such a thing right now,

you definitely should keep this on the list--because if your company grows

and you actually have the ability to hire someone on the outside to

basically audit your servers, you definitely should do that.

Because they could find something that you've

missed, and they might even save you from a major incident.

[Music].

Now, for number 10, the last item on my list. It's all about business continuity.

How are you, as the administrator, going to ensure that your company is back

up and running quickly after an incident?

And how long do you think it'll take you to get everything back up and running?

If your answer to that question is, "Well, a week, because I have to rebuild

everything. I have to install all the operating systems. I have to patch everything.

I have to reinstall all the applications"--if that's the answer, you're doing it wrong.

You should have some sort of automation, images,

backups, or something that is going to get you back up and running as

quickly as possible. The quicker you can

get everything up and running, the better. And if you have an

auto-healing environment--which means if a server falls over, a new server,

like a virtual server, is provisioned automatically in its place--

and that's especially true with containers, for example--you're doing it right.

You're doing a great job. Because the answer to that question is,

"Well, the server is never down because it

automatically brings one back up," and that's really cool. But your answer

to this question really determines how

good of a business continuity plan you actually have.

And if you don't have a plan, you really

should draft one. If all of your servers fell over tomorrow, what would be the

process for getting everything built back

up where it was before you had that

incident? And that's going to determine

what goes into your business continuity plan.

Now, this is something that we could talk about in a future video,

but I wanted to plant that seed right

now--because a business continuity plan is very important to have.

So, there you go. Those are my 10 tips for hardening the

security of your Linux servers. I hope it was helpful.

Now, I know that a lot of those tips were somewhat entry-level,

but again, this is the first episode of this series,

and I wanted to give you guys the overall list of

important things to consider. And then, in

future videos, we will take a look at more of these concepts in greater detail.

So, what are some concepts that you think

I should cover in this series? What's important to you?

Let me know in the comments down below.

I look forward to hearing what you have to say,

and I will go ahead and create episode two

in this series as soon as I possibly can.

So definitely subscribe to my channel if

you haven't already done so, and I'll see you again very soon.

Thanks for watching.

[Music].