CompTIA Linux+: Logical Volumes & Filesystem Hierarchy #5

You're watching ITPRO.TV.

Hello, and thank you for watching ITPRO.TV,

helping you level up with IT learning

everywhere you go. I'm your host, Zach

Memos, for this episode of CompTIA Linux Plus.

Logical Volume Manager is the name of

this episode, but you knew that, didn't

you? And the person we know and we love

and we trust, Don Pezet, is here to help us out.

Don, good to see you. Hey. Glad to

be back. And, you know, tackling a pretty

cool technology this episode, we're going

to be looking at the LVM, or the Logical

Volume Manager. It's a really neat

utility that allows us to have a lot of

flexibility with our file systems.

In previous episodes, I talked about how

we don't want to change file systems

after we create them. LVM actually makes

that a lot easier. So in this episode,

we're going to see,

you know, what the LVM is, obviously, but

how to install it, how to get it

configured, how to arrange our disks

using the LVM, and then provide us that

flexibility down the road. So that's all

coming up right here in this episode.

So what do we need to know to get our

system ready for the LVM? Alright. So

the odds are that your distro supports

the LVM. It might not be installed, but

you should have support for it. Most of

them do have it installed by default.

Unless you do, like, a minimal install, it might not.

We can check that really easily, so that's

going to be kind of step one. The other

thing is we're gonna need some hard

drives. Right? The LVM is a Logical

Volume Manager. It's requiring, to

create logical volumes or kind of

virtual disks, and those have to lay on

top of actual physical disks.

So we're going to need some in order to do this.

Now, you can use LVM with your, you know,

with your root directory and your home

and so on. So you can have a lot of

things mounted that way. In fact, when you

install Linux, you probably had the

choice to pick to use the LVM. If you

watched our installation episodes, I didn't choose that.

So my disks are currently set up without using the LVM.

If I want to create it from scratch, I'll

just have to know a handful of

commands. So let me run through a couple

of the basics here. The thing, I'm

going to make sure that LVM is installed.

There's a few ways to do it, but the

easiest way is to use your package

manager. So I'm going to do a

yum list lvm*. The main package

you're looking for is called lvm2

because it's version 2 of the Logical

Volume Manager. And so when I look at

mine, I see I've got installed packages:

lvm2 and lvm2-libs. The libraries--

those are already installed, so I'm good

to go. If they weren't installed, I would

just do a sudo yum install lvm2.

If you're on an Ubuntu machine, I believe

the package there is actually just

called lvm. I don't think it has the

number, but it is version 2 that it

installs by default. So I've got it. It's right there.

And so now I know that I've got the

tools. The next part are the disks.

I'm going to need some disks, and I'm going

to be attaching them to the LVM system.

So when I do that, it actually expects

you to already have a partition on the

disk. You don't have to format it, but it at

least needs to have a partition. And,

normally, we fret about the size of the

partition and the disk type and all that

stuff. LVM really doesn't care about that.

It just needs to have a partition in

place. So if you've already got one,

you're in business. I'm going to do an lsblk,

and I've got three disks: sdb, sdc, and sdd. And each of them--

they're one-terabyte disks, and

they've all got a single partition on

them that is one terabyte in size. That's

perfect for what I need here in this lab.

If your disks don't have a partition, you

just need to create one, and we saw that

in the last episode--or actually two

episodes ago. You can use the fdisk utility.

You know, sometimes we worry about using

GPT versus MBR. Here, none of that's

really going to matter because the

logical volumes are what we're going to

manipulate. So pick whatever partition type you like,

throw it on there. You don't need to

format it. We don't need to drop a file

system on it. Just have the partition

present, and then you'll be ready to

launch it with LVM and start to get your

volumes set up. So now that we're

prepared, how do we actually create

logical volumes, Don? Alright. So now we

can go in and we start to create this, but

it is actually a little bit--

I'm going to call it complex. It's not

that complex, though, but there's actually

three steps we have to get through.

Let me bring up a diagram here to show you

so we can kind of have a better understanding of this.

Inside of the LVM, there's three

different things you work with: there's

physical volumes, there's volume groups,

and there's logical volumes. Alright?

We have to create all three.

We're going to start with the physical volumes.

Now, notice it says physical volumes, not

physical disks. Right? I have

three physical disks, and they've got a

partition on them, but they're not

considered volumes yet. We're going to

bring those physical volumes--those disks--

into LVM. So we've got to create physical

volumes, and these will match up with our

physical disks. I've got three disks,

so I'm going to create three physical volumes. Right?

Then we create a volume group.

A volume group is where we gather the physical disks

together to create a combined pool of storage.

Alright? In the diagram here,

there's three two-terabyte drives, so

they get gathered together into six terabytes of storage.

Once that's done, then we can start

slicing and pulling from that storage

however we want. I can say, alright,

I need a three-terabyte disk, and so we can

slice that out of the volume group by

creating a logical volume. Now,

three terabytes is bigger than any one of

these physical disks. LVM will manage

spreading that data across the disks to

be able to allow for that space. And

what's really neat is down the road, if I

decide, man, I'm running out of space,

I need to add another terabyte to that

disk--we can go in and expand that logical volume.

We can add another terabyte onto the side of it--just slice

it right out of the volume group. Or what

they've done here in the example is

sliced out a second three-terabyte disk

and pulled it right out. So LVM lets us

do that, but it requires these three components. We're going to have to create each one.

Fortunately, we've got tools that do all

that. It's all when you install that lvm2

package--it installs a whole collection of tools.

To help you with this,

if you memorize these three names, which

you definitely wanna do for the exam,

the tools almost always start with

letters that match up with what you're

doing. So the physical volume tools

almost all start with pv, and then the

logical volumes start with lv, and the volume

groups start with vg. So when you take a

look at your system--let me just jump over here.

If I take a look inside of /usr/sbin,

and I look at--I want to work with

physical volumes, so I'll just look for pv*.

Alright? I can see right here, I've

got pvcreate, pvchange, pvremove, pvresize.

These are all my physical volume

utilities that let me mess around with

them. And then if I want to look at

volume groups, I can look for vg*, and

I'll find all of them--almost the same as

what we saw for physical volumes, except

we have some new options in here for,

like, vgimport, vgreduce for shrinking and growing--

you know, all those different ones.

And then lastly, we had logical volumes--that was lv*,

and I'll see each of them tucked away

inside of there, like the

lvdisplay and so on. Some of them are

aliases or shortcuts. A lot of them are

interchangeable. Like, lvdisplay will

show me all of my logical volumes.

lvs actually stands for logical volume show.

It will show me my logical volume.

So, some of these kinds of overlap with each

other, but it’s pretty easy to spot the

utilities, and we just need to go through

creating them. You create from the

ground up. We’re going to create our

physical volumes, then we’re going to

create our logical volume--sorry--

we create our physical volumes, then

we create our volume groups, then we create

our logical volumes on top.

So that’s the basic process that we’re going to go

through. Let me just run through

that. I’m gonna start a little

more simple than in this diagram:

I’m just gonna do two disks. I’ve got two

one-terabyte disks. I’m gonna combine them

into a two-terabyte volume group that

I can then slice out into a logical volume. Alright?

So what I can do is I’m gonna start with pvcreate, alright?

And actually, let me sudo this so that

I’m an administrator.

So I'm gonna sudo the pv create command, and I need to

tell it the disks that I'm bringing in.

Now remember I said we need a partition

on each one. We saw I already have some

partitions. So I'm gonna bring in /dev/sdb1 and /dev/sdc1. Alright.

Those are the two disks. I have a 1/sdd1.

I'm gonna hold on to it because I

wanna show you guys adding more

storage after the fact. So I'm just going

to grab these two. So when I do that, it

sees the disk. And remember how I said

you don't have to format it? Here's why.

It sees I've already got a format on

there, and it's gonna wipe it.

It's gonna erase whatever's on there.

It needs the partition. It doesn't need the

file system. So I'll say yes to let it

wipe it. It's gonna do the same thing

on the other disk. There we go. So they're

wiped, and now they've been brought into

the LVM. And I should be able to see that by doing pv display,

which is my physical volume display. Oops.

Non-root user. So let me just rerun that. Whoops.

And there we go. And so now it's gonna

show me those two physical volumes, sdb1

and sdc1. They're one terabyte. They each

have a UUID that's been assigned, and

I can see all of that right there. So

they've been brought in the way that I want them.

Now I can create a volume group to

combine these two together, and I'm just gonna use,

you guessed it, vg create. Now this

command is a little bit different.

I'm gonna create a volume group, and that

volume group needs to have a logical

name that I can map to. When I have a

hard drive, the hard drive is referenced

as /dev/something, right, a device ID.

Well, our volume groups need to have device IDs also, and there's a

service that's running in the background

called the Logical Volume Manager

Mapper, the mapper service. And the mapper

service is what's gonna create /dev

entries for our logical volume, so

it's really important. So when I start to

create a volume group, I have to give it

a name for what I want the volume group

to be, and that's gonna become part of

the device name for my logical volumes.

They'll be /dev/mapper/ and then the

volume group name followed by / and the

logical volume name. So it gets kind of

long. So when I create this one, I need to give it a name.

I would normally name this based off of

what I intended the pool storage to

be used for. But here on the show, just to

keep things a little bit easy, I'm gonna

call it vg1. This is volume group one.

And that way, you guys will easily be able to

tell, you know, that I'm working with a

volume group. So I'll stick with vg1, and

then I need to attach the physical

volumes to it. My physical volumes were

/dev/sdb1 and /dev/sdc1.

Alright. So that's gonna create the

volume group, and it's going to attach

those two physical volumes to it. If I do a vg display...oh,

let me sudo that. There we go. I can see

that it's here. I can see that it has somewhere in here,

my size here, my volume group size.

It's only showing 15 GB right now, so I need

to give that a moment for it to kind of

sync up. But if I scroll back up here...oh,

sorry. I'm looking at the wrong volume

group. I already have a volume group for

something else. If I looked at the VG

name, see how that one's called CentOS? Mhmm.

That's one of my, you know, one of the

ones the operating system created, not

mine. So I need to scroll up. Here's vg1,

and it's got the right storage. Thank

goodness. So you can see right here, I have

1.95 terabytes of storage available

inside of this volume group,

ready to be sliced out into logical

volumes. And to create those logical

volumes, we're gonna use lv create,

right, which I'm going to sudo. One of

these days, I'll remember to do that right off the bat.

With a logical volume, we're slicing

out storage and turning that into a disk.

So I'm gonna do lv create, and I need to tell it the size

that I want this to be. Okay? I want mine

to be, let's say I'll just do 200 GB

or, you know, I could do the full 2

terabytes if I wanted or 1.5 terabytes.

You know, maybe I'll do 1,500

gigabytes. So I'll put in 1500g,

and then I need to tell it the volume

group that I'm getting the storage from.

I might have more than one volume group.

I just have one, so I'll type vg1.

Then I need to give it a name. And, again,

normally, I would name this, like, web

storage or file storage or, you know, something

that told me what it was for. But in this

case, to keep things easy, I'm gonna

call it lv1. This is logical volume number one. Alright.

So when I go to create that, it builds it,

and we can use lv display to be able to verify that. Alright?

So when I take a look here again, I'm gonna

ignore the ones the operating system

created and find mine. Here's lv1.

That's a part of volume group one, and it is 1.46

or basically one and a half terabytes in

size. Bigger than one of my disks. Right?

and so i sliced it out i still have more

space available but that one's grabbed

right there and if you look at the

device name that it assigned

we've got slash dev slash vg1 slash lv1

right so it's not so hard to remember

it's not slash dev sda or sd because

there's no physical device these are

virtual mappings slash dev vg1 lv1 now

in red hat and centos they do a really

good job of mapping it that way so it's

easy to remember you just do the volume

group followed by the logical volume in

other distros they'll nest it under a

mapper folder which i likely have if i

go into slash dev and take a look uh

yeah here's that mapper folder that i

was talking about and i don't know if

it'll let me go into there oh it did so

if i go into the mapper folder see

there's a folder in there called vg1 lv1

so that's another name for that device i

could use slash dev slash mapper

slash

vg1-lv1 like that that would also be a

device name that points to the same

place but it's kind of nice having slash

dev slash vg1 lv1 like that that's

easier to remember it makes a little

more sense for us humans just know that

not every distro does that so look for

that mapper folder if you need to but at

this point i've created the physical

volume i've created the logical volumes

uh you know the the volume group in the

middle uh we've created all that it's

running i use the set of commands that

are the most common for me there are

more commands though that are available

so for example i did uh pv display vg

display and so on i could have also done

uh pvs

and uh whoops it is

not liking the fact that i left a bunch

of other stuff on that command so let me

read that there we go pbs uh which gives

us kind of an abbreviated view of the

volumes when i did pv display we got a

lot more information pvs it's kind of

summarized and so i can see the sd b1

sdc1 that are attached i also have uh

vgs for volume group

and so we can see that again kind of

summarized and lastly was lvs to see the

logical volumes so if you just want a

quick view those tools are also really

handy to be able to look at

and then there's like pv scan which will

scan your system and trying to find

physical volumes to bring in so there's

a number of other utilities too but the

basic creation

is done with those handful of commands

right there are we are able to format

and mount the logical volumes like

regular disks done oh yeah yeah so after

this

it acts just like a normal disk it's got

a slightly weird name but otherwise the

normal disk so for example if i want to

format this one let's go

ext4 right so i would do sudo

mkfs.ext4 so i'm going to format the

disk and i'll point to slash dev

vg1 slash lv1 and it's going to find

that disk and the mkfs command will

think it's a physical disk and it'll go

in and format it and now i've got a one

and a half terabyte disk that's ready uh

i can then mount that uh so maybe i'll

uh let me create a folder here i'm gonna

mount that in slash mnt slash

storage

and so i will mount slash dev slash vg1

slash sg1 all right now if you want lv1

and i'm going to mount that into slash

mount storage like that so now it's

mounted and i can go and start putting

files in there and working with it if i

do a df-h i can see slash mount size

storage right there one and a half

terabytes of space available it's ready

for me to start dumping things in there

and i work with like normal do notice on

the left side it's not showing slash dev

vg1 lv1 it's showing that mapper folder

that i mentioned slash dev mapper slash

vg1-lv1 so you may see that notation

kind of switch as you work with it but

they both function the same way i

mounted it it's online

i can forget about the lvm at this point

and just go on with my life and

and never have to think twice about it

it functions just like any other disk

well are they persistent or do we need

to add them to the file system table oh

just like a normal disk we would need to

add this to the the file system table or

f-stab etc f-stab

it's not really any different although i

will tell you

you need to be a little bit careful with

the file system table because uh

a lot of disks get mounted right away

when the system boots and sometimes the

lvm service hasn't started yet in which

case

the mapper might not have had time to do

its job so when you're mounting inside

of the file system table it's usually

better to use the slash dev mapper name

as opposed to using the slash dev vg1

slash whatever because that name might

not exist yet at boot time you can try

it out on yours to see what works but

basically you would just add that to

slash etc f-stab just like any other

disk except for the device name well

here you can see they've got one already

mounted slash var logs audit is slash

dev

mapper slash whatever uh so as long as

you use the mapper name you'll be you'll

be in good shape and you can mount it

and it'll be persistent so don how would

we go about adding more storage all

right so i mentioned that beginning the

episode i had that third disc that i was

holding i got another terabyte so let's

just say over over time

i

fill up this one and a half terabyte

storage drive that i just made and i

want to add more storage well i actually

i've got 500 gigs more storage in this

volume group already that i could slice

out and add and expanded 2 terabytes but

maybe i know that i'm going to add

another terabyte of data so i need to

add more than that 500 gigs it's

available i can go and add another

physical disk to my server and i can

incorporate that into the lvm i can

bring it in as another physical volume

and expand out my volume group once i've

done that i can expand out my logical

volume and then lastly i can expand out

my file system and before you know it

i'll have more storage so let's uh let's

see what that looks like

basically i'm going to go to my command

prompt and first off i would need to

shut my system down add another hard

drive right and then we would need to

create a partition on it i mentioned we

always need to have a partition there if

i do a lsblk i can see that sdd disk

right there

it's already got a partition on it and

actually i didn't bring this up earlier

but i can see the volume group and

logical volume names have been attached

inside of lsblk so i can see those guys

are tied up into logical volumes

all right so i want to incorporate sdd1

into my physical volumes so i'm going to

bring that in just like we did at the

beginning of the show i'll do a sudo pv

create i'm creating a physical volume

yes and that'll be slash dev

sdd1 all right so that's going to bring

it in all right now that just makes it a

physical volume if it had a partition on

it it would have formatted it and and

gotten it ready but it didn't actually

connect it to the volume group so now i

need to add it to the volume group and

for that

uh it's actually a little bit different

when i did it earlier i was creating the

volume group and you can add physical

volumes at the time well now i've

already got a volume group in place so i

i can't can't kind of use the same

command from before i've already created

it so instead i need to use a different

command which is the vg

extend command all right so vg extends

says take the existing volume group and

add more to it i do need to sudo this

which i've

literally forgotten every single time

and so i'm going to be extending vg1 and

then i need to tell it the disk that i'm

adding to it i'm adding slash dev slash

sdd1 and so when i run that i'll see

volume group 1 successfully extended so

what does that mean well if we run

pvs

we'll see my physical volumes right and

i can see sdd1 right here has been added

all right under free space i see it's

completely free versus the other ones

that have had space used up if i do a

vgs

then i'll see my volume groups and

notice how volume group 1

shows that it's actually 2.93 terabytes

it's 3 terabytes and that it's got one

and a half terabytes free there's the

500 gigs from the previous volumes that

i didn't use and now there's a whole

extra terabyte that's been stuck onto it

so it's expanded quite a bit

but

now that i've got that space the volume

group has been updated i need to upgrade

or update the logical volume and for

that i'm going to be extending out the

logic volume now before i do that let me

just run lvs

and so looking at it here i can see lv1

right now is one and a half terabytes so

i'm going to run sudo

lv resize and i'm going to resize that

partition i want to grow it by one

terabyte i'm going to add some more

space to it

so i'll come in and just say dash l and

then

uh plus one g so i'm adding one gigabyte

to it

and then or i don't need one gig one

terabyte there we go something actually

is meaningful

i'm going to add a terabyte to it plus

one t and then i'll specify this is

slash dev slash vg1 slash lv1 that's the

logical volume that i want to expand or

i could use the map or whichever one i

wanted all right and when i run that

it's going to go and extend it and it

tells me that it extended it from

1.46 terabytes to 2.46 terabytes it

added that extra space to expand that

out

you might think you're done and

technically as far as the lvm is

concerned you are done right if i do a

lvs and look at my logical volumes it's

now two and a half terabytes so great

we're done

except the file system won't recognize

that yet if you do a df-h

and take a look at it it still shows as

being a one and a half terabyte

partition and that's because i formatted

it with ext4 right and the ext4

partition that's on there is still one

and a half terabytes it's got an extra

terabyte of free space on it but it

doesn't know that

the lvm has expanded everything that it

can expand now it's up to your file

system so if you're running xfs or

ext4 or whatever you'll need to use that

file systems tooling to be able to

expand it out so this is where things

start to differentiate i'm running ext4

on this one so i can just use its tools

with it you have a utility called

resize

2fs

with xfs you have to use the xfs admin

command or no actually it's xfs

underscore resize i think is the one for

that be careful with it if you're

running red hat

red hat will tell you they do not

officially support resizing xfs

partitions so you can do it but they

don't support it because they tell you

you just need to back up format restore

so that's kind of how they are but uh

but with ext we can just use the resize

2fs command and i'll point that at slash

dev slash vg1

lv1 and if you don't provide any other

options this utility will just expand it

to fill all the available space so when

i run that

it sees there's more space it goes and

acquires that and if i take a look at my

storage now

now that disk shows as being two and a

half terabytes in size and i did it

while it was online i never unmounted it

right it stayed mounted the whole time

users could have been in the system

writing and so on and it just tacks it

on adds that space and we just grew it

and that's really the big advantage of

the lvm is that we can grow our

partitions

without actually having to kick our

users out without having to reboot a

server we can do it all dynamically

while people are in the system working

you know happy shiny people holding

hands and all that stuff and it's done

if i do a lvs or lv display i can see

i've got that two and a half terabytes

is available there and the file system

sees it as well so we are now expanded

and done so after all that i'm going to

ask is it difficult to remove the lvm if

we no longer need it uh you know it's

not difficult to remove it if you're if

you're done with it uh well if you want

to move it to another computer right

maybe i've got this on one server i want

to move it somewhere else you can just

unmount all this stuff

and then take it over to another

computer and reincorporate it the pv

scan utility is designed to find your

physical volumes

when it finds them all it'll bring it

into the lvm and it'll see the volume

group and the logic volumes are all

there so it's kind of easy to

reincorporate somewhere else but if i'm

actually done with it i want to turn

these back into regular disks right

because right now it's a bit of a mess

if i do

lsblk you'll see all these different

kind of notes in here about what it

created with the

the volume groups and the logical

volumes and so on so if i want to just

reset these back to becoming regular

disks we can move through a couple of

steps first i need to unmount the folder

that it's tied to which is slash mnt

storage so i'm going to unmount it

using the umount command so umount slash

mnt storage and then we basically just

go through and destroy everything

backwards so we start at the top of the

diagram with the logical volumes we

destroy that then we destroy the volume

group then we destroy the physical

volumes and when you're done now you've

got regular old disks available again so

what that looks like is we do sudo

lv remove

and i'll remove slash dev vg1 slash lv1

so i'm going to remove that logical

volume and i would remove each one if i

had more than one i just have the one

that was lv remove then we do

vg remove to remove the volume group and

that's slash dev vg1 for me so we'll

remove it it's gone if it has no logical

volumes then you don't get a prompt to

confirm because there's no data at that

point so it just removes and then lastly

we remove the physical volumes which is

a little bit harder because we have to

type each one so i'll do sudo pv remove

and then i'm going to remove dev

sdb1 slash dev slash

sdc1 and slash dev slash sd1 all right

each one removing them and see how it

says successfully wiped it actually

kills off that uh that file system and

now if i do an lsblk

i'll see where those guys are just three

regular old disks with regular old

partitions sitting on them waiting for

me to format them and put them back in

use as

non-lvm disks so pretty easy to remove

all this stuff

logical volume manager another wonderful

episode don thank you very much and

before we move on what else would you

like to say uh you know most of the

distros that are out there turn lv lvm

on by default if you're doing a desktop

install so you likely already have it

on servers it's usually not on by

default they're just standard

assignments and honestly if you're not

going to be expanding your disk storage

then the lvm doesn't really bring you

any benefits so i've there's some

snapshotting features that are kind of

nice that we didn't really get into uh

but you can actually do that stuff

without the lvm too so when i do server

deployments it is rare that i use the

lvm because there's just no real benefit

to it so keep that in mind just because

the feature's there doesn't mean you

have to use it on desktop though it is

kind of nice if you've got a storage

disk and you're running out of space

that you can add more if you've got a

laptop though pretty hard to add extra

storage to a laptop you're usually

replacing a disk and the lvm doesn't

help with that so do understand that

it's not

always the right solution sometimes it's

perfectly fine just go with regular

disks and you know create your petitions

on there can be a little easier to

manage sometimes too great advice thanks

don here's some more great advice watch

every episode of comptia linux plus

you're going to be very glad you did

it's going to help you now in the future

and don't forget about the supplementary

information in our course library and

we'll do only one thing help you be more

successful so check that out too and

tell everybody you know about it pro tv

ig pro tv is binge worthy thanks for

watching i'm zach memis and i'm don

pazette we will see you again soon

[Music]

thank you for watching i.t pro tv

welcome to itpro tv i'm your host rob

live from san francisco

you're watching it pro tv

hello and thank you for watching it pro

tv helping you level up with it learning

everywhere you go i'm your host zach

memphis for this episode of comptia

linux plus file system hierarchy

standard is the name of this episode and

once again don pizet is here to show us

the way don good to see you nice to work

with you again yep glad to be here and

you know we're going to tackle a pretty

neat topic here which is that file

system hierarchy standard uh many of you

as you learn linux have probably

experienced where you browse to the hard

drive and you see all these crazy

folders folders that oftentimes just

have three letter names that don't make

a whole lot of sense and

that's a that's a pretty big obstacle to

getting in and learning a new operating

system so what we're going to do right

here is we're going to take a look at

the file system hierarchy standard and

see what exactly are those folders what

do they do what is their purpose how are

they arranged how it varies from distro

to distro and get a little bit more

familiarity with that file system if

you're coming from something like

windows

well you just have a couple of folders

that it uses inside of linux though

there's actually a lot of different

folders and we need to be familiar with

them

it'll help you better use the system but

it will also help you with

troubleshooting and locating

configuration so definitely a good skill

to have and that's what we're going to

be learning about right here in this

episode so don who created the

fhs all right so the fhs has actually

been around a long time right and that's

easier to say than file system hierarchy

i guess

which oddly enough if i also say that

file system is actually two words right

file and system yeah but in the standard

they decided to combine them which is

not dictionary correct but but they did

so it is the fhs and basically what

happened is remember that a lot of the

linux stuff that we work with came from

the unix world so back in the 1970s at a

t bell labs they created the unix

operating system and they had to use a

file system to identify practically

everything even hardware was represented

as files well it was just a couple of

guys and they were making the stuff up

as they went well fast forward to today

where you've got

linux and freebsd and openbsd and all

these different

platforms that are all reliant on that

same file system structure how are you

supposed to create applications that are

compatible from one distro to another if

you don't know where the files are

supposed to go

so

they teamed up at one point this was

decades ago now and came up with a

standard that said here's the folders

that are the locations that we expect

things to be in in order for your system

to work and by making consistency that

helps with compatibility you know with

linux anybody can go out there so you

you right now if you want could go and

take the linux kernel source code and

grab some gnu utilities or any other

number of utilities that are out there

and package them together and create

your own distro and you can map the file

system however you want you can create

files with whatever name you want

folders whatever name you want put

configurations wherever you want you

have that flexibility it's free right

but if you're expecting third-party

applications to run

they're expecting to see certain folders

and that's where we we end up where we

are today where we have a standard and

the standard was put together i kind of

got off tangent so going back to your

question who created this thing uh it

actually came from the linux foundation

and we've talked briefly about the linux

foundation i think i've got their web

page pulled up here

uh whoops the linux foundation is an

organization that is or was headed up by

linus torvalds himself uh so you know

obviously the creator of the linux

kernel is going to be involved here but

it's actually got members from a number

of organizations companies like red hat

and canonical are a part of it uh the

the individuals that create projects

like slackware and other things they're

all a part of it and so it was a group

effort amongst all the major distros at

the time to come up with this standard

and it's not like they were really

inventing the standard most of it just

came from unix and everybody just agreed

hey from now on we're going to follow

this standard so when i log into a red

hat enterprise linux box i can expect

slash etc to be there and i can expect

configuration files to be in that folder

and then when i go to an ubuntu box i

can expect slash etc to be there and

configuration files to be there and even

if i jump outside of linux and i go to a

bsd system i can still expect slash etc

to be there and that it's going to have

configuration files because that's what

was agreed upon so the linux foundation

they're the ones who put this all

together

almost everybody adheres to it so it

works out pretty well but it does

really kind of come from linus torvalds

himself as part of of his leadership in

the linux foundation well done where can

we view the official standard uh so they

do publish the standard it's online in

what is a spectacularly old web page

this is it right here

so it does look a bit old if you want to

look at it i'll put the link in the show

notes but it's uh pathname.com

p-a-t-h-n-a-m-e.com pathname.com

fhs that'll take you right here to this

amazing web page uh which is uh is

pretty neat

what's really neat about it is how

static it's been it hasn't changed much

i can see right here the current version

is 2.3 which was announced january 29th

2004 which means as of the filming of

this episode that was 15 years ago there

has not been a change to the fhs in 15

years it's a long time uh and right on

here you can actually come through and

view the uh the standard they have it in

pdf raw text and also html which is

probably the easiest and if you browse

into that you'll see the actual standard

laid out right here and they're

basically defining all the different

folders defining which ones are required

which ones are optional and what role

that folder is supposed to serve what is

it supposed to do and you can kind of

see that here like slash dev is supposed

to contain device files or etc is

supposed to have host specific system

configuration and if you drill into it a

little more deeply

we can come in and find specific options

about it variations types of files that

might be expected to be found in there

in earlier episodes we've seen files

like the passwd file so right here it's

defined the fhs doesn't just define

folders it also defines individual files

that are inside of it and so if you've

ever found yourself like scratching your

head what why is the

wd file why is it the same everywhere

it's because it's right here it's part

of the standard everybody adheres to

that uh and you can kind of browse

through and see it's all freely posted

right there online do all linux

distributions adhere to the fhs you know

i'm gonna say that right eventually the

fhs thank you they uh they should but

but as is the case with most

technologies they they don't all do it

uh

most vendors are actually really good

about it so i mentioned ubuntu around

canonical and red hat those are kind of

the two biggest commercial players in

the linux world uh they adhere to it

pretty well but they both deviate a

little ways in fact i'm not sure i could

name a linux distro that completely

conforms to the standard because there's

some there's some weird stuff in this

standard that maybe was relevant back in

2004

and today is is not so relevant so uh

especially when it comes to security

there's some things that have changed

there and then the way that we need to

lay things out so um when i talk about

variations

let me give you an example i don't know

if i can find this real quick in the

standard but uh there's a folder

slash s-bin and here they define it

right here for system binaries okay so

the slash s-bin folder it's got well

system binders great well we also have a

slash usr slash sbin right uh which is

under slash usr so here we've got the

slash usr hierarchy under section four

and you can find slash usr slash sbin in

the list there non-essential

standard system libraries so both

folders have system libraries but one

folder is for essential stuff and one is

non-essential

well

why do we need to separate those out now

in the old days there was a case that

hard drive space was at a premium maybe

you're allocating out partition spaces

to say i've got this much space for

s-bin and this much more space for slash

user spinner or whatever there was a use

case for that but today storage is cheap

you know

we don't necessarily need separate

partitions and so a lot of distros have

started combining those and you know i'm

on a centos system let me show you here

this would be the same uh i believe

ubuntu is doing the same today so you

should be the same on either one but if

i navigate in my centos box i can go to

cd slash s-bin

right and i can take a look inside of

there and i see a bunch of of

executables right so here's these

system binaries and and libraries so

there it is

and then i can go into slash usr slash s

bin

and i can take a look and i see a bunch

of files and binaries

if you're a sharp observer you might

notice that they're the same right i'm

seeing the same files in both folder and

that's because slash usr slash sbin is a

real folder

if i pull up the full listing here see

how i've got s-bin and it's flagged as a

directory right there it's a real folder

but

if i take a look at

slash s-bin

it's a link it's a sim link so what

happened is the centos team really red

hat and the rel team um they made the

decision that there really wasn't need

to separate these out it just creates

confusion so they combined everything

into slash usr spin and then they just

sim linked espen and that's important to

note they sim linked it they said it is

still important to have slash s-bim

because many applications expect that

but we're not going to maintain separate

files anymore we're going to go ahead

and combine it and you'll see that like

up here i've got slash bin same kind of

thing

lib lib64

where if a vendor decides to deviate

from the standard they normally put sim

links in place so that they still

technically conform to the standard even

though the files are in a different

location so if you learn the standard

you'll be pretty safe you know if i'm

navigating my file system and i go to

slash s bin

not slash usr slash spin thanks to that

sim link i still find everything that i

want i can still use the system the way

that i expect so it's pretty rare that

somebody will deviate from the standard

and not put sim links in place to

preserve that compatibility it's pretty

rare for somebody to step in and say i

don't care about compatibility and that

usually only happens on really

really specialized distributions things

like pfsense right that's uh well that's

psd not linux but anyhow uh it's

specialized for firewalls and they say

we don't need compatibility you're not

loading other applications it's just

gonna run what we've put on there and so

those are scenarios where they might

deviate but most people do conform to

the standard well don where would we

normally find applications

all right so our applications that we

run right i just actually showed two

examples right there uh slash bin or

sorry slash s bin and slash usr spin but

applications can actually come from a

number of places uh there's there's

really

i'd say five locations by default but

you can put an application wherever you

want and when you install an application

it can go anywhere you can end up all

over the place that makes it hard to

find that's when in earlier episode we

learned about some commands to be able

to locate applications but for example

if i go into my root folder here

there's a couple of folders that are

really designed to be able to store

those applications and

me zoom in a little bit here

all right the first one is slash bin

right slash bin bin is short for

binaries right and applications are

typically binary files that we flag is

executable and they can run so slash bin

seems like a really great place for

those to be yeah and then there's the

usr the user variant of it which is

slash usr slash bin same thing for s bin

and slash usr

s bin so those are a couple places but

these folders these aren't where like

like if i install openoffice open office

isn't going into slash s-bin or at least

not normally

so

applications that i install

those are considered optional

applications and they will normally go

into slash opt so opt right there so

that's kind of that fifth place so slash

bin slash spin slash user slash bin

slash user spin and slash opt now when

you install an application there's

actually a number of vendors that will

maybe not put it in slash opt to let you

pick somewhere maybe even runs from your

home folder and they may actually create

links to their application inside of

slash user s bin or somewhere and that

that creates that uncertainty of like

where where the heck is this app how am

i going to find it but let me run over

the official standard and what these

folders are for right the first one is

that slash bin uh this one is what's

considered essential command line

utilities that are available to all

users all right so this is stuff that uh

is really considered essential for the

system to work and if you were to browse

around on a system where these were

separated and poke around in there it's

going to be things like um you know

system ctl uh well actually i'd been

administrative commander would be an s

pen uh so there's a little variation

there but commands that are critical

just the regular operation things like

mount right in order to mount a file

system i need the mount command so it

would need to be in slash bin

uh slash s-bin on the other hand that

stands for system binaries and system

binaries are required for the system to

boot up if it's missing a particular

binary it can't even boot anymore when

the linux kernel runs it then needs to

fire up the operating system it needs to

perform operations and

it needs a certain set of commands those

are going to be found inside of slash s

bin

then you've got the user one slash usr

bin

that one is going to have the the rest

of our user commands user commands that

aren't considered essential we don't

have to have them but they're nice to

have so they end up in slash user slash

bin

and then slash user slash sbin those are

non-essential system binaries so the

ones for the system to use for the

kernel to use really

but they're not essential the system

would boot up if those were all missing

now

those four right there really all

pertain to system commands not

applications and that's where we have

slash opt that's where we're going to

find things like

openoffice or whatever we installed it

in fact here let's take a look at mine

so on my system i know it's simlinked so

if i go to slash bin i'm actually seeing

slash usr band they're really the

combination of the two

and if we look around inside of there

we're going to find things like the zip

utility if i want to create an archive

right uh or i'll probably find gzip in

here as well there's yum to be able to

do

package updates and operating system

updates uh

i don't know there's a ton of junk in

here uh the cat utility and things and

these are all these are all really

useful utilities ones we use all the

time right but they're not essential i

don't i don't have to have them in order

for the system to boot but i might have

to have them in order to perform some

kind of operation right uh but it's not

like the system depends on them to boot

if i look inside of slash s-bin on the

other hand i'm gonna see a bunch of

things in here that deal with like

interacting with the file system right

uh things like uh fdisk and so on uh

we'll see

other utilities in here uh like for

interacting with cpu for printing

subsystem and so on this is stuff that

users aren't normally gonna run

sometimes we might though if you watched

our storage episodes we used all these

logical volume commands when we were

working with the logical volume manager

lvm well if you look see how they're all

glowing blue they're not actually here

in this folder they're linked from

somewhere else if i do a dash l

on that

i can see that well actually most of

these are just aliases aren't they so

they're all just linked to the lvm

command which actually is right here in

this folder so not not somewhere else uh

so some of these aren't even really

commands they're just aliases or

shortcuts uh the mkfs commands are like

that uh if we pull up all of those

uh well they actually do show up as

binaries mkfs.ext2 and so on but some of

them like these guys right here are just

aliases off of a base command so you'll

see variations there if i look in slash

opt actually i don't think i've

installed anything on this system uh

looks like i have some kind of red hat

binaries in here whoops

and actually i don't yeah that's an

empty empty little world right here i

haven't installed any applications on

this system because we you know just

installed it for the show but if i start

installing things like openoffice and

firefox and so on if they come from

system packages then they may actually

end up in the main binary folders but if

i'm installing them from a third-party

site then they'll typically end up right

here inside of slash opt that's that

optional software and i've noticed over

the years that linux distribution is the

worst about using this folder

bsd systems are usually the best about

always using pushing all sorts of things

in there so you'll you'll see variations

there but

i guess i've just done a really long

answer to your short questions actually

where are apps those are kind of the

five main places where you're going to

find the application files what are some

of the other key folders we need to be

familiar with doc all right there are a

lot of other folders in here uh and kind

of looking at actually you know let me

i'm gonna use the gui i'm gonna cheat i

don't normally use the gui

uh for whatever reason but let me just

browse into the file system here here we

go

all right so taking a look at these

folders when you browse into your file

system there's a lot of crazy folders in

here some that are

super mega insane important right and

then there's some that you don't really

care about right and if you don't know

them if you're not familiar with them

then they don't make a whole lot of

sense so let's just kind of run through

some of the critical ones

if you read the fhs standard there's

there's actually

something like 50 folders that are

defined you don't need to know them all

for the exam you don't even need to know

them all for real life there's really

just a handful that we deal with on a

regular basis probably the most

important one is this guy right here

boot right the boot folder it's almost

always a separate partition and so this

is more of a mount point than a folder

and that's because it contains the linux

kernel

and other files that are basically fired

up first when your system boots up in

the very beginning this folder gets

called if you're doing disk encryption

you can't encrypt the boot folder

because otherwise you wouldn't be able

to read your boot files so this will be

a separate partition that's unencrypted

it could be combined on a single

partition but it you know causes

problems with things like lux

disk encryption so that's the boot

folder and if you ever browse into it

you'll see things in there basic

configuration files you'll see some

stuff about

vm limits that's the actual kernel file

right there that's the first thing that

gets run and i've got grub that's my

bootloader that helps it find the

partitions of the operating system

efi that's my for my uefi bios so that

it's able to locate and boot the

operating system so all really critical

stuff and that's found in slash boot now

you rarely mess with what's in slash

boot in fact on a lot of distros this

will be mounted read only because you

don't want somebody messing with the

kernel on the disk and so it's kind of

flagged that way uh it depends yours may

or may not be flag read only and you can

always mount it read write and that's

what happens when you do an update or

you need to change your bootloader uh

it'll have to get mounted that way

there's slash dev which is short for

device and this is where all of our

device definitions go

in linux every piece of hardware is

represented as a file and if you watched

our storage episodes we saw that with

hard drives all of our hard drives were

slash dev slash sda1 or slash dev slash

uh uh you know logical mapper slash

whatever uh

it doesn't just stop with hard drives

you'll see that with your console your

displays your video card printers all

sorts of things will show up in here as

a device file so that's kind of what i'm

looking at here on this screen is that

big list of files

for all these different devices and if

it's not in here then that means your

system doesn't know about that piece of

hardware you're probably missing a

driver or something else that's critical

to make that work so kind of an

important folder when it comes to

troubleshooting hardware

also an important folder if you're

trying to mount a partition

you'll find the name right inside of

here

all right let me go back up

let's see that was slash dev etc that

was super important right uh this is

where our systems configuration files

will go and we've gone into this folder

in a number of episodes already and

we'll continue to go into it for some

other ones uh but basically

when you fire up a service or an

application it needs to know a

configuration and that configuration is

expected to be stored here it's mostly

just packed full of text files so it's

not a very big folder but those text

files are really important because it's

telling your system how to configure

itself

slash home that's where your user home

folders go so all your personal files

and this will oftentimes be a separate

partition as well so that if we fill it

up it doesn't mess with the regular os

it still has room uh but it doesn't have

to be in fact i think on my system it is

actually part of my main partition uh

but anyhow that that's where your your

user stuff will go uh slash lib those

are for libraries or linkable libraries

when applications run they rely on

libraries to provide basic functions and

they're usually shared between

applications and so you'll find those in

that i'm on a 64-bit system so i also

have lib64 when linux was created it was

all created around a 32-bit ecosystem

and so when you see a folder like lib

it's expecting 32-bit libraries to be in

there

uh see some other ones mnt mnt is an

optional one you might not have it it's

short for mount in the in the olden days

when we would mount a hard drive or a cd

drive or whatever we would mount it to a

folder under mnt a lot of distros don't

do that anymore the ones that do auto

mount will usually mount it under the

run folder and i see i've got run also

run is not in the fhs not in the

official standard it's just something

they made up uh but it's there and it

just shows you can choose to deviate if

you want to uh let's see slash proc

whoops i'm gonna break my system uh

slash proc is a really neat one because

this looks like a folder actually isn't

right this is a virtual file system

every program that you run is a process

and the process is represented as a file

inside of slash proc so it looks like

you're navigating around in fact if i

let me go back here

if i go into

slash proc and pull up a listing i see

all these numbers right what are those

numbers well if i were to pull up a list

of applications that are running on my

system i see all of these process id

numbers right here that's the process id

of that binary running in ram well in

the file system that all gets

represented right here as files and you

can even browse inside of them so if i

pick one at random like

10971

and pull up in there i can start to find

out information about that application

that's running these are not real files

this is all virtual it's being

dynamically generated as i navigate

through and it's part of that whole idea

of representing everything as a file not

everything in linux is represented as a

file there's an operating system called

plan nine where they went hardcore and

everything literally everything is

represented as a file in linux

most everything is represented as a file

and so that's kind of what we see in

slash proc

similar to that slash sys if you go into

slash sys

this one is

not showing me information about

processes but showing me information

about the system the you know the linux

kernel and what it's running and so

you'll see things like device detection

if you're running a hypervisor kernel

data that's all tucked away in here

and so when you run something like you

name dash a to find out what kernel

you're running in the version that's

actually coming from inside of slash sys

it's looking in there to find it uh and

represent that uh put it on your screen

uh i already talked about slash usr a

little bit for slash usr bin and usr

spin

this one is supposed to contain our

uh

really binaries and any other kind of

data that's designed to be shared

between users and i don't mean shared

across the network i mean shared between

users right here on the system if

there's a library file and i have a user

account zac has user account there's no

point in storing two copies of that

library we can put it in usr and now we

both have access to it so that was the

original intent of slash usr

on servers that makes a lot of sense on

desktops it doesn't and so it really

just kind of became the de facto dumping

grounds for everything everything is

considered to be shared amongst users

these days and that's why you're seeing

more and more distros linking to the

slash usr folder but if you look there's

more than binaries in there there's

libraries so i see lib and lib64 you

might have an etc folder here for shared

configurations source folder

games if you've installed games i didn't

think i had but apparently oh it's empty

all right so so i don't have anything on

there but these are things that every

user would then have access to inside of

slash usr uh neat little fact here this

is supposed to be mounted read only

based on the fhs that way users can't

tamper with binaries that other users

might be accessing

very few distros actually mount this

read only today though so this is

another area where a lot of people

deviate from the standard

and another important folder for us

let's see that was usr

is var that one which is just short for

various uh and this is a various folder

uh but it's not too much variety there's

actually a couple of things we expect to

be in there first off our log files log

files for our applications for the

system for journal d for all these other

different things those log files get

dumped into slash var log but also our

print spooler right when we print a job

it gets written to the hard disk first

before being sent to this printer and so

that's inside of slash var spool and if

you poke around in there you'll find a

lot of things like there's spool like i

mentioned

and log is right above it

but you'll see other things if you're

running an email server your mail cues

will be in there the point about var is

that it's supposed to be considered

temporary the stuff that's in there

isn't supposed to be mission critical if

i lose my logs the system's actually

fine i'll be okay if i lose my principle

it's fine i'll be okay if i lose email

in transit that kind of thing it's not

really critical it's just temporary and

in in the older days you'd put your

cheaper storage in slash var because you

didn't care about it as much today

storage is so cheap and so reliable that

slash var is usually pretty stable and

so people will put things like security

logs in there that they intend on

keeping long term even though that's not

the original intent of the folder don do

we really need to have all these folders

memorized once you just went over

yeah so um for the exam yes yeah you do

at least the ones that i named off right

don't don't go and memorize the whole

fhs that's crazy uh the ones that i just

highlighted those are kind of the

critical ones you want to know those for

the exam uh in real life land you can

actually cheat a good bit if you don't

remember these folders it'd be fine

because it's so easy to find files in

fact we have a whole another episode

coming up on how to locate stuff in

linux but just to kind of do a preview

of that um

let me

get to a console here so let's say

let's say i want to find a file um

yeah i'm i'm going to be using the

uh would be a good one like the ssh

command right i'm going to ssh into a

server and i want to make sure that ssh

is a certain version and so i need to

take a look at the ssh file itself and

and you know test it well where is it

stored i might have to think to myself

all right well is that an essential

binary or is it non-essential is it in

slash bin slash s bin slash usr bin

it could be any number of places or is

it in slash opt has it been overwritten

where is it at and so you can use

commands like which

and the which command if you say which

followed by a

a

a binary it's going to figure out which

actual binary would be run if i typed

that command and so it tells me this one

is slash usr

bin

ssh now there could be more than one

copy of ssh though and so we can use a

different command which is where

is

ssh and then it's going to find if

there's multiple copies it'll show me

the multiple copies and even things like

the help documentation and the

configuration so it's telling me right

here the binary is slash usr slash bin

ssh the configuration is slash etc ssh

and the help documentation is slash user

slash share slash man slash man one

slash ssh dot one dot gz

rolls right off the tongue so it does it

just rolled right off we don't have to

remember it if we can remember the where

is command right you can run that into

it there's also locate and find locate

and find i'll cover in another episode

but they're useful for finding not just

binaries but any file if i if i'm just

trying to find a file i forget where i

saved it i can use locate i can use find

they'll

find it obviously let's send the name or

locate it and then we can get that so

if you don't have it memorized you'll

get by just fine in most scenarios but

it takes time if you have it memorized

and you know exactly where to go you

save a lot of time and you're more

efficient so that's the way i always

look at it when it comes to memorization

awesome don file system hierarchy

standard and this is another great

episode fhs if you please and done

before we move on anything else you'd

like to say

uh you know this is all fairly static it

doesn't change very much and i know this

is our linux plus show but if you learn

the file system hierarchy standard when

you jump over to like macs that run mac

os they actually conform to a lot of

this and if you jump over to bsd a lot

of them conform to this too so you learn

it once but it applies to more than one

operating system that's a pretty cool

thing so definitely spend some time and

get familiar with it great advice thank

you don and a great episode as usual

comptia linux plus is a wonderful series

and it's designed to help you so make

sure you watch every single episode

you'll be glad you did and don't forget

about our course library where there's

supplementary information that's there

to do one thing help you go even further

so check that out as well and tell

everybody you know about it pro tv it

pro tv is binsworthy thanks for watching

i'm zach memphis and i'm don pizza we

will see you again soon

[Music]

thank you for watching it pro tv