CompTIA Linux+: Managing Services #11

you're watching it pro tv

hello thank you for watching it pro tv

helping you level up with it learning

everywhere you go i'm zach mimis your

host for this episode of comtea linux

plus and in this episode managing

services with system d

and we've asked our expert don bissett

to tell us more don good to see you

thank you for being here glad to be here

zach i'm ready to dive into the world of

system d which is a a really powerful

new technology i'm gonna say new it's

actually been around since i think 2008

but really gained prominence in the last

five to six years uh it is responsible

for powering the majority of linux

distros that are out there so really

important that we need to learn how it

works what it does and because

everything it's doing is behind the

scenes so in this episode what we're

going to do is kind of pull back the

curtain and get a chance to see exactly

what systemd is like how it functions

how we can interact with it to keep our

system up and running and make sure our

services are operating properly well how

are services controlled under linux

so you know there's a few different ways

this stuff can be controlled and we're

going to focus on systemd but it's not

the only way so that there's many

systems that use the original unix init

system so you know when linux boots up

there's a kernel right and the linux

kernel gets called that's really

important that's the part that linus

torvalds wrote but the kernel itself

doesn't interact with the user much it's

kind of limited in what it can do so

it's it's actually i don't want to say

it's not impressive but from a from the

standpoint of computer technology is

very impressive but from a standpoint of

human interaction it's not very

impressive at all so what the kernel

does is as soon as it fires up the next

thing that it does is it launches what's

called an init process right and the

init process is what actually starts all

the various services and makes your

operating system work it seems super

impressive to us but technologically

it's really basic in fact in the

original

versions of unix they use what was just

called the cis v init the sysv system 5

was for tracking one of their releases

uh and the this is via net what it would

do is it just had a folder with a bunch

of scripts in it and it would run the

scripts one after the other and that

would start your system up and then when

you got to a log imprompt it was because

at some stage a script was run that

launched a shell or a console and then

made that available to you

that's all it was right well

that sounds really simple because it was

really simple and it had certain

drawbacks certain problems certain

things that made it not efficient the

the number one problem

was that cisvi and it ran everything in

a serial order one after the other so it

would run script one then script two

then script three and if script three

took a long time or if it hung the

system just waited right there right so

it wasn't terribly fast so that was one

problem that it had the other problem

was that the scripts were just run when

the system booted up right and

after that if something changed on the

system the scripts wouldn't detect

unless you recalled them that wasn't a

problem back in the 1970s and in the

1980s but fast forward to

2019 where we are

we are constantly adding and removing

hardware from our systems you might have

usb thumb drives or webcams or whatever

that plug in and go

imagine if you plugged in a usb key

and the system didn't even know that

happened right and we had to run these

scripts and tell it to look for that

missing hardware so that is is a problem

with cisv init

system d was created as the next

generation of that to try and replace

this via net and do it in a different

way so what systemd does is it's not

just a series of scripts systemd is an

actual program it's an actual uh demon

it actually stands for system demon and

if you've never heard that term it's d a

e m o n daemon and in the the unix and

linux world any program that runs in the

background is part of the operating

system that runs unattended from the

user is called a demon it's just you

know doing its little job in the

background it actually comes from a

physics book that described some weird

physical motion

it's a crazy history some other book

yeah but but basically uh you know these

demons are what the system is running in

the background and it used to be the

init scripts that would call them now

it's the the systemd executable so this

program runs and it doesn't just run

when the system boots up it stays

running so systemd stays running the

whole time your system is online and

it's able to do things like well one it

can detect hardware changes because it's

always running

two

it can do things in parallel it can say

all right i'm gonna run these five

routines all at the same time and if one

hangs it's no big deal because i'm still

doing the other four and so it's faster

right system d is faster more stable

there are some people out there that

don't like system d you'll hear some

people complain about it from time to

time because it moves away from

tradition right traditionally you can

control your entire unix-like operating

system by modifying some text files with

systemd it's not like that it does have

configuration files but it's actually

doing a lot of stuff in a binary and

some people if you're like a unix purist

that that's not something you're happy

about but i'll tell you as a linux

administrator myself i love it i think

it's faster it works really well and

it's a great way to deal things so

systemd

is is pretty prominent and it's

typically how we're gonna be

manipulating services so going back to

zack's original question hey how are

these services controlled

you power on your system right uh your

bios kicks in it reaches out to the hard

drive it finds the boot files loads up

that linux kernel then the linux kernel

launches init and in this case it would

be a systemd that it fires up and then

systemd goes the routine of starting

everything else that you need to

interact with your system and bring it

online if it's supposed to right if

there's things that aren't supposed to

come on it leaves them off they system d

is what's actually controlling that so

how do we know if we are actually using

system d all right so uh you know a good

question i will say that all of the

major linux distros these days use

systemd so if you get ubuntu you get red

hat enterprise linux centos fedora mint

um

it just goes on and on all of the major

distros use systemd by default so the

odds are you are using system

but it's not 100 there are some distros

out there that have chosen not to go

with systemd that are following the old

model the sysv init or there was

actually another uh init system called

upstart upstart was really just a slight

modification assist via knit it wasn't a

full-on like change

so you may find some systems to do that

typically if you're running older

versions like if you're running centos 7

you're using system d but if you're

running centos 6 you're using upstart uh

and if you're running centos five you're

running sysv in it right so you know

it's kind of changed over time uh if

you're running ubuntu they've been

running systemd for the last three or

four versions i think and then before

that they were on sysv init

but there are current distros like

slackware slackware was actually the

very first linux to show that i ran back

in 1993 1992 three i think 1993. either

way uh it ran cis via knit back then and

it runs cisv in it today uh they have

not moved over to systemd because they

follow that model of a pure unix type

deployment um there's other ones there's

variants of debian i think there's one

it's called devon that actually uses uh

cisv init instead of systemd but none of

the major distro so if you're deploying

something in a business if you you're

deploying something where you need

support all of those companies are using

systemd but we can verify right let's be

sure right so i've got a centos box here

in front of me and i'm running centos 7

and i know that it defaults to systemd i

shouldn't say default like it just runs

systemd but if we want to verify that

there's a few different ways

i mentioned the boot order earlier and

when i turn the computer on the kernel

fires up and the first thing it does is

it starts init right so technically your

init process is the very first process

that runs it's number one so if we use a

utility like ps right this is the the

process show command it shows me all the

processes running in the background if i

do a ps

aux right that says show all of the

processes with the user that's attached

to them

and uh x i can't remember what x does

it's like a

services or something i can't remember

what x is but if i do ps aux and i run

that it will show me all the processes

that are running and if you look at

these process numbers right it's this

second column i've got process

13999 process 14 416. when a process run

it gets a number and when it closes the

number goes away and it just kind of

keeps incrementing and growing so over

14 000 processes have run since i booted

this machine up yeah it's a couple which

hasn't been up that long right i've been

up for an hour and a half

and so so that's a big number but if i

scroll up on that list let's just scroll

scroll scroll the numbers get smaller

and smaller these are processes that ran

early on and are still running and i can

find process number one right there pid

number one and if i look to the side

over here it's slash usr slash lib

system d

system d that tells me that i'm running

system d i'm on centos let me jump over

to ubuntu uh so this is an ubuntu

machine

and i can do the exact same thing here

if i jump into my terminal

and i'll just do ps aux and i'm just

going to feed it in the head command

save a little time and there's my

process id1 and i can see oh

this one's actually a little interesting

it's telling me slash sbin slash init

all right now wait a minute i said this

defaulted to systemd

that's not systemd if you see slash s

bin slash init that's cis v init okay

however if i relied on this i might get

tricked and let me show you what i mean

because i'm i'm like 99.9 positive this

should be systemd i didn't check before

the show so that's my fault but

if you pull up a a listing i'm going to

do an ls-l of slash s-bin i want to see

the binaries they're in that file or in

that folder okay and i go in here and

find init actually let me just do spin

slash init there we go

they're tricking us right it's a sim

link they have sim linked slash s bin

slash init does exist

and it's being pointed to slash lib

slash system d system d so there we go

so it it looked like in the process

output that we want now that means that

we can't really trust the process output

can't we the psa ux it works if we're

full on running systemd but it could

trick us with older stuff but if we

actually look at slash s bin slash init

it's in the official standard

that any operating system that's using

the linux kernel has to have a file in

slash s bin slash init even if it's not

what's called it has to be there and so

that's a more reliable way to look and

see what we're running if i jump back

over to

my centos box and do the same thing

in slash s bin i'm going to look for so

i'm running ls dash l to get a long

output slash s bin init

and i can see they've linked it there as

well the difference is in the process

command that ps aux on centos it's

actually showing me the true binary on

ubuntu they're showing me the link

instead of the binary so they've changed

the way that command works but i can see

here they're both clearly running s pen

now i mean uh systemd now if i looked in

here and it actually said slash spin

slash init and it was green and that was

that then i'm running cis vie init or

upstart upstart i mentioned was like an

upgrade to csv init not a replace not a

uh like a true change so it would be

slash s bin slash init as well so i

wouldn't be able to tell the difference

between upstart and cisv in it but that

doesn't matter for us i know we're

running systemd

so system b starts right after the

kernel right yeah and then what happens

so it's a big deal right it's the first

thing and if system d crashes your

system crashed right it's powering

everything

so once it fires up then it's got to get

to work and it says okay now it's time

for me to do my job i need to start some

services i need to

open up some ports on the system for

network communication and i need to

figure out whether we're going to load a

gui or go straight to the command line

it has to do all these different things

and it does that through the use of unit

files all right unit files are stored in

if i change directory i'm going to go

into slash lib

system d

slash system all right if you go into

that folder right there and pull up a

listing what you're going to find is a

ton of files and each file will be a

name and then it'll end in dot

service or dot

timer.target.slice each one is kind of

something different most of them are dot

service right those dot service files

are identifying demons demons are gonna

run in the background those are services

or applications that run and just keep

running and do the thing all right

other ones though like dot target that's

a user environment right so there's a

dot target file for if i want to have a

gui right it's actually i think called

graphical or something or another uh

it's in here

remembering the names of these is pretty

hard because there's so many yeah

graphical dot target right there uh and

then there's a command line one which is

actually called multi-user.target

um there's several others that are like

that you'll also find dot socket which

is identifying a network port that's

going to be opened or possibly a name

pipe or something that'll allow an

application to talk to another

application via network service

let's see that was targets services and

sockets those are the most common uh you

also see dot mount so those are where

it'll mount a hard drive partition you

still have the file system table the

slash etc f-stab file but you don't need

it anymore if you don't want because

systemd can use dot mount unit files to

be able to specify what to reach out to

so when systemd starts it's actually

parsing all of the stuff in this

directory and looking at it and saying

all right what do i need to do and if we

pick pick somebody here somebody easy

ssh all right so ssh

that's the secure shell daemon and it's

running by default on most linux systems

so that we can create a secure remote

shell connect in and get a command

prompt right so it should be starting

when the system starts if i look inside

of that file so let me just get

sshd.service

there we go there's not a whole heck of

a lot in here right so it starts with

this

blocked quote right here where it says

unit

and from there it goes on to specify

information about that daemon so i can

see that this is the open ssh server

daemon

and it has documentation right there's a

man file that's available for it so it's

got that registered here and then it's

saying we want to start it but don't

start it right away you need to make

sure that the network.target environment

has started right we have networking

working before we can start ssh and we

need the sshd keygen service running so

this is setting up a hierarchy right

like an order and dependencies that

systemd is able to use so it says okay i

can't start this one first i need to

start network.target and i need to start

sshd-keygen.servers

so once those are started then i can

come back and start this one and

remember it's doing all this in parallel

so it can reach out and try and do

multiple things at once

and then after that it says once right

after says i can't start until these are

started but once means once i start i

want this other thing to start as well

now

sshd-keygen service is listed twice

and what that's saying is it needs to be

started when i start

but if something goes wrong i need that

one started again because i depend on it

right that's establishing that

dependency too so we've got those

entries and then after that we see the

actual service right and so it's telling

me i can find my configuration in etc

sysconfig sshd that's where the

configuration is going to be and then

when we execute when we start here's the

command that it runs slash user slash

spin sshd that's an executable dash d

tells it to run as a daemon run in the

background otherwise when that shell

closed the the application would close

we can see what to do when we close it

and it's going to run the kill command

and it's going to shut that one down

you can see how to kill it uh your just

uses a process execute sounds really

violent but just exiting

and then restart if there's a failure if

it crashes it's going to try and restart

it and if it does a restart it'll wait

42 seconds right this is all defined

right here when we install this service

if i want it running every time it's

going to install it and attach it to

multi-user.target so when that starts

which is our our command line when the

command line starts it's going to make

sure that ssh starts as well these are

unit files and the unit files are just

text you can you can actually go in and

edit them if you want it's not a great

idea to edit these most of the time when

you install when you install the sshd

service it creates these files for you

right so i didn't create these they were

made for me if i want to modify them i

usually will not modify it here because

if we get an update right so sshd gets

an update and they update this unit file

it's going to overwrite my changes i'm

going to lose them so if you're going to

make changes don't make them in this

folder so i'm let me get back in here

actually let me zoom back out there we

go

so i'm in slash lib systemd system you

really should never edit anything in

slash lib it should just be left alone

but if you go into slash etc

slash system d slash system all right

and that folder might not exist you

might have to make it on centos it's

there by default so slash etc systemd

system as opposed to slash lib system d

slash system

when you look in here

you'll find service files and target

files and once and so on these are where

we're overriding what's done inside of

systemd's folder so if i want to make a

change i shouldn't go and change the

files in slash lib i can come right in

here and create a file with the same

name and when i do that this file will

override the other one and when an

update comes along it'll leave my file

alone right now i haven't overridden

anything and so everything in here was

done by the system looks like dbus is

doing some stuff we even have cis in it

which is for backwards compatibility

with cisv init so some of these entries

are already in here most of these are

just links

in fact i think all of them are dis

links um

well no some of them just have weird

permissions on them but many of them are

links to other locations and you'll see

them pointing to slash lib or somewhere

to override an activity

from somewhere else so that's how that

kind of functions and that's really the

heart and soul of systemd systemd is a

binary it's an executable that runs uh

in fact i think i browse right past it

if i go into slash lib system

d

and we look in here

somewhere right here there we go there's

the systemd executable it's actually not

very flashy at all

um

it's

let me do a human readable size on this

one

it's a blazing 1.6 megabytes

and yet that 1.6 megabytes is powering

my entire user environment right now the

kernel is handling my hardware

interaction and then systemd is making

all of my software work i mean this is

super important software and it's really

really small and then it relies on all

those configuration files to do its job

well do we have to create unit files for

new services done

you can so if i'm installing a new

service you can go in and create and

define them yourself i've had to do that

on occasion but normally the services

will create them automatically right so

let's say um let me give you an example

i wanna i wanna be a web server right so

i wanna install apache well if i take a

look inside of

my systemd library folder so i'm back in

slash lib systemd system

if i look for any file that starts with

http oh shoot i actually see it right

there httpd.service so it looks like i'm

already a web server

that file is defining the apache service

and getting it set up to do its job

right well i didn't create that file at

least i don't remember doing it so when

i installed httpd

it must have created that right if i

were to remove httpd so if i do

a sudo yum

remove or

we'll just remove

httpd

there we go so i'm going to uninstall it

so i'm uninstalling httpd

and now i'll just take a a look again

and i see there's no unit file defined

for it anymore so when it uninstalled

httpd it knew to take away that unit

file and if i install it back again

so i'm just going to come in here and do

an install

and when i do that it's going to go

through it's going to do the install

there it goes and if i take a look

i can see it created that file again

right so most services if you're

installing them from a package file

whoever the developer is that created

that software will know hey this is the

unit file i need and i'll go ahead and

create it for you so you really don't

have to worry about it you don't have to

go in and edit these things it just does

it now i will tell you sometimes you

install software and it doesn't create

the unit file for you right

and that might be software that's

designed to be run on demand okay

you need unit files for software that

runs in the background that runs

automatic without interaction i want

this web server running all the time it

needs unifile but if i have something

like openoffice right openoffice i need

it to run when i'm interacting with it i

don't want it running in the background

so there's not going to be a unit file

for openoffice those are just user

applications all right but sometimes

when you install a service it still

doesn't create the unit file because

most services don't start automatically

you've got to configure them and we do

that with systemctl

systemctl is a utility that is packaged

alongside systemd it's the system

control utility and it lets us control

system d and do things that otherwise

might require us to edit some files so

for example if i want httpd to start

right now

i could do sudo

systemctl

start

httpd right and it's going to read the

unit file it's actually looking for that

unit file and

i should have typed this the whole way

and said httpd.service

because that's the unit file name and

it's going to look for it find it and

start it okay if you leave the extension

off it looks for services first because

i'm i'm telling it to start something

and that way it doesn't look for

socket.target and those guys right but

if you want the full name would actually

have service on the end so it's going to

find it it's going to start it all right

now we can start a service even if it

doesn't have a unit file if it's got

dependencies things will break but even

if it didn't have one you could do that

but when the system reboots

that service is not going to be started

by default unless we tell it and we

could go in and modify the the actual

service file and or the the unit file

and actually tell it yes

multi multi-user or graphical depends on

this file we need hdbd to start right or

you can just do it right here with

systemctl you can say systemctl

enable

httpd right and when i run that look

what it did

created a sim link from slash etc

systemd system

multi-user.target.once slash

httpd.service so it put a sim link in

slash etc

right and it's pointing to slash user

slash lib system d

system slash http dot service it didn't

edit anything inside of slash lib or

slash user slash lib it edited instead

inside of etc and even then it wasn't

really an edit it was just a link that

says now whenever i start the multi-user

target i will also start httpd

it did that for me i didn't have to

create unit files i didn't have to go

and modify etc it was just done

automatically and if i say you know what

i want the web server but i don't want

it to start automatically i can just

come in and say disable

and when you do that it removes the link

it didn't disable apache i can still run

apache if i do

sudo systemctl start

httpd

like that

and then i can do status so

sudo systemctl status httpd and it will

actually show me that it's active and

running right now it's up and

operational okay

but looking up here it says that it's

loaded but it also tells me that it's

disabled which means if i reboot apache

is not going to be running anymore right

if i go back and enable it again

let me do that

and then we pull that status what i'm

going to see now is that the service is

enabled so if i reboot apache is going

to come back up and we'll be in business

so the important part here is that we do

all of this

without ever actually editing the file

system ctl does it all behind the scenes

for us so this is another area we kind

of deviate away from the way that unix

used to be where we'd edit all these

script files now it's all automated

through systemct how does systemd keep

track of which services to run

so we've kind of seen that a little bit

with our dependencies but it really

boils down to the target files right the

target files are what defines what's

going to get fired up and and so on uh

so let's take a look at one of those

target files uh if i give what folder am

i in i'm in the right folder slash lib

system t system

if i pull up a listing of star.target

all right you'll see a lot of targets

right targets are like collections of

unit files it says hey if i want the

network to function i've got a network

target and here's all the stuff i need

for the network to function and you'll

see other things in here that are like

that you know if i want uh remote crypt

encryption to work or reboot to work or

swap files to work they've all got

different targets and those targets are

defining all the services needed to make

that work so if i want to boot to a

command line

just to a command line then

i need what's in multi-user.target

if i take a look at that file let's use

less on

multi-user dot target

i can look in there and

it's actually not a very big file at all

right but that's because it's including

some things that it needs it says all

right multi-user can start whenever it

wants right so i don't see a actually do

i have a required don't i here's a

require that says basic.target so before

multi-user can start anything in

basic.target has to start then this one

can start and once it starts afterwards

we'll fire up basic.target if it isn't

already but we've got rescue.service

rescue.target we've got a few other

things uh that can be triggered as well

to you know get our system up and and

running well let's look at basic.target

so if i just i'll cat basic.target and i

look in here it's also not very big at

all right but it's got that it requires

systenit.target

and it's calling several other things

too and if i look at

sysinit.target we're kind of following

this back right going back back back and

we see how it's calling more and more

things

all of this is coming together to build

the environment that that is required

for our system and boot if i want a

graphical user interface then i would

need graphical.target

and you'll see where it depends on

multi-user the command line has to work

before i can launch stuff to get the gui

right they all are in their

interdependent as a human it would take

me forever to sift through all these

files and build out the tree of

dependencies but systemd does it every

time you boot it parses through these

files figures out the dependencies it uh

it actually doesn't monitor the files on

an ongoing basis if i change a file

right now it won't recognize it unless i

tell it to reload right so you can go in

and well you can reboot your system it's

one way to do it or like with a service

you use the restart command and it'll

reload the unit file for that service

but otherwise it's figuring out all

those dependencies kind of mapping it

together it gets pretty complex because

the sheer amount of files but at the end

of the day they are just simple text

files you can view it all well can we

change between the targets oh you can

yeah you know my system right now if i

were to reboot

it's set to default to the graphical

user interface so it it's basically set

to boot to graphical.target right but if

this is a server i don't need a gui

i might want to have the gui installed

so when i'm sitting at the server i can

use it but the rest of the time i don't

need a gui run so i could actually

switch targets and tell it hey don't

don't run graphical anymore just run

multi-user and we can do that pretty

easy um so here i am in my uh my my

command prompt and i could be in any

prompt it really wouldn't or any folder

it wouldn't matter which one i'm gonna

do a sudo system systemctl

isolate and so i'm telling you i want to

to target a specific

target that's a little redundant and

then i'll just say multi-user.target

all right multi-user means yes it's

command line but it's not just one

console just for you it still allows

network services and other things to run

and so when i run that

see how my system freaked out right it

recognized immediately okay we're not

going to do the gui target anymore we're

going to just do a command line and so

now i can come in and i can log in

assuming i can remember my password

and there we go i'm not in a gui i'm

just in the command line and so now if i

reboot i'm going to boot just to

this interface right here you know this

is my actually i need to go and modify

my grub

loader to tell it exactly what i want um

but at this point i'm now in just

command line x windows and all the gui

stuff is not running at all

i can switch back with that same command

uh well a little bit different so if i

do sudo systemctl isolate and then i'll

do graphical.target

and that's going to switch me back and

now it's loading x window see how it's

taking a moment it's not like x windows

was running in the background it's got

to relaunch all that stuff and get me in

here and uh and get me put back in place

now because this is happening after my

boot operation and all that you see how

it's kind of like screwed up my

resolution and all that

that's one of the trickier parts about

moving from one to another if

something's not defined in the unit

files or some other configuration file

then stuff gets lost actually it cleaned

up well sort of i got large icons now so

so um you know there's there's a lot of

people like me if you do this stuff a

lot you'll end up writing scripts to

kind of automate some of the changes and

so for example i have a little script in

my home directory

this will show you how often i do this

to manually set my resolution so that i

can just run it real quick

and then i'm right back and now my

icon's the right size and all so

you can switch between targets at the

drop of a hat but just know that there

are certain penalties to it there's

certain things that happen on your

system you might not expect so

definitely test it out you know run

through the routine a few times but

that's basically the nuts and bolts of

systemd right there there's a lot more

to it but the basic operation is there's

a ton of text files it reads them and

performs as they dictate great episode

managing services with system d

wonderfully done done and before we move

on what else would you like to say uh

you know if you want to learn more about

systemd definitely check out that

there's a systemd website uh it'll be

like your first google response i'm sure

but there's also a whole suite of other

utilities i use systemctl there's

journald for the special logging system

that systemd uses uh there's hostname

ctl for changing your computer hostname

there's a number of other ctl commands

that are all part of what systemd

creates so there's a lot more than we

went over in this episode i was just

focusing on really what's involved in

getting your system up and operational

and getting your services running great

advice as usual here's some more great

advice watch every episode of comtea

linux plus you're going to be glad you

did it's going to help you now and in

the future and keep studying take

advantage of the supplementary

information in our course library it's

there to do one thing help you be even

more successful so check that out as

well and tell everybody you know about

it pro tv you know i t pro tv is binge

worthy thanks for watching i'm zach

mamas and i'm don pizza and we will see

you again soon

[Music]

thank you for watching it pro tv

welcome to it pro tv i'm your host

live from san francisco

you're watching it pro tv

hello thank you for watching it pro tv

helping you level up with it learning

everywhere you go i'm your host zach

memphis for this episode of comptia

linux plus

managing services with sis v in it is

the name of this here episode and don

pizzat is going to be showing us the way

don good to see you thanks for joining

us well glad to be back zach and like i

said we're going to be tackling cisv and

it you know in the last episode we got a

chance to see systemd which is the most

popular initialization system in linux

right now however if you're working out

there in the field there are still many

many systems deployed that use the older

sysv in it in fact on servers it's

actually pretty common so definitely

something we want to learn about and

that's what we're going to do right here

in this episode we're going to learn a

little bit about how cisvne how cisvi

init works

uh we'll see how it works we'll see how

to manage it manipulate it and just kind

of see how to operate in an environment

that is running sysvena fantastic don

how are services controlled under linux

all right so um basically all of our

services inside of linux depend on an

initialization of some sort right and

system d we've already seen is one way

right your system boots up and system d

is well kind of follow a boot process

right your kernel kicks in and then it

launches your init and so system d would

launch and then it would control

services from there but with csv init

it's a little bit different your system

boots up your kernel fires up and then

the kernel calls a series of scripts cis

via init is not a binary execute well i

guess it sort of is there's a little bit

of a binary but the majority of what it

does is all being driven by script files

we saw with systemd it was driven by

unit files which aren't really scripts

they're kind of like descriptions and

then systemd knows what to do with it

and executes cisvnit just runs scripts

whatever happens to be in the script it

runs and you can put literally anything

in there like starting a service and

passing parameters to a service and

checking to make sure the service is

running or the permissions are set right

all of that is different things that

cisv and it can do and it does it in a

significantly different way than system

d does now many people feel the systemd

is the

better of the two options for modern

systems but if you've got a server that

millions of users rely upon and it needs

to be rock solid cisb and it's been

around since the late 1970s early 1980s

it's a very stable tried-and-true system

and that's why when you go out and look

at servers you'll find many still do it

let me give you an example if you're

deploying linux servers in amazon web

services aws they have what's called

amazon linux and amazon linux is based

off of

six which is based on upstart which is

based on v in it so it's kind of a long

chain so when you go into an amazon

linux ami it's going to be using

sysvenit initialization strings and all

of that so you've got to know it

amazon linux 2 which only recently

reached production that one now uses

systemd so they're moving that direction

but most people haven't ported their

workload over to it yet so it is very

very common to encounter cisvnit systems

so how do we know if we are using sysv

in it yeah so you know if you've just

been hired on somewhere you're taking

over a server you might not necessarily

know whether it's running sysv in it or

not maybe you've got all your distros of

version numbers memorized and you just

know off bat but if not we can double

check you know i mentioned that the

kernel is going to fire off the

initialization system and that might be

systemd

or it might be cisv in it well that

process of firing off initialization is

the very first thing that it does so we

can just look at process id number one

and that will usually tell us whether

we're running system d or cis via init

right so for example here let me take

this i've got two boxes here this one is

a centos 6 box so

it's running centos 6.7 which is

currently supported so this isn't like

this is old and crusty like you can run

this in a production environment and if

i come in and do a ps aux right

and i'm going to just pipe that into the

head command so i want to see the first

10 lines of my process list when i look

at the top processes here process number

one right here fired off by root if i

come along to the side here i can see

that that's slash sbin slash init if it

just says that slash spin slash init

then odds are you're running sysv in it

now it's not a guarantee and that's

because

some of the os distributions like ubuntu

ubuntu will actually show slash s bin

slash init also

but when you go and look at that binary

if you do an ls dash l slash sbin slash

init when you look at the binary on an

ubuntu system on a newer ubuntu system

it will actually be a sim link

pointing to system d right so be careful

with this one in fact if i go to like my

here's a centos 7 box

let me get into a terminal here and i'll

do the same kind of thing with ps aux

and i'll pipe that into head

and that very first process i can see

right here is pointing to slash user

slash lib systemd systemd so this is

pretty obvious right ubuntu at least in

the newest ones in the 18 releases

they're a little bit trickier so always

be on the lookout for that but that's

probably the easiest way to tell whether

you're running cis via init or not so

what you're looking for is that slash s

bin slash init that tells you what

you're running um and once you've got

that figured out now you know that you

need to administer this system a

slightly different way than you would a

systemd system so

sis v and it starts right after the

kernel right so then what happens all

right so cispean it does have a binary

right so it's slash s bin slash in it

right so our our bios fires up on the

system and it does all the hardware

detection then it reaches out it finds a

hard drive partition it finds the slash

boot partition finds the kernel launches

the kernel the kernel then turns around

and says okay i'm a sisvie init system

and it launches slash spin slash init

and when it does

it then proceeds to start launching a

series of scripts that that init

executable is then just going to run a

series of scripts and it runs them in

serial not in parallel system d does

stuff in parallel it'll do more than one

thing at the same time but with sysv

init it runs one script at a time and

when it finishes then it runs the next

one when that's done then it runs the

next one and so on and it all kicks off

with one script let me show you here on

this system if you go into

etc

you'll find a folder in there

called

rc.d this folder right here now this

folder has changed a lot over the years

originally there were there were

different folders in fact see all these

light blue folders up here yeah rc rc0.d

rc.local rc.assistant all these are like

previous iterations of what uh

uh of what cis vienna or like system

four init or system three in it like

each of them kind of relied on different

folders they're there for backwards

compatibility but now pretty much

everything is tucked away inside of rc.d

and if we go into that folder

and look you'll see a number of

subfolders based on all sorts of

different stuff and each one of these

kind of influences what sysvnit does

when it launches and the main one that

we want to pay attention to is this guy

right here rc dot system it

when cis v and it starts this is the

first thing that it calls it reaches out

to this script and it executes it and

then from there it will proceed to

spawn other things and there's a lot of

things that influence what it's going to

spawn uh which are all kind of tied to

uh parts of what it does but this one's

really the heart and soul of it so

sis

slash spin slash and it runs and it

calls this script to then initialize the

system does that one script then call

other scripts absolutely yep so uh it it

runs and as it's finishing it's then

launching other things based on a

certain criteria right and that criteria

is what's called a run level so notice

all these other folders over here

rc0.d1.d2.d

those are are tied to run levels right

so in in linux you have seven run levels

and each one indicates a different

status of the machine so for example

zero actually means the system is shut

down

now you might wonder to yourself there

like

why would i have a run level for

shutdown well you got to tell the system

to shut down and you might actually have

scripts that are a part of that when

your system is shutting down it needs to

you know basically

perform a few tasks maybe you know stop

rights to the hard disk or so on and

close out what it's doing and those will

be part of the shutdown script you also

have

rc6 here which is reboot and so that's

when your system is rebooting it might

have certain tests they might be the

same they might be totally empty there

might be no tasks tied to reboot and

shutdown but that's what those run

levels are the other run levels

are kind of up to the distro that they

can operate in different modes and

they've they've kind of standardized on

it so most of the distros follow the

same pattern but it's not a guarantee so

always be careful if you're not sure

check your distro's documentation a lot

of them

will document what their run levels are

in their initialization table so there's

a file if i i'm just going to cat slash

etc

init tab that's the initialization table

and when you look in that file most

distros do take the time to document the

run levels and so i can see right here

for centos and this applies to uh centos

and red hat enterprise linux i can see

the seven run levels so see how the zero

is halt halt is when the computer's

turned off uh and then six is reboot one

is single user mode that means that only

one user can be logged into the system

and nobody else can be logged in right

that's actually pretty handy if you're

an administrator sometimes you want to

modify a file but it's always in use

somebody's accessing it or there's a

file share people are accessing it over

the network you need to make sure you

have exclusive access to it so you can

boot in single user mode and you know

you'll have that there's multi-user

without it says without nfs it's

actually multi-user without networking

so it boots up in the full mode but with

no networking services at all and that

way you can ensure nobody over the

network is getting into the system if

you think you've been hacked here's a

great way to boot and shut off your

network and now you can get in and

examine the system and you know you're

not communicating out

you have full multi-user mode which is

run level three and that's where you're

saying look i want to boot up to a

command line interface

but i'm allowing multiple users people

can ssh in or i have more than one

console so a lot of people can be

working in the system that's the run

level that most servers run at

run level four it's actually not used i

i don't think i've seen a distro yet

that makes use of it but it's available

so if you want to create a custom run

level that's usually what run level 4 is

reserved for

run level 5 it's labeled here as x11 but

that means we want to boot a gui we want

to boot a graphical user interface which

is typically the x environment but you

know you might have some alternate user

interface

and that one is multi-user with

networking and a gui so it's kind of

like the

the full shebang right it boots

everything and then you get access to

the system so those are the run levels

and

looking at this slash etc rcd dot d

folders kind of hard to say

all these subfolders rc1.d rc2.d these

contain scripts so when i boot into

multi-user mode command line only

rc dot system it is going to run and

then it's going to look in rc3.d

and it's going to look at what scripts

are in there and look at all these

scripts that i've got i've got where

it's starting the quota daemon and

iptables for the firewall the lvm

monitor for my logical volume manager uh

network manager is starting postfix all

these different things will start when

my system starts these are tied to

various services right postfix uh that's

an email service

ssjshd that's my secure shell daemon

these are all being started that's how

it knows what services to fire up and

it's all controlled by the run level if

i look in uh let's go into rc

5.d

all right so this is if i boot with a

full gui

and i'll see things are a little bit

different there's a lot of overlap right

so i still see sshd right there i still

see post fix right but i'll also see

some things in here that are tied to

x windows let's see i should be seeing

gdm in here somewhere i must be skimming

over it

it's hard when you're under pressure on

the show but somewhere in here is gdm

which starts the graphical device uh

device manager they have to let you get

logged in and all of that so it's

additional things that will get loaded

you may see variations depending on

which run level you boot that's what all

happens right here

then there's one more thing that may not

get loaded but typically doesn't uh back

in rc.d

there's one more file

rc.local okay on really old systems i'm

talking unix all right

it would run rc.systenit

then it would run all the scripts for

the run level and then last it would run

rc.local and that was your way to

override anything else that was run

before you kind of undo or redo stuff

most distros

have the file for backwards

compatibility but don't do anything with

it so it's usually empty or it may not

even be called at all you'd have to

check your run level scripts to see if

they return to it or check rc.cisnet so

like if i take a look at my rc.local

let's see what's in it um yeah it's

pretty much empty in fact it just uh

touches a file to indicate that it's

locked and that's it so

it's not used in this case but it is

there so if you want to make use of it

you can versus if i take a look at

rc.systenit

and take a look at that one there's all

sorts of stuff in here as it goes

through mounting your disks or turning

on se linux right there uh you know

doing all sorts of other stuff this is a

big deal right it's got a lot of work to

do rc.local is empty

do we have to modify the script files

for each uh run level when we install a

service

all right so a good question right so i

showed where sshd was in there right so

my ssh demon is starting automatically

and it's in more than one run level

right so if i install a service like

let's say i want to install a web server

right so i'm going to install apache and

when i install apache i want it to run

on my system boots okay well

if you notice with ssh it was set to

start at run level three and it was set

to start at run level five uh they

skipped run level four because it's not

used and they skipped run level two

because run level two boots with no

networking so why why bother starting a

web server if there's no networking

right

or ssh in that case

i might want a web server to do the same

thing so i want the web server to start

with run level three and run level five

so to do that i mean we could manually

create scripts and put them in there but

what you'll find is that most of this is

handled for us through a handful of

commands all right let me let me just

show you kind of how this works so on my

system i'm going to do a sudo yum

install httpd all right so i'm going to

install apache right

and apparently it's already installed

great

so i've already got it installed now

once it's installed you can go through

and create scripts so if i go back to uh

let's see

rc3.d

and take a look in here i don't recall

seeing http in the list oh it is right

there

k15 httpd so i've already got it

installed and it's already set to start

so i picked a bad example all right

let's do uh maybe we'll do a database

instead is mysql on here let me just

skim the list and make sure i don't make

the same mistake twice all right we'll

do mysql so i'm going to do a sudo yum

install

mysql

and i'll do mysql-server as well and

we'll do a dash-y so it just does it all

right so it's going to install the

database server now when the database

server is installed it's not set to run

automatically or at least it shouldn't

be so i'm going to pull a list back up

here and look

and i do see it actually created a link

right there so it's smart enough to

create the links with what the default

run level should be if we want it on

three and we want it on five it shows up

right there now

what if that's not what we want though i

don't want mysql to run maybe i i only

want it to run when my gui is running

but i don't want it to run when i'm

booting to command line only or you know

something crazy like that right well you

can actually manipulate these pretty

easy without having to write or even

edit the scripts at all right so you

certainly can come in here most of these

are just sim links see how they're light

blue if you uh pull up a long listing

you'll see they're almost all sim linked

to the init.d folder

and in there is where these text files

are kind of being created but we can

manipulate all of this a few different

ways

the easiest way is to use the check

config command chk

co and fig check config what check

config does is it lets you turn services

on or off for different run levels

it defaults to run level three and five

so if i do a check config

and then i say

httpd on

that's going to turn oop i need to sue

do that let me do that command there

there we go that's going to turn on

httpd for run levels 3 and 5. if i boot

at run level 1 or 2 or 4 it's not going

to start but 3 and 5 it will

and that's kind of its default operation

if i say off

then httpd is not going to start in run

level anything right it's going to turn

them all off right but i can go a step

further with this command and i can be

really specific i can say for example

check config httpd

dash dash level

and then specify the levels i want the

default would be three five like that so

run levels three and five that's where

it's going to start maybe i want it to

run well i'll just leave it at 3 and 5.

let's take

mysql

oops

sorry i have to do one more option here

and say on there

so i left off a little on tag so it got

unhappy with me

let's say that i want mysql to run in

run level three and run level five and

run level two right mysql actually

doesn't need the network in a

traditional sense you can use name pipes

to access it so maybe i want that

database up and running even when the

network stack isn't up and going so for

that one i could do a sudo check config

dash dash level 235

mysqld on

and now it's set to go with multiple run

levels so it's kind of a cool one

it's easy to lose track of this stuff

though and we could go and sift through

the scripts and figure out where it's

set to run and where it's set not to so

if i go into uh

slash etc rc.d slash

rc2.d and take a look in there i can

find

wait a minute http is in the list

right it's actually sim linking to all

of these but the script is telling it

only to run at a certain run level and

that run level is left off so if we try

and track it right here through these

sim links

it really doesn't help we don't actually

know if it's going to start or not so a

better way to figure out what's going to

start and what isn't is to use check

config dash dash list

and that's going to list your services

and tell you what run levels they're

going to execute at so if i run that i

get this nice big list right here

and it's got all seven run levels with a

simple on or off next to each one so i

can find http right uh whoops

if i can scroll back up to it i can find

it right here

and the mouse is my biggest challenge

with this stuff

and i can see run levels one

zero one and two it's off run level 3 is

on 4 is off 5 is on 6 is off so just 3

and 5 are on even though i saw an httpd

link in slash etc

rc.d slash rc2.d and if i look at mysql

right here

i can see that it's a little different

in that it's two three and five that are

all turned on okay this is a better way

to see what's being run at each run

level if you just look at the links you

can't tell you'd have to follow the link

and then you have to edit the script or

at least view the script that it links

to and in that script it'll have an if

statement at the beginning that says

like if run level equals and it'll list

the run levels where it's going to start

that's a pain

check config dash list you'll see it all

right there and if you have a huge list

you can always pipe that into the grep

command and once it's in the grip

command uh

you can just say like i'm looking for

mysql and then you just get that one

line that makes life a little bit easier

there so pretty easy to verify uh we

could also just do it manually like

maybe maybe i'm not so concerned about

when the system starts i just need to

use my sql for a little while right now

okay so for that you'll use the service

command and we can just come in and say

sudo service

and then start and whatever the services

we want to start if i want to start

httpd there we go uh oops i got it

backwards uh service

httpd start

there we go

sorry you got to say the service name

first and start second i got it

backwards so sudo service start httpd

and now it starts up and i can do the

same thing with like mysqld which it'll

probably be unhappy that i haven't

secured it yet but we'll do it anyway

we can start that one up yeah it's

warning me that i need to run the

security startup script but uh but there

it goes so now those two services are

running

but

let's say i'm at run level five right

now and those are up and running and i

reboot my system if those aren't set to

start with run level five they won't be

started when i reboot so using the

service command like this is not

permanent it's really just kind of a

temporary thing just to get uh get the

service up and running and so on and you

can do a service

status

for any service

and take a look at like hdbd

and

oops

backwards again darn it service httpd

status there we go

and if we do that when i'll get it right

eventually we can see if the service is

actually actively running right now you

may find where it started with the

script and then something went wrong and

it closed and so even though rc.d is

telling it yeah you need to start this

at boot it may not be booted so this is

a good way to quick check that

and see and i'll do the same thing with

mysqld and i can verify right there

they're both up and running and happy

down how does this v in it know which

run level to boot to

uh you know i glanced over it earlier so

we kind of have some defaults right

where we're saying boot to the graphical

user interface or boot to the command

line uh and your your os your

distribution will typically default to

one or the other that comes from the

slash etc init tab folder that our file

that i opened earlier uh if i cat that

one slash etc

init tab all right so this is the file i

brought up on screen earlier to show the

run levels well there's one tiny little

line at the very end that i didn't

mention this this guy right here that

says id colon

5 colon init default colon all right so

this is telling the system what run

level to boot to by default so mine is

at the default to run level 5 which is a

full graphical user interface if i'm

setting up a server i probably don't

want the gui so i'll tell it to boot to

the command line to boot to multi-user

mode which is run level three so to do

that i would just edit this file and

change that five to a three and then

when you reboot it's gonna boot to the

command line and when you're ready you

can always change it back to a five

again and it'll boot up and and that'll

be kind of the the run level that it

goes to now that's if i want to make a

permanent change

let's say it's just a temporary thing

right maybe i'm having some kind of

crazy issues with x windows and

i just want to

to dump out and get rid of all of x

windows and then come back into x

windows but not

kind of not have to restart all my

services or whatever so we can actually

change the run level whenever we want

while the system is running now it can

have disastrous effects so we need to be

a little bit careful with it you know

all these scripts are going to execute

and do stuff

but if you ever want to change your run

level just temporarily there's a command

you can run which is init

if you run init

followed by a number let me pull out the

man file for that so if you just run

init followed by a number it will

immediately change your system

into whatever that other

init level happens to be now if you're

going to do this be really careful of

other users in the system you can you

can really mess with them especially if

you're about to drop a single user mode

so you might want to use the who command

to see who's logged in and how they're

accessing the system to say all right

here's people i need to contact before i

change this run level but assuming

you're the only one in here let me take

this system and i'm going to go to

multi-user mode right so i'm going to

say sudo init three i want to drop to

run level three and when i run that

my system drops to the itsy-weensy tiny

little console

and so i get to see that and we get out

the magnifying glass yeah yeah we'd have

to use it here uh so we lose all the you

know the cool

um

uh

screen enhancements and so on and now

we're just in straight command line mode

and there we are right so now i'm kind

of operating in this

this different world x windows has been

completely unloaded and it's executed

the scripts found in that rc3.d folder

to move me into this kind of mode and i

can switch back if i want by saying init

well i gotta sue do that sudo in it

and then move back into run level five

and when i run that

there we go i see that it takes me back

to my centos login screen

and i can get logged back in

and now i'm back into the gui so we

quickly and easily change our run level

like that now uh

most of the time we're going to do it

just like i did right there it's going

to happen fast

there may be times where you want to

schedule changing the run level and the

the official recommended way to do that

is use cron you can set up a cron job to

change the run level at a certain time

if you want

that's one technique you can use there

used to be a command called telling

and what telonite would do

is it would let you set a time delay for

when the run level would change and many

systems don't have the command anymore

and let's see if i do but you have to

spell it right if you want to find it i

do have it okay so there it is uh

telling it it lets you change the system

run level so just like the init command

but the difference is it takes some

options the init command really didn't

take any options this one does and one

of the options is dash t which is in

here somewhere well i didn't see it but

it does exist uh where you can come in

you can say

telling it

and then what number you want to go to

so i'll say i'm going to run to jump to

run level three and you can say dash t

followed by a time and you know maybe i

want to wait 30 seconds before i do it

and so i can specify that i would need

to suit it that command

and then i'll do that dash t30 and run

it okay so that's going to start that

timer well notice how mine just totally

ignored the timer that's because the

telenik command it's an old command it's

not supported by most a lot of districts

don't even have it and the ones that do

don't fully support it so it's kind of

the old way so instead if you want to

schedule it you're a little better off

by jumping in and doing uh a cron job to

run in it at a certain time fantastic

information managing services with cis v

in it great episode as usual thank you

don and before we move on i think you

have something else you'd like to say

yeah you know i've mentioned before that

cisvnit is really old and the systemd is

really new and shiny and i personally

like system d

but i think the sysv in it is still

really really important you are going to

find many many servers that are out

there that rely upon it and if you start

jumping into other operating systems

like freebsd and open bsd and those guys

they still use cisv in it too systemd

hasn't made the leap over those sides

yet so learning how this works is a

skill that you'll be able to apply many

different places definitely definitely a

valuable one to have great advice thank

you don here's some more great