CompTIA Security+ Full Course: Security Network Monitoring & SIEMs

foreign

[Music]

monitoring has been around for a lot of

time probably ever since the first

networks were invented just like with

any system just like with any electronic

device we tend to want to be able to

monitor if everything is going okay we

want to receive warnings we want to be

alerted when something goes wrong when

when something fails and this type of

monitoring is tremendously useful

especially in larger networks over time

this monitoring has extended to security

monitoring as well so we're not just

concerned about how is the network doing

if it's working well if you don't have

any failed devices but we're also

starting to look at the network traffic

how is the network utilized who uses it

who attempts to access it what type of

traffic are they generating and if we

try to gather all this type of

information we try to make sense of it

we try to correlate it with a smart

enough device we might be able to detect

attempts at intrusion or attacks that

are about to happen or that have

happened in the past or proves that

we've been compromised or somebody in

the network has been infected and all

that information is in there if you know

where to look for and also if you have

the right tools to look for it

in general the term internal detection

refers to a system that is able to

monitor whatever can be observed in a

network and in most cases we're talking

about two things that can be observed

first of all we have Network traffic

and then we have application events or

logs generated by the operating systems

by the applications running on those

os's and so on so coming back here to

our Network Focus we're talking about

intrusion detection at our Network level

we're going to call this one a network

based intrusion detection system or nids

and we have many commercial Solutions as

well as open source ones that are able

to perform this type of network-based

intrusion detection of course all the

major security vendors are doing it in

many examples you're going to see the

IDS functionality built into the

functionality of a larger firewall or a

larger UTM device especially for major

vendors out there but you also have

Solutions in the in the open source area

such as sort suricata or Zeke or bro

they're all available some of them also

have commercial versions as well but

they also provide you with three

versions that you can freely install and

try and run in your own environment now

the way these uh intrusion detection

systems work by definition is that they

rely on a database of signatures and

those signatures are basically just a

way to describe how a specific traffic

pattern is supposed to look like in

order to detect a specific type of

attack or attempt at an intrusion so we

might be looking at a sequence of

packets so that looks in a certain way

we might be looking at a specific type

of packet that doesn't play by the

normal protocol rules that it belongs to

or a specific type of payload or simply

just a a signature a byte sequence that

can be found in the in the packet

payload that indicates the fact that the

payload is malicious and this behavior

is very similar to what you're seeing in

antivirus scanning or anti-malware

scanning we're simply looking for a

sequence of bytes that indicates that

well if we find the sequence of bytes in

a specific executable file it means that

the file is infected with that specific

virus that the the sequence belongs to

now when intrusion detection again we

we're kind of doing the same thing right

we're looking for patterns but we're not

just scanning individual packets

sometimes we need to collect more

packets in a sequence in order to

determine if the behavior of the client

that is generating those packets is

abnormal and if it's abnormal what does

it indicate an attack pattern or not the

long story short intrusion detection

systems are strongly dependent on a

database of signatures now more advanced

nitrogen detection systems could also

correlate this network information with

log information so we're seeing

something fishy in the network by

looking at the network traffic let's

check the application logs that the

traffic is is going towards for example

let's see how that application reacts

and if we if we can see some abnormal

logs being generated by the app as well

now correlating that information the

traffic analogs might tell us more about

the actual attack or might increase the

confidence of the fact that we really

have identified a valid attack signature

not all solutions are able to do this of

course also a very important distinction

for intrusion detection system with an

emphasis on detection is the fact that

these systems are never able to block

the malicious traffic once they identify

it it's just like the name says just

detection it's not prevention all right

so we're not stopping the traffic we

might be able to see an attack signature

we might be able to raise some alerts

generate some syslogs but we're not

going to be able to block that specific

type of traffic one positive side for

this is that well if the device is not

inside of the traffic path then the

attacker might not even be able to

detect it

so most likely the the IDS is going to

work with a copy of the traffic just to

analyze it but it's not going to be able

to stop the malicious traffic and the

attacker is not going to be able to

detect the IDS device and might not even

be able to compromise it if if they

intend to in most situations the IDS

device doesn't even have a valid IP

address within the network that they're

monitoring so it cannot be addressed it

cannot be compromised by communicating

with it directly alright so since we

mentioned the fact that an IDs works

with just a copy of the traffic let's

see how can we generate that copy of

traffic right they're not within the

traffic path so we need to make a copy

of the traffic and just send it in a

separate channel on a separate channel

to the IDS device for analysis now my

way of doing this is by enabling Port

mirroring or span on in Cisco speak this

is switchboard analyzer just a

functionality on layer 2 or layer 3

switches that allow us to configure the

switch and we're basically telling it

well whatever traffic you're seeing on

ports let's say one two and three make a

copy of that traffic and forward it out

of port number eight and of course we're

assuming that on Board number eight

there's an IDs device connected right

there so we're basically telling the

switch to make a copy of all the

interesting traffic and send it towards

the IDS and of course you might be

thinking here well what if the switch is

overloaded what if there's more traffic

generated on those ports than the uh the

mirror report can actually support well

that's true it might happen so in in

cases when the switch is overloaded and

there's too much traffic in the network

packets might be dropped and also frames

with errors might not be forwarded to

the uh to the merits board either so we

might not be able to see 100 of all the

traffic but in most cases it's going to

be enough and it's also one of the

features that basically doesn't require

you to install anything else in the

network it's just a functionality just a

configuration effort just a couple of

commands on a switch another method for

duplicating traffic is by using a

passive order and active it's basically

layer 1 device called the tap a test

access port it's nothing else than a

kind of like a t connector where the

main cable goes from one end to the next

and there's a third cable that actually

receives a copy of the entire traffic

going through that that segment of cable

the device is not a smart one so it's

it's not like a switch it's not going to

look at the destination frames and

forward uh entire packets it's simply

going to duplicate the electrical or the

optical signals that it sees on the wire

and it's going to make a complete and

identical copy of those signals onto the

third connection which of course is uh

is ideally connected to the IDS device

now this type of approach is again

completely undetectable

span is not detectable either right and

also copies entire frames regardless if

those frames contain errors or not as we

said with Port mirroring while the

frames need to be correct in order to be

copied well with a tap the tap doesn't

care it's basically just a signal

repeater and we can do this for both

copper cables and so electrical signals

as well as fiber optic so Optical

signals the tap will not care we'll just

blindly copy all the signals that it

receives and finally the third method

for monitoring traffic is by having the

IDS device in the traffic path

but acting as a transparent device again

without an IP address or basically

becoming a layer 2 device that is part

of the same VLAN that they're they're

bridging but they cannot be addressed on

the network they cannot be detected on

the network and they if it's a true IDs

device then it's not going to be able to

block the actual traffic that goes

through it now having the device placed

inside of the traffic path opens us to

the possibility of actually blocking the

traffic and that's going to be a

different type of solution called

intrusion prevention system and we'll

get there in just a moment there's one

more type of intrusion detection device

or solution and that is a software

solution that can be installed directly

on the workstations so I'm not talking

about a box that listens to network

traffic on an entire segment but we're

talking here about a software solution

basically a program that runs on your

endpoint machine on your host machine be

it a laptop or a desktop now this one is

called host based instruction detection

because it runs on the host and it does

have pretty much the same benefit it's

or the same abilities as a network-based

internet detection so it's able to look

at the network traffic going in and out

of your network interface it's able to

look at the logs generated by the

applications on your system but since

they are running as an application on

your system they can become even smarter

because they might have access now to

the actual process table they might be

looking at the kernel you might be able

to look at the memory to see what

processes are running when did they

execute who executed them with what

privileges and they can also openly look

at encrypted traffic so if you are

communicating over SSL with a website

well a network-based internet connection

might not be able to understand anything

that's going back and forth because it's

encrypted but your host based intrusion

detection

is located at the end of that encrypted

tunnel so it is able to see that

unencrypted traffic before it even

enters the encrypted tunnel and right

after it leaves the encrypted tunnel so

it's able to actually watch the entire

traffic flow in an unencrypted form and

again since we have pretty much full

permissions on on the monitored host in

order to be able to properly monitor the

you know the process table and the

network connections and the network

traffic we could also have a look at the

files on the disk

why would you do that well that's

because monitoring the Integrity of the

files on the desk especially the

Integrity of the operating system files

and being able to detect when that

Integrity fails when a system file is

being replaced with a malicious one when

a system file is is becoming encrypted

or it is replaced with a completely

different version that might be an

indication of compromise that might be

an indication of the fact that you have

been infected with malware so Solutions

or functionality additional to host

based internet detection that monitor

files on your system especially

operating system files these are called

file Integrity Monitoring Solutions and

remember that we said that when we place

the intrusion detection device in the

traffic path

device actually becomes able to also

block the traffic that goes through it

which can make it an intrusion

prevention system right so detection

just alerts just generate alerts or

events intrusion prevention is about

actually actually taking action or

acting upon the detected intrusion so

what can a such a device actually do

whenever they're they're seeing

something fishy going on inside of the

network well they could do something as

simple as simply sending a TCP reset

packet to the originator of the

malicious connection they could also

have some more advanced functionality

especially if they if it's the same

device that acts as a firewall they

might be dynamically able to generate a

firewall rule to block similar traffic

like the one that was just detected as

being part of an attempt for for an

attack or for a compromise we could be

choosing if we're detecting something

that looks like a denial of service

attack we'd be choosing to limit the

amount of bandwidth that is allocated to

that specific type of traffic kind of

like policing that we're doing in into

our inequality of service in any case

any type of action that the IPS device

can take against the malicious traffic

we're going to call it active response

and depending on how complex the device

is and how powerful the device is you

might actually choose to look not just

at simple IPS or IDs signatures but also

look for malware signatures yeah that's

that's going to require you to you know

to decode encrypted traffic it's going

to require you to identify potential

protocols that might be carrying files

gather all those related packets that

belong to the same TCP Stream So the

same flow assemble them into an

executable file store that in memory or

attempt to scan it with an antivirus

engine and then determine if that flow

was actually malicious or not now this

requires a lot of processing power this

is going to create some sort of delay in

the networks of the users they're going

to see their their download uh unable to

finish or the application responding

slowly until the firewall the UTM device

or the nutrition prevention system is

actually able to scan those files

against malware signatures on a lighter

approach we could also just be looking

at URLs looking for malicious domains or

domains that show associated with

malware or with the command and control

servers we might be looking at URLs in

order to categorize those URLs and

figure out the reputation of that URL

and decide whether we want the

communication to that specific website

to proceed or not so regardless if the

device is an IPS or an IDs the detection

methods are pretty much the same now the

difference is just in what the device is

actually doing is it only alerting or is

it actually taking an active response

approach to the traffic but the

detection part is pretty much the same

right and when talking about detection

we are going to start with the basic

type of detection that is where we're

just looking for signatures in the

database which of course means that we

need to have an up-to-date database for

the device to be able to detect the

latest and the greatest attack now this

is basically one of the reasons why

people choose to pay for commercial

Solutions because databases maintained

by a dedicated software or security

vendor that deals with intrusion

prevention those databases are going to

be much more often updated and kept up

to date in order to mirror as best as

possible the database of all the known

attack patterns ever detected in the

world now with open source Solutions

you're gonna you're still going to have

a pretty good level of protection but

you might not be able to detect an

attack that was just identified

six hours ago nevertheless and

regardless how up-to-date your database

is you're still limited by the attack

patterns listed in that database if an

attack emerges and doesn't match

anything in your database it's still

going to go through

which leads us to a different approach

and that is a behavioral approach so

instead of looking at specific streams

of bytes specific headers specific

sequences of packets let's look at the

overall behavior of an application or of

a protocol

does it look like it's doing what's

supposed to do is it generating more

packets than we're used to seeing is it

uh generating more traffic is it

generating an abnormal amount of control

information as opposed to a real

transfer data and we call this

behavioral monitoring now in order for

Behavioral monitoring to work we need to

have something to compare that behavior

too and say well if it goes outside of

the known ranges

then it looks like something's fishy

well that known range is supposed to be

your Baseline so such a device or such a

system is supposed to be trained first

you're supposed to just leave it inside

of the network for let's say a week or

two just figure let it figure out how

does a normal Monday morning look like

in your network when everybody comes

into work and they start logging in and

start updating their their uh their

machines and perhaps even their mobile

phones only on the company Wi-Fi but

nevertheless you have to leave that

interim prevention solution learn what

does your normal traffic look like when

people start accessing internal

applications where people start

accessing internet destinations when

people start communicating sharing files

between each other when when backups

start to happen at midnight perhaps

right you have to let it learn so that

in a couple of weeks when something goes

outside of the known range where an

application behaves the way it did not

behave in the first training weeks then

it's going to be able to raise an alarm

and perhaps indicate the fact that the

application has been compromised or that

somebody is using it in order to elevate

their privileges or just compromise your

network and as you can probably guess

this is one area where machine learning

is going to provide you a lot of benefit

given that you take the time and efforts

to educate to teach the machine Learning

System what does your normal Baseline

look like now of course regardless how

complex or how well tuned Your solution

is going to be there will be false

positives and there will be false

negatives which is why I always tell

tell students there's a old saying that

I heard from someone in Cisco a long

long time ago and they said that IPS

Without Eyes

is useless so IPS without human eyes is

useless there there's always going to be

the need to have a human being right

there evaluating and analyzing whether

the alerts generated by the intrusion

prevention or detection system are valid

or not does it need more fine-tuning or

do we need to raise an alarm so what

devices can we actually find that

implement this type of advanced

functionality be it detection or

prevention well unfortunately this is

the uh the place where we're slowly

stepping into the marketing area that's

because the devices that we're going to

be listing here are not completely

different devices but over time

different naming conventions have

emerged different marketing names have

been invented to make them sound cool to

make them sound different from what the

other vendors were doing so we're going

to start with the next generation of

firewall and we will have this type of

Next Generation

for about 12 or 15 years already I've

been hearing the Next Generation term in

in uh in I.T security for so long that

I'm starting to wonder

are we still next generation are we have

we skipped the generation are we now in

the next next generation or where does

it stop where does it end where where

does the Next Generation begin right now

unfortunately marketing people don't

really ask themselves these questions so

we're kind of stuck with this

terminology for now and we're gonna keep

calling you next Generation until I

don't know when but regardless a Next

Generation firewall is basically just a

layer 7 firewall that's an application

layer firewall which is able to look at

the application layer payload so we're

actually seeing the data being sent

we're not just looking at the packet

headers and it also has some sort of

detection or prevention system built in

okay so we have an IPS or an IDs built

in which leads us back to the discussion

that we had before so we have an

application layer firewall which can be

enriched with additional functionality

now that we have access to the actual

application payload well why why not

look for intrusion signatures why not

look for malware signatures why not look

for spam signatures right so depending

on how complex the device is if it at

least has IPS functionality built in

we're going to call it the Next

Generation firewall and here's the funny

part if the Next Generation firewall has

a bunch of other additional features on

top of the IPS functionality such as

malware scanning antivirus scanning

perhaps looking at the files and being

able to implement some data loss

prevention policies uh it's able to look

at the URLs and categorize them and

analyze the reputation of the web pages

and pretty much everything that we could

possibly think of that we could be doing

just by looking at the application data

then we're going to call this a unified

threat management device a UTM device

again I don't think I need to repeat

this but the more complex the device

becomes the more stuff it needs to do in

order to decide weather to allow a

packet or not the more resources the

more CPU intensive is going to be the

more memory is going to require and the

more delay is going to introduce in the

network so keep this in mind even though

it kind of sounds cool right to have all

that security functionality in a single

box

which by the way you try to make sure

it's not a single box of failure single

point of failure all right

even though it sounds cool to have all

this functionality in one place

it's going to hit your performance

pretty badly right so keep this in mind

don't just enable everything blindly

because the end users the applications

and well God forbid the your your

customers you're paying customers

they're going to feel the the effects of

your of your awesome UTM device and

their application experience is going to

suffer now a special type of network

monitoring device can also be considered

a web application firewall we've briefly

mentioned about where allocations

firewalls in in a previous video and we

said that uh a web application firewall

is just a dedicated firewall that is

specifically trained and educated to

look at attack signatures aimed at web

applications so we're looking for things

such as cross-site scripting we're

looking for

um you know directory traversals we're

looking at SQL injection attacks we're

looking at pretty much anything that

could be performed by malicious user

that is trying to exploit a input

validation flaw in a web application so

it's still an application layer firewall

it still looks at the the application

layer payload it's just that it's a bit

more let's say picky about what type of

traffic is it going to analyze so it's

only going to look at web traffic and

it's only going to look for uh web

attacks web application attacks it's

mostly going to rely on signatures

that's because we cannot really do much

when it comes to requests coming in from

from our clients behavioral analysis

doesn't really play well here because

most attacks especially where vacation

attacks are just one single request one

single query with a malicious payload so

so in many situations it's gonna be

either black or white right we we're

detecting an attempt at an intrusion

we're detecting an attack in that

require West or not it's pretty much not

going to be much of a gray area with

verification firewalls and you could

deploy a WAFF as a separate device it

could be a physical box it could be a

virtual machine it could be a

functionality within a UTM device again

all in one wonders but it can also be a

part of the web server itself so we have

plugins that install alongside the

actual web server that is hosting the

web application such as plugins for the

Apache web server for the IIs web server

on on Windows server or for nginx so

we're installing these plugins right

there and their purpose is to scan the

traffic that's coming in from the

clients before allowing that request to

be processed by the web server having

something such as a plugin that runs

alongside the web server on the same

machine on the same box opens us to the

risk of either having that machine

compromised by an attacker who this time

doesn't Target the web application but

targets the scanning engine and can

intentionally cause for example a denial

of service give it so much traffic to

analyze that the web server running on

the same machine is unable to actually

respond to valid requests so there you

have it that's the now service attack

now when it comes to actually monitoring

the network traffic we said that a

solution would be to just simply mirror

all the traffic and then look for

specific deck patterns inside of that

traffic now that might not be always

feasible because the amount of traffic

entering a data center or the server

form that hosts an application might be

huge right so in some situations we

might not be able to analyze the exact

amount of traffic that goes in but we

might be able to generate a summary of

that traffic and then Analyze That

summary for intrusion attempts now this

traffic summary is sometimes found under

the terminology of netflow or S flow or

J flow which is basically just a

technology implemented by various

vendors out there in which instead of

creating an exact copy of the traffic

we're simply summarizing that traffic

and then reporting that summary back to

some analysis software so we're only

telling it what type of sources what

type of destinations have communicated

how many bytes were used what type of

protocols has been have been used uh

what type of flags have been set in that

specific type of traffic but we don't

put the burden of sending the entire

actual traffic in the entire payload to

that analysis software now this also

means that we're losing application

layer visibility all right since we're

just summarizing the type of traffic

we're only describing the metadata about

the traffic we're losing everything that

pertains to the application layer but

we're gaining a lot of performance and

we can also store this summary

information long term for further

analysis somewhere along the line in the

future sometimes looking at traffic it's

simply not feasible maybe we cannot grab

all the traffic that's running through

the network maybe we don't have network

devices smart enough to generate those

so those summaries those those flow

reports for us so another solution would

be to Simply have a software monitoring

service Ocean or so-called a network

performance monitor that queries

periodically your networking devices

queries your your routers your switches

uh your wireless LAN controllers your

firewalls perhaps about the status of

their physical resources status of their

interfaces how much traffic is going

through their interfaces I was the CPU

load what's the memory usage what's the

structure of the routing table how does

the r table look like how is the dhtp

traffic looking like right so any type

of monitoring information that can be

extracted out of these networking

devices which in turn can be correlated

in order to figure out if we can see

some anomalies in there one such

solution is for example solarwinds npm

Network performance monitor which is a

dedicated solution for monitoring not

just networking devices but also servers

and virtual machines about their their

their health right how are their network

interfaces looking like how much load is

there on their Hardware resource or

their Hardware components are they

generating any alerts do we have failed

interfaces do we have failed processes

we have something that's uh failed links

are we detecting errors or overloaded

devices stuff like that

now this type of performance monitoring

can be done over a variety of protocols

in most cases the SNMP protocol is going

to be used because it allows us to

report a lot of the hardware counters

and a lot of the interesting information

that we want to gather in store long

term also we might be using wmi such as

Windows management instrumentation and a

couple other protocols as well and of

course we could enrich this collection

by collecting logs from the monitored

devices and appliances as well and we

could be collecting those logs over

syslog so we need to configure the

device to actually send those syslogs

messages or at least a copy of them to

the monitoring device or we could rely

on an agent an additional piece of

software installed on the server on the

virtual machine that periodically

reports back to us everything of

interest regarding the that specific

host when talking about dedicated

software design specifically designed to

analyze a lot of information coming from

the network beat Network traffic Network

summaries such as netflow logs and any

kind of application data that solution

is most likely going to be called a seam

a security information and event

management now the keyword and a

definition of seam is correlation that

is it's not just a place where you just

dump all that information in a huge

database it's a place that as you dump

that information is going to look for

patterns inside of it it's going to try

to correlate Network traffic with logs

or application data with with netflow

data in order to figure out if some

anomalous behavior is detected in your

network so same solution and by the way

these are pretty expensive Solutions out

there are never designed to be just log

storage right they're engines smart

engines based on machine learning that

aim to detect patterns of intrusion by

analyzing and correlating information

found in multiple log files and what's

interesting about the implementation of

seams is that they're supposed to

collect logs from your network devices

from your security devices even from

your workstations and your mobile

devices perhaps and they're able to

understand and correlate all that

information and normalize all that

information even if it comes from tens

or hundreds of vendors or thousands of

devices

and they're able to normalize that

information and make it look the same so

that in the end

it can look for patterns inside of it

and it also allows you to perform

queries in a language quite similar to a

regular SQL language and query all that

information regardless of the fact that

it actually came from or tens of

hundreds of different vendors and since

a seam without machine learning

functionality is not a very useful theme

we could use that machine learning

features to look at user Behavior as

well because in the end we're trying not

to detect just you know attack patterns

we're also trying to identify who is

conducting them and a great risk comes

from Insider threats so if we are able

to monitor what our users are doing

we're not talking here about just

watching what websites they're they're

visiting or taking frequent screenshots

of their of their workstations now we're

not doing that but we're looking at the

behavior that they're exhibiting

whenever they are interacting with

specific applications and if the scene

has such an ability we call that ability

user and entity Behavior Analysis don't

think that we're only uh performing here

a Witch Hunt against uh Insider threats

think about the fact that we might be

able to detect abnormal behavior because

a user account has been compromised by a

hacker and that hacker is now acting on

behalf of that user the user might have

nothing to do with that abnormal

behavior might not even know about it

might not even be logged in at that

specific point in time but the attacker

might be acting on behalf of that user

if we're able to detect that abnormal

behavior we might be able to detect the

attack going on right then and stepping

just a bit into the realm of Science

Fiction here I know that some vendors

will say no this is not science fiction

where we're selling this we've had huge

success with this well yes and no I'm

gonna keep being a bit skeptical as to

how efficient this approach is what I'm

talking here about is sentiment analysis

or emotion AI that is analyzing user

behavior in what contents the user is

actually creating as in blog posts

social media posting

we're not talking here about actual you

know analyzing the contents of emails

and and chats because that might you

know step into the Privacy area which we

might not want to do that but by

analyzing publicly available information

generated by those users we might be

able to detect disgruntled employees we

might be able to detect uh unsatisfied

clients that might create some bad

reputation for the company perhaps even

before they become so upset as to take

action or malicious action against our

company again take this with a brain of

salt and don't just think that if it if

it sounds awesome on paper it has to be

awesome in real life if it sounds too

good to be true then it probably is too

good to be true

and finally the last term here that I

wanted you to know about is soar

security orchestration Automation and

response that's a mouthful I know it's

usually functionality built into SIM

Solutions or it can be just a standalone

solution but it basically uh it tries to

address is the problem of too much

information that is being overwhelmed by

too many alerts too many security events

too many security incidents too many

incidents that we need to determine if

there's security related or not

basically the hell of any I.T Department

that deals solely with monitoring the

network and the applications and the

idea behind this is that a solar

solution is supposed to use some machine

learning techniques in order to not just

to figure out which anomalous events are

occurring in the network but by

analyzing those anomalous events it is

able to take some action against them so

it could at the point uh determine if an

attack is going on even if it happened

in the middle of the night and take

action immediately by blocking some

ports by creating an access list by

disabling temporary disabling some user

accounts that might have been

compromised so that security

orchestration Automation and response

just be sure everybody is clear on this

especially for the exam where does the

theme get this information from where

first of all it's going to get it from

logs right syslogs that's going to be

the main source of information how do

you collect logs well you don't really

collect them you expect those devices to

send those to you so those devices need

to be configured uh be it networking

devices there might be servers there

might be virtual machines whatever type

of device you have just configure them

to send your logs to a secondary

destination if the seam is not the

primary one just make sure they send a

copy of those syslogs to the same device

as well next the scene can also collect

data by installing agents on specific

systems now of course we might not be

able to install agents on let's say

routers or switches apart from some

recent devices that are running Docker

containers perhaps but in most cases CM

agents are designed to be installed on

Windows and Linux systems then they're

running as background processes that

periodically scanned the system and

report back to the seam uh the logs

generated by the operating system the

running applications the long generated

by the applications actually running on

that host depending on how the agent is

configured the built-in listeners or

collectors that you're seeing here on

the slide refers to the fact that the

seam is pre-configured or has plugins

that allow it to understand what

different vendors are reporting back to

it so it's going to have different

plugins to understand Lots coming in

from you know Cisco devices HP devices

Dell VMware whatever vendor it is it

needs some sort of a plugin to

understand that specific log format and

more than one than that it needs to

understand the contents the payload of

what the log is saying SNMP traps again

most monitoring information is going to

come in through an SNMP query or as an

SNMP trap generated by the device back

to the seam and also netflow netflow or

different variants implemented by

different vendors are basically just

summaries of the traffic flows detected

over a certain period of time collected

and then sent over to the scene device

in order for that traffic summary to be

analyzed finally the scene can also

capture raw packet data if it has

dedicated sensors that are able to

generate a copy of the traffic and send

it back to the seam or we can even have

sensors installed inside of network that

are monitoring real traffic and they're

only telling back to the seam or

reporting back to the sim a summary of

that traffic this is very useful when

your devices don't have enough reporting

on monitoring capabilities to report

back to the Sim device instead you need

to install some specific sensors that

look at the traffic and then tell the

seam the necessary information that it

needs to perform those correlations

sometimes a sensor such as this one YB

and IPS or an IDs device even log

normalization is a feature built into

most Sim Solutions out there and

normalization is required and it's a

very important feature because the seam

is designed to collect information from

hundreds of vendors and thousands of

different appliances each of them

running different operating systems on

different versions and they're all

building syslogs and SNP traps in

different formats some some are

reporting them as a text some are

generating logs in binary format some

logs are in Json format some are in XML

format or or CSV format depending on how

the vendor actually designed its logging

and monitoring abilities we might even

find differences as to how the logs are

actually encoded some of them are might

be using UTF some of them might be using

some Regional encoding we might even run

into some issues due to the fact that

the new line character is represented

differently between Windows and Linux

systems and that also might be reflected

in the payload included in the logs that

we're receiving as as part of the

monitoring process not to mention the

fact that the SNMP mips basically the

the database schemas that each vendor is

using for their own software Solutions

or Hardware Appliances these are

completely different not just among

vendors but also among different

products from the same vendor so in

order to have all this bunch of

information collected in some

centralized location and to be able to

query all this information and to be

able to approach it in a in a consistent

manner we need normalization that is

taking all this information coming from

so many vendors in so many formats and

making that information look exactly the

same so that it can be stored in a

single database that can be queried at

once regardless of the source of that

information so what are we using to

normalize all this information coming

from all these vendors well you guessed

it we're gonna need some plugins some of

these plugins come from this Sim vendor

itself so they're going to be

pre-packaged with vendors vendor plugins

from for major vendors out there some of

these plugins are going to come from the

actual vendors so if a smaller vendor

create hate them let's say smaller

firewalls at some point and they want to

be able to integrate with the

large-scale seam Solutions they're going

to provide you with a plugin for their

own environment as well and another type

of normalization that is really really

important is timestamp normalization

don't forget that we're looking for

anomalies in a network traffic and in

network events and if we don't have

timestamp normalization if we don't make

sure that all the events that we're

looking at are actually stored with

their right timestamp at their right

moment in time when they actually

happened we have no chance of detecting

anomalies in the network so we might

have devices that are that have a badly

configured clock we might have devices

that have been configured for different

time zones we might have devices that

display time or or timestamp those time

values in the in their logs in in one

format versus another format some of

them might be using 24 hour some of them

might be using 12 hours some of them

might include the daylight savings time

some of them might be using a UTC or

Unix ebook time it's up to the vendor so

normalizing these timestamps is also a

very very important topic here that

needs to be taken care of by the same

solution before that event indicated by

that specific timestamp is stored in the

database alongside with the others now

the way a Sim solution can look for

anomalies in that huge database that we

just talked about well it could be done

in a number of ways we could just rely

on simple if then else matches so we're

looking for you know specific events

specific types of logs being generated

in a specific time range perhaps this

type of approach is the fastest one

because it it's basically boils down to

a simple query in that huge database

stored by the CM and Appliance

unfortunately if there are unknown

threats if there are attacks that we

know nothing about that we don't have a

signature for them we don't know what to

look for we're not going to be able to

detect them kind of makes sense right so

another approach would be heuristic rule

matching this is a type of rule matching

where we're not exactly looking for an

exact match

for the specific type of event but we're

looking for something that it's pretty

close to it all right so this type of

approach

relies on a more permissive set of rules

so if it doesn't 100 match or rule let's

say if we have some events that are

pretty close to it and match it like

let's say 80 or 90 percent

now this also requires you to fine-tune

your rule set so if at some point by

doing heuristic rule matching you're

detecting some anomalies but you don't

have a rule that matches that anomaly

100 well you'd better create it right

you'd better fine tune your rule set and

add some more rules or tweak the

existing ones to match that newly

detected anomaly and just to recap this

year in behavioral analysis implemented

in a CM relies on the fact that you need

to build a baseline you need to tell the

Sim how does your normal look like how

does your normal traffic look like how

does your normal logs generated by all

the devices and all the applications in

your network looks like so that in turn

can be used as a starting point in order

to detect potential well mismatches that

might indicate attacks or attempts at

compromising your network now of course

this is going to create a lot of false

positives so you might into a situation

where an alert is being raised because

in an application starts generating some

huge backups because some admin has

modified the backup policy now the same

device sees a lot of traffic in there

racism alert raises everyone from their

sleep at 3am in the morning and saying

that oh my God this looks like a data

exfiltration attempt somebody's is

dumping all the data from our database

and then an admin has to come in and

intervene and say my dear seem what's

happening in there what you're seeing is

just a full backup happening at 3am in

the morning it's okay right don't freak

out about it okay so it does require

human intervention for fine tuning these

rules

on the other hand we have anomaly

analysis and this is by definition a

type of analysis that is performed

whenever we're comparing observed

Behavior with known standard Behavior

especially when we're comparing what

we're seeing as part of a protocol's

behavior with what this theme device

knows that the protocol is supposed to

behave according to its RFC according to

its definition finally with Trend

analysis we're going to be looking at

historic data and try to extrapolate it

for example if we see that the backups

are increasing every single week because

more data and more data is generated the

same device might be able to generate a

pattern so that if we see five gigabytes

and backup this week and eight gigabytes

of backups next week when it is going to

see 12 gigabytes two weeks from now it's

not going to raise an alert because it

expected the backup volume to increase

by that amount but I don't need to tell

you that not everything can be safely

predicted this way finally after all

that advanced correlation and machine

learning and AI features the seams

actually can be used as a database for

event storage and they can be queried by

human users by admins if you know what

to look for perhaps you just need to

investigate some event perhaps you need

to perform some some forensic analysis

so those databases become available to

you to any admin basically simply by

creating specific rules in order to

match specific types of events stored in

there so you could create simple rules

that are they're matching based on

specific conditions look for one

specific IP address or look for a

specific time range look for one

specific string that might occur in all

those log payloads maybe look for a user

and see what are the events that are

that are generated by the user or that

implicate that you user and so on and so

forth so the seam appliances are going

to allow you to create some queries very

similar to what you might be already

used to if you ever used SQL in the past

because all that data is basically

stored in a relational database which

can be queried with an SQL like a

language and finally don't forget that

at the end of the day not everybody has

money to invest in a Sim solution so you

might end up having to analyze your logs

by yourself just navigating a bunch of

logs and this is where a bunch of text

matching utilities especially some

utilities that are built into most Linux

distributions are going to come in and

help you tremendously now this is not

Linux course and the exam is not going

to expect you to know everything about

all these command line commands but I

would say that knowing at least the

commands right here on the slide is

going to help you figure out a couple of

the outputs on the exam right so without

going into too much detail here let's

have a look in one of my folders here

that stores log files and a new Ubuntu

distribution this is running on WSL

right windows subsystem for Linux we

have a log file right here dpkg log

which is logged that's generated by the

package managers so this log is going to

tell me which package based operations

are have been conducted on this machine

from its big It's Beginning from its

installation right what did I install

what did I uninstall what did I upgrade

so it might be some useful information

in here so let's just see a couple of

these commands cat is the concatenate

command in Linux and can also be used to

list the contents of

text files so CAD dpk G log is going to

provide you a bunch of listing right

here trying to display all the contents

of the text file right at the console

now this file right here we can also

pipe it so resend the result of this cat

command to another command which could

be word count word count minus L this is

going to count the lines in its log file

so you can see it's over 9 000 lines

long pretty tough to search for some

information in a 9000 line log file so

what we can do right here is for example

limit the amount of information that

we're displaying on the screen this is

where the head or tail commands come in

the head command as you can probably

guess is going to provide you with a

listing of the first 10 lines in this

log file similarly the tail command is

going to provide you a listing of the

last 10 lines in a log file the tail

command is very useful for log files

that get appended frequently so you just

want to see the last modification

locations made in this file use the tail

command of course the number of lines is

configurable we're not going to go into

all these parameters right now if you're

interested in finding out more about any

Linux command any Linux utility just use

the Man pages man tail

and it's going to provide you with the

manual pages that are going to tell you

what are all the possible configuration

Flags or settings that can be added to

this command here's the dash n for

example number of lines I'll put the

last number of lines you can add it as a

minus n parameter or dash dash line

sequels how many lines you want to

display on the screen quit with the

letter Q now the graph utility is a

regular expression evaluator which can

be of course used to run some complex

regular Expressions which are going to

help you tremendously dig through a lot

of information extract what is actually

useful to you but you can also do some

very simple uh string matching using rep

for example if we are displaying the

dpkg log here and piping this to the uh

to the grep command and search for let's

say installation of a specific package

such as let me see ansible right I did

use this machine for ansible in the past

so there you go these are all the log

entries in here generated by the ansible

package notice that we've been through a

number of advancible versions in here

starting from version 2.8.1 we went

through 2.919 2.927 and so on we can

even see the evolution of this package

on this machine now this is just a very

very simple example here I just wanted

to let you know that you do have a lot

of utilities available at your disposal

for manual log searching if you don't

have a SIM solution available all right

now there's a lot more to talk about

this but since this is not a Linux

trading we're gonna stop right here

alright everyone thanks so much for

watching I know there's been a lot of

information in this video but I hope you

found this useful and informative and I

hope to see you on the next video as

well don't forget to leave a comment if

you like this support the channel if you

can if you wish if you find this useful

in your studies and see you in the next

video bye bye

[Music]

foreign

[Music]