-
foreign
-
[Music]
-
monitoring has been around for a lot of
-
time probably ever since the first
-
networks were invented just like with
-
any system just like with any electronic
-
device we tend to want to be able to
-
monitor if everything is going okay we
-
want to receive warnings we want to be
-
alerted when something goes wrong when
-
when something fails and this type of
-
monitoring is tremendously useful
-
especially in larger networks over time
-
this monitoring has extended to security
-
monitoring as well so we're not just
-
concerned about how is the network doing
-
if it's working well if you don't have
-
any failed devices but we're also
-
starting to look at the network traffic
-
how is the network utilized who uses it
-
who attempts to access it what type of
-
traffic are they generating and if we
-
try to gather all this type of
-
information we try to make sense of it
-
we try to correlate it with a smart
-
enough device we might be able to detect
-
attempts at intrusion or attacks that
-
are about to happen or that have
-
happened in the past or proves that
-
we've been compromised or somebody in
-
the network has been infected and all
-
that information is in there if you know
-
where to look for and also if you have
-
the right tools to look for it
-
in general the term internal detection
-
refers to a system that is able to
-
monitor whatever can be observed in a
-
network and in most cases we're talking
-
about two things that can be observed
-
first of all we have Network traffic
-
and then we have application events or
-
logs generated by the operating systems
-
by the applications running on those
-
os's and so on so coming back here to
-
our Network Focus we're talking about
-
intrusion detection at our Network level
-
we're going to call this one a network
-
based intrusion detection system or nids
-
and we have many commercial Solutions as
-
well as open source ones that are able
-
to perform this type of network-based
-
intrusion detection of course all the
-
major security vendors are doing it in
-
many examples you're going to see the
-
IDS functionality built into the
-
functionality of a larger firewall or a
-
larger UTM device especially for major
-
vendors out there but you also have
-
Solutions in the in the open source area
-
such as sort suricata or Zeke or bro
-
they're all available some of them also
-
have commercial versions as well but
-
they also provide you with three
-
versions that you can freely install and
-
try and run in your own environment now
-
the way these uh intrusion detection
-
systems work by definition is that they
-
rely on a database of signatures and
-
those signatures are basically just a
-
way to describe how a specific traffic
-
pattern is supposed to look like in
-
order to detect a specific type of
-
attack or attempt at an intrusion so we
-
might be looking at a sequence of
-
packets so that looks in a certain way
-
we might be looking at a specific type
-
of packet that doesn't play by the
-
normal protocol rules that it belongs to
-
or a specific type of payload or simply
-
just a a signature a byte sequence that
-
can be found in the in the packet
-
payload that indicates the fact that the
-
payload is malicious and this behavior
-
is very similar to what you're seeing in
-
antivirus scanning or anti-malware
-
scanning we're simply looking for a
-
sequence of bytes that indicates that
-
well if we find the sequence of bytes in
-
a specific executable file it means that
-
the file is infected with that specific
-
virus that the the sequence belongs to
-
now when intrusion detection again we
-
we're kind of doing the same thing right
-
we're looking for patterns but we're not
-
just scanning individual packets
-
sometimes we need to collect more
-
packets in a sequence in order to
-
determine if the behavior of the client
-
that is generating those packets is
-
abnormal and if it's abnormal what does
-
it indicate an attack pattern or not the
-
long story short intrusion detection
-
systems are strongly dependent on a
-
database of signatures now more advanced
-
nitrogen detection systems could also
-
correlate this network information with
-
log information so we're seeing
-
something fishy in the network by
-
looking at the network traffic let's
-
check the application logs that the
-
traffic is is going towards for example
-
let's see how that application reacts
-
and if we if we can see some abnormal
-
logs being generated by the app as well
-
now correlating that information the
-
traffic analogs might tell us more about
-
the actual attack or might increase the
-
confidence of the fact that we really
-
have identified a valid attack signature
-
not all solutions are able to do this of
-
course also a very important distinction
-
for intrusion detection system with an
-
emphasis on detection is the fact that
-
these systems are never able to block
-
the malicious traffic once they identify
-
it it's just like the name says just
-
detection it's not prevention all right
-
so we're not stopping the traffic we
-
might be able to see an attack signature
-
we might be able to raise some alerts
-
generate some syslogs but we're not
-
going to be able to block that specific
-
type of traffic one positive side for
-
this is that well if the device is not
-
inside of the traffic path then the
-
attacker might not even be able to
-
detect it
-
so most likely the the IDS is going to
-
work with a copy of the traffic just to
-
analyze it but it's not going to be able
-
to stop the malicious traffic and the
-
attacker is not going to be able to
-
detect the IDS device and might not even
-
be able to compromise it if if they
-
intend to in most situations the IDS
-
device doesn't even have a valid IP
-
address within the network that they're
-
monitoring so it cannot be addressed it
-
cannot be compromised by communicating
-
with it directly alright so since we
-
mentioned the fact that an IDs works
-
with just a copy of the traffic let's
-
see how can we generate that copy of
-
traffic right they're not within the
-
traffic path so we need to make a copy
-
of the traffic and just send it in a
-
separate channel on a separate channel
-
to the IDS device for analysis now my
-
way of doing this is by enabling Port
-
mirroring or span on in Cisco speak this
-
is switchboard analyzer just a
-
functionality on layer 2 or layer 3
-
switches that allow us to configure the
-
switch and we're basically telling it
-
well whatever traffic you're seeing on
-
ports let's say one two and three make a
-
copy of that traffic and forward it out
-
of port number eight and of course we're
-
assuming that on Board number eight
-
there's an IDs device connected right
-
there so we're basically telling the
-
switch to make a copy of all the
-
interesting traffic and send it towards
-
the IDS and of course you might be
-
thinking here well what if the switch is
-
overloaded what if there's more traffic
-
generated on those ports than the uh the
-
mirror report can actually support well
-
that's true it might happen so in in
-
cases when the switch is overloaded and
-
there's too much traffic in the network
-
packets might be dropped and also frames
-
with errors might not be forwarded to
-
the uh to the merits board either so we
-
might not be able to see 100 of all the
-
traffic but in most cases it's going to
-
be enough and it's also one of the
-
features that basically doesn't require
-
you to install anything else in the
-
network it's just a functionality just a
-
configuration effort just a couple of
-
commands on a switch another method for
-
duplicating traffic is by using a
-
passive order and active it's basically
-
layer 1 device called the tap a test
-
access port it's nothing else than a
-
kind of like a t connector where the
-
main cable goes from one end to the next
-
and there's a third cable that actually
-
receives a copy of the entire traffic
-
going through that that segment of cable
-
the device is not a smart one so it's
-
it's not like a switch it's not going to
-
look at the destination frames and
-
forward uh entire packets it's simply
-
going to duplicate the electrical or the
-
optical signals that it sees on the wire
-
and it's going to make a complete and
-
identical copy of those signals onto the
-
third connection which of course is uh
-
is ideally connected to the IDS device
-
now this type of approach is again
-
completely undetectable
-
span is not detectable either right and
-
also copies entire frames regardless if
-
those frames contain errors or not as we
-
said with Port mirroring while the
-
frames need to be correct in order to be
-
copied well with a tap the tap doesn't
-
care it's basically just a signal
-
repeater and we can do this for both
-
copper cables and so electrical signals
-
as well as fiber optic so Optical
-
signals the tap will not care we'll just
-
blindly copy all the signals that it
-
receives and finally the third method
-
for monitoring traffic is by having the
-
IDS device in the traffic path
-
but acting as a transparent device again
-
without an IP address or basically
-
becoming a layer 2 device that is part
-
of the same VLAN that they're they're
-
bridging but they cannot be addressed on
-
the network they cannot be detected on
-
the network and they if it's a true IDs
-
device then it's not going to be able to
-
block the actual traffic that goes
-
through it now having the device placed
-
inside of the traffic path opens us to
-
the possibility of actually blocking the
-
traffic and that's going to be a
-
different type of solution called
-
intrusion prevention system and we'll
-
get there in just a moment there's one
-
more type of intrusion detection device
-
or solution and that is a software
-
solution that can be installed directly
-
on the workstations so I'm not talking
-
about a box that listens to network
-
traffic on an entire segment but we're
-
talking here about a software solution
-
basically a program that runs on your
-
endpoint machine on your host machine be
-
it a laptop or a desktop now this one is
-
called host based instruction detection
-
because it runs on the host and it does
-
have pretty much the same benefit it's
-
or the same abilities as a network-based
-
internet detection so it's able to look
-
at the network traffic going in and out
-
of your network interface it's able to
-
look at the logs generated by the
-
applications on your system but since
-
they are running as an application on
-
your system they can become even smarter
-
because they might have access now to
-
the actual process table they might be
-
looking at the kernel you might be able
-
to look at the memory to see what
-
processes are running when did they
-
execute who executed them with what
-
privileges and they can also openly look
-
at encrypted traffic so if you are
-
communicating over SSL with a website
-
well a network-based internet connection
-
might not be able to understand anything
-
that's going back and forth because it's
-
encrypted but your host based intrusion
-
detection
-
is located at the end of that encrypted
-
tunnel so it is able to see that
-
unencrypted traffic before it even
-
enters the encrypted tunnel and right
-
after it leaves the encrypted tunnel so
-
it's able to actually watch the entire
-
traffic flow in an unencrypted form and
-
again since we have pretty much full
-
permissions on on the monitored host in
-
order to be able to properly monitor the
-
you know the process table and the
-
network connections and the network
-
traffic we could also have a look at the
-
files on the disk
-
why would you do that well that's
-
because monitoring the Integrity of the
-
files on the desk especially the
-
Integrity of the operating system files
-
and being able to detect when that
-
Integrity fails when a system file is
-
being replaced with a malicious one when
-
a system file is is becoming encrypted
-
or it is replaced with a completely
-
different version that might be an
-
indication of compromise that might be
-
an indication of the fact that you have
-
been infected with malware so Solutions
-
or functionality additional to host
-
based internet detection that monitor
-
files on your system especially
-
operating system files these are called
-
file Integrity Monitoring Solutions and
-
remember that we said that when we place
-
the intrusion detection device in the
-
traffic path
-
device actually becomes able to also
-
block the traffic that goes through it
-
which can make it an intrusion
-
prevention system right so detection
-
just alerts just generate alerts or
-
events intrusion prevention is about
-
actually actually taking action or
-
acting upon the detected intrusion so
-
what can a such a device actually do
-
whenever they're they're seeing
-
something fishy going on inside of the
-
network well they could do something as
-
simple as simply sending a TCP reset
-
packet to the originator of the
-
malicious connection they could also
-
have some more advanced functionality
-
especially if they if it's the same
-
device that acts as a firewall they
-
might be dynamically able to generate a
-
firewall rule to block similar traffic
-
like the one that was just detected as
-
being part of an attempt for for an
-
attack or for a compromise we could be
-
choosing if we're detecting something
-
that looks like a denial of service
-
attack we'd be choosing to limit the
-
amount of bandwidth that is allocated to
-
that specific type of traffic kind of
-
like policing that we're doing in into
-
our inequality of service in any case
-
any type of action that the IPS device
-
can take against the malicious traffic
-
we're going to call it active response
-
and depending on how complex the device
-
is and how powerful the device is you
-
might actually choose to look not just
-
at simple IPS or IDs signatures but also
-
look for malware signatures yeah that's
-
that's going to require you to you know
-
to decode encrypted traffic it's going
-
to require you to identify potential
-
protocols that might be carrying files
-
gather all those related packets that
-
belong to the same TCP Stream So the
-
same flow assemble them into an
-
executable file store that in memory or
-
attempt to scan it with an antivirus
-
engine and then determine if that flow
-
was actually malicious or not now this
-
requires a lot of processing power this
-
is going to create some sort of delay in
-
the networks of the users they're going
-
to see their their download uh unable to
-
finish or the application responding
-
slowly until the firewall the UTM device
-
or the nutrition prevention system is
-
actually able to scan those files
-
against malware signatures on a lighter
-
approach we could also just be looking
-
at URLs looking for malicious domains or
-
domains that show associated with
-
malware or with the command and control
-
servers we might be looking at URLs in
-
order to categorize those URLs and
-
figure out the reputation of that URL
-
and decide whether we want the
-
communication to that specific website
-
to proceed or not so regardless if the
-
device is an IPS or an IDs the detection
-
methods are pretty much the same now the
-
difference is just in what the device is
-
actually doing is it only alerting or is
-
it actually taking an active response
-
approach to the traffic but the
-
detection part is pretty much the same
-
right and when talking about detection
-
we are going to start with the basic
-
type of detection that is where we're
-
just looking for signatures in the
-
database which of course means that we
-
need to have an up-to-date database for
-
the device to be able to detect the
-
latest and the greatest attack now this
-
is basically one of the reasons why
-
people choose to pay for commercial
-
Solutions because databases maintained
-
by a dedicated software or security
-
vendor that deals with intrusion
-
prevention those databases are going to
-
be much more often updated and kept up
-
to date in order to mirror as best as
-
possible the database of all the known
-
attack patterns ever detected in the
-
world now with open source Solutions
-
you're gonna you're still going to have
-
a pretty good level of protection but
-
you might not be able to detect an
-
attack that was just identified
-
six hours ago nevertheless and
-
regardless how up-to-date your database
-
is you're still limited by the attack
-
patterns listed in that database if an
-
attack emerges and doesn't match
-
anything in your database it's still
-
going to go through
-
which leads us to a different approach
-
and that is a behavioral approach so
-
instead of looking at specific streams
-
of bytes specific headers specific
-
sequences of packets let's look at the
-
overall behavior of an application or of
-
a protocol
-
does it look like it's doing what's
-
supposed to do is it generating more
-
packets than we're used to seeing is it
-
uh generating more traffic is it
-
generating an abnormal amount of control
-
information as opposed to a real
-
transfer data and we call this
-
behavioral monitoring now in order for
-
Behavioral monitoring to work we need to
-
have something to compare that behavior
-
too and say well if it goes outside of
-
the known ranges
-
then it looks like something's fishy
-
well that known range is supposed to be
-
your Baseline so such a device or such a
-
system is supposed to be trained first
-
you're supposed to just leave it inside
-
of the network for let's say a week or
-
two just figure let it figure out how
-
does a normal Monday morning look like
-
in your network when everybody comes
-
into work and they start logging in and
-
start updating their their uh their
-
machines and perhaps even their mobile
-
phones only on the company Wi-Fi but
-
nevertheless you have to leave that
-
interim prevention solution learn what
-
does your normal traffic look like when
-
people start accessing internal
-
applications where people start
-
accessing internet destinations when
-
people start communicating sharing files
-
between each other when when backups
-
start to happen at midnight perhaps
-
right you have to let it learn so that
-
in a couple of weeks when something goes
-
outside of the known range where an
-
application behaves the way it did not
-
behave in the first training weeks then
-
it's going to be able to raise an alarm
-
and perhaps indicate the fact that the
-
application has been compromised or that
-
somebody is using it in order to elevate
-
their privileges or just compromise your
-
network and as you can probably guess
-
this is one area where machine learning
-
is going to provide you a lot of benefit
-
given that you take the time and efforts
-
to educate to teach the machine Learning
-
System what does your normal Baseline
-
look like now of course regardless how
-
complex or how well tuned Your solution
-
is going to be there will be false
-
positives and there will be false
-
negatives which is why I always tell
-
tell students there's a old saying that
-
I heard from someone in Cisco a long
-
long time ago and they said that IPS
-
Without Eyes
-
is useless so IPS without human eyes is
-
useless there there's always going to be
-
the need to have a human being right
-
there evaluating and analyzing whether
-
the alerts generated by the intrusion
-
prevention or detection system are valid
-
or not does it need more fine-tuning or
-
do we need to raise an alarm so what
-
devices can we actually find that
-
implement this type of advanced
-
functionality be it detection or
-
prevention well unfortunately this is
-
the uh the place where we're slowly
-
stepping into the marketing area that's
-
because the devices that we're going to
-
be listing here are not completely
-
different devices but over time
-
different naming conventions have
-
emerged different marketing names have
-
been invented to make them sound cool to
-
make them sound different from what the
-
other vendors were doing so we're going
-
to start with the next generation of
-
firewall and we will have this type of
-
Next Generation
-
for about 12 or 15 years already I've
-
been hearing the Next Generation term in
-
in uh in I.T security for so long that
-
I'm starting to wonder
-
are we still next generation are we have
-
we skipped the generation are we now in
-
the next next generation or where does
-
it stop where does it end where where
-
does the Next Generation begin right now
-
unfortunately marketing people don't
-
really ask themselves these questions so
-
we're kind of stuck with this
-
terminology for now and we're gonna keep
-
calling you next Generation until I
-
don't know when but regardless a Next
-
Generation firewall is basically just a
-
layer 7 firewall that's an application
-
layer firewall which is able to look at
-
the application layer payload so we're
-
actually seeing the data being sent
-
we're not just looking at the packet
-
headers and it also has some sort of
-
detection or prevention system built in
-
okay so we have an IPS or an IDs built
-
in which leads us back to the discussion
-
that we had before so we have an
-
application layer firewall which can be
-
enriched with additional functionality
-
now that we have access to the actual
-
application payload well why why not
-
look for intrusion signatures why not
-
look for malware signatures why not look
-
for spam signatures right so depending
-
on how complex the device is if it at
-
least has IPS functionality built in
-
we're going to call it the Next
-
Generation firewall and here's the funny
-
part if the Next Generation firewall has
-
a bunch of other additional features on
-
top of the IPS functionality such as
-
malware scanning antivirus scanning
-
perhaps looking at the files and being
-
able to implement some data loss
-
prevention policies uh it's able to look
-
at the URLs and categorize them and
-
analyze the reputation of the web pages
-
and pretty much everything that we could
-
possibly think of that we could be doing
-
just by looking at the application data
-
then we're going to call this a unified
-
threat management device a UTM device
-
again I don't think I need to repeat
-
this but the more complex the device
-
becomes the more stuff it needs to do in
-
order to decide weather to allow a
-
packet or not the more resources the
-
more CPU intensive is going to be the
-
more memory is going to require and the
-
more delay is going to introduce in the
-
network so keep this in mind even though
-
it kind of sounds cool right to have all
-
that security functionality in a single
-
box
-
which by the way you try to make sure
-
it's not a single box of failure single
-
point of failure all right
-
even though it sounds cool to have all
-
this functionality in one place
-
it's going to hit your performance
-
pretty badly right so keep this in mind
-
don't just enable everything blindly
-
because the end users the applications
-
and well God forbid the your your
-
customers you're paying customers
-
they're going to feel the the effects of
-
your of your awesome UTM device and
-
their application experience is going to
-
suffer now a special type of network
-
monitoring device can also be considered
-
a web application firewall we've briefly
-
mentioned about where allocations
-
firewalls in in a previous video and we
-
said that uh a web application firewall
-
is just a dedicated firewall that is
-
specifically trained and educated to
-
look at attack signatures aimed at web
-
applications so we're looking for things
-
such as cross-site scripting we're
-
looking for
-
um you know directory traversals we're
-
looking at SQL injection attacks we're
-
looking at pretty much anything that
-
could be performed by malicious user
-
that is trying to exploit a input
-
validation flaw in a web application so
-
it's still an application layer firewall
-
it still looks at the the application
-
layer payload it's just that it's a bit
-
more let's say picky about what type of
-
traffic is it going to analyze so it's
-
only going to look at web traffic and
-
it's only going to look for uh web
-
attacks web application attacks it's
-
mostly going to rely on signatures
-
that's because we cannot really do much
-
when it comes to requests coming in from
-
from our clients behavioral analysis
-
doesn't really play well here because
-
most attacks especially where vacation
-
attacks are just one single request one
-
single query with a malicious payload so
-
so in many situations it's gonna be
-
either black or white right we we're
-
detecting an attempt at an intrusion
-
we're detecting an attack in that
-
require West or not it's pretty much not
-
going to be much of a gray area with
-
verification firewalls and you could
-
deploy a WAFF as a separate device it
-
could be a physical box it could be a
-
virtual machine it could be a
-
functionality within a UTM device again
-
all in one wonders but it can also be a
-
part of the web server itself so we have
-
plugins that install alongside the
-
actual web server that is hosting the
-
web application such as plugins for the
-
Apache web server for the IIs web server
-
on on Windows server or for nginx so
-
we're installing these plugins right
-
there and their purpose is to scan the
-
traffic that's coming in from the
-
clients before allowing that request to
-
be processed by the web server having
-
something such as a plugin that runs
-
alongside the web server on the same
-
machine on the same box opens us to the
-
risk of either having that machine
-
compromised by an attacker who this time
-
doesn't Target the web application but
-
targets the scanning engine and can
-
intentionally cause for example a denial
-
of service give it so much traffic to
-
analyze that the web server running on
-
the same machine is unable to actually
-
respond to valid requests so there you
-
have it that's the now service attack
-
now when it comes to actually monitoring
-
the network traffic we said that a
-
solution would be to just simply mirror
-
all the traffic and then look for
-
specific deck patterns inside of that
-
traffic now that might not be always
-
feasible because the amount of traffic
-
entering a data center or the server
-
form that hosts an application might be
-
huge right so in some situations we
-
might not be able to analyze the exact
-
amount of traffic that goes in but we
-
might be able to generate a summary of
-
that traffic and then Analyze That
-
summary for intrusion attempts now this
-
traffic summary is sometimes found under
-
the terminology of netflow or S flow or
-
J flow which is basically just a
-
technology implemented by various
-
vendors out there in which instead of
-
creating an exact copy of the traffic
-
we're simply summarizing that traffic
-
and then reporting that summary back to
-
some analysis software so we're only
-
telling it what type of sources what
-
type of destinations have communicated
-
how many bytes were used what type of
-
protocols has been have been used uh
-
what type of flags have been set in that
-
specific type of traffic but we don't
-
put the burden of sending the entire
-
actual traffic in the entire payload to
-
that analysis software now this also
-
means that we're losing application
-
layer visibility all right since we're
-
just summarizing the type of traffic
-
we're only describing the metadata about
-
the traffic we're losing everything that
-
pertains to the application layer but
-
we're gaining a lot of performance and
-
we can also store this summary
-
information long term for further
-
analysis somewhere along the line in the
-
future sometimes looking at traffic it's
-
simply not feasible maybe we cannot grab
-
all the traffic that's running through
-
the network maybe we don't have network
-
devices smart enough to generate those
-
so those summaries those those flow
-
reports for us so another solution would
-
be to Simply have a software monitoring
-
service Ocean or so-called a network
-
performance monitor that queries
-
periodically your networking devices
-
queries your your routers your switches
-
uh your wireless LAN controllers your
-
firewalls perhaps about the status of
-
their physical resources status of their
-
interfaces how much traffic is going
-
through their interfaces I was the CPU
-
load what's the memory usage what's the
-
structure of the routing table how does
-
the r table look like how is the dhtp
-
traffic looking like right so any type
-
of monitoring information that can be
-
extracted out of these networking
-
devices which in turn can be correlated
-
in order to figure out if we can see
-
some anomalies in there one such
-
solution is for example solarwinds npm
-
Network performance monitor which is a
-
dedicated solution for monitoring not
-
just networking devices but also servers
-
and virtual machines about their their
-
their health right how are their network
-
interfaces looking like how much load is
-
there on their Hardware resource or
-
their Hardware components are they
-
generating any alerts do we have failed
-
interfaces do we have failed processes
-
we have something that's uh failed links
-
are we detecting errors or overloaded
-
devices stuff like that
-
now this type of performance monitoring
-
can be done over a variety of protocols
-
in most cases the SNMP protocol is going
-
to be used because it allows us to
-
report a lot of the hardware counters
-
and a lot of the interesting information
-
that we want to gather in store long
-
term also we might be using wmi such as
-
Windows management instrumentation and a
-
couple other protocols as well and of
-
course we could enrich this collection
-
by collecting logs from the monitored
-
devices and appliances as well and we
-
could be collecting those logs over
-
syslog so we need to configure the
-
device to actually send those syslogs
-
messages or at least a copy of them to
-
the monitoring device or we could rely
-
on an agent an additional piece of
-
software installed on the server on the
-
virtual machine that periodically
-
reports back to us everything of
-
interest regarding the that specific
-
host when talking about dedicated
-
software design specifically designed to
-
analyze a lot of information coming from
-
the network beat Network traffic Network
-
summaries such as netflow logs and any
-
kind of application data that solution
-
is most likely going to be called a seam
-
a security information and event
-
management now the keyword and a
-
definition of seam is correlation that
-
is it's not just a place where you just
-
dump all that information in a huge
-
database it's a place that as you dump
-
that information is going to look for
-
patterns inside of it it's going to try
-
to correlate Network traffic with logs
-
or application data with with netflow
-
data in order to figure out if some
-
anomalous behavior is detected in your
-
network so same solution and by the way
-
these are pretty expensive Solutions out
-
there are never designed to be just log
-
storage right they're engines smart
-
engines based on machine learning that
-
aim to detect patterns of intrusion by
-
analyzing and correlating information
-
found in multiple log files and what's
-
interesting about the implementation of
-
seams is that they're supposed to
-
collect logs from your network devices
-
from your security devices even from
-
your workstations and your mobile
-
devices perhaps and they're able to
-
understand and correlate all that
-
information and normalize all that
-
information even if it comes from tens
-
or hundreds of vendors or thousands of
-
devices
-
and they're able to normalize that
-
information and make it look the same so
-
that in the end
-
it can look for patterns inside of it
-
and it also allows you to perform
-
queries in a language quite similar to a
-
regular SQL language and query all that
-
information regardless of the fact that
-
it actually came from or tens of
-
hundreds of different vendors and since
-
a seam without machine learning
-
functionality is not a very useful theme
-
we could use that machine learning
-
features to look at user Behavior as
-
well because in the end we're trying not
-
to detect just you know attack patterns
-
we're also trying to identify who is
-
conducting them and a great risk comes
-
from Insider threats so if we are able
-
to monitor what our users are doing
-
we're not talking here about just
-
watching what websites they're they're
-
visiting or taking frequent screenshots
-
of their of their workstations now we're
-
not doing that but we're looking at the
-
behavior that they're exhibiting
-
whenever they are interacting with
-
specific applications and if the scene
-
has such an ability we call that ability
-
user and entity Behavior Analysis don't
-
think that we're only uh performing here
-
a Witch Hunt against uh Insider threats
-
think about the fact that we might be
-
able to detect abnormal behavior because
-
a user account has been compromised by a
-
hacker and that hacker is now acting on
-
behalf of that user the user might have
-
nothing to do with that abnormal
-
behavior might not even know about it
-
might not even be logged in at that
-
specific point in time but the attacker
-
might be acting on behalf of that user
-
if we're able to detect that abnormal
-
behavior we might be able to detect the
-
attack going on right then and stepping
-
just a bit into the realm of Science
-
Fiction here I know that some vendors
-
will say no this is not science fiction
-
where we're selling this we've had huge
-
success with this well yes and no I'm
-
gonna keep being a bit skeptical as to
-
how efficient this approach is what I'm
-
talking here about is sentiment analysis
-
or emotion AI that is analyzing user
-
behavior in what contents the user is
-
actually creating as in blog posts
-
social media posting
-
we're not talking here about actual you
-
know analyzing the contents of emails
-
and and chats because that might you
-
know step into the Privacy area which we
-
might not want to do that but by
-
analyzing publicly available information
-
generated by those users we might be
-
able to detect disgruntled employees we
-
might be able to detect uh unsatisfied
-
clients that might create some bad
-
reputation for the company perhaps even
-
before they become so upset as to take
-
action or malicious action against our
-
company again take this with a brain of
-
salt and don't just think that if it if
-
it sounds awesome on paper it has to be
-
awesome in real life if it sounds too
-
good to be true then it probably is too
-
good to be true
-
and finally the last term here that I
-
wanted you to know about is soar
-
security orchestration Automation and
-
response that's a mouthful I know it's
-
usually functionality built into SIM
-
Solutions or it can be just a standalone
-
solution but it basically uh it tries to
-
address is the problem of too much
-
information that is being overwhelmed by
-
too many alerts too many security events
-
too many security incidents too many
-
incidents that we need to determine if
-
there's security related or not
-
basically the hell of any I.T Department
-
that deals solely with monitoring the
-
network and the applications and the
-
idea behind this is that a solar
-
solution is supposed to use some machine
-
learning techniques in order to not just
-
to figure out which anomalous events are
-
occurring in the network but by
-
analyzing those anomalous events it is
-
able to take some action against them so
-
it could at the point uh determine if an
-
attack is going on even if it happened
-
in the middle of the night and take
-
action immediately by blocking some
-
ports by creating an access list by
-
disabling temporary disabling some user
-
accounts that might have been
-
compromised so that security
-
orchestration Automation and response
-
just be sure everybody is clear on this
-
especially for the exam where does the
-
theme get this information from where
-
first of all it's going to get it from
-
logs right syslogs that's going to be
-
the main source of information how do
-
you collect logs well you don't really
-
collect them you expect those devices to
-
send those to you so those devices need
-
to be configured uh be it networking
-
devices there might be servers there
-
might be virtual machines whatever type
-
of device you have just configure them
-
to send your logs to a secondary
-
destination if the seam is not the
-
primary one just make sure they send a
-
copy of those syslogs to the same device
-
as well next the scene can also collect
-
data by installing agents on specific
-
systems now of course we might not be
-
able to install agents on let's say
-
routers or switches apart from some
-
recent devices that are running Docker
-
containers perhaps but in most cases CM
-
agents are designed to be installed on
-
Windows and Linux systems then they're
-
running as background processes that
-
periodically scanned the system and
-
report back to the seam uh the logs
-
generated by the operating system the
-
running applications the long generated
-
by the applications actually running on
-
that host depending on how the agent is
-
configured the built-in listeners or
-
collectors that you're seeing here on
-
the slide refers to the fact that the
-
seam is pre-configured or has plugins
-
that allow it to understand what
-
different vendors are reporting back to
-
it so it's going to have different
-
plugins to understand Lots coming in
-
from you know Cisco devices HP devices
-
Dell VMware whatever vendor it is it
-
needs some sort of a plugin to
-
understand that specific log format and
-
more than one than that it needs to
-
understand the contents the payload of
-
what the log is saying SNMP traps again
-
most monitoring information is going to
-
come in through an SNMP query or as an
-
SNMP trap generated by the device back
-
to the seam and also netflow netflow or
-
different variants implemented by
-
different vendors are basically just
-
summaries of the traffic flows detected
-
over a certain period of time collected
-
and then sent over to the scene device
-
in order for that traffic summary to be
-
analyzed finally the scene can also
-
capture raw packet data if it has
-
dedicated sensors that are able to
-
generate a copy of the traffic and send
-
it back to the seam or we can even have
-
sensors installed inside of network that
-
are monitoring real traffic and they're
-
only telling back to the seam or
-
reporting back to the sim a summary of
-
that traffic this is very useful when
-
your devices don't have enough reporting
-
on monitoring capabilities to report
-
back to the Sim device instead you need
-
to install some specific sensors that
-
look at the traffic and then tell the
-
seam the necessary information that it
-
needs to perform those correlations
-
sometimes a sensor such as this one YB
-
and IPS or an IDs device even log
-
normalization is a feature built into
-
most Sim Solutions out there and
-
normalization is required and it's a
-
very important feature because the seam
-
is designed to collect information from
-
hundreds of vendors and thousands of
-
different appliances each of them
-
running different operating systems on
-
different versions and they're all
-
building syslogs and SNP traps in
-
different formats some some are
-
reporting them as a text some are
-
generating logs in binary format some
-
logs are in Json format some are in XML
-
format or or CSV format depending on how
-
the vendor actually designed its logging
-
and monitoring abilities we might even
-
find differences as to how the logs are
-
actually encoded some of them are might
-
be using UTF some of them might be using
-
some Regional encoding we might even run
-
into some issues due to the fact that
-
the new line character is represented
-
differently between Windows and Linux
-
systems and that also might be reflected
-
in the payload included in the logs that
-
we're receiving as as part of the
-
monitoring process not to mention the
-
fact that the SNMP mips basically the
-
the database schemas that each vendor is
-
using for their own software Solutions
-
or Hardware Appliances these are
-
completely different not just among
-
vendors but also among different
-
products from the same vendor so in
-
order to have all this bunch of
-
information collected in some
-
centralized location and to be able to
-
query all this information and to be
-
able to approach it in a in a consistent
-
manner we need normalization that is
-
taking all this information coming from
-
so many vendors in so many formats and
-
making that information look exactly the
-
same so that it can be stored in a
-
single database that can be queried at
-
once regardless of the source of that
-
information so what are we using to
-
normalize all this information coming
-
from all these vendors well you guessed
-
it we're gonna need some plugins some of
-
these plugins come from this Sim vendor
-
itself so they're going to be
-
pre-packaged with vendors vendor plugins
-
from for major vendors out there some of
-
these plugins are going to come from the
-
actual vendors so if a smaller vendor
-
create hate them let's say smaller
-
firewalls at some point and they want to
-
be able to integrate with the
-
large-scale seam Solutions they're going
-
to provide you with a plugin for their
-
own environment as well and another type
-
of normalization that is really really
-
important is timestamp normalization
-
don't forget that we're looking for
-
anomalies in a network traffic and in
-
network events and if we don't have
-
timestamp normalization if we don't make
-
sure that all the events that we're
-
looking at are actually stored with
-
their right timestamp at their right
-
moment in time when they actually
-
happened we have no chance of detecting
-
anomalies in the network so we might
-
have devices that are that have a badly
-
configured clock we might have devices
-
that have been configured for different
-
time zones we might have devices that
-
display time or or timestamp those time
-
values in the in their logs in in one
-
format versus another format some of
-
them might be using 24 hour some of them
-
might be using 12 hours some of them
-
might include the daylight savings time
-
some of them might be using a UTC or
-
Unix ebook time it's up to the vendor so
-
normalizing these timestamps is also a
-
very very important topic here that
-
needs to be taken care of by the same
-
solution before that event indicated by
-
that specific timestamp is stored in the
-
database alongside with the others now
-
the way a Sim solution can look for
-
anomalies in that huge database that we
-
just talked about well it could be done
-
in a number of ways we could just rely
-
on simple if then else matches so we're
-
looking for you know specific events
-
specific types of logs being generated
-
in a specific time range perhaps this
-
type of approach is the fastest one
-
because it it's basically boils down to
-
a simple query in that huge database
-
stored by the CM and Appliance
-
unfortunately if there are unknown
-
threats if there are attacks that we
-
know nothing about that we don't have a
-
signature for them we don't know what to
-
look for we're not going to be able to
-
detect them kind of makes sense right so
-
another approach would be heuristic rule
-
matching this is a type of rule matching
-
where we're not exactly looking for an
-
exact match
-
for the specific type of event but we're
-
looking for something that it's pretty
-
close to it all right so this type of
-
approach
-
relies on a more permissive set of rules
-
so if it doesn't 100 match or rule let's
-
say if we have some events that are
-
pretty close to it and match it like
-
let's say 80 or 90 percent
-
now this also requires you to fine-tune
-
your rule set so if at some point by
-
doing heuristic rule matching you're
-
detecting some anomalies but you don't
-
have a rule that matches that anomaly
-
100 well you'd better create it right
-
you'd better fine tune your rule set and
-
add some more rules or tweak the
-
existing ones to match that newly
-
detected anomaly and just to recap this
-
year in behavioral analysis implemented
-
in a CM relies on the fact that you need
-
to build a baseline you need to tell the
-
Sim how does your normal look like how
-
does your normal traffic look like how
-
does your normal logs generated by all
-
the devices and all the applications in
-
your network looks like so that in turn
-
can be used as a starting point in order
-
to detect potential well mismatches that
-
might indicate attacks or attempts at
-
compromising your network now of course
-
this is going to create a lot of false
-
positives so you might into a situation
-
where an alert is being raised because
-
in an application starts generating some
-
huge backups because some admin has
-
modified the backup policy now the same
-
device sees a lot of traffic in there
-
racism alert raises everyone from their
-
sleep at 3am in the morning and saying
-
that oh my God this looks like a data
-
exfiltration attempt somebody's is
-
dumping all the data from our database
-
and then an admin has to come in and
-
intervene and say my dear seem what's
-
happening in there what you're seeing is
-
just a full backup happening at 3am in
-
the morning it's okay right don't freak
-
out about it okay so it does require
-
human intervention for fine tuning these
-
rules
-
on the other hand we have anomaly
-
analysis and this is by definition a
-
type of analysis that is performed
-
whenever we're comparing observed
-
Behavior with known standard Behavior
-
especially when we're comparing what
-
we're seeing as part of a protocol's
-
behavior with what this theme device
-
knows that the protocol is supposed to
-
behave according to its RFC according to
-
its definition finally with Trend
-
analysis we're going to be looking at
-
historic data and try to extrapolate it
-
for example if we see that the backups
-
are increasing every single week because
-
more data and more data is generated the
-
same device might be able to generate a
-
pattern so that if we see five gigabytes
-
and backup this week and eight gigabytes
-
of backups next week when it is going to
-
see 12 gigabytes two weeks from now it's
-
not going to raise an alert because it
-
expected the backup volume to increase
-
by that amount but I don't need to tell
-
you that not everything can be safely
-
predicted this way finally after all
-
that advanced correlation and machine
-
learning and AI features the seams
-
actually can be used as a database for
-
event storage and they can be queried by
-
human users by admins if you know what
-
to look for perhaps you just need to
-
investigate some event perhaps you need
-
to perform some some forensic analysis
-
so those databases become available to
-
you to any admin basically simply by
-
creating specific rules in order to
-
match specific types of events stored in
-
there so you could create simple rules
-
that are they're matching based on
-
specific conditions look for one
-
specific IP address or look for a
-
specific time range look for one
-
specific string that might occur in all
-
those log payloads maybe look for a user
-
and see what are the events that are
-
that are generated by the user or that
-
implicate that you user and so on and so
-
forth so the seam appliances are going
-
to allow you to create some queries very
-
similar to what you might be already
-
used to if you ever used SQL in the past
-
because all that data is basically
-
stored in a relational database which
-
can be queried with an SQL like a
-
language and finally don't forget that
-
at the end of the day not everybody has
-
money to invest in a Sim solution so you
-
might end up having to analyze your logs
-
by yourself just navigating a bunch of
-
logs and this is where a bunch of text
-
matching utilities especially some
-
utilities that are built into most Linux
-
distributions are going to come in and
-
help you tremendously now this is not
-
Linux course and the exam is not going
-
to expect you to know everything about
-
all these command line commands but I
-
would say that knowing at least the
-
commands right here on the slide is
-
going to help you figure out a couple of
-
the outputs on the exam right so without
-
going into too much detail here let's
-
have a look in one of my folders here
-
that stores log files and a new Ubuntu
-
distribution this is running on WSL
-
right windows subsystem for Linux we
-
have a log file right here dpkg log
-
which is logged that's generated by the
-
package managers so this log is going to
-
tell me which package based operations
-
are have been conducted on this machine
-
from its big It's Beginning from its
-
installation right what did I install
-
what did I uninstall what did I upgrade
-
so it might be some useful information
-
in here so let's just see a couple of
-
these commands cat is the concatenate
-
command in Linux and can also be used to
-
list the contents of
-
text files so CAD dpk G log is going to
-
provide you a bunch of listing right
-
here trying to display all the contents
-
of the text file right at the console
-
now this file right here we can also
-
pipe it so resend the result of this cat
-
command to another command which could
-
be word count word count minus L this is
-
going to count the lines in its log file
-
so you can see it's over 9 000 lines
-
long pretty tough to search for some
-
information in a 9000 line log file so
-
what we can do right here is for example
-
limit the amount of information that
-
we're displaying on the screen this is
-
where the head or tail commands come in
-
the head command as you can probably
-
guess is going to provide you with a
-
listing of the first 10 lines in this
-
log file similarly the tail command is
-
going to provide you a listing of the
-
last 10 lines in a log file the tail
-
command is very useful for log files
-
that get appended frequently so you just
-
want to see the last modification
-
locations made in this file use the tail
-
command of course the number of lines is
-
configurable we're not going to go into
-
all these parameters right now if you're
-
interested in finding out more about any
-
Linux command any Linux utility just use
-
the Man pages man tail
-
and it's going to provide you with the
-
manual pages that are going to tell you
-
what are all the possible configuration
-
Flags or settings that can be added to
-
this command here's the dash n for
-
example number of lines I'll put the
-
last number of lines you can add it as a
-
minus n parameter or dash dash line
-
sequels how many lines you want to
-
display on the screen quit with the
-
letter Q now the graph utility is a
-
regular expression evaluator which can
-
be of course used to run some complex
-
regular Expressions which are going to
-
help you tremendously dig through a lot
-
of information extract what is actually
-
useful to you but you can also do some
-
very simple uh string matching using rep
-
for example if we are displaying the
-
dpkg log here and piping this to the uh
-
to the grep command and search for let's
-
say installation of a specific package
-
such as let me see ansible right I did
-
use this machine for ansible in the past
-
so there you go these are all the log
-
entries in here generated by the ansible
-
package notice that we've been through a
-
number of advancible versions in here
-
starting from version 2.8.1 we went
-
through 2.919 2.927 and so on we can
-
even see the evolution of this package
-
on this machine now this is just a very
-
very simple example here I just wanted
-
to let you know that you do have a lot
-
of utilities available at your disposal
-
for manual log searching if you don't
-
have a SIM solution available all right
-
now there's a lot more to talk about
-
this but since this is not a Linux
-
trading we're gonna stop right here
-
alright everyone thanks so much for
-
watching I know there's been a lot of
-
information in this video but I hope you
-
found this useful and informative and I
-
hope to see you on the next video as
-
well don't forget to leave a comment if
-
you like this support the channel if you
-
can if you wish if you find this useful
-
in your studies and see you in the next
-
video bye bye
-
[Music]
-
foreign
-
[Music]