-
great so I think we can start since the
-
meeting is recorded So if everyone uh
-
jump jumps in later can can watch the
-
recording
-
so hi everyone and welcome to these
-
um Cloud skill challenge study session
-
around a create classification models
-
with Azure machine learning designer
-
so today I'm thrilled to be here with
-
John uh John do my introduce briefly
-
yourself
-
uh thank you Carlotta hello everyone
-
Welcome to our Workshop today I hope
-
that you are all excited for it I am
-
John Aziz a gold Microsoft learn student
-
ambassador and I will be here with uh
-
Carlota to like do the Practical part
-
about this module of the cloud skills
-
challenge thank you for having me
-
perfect thanks John so for those who
-
don't know me I'm kellota
-
based in Italy and focus it on AI
-
machine learning Technologies and about
-
the use in education
-
um so
-
um these Cloud skill challenge study
-
session is based on a learn module a
-
dedicated learn module I sent to you uh
-
the link to this module uh in the chat
-
in a way that you can follow along the
-
model if you want or just have a look at
-
the module later at your own pace
-
um
-
so before starting I would also like to
-
remember to remember you uh the code of
-
conduct and guidelines of our student
-
Masters community so please during this
-
meeting be respectful and inclusive and
-
be friendly open and with coming and
-
respectful of other each other
-
differences
-
if you want to learn more about the code
-
of content you can use this link in the
-
deck again.ms slash s-a-c-o-c
-
and now we are
-
um we are ready to to start our session
-
so as we mentioned it we are going to
-
focus on classification models and Azure
-
ml uh today so first of all we are going
-
to um identify uh the kind of
-
um of scenarios in which you should
-
choose to use a classification model
-
we're going to introduce Azure machine
-
learning and Azure machine designer
-
we're going to understand uh which are
-
the steps to follow to create a
-
classification model in Azure mesh
-
learning and then John will
-
um
-
lead an amazing demo about training and
-
Publishing a classification model in
-
Azure ml designer
-
so let's start from the beginning let's
-
start from identifying classification
-
machine learning scenarios
-
so first of all what is classification
-
classification is a form of machine
-
learning that is used to predict which
-
category or class an item belongs to for
-
example we might want to develop a
-
classifier able to identify if an
-
Incoming Email should be filtered or not
-
according to the style the center the
-
length of the email Etc in this case the
-
characteristics of the email are the
-
features
-
and the label is a classification of
-
either a zero or one representing a Spam
-
or non-spam for the including email so
-
this is an example of a binary
-
classifier if you want to assign
-
multiple categories to the incoming
-
email like work letters love letters
-
complaints or other categories in this
-
case a binary classifier is not longer
-
enough and we should develop a
-
multi-class classifier so classification
-
is an example of what is called
-
supervised machine learning
-
in which you train a model using data
-
that includes both the features and
-
known values for label
-
so that the model learns to fit the
-
feature combinations to the label then
-
after training has been completed you
-
can use the train model to predict
-
labels for new items for for which the
-
label is unknown
-
but let's see some examples of scenarios
-
for classification machine learning
-
models so we already mentioned an
-
example of a solution in which we would
-
need a classifier but let's explore
-
other scenarios for classification in
-
other Industries for example you can use
-
a classification model for a health
-
clinic scenario and use clinical data to
-
predict whether patient will become sick
-
or not
-
uh you can use
-
um
-
oh sorry so when I became muted it's a
-
long time or you can use you can use uh
-
some models for classification for
-
example you can use you're saying this
-
uh so I I was I was
-
this one like you you have been muted
-
for uh one second okay okay perfect
-
perfect uh yeah I was talking sorry for
-
that so I was talking about the possible
-
you can use a classification model like
-
have Clinic scenario Financial scenario
-
or other third one is business type of
-
scenario you can use characteristics or
-
small business to predict if a new
-
Venture will will succeed or not for
-
example and these are all types of
-
binary classification
-
uh but today we are also going to talk
-
about Azure machine learning so let's
-
see
-
um what is azure Mash learning so
-
training and deploying an effective
-
machine learning model involves a lot of
-
work much of it time consuming and
-
resource intensive so Azure machine
-
learning is a cloud-based service that
-
helps simplify some of the tasks it
-
takes to prepare data train a model and
-
also deploy it as a predictive service
-
so it helps that the scientists increase
-
their efficiency by automating many of
-
the time consuming tasks Associated to
-
creating and training a model
-
and it enables them also to use
-
cloud-based compute resources that scale
-
effectively to handle large volumes of
-
data while incurring costs only when
-
actually used
-
to use Azure machine learning you
-
fasting fast you need to create a
-
workspace resource in your Azure
-
subscription and you can then use these
-
workspace to manage data compute
-
resources code models and other
-
artifacts after you have created an
-
Azure machine learning workspace you can
-
develop Solutions with the Azure machine
-
learning service either with developer
-
tools or the Azure machine Learning
-
Studio web portal
-
in particular International Learning
-
Studio is a web portal for machine
-
learning Solutions in Azure and it
-
includes a wide range of features and
-
capabilities that help data scientists
-
prepare data train models publish
-
Predictive Services and monitor also
-
their usage
-
so to begin using the web portal you
-
need to assign the workspace you created
-
in the Azure portal to the Azure machine
-
Learning Studio
-
as its core Azure Mash learning is a
-
service for training and managing
-
machine learning models for which you
-
need compute resources on which to run
-
the training process
-
compute targets are um one of the main
-
basic concept of azure Mash learning
-
they are cloud-based resources on which
-
you can run model training and AD
-
exploration processes
-
so initial machine Learning Studio you
-
can manage the compute targets for your
-
data science activities and there are
-
four kinds of of compute targets you can
-
create we have the computer instances
-
which are vital machine set up for
-
running machine learning code during
-
development so they are not designed for
-
production
-
then we have compute clusters which are
-
a set of virtual machines that can scale
-
up automatically based on traffic
-
we have inference clusters which are
-
similar to compute clusters but they are
-
designed for deployment so they are a
-
deployment targets for Predictive
-
Services that use train models
-
and finally we have attached compute
-
which are any compute Target that you
-
manage yourself outside of azramel like
-
for example virtual machines or Azure
-
databricks clusters
-
so we talked about Azure machine
-
learning but we also mentioned it
-
mentioned Azure machine learning
-
designer what is azure machine learning
-
designer so initial machine Learning
-
Studio there are several ways to author
-
classification machine learning models
-
one way is to use a visual interface and
-
this visual interface is called designer
-
and you can use it to train test and
-
also deploy machine learning models and
-
the drag and drop interface makes use of
-
clearly defined inputs and outputs that
-
can be shared reused and also Version
-
Control
-
and using the designer you can identify
-
the building blocks or components needed
-
for your model place and connect them on
-
your canvas and run a machine learning
-
job
-
so
-
um each designer project so each project
-
in the designer is known as a pipeline
-
and in the design we have a left panel
-
for navigation and a canvas on your
-
right hand side in which you build your
-
pipeline visually so pipelines let you
-
organize manage and reuse complex
-
machine learning workflows across
-
projects and users
-
a pipeline starts with the data set from
-
which you want to train the model
-
because all begins with data when
-
talking about data science and machine
-
learning and each time you run a
-
pipeline the configuration of the
-
pipeline and its results are stored in
-
your workspace as a pipeline job
-
so the second main concept of azure
-
machine learning is a component so going
-
hierarchically from the pipeline we can
-
say that each building block of a
-
pipeline is called a component
-
learning component encapsulate one step
-
in a machine learning pipeline so it's a
-
reusable piece of code with inputs and
-
outputs something very similar to a
-
function in any programming language
-
and in a pipeline project you can access
-
data assets and components from the left
-
panels
-
asset Library tab as you can see
-
um here in the screenshot in the deck
-
so you can create data assets on using
-
an adoc page called Data Page and a data
-
set is a reference to a data source
-
location
-
so this data source location could be a
-
local file a data store a web file or
-
even an age group open that set
-
and these data assets will appear along
-
with standard sample data set in the
-
designers asset Library
-
um
-
and another basic concept of azure ml is
-
azure machine learning jobs
-
so basically when you submit a pipeline
-
you create a job which will run all the
-
steps in your pipeline so a job execute
-
a task against a specified compute
-
Target
-
jobs enable systematic tracking for your
-
machine learning experimentation in
-
Azure ml
-
and once a job is created azramel
-
maintains a run record uh for for the
-
job
-
um but let's move to the classification
-
steps so
-
um let's introduce uh how to create a
-
classification model in Azure ml but you
-
will see it in more details in a
-
handsome demo that John will will guide
-
through in a few minutes
-
so you can think of the steps to train
-
and evaluate a classification machine
-
learning model as four main steps so
-
first of all you need to prepare your
-
data so you need to identify the
-
features and the label in your data set
-
you need to pre-process so you need to
-
clean and transform the data as needed
-
then the second step of course is
-
training the model
-
and for training the model you need to
-
split the data into two groups a
-
training and a validation set
-
then you train a machine learning model
-
using the training data set and you test
-
the machine learning model for
-
performance using the validation data
-
set
-
the third step is performance evaluation
-
um which means comparing how close the
-
model's predictions are to the known
-
labels and these lead us to compute some
-
evaluation performance metrics
-
and then finally
-
um so these three steps are not
-
um not performed uh every time in a
-
linear manner it's more an iterative
-
process but once you obtain you achieve
-
a a performance with which you are
-
satisfied so you are ready to let's say
-
go into production and you can deploy
-
your train model as a predictive service
-
into a real-time uh to a real-time
-
endpoint and to do so you need to
-
convert the training pipeline into a
-
real-time inference Pipeline and then
-
you can deploy the model as an
-
application on a server or device so
-
that others can consume this model
-
so let's start with the first step which
-
is prepaid data reward data can contain
-
many different issues that can affect
-
the utility of the data and our
-
interpretation of the results so also
-
the machine learning model that you
-
train using this data for example real
-
world data can be affected by a bed
-
recording or a bad measurement and it
-
can also contain missing values for some
-
parameters and Azure machine learning
-
designer has several pre-built
-
components that can be used to prepaid
-
data for training these components
-
enable you to clean data normalize
-
features join tables and and more
-
let's come to uh training so to train a
-
classification model you need a data set
-
that includes historical features so the
-
characteristics of the entity for which
-
one to make a prediction and known label
-
values the label is the class indicator
-
we want to train a model to predict it
-
and it's common practice to train a
-
model using a subset of the data while
-
holding back some data with which to
-
test the train model and this enables
-
you to compare the labels that the model
-
predicts with the actual known labels in
-
the original data set
-
this operation can be performed in the
-
designer using the split data component
-
as shown by the screenshot here in the
-
in the deck
-
there's also another component that you
-
should use which is the score model
-
component to generate the predicted
-
class label value using the validation
-
data as input so once you connect all
-
these components
-
um the component specifying the the
-
model we are going to use the split data
-
component the trained model component
-
and the score model component you want
-
to run an a new experiment in the
-
initial map which will use the data set
-
on the canvas to train and score a model
-
after training a model it is important
-
we say to evaluate its performance to
-
understand how bad how how good sorry
-
our model is performing
-
and there are many performance metrics
-
and methodologies for evaluating how
-
well a model makes predictions the
-
component to use to perform evaluation
-
in Azure ml designer is called as
-
intuitive as it is evaluate model
-
once the job of training and evaluation
-
of the model is completed you can review
-
evaluation metrics on the completed job
-
Page by right clicking on the component
-
in the evaluation results you can also
-
find the so-called confusion Matrix that
-
you can see here in the right side of of
-
this deck
-
a confusion Matrix shows cases where
-
both the predicted and actual values
-
were one uh the so-called true positives
-
at the top left and also cases where
-
both the predicted and the actual values
-
were zero the so-called true negatives
-
at the bottom right while the other
-
cells show cases where the predicting
-
and actual values differ
-
called false positive and false
-
negatives and this is an example of a
-
confusion Matrix for a binary classifier
-
why for a multi-class classification
-
model the same approach is used to
-
tabulate each possible combination of
-
actual and predictive value counts so
-
for example a model with three possible
-
classes would result in three times
-
three Matrix
-
the confusion Matrix is also useful for
-
the metrics that can be derived from it
-
like accuracy recall or precision
-
um we we say that the last step is
-
deploying the train model to a real-time
-
endpoint as a predictive service and in
-
order to automate your model into
-
service that makes continuous
-
predictions you need first of all to
-
create any and and then deploy an
-
inference pipeline the process of
-
converting the training pipeline into a
-
real-time inference pipeline removes
-
training components and adds web service
-
inputs and outputs to handle requests
-
and the inference pipeline performs they
-
seem that the transformation as the
-
first pipeline but for new data then it
-
uses the train model to infer or predict
-
label values based on its feature
-
um so I think I've talked a lot for now
-
I would like to let John show us
-
something in practice uh with uh with
-
the Hands-On demo so please John go
-
ahead sharing your screen and guide us
-
through this demo of creating a
-
classification with the Azure machine
-
learning designer
-
uh thank you so much Carlotta for this
-
interesting explanation of the Azure ml
-
designer and now
-
um I'm going to start with you in the
-
Practical demo part so uh if you want to
-
follow along go to the link that Carlota
-
sent in the chat so like you can do
-
the demo or the Practical part with me
-
I'm just going to share my screen
-
and
-
go here so uh
-
where am I right now I'm inside the
-
Microsoft learn documentation this is
-
the exercise part of this module and we
-
will start by setting two things which
-
are a prequisite for us to work inside
-
this module which are the users group
-
and the Azure machine learning workspace
-
and something extra which is the compute
-
cluster that calendar Target about so I
-
just want to make sure that you all have
-
a resource Group created inside your
-
auditor inside your Microsoft Azure
-
platform so this is my research group
-
inside this is this Resource Group I
-
have created an Azure machine learning
-
workspace so I'm just going to access
-
the workspace that I have created
-
already from this link I am going to
-
open it which is the studio web URL and
-
I will follow the steps so what is this
-
this is your machine learning workspace
-
or machine Learning Studio you can do a
-
lot of things here but we are going to
-
focus mainly on the designer and the
-
data and the compute so another
-
prequisite here as Carlotta told you and
-
we need some resources to power up the
-
the classification the processes that
-
will happen
-
so we have created this Computing
-
cluster
-
and we have like Set uh some presets for
-
it so
-
where can you find this preset you go
-
here under the create compute you'll
-
find everything that you need to do so
-
the size is the Standard ds11 Version 2
-
and it's a CPU not GPU because we don't
-
know the GPU and we don't need a GPU and
-
a like it is ready for us to use
-
the next thing which we will look into
-
is the designer how can you access the
-
designer
-
you can either click on this icon or
-
click on the navigation menu and click
-
on the designer for me
-
um
-
now I am inside my designer
-
what we are going to do now is the
-
pipeline that Carlotta told you about
-
and from where can I know these steps if
-
you follow along in the learn module you
-
will find everything that I'm doing
-
right now in details uh with screenshots
-
of course so I'm going to create a new
-
pipeline and I can do so by clicking on
-
this plus button
-
it's going to redirect me to the
-
designer authoring the pipeline uh where
-
I can drag and drop data and components
-
that the Carlota told you the difference
-
between
-
and here I am going to do some changes
-
to the settings I am going to connect
-
this with my compute cluster that I
-
created previously so I can utilize it
-
from here I'm going to choose this
-
compute cluster demo that I have showed
-
you before in the Clusters here
-
and I am going to change the name to
-
something more meaningful instead of
-
byline and the date of today I'm going
-
to name it diabetes
-
and
-
let's just check this training
-
let's say training 0.1 or okay
-
and I am going to close this tab and in
-
order to have a bigger place to work
-
inside because this is where we will
-
work where everything will happen so I
-
will click on close from here
-
and I will go to the data and I will
-
create a new data set
-
how can I create a new data set there is
-
multiple options here you can find from
-
local files from data store from web
-
files from open data set but I'm going
-
to choose from web files as this is the
-
way we're going to create our data
-
from here the information of my data set
-
I'm going to get them from the Microsoft
-
learn module so if we go to the step
-
that says create a data set
-
under it it illustrates that you can
-
access the data from inside the asset
-
library and inside your asset liability
-
you'll find the data and find the
-
component and I'm going to select
-
this link because this is where my data
-
is stored if you open this link you will
-
find this is this is a CSV file I think
-
yeah and you can like all the data are
-
here
-
all right now let's get back
-
um
-
[Music]
-
and you are going to do something
-
meaningful but because I have already
-
created it before twice so I'm gonna
-
like add a number to the name
-
uh the data set is tabular and there is
-
the file but this is a table so we're
-
going to choose the table
-
[Music]
-
for data set time
-
now we will click on next that's gonna
-
review or uh display for you the content
-
of this file that you have
-
like imported to this workspace
-
and for these settings these are like
-
related to our filed format
-
so this is a delimited file and it's not
-
plain text it's not a Json the delimiter
-
is comma as like we have seen that they
-
those
-
so I'm choosing
-
errors because the only the first five
-
[Music]
-
for example okay uh if you have any
-
doubts if you have any problems please
-
don't hesitate to all right through me
-
in the chat
-
and like what what is blocking you and
-
me and Carlota will try to help you and
-
like whenever possible
-
and now this is the new preview for my
-
data set I can see that I have an ID I
-
have patient ID I have pregnancies I
-
have the age of the people
-
have the body mass together I think
-
and they have diabetical or not as a
-
zero and one zero indicates a negative
-
the person doesn't have diabetes and one
-
indicates a positive that this person
-
has diabetes okay
-
now I'm going to click on next here I am
-
defining my schema all the data types
-
inside my columns the column names which
-
columns to include which to exclude and
-
here we will include everything except
-
the path of the bath color and we are
-
going to review the data types of each
-
column so let's review this first one
-
this is numbers numbers then it's the
-
integer and this is
-
um like decimal
-
dotted
-
decimal number so we are going to choose
-
this data type
-
and for this one
-
it says diabetic and it's a zero under
-
one and we are going to make it as
-
integerables
-
now we are going to click on next and
-
move to reviewing everything this is
-
everything that we have defined together
-
I will click on create
-
and
-
now the first step has ended we have
-
gotten our data ready
-
now what now we're going to utilize the
-
designer
-
um Power we're going to drag and drop
-
our data set to create the pipeline
-
so I have like click on it and drag it
-
to this space it's gonna appear to you
-
and we can inspect it by right click and
-
choose preview data
-
to see what we have created together
-
from here you can see everything that we
-
have like seen previously but in more
-
details and we are just going to close
-
this now what now we are gonna do the
-
processing that Carlota like mentioned
-
these are some instructions about the
-
data about how you can loot them how you
-
can open them but we are going to move
-
to the transformation or the processing
-
so as Carlotta told you like any data
-
for us to work on we have to do some
-
processing to it
-
to make it easy easier for the model to
-
be trained and easier to work with so uh
-
we're gonna do the normalization and
-
normalization meaning is uh
-
to scale our data either down or up but
-
we're going to scale them down
-
and like we are going to decrease and
-
relatively decrease
-
the the values all the values to work
-
with lower numbers and if we are working
-
with larger numbers it's going to take
-
more time if we're working with smaller
-
numbers it's going to take less time to
-
calculate them and that's it so
-
where can I find the normalized data I
-
can find it inside my component
-
so I will choose the component and
-
search for normalized data
-
I will drag and drop it as usual and I
-
will connect between these two things
-
by clicking on this spot this like
-
Circle and
-
drag and drop until the next circuit
-
now we are going to Define our
-
normalization method
-
so I'm going to double click on the
-
normalized data
-
it's going to open the settings for the
-
normalization
-
as better transformation method which is
-
a mathematical way
-
that is going to scale our data
-
according to
-
we're going to choose min max and for
-
this one we are going to choose use 0
-
for constant column we are going to
-
choose true
-
and we are going to Define which columns
-
to normalize so we are not going to
-
normalize the whole data set we are
-
going to choose a subset from the data
-
set to normalize so we're going to
-
choose everything except for the patient
-
ID and the diabetic because the patient
-
ID is a number but it's a categorical
-
data it describes a vision it's not a
-
number that I can sum I can say patient
-
ID number one plus patient ID number two
-
no this is a patient and another
-
location it's not a number that I can do
-
mathematical operations on so I'm not
-
going to choose it so we will choose
-
everything as I said except for the
-
diabetic and the patient might I will
-
click on Save
-
and it's not showing me a warning again
-
everything is good
-
now I can click on submit
-
and review my normalization output
-
um
-
so uh if you click on submit here
-
and you will like choose create new and
-
set the name that is mentioned here
-
inside the notebook so it it tells you
-
to like create a job and name it name
-
the experiment Ms learn diabetes
-
training because you will continue
-
working on and building component later
-
I have it already like created I am the
-
like we can review it together so uh let
-
me just open this in another tab I think
-
I have it
-
here
-
okay
-
so these are all the jobs that I have
-
read them
-
all the jobs there let's do this over
-
these are all the jobs that I have
-
submitted previously
-
and I think this one is the
-
normalization job so let's see the
-
output of it
-
as you can see it says uh check mark yes
-
which means that it worked and we can
-
preview it how can I do that right click
-
on it choose preview data
-
and as you can see all the data are
-
scaled down
-
so everything is between zero
-
and uh one I think
-
so like everything is good for us now we
-
can move forward to the next step
-
which is to create the whole pipeline so
-
uh Carlota told you that
-
we're going to use a classification
-
model to create this data set so uh let
-
me just drag and drop everything
-
to get runtime and we're doing
-
about about everything by
-
so
-
as a result we are going to explain
-
yeah so I'm going to give this split
-
data I'm going to take the
-
transformation data to split data and
-
connect it like that
-
I'm going to get a three model
-
components because I want to train my
-
model
-
and I'm going to put it right here
-
okay
-
like let's just move it down there okay
-
and we are going to use a classification
-
model
-
a two class
-
logistic regression model
-
so I'm going to give this algorithm to
-
enable my model to work
-
this is the untrained model this is
-
here
-
the left the left
-
the left like Circle I'm going to
-
connect it to the data set and the right
-
one we are going to connect it to
-
evaluate model
-
evaluate model so let's search for
-
evaluate model here
-
so because we want to do what we want to
-
evaluate our model and see how it it has
-
been doing it is it good is it bad
-
um sorry like
-
this is
-
down there
-
after the school
-
so we have to get the score model first
-
so let's get it
-
and this will take the trained model and
-
the data set
-
to score our model and see if it's
-
performing good or bad
-
and
-
um
-
after that like we have finished
-
everything now we are going to do the
-
what
-
the presets for everything
-
as a starter we will be splitting our
-
data so
-
how are we going to do this according to
-
what to the split rules so I'm going to
-
double click on and choose split rows
-
and the percentage is
-
70 percent for the and 30 percent of the
-
data for
-
the valuation or for the scoring okay
-
I'm going to make it a randomization so
-
I'm going to split data randomly and the
-
seat is uh
-
132 23 I think yeah
-
and I think that's it
-
the split says why this holes and that's
-
good
-
now for the next one which is the train
-
model we are going to connect it as
-
mentioned here
-
and like we have done that and then why
-
am I having here like let's double click
-
on it yeah it has like it needs the
-
label column that I am trying to predict
-
so from here I'm going to choose
-
diabetic I'm going to save
-
I'm going to close this one
-
so it says here
-
the diabetic label the model it will
-
predict the zero and one because this is
-
a binary classification algorithm so
-
it's going to predict either this or
-
that
-
and
-
um
-
I think that's everything to run the the
-
pipeline
-
so everything is done everything is good
-
for this one we're just gonna leave it
-
like for now because this is the next
-
step
-
um this will like be put instead of the
-
score model but then it's
-
delete it for now
-
okay
-
now we have to submit the job in order
-
to see the output of it so I can click
-
on submit and choose the previous job
-
which is the one that I have showed you
-
before
-
and then let's review its output
-
together here
-
so if I go to the jobs
-
if I go to Ms learn maybe it is training
-
I think it's the one that lasted the
-
longest this one here
-
so here I can see
-
the job output what happened inside
-
the model as you can see
-
so the normalization we have like seen
-
before the split data I can preview it
-
the result one or the result two as it
-
splits the data to 70 here and three
-
thirty percent here
-
um I can see the score model which is
-
like something that we need
-
to review
-
um inside the scroll model uh like from
-
here
-
we can see that
-
let's get back here
-
like this is the data that the model has
-
been scored and this is a scoring output
-
so it says code label true and if he is
-
not diabetic so this is
-
um
-
around prediction let's say
-
for this one it's true and true and this
-
is like a good like what do you say
-
prediction and the probabilities of this
-
score
-
which means the certainty of our model
-
of that this is really true it's 80 for
-
this one is 75
-
so these are some cool metrics that we
-
can review to understand how our model
-
is performing it's performing good for
-
now
-
let's check our evaluation model
-
so this is the extra one that I told you
-
about instead of the like
-
score model only we are going to add
-
what evaluate model
-
after it so here
-
we're going to go to our asset library
-
and we are going to choose the evaluate
-
model
-
and we are going to put it here and we
-
are going to connect it and we are going
-
to submit the job using the same name of
-
the job that we used previously
-
let's review it uh also so after it
-
finishes you will find it here so I have
-
already done it before this is how I'm
-
able to see the output
-
so let's see
-
what what is the output of this
-
evaluation process
-
here it mentioned to you that there are
-
some metrics
-
like the confusion Matrix which Carlotta
-
told you about there is the accuracy the
-
Precision the recall and F1 School
-
every Matrix gives us some insight about
-
our model it helps us to understand it
-
more more and
-
like understand if it's overfitting if
-
it's good if it's bad and really really
-
like understand how it's working
-
now I'm just waiting for the job to load
-
until it loads
-
um
-
we can continue to
-
to work on our
-
model so I will go to my designer I'm
-
just going to confirm this
-
and I'm going to continue working on it
-
from
-
where we have stopped where have we
-
stopped
-
we have stopped on the evaluate model so
-
I'm going to choose this one
-
and it says here
-
select experiment create inference
-
pipeline so
-
I am going to go to the jobs
-
I'm going to select my experiment
-
I hope this works
-
okay salute finally now we have our
-
evaluate model output
-
let's previews evaluation results
-
and uh
-
cool come on
-
finally now we can create our inference
-
pipeline so
-
I think it says that
-
um
-
select the experiment then select Ms
-
learn so
-
I am just going to select it
-
and finally now we can the ROC curve we
-
can see it that the true positive rate
-
and the force was integrate the false
-
positive rate is increasing with time
-
and also the true positive rate true
-
positive is something that it predicted
-
that it is uh positive it has diabetes
-
and it's really a it's really true it
-
the person really has diabetes okay and
-
for the false positive it predicted that
-
someone has diabetes and someone doesn't
-
has it this is what true position and
-
false positive means this is The Recoil
-
curve so we can like review the metrics
-
of our model this is the lift curve I
-
can change the threshold of my confusion
-
Matrix here
-
and this could look don't want to add
-
anything about the the the graphs and
-
you can do so
-
um
-
yeah so just wanted to if you go yeah I
-
just wanted to comment comment for the
-
RSC curve uh that actually from this
-
graph the metric which uh usually we're
-
going to compute is the end area under
-
under the curve and this coefficient or
-
metric
-
um it's a confusion
-
um is a value that could span from from
-
zero to one and the the highest is
-
um
-
this the highest is the the score so the
-
the closest one
-
um so the the highest is the amount of
-
area under this curve
-
um the the the highest performance uh we
-
we've got from from our model and
-
another thing is what John is
-
um playing with so this threshold for
-
the logistic
-
regression is the threshold used by the
-
model
-
um to
-
um
-
to predict uh if the category is zero or
-
one so if the probability the
-
probability score is above the threshold
-
then the category will be predicted as
-
one while if the the probability is
-
below the threshold in this case for
-
example 0.5 the category is predicted as
-
as zero so that's why it's very
-
important to um to choose the the
-
threshold because the performance really
-
can vary
-
um
-
with this threshold value
-
uh thank you uh so much uh kellota and
-
as I mentioned now we are going to like
-
create our inference pipeline so we are
-
going to select the latest one which I
-
already have it opened here this is the
-
one that we were reviewing together this
-
is where we have stopped and we're going
-
to create an inference pipeline we are
-
going to choose a real-time inference
-
pipeline okay
-
um from where I can find this here as it
-
says real-time inference pipeline
-
so it's gonna add some things to my
-
workspace it's going to add the web
-
service inboard it's going to have the
-
web service output because we will be
-
creating it as a web service to access
-
it from the internet
-
uh what are we going to do we're going
-
to remove this diabetes data okay
-
and we are going to get a component
-
called Web
-
input and what's up let me check
-
it's enter data manually
-
we have we already have the with input
-
present
-
so we are going to get the entire data
-
manually
-
and we're going to collect it to connect
-
it as it was connected before like that
-
and also I am not going to directly take
-
the web service sorry escort model to
-
the web service output like that
-
I'm going to delete this
-
and I'm going to execute a python script
-
before
-
I display my result
-
so
-
this will be connected like okay but
-
so
-
the other way around
-
and from here I am going to connect this
-
with that and there is some data uh that
-
we will be getting from the node or from
-
the the examination here and this is the
-
data that will be entered like to our
-
website manually okay this is instead of
-
the data that we have been getting from
-
our data set that we created so I'm just
-
going to double click on it and choose
-
CSV and I will choose it has headers
-
and I will take or copy this content and
-
put it there okay
-
so let's do it
-
I think I have to click on edit code now
-
I can click on Save and I can close it
-
another thing which is the python script
-
that we will be executing
-
um yeah we are going to remove this also
-
we don't need the evaluate model anymore
-
so we are going to remove
-
script that I will be executing okay
-
I can find it here
-
um
-
yeah
-
this is the python script that we will
-
execute and it says to you that this
-
code selects only the patient's ID
-
that's correct label the school
-
probability and return returns them to
-
the web service output so we don't want
-
to return all the columns as we have
-
seen previously
-
uh the determines everything
-
so
-
we want to return certain stuff the
-
stuff that we will use inside our
-
endpoint so I'm just going to select
-
everything and delete it and
-
paste the code that I have gotten from
-
the uh
-
the Microsoft learn Docs
-
now I can click on Save and I can close
-
this
-
let me check something I don't think
-
it's saved it's saved but the display is
-
wrong okay
-
and now I think everything is good to go
-
I'm just gonna double check everything
-
so uh yeah we are gonna change the name
-
of this uh
-
Pipeline and we are gonna call it
-
predict
-
diabetes okay
-
now let's close it and
-
I think that we are good to go so
-
um
-
okay I think everything is good for us
-
I just want to make sure of something is
-
the data is correct the data is uh yeah
-
it's correct
-
okay now I can run the pipeline let's
-
submit
-
select an existing Pipeline and we're
-
going to choose the MS layer and
-
diabetes training which is the pipeline
-
that we have been working on
-
from the beginning of this module
-
I don't think that this is going to take
-
much time so we have submitted the job
-
and it's running
-
until the job ends we are going to set
-
everything
-
and for deploying a service
-
in order to deploy a service okay
-
um
-
I have to have the job ready so
-
until it's ready or you can deploy it so
-
let's go to the job the job details from
-
here okay
-
and until it finishes
-
Carlotta do you think that we can have
-
the questions and then we can get back
-
to the job I'm deploying it
-
yeah yeah yeah so yeah yeah guys if you
-
have any questions
-
uh on on what you just uh just saw here
-
or into introductions feel free this is
-
a good moment we can uh we can discuss
-
now while we wait for this job to to
-
finish
-
uh and the
-
can can
-
we have the energy check one or like
-
what do you think uh yeah we can also go
-
to the knowledge check
-
um
-
yeah okay so let me share my screen
-
yeah so if you have not any questions
-
for us we can maybe propose some
-
questions to to you that you can
-
um
-
uh to check our knowledge so far and you
-
can uh maybe answer to these questions
-
uh via chat
-
um so we have do you see my screen can
-
you see my screen
-
yes
-
um so John I think I will read this
-
question loud and ask it to you okay so
-
are you ready to transfer
-
yes I am
-
so
-
um you're using Azure machine learning
-
designer to create a training pipeline
-
for a binary classification model so
-
what what we were doing in our demo
-
right and you have added a data set
-
containing features and labels uh a true
-
class decision Forest module so we used
-
a logistic regression model our
-
um in our example here we're using A2
-
class decision force model
-
and of course a trained model model you
-
plan now to use score model and evaluate
-
model modules to test the train model
-
with the subset of the data set that
-
wasn't used for training
-
but what are we missing so what's
-
another model you should add and we have
-
three options we have join data we have
-
split data or we have select columns in
-
in that set
-
so
-
um while John thinks about the answer uh
-
go ahead and
-
um
-
answer yourself so give us your your
-
guess
-
put in the chat or just come off mute
-
and announcer
-
a b yes
-
yeah what do you think is the correct
-
answer for this one I need something to
-
uh like I have to score my model and I
-
have to evaluate it so I I like I need
-
something to enable me to do these two
-
things
-
I think it's something you showed us in
-
in your pipeline right John
-
of course I did
-
uh we have no guests yeah
-
can someone like someone want to guess
-
uh we have a b yeah
-
uh maybe
-
so uh in order to do this in order to do
-
this I mentioned the
-
the module that is going to help me to
-
to divide my data into two things 70 for
-
the training and thirty percent for the
-
evaluation so what did I use I used
-
split data because this is what is going
-
to split my data randomly into training
-
data and validation data so the correct
-
answer is B and good job eek thank you
-
for participating
-
next question please
-
yes
-
answer so thanks John
-
uh for uh explaining us the the correct
-
one
-
and we want to go with question two
-
yeah so uh I'm going to ask you now
-
karnata you use Azure machine learning
-
designer to create a training pipeline
-
for your classification model
-
what must you do before you deploy this
-
model as a service you have to do
-
something before you deploy it what do
-
you think is the correct answer
-
is it a b or c
-
share your thoughts without in touch
-
with us in the chat and
-
um and I'm also going to give you some
-
like minutes to think of it before I
-
like tell you about
-
yeah so let me go through the possible
-
answers right so we have a uh create an
-
inference pipeline from the training
-
pipeline
-
uh B we have ADD and evaluate model
-
module to the training Pipeline and then
-
three we have uh clone the training
-
Pipeline with a different name
-
so what do you think is the correct
-
answer a b or c
-
uh also this time I think it's something
-
we mentioned both in the decks and in
-
the demo right
-
yes it is
-
it's something that I have done like two
-
like five minutes ago
-
it's real time real time what's
-
um
-
yeah so think about you need to deploy
-
uh the model as a service so uh if I'm
-
going to deploy model
-
um I cannot like evaluate the model
-
after deploying it right because I
-
cannot go into production if I'm not
-
sure I'm not satisfied over my model and
-
I'm not sure that my model is performing
-
well
-
so that's why I would go with
-
um
-
I would like exclude B from from my from
-
my answer
-
uh while
-
um thinking about C uh I don't see you I
-
didn't see you John cloning uh the
-
training Pipeline with a different name
-
uh so I I don't think this is the the
-
right answer
-
um while I've seen you creating an
-
inference pipeline uh yeah from the
-
training Pipeline and you just converted
-
it using uh a one-click button right
-
yeah that's correct so uh this is the
-
right answer
-
uh good job so I created an inference
-
real-time Pipeline and it has done it
-
like it finished it finished the job is
-
finished so uh we can now deploy
-
ment
-
yeah
-
exactly like on time
-
I like it finished like two seconds
-
three three four seconds ago
-
so uh
-
until like um
-
this is my job review so
-
uh like this is the job details that I
-
have already submitted it's just opening
-
and once it opens
-
um
-
like I don't know why it's so heavy
-
today it's not like that usually
-
yeah it's probably because you are also
-
showing your your screen on teams
-
okay so that's the bandwidth of your
-
connection is exactly do something here
-
because yeah finally
-
I can switch to my mobile internet if it
-
did it again so I will click on deploy
-
it's that simple I'll just click on
-
deploy and
-
I am going to deploy a new real-time
-
endpoint
-
so what I'm going to name it I'm
-
description and the compute type
-
everything is already mentioned for me
-
here so I'm just gonna copy and paste it
-
because we like we have we are running
-
out of time
-
so it's all Azure container instance
-
which is a containerization service also
-
both are for containerization but this
-
gives you something and this gives you
-
something else for the advanced options
-
it doesn't say for us to do anything so
-
we are just gonna click on deploy
-
and now we can test our endpoint from
-
the endpoints that we can find here so
-
it's in progress if I go here
-
under the assets I can find something
-
called endpoints and I can find the
-
real-time ones and the batch endpoints
-
and we have created a real-time endpoint
-
so we are going to find it under this uh
-
title so if I like click on it I should
-
be able to test it once it's ready
-
it's still like loading but this is the
-
input and this is the output that we
-
will get back so if I click on test and
-
from here I will input some data to the
-
endpoint
-
which are the patient information The
-
Columns that we have already seen in our
-
data set the patient ID the pregnancies
-
and of course of course I'm not gonna
-
enter the label that I'm trying to
-
predict so I'm not going to give him if
-
the patient is diabetic or not this end
-
point is to tell me this is the end
-
point or the URL is going to give me
-
back this information whether someone
-
has diabetes or he doesn't so if I input
-
these this data I'm just going to copy
-
it and go to my endpoint and click on
-
test I'm gonna give the result pack
-
which are the three columns that we have
-
defined inside our python script the
-
patient ID the diabetic prediction and
-
the probability the certainty of whether
-
someone is diabetic or not based on the
-
uh based on the prediction so that's it
-
and like uh I think that this is really
-
simple step to do you can do it on your
-
own you can test it
-
and I think that I have finished so
-
thank you
-
uh yes we are running out of time I I
-
just wanted to uh thank you John for for
-
this demo for going through all these
-
steps to
-
um create train a classification model
-
and also deploy it as a predictive
-
service and I encourage you all to go
-
back to the learn module
-
um and uh like depend all these topics
-
at your at your own pace and also maybe
-
uh do this demo on your own on your
-
subscription on your Azure for student
-
subscription
-
um and I would also like to recall that
-
this is part of a series of study
-
sessions of cloud skill challenge study
-
sessions
-
um so you will have more in the in the
-
in the following days and this is for
-
you to prepare let's say to to help you
-
in taking the a cloud skills challenge
-
which collect
-
a very interesting learn module that you
-
can use to scale up on various topics
-
and some of them are focused on AI and
-
ml so if you are interested in these
-
topics you can select these these learn
-
modules
-
um so let me also copy
-
um the link the short link to the
-
challenge in the chat uh remember that
-
you have time until the 13th of
-
September to take the challenge and also
-
remember that in October on the 7th of
-
October you have the you can join the
-
student the the student developer Summit
-
which is uh which will be a virtual or
-
in for some for some cases and hybrid
-
event so stay tuned because you will
-
have some surprises in the following
-
days and if you want to learn more about
-
this event you can check the Microsoft
-
Imaging cap Twitter page and stay tuned
-
so thank you everyone for uh for joining
-
this session today and thank you very
-
much Sean for co-hosting with with this
-
session with me it was a pleasure
-
thank you so much Carlotta for having me
-
with you today and thank you like for
-
giving me this opportunity to be with
-
you here
-
great I hope that uh yeah I hope that we
-
work again in the future sure I I hope
-
so as well
-
um so
-
bye bye speak to you soon
-
bye