- Thanks, Jennifer.
All right, so it is exactly four o'clock,
so I will go ahead
and get us started to make
sure we start on time.
Good afternoon, everyone.
Thank you so much for joining us today.
My name is Frances Barreto.
I am one of the district success managers
that support our amazing
partners in Arizona.
And you do have a dedicated
team here to support you
as you see on the screen.
That includes Chelsea Hatchard,
who's our district partnership manager,
Sheryl Dossola supports as
the rostering specialist.
We have Victoria Cheng,
who is another district success manager,
and then Jennifer
Cummings and India Quarles
are your professional
learning specialists.
We are especially excited
to have Sarah Robertson
from the Khan Academy Writing
Coach team with us today,
and she's gonna be sharing
how you can help your
students become more confident
and skilled writers using
Khanmigo's Writing Coach tool.
This is the first
out of several professional
learning opportunities
that you're going to be able
to participate in the spring,
so keep an eye out for an email
that should be coming your way March 11th
with the registration links
for our Arizona Spring Learning series.
And remember that if you register,
you'll automatically
receive the recording,
even if you can't attend live.
Please feel free to drop any questions
or thoughts in the chat
throughout the session.
Jennifer and I will be
actively in the chat
and we'll also be sending
the recording your way
after this session as well.
So without further ado,
I'm gonna go ahead and
pass it on to Sarah.
- Thank you, Frances.
- [Frances] You're welcome.
- I'm gonna share my presentation now.
All right, hopefully you can all see
where it says Meet Khanmigo Writing Coach.
Yes, okay, awesome.
I'm gonna try and have
the chat open as well
so I can keep an eye on
what you all are saying.
Welcome, everyone.
I'm really excited to be here.
A little bit about myself.
My name is Sarah Robertson.
I am a principal product
manager here at Khan Academy,
focusing on literacy experiences
and classroom tools to support ELA
and humanities teachers specifically.
And some background of
sort of where I came from.
I was a middle school
ELA teacher in Boston.
That's a picture of me when
I had a lot less gray hair
than I have now.
And when I was a teacher,
I taught seventh grade.
Many of my students were
significantly below grade level.
I taught a variety of humanities classes
during my time in the classroom,
but the hardest of all the classes
that I taught was writing.
I think as a writing teacher,
I assigned 25 different essays
throughout the course of
one single school year.
And like most middle and
high school teachers,
I had about 100 students,
maybe more than that,
so that's about 2,500
distinct essay drafts
for me to guide students
through producing,
provide timely, detailed,
personalized feedback on,
evaluate against a rubric.
I worked easily about 12 hours a day
and nearly every weekend,
and even still, it always
felt like I could never get
to all the students in the class
who were asking for my help.
I could never keep a close enough eye
on the students who I knew needed my help,
but weren't asking for it.
I could never deliver
personalized feedback
to students quickly enough,
even if I tried really hard.
I never really knew about
what my students needed
at any given moment
or how to better support them.
And the burnout from this year
of teaching writing
specifically was so real.
And keep in mind this was before COVID,
before learning loss,
and before generative AI,
so I have a lot of empathy for many of you
who are currently in the classroom.
So today I am excited to share with you
the solution that we built
in response to these very
real classroom challenges.
So we'll talk a bit about
why we built Writing Coach
and our intentions behind its design,
and then I'm gonna dig
into how to actually use it
and show you how to find it
and how to assign things.
I'll show you what the
student experience is like
and what the teacher
and administrator
reports are like as well.
And then I will send you
off and hopefully you can go
and use it and explore it yourselves.
And then I'll actually give you my email
so you can send me the
feedback that you have.
All right, so I had asked
if we could make this
a meeting-type webinar
so you all could be a
little bit more interactive.
So now we have the chat.
Thank you, Aviv, for setting that up.
I would love for you all
to answer this question in the chat.
I'm wondering if you could just name,
what are some things that
you know students need
in order to reach grade-level
proficiency in writing?
Take a minute and think
and put your thoughts into the chat.
Kim says, "Vocabulary, sentence fluency,
punctuation, practice
help with sentence structure,
grammar, punctuation."
Absolutely.
Format of the writing, yes.
Practice and timely feedback, yes.
I'm seeing a lot of practice,
feedback, direct instruction.
Out of curiosity, does anybody know
how much time the National
Commission on Writing recommends
that middle and high school
students should spend
on writing practice?
Let's say each day.
How many minutes per
day do they recommend?
Guess, put it in the chat.
30 minutes.
60 minutes.
30 minutes.
All right, you guys are well informed.
It's 60 minutes per day.
I taught writing,
so in my classroom,
kids were writing a lot of the time
when they weren't watching me model things
or teaching things,
but in many schools,
there isn't a separate writing class
and most students are not
actually getting the 60 minutes
of writing practice per day,
as I'm sure you all are well aware.
Other things that I saw
mentioned that students need,
they need the direct instruction
specifically in things like
the rhetorical situation,
audience, purpose, tone, rhetoric,
text structure, sentence
structure, grammar,
the writing process,
how we begin to write,
how we think about writing,
how to conduct research,
how to incorporate evidence,
how to cite evidence.
They also need things like
exposure to mentor texts,
they need modeling, they need
vocabulary-building practice,
they need feedback
and that feedback has to
be specific and actionable
and it also has to be
delivered not three weeks
from when they wrote the essay.
And then of course, reading
practice and comprehension
is part of that too.
And they also just need to feel motivated,
that's a really big piece.
Again, looking for you
all to go to the chat,
what are some of the things
that make teaching writing
especially challenging?
Go ahead and put your
thoughts in the chat.
Time, yes, for sure.
Yep.
Never enough.
Differentiation.
Yes, Robert.
The feedback piece is,
yes, really challenging.
Motivation, yep, yep.
Definitely time for
providing that feedback,
especially if you have a lot of students.
Grade-level reading, yep.
Exactly, agree.
Texting all the time.
Yeah, productive struggle,
that's a big one.
And I think the motivation
piece plays into that too.
So I spend a lot of my time talking to
and learning from ELA teachers,
and writing is consistently
one of the hardest parts
of the job as I experienced firsthand.
These are some quotes from interviews
that we've done with ELA teachers
and I also happen to grab some screenshots
from a few of the recent comments
in various ELA teacher Facebook
groups that I'm still in.
So yeah, kids are not
writing on grade level
and it's really no wonder why.
A big piece of this, as you all mentioned,
is timely, specific, actional
feedback is so critical,
but with one teacher to 100-plus students,
it's nearly impossible and
teachers are really overwhelmed.
Also, since I've been in the classroom,
which is now, I think,
approaching 10 years ago,
it's gotten a lot harder.
My husband is a writing professor,
he teaches college freshmen,
so I hear from his perspective a lot
about how student AI use has
affected writing instruction
and practice in his classroom.
I also hear this from many of
the teachers that I speak to
through Khan Academy
and I expect that some
of you might have had
some personal experience
dealing with this too.
And there are various things out there.
Somebody here mentions,
"I've seen several AI
checkers online recently.
How accurate are those?"
They're not,
they're not accurate, unfortunately.
We all wish that there was
like one beautiful magical
solution to this problem,
but it has been really challenging
for teachers to be able to spot students
who are using this technology.
And it can be, as I'm sure you know,
very difficult to challenge a student
who you think has produced
something with ChatGPT,
but you don't have the evidence for that.
Okay, for this one,
I wanted to touch upon
another prescient question.
About a year ago I led a workshop
with high school students
on AI and writing
and one of the students said to me,
and this was like a very precocious,
I don't know, 14-year-old girl,
she said, "I just don't see the point
in learning to write anymore,
AI can do it for me.
Why do I need to learn to do this thing
that AI can just do for me?"
And I want you to take a minute
and you can answer in the chat,
or if you feel so inclined,
you can come off mute
and just say it out loud,
do you agree or disagree?
Do you think that it's still important?
I feel like this is
probably a biased group,
but do you think it's important
to still learn to write,
for students to still
learn to write, and why?
What would you say to that student?
So yes, I'm gonna pause here,
and if you have something to say,
feel free to come off mute and
just say or put it in chat.
- Hi Sarah, this is Johna Wallace.
I'm gonna say this out loud,
I think that students do
need to learn how to write.
My son works in finance
and he has to be able to
communicate to other people
what he has done with his clients,
and if he cannot write coherently,
that communication is not there.
Also with the writing,
it helps clarify students' thinking
and their ability to
communicate with one another,
so they need to know how to do that.
- 100%.
You said it so perfectly.
Everything you said.
Writing is thinking,
writing is communicating.
Without knowing how to write,
we can't actually sometimes process
what we even think about something.
We don't have that critical muscle,
we're not building that critical muscle
of processing information and
working through information
and communicating it in ways
that reflect our own personal experiences
and our own beliefs.
I came across this Joan
Didion quote a while ago
and it sort of gave me exactly
what I think I was trying to convey
when I was presented with
these kinds of opinions
from various people,
specifically students who were like,
why do I need to do this anymore?
And Joan Didion says,
"I write entirely to find
out what I'm thinking,
what I'm looking at,
what I see and what it means."
And I get chills when
I say that every time
because as somebody who,
I consider myself a writer
and a writing teacher
and somebody who supports
writing teachers now,
and writing is so much more
than the written content, right?
It's not,
the process of writing means something
and it's important
and it has real human value.
And I think as teachers,
it's our jobs not just to
teach students how to write,
but to teach them about
why they still should be
learning to write, right?
There's also a robust body
of evidence that shows
that there are real cognitive
and psychological benefits from writing.
It helps students,
again, process emotions,
process their thinking,
figure out what they feel,
figure out what they believe.
It's a really wonderful tool
that we all have in our
toolbox as human beings.
So I agree,
I don't think that large
language models or AI
is a reason that we should
not teach writing anymore.
I don't think anybody
would be surprised by that.
And if anything, I actually
think it's more important now
than it has ever been,
but it is still very critical
to educate students about
AI and large language models
and to use it ethically and
responsibly in the classroom.
And we could do an entire webinar
specifically on this topic of effective
and ethical AI use in classrooms alone,
but I sort of jotted down a
list of some of the things
to consider when we are
incorporating AI in the classroom.
The first thing is,
is the use of this AI actually
helping students learn
or are we using it as a shortcut?
Will students become reliant on the AI?
And if it's being used as
some sort of scaffolding,
how can we gradually remove
that scaffolding over time?
Do students understand
how the content from the AI
is actually being generated?
Where it comes from,
when to trust it, when
definitely not to trust it,
when to say something about the fact
that they might have used it in some way.
And are they using AI in ways
that allow them to focus
more of their efforts
on higher-order thinking skills
and also on the human
element of writing too?
There are many opinions on these topics,
but I do believe that there are many ways
in which even if we use AI
for things like brainstorming,
I think it depends on the situation,
but in some ways some might argue
that that's actually taking away
some of the really
important thinking skills
that students do when they brainstorm.
It doesn't hurt maybe to have somebody
to bounce ideas off of
or to jumpstart things,
but worlds in which AI
generates a first draft
or AI generates a list of ideas,
we should just be thinking
about what is the AI doing
that a student previously
could have been doing,
and are we taking something
away of value to that student?
Okay.
So with all of that in mind,
let's learn about how and
why we made Writing Coach
and then I'm going to
show you how to use it.
Khan Academy is a well-known company.
It is not necessarily as
well-known for writing and reading
and ELA or humanities as it is for STEM,
but our mission has always been
to provide a free world-class education
to anyone and anywhere,
and we've always wanted to provide
that same level of
support in the humanities.
So when large language models
became broadly available
in late 2022,
we immediately began exploring ways
that we could leverage the technology
to achieve that mission.
And it's probably no
surprise that we found LLMs
to be especially helpful in
the writing space specifically.
So like all great products,
Writing Coach was also designed
as a solution to some very real problems.
So what we know about writing
is that, of course, students are expected
to meet grade-level standards.
They need to be writing
60 minutes per day.
They need support,
not just at the end of the writing process
when they've produced a
draft and they need feedback,
but they actually need
help getting started,
they need help outlining,
they need help drafting,
and they need to learn
how to do those different
parts of the process.
And then, of course, as we all mentioned,
they need specific, actionable,
personalized, timely
feedback in order to improve.
But in reality, only about a
quarter of secondary students
meet grade-level writing standards.
The vast majority of students
are not clocking even close
to their recommended
hour of practice a day.
Most teachers have way more students
than they can give personalized
writing process support to.
And I mention this math set all the time.
If the average teacher has 100 students
and limits herself to 10
minutes of feedback per essay,
which is something I would do,
I would set my timer 10 minutes per essay,
that's 17 hours for one
draft of one assignment
for me to give feedback to every kid,
17 hours for one assignment.
And if kids are writing an hour a day,
I just don't,
the math isn't mathing,
as the kids say these days,
it's just not really possible.
So in 2023, we started
working on a solution
and then last fall we
launched Writing Coach
to our school and district partners.
And when I talk about Writing Coach,
I think oftentimes
people hear AI, writing,
and they sort of have
a picture in their head
of what this is.
So I like to kind of explain what it is
against what it's not
and how it's different from
some of the other things
that people might be familiar with.
So you might be familiar with tools,
for example, like Grammarly.
I use Grammarly,
Grammarly is great.
Grammarly gives AI-generated
feedback to students,
but it typically provides more sentence
or phrase-level suggestions
and allows them to click a
button and accept the suggestion
without really having to figure out
how to change it themselves.
You might be familiar with other tools
who can kind of generate feedback quickly
for the teacher using AI
or even for the students.
Some of these things
are paid, some are free,
but usually the feedback
is rather limited,
there's not that much feedback,
or the teacher gets the feedback,
but then has to figure out
how to get it to the student.
And to this day, I haven't
really found anybody else
who guides students through
the actual writing process
stage by stage in the way that we do.
And if you're familiar with ChatGPT,
which I'm assuming all of you are,
it's obviously always more than happy
to just rewrite a student's essay for them
or produce it from scratch.
And there's positive use cases
for all of these different kinds of tools,
but the primary purpose behind them
is usually either to save time
or to produce better writing,
not to produce better writers.
And that is really how
Writing Coach is different.
The intention behind Writing
Coach is to help teachers
create better writers and
not just better writing.
So it is an instructional tool
with the primary purpose
of student learning
and teacher transparency.
It's a tool designed to guide students
through the full essay-writing process
while preventing cheating.
It's a way for teachers to give students
more writing practice
and real-time support.
And we have data dashboards for teachers
that have at-a-glance
and in-depth insights
into their students' writing processes.
So it's not just a digital
essay-writing platform,
it's not something that grades
or evaluate student writing,
it gives feedback, it doesn't score,
and that's really on purpose
because I could talk about
this for a long time,
but I think if we're telling students
that their voices matter,
we should have a human
reading what they have to say
at some point.
It's not a student productivity tool,
it's not a feedback generator,
and it's not a tool that provides
easy-to-accept suggestions
for improving things like
grammar and mechanics.
So just really quick,
we launched Writing Coach now
about six months ago or so
and the feedback that we've gotten
has been very, very positive
from the teachers and the
students who have used it so far.
We had one teacher talk
about how the feedback
that Khanmigo generated
was very, very close
to what they would have written themselves
on student writing.
In some cases it was more
detailed or more specific.
Another teacher said that they used this
for an IB essay prep course
and they had seen this
make the biggest difference
in helping students with that essay
than any any tool they've ever used.
We had one teacher in a district
who used this consistently
and her class had every
single student pass
that state writing assessment.
And so we were really
excited about what this is
and how it works.
So let's just dive into
what it looks like.
Okay, so the first step
in using Writing Coach
is to create a Writing Coach assignment,
and a Writing Coach assignment
is an essay assignment.
To begin, you should start by
logging in to Khan Academy.
So you can access Writing Coach
directly through Khan Academy.
As you can see here in the top
of every Khan Academy screen,
you see a Khanmigo drop-down
and you can click on Writing
Coach and get there that way.
And if you're logged in
to a Khan Academy account,
you'll be logged in to
a Writing Coach account,
Writing Coach is part of Khan Academy,
but you can also just go
to khanmigo.ai/writingcoach
and that will also take you
directly to Writing Coach.
Once you are in Writing
Coach as a teacher,
you will see three tabs on the left,
one says Assignments,
one says My Essays,
and one says About Writing Coach.
Assignments, self-explanatory,
that is where you create
assignments for students.
The My Essays space is kind
of like a playground for you
to play around with sample
essays that we've created.
You can test out or demo essays
that you might have
assigned to your students
so that you can model
how to use it for the
first time with them.
But to access your assignments
and create assignments,
you would do that from
your Assignments tab.
And you just should be looking
for this Create Assignment button
in the top right of the screen.
So a Writing Coach essay
assignment starts with a prompt
and it should be a prompt
that you are providing.
The essay instructions, I
can't emphasize this enough,
are the most important part.
The more details you include,
the more of a stickler
Khanmigo will be when, they,
Khanmigo is guiding students
through the writing process.
So if you say things like,
make sure in your instructions,
make sure you include three
pieces of textual evidence,
Khanmigo will take that information
and check that the student
has that amount of evidence
whenever it's giving feedback.
If you say you need to
use MLA in-text citations
or APA in-text citations,
Khanmigo will check to make
sure that students did that.
If you say that they need to have
a concession/rebuttal paragraph
in their argumentative essay,
again, Khanmigo will have that context
and then make sure students
are meeting those requirements.
Also, if you are doing an essay
that is in order to prep
students for the ACT
or another test, an AP test or something,
you can say in the
instruction specifically,
this is an AP practice test
or this is an ACT practice essay,
and that also helps Khanmigo understand,
okay, now I have all of the
information in the world
about what makes a great ACT essay
and I know what an exemplary
ACT essay looks like
and I know what the rubric looks like.
Just telling Khanmigo
this is an ACT essay,
Khanmigo will take that context
and tailor its interactions accordingly.
One thing I do wanna point out,
'cause this is a common
source of confusion,
the grade level drop-down here,
you can select any grade level you want.
Right now we have fifth/sixth,
which is kind of like a beta level
because it will still be
a little bit advanced.
And then we have the seventh
all the way through 12th.
We use this to get Khanmigo
to tailor the language that it's using
when it's chatting with students,
but also to tailor the
feedback that it gives.
So if you select fifth/sixth,
it might give less feedback
or less harsh feedback
than if you select 12th
grade for the same essay.
And that's, again, in order to make sure
that Khanmigo is leveling
itself appropriately.
Students will not see the
grade level that you select.
So if you teach 11th graders
and it's the beginning of the year
and you're kind of building things up,
you can select seventh
grade as their level
if you really want the feedback
to be a little bit more just broken down
or to kind of build them up
to the higher grade levels of
feedback throughout the year.
So that's a scaffolding
tool that I like to mention
because it's not always clear.
Students won't see the grade level,
so they're not gonna feel demoralized
if you choose a level that's below
the level they're actually in.
Another thing I like to tell people.
So when you are designing
your essay instructions,
again, for the assignment,
things that you can use are
details from the actual rubrics
that you will be using
or that are relevant.
So for example, if
you're preparing students
for the Arizona six through
eight argumentative essay,
you're assigning something,
you can use former released
essay prompts, for example.
One thing you can do in your
essay instructions is to say,
this is a writing practice test
for the ASA seventh grade essay.
Students will be evaluated on purpose,
focus and organization,
evidence and elaboration, conventions.
And you can even say things
from that score four,
so you can literally copy and paste.
The response should have a
strongly maintained claim
with little or no,
wait, yeah,
loosely related material.
So you're pulling the exact information
from that score four.
So that means that when
Khanmigo is helping the student
through the essay writing,
drafting and revising and all of that,
it will have that in mind that
those are the expectations
that will make this a great essay.
So if you wanna get more
information in there,
that's usually always very helpful
to make sure that Khanmigo
is preparing students
for the essay.
No, you cannot upload the rubric yet,
that is something that
we are hoping to do,
but instead you can copy and paste details
from this part of the rubric.
Again, you really only need to tell it
this is what you would
need to do to get a four.
You don't need to provide the rest of it
because, again, this
isn't an evaluative tool,
it's not gonna score students,
so it's really about giving them feedback
that will help them get
the highest score possible.
And then for ACT, I think I remember,
am I remembering correctly
that high schools in Arizona use the ACT?
Okay.
You don't have to do that level of detail,
you can just say this
is an ACT practice essay
because Khanmigo knows ACT practice essay
and knows what a great ACT
practice essay looks like,
it knows what will get
you the highest score,
and it will make sure that
it's giving you feedback
and tailoring directions
according to those expectations for ACT.
So once you have created your assignment,
I think the only other details
we ask for are due date
and then you can create a
class if you haven't yet
or assign it to a class you
already have on Khan Academy.
Your students can access it
from their Khan Academy main dashboard
or they can go directly to
Writing Coach themselves.
You'll also, when you
create your assignment,
you'll see a link that
you can share with them,
you can put it in Google
Classroom or Canvas
or whatever you use,
and they can go straight
to the assignment.
Students can also go to
khanmigo.ai/writingcoach
and log in to their Khan
Academy account from there.
But they should be able
to access their assignment
directly in Writing Coach
or from Khan Academy
or from the link that you share.
So let's walk through what
this looks like for students.
On the left here,
students will see the assignment details
that you provided in the
essay instructions field
when you were creating the assignment.
And so they'll see the title of the essay,
they will see the type of
essay that you selected,
and right now we support
argumentative, persuasive,
expository, explanatory,
and literary analysis.
And then we also will be adding soon
AP Language and Composition and SAT.
We don't have an ACT one yet,
but again, you can just choose
persuasive, argumentative,
or anything relevant,
and then you can specifically
say it's an ACT essay
in the instructions.
And then students see the essay prompt
and instructions that you provided
and then they have Khanmigo on the side.
So for understanding,
it's really just about
breaking down the prompt,
making sure the student
understands the requirements,
making sure they can ask questions
about anything that they don't
get that you put in there.
So they can do things like ask Khanmigo
to define specific terms
or break things down,
or even if you provided
a lot of information,
it can kind of help
them search through that
to kind of find specific
answers to their questions.
And they can come back
to this at any point.
So if at any point they're like,
what were those directions again,
they can come back to
this and ask questions,
but they can also access
these instructions
throughout the whole rest
of the writing process.
Okay.
When students are ready to get started,
and some additional information
that I don't think I mentioned earlier
is that if in cases
where the essay requires
that students do readings
or they have like the source packet
or they have a text
that they are supposed to
be gathering evidence from,
that is something that
they can either do that
before they start their
Writing Coach assignment
or they can do it between
understanding and outlining.
Outlining works best,
I mean, it can actually help them
with making sure they're
selecting relevant evidence,
but it works best if they've
already read the text
or they've already looked at the text
and they're familiar with the topic.
And during the understanding phase,
Khanmigo will kind of
nudge them to figure out,
have you read the text yet?
Do you need to go back?
And it will kind of give them
advice for what to do next
before they start outlining.
But when students are
ready to get started,
this is the basic outline template,
probably looks relatively familiar.
It starts out as a basic
kind of five-paragraph essay,
but we made it flexible
so that students can remove paragraphs,
they can add additional paragraphs,
they can do things like
add additional evidence
or reasoning,
they can add or remove the source fields,
they can add,
I think we have, yeah,
reasoning explanation,
I might have mentioned that,
and move paragraphs around.
So we tried to design it
so that it would support
many different kinds of academic essays.
Even if you have a one-paragraph essay,
an extended open response,
it will work that way as well
as long as you're clear
in the essay instructions
about the format of what
you want students to do.
For each of the outline templates,
we also provide some pretty
simple exemplar sentences
for each of the fields.
So if you chose persuasive
or argumentative,
we will have an example thesis
related to from like a
persuasive or argumentative essay
of a completely different topic.
They're all Khan Academy-related,
so I don't think we have any
risk of picking the same topic
as you would choose for your students.
But then Khanmigo, again, is on the side
and Khanmigo can help
students get started.
If the student gets started and is stuck,
it's there to help them with that,
Khanmigo can see their outline,
it can check the outline
against the assignment instructions,
again, that the teacher has provided.
So if the student doesn't
know what to do next,
Khanmigo will offer some suggestions,
but Khanmigo will never ever provide
something for the student
to actually put into their outline.
So Khanmigo is not gonna
give them evidence,
it's not going to write a
thesis statement for them,
it's not going to provide
examples or main points
or any of that,
it will look at what the student has
and it will give the
student advice and tips
and guidance for how
they can make progress
and what they're supposed to be doing.
It can also,
oh yes,
when the student is finished outlining,
one of the other kind of
interesting things about this
is you might have some students
who just don't really
wanna talk to Khanmigo
or they don't really feel like chatting
or they don't really know how to chat.
Even if you never talk to Khanmigo,
when the student moves
from outlining to drafting,
we check it anyway.
So we have Khanmigo,
again, look at the outline
and look at the assignment instructions,
and at this step,
we're really just checking
for pretty high-level basic,
not like, is this a well crafted essay?
It's like, does it have
a thesis statement?
And is the thesis
statement actually related
to what the essay prompt is asking?
Does the student have enough evidence
according to what the teacher asked for?
Does it have main points
that actually support
or could reasonably support the thesis?
I sometimes had seventh graders
who would start with a thesis
on one side of an argument,
and then halfway through,
they'd switch sides
and wouldn't even realize
that they did that.
So that's one of the
things that we check for.
And again, if you ask for
something like a counterargument
in your instructions,
we'll check for that here.
And if the student has not met
those very basic requirements,
there will be kind of a
stopping point where we say,
it looks like your teacher asked for this
and you didn't quite do this part,
so it will let them go back
and they can fix what
they didn't do correctly
and then they will move on
to drafting at that point.
During the drafting phase,
we try to keep it pretty simple,
we don't want it to be
too much distraction.
So again, the student can access
the assignment instructions
whenever they need to.
They can also access the
outline that they just wrote
from the previous step,
and from that outline,
they can copy and paste
the different contents
of their essay that they worked on.
They can take a piece of
evidence and put that in
or their thesis statement, et cetera.
They can also just hide this completely
and just have a nice blank
drafting area if they want.
But they can chat with
Khanmigo if they want to.
And again, Khanmigo in the drafting phase
is going to focus on helping
the student get their words
on the paper,
looking at their outline
at the appropriate points,
and pulling the information
that the student has already
written out in their outline
and helping them kind of
begin to craft their arguments
and their points and their prose.
It's not necessarily going to
be giving a lot of feedback
at this point because, again,
the drafting phase is really about
get your words on the paper,
get your thoughts on the paper,
get your ideas on the paper,
and then the revising
stage will come next.
So Khanmigo at this point
is just helping students
get a first draft together
and not too much focusing
on the perfection,
trying to make sure every word is perfect.
And again, Khanmigo won't write for them.
You can see here the
student might ask them to,
they might ask them to nicely,
and Khanmigo will gently, politely decline
and help them kind of work
through whatever it is
they're struggling with.
So when the student has their first draft,
they will at that point move
on to the revising stage.
And in revising, the
first thing that happens
is Khanmigo will generate
feedback in five categories,
which you can see up here.
Introduction and claim is the first one,
then we have evidence and reasoning,
and then there's structure
and organization,
conclusion, and style and tone.
Note that in Writing Coach,
Khanmigo will give students feedback
in all of the same areas
as the AASA and ACT writing rubrics,
but it might be categorized
slightly differently.
So for this grade six
through eight AASA essays
for example,
feedback related to purpose,
focus, and organization
that would fall under either
introduction or structure.
Evidence and elaboration feedback
would fall under evidence and reasoning.
Conventions would fall under
style and tone, et cetera.
And then same for SAT.
Feedback related to
development and support
would be under evidence and reasoning,
organization would be
under structure, et cetera.
So that the labels are different,
but the feedback is the same.
But I think one of the most
important things to know
about the revising stage of Writing Coach
is that it's not just about the feedback.
So of course, in this example,
the student has 20 suggestions
under each of these five categories total
and we let them kind of
focus on one area at a time,
but we don't just give them the feedback
and then say, good luck, hope you do well,
we built in revising tools.
So we give, at some point,
praise or positive feedback,
but for the critical
feedback and the suggestions,
there are actual actionable
steps that students can take
to make the changes on the
draft right there on the page
or ask Khanmigo for follow-up
guidance or support.
So some of the things that
students can do are ask Khanmigo
to give them an example,
and this is, I think, one
of the most popular things
a student would choose to do.
So if Khanmigo says
your introduction doesn't
have enough context
for the reader,
that's, I think, for a very
common piece of feedback
for introduction,
the student can say, give me an example,
and Khanmigo will take an essay
on a completely different topic
and then give them an example
of what it looks like not to have context
and then it would look
like to have that context,
and that we have seen has
been really, really valuable
for students who get a piece of feedback
and they're just like, I don't get it.
So that's very, very helpful.
Then we have explain suggestion,
and that's really helpful.
If Khanmigo is being a little bit wordy,
they can ask it to explain it
and that'll usually result
in Khanmigo breaking it down.
If students are struggling readers,
you can also have them say
after they've explained it,
you say, make it simpler, say it simpler,
just keep kind of pushing
Khanmigo if they need to.
Students can just ask a
general question as well.
But then if they go in
and they change their essay
based on the feedback,
they can then immediately
just say, check my revision,
and Khanmigo will look
at that piece of feedback
and at the change the student just made
and let them know if they've fixed it,
let them know if they've
addressed it or not,
and then Khanmigo will say,
you made a little bit of an improvement,
but you could still make it clear
or you can still kind of provide
a little bit more information here
or give them whatever
follow-up guidance is necessary
until the student has what they need.
And what else?
They also have access
to their outline again
during revising.
And as they are working
through their feedback,
they can mark them as resolved
and keep track of where they're
at in their revision stage.
As their teacher,
and you'll see this in a minute,
you can see how much feedback they got
and how much they've resolved,
so you can keep track
of how close they are
to being done with this second draft.
And when they are finished,
the student can click mark as complete
and that's to let you
know that they're done
and happy with their draft.
And they get the option of exporting it,
so they can export it
to Microsoft Word or PDF
or even save it as a Google doc.
One other thing to note is
that we don't currently support
a work cited page.
Usually for test prep,
that's not a concern,
but for longer research
essays or whatever,
if you want students
to include work cited,
I would have them do that after
they've exported it to a doc
or a Google doc,
they can add the work cited there
because this isn't built
to kind of support feedback
on that end page.
You can have students
provide in-text citations
and it will give them feedback
on the in-text citations,
just not the work cited
page or the bibliography.
All right, let's look
at the teacher report.
So this is the class
report for an assignment.
So if you created a
persuasive essay assignment
for a given class,
from your Assignments page in
your Writing Coach dashboard,
you can click on that and
you can see this report.
We designed this report to be
a way for you, as a teacher,
to get an at-a-glance
view of where students are
and where you might want to dig in
to pay a little bit more close attention
to what happened with a
specific student's essay.
So the information that we show you here
is the step of the process
that the student is in.
So it'll tell you if a
student hasn't started yet,
if they are in the understanding
phase or outlining,
if they're drafting or revising
or if they're completed.
We also have little flags
that will tell you the
student is completed,
but they edited their essay
after the due date came,
so you might wanna just see
how much they actually did by the due date
if you have some procrastinators,
or if students have past
due date assignments,
you can see that as well.
You can see the last
time that they updated it
or how much time they spent overall,
and this is actively spent.
So if they have the tab open
and then go to sleep and
come back the next day,
it's not gonna count those
hours that they were sleeping.
This is time actively spent on the page,
chatting with amigo or typing or drafting.
Word count is the current word count
for their current latest draft.
Then this is where you would see
of the 21 pieces of
feedback Khanmigo gave them,
they've resolved this many of them.
And then these are the originality flags.
Originality flags are designed
to help you figure out,
again, where to drill down.
So if you see a critical
flag and you hover over it,
then you'll see what exactly
happened and where it happened
and then you can click into that
and see exactly what went down.
And I'll show you a live
view of that in a moment.
This is also a great place to figure out
if I have a student who
didn't get much feedback
or got a lot of feedback and
didn't resolve much of it
or didn't spend very much time writing,
but has a really high word count,
those are the kinds of things too
that you might wanna be like,
let me just see exactly
what happened here.
So it's really designed
to be a way to document
at a high level parts
of the writing process
and let you figure out where
to drill down into that.
So when you do drill
down into one student,
this is an example
of where I would've clicked
on an originality flag,
a critical flag.
So it'll take me straight to
the moment where this happened.
In this case, it was a student
who, during the drafting phase,
pasted 58 words from somewhere
that was not their outline.
And I can see the words that they pasted,
those words are highlighted.
I can also see their chat
at the time that they were drafting.
So I can see here some clues
that maybe the student wasn't following
academic integrity rules.
I can just have a little
bit more information
about why it was flagged
and then give me the opportunity
to follow up with that student.
But there's no originality flags,
I might not even need to
drill down into this report,
but I can if I want to
because it has the whole record
of the student's writing process.
So I could click on Understanding
and I can see the chat the
student had with Khanmigo.
I can click into Outlining
and see their whole outlining history.
I can see their whole drafting history
and all of their chats.
I can see for feedback
all the feedback Khanmigo gave.
I can see the revision student made.
I can see the chats
students had with Khanmigo
about their revisions,
what their essay looked
like at the time it was due,
what it looked like when
they marked it as complete
and all of that kind
of record information.
So again, obviously we
don't expect teachers
to review all of this information
for every single student,
we're not trying to make more
work for writing teachers,
but it is here for the cases
when you do want to drill down
or you do need to drill down
and you wanna kind of get a
sense of what's happening.
And we are also mid process
of exploring other ways
that we can help surface
more high-level insights
for teachers at the assignment level
so that they know about other information
around when they might wanna drill down
or instructional insights
and things like that.
And then quickly,
if you are an administrator
for a Khan Academy District,
you also have administrator reporting
and this includes data such
as Writing Coach usage.
So that includes the time spent
using Writing Coach among your district,
so how much time students
have spent or teachers,
and then also the percent and
the total number of students
who have used Writing Coach
during a given timeframe.
So I'm going to now show you,
'cause we have a few minutes,
a live view.
Let me change.
Oh, can you see?
Wait, no.
One moment.
- Yeah, we're still
seeing the report, Sarah.
- Okay.
Oh, do you see assignment review?
- Yes, assignment review.
- Okay, okay, that did work.
So this is an actual assignment report
with fake student data, of course.
In this case I assigned
a school start time persuasive essay.
I can view the essay instructions
that I provided to Khanmigo
and the students here.
And this is what it looks like
to view the student progress.
So again, I can see who is at which stage,
how far they've gotten.
And then for the originality flags,
if I hover over this,
I get kind of a high-level understanding
of what was flagged.
Critical flags are shown
in cases where students
are pasting a lot of text
from outside of their outline
or they're specifically doing
this in drafting or revising.
Questionable flags are they
might have pasted a phrase
or they might have put
things into their outline
that isn't evidence or a source
that you might wanna look at.
And you can kind of get a sense
of what the different levels are here
and how those things are triggered.
And right now it is just paste events.
So what we are looking at are
students using the content
that they outlined with Khanmigo.
So if they are creating
an outline with Khanmigo
and then they're drafting
and they are pasting an information
that's not from the outline,
that will trigger a flag,
but if they're pasting from their outline
or if it's coming from their outline,
that's fine.
We know that Khanmigo
worked with them on it,
we know that this was
something that they did
in our record of the experience,
and that will not be flagged.
Let me click into another
one of these students.
So if I click on this student again,
you can see first the final draft.
So if I'm just here
and I wanna look at the student's
final draft and grade that
or just see what it looked
like at the very end,
I can jump straight to that.
But I can also click to understanding
and I can see how much time they spent,
what they talked to Khanmigo
about during this stage,
the whole conversation if I want to.
I can also see, again,
their record of outlining.
So I can see where they
provided a thesis statement,
here they have some evidence,
they started filling in some more details.
And then I can see their whole chat
during the whole evidence,
or sorry, during the
whole outlining stage.
And then for drafting,
you can see that the students started
by asking Khanmigo to
help them get started.
It prompted them to start
with their introduction
and then the student asked
Khanmigo for feedback
and Khanmigo recommended
they provide more context.
And also giving it too specific
feedback about the thesis,
but again,
this stage is not really
about the specific feedback,
it's just about get the words on the page.
So I can see, again, any
revisions that they made
while they were drafting
and then their final draft.
And then for revising, same thing,
I can see all the feedback
they got from Khanmigo.
I can see what they
chatted with Khanmigo about
when it came to the
feedback that they got.
Here I can see that the
student asked Khanmigo
to check that they did their
MLA in-text citation correctly.
And then I can see where
they've done revisions.
The student got a questionable flag here,
so I can look at that.
But I can see that the flag
was related to a new piece of evidence
that the student pasted.
So I know that that's a non-issue.
You're pasting evidence from somewhere,
I know you didn't write
it 'cause it's evidence,
so that's completely fine.
And all the way down to
where their final draft was.
So that is the teacher report
and the individual student report.
Let me go back to my deck.
All right, so we have a few minutes left.
These are the instructions for you
to get started on this now
if you're interested in trying this out.
Again, if you already have
a Khan Academy account,
then you already have a
Writing Coach account.
So you can go to Khan Academy
or you can go to khanmigo.ai/writingcoach
and you can get to
Writing Coach from there.
And log in to your Khan Academy account,
and again, that will take
you right into Writing Coach.
Some things if you're just getting started
that you could do.
You can start by exploring
the student experience.
So if you are in a teacher account,
let me go back to this.
I mentioned these three tabs here.
If you click on My Essays,
this will let you experience
some of the student experience.
Note, however,
if you are not a district
partner with Khanmigo,
then your students won't be able
to just write their own
essay whenever they want,
that's only available
to district partners,
but all students will be able
to access assignments from here.
And a thing that you have as a teacher
that your students don't
have are these sample essays.
So if you are just trying to get a sense
of the student experience,
these are pretty helpful
because you can click into them
and it'll start you off at understanding
and you can chat with Khanmigo
as if it were like a fresh assignment.
You can go through outlining
and we have pre-filled in content
so you don't have to write an
entire outline for yourself,
you can fill it in
and then kind of see
the drafting experience.
Again, we pre-fill in an essay for you
so you can try that out
and then you can get
Khanmigo to generate feedback
and do the revising stage.
The sample essay tool is also very helpful
if you are demoing Writing
Coach with your students
for the first time
because it helps them see
what they will see when they're
working on their assignment,
and you can walk them through the stages
and how to use Khanmigo
and how to access all of the
different tabs at the bottom
when they're on various stages.
So I'd recommend checking out
the student experience first,
and then when you're ready,
you can create your first
Writing Coach essay.
Again, you go straight
to your Assignments tab
and you click Create Assignment
and you fill in the
information from there.
And I am very eager to
hear any and all feedback
that anyone has about this.
So if you use it yourself,
if you share it with another
teacher and they use it,
I have no problem
sharing my email address,
please reach out to me,
it's sarahrobertson@khanacademy.org.
Let me know what your experience was like,
send me your ideas,
send me your critical feedback,
send me anything that you have
that could help us continue
to make Writing Coach a
really valuable experience
for you and your students.
And yeah, that is it for me.
I will pass it back, I
think, to Frances or Aviv.
I don't know if we have
room or time for questions.
- Yeah, we had a couple of questions.
One is around,
I know you talked about right now
we don't have the ability
to upload rubrics.
The question was asked
around uploading sources,
because, how will Khanmigo
know if the student-
- [Sarah] Great question.
- making up evidence or
the evidence is accurate?
- Yeah, that's a great question.
So we don't have a way right now
to just kind of attach a PDF
or upload a reference text,
however, if the reference
text is a well-known text
and you include the name of that text
in the actual essay instructions,
Khanmigo will know about the text,
it will be able to kind of provide,
spot any issues with the text
that are wildly inaccurate
or just very off base.
It is not really designed to do things
like check that the
evidence is legit evidence,
that's not really something that it does.
What it can do is make sure the evidence
that the students provided is aligned
to the points that they're trying to make,
to the argument that they're making,
make sure that if they
choose a piece of evidence,
that they are explaining how
it helps them prove the point
that they're trying to make.
It can help them do things
like fix their introductions to evidence,
I remember that was one
of the biggest things
that I had to deal with
with my middle schoolers,
and then their citations as well.
But it won't be able to be like,
hey, that's not a real quote,
'cause it wasn't designed to do that.
But in the future when we do have things
like the ability for you
to attach a reference text,
that is something reasonable
that we would be able to support.
- Thank you, Sarah.
- [Sarah] Yeah.
- There was another question around,
can you assign an essay to
a class that you created
separate from your roster?
For example, if you had a
combined group for tutoring.
- Yes.
- [Frances] Yes.
- You can, yep.
It just has to be a Khan Academy class,
which you can actually do.
So when you are,
I'll show you actually,
when you are in your teacher experience
and you are creating your assignment,
you will be able to assign it
to an existing class that you have.
Or oh, this might just be because,
oh, I think I'm in a different,
you should be able to see any
other classes that you have
that maybe are not even
Khanmigo-rostered classes.
So if you have another
class on Khan Academy
that isn't a district-rostered class,
like a tutoring group,
you can assign an essay to that class.
In some cases you can do
it straight from here.
There's an option sometimes
that is create a new class,
I'm not sure why that's
not showing up here.
But if you are on Khan Academy
and you're creating a class
there, like a manual class,
you will see that in the list here
'cause this will show you
any of your Khan Academy
classes that you have.
- That's right.
- [Sarah] Mm-hmm.
- All right.
Is the feedback able to be differentiated
by student ability,
but just still at same grade level?
For example, some feedback simpler,
but still at grade level?
- Yes and no.
We do have some features
that are student-specific
where a student can change, for example,
their Khanmigo reading level.
It's not super expansive,
there's like basic,
which is actually the
default for all students,
and then there's maybe two
other levels that are higher.
So if you have really advanced students,
you can have them increase
their Khanmigo level,
but most students will start
out at the basic level.
If you have students who
are English learners,
you also can have them,
if you want, this isn't required,
but you can have them change
their Khanmigo chat language
to their native language,
and then they can talk to
Khanmigo in their native language,
but they're still looking
at an English essay
that they're writing
and Khanmigo will still be able
to kind of communicate with them
and look at their essay in
English and help them with that.
Other than that,
the scaffolding is really at
that point up to the student.
So if the student needs help,
Khanmigo will be there to help them.
And there is that checkpoint
between outlining and drafting
where it's checking to see,
did they meet the requirements?
So if you have students
who are way behind,
then that is a point where they
might spend a lot more time
outlining with Khanmigo,
and it will do that automatically
in that if they really are off topic
or they're missing a lot of pieces,
Khanmigo will repeatedly kind
of help them through that
until they have a good outline,
whereas other students
would just speed straight
through to drafting
'cause they wouldn't have those
checkpoints to worry about.
But yeah, we hear this a lot,
there's a lot of concerns,
especially from the past few years
where it just sounds
like students need a lot more scaffolding
than they have in previous years.
And so one of our biggest priorities
is figuring out better
ways to support kiddos
and especially find ways to differentiate
to the students who need it the most.
So if you have more ideas,
please send them to me.
- Awesome, thank you.
Thank you, Sarah, so much.
Thank you, everybody that joined us.
I've also dropped in my email in the chat.
In case you guys think
of any other questions,
please feel free to email Sarah,
email me.
We are here to help
and support in any way that you guys need
and we truly value your feedback,
your ideas and your feedback.
So thank you, thank you.
- Thank you all so much.