Writing Coach Meeting for AZ Partners

- Thanks, Jennifer.

All right, so it is exactly four o'clock,

so I will go ahead

and get us started to make
sure we start on time.

Good afternoon, everyone.

Thank you so much for joining us today.

My name is Frances Barreto.

I am one of the district success managers

that support our amazing
partners in Arizona.

And you do have a dedicated
team here to support you

as you see on the screen.

That includes Chelsea Hatchard,

who's our district partnership manager,

Sheryl Dossola supports as
the rostering specialist.

We have Victoria Cheng,

who is another district success manager,

and then Jennifer
Cummings and India Quarles

are your professional
learning specialists.

We are especially excited
to have Sarah Robertson

from the Khan Academy Writing
Coach team with us today,

and she's gonna be sharing

how you can help your
students become more confident

and skilled writers using
Khanmigo's Writing Coach tool.

This is the first

out of several professional
learning opportunities

that you're going to be able
to participate in the spring,

so keep an eye out for an email

that should be coming your way March 11th

with the registration links

for our Arizona Spring Learning series.

And remember that if you register,

you'll automatically
receive the recording,

even if you can't attend live.

Please feel free to drop any questions

or thoughts in the chat
throughout the session.

Jennifer and I will be
actively in the chat

and we'll also be sending
the recording your way

after this session as well.

So without further ado,

I'm gonna go ahead and
pass it on to Sarah.

- Thank you, Frances.
- [Frances] You're welcome.

- I'm gonna share my presentation now.

All right, hopefully you can all see

where it says Meet Khanmigo Writing Coach.

Yes, okay, awesome.

I'm gonna try and have
the chat open as well

so I can keep an eye on
what you all are saying.

Welcome, everyone.

I'm really excited to be here.

A little bit about myself.

My name is Sarah Robertson.

I am a principal product
manager here at Khan Academy,

focusing on literacy experiences

and classroom tools to support ELA

and humanities teachers specifically.

And some background of
sort of where I came from.

I was a middle school
ELA teacher in Boston.

That's a picture of me when
I had a lot less gray hair

than I have now.

And when I was a teacher,
I taught seventh grade.

Many of my students were
significantly below grade level.

I taught a variety of humanities classes

during my time in the classroom,

but the hardest of all the classes

that I taught was writing.

I think as a writing teacher,

I assigned 25 different essays

throughout the course of
one single school year.

And like most middle and
high school teachers,

I had about 100 students,

maybe more than that,

so that's about 2,500
distinct essay drafts

for me to guide students
through producing,

provide timely, detailed,
personalized feedback on,

evaluate against a rubric.

I worked easily about 12 hours a day

and nearly every weekend,

and even still, it always
felt like I could never get

to all the students in the class

who were asking for my help.

I could never keep a close enough eye

on the students who I knew needed my help,

but weren't asking for it.

I could never deliver
personalized feedback

to students quickly enough,

even if I tried really hard.

I never really knew about
what my students needed

at any given moment

or how to better support them.

And the burnout from this year

of teaching writing
specifically was so real.

And keep in mind this was before COVID,

before learning loss,

and before generative AI,

so I have a lot of empathy for many of you

who are currently in the classroom.

So today I am excited to share with you

the solution that we built

in response to these very
real classroom challenges.

So we'll talk a bit about
why we built Writing Coach

and our intentions behind its design,

and then I'm gonna dig
into how to actually use it

and show you how to find it
and how to assign things.

I'll show you what the
student experience is like

and what the teacher

and administrator
reports are like as well.

And then I will send you
off and hopefully you can go

and use it and explore it yourselves.

And then I'll actually give you my email

so you can send me the
feedback that you have.

All right, so I had asked
if we could make this

a meeting-type webinar

so you all could be a
little bit more interactive.

So now we have the chat.

Thank you, Aviv, for setting that up.

I would love for you all

to answer this question in the chat.

I'm wondering if you could just name,

what are some things that
you know students need

in order to reach grade-level
proficiency in writing?

Take a minute and think

and put your thoughts into the chat.

Kim says, "Vocabulary, sentence fluency,

punctuation, practice

help with sentence structure,
grammar, punctuation."

Absolutely.

Format of the writing, yes.

Practice and timely feedback, yes.

I'm seeing a lot of practice,
feedback, direct instruction.

Out of curiosity, does anybody know

how much time the National
Commission on Writing recommends

that middle and high school
students should spend

on writing practice?

Let's say each day.

How many minutes per
day do they recommend?

Guess, put it in the chat.

30 minutes.

60 minutes.

30 minutes.

All right, you guys are well informed.

It's 60 minutes per day.

I taught writing,

so in my classroom,

kids were writing a lot of the time

when they weren't watching me model things

or teaching things,

but in many schools,

there isn't a separate writing class

and most students are not
actually getting the 60 minutes

of writing practice per day,

as I'm sure you all are well aware.

Other things that I saw
mentioned that students need,

they need the direct instruction

specifically in things like
the rhetorical situation,

audience, purpose, tone, rhetoric,

text structure, sentence
structure, grammar,

the writing process,
how we begin to write,

how we think about writing,
how to conduct research,

how to incorporate evidence,
how to cite evidence.

They also need things like
exposure to mentor texts,

they need modeling, they need
vocabulary-building practice,

they need feedback

and that feedback has to
be specific and actionable

and it also has to be
delivered not three weeks

from when they wrote the essay.

And then of course, reading
practice and comprehension

is part of that too.

And they also just need to feel motivated,

that's a really big piece.

Again, looking for you
all to go to the chat,

what are some of the things

that make teaching writing
especially challenging?

Go ahead and put your
thoughts in the chat.

Time, yes, for sure.

Yep.

Never enough.

Differentiation.

Yes, Robert.

The feedback piece is,
yes, really challenging.

Motivation, yep, yep.

Definitely time for
providing that feedback,

especially if you have a lot of students.

Grade-level reading, yep.

Exactly, agree.

Texting all the time.

Yeah, productive struggle,
that's a big one.

And I think the motivation
piece plays into that too.

So I spend a lot of my time talking to

and learning from ELA teachers,

and writing is consistently
one of the hardest parts

of the job as I experienced firsthand.

These are some quotes from interviews

that we've done with ELA teachers

and I also happen to grab some screenshots

from a few of the recent comments

in various ELA teacher Facebook
groups that I'm still in.

So yeah, kids are not
writing on grade level

and it's really no wonder why.

A big piece of this, as you all mentioned,

is timely, specific, actional
feedback is so critical,

but with one teacher to 100-plus students,

it's nearly impossible and
teachers are really overwhelmed.

Also, since I've been in the classroom,

which is now, I think,
approaching 10 years ago,

it's gotten a lot harder.

My husband is a writing professor,

he teaches college freshmen,

so I hear from his perspective a lot

about how student AI use has
affected writing instruction

and practice in his classroom.

I also hear this from many of
the teachers that I speak to

through Khan Academy

and I expect that some
of you might have had

some personal experience
dealing with this too.

And there are various things out there.

Somebody here mentions,

"I've seen several AI
checkers online recently.

How accurate are those?"

They're not,

they're not accurate, unfortunately.

We all wish that there was

like one beautiful magical
solution to this problem,

but it has been really challenging

for teachers to be able to spot students

who are using this technology.

And it can be, as I'm sure you know,

very difficult to challenge a student

who you think has produced
something with ChatGPT,

but you don't have the evidence for that.

Okay, for this one,

I wanted to touch upon
another prescient question.

About a year ago I led a workshop
with high school students

on AI and writing

and one of the students said to me,

and this was like a very precocious,

I don't know, 14-year-old girl,

she said, "I just don't see the point

in learning to write anymore,

AI can do it for me.

Why do I need to learn to do this thing

that AI can just do for me?"

And I want you to take a minute

and you can answer in the chat,

or if you feel so inclined,

you can come off mute
and just say it out loud,

do you agree or disagree?

Do you think that it's still important?

I feel like this is
probably a biased group,

but do you think it's important
to still learn to write,

for students to still
learn to write, and why?

What would you say to that student?

So yes, I'm gonna pause here,

and if you have something to say,

feel free to come off mute and
just say or put it in chat.

- Hi Sarah, this is Johna Wallace.

I'm gonna say this out loud,

I think that students do
need to learn how to write.

My son works in finance

and he has to be able to
communicate to other people

what he has done with his clients,

and if he cannot write coherently,

that communication is not there.

Also with the writing,

it helps clarify students' thinking

and their ability to
communicate with one another,

so they need to know how to do that.

- 100%.

You said it so perfectly.

Everything you said.

Writing is thinking,

writing is communicating.

Without knowing how to write,

we can't actually sometimes process

what we even think about something.

We don't have that critical muscle,

we're not building that critical muscle

of processing information and
working through information

and communicating it in ways

that reflect our own personal experiences

and our own beliefs.

I came across this Joan
Didion quote a while ago

and it sort of gave me exactly

what I think I was trying to convey

when I was presented with
these kinds of opinions

from various people,

specifically students who were like,

why do I need to do this anymore?

And Joan Didion says,

"I write entirely to find
out what I'm thinking,

what I'm looking at,

what I see and what it means."

And I get chills when
I say that every time

because as somebody who,

I consider myself a writer
and a writing teacher

and somebody who supports
writing teachers now,

and writing is so much more
than the written content, right?

It's not,

the process of writing means something

and it's important

and it has real human value.

And I think as teachers,

it's our jobs not just to
teach students how to write,

but to teach them about

why they still should be
learning to write, right?

There's also a robust body
of evidence that shows

that there are real cognitive

and psychological benefits from writing.

It helps students,
again, process emotions,

process their thinking,

figure out what they feel,

figure out what they believe.

It's a really wonderful tool

that we all have in our
toolbox as human beings.

So I agree,

I don't think that large
language models or AI

is a reason that we should
not teach writing anymore.

I don't think anybody
would be surprised by that.

And if anything, I actually
think it's more important now

than it has ever been,

but it is still very critical

to educate students about
AI and large language models

and to use it ethically and
responsibly in the classroom.

And we could do an entire webinar

specifically on this topic of effective

and ethical AI use in classrooms alone,

but I sort of jotted down a
list of some of the things

to consider when we are
incorporating AI in the classroom.

The first thing is,

is the use of this AI actually
helping students learn

or are we using it as a shortcut?

Will students become reliant on the AI?

And if it's being used as
some sort of scaffolding,

how can we gradually remove
that scaffolding over time?

Do students understand

how the content from the AI
is actually being generated?

Where it comes from,

when to trust it, when
definitely not to trust it,

when to say something about the fact

that they might have used it in some way.

And are they using AI in ways

that allow them to focus
more of their efforts

on higher-order thinking skills

and also on the human
element of writing too?

There are many opinions on these topics,

but I do believe that there are many ways

in which even if we use AI
for things like brainstorming,

I think it depends on the situation,

but in some ways some might argue

that that's actually taking away

some of the really
important thinking skills

that students do when they brainstorm.

It doesn't hurt maybe to have somebody

to bounce ideas off of
or to jumpstart things,

but worlds in which AI
generates a first draft

or AI generates a list of ideas,

we should just be thinking
about what is the AI doing

that a student previously
could have been doing,

and are we taking something
away of value to that student?

Okay.

So with all of that in mind,

let's learn about how and
why we made Writing Coach

and then I'm going to
show you how to use it.

Khan Academy is a well-known company.

It is not necessarily as
well-known for writing and reading

and ELA or humanities as it is for STEM,

but our mission has always been

to provide a free world-class education

to anyone and anywhere,

and we've always wanted to provide

that same level of
support in the humanities.

So when large language models
became broadly available

in late 2022,

we immediately began exploring ways

that we could leverage the technology

to achieve that mission.

And it's probably no
surprise that we found LLMs

to be especially helpful in
the writing space specifically.

So like all great products,

Writing Coach was also designed

as a solution to some very real problems.

So what we know about writing

is that, of course, students are expected

to meet grade-level standards.

They need to be writing
60 minutes per day.

They need support,

not just at the end of the writing process

when they've produced a
draft and they need feedback,

but they actually need
help getting started,

they need help outlining,
they need help drafting,

and they need to learn

how to do those different
parts of the process.

And then, of course, as we all mentioned,

they need specific, actionable,

personalized, timely
feedback in order to improve.

But in reality, only about a
quarter of secondary students

meet grade-level writing standards.

The vast majority of students
are not clocking even close

to their recommended
hour of practice a day.

Most teachers have way more students

than they can give personalized
writing process support to.

And I mention this math set all the time.

If the average teacher has 100 students

and limits herself to 10
minutes of feedback per essay,

which is something I would do,

I would set my timer 10 minutes per essay,

that's 17 hours for one
draft of one assignment

for me to give feedback to every kid,

17 hours for one assignment.

And if kids are writing an hour a day,

I just don't,

the math isn't mathing,

as the kids say these days,

it's just not really possible.

So in 2023, we started
working on a solution

and then last fall we
launched Writing Coach

to our school and district partners.

And when I talk about Writing Coach,

I think oftentimes
people hear AI, writing,

and they sort of have
a picture in their head

of what this is.

So I like to kind of explain what it is

against what it's not

and how it's different from
some of the other things

that people might be familiar with.

So you might be familiar with tools,

for example, like Grammarly.

I use Grammarly,

Grammarly is great.

Grammarly gives AI-generated
feedback to students,

but it typically provides more sentence

or phrase-level suggestions

and allows them to click a
button and accept the suggestion

without really having to figure out

how to change it themselves.

You might be familiar with other tools

who can kind of generate feedback quickly

for the teacher using AI
or even for the students.

Some of these things
are paid, some are free,

but usually the feedback
is rather limited,

there's not that much feedback,

or the teacher gets the feedback,

but then has to figure out
how to get it to the student.

And to this day, I haven't
really found anybody else

who guides students through
the actual writing process

stage by stage in the way that we do.

And if you're familiar with ChatGPT,

which I'm assuming all of you are,

it's obviously always more than happy

to just rewrite a student's essay for them

or produce it from scratch.

And there's positive use cases

for all of these different kinds of tools,

but the primary purpose behind them

is usually either to save time
or to produce better writing,

not to produce better writers.

And that is really how
Writing Coach is different.

The intention behind Writing
Coach is to help teachers

create better writers and
not just better writing.

So it is an instructional tool

with the primary purpose
of student learning

and teacher transparency.

It's a tool designed to guide students

through the full essay-writing process

while preventing cheating.

It's a way for teachers to give students

more writing practice
and real-time support.

And we have data dashboards for teachers

that have at-a-glance
and in-depth insights

into their students' writing processes.

So it's not just a digital
essay-writing platform,

it's not something that grades
or evaluate student writing,

it gives feedback, it doesn't score,

and that's really on purpose

because I could talk about
this for a long time,

but I think if we're telling students

that their voices matter,

we should have a human
reading what they have to say

at some point.

It's not a student productivity tool,

it's not a feedback generator,

and it's not a tool that provides

easy-to-accept suggestions

for improving things like
grammar and mechanics.

So just really quick,

we launched Writing Coach now
about six months ago or so

and the feedback that we've gotten

has been very, very positive

from the teachers and the
students who have used it so far.

We had one teacher talk
about how the feedback

that Khanmigo generated
was very, very close

to what they would have written themselves

on student writing.

In some cases it was more
detailed or more specific.

Another teacher said that they used this

for an IB essay prep course

and they had seen this
make the biggest difference

in helping students with that essay

than any any tool they've ever used.

We had one teacher in a district
who used this consistently

and her class had every
single student pass

that state writing assessment.

And so we were really
excited about what this is

and how it works.

So let's just dive into
what it looks like.

Okay, so the first step
in using Writing Coach

is to create a Writing Coach assignment,

and a Writing Coach assignment
is an essay assignment.

To begin, you should start by
logging in to Khan Academy.

So you can access Writing Coach

directly through Khan Academy.

As you can see here in the top
of every Khan Academy screen,

you see a Khanmigo drop-down

and you can click on Writing
Coach and get there that way.

And if you're logged in
to a Khan Academy account,

you'll be logged in to
a Writing Coach account,

Writing Coach is part of Khan Academy,

but you can also just go
to khanmigo.ai/writingcoach

and that will also take you
directly to Writing Coach.

Once you are in Writing
Coach as a teacher,

you will see three tabs on the left,

one says Assignments,

one says My Essays,

and one says About Writing Coach.

Assignments, self-explanatory,

that is where you create
assignments for students.

The My Essays space is kind
of like a playground for you

to play around with sample
essays that we've created.

You can test out or demo essays

that you might have
assigned to your students

so that you can model

how to use it for the
first time with them.

But to access your assignments
and create assignments,

you would do that from
your Assignments tab.

And you just should be looking

for this Create Assignment button

in the top right of the screen.

So a Writing Coach essay
assignment starts with a prompt

and it should be a prompt
that you are providing.

The essay instructions, I
can't emphasize this enough,

are the most important part.

The more details you include,

the more of a stickler
Khanmigo will be when, they,

Khanmigo is guiding students
through the writing process.

So if you say things like,

make sure in your instructions,

make sure you include three
pieces of textual evidence,

Khanmigo will take that information

and check that the student
has that amount of evidence

whenever it's giving feedback.

If you say you need to
use MLA in-text citations

or APA in-text citations,

Khanmigo will check to make
sure that students did that.

If you say that they need to have

a concession/rebuttal paragraph

in their argumentative essay,

again, Khanmigo will have that context

and then make sure students
are meeting those requirements.

Also, if you are doing an essay

that is in order to prep
students for the ACT

or another test, an AP test or something,

you can say in the
instruction specifically,

this is an AP practice test

or this is an ACT practice essay,

and that also helps Khanmigo understand,

okay, now I have all of the
information in the world

about what makes a great ACT essay

and I know what an exemplary
ACT essay looks like

and I know what the rubric looks like.

Just telling Khanmigo
this is an ACT essay,

Khanmigo will take that context

and tailor its interactions accordingly.

One thing I do wanna point out,

'cause this is a common
source of confusion,

the grade level drop-down here,

you can select any grade level you want.

Right now we have fifth/sixth,

which is kind of like a beta level

because it will still be
a little bit advanced.

And then we have the seventh
all the way through 12th.

We use this to get Khanmigo

to tailor the language that it's using

when it's chatting with students,

but also to tailor the
feedback that it gives.

So if you select fifth/sixth,

it might give less feedback
or less harsh feedback

than if you select 12th
grade for the same essay.

And that's, again, in order to make sure

that Khanmigo is leveling
itself appropriately.

Students will not see the
grade level that you select.

So if you teach 11th graders

and it's the beginning of the year

and you're kind of building things up,

you can select seventh
grade as their level

if you really want the feedback

to be a little bit more just broken down

or to kind of build them up

to the higher grade levels of
feedback throughout the year.

So that's a scaffolding
tool that I like to mention

because it's not always clear.

Students won't see the grade level,

so they're not gonna feel demoralized

if you choose a level that's below

the level they're actually in.

Another thing I like to tell people.

So when you are designing
your essay instructions,

again, for the assignment,

things that you can use are
details from the actual rubrics

that you will be using
or that are relevant.

So for example, if
you're preparing students

for the Arizona six through
eight argumentative essay,

you're assigning something,

you can use former released
essay prompts, for example.

One thing you can do in your
essay instructions is to say,

this is a writing practice test

for the ASA seventh grade essay.

Students will be evaluated on purpose,

focus and organization,

evidence and elaboration, conventions.

And you can even say things
from that score four,

so you can literally copy and paste.

The response should have a
strongly maintained claim

with little or no,

wait, yeah,

loosely related material.

So you're pulling the exact information

from that score four.

So that means that when
Khanmigo is helping the student

through the essay writing,

drafting and revising and all of that,

it will have that in mind that
those are the expectations

that will make this a great essay.

So if you wanna get more
information in there,

that's usually always very helpful

to make sure that Khanmigo
is preparing students

for the essay.

No, you cannot upload the rubric yet,

that is something that
we are hoping to do,

but instead you can copy and paste details

from this part of the rubric.

Again, you really only need to tell it

this is what you would
need to do to get a four.

You don't need to provide the rest of it

because, again, this
isn't an evaluative tool,

it's not gonna score students,

so it's really about giving them feedback

that will help them get
the highest score possible.

And then for ACT, I think I remember,

am I remembering correctly

that high schools in Arizona use the ACT?

Okay.

You don't have to do that level of detail,

you can just say this
is an ACT practice essay

because Khanmigo knows ACT practice essay

and knows what a great ACT
practice essay looks like,

it knows what will get
you the highest score,

and it will make sure that
it's giving you feedback

and tailoring directions

according to those expectations for ACT.

So once you have created your assignment,

I think the only other details
we ask for are due date

and then you can create a
class if you haven't yet

or assign it to a class you
already have on Khan Academy.

Your students can access it

from their Khan Academy main dashboard

or they can go directly to
Writing Coach themselves.

You'll also, when you
create your assignment,

you'll see a link that
you can share with them,

you can put it in Google
Classroom or Canvas

or whatever you use,

and they can go straight
to the assignment.

Students can also go to
khanmigo.ai/writingcoach

and log in to their Khan
Academy account from there.

But they should be able
to access their assignment

directly in Writing Coach
or from Khan Academy

or from the link that you share.

So let's walk through what
this looks like for students.

On the left here,

students will see the assignment details

that you provided in the
essay instructions field

when you were creating the assignment.

And so they'll see the title of the essay,

they will see the type of
essay that you selected,

and right now we support
argumentative, persuasive,

expository, explanatory,
and literary analysis.

And then we also will be adding soon

AP Language and Composition and SAT.

We don't have an ACT one yet,

but again, you can just choose
persuasive, argumentative,

or anything relevant,

and then you can specifically
say it's an ACT essay

in the instructions.

And then students see the essay prompt

and instructions that you provided

and then they have Khanmigo on the side.

So for understanding,

it's really just about
breaking down the prompt,

making sure the student
understands the requirements,

making sure they can ask questions

about anything that they don't
get that you put in there.

So they can do things like ask Khanmigo

to define specific terms
or break things down,

or even if you provided
a lot of information,

it can kind of help
them search through that

to kind of find specific
answers to their questions.

And they can come back
to this at any point.

So if at any point they're like,

what were those directions again,

they can come back to
this and ask questions,

but they can also access
these instructions

throughout the whole rest
of the writing process.

Okay.

When students are ready to get started,

and some additional information

that I don't think I mentioned earlier

is that if in cases
where the essay requires

that students do readings

or they have like the source packet

or they have a text

that they are supposed to
be gathering evidence from,

that is something that
they can either do that

before they start their
Writing Coach assignment

or they can do it between
understanding and outlining.

Outlining works best,

I mean, it can actually help them

with making sure they're
selecting relevant evidence,

but it works best if they've
already read the text

or they've already looked at the text

and they're familiar with the topic.

And during the understanding phase,

Khanmigo will kind of
nudge them to figure out,

have you read the text yet?

Do you need to go back?

And it will kind of give them
advice for what to do next

before they start outlining.

But when students are
ready to get started,

this is the basic outline template,

probably looks relatively familiar.

It starts out as a basic
kind of five-paragraph essay,

but we made it flexible

so that students can remove paragraphs,

they can add additional paragraphs,

they can do things like
add additional evidence

or reasoning,

they can add or remove the source fields,

they can add,

I think we have, yeah,
reasoning explanation,

I might have mentioned that,

and move paragraphs around.

So we tried to design it
so that it would support

many different kinds of academic essays.

Even if you have a one-paragraph essay,

an extended open response,

it will work that way as well

as long as you're clear
in the essay instructions

about the format of what
you want students to do.

For each of the outline templates,

we also provide some pretty
simple exemplar sentences

for each of the fields.

So if you chose persuasive
or argumentative,

we will have an example thesis

related to from like a
persuasive or argumentative essay

of a completely different topic.

They're all Khan Academy-related,

so I don't think we have any
risk of picking the same topic

as you would choose for your students.

But then Khanmigo, again, is on the side

and Khanmigo can help
students get started.

If the student gets started and is stuck,

it's there to help them with that,

Khanmigo can see their outline,

it can check the outline

against the assignment instructions,

again, that the teacher has provided.

So if the student doesn't
know what to do next,

Khanmigo will offer some suggestions,

but Khanmigo will never ever provide

something for the student

to actually put into their outline.

So Khanmigo is not gonna
give them evidence,

it's not going to write a
thesis statement for them,

it's not going to provide
examples or main points

or any of that,

it will look at what the student has

and it will give the
student advice and tips

and guidance for how
they can make progress

and what they're supposed to be doing.

It can also,

oh yes,

when the student is finished outlining,

one of the other kind of
interesting things about this

is you might have some students

who just don't really
wanna talk to Khanmigo

or they don't really feel like chatting

or they don't really know how to chat.

Even if you never talk to Khanmigo,

when the student moves
from outlining to drafting,

we check it anyway.

So we have Khanmigo,
again, look at the outline

and look at the assignment instructions,

and at this step,

we're really just checking
for pretty high-level basic,

not like, is this a well crafted essay?

It's like, does it have
a thesis statement?

And is the thesis
statement actually related

to what the essay prompt is asking?

Does the student have enough evidence

according to what the teacher asked for?

Does it have main points
that actually support

or could reasonably support the thesis?

I sometimes had seventh graders

who would start with a thesis
on one side of an argument,

and then halfway through,
they'd switch sides

and wouldn't even realize
that they did that.

So that's one of the
things that we check for.

And again, if you ask for
something like a counterargument

in your instructions,

we'll check for that here.

And if the student has not met

those very basic requirements,

there will be kind of a
stopping point where we say,

it looks like your teacher asked for this

and you didn't quite do this part,

so it will let them go back

and they can fix what
they didn't do correctly

and then they will move on
to drafting at that point.

During the drafting phase,

we try to keep it pretty simple,

we don't want it to be
too much distraction.

So again, the student can access
the assignment instructions

whenever they need to.

They can also access the
outline that they just wrote

from the previous step,

and from that outline,

they can copy and paste
the different contents

of their essay that they worked on.

They can take a piece of
evidence and put that in

or their thesis statement, et cetera.

They can also just hide this completely

and just have a nice blank
drafting area if they want.

But they can chat with
Khanmigo if they want to.

And again, Khanmigo in the drafting phase

is going to focus on helping
the student get their words

on the paper,

looking at their outline
at the appropriate points,

and pulling the information

that the student has already
written out in their outline

and helping them kind of
begin to craft their arguments

and their points and their prose.

It's not necessarily going to
be giving a lot of feedback

at this point because, again,

the drafting phase is really about

get your words on the paper,

get your thoughts on the paper,

get your ideas on the paper,

and then the revising
stage will come next.

So Khanmigo at this point
is just helping students

get a first draft together

and not too much focusing
on the perfection,

trying to make sure every word is perfect.

And again, Khanmigo won't write for them.

You can see here the
student might ask them to,

they might ask them to nicely,

and Khanmigo will gently, politely decline

and help them kind of work

through whatever it is
they're struggling with.

So when the student has their first draft,

they will at that point move
on to the revising stage.

And in revising, the
first thing that happens

is Khanmigo will generate
feedback in five categories,

which you can see up here.

Introduction and claim is the first one,

then we have evidence and reasoning,

and then there's structure
and organization,

conclusion, and style and tone.

Note that in Writing Coach,

Khanmigo will give students feedback

in all of the same areas

as the AASA and ACT writing rubrics,

but it might be categorized
slightly differently.

So for this grade six
through eight AASA essays

for example,

feedback related to purpose,
focus, and organization

that would fall under either
introduction or structure.

Evidence and elaboration feedback

would fall under evidence and reasoning.

Conventions would fall under
style and tone, et cetera.

And then same for SAT.

Feedback related to
development and support

would be under evidence and reasoning,

organization would be
under structure, et cetera.

So that the labels are different,

but the feedback is the same.

But I think one of the most
important things to know

about the revising stage of Writing Coach

is that it's not just about the feedback.

So of course, in this example,

the student has 20 suggestions

under each of these five categories total

and we let them kind of
focus on one area at a time,

but we don't just give them the feedback

and then say, good luck, hope you do well,

we built in revising tools.

So we give, at some point,
praise or positive feedback,

but for the critical
feedback and the suggestions,

there are actual actionable
steps that students can take

to make the changes on the
draft right there on the page

or ask Khanmigo for follow-up
guidance or support.

So some of the things that
students can do are ask Khanmigo

to give them an example,

and this is, I think, one
of the most popular things

a student would choose to do.

So if Khanmigo says

your introduction doesn't
have enough context

for the reader,

that's, I think, for a very
common piece of feedback

for introduction,

the student can say, give me an example,

and Khanmigo will take an essay

on a completely different topic

and then give them an example

of what it looks like not to have context

and then it would look
like to have that context,

and that we have seen has
been really, really valuable

for students who get a piece of feedback

and they're just like, I don't get it.

So that's very, very helpful.

Then we have explain suggestion,

and that's really helpful.

If Khanmigo is being a little bit wordy,

they can ask it to explain it

and that'll usually result
in Khanmigo breaking it down.

If students are struggling readers,

you can also have them say

after they've explained it,

you say, make it simpler, say it simpler,

just keep kind of pushing
Khanmigo if they need to.

Students can just ask a
general question as well.

But then if they go in

and they change their essay
based on the feedback,

they can then immediately
just say, check my revision,

and Khanmigo will look
at that piece of feedback

and at the change the student just made

and let them know if they've fixed it,

let them know if they've
addressed it or not,

and then Khanmigo will say,

you made a little bit of an improvement,

but you could still make it clear

or you can still kind of provide

a little bit more information here

or give them whatever
follow-up guidance is necessary

until the student has what they need.

And what else?

They also have access
to their outline again

during revising.

And as they are working
through their feedback,

they can mark them as resolved

and keep track of where they're
at in their revision stage.

As their teacher,

and you'll see this in a minute,

you can see how much feedback they got

and how much they've resolved,

so you can keep track
of how close they are

to being done with this second draft.

And when they are finished,

the student can click mark as complete

and that's to let you
know that they're done

and happy with their draft.

And they get the option of exporting it,

so they can export it
to Microsoft Word or PDF

or even save it as a Google doc.

One other thing to note is
that we don't currently support

a work cited page.

Usually for test prep,
that's not a concern,

but for longer research
essays or whatever,

if you want students
to include work cited,

I would have them do that after
they've exported it to a doc

or a Google doc,

they can add the work cited there

because this isn't built
to kind of support feedback

on that end page.

You can have students
provide in-text citations

and it will give them feedback
on the in-text citations,

just not the work cited
page or the bibliography.

All right, let's look
at the teacher report.

So this is the class
report for an assignment.

So if you created a
persuasive essay assignment

for a given class,

from your Assignments page in
your Writing Coach dashboard,

you can click on that and
you can see this report.

We designed this report to be
a way for you, as a teacher,

to get an at-a-glance
view of where students are

and where you might want to dig in

to pay a little bit more close attention

to what happened with a
specific student's essay.

So the information that we show you here

is the step of the process
that the student is in.

So it'll tell you if a
student hasn't started yet,

if they are in the understanding
phase or outlining,

if they're drafting or revising
or if they're completed.

We also have little flags

that will tell you the
student is completed,

but they edited their essay
after the due date came,

so you might wanna just see

how much they actually did by the due date

if you have some procrastinators,

or if students have past
due date assignments,

you can see that as well.

You can see the last
time that they updated it

or how much time they spent overall,

and this is actively spent.

So if they have the tab open

and then go to sleep and
come back the next day,

it's not gonna count those
hours that they were sleeping.

This is time actively spent on the page,

chatting with amigo or typing or drafting.

Word count is the current word count

for their current latest draft.

Then this is where you would see

of the 21 pieces of
feedback Khanmigo gave them,

they've resolved this many of them.

And then these are the originality flags.

Originality flags are designed
to help you figure out,

again, where to drill down.

So if you see a critical
flag and you hover over it,

then you'll see what exactly
happened and where it happened

and then you can click into that

and see exactly what went down.

And I'll show you a live
view of that in a moment.

This is also a great place to figure out

if I have a student who
didn't get much feedback

or got a lot of feedback and
didn't resolve much of it

or didn't spend very much time writing,

but has a really high word count,

those are the kinds of things too

that you might wanna be like,

let me just see exactly
what happened here.

So it's really designed
to be a way to document

at a high level parts
of the writing process

and let you figure out where
to drill down into that.

So when you do drill
down into one student,

this is an example

of where I would've clicked
on an originality flag,

a critical flag.

So it'll take me straight to
the moment where this happened.

In this case, it was a student

who, during the drafting phase,

pasted 58 words from somewhere
that was not their outline.

And I can see the words that they pasted,

those words are highlighted.

I can also see their chat

at the time that they were drafting.

So I can see here some clues

that maybe the student wasn't following

academic integrity rules.

I can just have a little
bit more information

about why it was flagged

and then give me the opportunity

to follow up with that student.

But there's no originality flags,

I might not even need to
drill down into this report,

but I can if I want to

because it has the whole record

of the student's writing process.

So I could click on Understanding

and I can see the chat the
student had with Khanmigo.

I can click into Outlining

and see their whole outlining history.

I can see their whole drafting history

and all of their chats.

I can see for feedback

all the feedback Khanmigo gave.

I can see the revision student made.

I can see the chats
students had with Khanmigo

about their revisions,

what their essay looked
like at the time it was due,

what it looked like when
they marked it as complete

and all of that kind
of record information.

So again, obviously we
don't expect teachers

to review all of this information
for every single student,

we're not trying to make more
work for writing teachers,

but it is here for the cases
when you do want to drill down

or you do need to drill down

and you wanna kind of get a
sense of what's happening.

And we are also mid process
of exploring other ways

that we can help surface
more high-level insights

for teachers at the assignment level

so that they know about other information

around when they might wanna drill down

or instructional insights
and things like that.

And then quickly,

if you are an administrator
for a Khan Academy District,

you also have administrator reporting

and this includes data such
as Writing Coach usage.

So that includes the time spent

using Writing Coach among your district,

so how much time students
have spent or teachers,

and then also the percent and
the total number of students

who have used Writing Coach
during a given timeframe.

So I'm going to now show you,

'cause we have a few minutes,

a live view.

Let me change.

Oh, can you see?

Wait, no.

One moment.

- Yeah, we're still
seeing the report, Sarah.

- Okay.

Oh, do you see assignment review?

- Yes, assignment review.
- Okay, okay, that did work.

So this is an actual assignment report

with fake student data, of course.

In this case I assigned

a school start time persuasive essay.

I can view the essay instructions

that I provided to Khanmigo
and the students here.

And this is what it looks like
to view the student progress.

So again, I can see who is at which stage,

how far they've gotten.

And then for the originality flags,

if I hover over this,

I get kind of a high-level understanding

of what was flagged.

Critical flags are shown

in cases where students
are pasting a lot of text

from outside of their outline

or they're specifically doing
this in drafting or revising.

Questionable flags are they
might have pasted a phrase

or they might have put
things into their outline

that isn't evidence or a source

that you might wanna look at.

And you can kind of get a sense

of what the different levels are here

and how those things are triggered.

And right now it is just paste events.

So what we are looking at are
students using the content

that they outlined with Khanmigo.

So if they are creating
an outline with Khanmigo

and then they're drafting

and they are pasting an information

that's not from the outline,

that will trigger a flag,

but if they're pasting from their outline

or if it's coming from their outline,

that's fine.

We know that Khanmigo
worked with them on it,

we know that this was
something that they did

in our record of the experience,

and that will not be flagged.

Let me click into another
one of these students.

So if I click on this student again,

you can see first the final draft.

So if I'm just here

and I wanna look at the student's
final draft and grade that

or just see what it looked
like at the very end,

I can jump straight to that.

But I can also click to understanding

and I can see how much time they spent,

what they talked to Khanmigo
about during this stage,

the whole conversation if I want to.

I can also see, again,
their record of outlining.

So I can see where they
provided a thesis statement,

here they have some evidence,

they started filling in some more details.

And then I can see their whole chat

during the whole evidence,

or sorry, during the
whole outlining stage.

And then for drafting,

you can see that the students started

by asking Khanmigo to
help them get started.

It prompted them to start
with their introduction

and then the student asked
Khanmigo for feedback

and Khanmigo recommended
they provide more context.

And also giving it too specific
feedback about the thesis,

but again,

this stage is not really
about the specific feedback,

it's just about get the words on the page.

So I can see, again, any
revisions that they made

while they were drafting

and then their final draft.

And then for revising, same thing,

I can see all the feedback
they got from Khanmigo.

I can see what they
chatted with Khanmigo about

when it came to the
feedback that they got.

Here I can see that the
student asked Khanmigo

to check that they did their
MLA in-text citation correctly.

And then I can see where
they've done revisions.

The student got a questionable flag here,

so I can look at that.

But I can see that the flag

was related to a new piece of evidence

that the student pasted.

So I know that that's a non-issue.

You're pasting evidence from somewhere,

I know you didn't write
it 'cause it's evidence,

so that's completely fine.

And all the way down to
where their final draft was.

So that is the teacher report

and the individual student report.

Let me go back to my deck.

All right, so we have a few minutes left.

These are the instructions for you

to get started on this now

if you're interested in trying this out.

Again, if you already have
a Khan Academy account,

then you already have a
Writing Coach account.

So you can go to Khan Academy

or you can go to khanmigo.ai/writingcoach

and you can get to
Writing Coach from there.

And log in to your Khan Academy account,

and again, that will take
you right into Writing Coach.

Some things if you're just getting started

that you could do.

You can start by exploring
the student experience.

So if you are in a teacher account,

let me go back to this.

I mentioned these three tabs here.

If you click on My Essays,

this will let you experience
some of the student experience.

Note, however,

if you are not a district
partner with Khanmigo,

then your students won't be able

to just write their own
essay whenever they want,

that's only available
to district partners,

but all students will be able

to access assignments from here.

And a thing that you have as a teacher

that your students don't
have are these sample essays.

So if you are just trying to get a sense

of the student experience,

these are pretty helpful

because you can click into them

and it'll start you off at understanding

and you can chat with Khanmigo

as if it were like a fresh assignment.

You can go through outlining

and we have pre-filled in content

so you don't have to write an
entire outline for yourself,

you can fill it in

and then kind of see
the drafting experience.

Again, we pre-fill in an essay for you

so you can try that out

and then you can get
Khanmigo to generate feedback

and do the revising stage.

The sample essay tool is also very helpful

if you are demoing Writing
Coach with your students

for the first time

because it helps them see

what they will see when they're
working on their assignment,

and you can walk them through the stages

and how to use Khanmigo

and how to access all of the
different tabs at the bottom

when they're on various stages.

So I'd recommend checking out
the student experience first,

and then when you're ready,

you can create your first
Writing Coach essay.

Again, you go straight
to your Assignments tab

and you click Create Assignment

and you fill in the
information from there.

And I am very eager to
hear any and all feedback

that anyone has about this.

So if you use it yourself,

if you share it with another
teacher and they use it,

I have no problem
sharing my email address,

please reach out to me,

it's [e-mail].

Let me know what your experience was like,

send me your ideas,

send me your critical feedback,

send me anything that you have
that could help us continue

to make Writing Coach a
really valuable experience

for you and your students.

And yeah, that is it for me.

I will pass it back, I
think, to Frances or Aviv.

I don't know if we have
room or time for questions.

- Yeah, we had a couple of questions.

One is around,

I know you talked about right now

we don't have the ability
to upload rubrics.

The question was asked
around uploading sources,

because, how will Khanmigo
know if the student-

- [Sarah] Great question.

- making up evidence or
the evidence is accurate?

- Yeah, that's a great question.

So we don't have a way right now

to just kind of attach a PDF
or upload a reference text,

however, if the reference
text is a well-known text

and you include the name of that text

in the actual essay instructions,

Khanmigo will know about the text,

it will be able to kind of provide,

spot any issues with the text
that are wildly inaccurate

or just very off base.

It is not really designed to do things

like check that the
evidence is legit evidence,

that's not really something that it does.

What it can do is make sure the evidence

that the students provided is aligned

to the points that they're trying to make,

to the argument that they're making,

make sure that if they
choose a piece of evidence,

that they are explaining how
it helps them prove the point

that they're trying to make.

It can help them do things

like fix their introductions to evidence,

I remember that was one
of the biggest things

that I had to deal with
with my middle schoolers,

and then their citations as well.

But it won't be able to be like,

hey, that's not a real quote,

'cause it wasn't designed to do that.

But in the future when we do have things

like the ability for you
to attach a reference text,

that is something reasonable

that we would be able to support.

- Thank you, Sarah.

- [Sarah] Yeah.

- There was another question around,

can you assign an essay to
a class that you created

separate from your roster?

For example, if you had a
combined group for tutoring.

- Yes.

- [Frances] Yes.
- You can, yep.

It just has to be a Khan Academy class,

which you can actually do.

So when you are,

I'll show you actually,

when you are in your teacher experience

and you are creating your assignment,

you will be able to assign it

to an existing class that you have.

Or oh, this might just be because,

oh, I think I'm in a different,

you should be able to see any
other classes that you have

that maybe are not even
Khanmigo-rostered classes.

So if you have another
class on Khan Academy

that isn't a district-rostered class,

like a tutoring group,

you can assign an essay to that class.

In some cases you can do
it straight from here.

There's an option sometimes
that is create a new class,

I'm not sure why that's
not showing up here.

But if you are on Khan Academy

and you're creating a class
there, like a manual class,

you will see that in the list here

'cause this will show you

any of your Khan Academy
classes that you have.

- That's right.

- [Sarah] Mm-hmm.

- All right.

Is the feedback able to be differentiated

by student ability,

but just still at same grade level?

For example, some feedback simpler,

but still at grade level?

- Yes and no.

We do have some features
that are student-specific

where a student can change, for example,

their Khanmigo reading level.

It's not super expansive,

there's like basic,

which is actually the
default for all students,

and then there's maybe two
other levels that are higher.

So if you have really advanced students,

you can have them increase
their Khanmigo level,

but most students will start
out at the basic level.

If you have students who
are English learners,

you also can have them,

if you want, this isn't required,

but you can have them change
their Khanmigo chat language

to their native language,

and then they can talk to
Khanmigo in their native language,

but they're still looking
at an English essay

that they're writing

and Khanmigo will still be able

to kind of communicate with them

and look at their essay in
English and help them with that.

Other than that,

the scaffolding is really at
that point up to the student.

So if the student needs help,

Khanmigo will be there to help them.

And there is that checkpoint
between outlining and drafting

where it's checking to see,
did they meet the requirements?

So if you have students
who are way behind,

then that is a point where they
might spend a lot more time

outlining with Khanmigo,

and it will do that automatically

in that if they really are off topic

or they're missing a lot of pieces,

Khanmigo will repeatedly kind
of help them through that

until they have a good outline,

whereas other students

would just speed straight
through to drafting

'cause they wouldn't have those
checkpoints to worry about.

But yeah, we hear this a lot,

there's a lot of concerns,

especially from the past few years

where it just sounds

like students need a lot more scaffolding

than they have in previous years.

And so one of our biggest priorities

is figuring out better
ways to support kiddos

and especially find ways to differentiate

to the students who need it the most.

So if you have more ideas,

please send them to me.

- Awesome, thank you.

Thank you, Sarah, so much.

Thank you, everybody that joined us.

I've also dropped in my email in the chat.

In case you guys think
of any other questions,

please feel free to email Sarah,

email me.

We are here to help

and support in any way that you guys need

and we truly value your feedback,

your ideas and your feedback.

So thank you, thank you.

- Thank you all so much.