5.3 - More on Experiments

I'm going to record
from the current slide.

Sorry. Today we're going to
be talking about Chapter 11.

More on experiments: confounding

and obscuring variables,

and there's a little girl
hiding behind a tree,

and she is the third
variable in your experiment.

You don't want her
to be hiding and

explaining the results
of your experiment.

Before I review with you,

I did want to go over some
assignments that are due.

This week should be pretty
light on assignments,

and that's because I
want you to be working

on your method and
your results section,

working on sprucing
up your intro

if you have any comments.

This Sunday, we're going to

have the quiz on Chapter 11 due,

and then next Sunday is
going to be pretty heavy.

I want your Pquiz on Chapter
12 and your second draft,

which will include a
revised introduction.

If you've got feedback from
me, you can also schedule

an appointment with me if
you want more feedback,

a method section, which
we'll talk about today.

An anticipated results and
data analysis section,

we'll also talk
about that today,

and a discussion section.

We'll talk about that Wednesday

if we don't get to it today,
the discussion section.

But by the end of this week,

you should have all of
the things you need

to be able to write those
sections of the paper.

Any questions? Alesia,

do you know what
measures you're using

to measure introversion
and extroversion?

I don't know the specific ones.

I was planning on looking
them up today, actually.

Rick or Haley,

if you need help looking for
those measures, let me know.

I'm more than happy to help you.

I've got enough
experience to be able

to do it pretty quickly,

and I don't want you to waste
a lot of time looking for

those measures. Sorry, Rick.

You should be using that
time to write your papers,

because that's what I care

about more than actual measures.

We can always talk about
the measures later.

Review questions.
Are you all ready?

Me going to feel really
bad about myself,

really good about myself,
depending on how you do.

No pressure. What's an
independent variable?

We're starting off pretty easy,

and then we're going to
get a little harder.

It's the variable that
gets manipulated.

Manipulated or measured
by the experimenter.

Then the dependent
variable is? Alesia.

What's the word?
It's also measured,

but it's going to depend on
the independent variable.

That's just how I remember them,

but it's what you're
really looking for.

It's your outcome variable.

It's influenced by your
independent variable.

Those are acceptable answers.

Ann, can you give
me an example using

your research question? Haley.

Should I say my
research question, too?

Sure.

My research question is,

how do age differences
and being a woman in

a leadership position affect
how they are viewed at work?

My IV, I actually have two.

The first one is age groups,

and that has an age group
of 20-45 and then 45 plus,

and then the other independent
variable is women in

leadership positions and men
in leadership positions.

Then my dependent variable
is how these women

are treated and viewed at work.

Just generally how people
are treated at work, right?

Yeah.

Perfect. Great. Because all
that makes sense so far.

That's an easy question.
What's an experiment?

I know what an experiment is,

but I always have trouble
putting it into words.

Just throw out some words
that you're thinking of.

It's when you're trying to

understand the relationship
of two variables

and you use an
experiment to do so.

That's great. You use

an experiment to establish

a relationship between
two variables,

but it is different
from a correlation

because what do you
say about X and Y?

X has to cause Y.

Yes. X has to cause Y.

It establishes causality.

What is one thing that

experimenters do to
establish causality?

Temporal precedence.

Temporal precedence. Yes,
this what I was looking for,

but it's another question
on the next slide.

It's temporal precedence.
Wait hold on,

not Haley. Rick or Alesia.

What came first?
What caused what?

Exactly. What caused what?

X comes before Y, so

your independent variable caused

your dependent variable?
Great. What else?

That's the difference
between a correlation just

shows that they have something
to do with each other.

Exactly. In a
correlational study,

you don't know what came first.

Perfect. What else?

Alesia, you're spot on.

You just need a little bit more.

What else? We talked
about the hat, remember?

Little frog hat.

It's random assignment
to one of X conditions.

Alesia, you're not
going to have random

assignment because
you're already

placing people into conditions,
pre existing groups.

Rick, you probably will
have random assignment,

and Haley, you won't have

random assignment because
you can't randomly

assign people to sex or gender.

But what is it about
random assignment?

Actually, we'll come
to that question soon.

Random assignment to
one of X conditions,

and you measure
dependent variables,

and you control for as
many variables as you can.

Alesia, I think this is
something you brought up,

or is this why they
try to control

as many variables as
you can in a room?

Yes, this is why
because we want to

establish that X causes Y.

Great. Let's see this one.

This one's a little tricky.

Why have two or more levels
of an independent variable?

Alesia, I used your example.

Why do we have extraversion and

introversion as the
two levels of an IV?

Why can't we just
do extraversion?

I guess you could

say you're not getting
the question answered.

You're only looking
at one aspect

of it rather than
the whole picture.

Yes. It's basically
for comparison.

You don't just want
descriptives of extraversion,

but you want to
see the comparison

between extraversion
or introversion.

Rick, you want to see
the differences between

pre-COVID and during
COVID, and Haley,

you want to see the
differences between

age group and sex,

and you want your study
to show that x1 one

differs from x2 on y. Rick,

this would be that
loneliness scores

differ pre-COVID
versus during COVID.

This is one of the questions
that Haley already answered.

What factors allow an experiment
to make causal claims?

One is temporal precedence,

so X comes before Y.

The independent variable comes

before the dependent variable.

Thus, we can say the
independent variable

caused the dependent variable.

What else? Is one thing

in common between
correlation and experiments.

Covariance. Variance.

Yes. Covariance.
What does that mean?

Is it just that there's a
relationship between the two?

Perfect. Then we have
temporal precedence,

which we already talked about,
and what's the last one?

Internal validity.

Yes. What does that mean?

Is there a C variable

that is associated with
the other two variables?

It's a little girl that's
hiding behind the tree.

If there's a little girl
hiding behind the tree,

does that mean that you
have good internal validity

or poor internal validity?

Poor internal validity.

Yes. If there's another
variable that's hiding

that might have caused changes

in your dependent variable,

that is low internal validity.

Damn, you guys are putting
it together. This is good.

Is it making more
sense now that I'm

putting it in the questions?

I'm not going to feel
too good about myself,

so I didn't get
through all of them.

You guys are doing a great job.

What's the difference in
the following designs?

Between versus within subjects.

This we just learned
last week, so it's okay.

It doesn't make sense.

At least feel like you have
it on the tip of your tongue.

I want to say that the
within subject design

has to do with more time
because you're doing

the same type of
experiment repeatedly,

whereas the between
it's happening once,

and then you have all your data.

It's very close. Within,

what is the same?

The participants.

The participants are the same.

Yes. Haley, go ahead.

Well, so the within is where

there's only one group
of participants,

and they're given all the levels

of the independent variable.

This would be, sorry.

No, go ahead.

It would be for
Rick's study where

he collects 100 people at Wu,

and then he has them fill

out the survey on
loneliness pre-COVID,

and then he has them
fill out the survey

of loneliness during COVID.

He has them go
through all levels

of the independent variable.

Then what about
between subjects?

There's different
groups of participants

placed into the different
levels of the IV.

Yeah. Between subjects is when

you have them placed
into different groups.

Rick's study would be
one group is randomly

assigned to pre-COVID
loneliness scale,

another group is assigned to

during COVID loneliness scale.

Does that make sense? Between
means you're comparing

between people and within

means you're comparing
within the individual.

You're exposing them to the
same independent variable.

Between; people are randomly

assigned to one of conditions.

Within; people are exposed
to all of the conditions.

For some of you like Alicia,

you can't randomly you can't

assign people to be extroverts
and then you're like,

now pretend to be
introvert or Haley,

you can't do that for
your study, either.

You can't be like,
imagine you are

a woman in her 30s in
a leadership position.

Now you're going to pretend
that you're a man in

his 40s in a leadership
position. You can't do that.

Another thing within subjects

that we talk about is that we

deal with problems
by counterbalancing.

Let me give you an example
using Rick's study.

Let's say Rick decides
that he wants all of

his participants to complete

the pre-COVID survey and

then complete the
during COVID survey.

There might be an
issue with that, why?

Maybe they see what's going on.

Maybe they see what's going on.

They're like, pre-COVID.

Now I'm going to
base my responses

during COVID on what
I responded earlier.

We deal with these problems
by counterbalancing.

What does that mean? [LAUGHTER]

Isn't it where you change up
the order of the conditions?

Exactly. Give me an
example with Rick's study.

I just read about it.

For Participant 1,

you do pre-COVID and
then do after COVID,

and then Participant 2,

you do the opposite,

you do post-COVID first,
and then pre-COVID.

Perfect. That way
you can eliminate

the order of the
independent variables

as a reason for the change
in the dependent variables.

Perfect. You guys got it.

You're putting it all together.

Here's that again.
What's the difference

between random selection
and random assignment?

What's the purpose of each?

Random selection is you're
choosing the sample

that will best represent

the population
you're looking at,

and the random assignment

is you're taking that sample and

you're assigning them
specific conditions.

Perfect. That's great.

For random selection,
the purpose is,

you want it to be
representative of your sample,

and then random assignment,
what's the purpose of it?

We talked about it earlier.

It's not covariance, it's
not temporal precedence.

Internal validity?

Yes. It's internal validity.

But why?

I was at the Pepsi study,

like randomly assigning
people to Pepsi or Coke.

I'm blinking, and I'm sorry.

No, that's okay.
Anyone else want to.

But you did it perfectly.

You did less words than
I did. [OVERLAPPING]

Keeps for one side

to be overloaded with all
something and the other one.

They'd be offset so they
wouldn't be represent. Sorry.

No, that's fine. Keep going.

One side would be,

all girls or something like
that or all Coke drinkers or.

Yes.

With random assignment, what you

do is that you evenly distribute

any individual difference
variables that

could account for differences
in your dependent variable.

The individual
difference variable

that we were talking about

was drinking Coke versus Pepsi.

Rick's hypothesis is that

people will experience
more loneliness

during COVID than pre-COVID.

But we also know that
Coke drinkers tend

to experience more
loneliness than

Pepsi drinkers because
Pepsi is better.

There would be low
internal validity

if in the during
COVID condition,

we had more coke drinkers

than in the pre-COVID condition

where we had more
Pepsi drinkers.

That individual
difference variable,

whether you like Coke or Pepsi,

could also affect your dependent

variable, which is loneliness.

You basically said
this too, Rick.

You said something like gender.

That's another individual
difference variable.

Maybe gender affects
feelings of loneliness

or feelings of competence

or whatever a
dependent variable is.

Okay.

Does that make sense?

Yes.

This was the hardest
question, Alicia.

You just got it.

Now this is basically all
we've learned until today.

All nine chapters or 10
chapters that we've gone over.

Any questions here?

If I put the hat on the
exam, will that help?

[LAUGHTER] I feel like
it. Remember the hat.

Back to the girl that's hiding.

We're going to talk
about experiments again,

but now we're going
to talk about

confounding and
obscuring variables.

It's going to be all
about this little girl,

the third variable that could

affect the results
of our studies.

I also just posted
the PowerPoint,

if you want to go
online and get it.

I was working on this
until class time,

so if you're going to go and get

it and take some
notes, that's cool.

For this chapter,
we're going to talk

about threats to
internal validity,

which we've already
established as having

a third variable that might

account for changes in
the dependent variable.

What we're really
looking for is,

did the independent
variable really

cause the difference in
the dependent variable?

Then we'll also
discuss null effects.

What if the independent variable

doesn't make a difference?

What if you find no difference

in your dependent variable?

Does that really mean that

your independent variable
cause no difference,

or is there a lurking
little girl that's

messing with your
data that makes it

seem like there's no difference?

Within the threats to
internal validity,

did the independent variable
really cause the difference?

We have these four
bullet points.

First, we're going to go over
the really bad experiment.

We'll also talk
about six potential

internal validity
threats in one group,

three potential internal
validity threats.

If there are so
many threats, why

do we even conduct experiments?

Hopefully, I'll convince you
that they are still useful.

The really bad experiment.

This is the experiment that
I talked about with within

subjects design when we
talked about brick study,

but Rick your
experiment isn't bad.

I was just using it as a
within subjects design.

You're doing between
subjects design,

so this makes it better.

The three examples on
this slide illustrate

what's known as a
really bad experiment.

Diagram A is just
a general diagram.

You have your participants,

you measure the dependent
variable beforehand,

you give them the treatment,

and then you measure
your dependent variable.

What you're looking
at, is changes in

your dependent variable as
a result of your treatment.

In experiment B, this is
discussed in your textbook.

Niko, a summer camp counselor
and psychology major

noticed that in a
cabin of 15 boys,

that there is an
especially rowdy bunch.

He's heard that a change
in diet might help

calm them down,
so he eliminates.

He measures rowdy
behavior first,

so they have really
rowdy behavior.

He reduces their sugar,

and then he measures their
rowdy behavior again.

He reduces their sugar for
two days, and as expected,

confirming his hypothesis,

when he reduces
sugar for two days,

their rowdy behavior
has decreased.

We'll talk about
why that might not

be caused by the sugar.

In another study, they
have 40 depressed women,

all of whom are
interested in receiving

psychotherapy to treat
their depression.

Dr. Yuki measures depression

before they start
cognitive therapy.

Then they start
cognitive therapy,

and at the end of the 12 weeks

session of cognitive therapy,

they measure depression again
and finds that on average,

people's levels of
depression have

decreased through what she's

assuming is through
the cognitive therapy.

Can anyone tell me why
these studies might not be

good before we get into it,

intuitively, before we get into

the actual terms for
all these things.

I want to say you're
assuming that

there's only one potential.

Yes.

Yeah, so one potential cause.

Let's go with this
one, for instance.

Niko is assuming that because
he reduced the sugar,

the boys are less rowdy.

But what's another explanation

for why they might
be less rowdy?

Maybe they went on a
long hike that day.

Yeah, maybe they went on
a long hike that day.

Maybe they get tired
of being rowdy.

You be rowdy for, like,
three days and then

you're tired out.
You're not as rowdy.

Great. Maybe there were

other activities.
What's another thing?

Maybe time. Maybe this
was when they first met,

and they're all
trying to show each

other how rowdy they are,

and at the end, they're
like, Okay, you

already know me. I don't care.

When you go on a
first date, right,

you try to, like,
be your best self,

and then after a while,
you're like, Okay,

I can't keep this up,
you're just normal.

Okay, what about this study?

For depression, they
have cognitive therapy,

and then they decrease
their depression.

Yeah, it could be due
to cognitive therapy.

What's another
potential explanation?

Like, seasonal factors, maybe?

That's a great explanation.

Yeah. Maybe Doctor Yuki

recruited people
during the winter,

and all these women
had depression.

Then she gives them
cognitive therapy.

Look, it's spring.

They have less depression.

It might not be because
of the cognitive therapy.

I might just be the
change in season.

Perfect time. Anything else?

At events.

Life events. Perfect. Can
you give us an example?

Like maybe at the beginning,

they just went through,

like, a really bad divorce,

and then after they were,

like, getting back
on their feet.

Yeah. Maybe she recruited all

of these women as part
of her practice, right?

Like, she knew that

these women were going
through a divorce.

She's like, Okay, let
me measure depressi

and then let me
give you therapy,

and then we'll measure
your depression again.

Maybe it was just
time. Yeah, those

are all good explanations.

The reason why I have you go

through this painstakingly is

because I want you to be able to

work through it in case you
forget something on the exam,

because all of the things
that you've talked

about have actual terms to them,

which we'll talk about shortly.

Alright. This is known as the
one-group posttest design.

So this is a graphical
representation of the changes.

On the Y axis, we
have rowdy behavior.

This for the campers.

Beginning of the week, they have

this much rowdy behavior,

about 70% rowdy behavior.

At the end of the week,

after the sugar-free diet,

they have about 43%
rowdy behavior.

Whoa, it's a huge decrease.

What about depression?
Pre-therapy,

these women had a score
of 17 in depression,

but after therapy, they

experienced a score of
about ten in depression.

Let's make sure all
the college campers

eat a sugar-free diet,

and let's make sure
that women who are

depressed all get
cognitive therapy.

Is that a good assumption
to make from that?

No, not necessarily.

Okay, so let's talk about

the internal validity threats

in one-group
pretest-posttest designs.

These are some of the examples
that you came up with,

but these are the terms

that go along with
those examples.

We'll talk about
maturation threats.

What does maturation sound like?

Terry mature.

Yeah. What does
mature sound like?

Like, what does it mean?

Like overtime.

I think that's what we
talked about getting

tired out. It got hold.

Yes, that's why I had

to go through that exercise
so that we could then talk.

I'm going to go over what
these mean, obviously,

but I just wanted to,

like, get your
intuition working.

History threats, regression
threats, attrition threats,

testing threats and
intimidation threats

, and combined threats.

We're just talking about
all the threats today.

Okay, maturation threats.

A change in behavior that

merges spontaneously. Over time.

For example, children
become better and faster at

solving addition and
subtraction problems

as they get older,

trees grow taller with
age, and over time,

people tend to recover on their

own from various
psychological disorders.

This is the original graph
that we had, or sorry, yeah,

the original graph
that we had with

this line showing us that women

in the therapy condition

decreased in depression
across time, right?

Women in the pre-therapy
condition experience

more depression than women in
the post-therapy condition.

But we don't have a
full comparison group.

This is what Alesia was saying.

We don't have something
to compare it to.

What if we had a
no-therapy group,

a group that actually
didn't receive any therapy,

and we see how they
change across time.

If the no therapy comparison
groups' depression level

decreased across time,

this suggests that it's
a maturation effect,

it's a maturation effect and not

necessarily that it's
the therapy effect.

But because we do have
that comparison line now,

which one seems like it
decreased more over time?

Their depression
decreased more over

time. Which one
does it seem like?

The therapy group.

Yeah. Because the therapy group,

their depression decreased
even more significantly over

time relative to the no therapy
group, then we can say,

Okay, it might be the
therapy that's causing

this decrease in depression

and not necessarily maturation.

Does that make sense? If they

wouldn't have received therapy,

they would have
still decreased in

depression quite a bit,

but it wouldn't have
been as significant as

including therapy on top
of just simple maturation.

The way that you prevent
maturation threats

is to try to have
comparison groups.

In a really bad experiment,

you don't have a
comparison group,

but in a true experiment, there

should be a comparison group.

Okay. If you wanted to say what

the true effect of therapy would

be on decreased depression,

you would subtract the effect
of just simple maturation.

It decreases about four points

from the total
effect of therapy.

It decreases about six points.

In reality, therapy only
helps about two points

above what simple
time would give you.

Does that make sense?

Two, six or actually
four points.

This is 18, and then
it goes to, like, 16,

that's a decrease of
two points, 18-12.

It's a decrease of six points.

6-2 is about four.

The unique contribution
of therapy

is a decrease in four
points of depression.

Okay. Does that make sense? Ylo,

looking confused, Rick,
does that make sense?

That part totally
makes sense again.

I'm going, Oh.

Where did the numbers come from?

Oh, well, it's just the
numbers that are here.

I'm just looking like,
18 is this point.

Then this point is like 16.

I don't know how to read them.

I just don't know why

18 is there and not 57
under or something.

Oh, the numbers are
just arbitrary.

I'm just using the numbers
that are on the graph.

But if you had this, you
could say, and this graph,

you could say the unique
effect of therapy was about

a four-point drop in

depression or whatever
the actual scale was.

Okay. Any other
questions or comments?

That's what I miss
about being in class

is seeing people's faces,
some people get it,

and then a lot of people don't,

so I like looking
at faces. But okay.

History threats to
internal validity.

What is a history threat?

This results when some
external or historical event

affects most members of

the treatment group at the
same time as treatment.

It's unclear whether the change

in the dependent variable for

the experimental group
was the result of

the treatment or of
the historical factor.

This is more about,
like an actual event.

An example is, suppose you
were studying the effects

of meditation on stress levels
among college students,

and while you were
conducting the study,

a violent event occurred
on the college campus.

The meditation
group did not show

significant decreases
in stress levels,

but was that because the
treatment wasn't effective?

It might not be, right?

Because you had this
really stressful event

happen on campus.

Maybe the meditation
was effective,

but the campus violence raised
people's stress levels.

Alright, this is another study

that was discussed in your
textbook in a college,

they had a go green campaign

where they emailed
college students

starting in September,

and they told them
to try to unplug

things that weren't
being used and to turn

off the lights whenever
they weren't in

a room to decrease

the number of kilowatt hours
that were being used, right?

To go green.

What they found the Go
green campaign in October.

This is the number of kilowatt
hours change across time.

The people who got
the emails decreased

their kilowatt hours from
September to November.

But what about the no campaign?

These people didn't
even receive an email.

They also decreased
in the kilowatt hours

used between September
and November.

It's a historical factor
that could have happened

between September and
November that might have

reduced their energy usage.

I want to say Halloween,

but I don't know if that's

gonna have anything
to do with it.

Yes, maybe Halloween, and
they're all going to parties,

so it's warm so they don't have

to use as much electricity.

But really, it's like in,

let's pretend that they're
in North Carolina.

You probably use your air
conditioner a lot less

between September and November.

Because both of
the groups seem to

track along the same trend line,

then it shows that the campaign

wasn't actually effective.

It was just a history threat.

They all went through the
same seasonal change.

However, if we find that in
the beginning of September,

both the people that weren't

in the campaign and
those who received

the email use the same
amount of energy.

But at the end of the campaign,

the people who
received the email use

significantly less energy than

those in the no
campaign condition,

then we can say that
the manipulation,

so whether they got
the email or not,

actually reduced their level
of kilowatt hours used.

Does that make sense? This is

an example of a
history threat that

affects your dependent variable.

This is an example.

Same history threat
is available,

but despite the same
history being available,

there is still a difference.

Does that make sense?
The green campaign

changed the level of

energy use greater than
that which we would

normally see based
on seasonal changes.

How do you avoid
a history threat?

You can't always avoid
a history threat,

but because you can't say, Well,

I don't want winter to happen.

You should measure
a control group,

so a group that
doesn't experience

any of the manipulation.

Something that happened
at UC Santa Barbara,

when I was a student,

they were looking at one of
my friends is a researcher

on romantic and also,
platonic relationships.

Between friends and how
close they feel to them,

they were also investigating
how stressful events could

influence the closeness
that you feel

to your friends and to
your significant others.

They started collecting data,

I think in the fall, and
they were collecting

it the entire academic year.

Then in May, we had the
shooting that happened.

Then because they were

collecting that data
for the entire year,

they had an entire year's
worth of data pre shooting,

and then they had
data post shooting.

Their research question actually

changed because of
that historical event.

Sometimes because of things
that happen in the world,

that might affect your
research question

or what you're actually
looking for in your data.

Does that make sense?
Or like COVID,

there were researchers
that were probably

collecting data on loneliness,

and then COVID hit and
they're like, Oh, my God.

Jackpot.

This sucks, but Jackpot.

Basically. This sucks, but
Jackpot because no one

else had data before
COVID on this.

It sucks, yes, Jackpot
sum. Does that make sense?

I think the message is just
collect data all the time

because you never
know what messed

up stuff is going to
happen in the world.

What about regression threats?

What's a regression threat?

Regression means to return to.

This is also known
as regression to

the mean or regression
to the average.

This is a statistical concept in

which extremely low or
extremely high okay,

extremely low or extremely
high performance at time one

is likely to be less
extreme at time two.

Closer to the average.

If you're in a really good
mood or a really bad mood,

you'll probably be in
a more moderate mood

tomorrow due to aggression
because that's your average.

I was always scared of this,

and I'll 'cause I
always got good grades.

I was like, oh, my God, there's
gonna be a regression to

the mean because this can't
be my real performance.

I'm going to have to get a
fail to get the average of

a [inaudible] this is so you
can remember it in that way.

Regression. If you
perform really high,

then if that's not your
true level of performance,

you can always dip down and
go back to your average.

It's hard to tell with
your own performance,

what, your true high
or your true low is.

But if you have
impostor syndrome,

then you're likely to think
that your high performance is

not really your performance.

Hopefully, it helps
you remember.

Regression and
internal validity.

Regression threats only occur in

pre test post test designs.

Specifically, they only
occur when a group has

an extreme pretest
score high or low.

You can anticipate
that the scores of

those participants will regress

towards the mean at post test.

One example of the

regression threat
that we talked about,

of course, will be
the depression score,

which we'll talk about
these graphs shortly.

But another one is the
campers with rowdy behavior.

Maybe they scored really,

really high on rowdy behavior,

but that's an extreme score.

They had to go back
to their meme.

Their average rowdy behavior
is probably not really high,

so they would expend a lot of

energy. Does that make sense?

Can you give me
another example of

regression to the meme?

Brick's like, no,
don't pick on me.

Alsia or Haley.

Would it be like like
basketball players doing,

like, really, really
well right before

their big NBA game of
the year, whatever.

I don't know the terms. But then

that's not normally
how good they perform.

They're just trying to get into

the final game or whatever.

Then they regress back to
their mean after the big game.

Yes. We all have sort of a
set baseline of our behavior.

How talkative we are in class.

Alsia do you talk this
much in other classes?

I know.

No. Because this is, like,

a high level of
talking in classes,

you probably regress
to the mean in

your other classes and
don't talk at all.

Is that true? Exactly, yeah.

That's true for me, too,
because I am an introvert.

On Mondays and Wednesdays
when I teach this class,

and when I teach
the other classes,

I'm talking a lot and
a lot and a lot for

like an hour and a
half to four hours.

After I'm done teaching,

I just go in the living room
and I turn off the lights,

and my boyfriend knows
not to bother me

because I regress to
the mean and I have to,

like, power back up.

Does that make sense? Is this

a better example of
regression to me?

Brick, do you have an
experience in your life?

Well, that's in a way,

to me, I'm relating to that.

After two hours of
focusing on the class,

that's why I got to
regress to my mean

of calm and take a walk
to counterbalance.

Yeah, it makes sense.

You need to reset.

I just want you to
think of those examples

in case it comes up on the exam.

How can you prevent
regression to the meme?

You look at comparison groups
and inspect the results.

If you look at the top graph,

you can see that both groups

have a similar level of
depression at pre test,

but the therapy group
had lower levels

of depression at post test.

Regression can be
ruled out so here,

it's telling us
that pre therapy,

the no therapy and
the therapy group

had similar levels
of depression.

But after receiving therapy,

the therapy group decreased into

depression more than
the no therapy group.

If it was truly a
regression to the mean,

it would look
something like this.

Let's talk about this one.

In the pre therapy group, Sorry.

In the pre therapy
condition or time,

the therapy group
had higher levels of

depression relative to
the no therapy group.

But after receiving therapy

or in that time
period of time, too,

what they found is that

the no therapy group didn't
change in depression,

whereas the therapy group
did decrease in depression.

This shows us an example
of regression to

the mean because in pre therapy,

the therapy group scored
really high on depression,

and then post therapy,

they scored much lower,

whereas there were no changes
in the no therapy group.

There's regression
because they start off on

a more extreme score and
then change after time.

Does that make sense in
terms of regression? Brick.

But that's the middle one,

the one we were
just talking about.

That's like what you want.

That reads right.

That's not necessarily
what you want.

Because this is telling us that

these people might have

changed just because they

were very extreme
in the beginning.

It could be what we want,

but it doesn't rule out
regression to the mean as

a potential cause because

they were very extreme
in the beginning.

Okay.

Does that make a
little bit of sense?

This could be potentially
what we want,

but we can't rule out regression

because of the extreme scores.

What about C?

The bottom graph depicts the
therapy group starting out

at a higher level of

depression than the
no therapy group,

but this pattern is
reversed at post-test.

This is reversed at post-test

when no therapy group had

higher depression scores
than the therapy group.

This couldn't be accounted
for by regression alone.

Therefore,

the independent variable
must have had an effect.

Because we have this
crossover effect,

this is basically what's telling

us what the average should be.

This is what the average
should be for typical people.

That average, it should
be for typical people

isn't really different than
the treatment condition,

then it's telling us that
there might be issues.

In this case, because
there is an extreme score,

it's telling us that
regression might be an issue.

But here, because they were
extreme to begin with,

and then they went extremely low

afterwards or eliminating
any regression.

We're saying that people are

really depressed to begin with,

and they got much better
than they would be expected

to if they were in the
no treatment condition.

Does that make sense? Rick,

if it doesn't can
explain it another way.

Well, it makes sense
when you said when

that straight horizontal line

is a no treatment condition.

I guess, put like
that, it makes sense.

I just thought it was like an

average condition or something.

No, it's a no treatment group.

It's like what would happen if

these people just didn't

receive any treatment
across time?

What would happen naturally?

Alycia, does that
make sense? Lily?

It does.

How does it affect
internal validity?

There's another
factor that could

account for the results in
your dependent variable,

and how can they be
prevented by having

a control group they
can compare to?

What about attrition threats?

We have a lot of these.

This is another dataset

where we actually
collected data from

freshmen to seniors
at UC Santa Barbara.

Attrition, does anyone know

what the word attrition means?

It's okay if you don't.

I think I only know because
it's in psychology.

This is a reduction.

It just means reduction,
but in this case,

it means a reduction
in participant numbers

from pre-test to post-test.

Attrition is only problematic
when it is systematic.

In the top graph, two
people drop out of

the study from
pre-test to post-test,

and those students or
those people who dropped

out scored really
high on Time 1.

That might be an
attrition threat.

This decreases the post-test
mean dramatically.

You don't know whether
it was the attrition.

These two people leaving or

the fact that they
scored really high on

it that led to the change in

the dependent variable,
which is the score.

In the bottom graph, attrition
isn't really a factor

because people who drop out have

scores that are closer
to the average.

If they drop out, the average
doesn't really change.

Therefore, the attrition
isn't a threat.

Another way to think
about it is right

now we're running an analysis
with the full dataset.

We have like 300 people that
we began with at Time 1,

and then like at Time 8 or 9.

I don't remember how many
times we collected data,

but many times, we
have about 200 people.

We have about 100
people that we lost,

and it could be that we lost
these people because of

a specific reason or
they just decided

not to participate
for a random reason.

But some ways that
we can look at it,

there's analysis
that I have to do

this week when I
finish teaching,

is to assess whether

our participants are
missing at random.

There are some statistical
techniques that you can use.

But let's say that in a
study that we did use,

I should share this favor

with you because you
probably think I'm lying,

but we basically
fat shamed women.

I didn't was one
of my colleagues.

Don't get angry at me. We had

overweight women, and
we recruited them.

I'm overweight,
too, so I feel it.

They were either in
a condition where

they were fat
shamed or they were

made to feel embarrassed
about their weight.

They were in a dating
scenario where very fit male,

they're all heterosexual, too,

would interact with them,

and it was a pretend
date scenario

where they would feel a lot of

self-consciousness or

a scenario in which
they didn't have

to really show their weight
to the other person.

But we would have attrition
if more people in

the fat shaming condition quit

the study relative to
the other condition.

We're saying that any effects

on the dependent
variable that we

see aren't necessarily due to

our manipulation of the
independent variable,

but might be because
we fat shamed women,

and they got upset,

so they dropped
out of the study.

That means that it's systematic.

It happens in a
specific condition.

Does that make sense? Alycia?

Attrition has to do
with the number of

participants you have at

any given time during
the experiment?

It has to do with one,

the reasons for why they quit.

Is there a reason that's
common among them?

Or is it just chance
that they dropped out?

If we had an equal number
of participants in

the fat shaming versus the non

fat shaming condition drop out,

we could say, well, maybe people

are experiencing midterms.

They're really stressed out.

But if we have only people in

the fat shaming
condition drop out,

then we're like, well,

it's because they're
experiencing this threat

and our independent variable

might be too threatening
and they're dropping out.

Thus this is an
attrition threat.

Our independent
variables causing

people to drop out of our study.

That's one, you lose
people systematically.

Two, if people that are either

really extreme scores or

really extreme scores
drop out of your study,

that affects your scores on
your dependent variable.

What we're looking
for here is just,

we're talking about
internal validity,

we're talking about
whether there is

anything like a little girl
hiding behind the tree

that could explain changes
in your dependent variable.

If we have all the women in

the fat shaming condition quit,

then when we run
the analysis and we

found significant
differences, yeah,

it might be because
we fat shamed women,

but it could also be

because some of the women
dropped out of the study.

There's an alternative
explanation for

the changes in our
dependent variable.

Is that more clear? It's
okay if you say no.

That makes more sense.

Rick?

It makes sense.

How can it be prevented?

One way that it can
be prevented is if

these people drop
out of a study,

remove all of their scores
completely from the data.

You remove them from
pre-test average,

so it doesn't affect your scores

on the dependent
variable as much.

The approach that we're taking

because we want the
analysis that we are doing,

you'll probably learn about 467,

468 requires a minimum
number of participants,

and we're barely
meeting that number,

so we don't want to
eliminate any people.

We're looking at pre-test
scores of dropouts.

If they have extreme scores,

then they're more
likely to threaten

the internal validity than
they have moderate scores.

We're looking at
their scores to see

whether they have
very extreme scores,

and if they don't going
to include them in

the data because we
can argue that they're

not necessarily affecting
our dependent variable.

Does that make sense?

Rick? It's a lot.

My mind was wondering.

No, it's okay. Ways
to prevent it,

drop their entire beta

set or look at the pretest
scores of dropout,

see if there's a specific
reason why they dropped out.

If they aren't extreme scores,

you can include them in
the dataset so that you

can still have more data points

to make stronger predictions.

That makes sense.

Haley, how are you doing?

I think I'm getting it.

It depends on if they're
fitting within that average,

they're just put them back in

because they're not
going to affect it.

They're not going to affect it.

It's no harm in including them.

But there is a harm in including
them if you're saying,

Time 1 scores are

significantly higher
than Time 2 scores,

but their mean only might be

higher because of
these two people.

I got it.

But if you remove
these two people,

then you can look at well,
actually, I can't cover it.

Pretend these two
scores don't exist.

If you look at just
the orange dots,

they look almost
exactly the same.

Then you would say there's

no significant
difference on whatever

your score is between
Type 1 and Time 2.

These people would be affecting

your interpretation
of the results,

whereas if these people
you remove them,

their average would still
look the same as that one.

It wouldn't affect your
interpretation of the results.

Is that This is tough.

Yeah. Okay. You can tell me
if it doesn't make sense,

and then I'll try to figure out

different ways to
talk about them.

But we'll review it on Wednesday

to see how well it's stuck.

Okay, so testing threats.

So what's a testing threat?

There's a type of order effect

in which there is a change in

participants as a
result of experiencing

the dependent variable
to test more than once.

Let's say that I
was testing people

on let's say that I was

testing all of you

on the same questions
over and over again.

I tested you on the questions I

gave you today and you did,

let's say, an eight out of t10.

Then I tested you
again tomorrow,

and you got a 10 out of 10.

One argument that
could be made is

that I'm a really great teacher,

and I taught you
really well today.

That's why you
increased in scores.

But the testing
threat would argue,

I didn't really do
that great of a job.

It's just because you've been
exposed to the questions,

and that's what made you improve

your performance.
Does that make sense?

Yeah.

Okay. Great. Scores might
go up due to practice.

It's called a
practice of effect.

Or their scores
might go down due

to fatigue the fatigue effect.

You might be annoyed with
me and be like, Oh, my God,

I've already seen
these questions

and actually gets more wrong.

Testing threats affect
internal validity

because it's not clear
if the treatment

caused a change in the
dependent variable

or whether practice fatigue did.

Here's the ability score.

Let's say the ability
on the test, pre test,

you don't do so hot,

post test, you do better.

Again, a treatment
group can help you

avoid any testing threats.

Let's say that in
the treatment group,

this is when you were exposed
to me an awesome teacher,

and this one is when
you were just exposed

to some random person on
YouTube teaching you.

If I wanted to
argue, for instance,

for a higher raise, saying
that I'm an awesome teacher,

and I increased
your performance,

I would need to show that there

was a significant difference

between the comparison group
and the treatment group.

It looks here like at post test,

I did do a better job than

a random person
that you could have

seen on YouTube in terms
of improving your ability.

Well, that's good.

[LAUGHTER] This is
just an example.

I'm just trying to give you guys

examples that might
be relatable.

Cool. All right.

More questions or
does this make sense?

It's just like being exposed to

the test can improve
your performance,

or it can worsen
your performance.

What's an example where it
can worsen your performance?

Were being exposed to the?

Yeah, burnout but, like,
give me an example.

Well, I go through
both of those things

on our test when we get

a chance to take them
too, the quit twice.

The quiz is a perfect
example is like,

my God, if I got a 90,

do I want to go through
that again, you know?

Because it's a real burnout.

I focus so much and
to do it again.

If it's a longer test,

that might affect my
score or my burnout.

Let's say you start off
with 100 units of energy.

When you take the test,
you get frustrated.

Then take the test again
you get an eight out of 10.

Now, you only have
70 units of energy,

so you have to take it again.

You might either do worse
because you're fatigued,

you're missing 30
units of energy,

or you might do
better because you're

exposed to the same questions.

That's a great example
of a testing threat.

What about
instrumentation threat?

This one, I think is the
easiest of all of them.

This is when the observers
change the criteria over time.

Let's say that I
have a test at time

1 and a test at time
2, and at time 1,

I give you really,
really hard questions

and you all do not
hot, not great.

But then at time 2, I give you

really easy questions
and you do better.

That's an instrumentation
threat because I'm giving you

different ways of measuring
your performance.

At time 1, I'm giving you
really hard questions,

and at time 2, I'm giving
you really easy questions.

It isn't that I
taught you better,

it's that I gave you different
ways of measuring ability.

The researcher uses
different forms of

a pretest and post test
that aren't equivalent.

One way to prevent this is to

use a post-test only design.

Randomly assign people
to either have me as

an instructor or random YouTube
person as an instructor,

and then measure
their performance on

moderate pace or moderately
difficult questions.

But if you do need a
pretest post-test design,

you should make sure
that the pretest

and the post-test
forms are equivalent.

I really wanted to see

whether my performance as

an instructor affected
your learning,

I would have so this is what
you guys are taking, right?

You guys are taking a pretest

to see what you know
about research methods.

That's one of the extra
credit assignments

and then you're taking

a post-test to see how

well I did in
teaching and how well

you retained the material.

How can they be prevented?

Let's see. How can
they be prevented?

A post-test only design or make

sure that pre test and
post tests are equivalent.

Post test would be
between subjects,

so I would randomly assign
you to either me or YouTube

and give you same level
of difficulty questions,

or I would have a pre
test and a post test.

Kind of what you're doing in
this class. Take a pre test.

How much do you know
about research methods?

Take the class and then see

how much you learn about
research methods in the class.

What's the difference between

instrumentation versus
testing threads?

Because they both have to do

with maybe seeing
something repeatedly.

But instrumentation is the
measurement instrument

has changed from
time 1 to time 2.

I use either really
difficult or really

easy questions
depending on the time,

and testing effects happen when

the participant changes
from time 1 to time 2.

Testing is that you actually

got better at knowing
the material.

You got practice with

the material or you got
fatigued from the material.

This would be testing.

It's something that
changes within you.

Instrumentation,
it's something that

happens on the measure.

It's either really hard
measure or really easy measure

or we have one measure
of depression,

then we change it to another
measure of depression.

That's the difference between

instrumentation and testing.

Instrumentation is more
about the instrument.

Testing is about the participant.
Alisa you are thrilled.

Instrumentation is
like you're changing

the methods you use
throughout the experiment.

The methods might be the

same except that the
measure is different.

Let's say that you were looking

at whether you can
make introverts more

extroverted by making them

attend a party with
mostly extroverts.

You measured introvert.
You're like,

It's not going to work.

But let's say
measure introversion

with one personality scale,

and then after the party,

you measured introversion with
another personality scale.

That would be an
instrumentation threat.

A testing threat might be
something like if you measured

the extent to which
introverts are

happy after attending a party,

you're like, Yeah, introverts

just an argument might be made.

Introverts just need to
hang out with more people,

and they'll become extroverted.

You take their levels
of happiness at time 1,

and then you take their
happiness at time 2,

which is after the party,

and they might be more upset,

not necessarily
because of the party,

but maybe because of the amount

of time that they
had to spend with

other people or other factors.

Yeah. Does that make sense?

The participant or the
instrument changes.

These are examples
I'm just coming

up from the top of my head,

so if they're awful, I'm sorry.

But I can come up
with different ones,

too. Combined threats.

Sometimes the threats combine
to create a super threat.

There's a selection
history threat.

This is when an outside event

or factor systematically affects

participants at one level of
the independent variable.

Let's say that students at

one university were in
your treatment group,

so students at PSU were
in my treatment group,

and students at WU were
in my control group,

and I wanted to see the effects
of meditation on stress.

Let's say that during
the course of the study,

a stressful event occurs at PSU,

that would affect the
results of your study.

That's a history threat,

and it's also a selection
threat because it

only occurs at one
of the two places.

What about a selection
attrition threat?

This is when participants in one

experimental group
experience attrition.

That's what I was
talking about with

women that are fat chained,

we call it weight stigma.

If we have more participants

in one experimental group
experience attrition.

If more people in the weight
stigma condition drop out,

then that is a selection
attrition threat.

Let's say the participants
in one study or

one group have to travel
one mile for the study,

and participants in the
other group have to

travel 20 miles for the study.

You might have more people
drop out in which condition.

The 20 mile group?

Oh, yeah.

Zone it out. Yeah.

The people that got
to drive further.

They have to walk. You have to

walk either 1 mile or 20 miles.

Oh, what? Oh.

Yeah. I might walk.

You might have more attrition in

the 20 mile group due to
the distance from the lab.

You can't be sure if

the differences between
the groups were

due to the independent variable

or the distance in attrition.

All these threats to
internal validity

are essentially saying,

there are all these
other things that

could be causing your
dependent variable,

and you need to
be aware of them.

We're doing a lot of

the things that we should be
doing in your experiments or

when I talk to you about

your experiments to make
sure that there are

no other variables that

could explain changes in
your dependent variable.

Like for Rick, we talked about

Pepsi versus Coke drinkers,

Alesia we could talk

about the extent of whether
people live alone or not

during the quarantine
because that could

affect their levels of
introversion versus extroversion.

Maybe people can change.

I feel like we're all
becoming a lot more awkward.

I don't know if that's
correct or not,

but so I'm fully vaccinated.

One of my friends
is too, and we had

a distance lunch recently.

I was just really awkward.

I mean, all of it
is really awkward.

Like, you have to sit far apart,

and then anyway, so
selection attrition threat.

Yes. Does that make sense?

We're redefining our
cultural interactions.

There you go. Yeah, we are.

It's really weird and, like,

when we started zooming,

I hate I've never had
to look on my face

before this often [LAUGHTER]

I know. I'm hard.

Yeah. Especially when
you're teaching.

Any other comments or questions?

See, that's another
interesting Razors question.

Like, how are we changing

our cultural expectations
about social interactions?

Right now, right in this moment.

It is very, very
awkward and weird.

Some people, okay,
so short side note.

Some people are worried
about children not being

able to detect
emotional expressions.

Especially among
infants and toddlers,

because that's when they

learn what happy face

and angry face and all
this stuff looks like.

If they're going out
and they're I can't

imagine being an infant
or toddler right now.

Oh, no.

Your norm is just
seeing, people's eyes.

No, but they actually
had some research,

and they showed that
their detection

of emotional expression
isn't changing.

But it must be really
weird for, like,

little babies and toddlers,

like in the next few
years and they're like,

why isn't anyone wearing a
mask anymore? If that happens.

I kind of have that experience
at work, 'cause, like,

I think most of my expression
comes from my mouth,

like, whether I'm
frowning or whether,

like, there's a small smile,

but, like, we have the mask,

so I feel like people when
they're talking to me,

they can't tell, like,

I might be smiling, but

they're probably
looking at me thinking,

She's so pissed off

right now [LAUGHTER]
even though I'm not.

I do a more squinty smile now.

Like, that's my smile when I'm

wearing my mask just to
remind people that I'm,

it's very, very strange.

I have two more
slides to go before I

switch to something else.

This one should be
easy relative to

the other internal threats.

Three potential internal
validity threats in any study.

Now that we've talked about
the other six, observer bias.

This is when the
researchers expectations

influence how we
interpret the results.

An example might be for
the depression study.

Doctor Yuki might be
a biased observer

of patients depression.

She expects to see our
patients improve whether they

do or do not. She
might rate them.

Instead of having them fill out

a self support on depression,

she might rate how
depressed they seem.

She might manipulate the data

to make it look like they
seem less depressed.

Nickle may also be a biased
observer of his campers.

He might expect the low
sugar diet to work,

so he views the boys post test
behavior more positively.

This is one of the
reasons why we

have confederates or other
people rate our data.

For instance, I have

research assistants who
I might say, like, oh,

can you rate this
data for positivity,

negativity, all these
other adjectives.

But I'm really looking for

is how biased people's
responses are.

But by including
different measures

and by not letting them
know what my hypothesis is,

I'm removing observer bias

because I might if I were
to look at the data,

I'm like, oh, this
guy's clearly biased.

I might inadvertently
make my data look

and analyze it in

such a way that it would be
statistically significant.

That's why there are lots
of checks to make sure that

your data are interpreted

in the most objective
way possible

to reduce observer bias.

Although comparison
groups can prevent

many threats to
internal validity,

they don't necessarily
control for observer bias.

Even if doctor Yuki used a
no therapy comparison group,

observaive bias
could still occur.

If she knew the participants
were in which group,

her biases could lead her
to see more improvement

in the therapy group than
in the comparison group.

We have to be aware
of our own bias.

A way to circumvent
that also is by using

self report measures because

participants have to
answer those questions,

and it's not us
affecting the data.

However, when participants
are affected,

this is called demand
characteristics.

Demand characteristics
is a bias that

occurs when participants
figure out what

a research study is
about and change

their behavior in the
expected direction.

Observer bias is about

the researcher changing how
they interpret the results.

Demand characteristics is about

the participant figuring out
what the study is about.

Sometimes participants want
to be good participants.

I think most of
the time they do.

They might answer your surveys
in a way that supports

your hypotheses or what they
think your hypotheses are.

Sometimes we have what is it?

Antagonistic
participants, so they

figure out your hypothesis,

and then they're like,

I'm not going to prove
your hypothesis,

right, or I'm not going to
support your hypothesis.

They answer questions in
a complete opposite way.

One way that we can control
for observer bias and

demand characteristics is by

using a double blind mask study.

What this means is,

I, as the experimenter,

would create a sheet
that had numbers on it.

I would be the one
who knows that one

equals pre COVID and two
equals during COVID.

I would be the one who knows

what the experimental
conditions are.

But let's say Alesia and

Rick and Hey are my
research assistants,

and all you know is,
if there are one,

you put them on this
computer, if there are two,

you put them on that computer.

You as experimenter don't know

what condition the
participants are in,

and the participants themselves

don't know what
experiment they're in

either because they only get

exposed to one of
the two surveys.

Does that make sense? If you as

a research assistants are

they're in the room
with a participant,

you don't know what
condition they're in.

You can't affect their behavior.

You can't interpret
their behavior in

a way that's consistent with

your manipulation of the
independent variable

because you don't know
what condition they're in,

and the participants can't

behave in a way that's
consistent with

your hypotheses because
they don't know

what condition they're in or
how many conditions exist.

Does that make sense? If you
were in one of my studies,

you might be in one of

four conditions or one
of five conditions.

Even if you tried to guess
what my study was about,

you might not be correct.

That's one way to avoid
demand characteristics.

If a double-blind
study isn't feasible,

sometimes it's really
difficult, especially,

let's say that Computer 1
breaks down and you're like,

oh, my God, Computer
1 broke down,

so I can't assign
participants to that.

Then I would have to tell you,

click on the "Pre-COVID"
survey for this person,

then click on the
"Post-COVID" survey.

That would be a mask study.

The participants might know
which group they're in,

but other people might not.

They might know based on

the survey tab that they're
in the pre-COVID group.

You might know that they're
in the pre-COVID group,

but someone else who might be

observing the study
and looking at

the behavior might not know
what group they're in.

We would try to mitigate

observer bias by doing

that and also demand
characteristics.

But the best, of course,

is the double-blind, but

it can get pretty
complicated really quick.

Another way that it can
get really messed up is,

let's say, Condition
1 is pre-COVID,

and Condition 2 is during COVID.

I wrote that down somewhere,

and then I threw away the paper,

and then all I have is

there's people in Condition
1 and Condition 2,

and I'm like, I don't
know what they mean.

That's another way that
double-blind studies can fail.

The placebo effects. You
guys like placebo effects.

Everybody likes them, because

this effect is
present when people

receive a treatment and improve,

but only because they
believe that they are

receiving a valid or
effective treatment.

There are ways
that we can design

a study to rule out
the placebo effect.

This is a double-blind
placebo study.

Let's say that we
had this experiment.

Participants are
told that they're

receiving a new therapy,

pill, or injection, but in fact,

it's missing the
active ingredient,

or they may be told that

they're getting a
new type of therapy,

but in fact, they simply

chatted with someone and
didn't get any therapy.

Nonetheless,
participants may improve

because they thought that
they had the therapy.

It's important to note that

placebo effects
aren't imaginary,

but placebos can be
strong treatments.

How can we design studies to
rule out the placebo effect?

Have any of you experienced
a placebo effect?

Probably.

Can you think of it? It usually

happens below
conscious awareness.

Can you think of a time?

Have you taken emergency,

and that made your cold last
a shorter amount of time?

Or you avoided getting

a cold because you
took an emergency?

There are a lot of
things like that

that aren't empirically studied,

necessarily, but that we all do.

Even as scientists, that we
might do things like that.

No placebo effect?

In the figure on the left,

so this figure right here,

you can see that both
groups symptoms.

We have symptoms on the y-axis,

pre-test and
post-test ratings of,

let's say, pain across time.

This is pre/post symptoms pain.

You can see that both groups
decrease across time.

But the group in the
true therapy condition,

the teal color,

decreased more in their
post-test rating relative

to those in the placebo therapy.

It suggests that the
therapy had some effect.

There was a placebo effect,

but there was also a

greater-than-placebo effect in

the true therapy condition.

But there could also be

other internal validity threats

that could have
explained that result.

Again, that could have
been due to maturation,

could have been done to
history or aggression,

testing, or
instrumentation threats.

Maturation, something
happened across time.

Maybe their pain just
decreased because

their body is healing, history.

What could be something
that's a threat

to history with decreased pain?

Maybe a bomb went off

and other parts of
their body hurt,

like their arm had
to be amputated.

Their knee pain seems

a lot less severe relative

to the arm pain that
they're experiencing.

That might be a history threat.

Rick, you seem surprised.

I'm just trying to
think of something,

a historical event that
could happen that could make

your knee pain seem less severe.

If something bigger
than you happens.

What about regression?

This might happen
when you go to the

doctor and they ask you,

on a scale 1-10, how
much pain are you in?

There's a long line,

and you might say, I'm at
a nine when you get there.

But then when they
see you in the room,

you're like, not really.

It might be a seven.

That could be a regression
test, a regression threat.

Testing could be what?

A threat to testing or

a testing threat is when
the participant changes.

They see a nurse and
they mention something,

and the nurse is like,
oh, that's really bad,

or oh, that's fine.

Then when they see
the actual doctor,

based on what the nurse said,

they could be like,

but it's not that bad oh,

but it's actually really bad.

Some outside influence can
change your own perception.

That's a good example of
testing and instrumentation.

Let's say that I
measure the amount of

pain in your injured leg by

squeezing it at the
first amount of time.

At the second amount of time,

I just give you a survey.

Of course, you're going to have

a lot more pain when I squeeze

your injured leg versus when
I just ask you about it.

That would be an instrument
of threat because I use

different measures
of pain or symptoms.

I just wanted to go through
all of them, maturation,

history, regression,
testing, and

instrumentation. Did all
of those make sense?

History would be
the bomb going off.

Regression would be like,

before you see the
doctor, you want to get

in, and you exaggerate.

But then, when you
actually see the doctor,

you know that they're
going to be able to tell

how much your pain
is, so you reduce it.

Testing is you have a
certain threshold for pain,

so you're like, oh, my God,

I'm experiencing a
Level 8 of pain,

and then the nurse sees
you and she's like,

no, that's not really a Level 8.

Level 8 is like you're
giving birth or something.

You're like, so that
changes your anchor.

Then instrumentation is using

grabbing your injured leg versus

giving you a survey.
Bring it all together.

In order to determine
whether there is,

in fact, a placebo effect,

the researcher might add a third

comparison group that doesn't

receive any therapy at all.

They don't receive
the true therapy

or the placebo therapy.

They don't receive
any therapy at all.

If you have a placebo
effect, be here.

If you have a placebo effect,

then no treatment group should

improve much as a placebo group.

Here, we have the
no therapy group,

we have the placebo
therapy group,

but we do have a
placebo effect because

both the placebo group and

the true therapy groups seem to

decrease in symptoms
across time,

and that seems to be greater

for them relative to
the no therapy group.

The placebo does seem to
have a greater effect on

symptoms. Does that make sense?

The true therapy might be going

to the physical therapist.

Placebo therapy might be
going to me and unlicensed.

Not at all expert on
physical therapy,

but I might just, like,

squish your leg around.

That's a placebo therapy.

Then no therapy is
you just go home.

Me just squishing your leg
around might help you feel

better because you think
it's an actual treatment,

relative to the
no-therapy group.

But the true therapy
group where you actually

get treatment by
physical therapist,

experiences the highest level
of decrease in symptoms.

Is that good?

Sure.

Do you want to explain it
in a different way, Rick?

No, I was definitely
following you. For sure.

The placebo effect can
still have effect,

but it doesn't mean
it's still imaginary.

It's imaginary, but
it's doing something.

This is another threat
to internal validity.

Like all the other threats
to internal validity,

including a comparison
group, which in this case,

is a group that receives
no therapy at all,

can help you see
whether you have

a placebo effect or not.

We're done with
this one for now,

we'll continue on Wednesday.

Let's go to the next
PowerPoint. Oh, no.

Where is it? Let's see.

Do you guys have anything
exciting coming up?

No.

No?

My birthday's this month.

I'm not going to do
anything what can I do?

But last year for my boyfriend's
birthday, he's vegan.

Well, he can't have dairy,

so we usually just get vegan
cakes 'cause it's easier.

I made him Atres
etches, vegan cake.

Do you guys Atres etches?

Oh, you should have
it. It's really good.

It's basically three milks,

but it's usually like
regular milk and

condensed milk and evaporated
milk or something.

It's a cake that's

sort of very moist and
has a lot of liquid,

and you dip it in the liquid and

it's so good and delicious.

Anyway, last year I made,

I'll send you or I'll show
you a picture next time.

I made him a vegan Dzeches,
which I just talked about.

It has three different milk.

I had to use, like,
coconut milk and

almond milk and some other milk.

But it came out really good.

I might start practicing that.

For my birthday.

There might be a lot
of cake in my life.

That's the most exciting
thing going on so far.

Yeah. All right,

Haley, anything exciting
coming up for you?

Um, it's my boyfriend's
birthday this month, as well.

What are you going to do
or when is his birthday?

The 14th.

We're probably just going
to get sushi and hang out.

To go sushi now,

'cause the restaurants in

my county are shut
back down again, so.

Yeah. Sushi is awesome.

I love sushi.

Yeah. I didn't have
sushi until I was like,

22, because I was
like, Oh, raw fish.

Then one of my friends is
like, You have to try it.

So I did, and I'm
like, Oh, my God.

Then when I went to UC Santa
Barbara for my PhD, like,

I would have poke bowls
multiple times a week.

Have you guys had a poke bowl?

I've had one once.

Oh, yeah. It's so good.

They don't have them
as much in Oregon,

but in California,
they had, of course,

everyone hates
Californians, but they had,

like a subway poke bowl
where you would be able to,

like, choose your
grain and then choose,

like how much stuff
you want in it.

That's what I miss. And
now I want a poke bowl.

Okay, so let's talk about
writing a method section.

Why we should write
a method section?

One, you want to
be able to write

a method section in
case anyone wants

to do your study again
or extend your study.

When I read your method section,

I should be able to say, Okay,

so Alesia measured extra
version using this,

and then she measured anxiety
symptoms measuring that.

I want to add in
gender as a factor.

I'm going to use
Alesia's method and

include gender as
a factor to see if

I get similar results to her.

That's essentially
what we're looking for

when we look at a
method section.

The section will allow you
to see who was recruited,

which affects the
generalizability of your results,

how the variables
were operationalized.

We measured loneliness using
the UCLA loneliness scale,

for instance, and what the
outcome measures were.

All of this method section

allows us to evaluate the
strength of the study,

identify any confounds and
extend existing research.

If I knew as

a soda researcher that coke
people are more lonely,

I would want Rick to

measure whether people are
Coke or Pepsi drinkers,

so we can address

that as a potential
confound in the paper.

Okay.

How do you format
your method section?

Again, like the
rest of your paper,

you're going to use past tense.

At this point in your writing,

when we actually
write the paper,

you would have already completed

the research topic,
or research project.

Make sure to center the
word method. No methods.

It's not methods, not plural.

It's singular method.

No bold, underline or italics.

Subheadings are flush left,

and italicized for
participants materials,

procedure design and analyses.

Okay, what about
the subheadings?

Participants,

how many participants
do you want to recruit?

The rule of thumb is 50
participants per cell.

What the hell is a cell? We'll
see it on the next slide.

But that's changing,
and it's unlikely that

you'll get approximately
50 participants per cell.

We talked a little
bit about power.

And in the last
four to five years,

we've been under a lot of
scrutiny as psychologists.

We've created different kind of

statistical techniques
that allow us to say,

if we want to make
sure that we have

less than a 5% chance or

5% rate of getting results
that are due to chance,

then we must recruit at
least 190 participants.

But before that, people
were like, Okay,

you can get 50
participants per cell,

which will be on the next slide.

Let's say that my
research question is to

investigate how music affects
athletic performance.

Music is an
independent variable,

and athletic performance
is a dependent variable.

Then I have my participants

listen to a specific
kind of music.

Independent variables music.

Yes, and the independent
variable has

two levels, techno or classical.

I have one dependent variable
of athletic performance,

and we base the sample size
off of the levels of the IV.

One cell means one
level of the IV.

Because you have one

independent variable
with two levels,

techno or classical music,

you would need 100 participants.

I think I have um a
graph on the next slide.

Okay. Do those numbers
track for you?

Yeah. Okay, so let's say that

I also want to look at gender.

I categorize people as
either male or female.

Now I have two independent
variables, music and gender.

Music still has two levels,

techno classical,

and gender has two
levels, male or female.

Now we have four cells.

We still have one dependent

variable athletic performance,

and we base a sample size off of

the number of and
levels of the IV.

Thus we would need 200
participants in my study.

This is what I'm talking
about when I say cells.

Hey, this is going
to be like yours.

You're going to have
male and female,

and then here you're going
to have the two age groups.

You would need about
200 participants.

Does that make sense?

Yes.

You'd have males in
their 20s and males

in whatever the
other age group is,

and then you'd have
females in their 20s and

females in whatever the
other age group is.

Alesia, you only

have one independent
variable with two levels.

How many participants
would you need?

A hundred.

Yep, Rick.

One hundred.

Yeah. Great.

Yeah, so let's just pretend
this and this is yours Rick.

So all people measure

loneliness and then we

have people pre COVID
and post COVID,

so you would only
need 100 people.

Okay, all right.

So that's just
about participants.

I want your number
of participants,

I also want any
screening criteria.

Are there any criteria that
you're using to screen them?

Are you making sure that

they aren't
clinically depressed?

Alisia, are you
making sure that they

all have diagnosed anxiety?

Because it would suck if

you had introverts
and extroverts,

but then none of
them had anxiety,

so how are you going to measure

differences in anxiety symptoms?

So your screening criteria
might be that they have

anxiety or have been

diagnosed with anxiety
at some point.

Haley, yours might be
that they're actual

working professional in
those fields, right?

Yeah.

Yeah. So you would be recruiting

what kind of people,
like business people?

I think so. Yeah. I
need to think about how

to operationalize

leadership positions
like from [OVERLAPPING]

You're good for the start.
So you will randomly

select five or 10 of
the top 50 Forbes,

whatever, money making
[LAUGHTER] companies

and survey CEOs and

COOs and other leadership
positions there.

So you're going to
randomly select

some of the businesses
and then you're going

to grab some of the participants

from those randomly
selected businesses.

Just aim for the moon
because you're not

really going to do it and
if you want to do it,

you probably can next
year, but we'll see.

Okay.

Okay. Then you'll also
include basic demographics.

So are they all WOU students?

Are they all in the
top 50 Forbes list?

I also want you to tell me about

their age, their gender, race,

ethnicity, any other variables

you think might be interesting.

You can also include
information if it's relevant.

So income levels might

be relevant if you're
studying the effects

of helping people or

helping behavior on
monetary donations.

yeah. So we did a study

with professionals that
make a lot of money,

and we're like, we're
offering to pay them $50.

But they make so much money,
why don't they donate?

Do you think they donated?

No. These people took
their $50 gift cards,

which is fine, but they
make so much money.

So we actually came up with

four different
charities that we all

support and we like

painstakingly wrote
the descriptions

of each charity and we're like,

you can donate to one
of these four charities

or click the last button.

To scroll and click
the last button

if you want the gift card to

yourself and the like 98%
gave it to themselves,

but income levels
might be different.

So maybe if we
recruited students

and we could see
whether students are

more likely to give
donations relative

to working professionals,
that would be interesting.

But that might not be
interesting for your study.

But income levels
might be irrelevant

if you're studying
coffee consumption,

or maybe not if you're

looking at how many times
people go to Starbucks.

I also want numbers.

So were any people excluded
from analyses and why,

you can just, Alisia, you
can make something up.

You can say something like
I excluded 10 participants

because they did not

complete the anxiety
symptom questionnaire.

ErricTingle, you
could say something

like people did not complete

the UCLA loneliness scale

and I had to delete
them from the data.

So for demographics, we
usually include percent

of male versus female,
their age range.

So participants were between
the ages of 18 and 30.

The mean average age was 24
and the standard deviation.

Where were the
participants recruited?

So Haley, maybe you sent

an email inviting them to
complete a Qual trix survey,

or maybe it was post COVID and

you went to one of
the conferences where

the top business leaders go and

you caught people as they were

on their way to
lunch and had them

complete a really fast survey.

Were they compensated, and
if they were compensated,

were they compensated
with course credit

or monetary incentives.

Most of the time,
especially in 467,

468, you'll compensate
them with course credit.

Participants completed the study

for half of course credit.

For the subsection
of the materials,

all materials used in this
study should be mentioned.

So for example, the
Beck Depression

inventory and cite
it appropriately.

Rick, you're going to have

the UCLA loneliness
scale and cite it.

Which measures did
you decide on?

Why? Why did you decide on
the UCLA loneliness scale?

You can say it has
good reliability.

It's been tested across
different samples.

It has good validity,

and it has good reliability.

It could just be a
sentence about that.

If the surveys are used,

please describe the survey.

So a short description of
measure and what it measures.

So the UCLA loneliness
scale measures

loneliness among young adults.

It has 10 items, I'm
just making this up,

and it is a Likert scale where

participants answer items like,

I am a lonely person
on a scale from

one strongly disagree to
seven, strongly agree.

People who score high
on the loneliness scale

tend to be more
lonely than those who

score low on the
loneliness scale.

So what does a high or low
score on the scale mean?

You should include
sample questions.

So like I just did, I
am a lonely person,

from strongly disagree
to strongly agree.

Talk about the
reliability, the validity.

I want you to include all items

from the original
measure in the appendix.

So you will have an appendix
at the end of your paper,

and right there I just want
you to type up the measures.

When using a pre existing
published measure

which you all should be using,

I want you to include
the full name

of the measure followed by

an abbreviation and citation
of the original author.

So one index that I

use a lot is the suspicions
of motives index,

and we shorten it by calling
it SOMI then I cite it.

So full name of measure,

abbreviation, and citation
of the original paper.

After that, you don't
have to call it

the suspicion of motives
index throughout

the rest of your method
or results section.

You can just call it SOMI.

So let me look up a paper

because this is all
sort of, hold on.

Like, I'm just speaking
and it might not make

complete sense and I'm just

thinking of me as a
student and I'd be like,

okay, that's great, but how

is it going to look
like in my paper?

Can you guys see
my search screen?

No.

No. Okay.

[inaudible].

I can't.

Okay. Share.

Cool. Can you see
this paper now?

This is one of my papers.

This is a short paper,

so you can use it to
look at the method.

For instance, method,
subjects in design,

186 Latino students
participated for $5.

You don't have to conclude
this effect size.

This was a headache,
because they made

me rewrite the paper
and include that.

This is a power thing that
I was talking to you about.

We set a goal of 45
participants per

condition and stopped when
that number was reached.

There's a new thing
that we have to do.

Seven participants didn't
complete the PEMS.

Ten failed to indicate that

the Latino candidate was hired,

and six did not
complete the Stroop.

Thus, the final sample was 163.

The average age was this.

We had 118 women,

44 men, and one person
who did not report.

The procedures, they received
an email invitation to

participate for pay and
provided informed consent.

Study A involved reviewing

and evaluating recent
recommendations.

Study B measured
cognitive performance.

This study was conducted
in compliance with

the Campus Institutional
Review Board.

Participants were randomly
assigned to conditions.

Then I talk about the
random assignments

, so candidate qualifications.

People either saw a
job description for

an entry level human
resources position

followed by the resumes of

one Latino male and two
white male candidates.

The race and gender
of the candidates

were manipulated
based on the names,

based on their resumes.

One was clearly the
best qualified,

one was the worst, and the
third was in the middle.

For half of the participants,

the best qualified
candidate was Latino.

For the other half,

the moderately qualified
candidate was Latino.

The worst qualified
candidate was always white.

After viewing all the resumes,

participants saw the human
resources officer's profile,

indicated his race and gender,

and read comments
about each candidate.

They read what they said
about each candidate and

then I also manipulated
the diversity rationale.

Whether we said that the person

would add diversity
to the company or

not and then I add
manipulation checks,

but this is what you would have

with your dependent measures.

For me, it was cognitive
performance on the Stroop task.

You guys remember the Stroop
task where it's like,

it has the word blue,
but it's in red.

You have to say the name of
the word and not the color

, so that's what I used.

You can see my method
section is very short.

I should be maybe
one or two pages.

For instance, 100 participants

at 100 students at

a public university participated

online for course credit.

This could be your study, Rick.

Previous studies, we don't.

All of this is trash.
You don't need this.

You can say seven
participants didn't

complete the
loneliness scale thus,

our final sample was
93 participants.

You can just flub the
numbers for the age,

how many women
versus how many men.

Procedure, participants received

an email invitation to

participate in the
study for course credit

and they were asked to complete

surveys regarding
their experiences

in the last few years.

Participants were randomly
assigned to one of

two conditions and then you
can say COVID manipulation.

Participants saw a
description page

that either said that they would

be answering questions related

to their experiences
before COVID,

or they were asked

to write their
experiences after COVID.

Then that'll be your
whole manipulation here.

Your dependent measure
will be loneliness,

so dependent measure
UCLA loneliness scale.

The UCLA loneliness scale was
used to measure loneliness.

Participants answered
these questions on

a one strongly disagree to
seven strongly agree scale,

and that's basically
your method section.

Does it make more
sense now that I'm

putting it in an actual paper?

Yes.

Yes.

This paper took me
forever to write.

I will tell you, I hate
writing. It's hard.

It is really hard and I
completely understand all of you.

We have 6 minutes left, God.

Step by step of how you
collected the data.

This is a recipe.

I want you to mix your
water, your flour and yeast,

add salt, let the bread
rise overnight, et cetera.

Make sure to include
enough details,

so another researcher
could replicate

your study where were your
surveys completed online,

in person in a classroom.

Was there a research
assistant present and so on.

Again, the procedure right

in the order the
events occurred.

I said I emailed them a survey,

they clicked on the link, blah.

Usually you start with
informed consent.

You discuss the
experimental design,

you discuss the conditions,

how the students were
assigned to condition.

When did the
experiment take place?

What were the instructions to

the participants and
did you deceive them?

Then if you deceive them,

did you include debriefing?

Your method should only

be about a page to
two pages long.

Very short. You can use
my paper as a sample.

I also included
another sample paper,

so let me open the link.

That paper was only
five or six pages long,

but it took me a long
time to get it done.

This is the link that's
included in your PowerPoint.

No I already have it posted
without the questions.

You'll write an abstract later,

but most of you have used this.

I can get everyone
in class right now,

use the thing I gave them,
the little template.

That's the intro. The
method, here it is.

Your participant, describes who

should be in your study,
the research design.

We can talk about that
also on Wednesday.

The measures, which ones
you used and procedure.

That's it. I also
included another paper,

I believe with a Purdue Owl.

I've been using the Purdue
Owl since I was an undergrad.

I would also use this.

They have sample papers.

No, I hate how small this is.

They did a literature review.

All of this sucks. That
would not be good.

One to model. I think
this one has data.

This also has a metanalysis,
so that might not be good.

Any of the papers
that you found for

your review should
also be good for

this to model after. Let's see.

Sure. This is another short
paper, one of my favorites.

Their procedure was just that.

I can also upload
another paper onto

our Canvas page so

that you get a sense of
how to write a paper.

Let's see. I think I
already have some.

Here we go. There you go.

Method, participants procedure,

measures. It could
just be a page.

It's already in our Canvas page.

You can start
writing your method

today or whenever you want.

That was a lot. Do you all
have any questions, comments?

Is this portion
of the paper due,

is it next week or
the week after?

It's next.

This portion of the paper
is due on the 16th.

Not this Sunday but
next Sunday. Sorry.

It's not due. Well, I'm
still going to do it now.

I'm giving y'all a lot of time

to be able to write this.
It's not just this.

It's going to be the
results section,

and we'll talk about the
results section on Thursday,

but that should also
be relatively short

because you don't
have any results.

I just want you to tell me
what analysis you would run.

We'll talk about that
on Thursday as well.

The method will be about a page.

Results will be
maybe half a page.

Discussion will maybe be
between one and three pages,

and we'll go over the
discussion on Monday.

Before this Sunday, we

will have gone over
method, results,

and then you should
work on that over

the weekend, and then on Monday,

we'll talk about the discussion,

and then you'll have the
entire week to work on

the discussion in the IRB.

You have two weeks.
Get started on it now.

I'm also a procrastinator.

I understand. I'm the worst.

I'm giving you way
more than enough time.

Let's just look
ahead really quick.

I know it's already time, but

I'm trying to give you ample
time to be able to get

this done and look for
examples and think

about it before we actually
have to write the paper.

We're almost done with lectures,

but I do want you all

to try to schedule
appointments with me to

review your papers because
I want everyone to

get any of this
class if you can.

We're Week 7. Next week is
the last week of lecture.

The paper must include
revised introduction.

You should already be working
on that, your method, page,

results and data
analysis strategy,

half a page, discussion,
maybe three pages.

We'll also talk about the IRB,

very easy solve that
we'll be due on the 16th.

Then you'll just have
P quizzes next Sunday.

That'll be the end.
Then you'll have

1-3 weeks to write
your final paper.

I'll get your
papers on the 16th.

I'll try to get it back
to you by the 30th.

Then you just have to make
minor edits depending on

how much feedback I give you,

and then you just have

the final exam and
submit your final paper.

But you'll have two weeks
of no classes with me,

and I want you to
use that time to

meet with Writing Center tutors,

meet with me, make appointments.

I hope that will be enough
time for you all to manage.

You'll really like
me, then, because

we won't have to go over
all these lectures.

The exam will open at 6:00 AM on

6/7 and closes on
6/10 at 11:59 PM.

You have three days to take it.

It'll be two hours long.

You can choose any
two-hour block

between those three
days to take it.

It will cover all
of the chapters.

Basically the questions
I've been asking you,

I think will be on
the exam, hint hint.

You can use any PowerPoints

that we have, you
can use any PDFs,

you can use anything
on the website,

you can use a text,
but don't Google.

I can tell when you
Google and you copy.

I've had that happen
before, and no.

Just try to explain
it in your own words.

That's why I try to have
you explain things back to

me because if you
can do that now,

you're going to be able
to do that on the exam.

We're almost done
with this class.

It's going by really
quick. But you guys,

at least your papers
have been really good.

I gave good feedback, I hope,

and you're working on it,

and I look forward to
seeing your second drafts.

If you need help, let me know,

but I think the intro is one

of the hardest parts to write.

The method and the
results should be easy.

Any other questions or concerns?

I was wondering if
I could talk to you

about the scales that

I could potentially use because
I have a couple in mind,

but I want to

make sure that I'm using
the right scales, I guess.

Sure. If no one
else has questions,

y'all can skedaddle, and
I will meet with Alicia.

I guess I'll email

you and ask about the
loneliness scale.

If you can just send me a link.

I'll just send it
to you right now.

That'd be great. That's
why I brought it up.

Please email me if you
want help with that.

I don't want you wasting
two hours of your life.

Thanks. I know it's frustrating.

I knew I shouldn't do it,

but I'm glad I looked up
to see what time it was.

I have a lot of years of

experience banging my
head against a wall,

trying to figure
these things out.

Use my experience. Here we go.

I'm also unsure about

what scales I should
use with mine, I guess.

Rick, I'm going to send
this to you right now.

Alicia, I just have
30 emails now.

Well, then I'm good.
I appreciate it.

I'm going to get started on it.

Thanks for your great class.

It helps. I'm out.

Thank you.

Bye.

I'm going to share
my Google screen.

Just ignore my emails.
It's a headache.

Oh my God. Alicia,

you have an idea of
some of the scales.

Haley, do you have an idea
of some of your scales?

Not really. No, not yet.

Alicia.

You said you had an idea, so.

Yeah, so like some of the
ones that I found in,

like, my reference
articles seem good.

Like for personality, like,

the Eysenck I think let's see,

let me look at what it was.

Well, there's two that I
could potentially use.

There's the Eysenck
personality questionnaire,

and then there's the 10
item personality inventory,

which that one's a lot shorter.

Do they both measure
extraversion?

Yeah, I think they both.

I think they both measure,

all personality like all big
five personality traits.

But I don't know if I want
to use, like, I know,

I think you mentioned it
once before where it's like,

I don't want my participants

necessarily to know
what I'm measuring.

Like, having at least,

like, one other like,

personality trait
might be helpful,

but I don't know how much
is too much, I guess.

Yeah. How long is the Eysenck?

It is let's see.
I think it's 48.

Yeah. I would just say that you

used part of the Eysenck
to measure extraversion,

conscientiousness
and agreeableness

or something like that.

To mask the true
purpose of your study

and then talk about each
of the sub measures.

Extraversion was measured
by asking questions like,

whatever the questions are,

agreeable illness
was measured by

using whatever the
questions are,

and then the other one,
conscientiousness.

You can use that. A
dependent variable,

do you have anxiety symptoms?

Yeah, there's a couple
that I could use.

One of them is the Beck
anxiety inventory.

Then I think I
found another one.

It's a brief symptom inventory,

which that one's like 53 items.

Then, I think it looks
at anxiety and, like,

a handful of other I don't
want to say disorders,

but, like, other types
of mental things.

Illnesses?

Yeah.

At this point, I think it's up

to you what you want to use.

Okay.

I would look through

both of the Beck
depression inventory

and the other anxiety
scale and look to see

whether one has more questions

on it that reveals a
variety of symptomology,

because you're looking at
differences in symptomology.

Does it also look at physical

versus psychological symptoms?

Just use whichever one
you think is best.

I think I looked at one with you

the Beck depression or Beck
anxiety inventory, right?

Yeah.

I think I liked that one,

but as a researcher, these are

the questions that
we have, right?

You just have to bite
the bullet and decide.

I'm looking for the inventory.

Can you read me a few questions?

For which one?

The anxiety inventory.

The Beck one or the
briefs one. The Beck one.

For the Beck anxiety one, it's,

like, a scale from 0-3,

and, like, a couple of them are

like numbness and
tingling, feeling hot.

Yeah, I like this one. I
have it on my computer.

Because I think it has

physical and psychological
measures of anxiety.

I think this one would be
good. This one doesn't seem

like too long to complete
for your participants.

Okay.

You just need to compare
this across the other one,

which is 48 questions.

Yeah. Because the other one
is like a scale of 0-4,

and then it's like in
the past seven days,

were you distressed by
these various things.

Then, yeah, it's like 53 items.

It really is up to you.

I would look at the other one
to see if they have, again,

psychological and physical
symptoms of anxiety because I

think that's what you're

looking for
differences in, right?

I would also include
another dependent measure

maybe something
positive, well being.

As a separate scale or?

Yeah, because if you're only

measuring personality
and anxiety,

then they'll probably
guess that you're

measuring personality
and anxiety.

Okay.

Yeah. Maybe one more measure.

But it sounds like you've
already found some good scales.

I think the problem now is
deciding which one to use.

At this point, I feel
like the Beck anxiety

is pretty widely
and commonly used.

I haven't heard
of the other one,

but I'm also not a
clinical psychologist,

so it's up to you.

Okay.

Yeah.

All right. Thank you.

Yeah, of course. Let me

know if you have
any more questions.

Okay.

All right, Hey, how
can I help you?

Okay. I guess my
real question is,

like, well, okay.

My independent variables
are age and gender,

and then dependent is how
they're treated at work and

I am just unsure of how to,

like, what I would use to

measure treatment
at work, I guess.

Let's see. Based on the
articles that you've read,

so you're going to need to

be more specific about
treatment at work.

Are you talking about
sexual harassment at work?

Yeah. In my introduction,

I made the distinction of
harmful workplace experiences.

That included a lot
of different things,

including, like,
sexual harassment or

just harassment
that's non sexual.

Okay. Here's a sexual
harassment questionnaire.

Or maybe a toxic work
environment questionnaire?

Yeah.

For your independent variables,

you're just basically
going to ask,

what is your gender.
How old are you?

Yeah.

This is a master's thesis.

You might need to
look for that scale.

I think the key here is for you

to more clearly operationalize

what you mean treatment at work.

Do you mean sexual harassment?

If you want that
to be a component,

then you might need to look for

a sexual harassment scale.

Let me try to find one.

I think this racialized
sexual harassment scale,

which you might not want.

Psychological climate
for sexual harassment.

I just want to see it all.

I think the sexual
harassment is like

the angle of what I want to
see within the age groups.

I think the treatment
at work, I guess,

would just be have

you experience sexual
harassment at work? Like that.

This is one climate.

It would be risky for me to file

a sexual harassment complaint.

Complaint would not
be king seriously,

would not be thoroughly
investigated, etc.

This is like the
climate at work.

Let me look at your paper

because I feel like
in your paper you

talked about some scales.

I still need to grade some.

Speed grader. I need to look at.

There are differences
in how people

are treated in the
workplace because of these.

What measure do they use?

Give me a second.
Sorry, no, I find.

That's okay. That article.

It says, Hold it one second.

Sorry. Like within
their method section?

I think they did
a meta analysis.

You might need to look at

the actual articles
that they cited.

But another one
that actually has

dependent measures is this one

a confirmatory study
of the relations

between workplace sexism,

sense of belonging, mental
health, job satisfaction.

So those might be things
that you're interested in.

Sexism, sense of belonging,

mental health, job satisfaction.

That one.

No sorry,

I'm reading to try
and find like there.

It says sorry, go ahead.

I've received fewer
opportunities for promotion.

I have less job stability.

Those are the skills
that you can use.

You can even look
at the schedule of

sexist events by
Klanef and Landren.

You don't have to
create your own skills.

Usually, this is why you
read papers because you want

to see what kind of skills
other people have used.

Let me see job satisfaction,

mental health, using
depression anxiety scale.

Job satisfaction
was measured using

the four item index of
effective job satisfaction.

I find real enjoyment in my job.

I like my job better than
the average person, etc.

Then, in my hypothetical
experiment,

I would just use the
job satisfaction,

a couple of questions
from there,

and the sexism questions
and just mix it up

so that the
participants don't find

out what I'm trying to ask

or hopefully they
don't find out.

Well, you're not going to mix

the job satisfaction
with the sexism items.

You will have them like
in a in a specific order.

Let me show you, like
what that looks like.

Let me show you
one of my studies.

This is what you'll
use in 467468.

Oh, I don't think that
one's the finish.

Oh, this one. It's 68
participants. Skip two.

They either get
geographic mobility

or they get diversity,

and then they get questions
of individual mobility.

Then they get questions
about group status.

Then they get questions about
all these other things.

You can randomize
the extent to which

these like how these
variables appear.

But you want to keep all the job

satisfaction questions together.

You want to keep all the
sexism questions together.

I didn't mean mix them together,

but just include other
scales of different things.

You can. This is my
manipulation check.

Please describe what
the articles about.

What years the article I read
today was about, whatever.

Does that make more sense?

You don't have to
create your own.

I think that's the most
stressful part for people.

You can just look through
the articles that you

cited and see what kind
of measures they used.

You don't have to use all of

the measures that this person,

whatever article we were
looking at used, this one.

You can use some from here.

You can use some from

here if you want to look at
sexual harassment climate.

Excuse me. Sorry.

Bless you.

Thank you. But I would maybe
choose three or four total.

I think for yours,

yours is pretty straightforward.

I don't think you're going to
need extra surveys to mask

your study you could just make
it about job experiences,

and all of these are
about job experiences.

Does that make
sense? Yes, that's

much clearer now. Thank you.

Of course. If you want me

to look at a draft, let me know.

Or if you want to decide
on your measures,

also let me know.

But at this point,
it's mostly about

how you want to operationalize

it and what you want to measure.

Sounds good.

Cool.

Maybe I'll ask you
on Wednesday to see

how your decisions going.
Maybe three or four.

I will look into that today.

Sounds good. Well,

good luck. I will see
you on Wednesday.

See you on Wednesday.