\n",
" Stop and think \n",
" Do these distibutions look identical? What is different and why?\n",
"
"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"These sample statistics follow a t-distribution, which is the basis of the t-test. \n",
"\n",
"You'll note the distributions are quite similar, except there are wider tails in the case with only 6 samples. This is because the t-distribution approaches the normal with more samples (with more degrees of freedom)."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Exercise 2: Relationship between p and t values"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"
\n",
" Exercise 2 \n",
" Using the following interactive widget, explore the critical value of t shown for different degrees of freedom (sample sizes) and alpha levels. Report in a cell below the critical value for a t-distribution with 9 degrees of freedom and alpha is 0.05, the critival value for a t-distribution with 50 degrees of freedom and alpha 0.05, and the critical value of a t-distribution with 25 degrees of freedom and alpha = 0.4.\n",
"
\n",
" Stop and think \n",
" How do the critival values change based on sample size?\n",
"
"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Your answer goes here \n",
"\n",
"- 9 df, alpha 0.05 : crit val 1.83\n",
"- 50 df, alpha 0.05 : crit val 1.676\n",
"- 25 df, alpha 0.4 : crit val 0.256\n",
"\n",
"When we have higher df (larger sample size), the distribution of the t-statistic will have smaller tails and more closely approximate the normal. Thus, the critical value is lower for df=50 than df=9."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Exercise 3: Computing a one sample t-test by hand"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"
\n",
" Exercise 3 \n",
" The following cell defines a small dataset as a numpy array. Compute the t-value for this array under the null hypothesis that the true mean is 0.25. You will find the functions np.mean(), np.std(), and np.sqrt() useful. Print the t-value out in a cell by itself. Then use the stats.t.cdf() function to compute the p-value associated with that t using a one sided test. \n",
"
"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [],
"source": [
"# Your answer here\n",
"scores=np.array([.5,.56,.76,.8,.9]) # here is your data\n",
"# compute the \"effect\" (i.e., difference between the mean of the values and the null hypothesis)\n",
"# compute the error (i.e., the standard error of the mean), pay attention to whether you are dividing by n-1 or n\n",
"# compute the t-value\n",
"# use stats.t.cdf() to compute the area in the tail of the correct t-distribution for a one sided test."
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"(0.0018983152219906874, 6.03672166714376, 4)"
]
},
"execution_count": 6,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# Your answer here\n",
"\n",
"scores=np.array([.5,.56,.76,.8,.9]) # here is your data\n",
"\n",
"# compute the \"effect\" (i.e., difference between the mean of the values and the null hypothesis)\n",
"delta = np.mean(scores)-0.25\n",
"\n",
"# compute the error (i.e., the standard error of the mean)\n",
"n = len(scores)\n",
"se = np.std(scores, ddof=1) / np.sqrt(n)\n",
"\n",
"# Pay attention to the degrees of freedom!!\n",
"# compute the t-value\n",
"# use stats.t.cdf() to compute the area in the tail of the correct t-distribution for a one sided test.\n",
"\n",
"t_stat = delta / se\n",
"pval = 1-stats.t.cdf(t_stat,df=n-1)\n",
"\n",
"(pval, t_stat, n-1)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"
\n",
" Stop and think \n",
" If you have trouble, try Googling these functions to find information about the arguments!\n",
"
"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Exercise 4: Using `pingouin` to do a on sample `ttest()`"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"
\n",
" Exercise 4 \n",
" The following cell shows how to do a one sample t-test from a numpy array. Repeat this for the null hypothesis of 0.25, 0.5, and 0.75. All tests should be one-sided. Write one sentence below each t-test to describe how you would report the test in a paper.\n",
"
"
],
"text/plain": [
" T dof alternative p-val CI95% cohen-d BF10 \\\n",
"T-test 6.036722 4 greater 0.001898 [0.54, inf] 2.699704 28.44 \n",
"\n",
" power \n",
"T-test 0.999169 "
]
},
"execution_count": 7,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"scores=np.array([.5,.56,.76,.8,.9])\n",
"print(np.mean(scores))\n",
"pg.ttest(x=scores, y=0.25, alternative='greater')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"With a mean of 0.704, we find that the scores are significantly greater than 0.25 ( t(4)=6.04, p < 0.01)"
]
},
{
"cell_type": "code",
"execution_count": 34,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"
\n",
"\n",
"
\n",
" \n",
"
\n",
"
\n",
"
T
\n",
"
dof
\n",
"
alternative
\n",
"
p-val
\n",
"
CI95%
\n",
"
cohen-d
\n",
"
BF10
\n",
"
power
\n",
"
\n",
" \n",
" \n",
"
\n",
"
T-test
\n",
"
2.712536
\n",
"
4
\n",
"
greater
\n",
"
0.026699
\n",
"
[0.54, inf]
\n",
"
1.213083
\n",
"
4.235
\n",
"
0.719802
\n",
"
\n",
" \n",
"
\n",
"
"
],
"text/plain": [
" T dof alternative p-val CI95% cohen-d BF10 \\\n",
"T-test 2.712536 4 greater 0.026699 [0.54, inf] 1.213083 4.235 \n",
"\n",
" power \n",
"T-test 0.719802 "
]
},
"execution_count": 34,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"import pingouin as pg\n",
"scores=np.array([.5,.56,.76,.8,.9])\n",
"pg.ttest(x=scores, y=0.5, alternative='greater')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"With a mean of 0.704, we find that the scores are significantly greater than 0.5 ( t(4)=2.71, p < 0.05)"
]
},
{
"cell_type": "code",
"execution_count": 36,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"
\n",
"\n",
"
\n",
" \n",
"
\n",
"
\n",
"
T
\n",
"
dof
\n",
"
alternative
\n",
"
p-val
\n",
"
CI95%
\n",
"
cohen-d
\n",
"
BF10
\n",
"
power
\n",
"
\n",
" \n",
" \n",
"
\n",
"
T-test
\n",
"
-0.61165
\n",
"
4
\n",
"
greater
\n",
"
0.713087
\n",
"
[0.54, inf]
\n",
"
0.273538
\n",
"
0.923
\n",
"
0.015119
\n",
"
\n",
" \n",
"
\n",
"
"
],
"text/plain": [
" T dof alternative p-val CI95% cohen-d BF10 \\\n",
"T-test -0.61165 4 greater 0.713087 [0.54, inf] 0.273538 0.923 \n",
"\n",
" power \n",
"T-test 0.015119 "
]
},
"execution_count": 36,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"import pingouin as pg\n",
"scores=np.array([.5,.56,.76,.8,.9])\n",
"pg.ttest(x=scores, y=0.75, alternative='greater')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"With a mean of 0.704, we find that the scores are not significantly greater than 0.75 ( t(4)=0.61, p > 0.05)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Exercise 5: Paired t-test example"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### STUDY DESCRIPTION\n",
"\n",
"Parents often sing to their children and, even as infants, children listen to and look at their parents while they are singing. Research by Mehr, Song, and Spelke (2016) sought to explore the psychological function that music has for parents and infants, by examining the hypothesis that particular melodies convey important social information to infants. Specifically, melodies convey information about social affiliation.\n",
"\n",
"The authors argue that melodies are shared within social groups. Whereas children growing up in one culture may be exposed to certain songs as infants (e.g., “Rock-a-bye Baby”), children growing up in other cultures (or even other groups within a culture) may be exposed to different songs. Thus, when a novel person (someone who the infant has never seen before) sings a familiar song, it may signal to the infant that this new person is a member of their social group.\n",
"\n",
"To test this hypothesis, the researchers recruited 32 infants and their parents to complete an experiment. During their first visit to the lab, the parents were taught a new lullaby (one that neither they nor their infants had heard before). The experimenters asked the parents to sing the new lullaby to their child every day for the next 1-2 weeks.\n",
"\n",
"Following this 1-2 week exposure period, the parents and their infant returned to the lab to complete the experimental portion of the study. Infants were first shown a screen with side-by-side videos of two unfamiliar people, each of whom were silently smiling and looking at the infant.The researchers recorded the looking behavior (or gaze) of the infants during this ‘baseline’ phase. Next, one by one, the two unfamiliar people on the screen sang either the lullaby that the parents learned or a different lullaby (that had the same lyrics and rhythm, but a different melody). Finally, the infants saw the same silent video used at baseline, and the researchers again recorded the looking behavior of the infants during this ‘test’ phase.For more details on the experiment’s methods, please refer to Mehr et al. (2016) Experiment 1."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The first thing to do is download the .csv formatted data file, using the link above, or just click [here](http://gureckislab.org/courses/fall19/labincp/data/MehrSongSpelke2016.csv). "
]
},
{
"cell_type": "code",
"execution_count": 41,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"
\n",
"\n",
"
\n",
" \n",
"
\n",
"
\n",
"
id
\n",
"
study_code
\n",
"
exp1
\n",
"
exp2
\n",
"
exp3
\n",
"
exp4
\n",
"
exp5
\n",
"
dob
\n",
"
dot1
\n",
"
dot2
\n",
"
...
\n",
"
dtword13
\n",
"
dtnoword13
\n",
"
totsing14
\n",
"
babylike14
\n",
"
singcomf14
\n",
"
totrecord14
\n",
"
othersong14
\n",
"
dtword14
\n",
"
dtnoword14
\n",
"
filter_$
\n",
"
\n",
" \n",
" \n",
"
\n",
"
0
\n",
"
101
\n",
"
\"LUL\"
\n",
"
1
\n",
"
0
\n",
"
0
\n",
"
\n",
"
\n",
"
09oct2012
\n",
"
29mar2013
\n",
"
05apr2013
\n",
"
...
\n",
"
0
\n",
"
0
\n",
"
\n",
"
\n",
"
\n",
"
\n",
"
\n",
"
0
\n",
"
0
\n",
"
1
\n",
"
\n",
"
\n",
"
1
\n",
"
102
\n",
"
\"LUL\"
\n",
"
1
\n",
"
0
\n",
"
0
\n",
"
\n",
"
\n",
"
16nov2012
\n",
"
10may2013
\n",
"
17may2013
\n",
"
...
\n",
"
0
\n",
"
0
\n",
"
\n",
"
\n",
"
\n",
"
\n",
"
\n",
"
0
\n",
"
0
\n",
"
1
\n",
"
\n",
"
\n",
"
2
\n",
"
103
\n",
"
\"LUL\"
\n",
"
1
\n",
"
0
\n",
"
0
\n",
"
\n",
"
\n",
"
26nov2012
\n",
"
11may2013
\n",
"
20may2013
\n",
"
...
\n",
"
0
\n",
"
0
\n",
"
\n",
"
\n",
"
\n",
"
\n",
"
\n",
"
0
\n",
"
0
\n",
"
1
\n",
"
\n",
"
\n",
"
3
\n",
"
104
\n",
"
\"LUL\"
\n",
"
1
\n",
"
0
\n",
"
0
\n",
"
\n",
"
\n",
"
19nov2012
\n",
"
11may2013
\n",
"
18may2013
\n",
"
...
\n",
"
0
\n",
"
0
\n",
"
\n",
"
\n",
"
\n",
"
\n",
"
\n",
"
0
\n",
"
0
\n",
"
1
\n",
"
\n",
"
\n",
"
4
\n",
"
105
\n",
"
\"LUL\"
\n",
"
1
\n",
"
0
\n",
"
0
\n",
"
\n",
"
\n",
"
29nov2012
\n",
"
15may2013
\n",
"
29may2013
\n",
"
...
\n",
"
0
\n",
"
0
\n",
"
4
\n",
"
3
\n",
"
4
\n",
"
0
\n",
"
0
\n",
"
0
\n",
"
0
\n",
"
1
\n",
"
\n",
" \n",
"
\n",
"
5 rows × 153 columns
\n",
"
"
],
"text/plain": [
" id study_code exp1 exp2 exp3 exp4 exp5 dob dot1 \\\n",
"0 101 \"LUL\" 1 0 0 09oct2012 29mar2013 \n",
"1 102 \"LUL\" 1 0 0 16nov2012 10may2013 \n",
"2 103 \"LUL\" 1 0 0 26nov2012 11may2013 \n",
"3 104 \"LUL\" 1 0 0 19nov2012 11may2013 \n",
"4 105 \"LUL\" 1 0 0 29nov2012 15may2013 \n",
"\n",
" dot2 ... dtword13 dtnoword13 totsing14 babylike14 singcomf14 \\\n",
"0 05apr2013 ... 0 0 \n",
"1 17may2013 ... 0 0 \n",
"2 20may2013 ... 0 0 \n",
"3 18may2013 ... 0 0 \n",
"4 29may2013 ... 0 0 4 3 4 \n",
"\n",
" totrecord14 othersong14 dtword14 dtnoword14 filter_$ \n",
"0 0 0 1 \n",
"1 0 0 1 \n",
"2 0 0 1 \n",
"3 0 0 1 \n",
"4 0 0 0 0 1 \n",
"\n",
"[5 rows x 153 columns]"
]
},
"execution_count": 41,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# get the baby data frame\n",
"baby_df = pd.read_csv('http://gureckislab.org/courses/fall19/labincp/data/MehrSongSpelke2016.csv')\n",
"# filter to only have the data from experiment 1\n",
"experiment_one_df = baby_df[baby_df['exp1']==1]\n",
"experiment_one_df.head()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Baseline phase: Conduct a one sample t-test\n",
"\n",
"You first want to show that infants' looking behavior did not differ from chance during the baseline trial. The baseline trial was 16 seconds long. During the baseline, infants watched a video of two unfamiliar people, one of the left and one on the right. There was no sound during the baseline. Both of the actors in the video smiled directly at the infant.\n",
"\n",
"The important question was to determine whether the infant looked more or less to either person. If they showed no preference, the infant should look at both people about 50% of the time. How could we determine whether the infant looked at both people about 50% of the time?\n",
"\n",
"The `experiment_one_df` data frame has a column called `Baseline_Proportion_Gaze_to_Singer`. All of these values show how the proportion of time that the infant looked to the person who would later sing the familiar song to them. If the average of these proportion is .5 across the infants, then we would have some evidence that the infants were not biased at the beginning of the experiment. However, if the infants on average had a bias toward the singer, then the average proportion of the looking time should be different than .5.\n",
"\n",
"Using a one-sample t-test, we can test the hypothesis that our sample mean for the `Baseline_Proportion_Gaze_to_Singer` was not different from .5.\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"
\n",
" Exercise 6 \n",
" The cell below shows how to get the looking time proportions from the baseline phase. Conduct a one-sample t-test using pinguoin to see if this data is different than a null hypothesis of 0.5 (no preference).
"
],
"text/plain": [
" T dof alternative p-val CI95% cohen-d BF10 \\\n",
"T-test 0.674375 31 two-sided 0.505071 [0.46, 0.58] 0.119214 0.233 \n",
"\n",
" power \n",
"T-test 0.100209 "
]
},
"execution_count": 46,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# Answer goes here\n",
"print('my mean = ', experiment_one_df['Baseline_Proportion_Gaze_to_Singer'].mean())\n",
"pg.ttest(x=experiment_one_df['Baseline_Proportion_Gaze_to_Singer'], y=0.5)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Remember how the experiment went. Infants watched silent video recordings of two women (Baseline). Then each person sung a song, one was familiar to the infant (their parents sung the song to them many times), and one was unfamiliar (singing phase). After the singing phase, the infants watched the silent video of the two singers again (test phase). The critical question was whether the infants would look more to the person who sung the familiar song compared to the person who sun the unfamiliar song, which is recorded as `Test_Proportion_Gaze_to_Singer`. If the infants did this, they should look more than 50% of the time to the singer who sang the familiar song. We have the data, we can do another one sample t-test to find out."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"
\n",
" The cell below shows how to get the looking time proportions from the test phase. First conduct a one sample t-test on this test data compared to a null hypothesis of 0.5. What do you find? Finally, conduct a paired t-test between the baseline and test phase using pinguoin to see if this data is different than a null hypothesis of 0.0 (no difference).\n",
"
\n",
"Alright. What did we find? You should take a stab at writing down what we found. You can use the same kind of language that I used from the first one sample-test. You should state the mean proportion, the t-value, the dfs, and the p-value. You should be able to answer the question, did the infants look longer at the singer who sang the familiar song? And, did they look longer than would be consist with chance at 50%.\n",
"
"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"With an average looking proportio of 59.3%, infants looked significant longer at the adult who sang the familiar song during the test phase ( t(31)=2.95, p < .01)"
]
}
],
"metadata": {
"celltoolbar": "Tags",
"kernel_info": {
"name": "python3"
},
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.9"
},
"nteract": {
"version": "0.15.0"
},
"toc": {
"base_numbering": 1,
"nav_menu": {},
"number_sections": true,
"sideBar": true,
"skip_h1_title": false,
"title_cell": "Table of Contents",
"title_sidebar": "Contents",
"toc_cell": false,
"toc_position": {},
"toc_section_display": true,
"toc_window_display": false
}
},
"nbformat": 4,
"nbformat_minor": 1
}