Introduction
This coursework is designed to allow you to demonstrate your skills in carrying out manual statistics calculations, interpreting SPSS results and study design. The assignment comprises four questions, you need to answer all questions. The questions are presented in three sections:
Section 1 (Q1) provides an experimental study scenario, the data collected from 96 participants and SPSS analysis outputs (in Appendix B). You are required to use this information to report the study design, method of statistical analysis applied, results of the study and your interpretation of findings.
Section 2 (Qs 2 and 3) provides another experimental study scenario and the data collected from 28 participants. For each statistical question, you are required to identify the experimental design, justify your choice of test, report the results, and interpret the results. You should carry out appropriate statistical analyses by hand and must include the workings in the submitted coursework.
Section 3 (Q4) requires you to write an account of the method for the experimental study that has been outlined in Section 2, plus the method for a follow-up interview study.
To answer Question 1, you need to extract relevant information from the SPSS outputs and then write a written report of the study design and your interpretation of the analysis findings. Note: the SPSS outputs will contain more information than you need and some of the information is duplicated due the nature of SPSS outputs. You must therefore be selective. We have also removed some unnecessary information. You are only required to interpret the information provided.
When answering the statistical questions (Q2 and Q3), you should briefly report the outcome of the statistical test and its interpretation (no longer than a paragraph, no graphs or tables are required). Note: In this section, you are not required to test for parametric assumptions or conduct post-hoc tests. Information about reporting statistical tests in writing is given in the Statistics Workbook and should conform to the APA (American Psychological Association) format for statistical reporting.
Please note the maximum word limit for Q4 on study design
You must complete this coursework on your own and not in collaboration with other
Specification
Electronic submission of completed coursework via Moodle by the deadline: 13th January
2022, 3pm. Indicate your name and student number on the documents that you submit.
Statistical analyses should be completed by hand and full workings shown. These may be hand-written and scanned/photographed or typed out. Workings may be submitted as an appendix to a single document or you may instead choose to submit two documents, one containing answers, the other containing calculations.
Appendix A provides the marking criteria for each section. Appendix B provides SPSS outputs for Q1.
Section 1 – interpretation of SPSS outputs
Background to flight interface experiment
You have been asked to analyse the experimental data from an experiment comparing pilot performance in a simulation task using two different interfaces: joystick (see Figure 1) versus an innovative direct brain interface (see Figure 2). The overall aim of the experiment is to explore the possibility that brain interfaces might be a better way for pilots to control aircraft in future than using physical control inputs.
Figure 1 – Aircraft joystick Figure 2 – 3D Printed MEG scanner
The experiment was based around a piece of simulator software published by NASA (the MATB-II, or Multi-Attribute Task Battery version 2), which includes many different types of tasks. This study used the “TRACK” module (compensatory tracking) alone, set to “MANUAL” throughout
[https://matb.larc.nasa.gov/]
Figure 3 – Compensatory tracking task, known as “TRACK” in MATB-II
Three levels of task difficulty were programmed in to simulate three levels of turbulence – low turbulence, medium turbulence, and high turbulence.
96 participants from a flight simulation training programme were recruited to the study. The participants, who were matched for training experience and flight skill levels, were randomly
assigned to perform the experimental task in one of six conditions. Each participant performed the task in only one experimental condition comprising of one level of turbulence and
one control input method. The data were then analysed in SPSS. The data collected are given in Table 1. Selected SPSS outputs from the analyses conducted are given in Appendix B.
Table 1. Compensatory task performance (RMSE score) in each simulation trial condition
Q1. SPSS interpretation [30 marks]
Using the information provided, describe the study design and state the statistical test conducted. Comment on how you know whether or not the data meet parametric assumptions. State the hypotheses tested and describe the experimental results for each test performed. Using relevant information from the SPSS outputs, explain how the data can be used to interpret the effect of flight control interface and turbulence level on performance in the simulation control task. You do not need to include graphs and tables in your answer but you may refer to graphs included in the SPSS output. For ease of referencing, a Figure number has been added in blue text above each graphical output in Appendix B. You should refer to relevant test results and descriptive statistics to support your interpretation of the study findings and state your conclusions in answer to the research question investigated.
Section 2 – statistical calculations
Background to the spacesuit study
Having established the best interface for spaceflight, the company is interested in developing its own spacesuits for use by the first tourists on the moon. They have collected a set of pre-existing spacesuits with a view to evaluating their strengths and weaknesses. There is a “Hard-shell” spacesuit (largely metal and composite construction), a “Soft-shell” spacesuit (largely fabric construction) and a prototype “Hybrid” suit that combines elements of the other two (part metal, part fabric), as shown in Figure 4. You are asked to carry out statistical tests based on the data collected (See Table 2) to
address the research questions below.
Figure 4. From L-R, Hard-shell, Soft-shell and Hybrid designs (all pictures NASA, public domain).
Data collected from the spacesuit experiment
9364 120 865 88 5 7 100 946 102 735 107 885 77 977 118 6 6 95 823 84 965 102
Comfort rating
1 to 9 scale, 1 = lowest comfort rating, 9 = highest comfort
Rock collection task time
Participant 1
Soft Shell
Hard Shell
Soft Shell
Hard Shell
Participant 2
Participant 3
Participant 4
Participant 5
Participant 6
Participant 7
Participant 8
Participant 9
Participant 10
Participan 11
6 6 6 11 98918
t Participan 12
t Participan 13
t Participan 14
t Participan 15
t Participan 16
t Participan 17
t Participan 18
t Participan 19
t Participan 20
t Participan 21
t Participan 22
t Participan 23
t Participan 24
t Participan 25
t Participan 26
t Participan 27
t Participan 28
7 2 6 745 839 731 621 616 865 9490
87 4 95 49
6 6 5 11 8 6 5 11
Q2. Twenty-eight participants with some experience of protective suits (divers, firefighters, pilots etc) were recruited to the study. Initially all participants tried on all three suits and gave a comfort rating on a 1 to 9 scale for each where 1 = lowest comfort level, 9 = highest comfort level. Evaluate whether the suits differ in comfort levels and advise as to the most suitable options for user comfort. Post-hoc tests are not required in this coursework. [20 marks]
Q3. In a follow-on experiment the researchers were interested in evaluating task performance on a rock sample collecting task in a swimming pool which was used to simulate reduced gravity. Unfortunately, as a one-off prototype the Hybrid suit was not rated for immersion in chlorinated water so the study was restricted to the Hard-shell and Soft-shell suits. Two groups of fourteen participants undertook the task wearing either the Hard-shell or the Soft-shell suit and their time to collect a set number of rocks and place them in bags was recorded. Determine whether there was a significant difference in task performance time between the two suits. [20 marks]
Section 3 – study design
Q4. Section 2 relates to an experimental study of different types of space suit. Following from this study, you are asked to plan and carry out a small interview study to explore in more detail how the different types of suit might impact on task performance. You are intending to publish the results from both the experimental study and the small follow up interview study in a single publication.
Write a single account of the method that covers the two studies. For the experimental study you should use the information given in section 2 in relation to the space suit study, together with your knowledge of experimental design and planning of studies as the basis for your account of the
method. For the small interview study, you should think carefully about how you would carry out such a study, in order to present a written account of your method for this study.
In your account of the method you should consider using sub-sections that are typical within a method section of a report or publication (e.g. general approach, materials, participants, experimental design, procedure and analysis). You will need to think about the best way to present the two different parts of the study in a single account of the method.
Your account of this method should be no more than 1300 words and you must write your word count below the answer for this question – [30 marks]
APPENDIX A – Marking Criteria SECTION 1: Q1 30 marks will be awarded for the following:
Complete and accurate statement of hypotheses, IVs and DVs. Identification of the analysis test that has been conducted [7 marks]
Interpretation of information relating to tests of parametric assumptions [3 marks] Complete and accurate statement of analysis results for each hypothesis tested [6
Interpretation of results with reference to descriptive statistics and, where appropriate, post- hoc analyses outcomes [12 marks]
Statement of study conclusions [2 marks]
SECTION 2: Q2 & Q3 [20 marks each] will be awarded on the following criteria
• Complete and accurate statement of hypotheses, IVs and DVs [5 marks]
• Correct test selected [2 marks]
• Correct test calculation methods applied [3 marks]
• Accurate interpretation of results with reference to descriptive statistics [5 marks]
• Adequate evidence of workings [5 marks]
SECTION 3: Q4 30 Marks will be awarded for the following:
• Structure and ease of reading of the account of the method [10 marks],
• The accuracy of the presentation of relevant details from the experimental study [5 marks]
• A plausible account of a small interview study, including relevant details on sampling and commentary on the types of questions that will be used in the study [5 marks]
• Explanation of the collection, collation and analysis of data for the experimental and interview studies [5 marks]
• Details of how principles of good study design are achieved in the conduct of the experimental and interview parts of the study [5 marks]
Univariate Analysis of Variance
Between-Subjects Factors
APPENDIX B – SPSS outputs for Question 1
Value Label
Control_input 1
Turbulence
48 48 32 32 32
Dependent Variable:
Performance
Tests of Between-Subjects Effects
Type III Sum of Squares
Mean Square
Corrected Model
Intercept 196656.510 Control_input 943.760 Turbulence 14945.646
.000 .000 .001 .000 .000
Control_input * Turbulence Error
Corrected Total
6515.437 221907.000 25250.490
a. R Squared = .742 (Adjusted R Squared = .728)
Profile Plots
18735.052a
196656.510
EXAMINE VARIABLES=Performance BY Condition
/PLOT BOXPLOT NPPLOT /COMPARE VARIABLES /STATISTICS DESCRIPTIVES /CINTERVAL 95
/MISSING LISTWISE /NOTOTAL.
Descriptives
Condition Statistic
Std. Error
Performance
Low turbulence + joystick Mean
95% Confidence Interval for Mean
5% Trimmed Mean
Std. Deviation Minimum Maximum
Interquartile Range
Low turbulence + BCI Mean
95% Confidence Interval for
5% Trimmed Mean Median
Std. Deviation Minimum Maximum
Interquartile Range
Medium turbulence + joystick Mean
95% Confidence Interval for Mean
5% Trimmed Mean
Std. Deviation Minimum Maximum
Interquartile Range
Lower Bound Upper Bound
Lower Bound Upper Bound
Lower Bound Upper Bound
1.653 24.04
31.09 27.51 26.50 43.729 6.613
.240 .564 -.475 1.091
1.879 29.87
37.88 33.86 33.50 56.517 7.518
.086 .564 -.385 1.091
2.474 41.04
51.59 46.63 46.50 97.963 9.898
Medium turbulence + BCI
High turbulence + joystick
95% Confidence Interval for Mean
Lower Bound
Upper Bound 45.18
5% Trimmed Mean
Std. Deviation
Interquartile Range
5% Trimmed Mean
Upper Bound 76.36
95% Confidence Interval for Mean
Lower Bound
Std. Deviation
5% Trimmed Mean
Upper Bound 56.31
Interquartile Range
95% Confidence Interval for Mean
Lower Bound
Std. Deviation
Interquartile Range
1.091 1.756
.564 1.091 2.369
High turbulence + BCI
.564 1.091 2.462
.564 1.091
Statistic df
.099 .134 .156
.150 .137 .185
Shapiro- . Statistic df
Tests of Normality
Kolmogorov-Smirnova
Performance
Low turbulence + joystick
Low turbulence + BCI Medium turbulence + joystick
Medium turbulence + BCI High turbulence + joystick High turbulence + BCI
16 .200* 16 .200* 16 .200*
16 .200* 16 .200* 16 .148
.981 .987 .944
.930 .937 .941
16 .971 16 .996 16 .400
16 .242 16 .316 16 .364
*. This is a lower bound of the true significance. a. Lilliefors Significance Correction
UNIANOVA Performance BY Control_input Turbulence
/METHOD=SSTYPE(3)
/INTERCEPT=INCLUDE
/POSTHOC=Turbulence(BONFERRONI)
/PLOT=PROFILE(Control_input Turbulence Turbulence*Control_input) TYPE=LINE
ERRORBAR=NO
MEANREFERENCE=NO YAXIS=AUTO
/EMMEANS=TABLES(Control_input) COMPARE ADJ(BONFERRONI) /EMMEANS=TABLES(Turbulence) COMPARE ADJ(BONFERRONI)
/EMMEANS=TABLES(Control_input*Turbulence) COMPARE(Control_input) ADJ(BONFERRONI) /EMMEANS=TABLES(Control_input*Turbulence) COMPARE(Turbulence) ADJ(BONFERRONI)
/PRINT DESCRIPTIVE HOMOGENEITY
/CRITERIA=ALPHA(.05)
/DESIGN=Control_input Turbulence Control_input*Turbulence.
Univariate Analysis of Variance
Between-Subjects Factors
Value Label
Control_input 1
48 48 32 32 32
Turbulence
Dependent Variable:
Descriptive Statistics
Performance
Control_input
Turbulence Mean Std. Deviation
Joystick Low Medium
High Total
BCI Low Medium
High Total
Medium 43.88
High 61.19
Total 45.26 16.303
Performance Based on Mean
Levene’s Test of Equality of Error Variancesa,b
Levene Statistic df1 df2
.114 .215 .216
Based on Median
Based on Median and with adjusted df
.115 Tests the null hypothesis that the error variance of the dependent variable is equal across groups.
a. Dependent variable: Performance
b. Design: Intercept + Control_input + Turbulence + Control_input * Turbulence
Based on trimmed mean
Dependent Variable: Performance
Type III Sum of Squares
Mean Square F
Tests of Between-Subjects Effects
Corrected Model
18735.052a
196656.510
Intercept 196656.510 Control_input 943.760 Turbulence 14945.646
.000 .000 .001 .000 .000
Control_input * Turbulence Error
Corrected Total
6515.437 221907.000 25250.490
a. R Squared = .742 (Adjusted R Squared = .728)
Estimated Marginal Means 1. Control_input
Dependent Variable:
Control_input
Dependent Variable:
(I) Control_input
Performance
48.396 42.125
Performance
(J) Control_input
Std. Error
Lower Bound
Upper Bound
50.836 44.565
95% Confidence Interval
1.228 45.956 1.228 39.685
Pairwise Comparisons
Mean Difference (I-J)
95% Confidence Interval for Differenceb
Joystick BCI
BCI Joystick
6.271* -6.271*
Std. Error
1.737 1.737
Lower Bound
2.820 -9.721
Upper Bound
9.721 -2.820
Based on estimated marginal means
*. The mean difference is significant at the .05 level. b. Adjustment for multiple comparisons: Bonferroni.
Univariate Tests
Sum of Squares df Mean Square F Sig.
The F tests the effect of Control_input. This test is based on the linearly independent pairwise comparisons among the estimated marginal means.
Dependent Variable: Performance
2. Turbulence
Dependent Variable: Performance
Turbulence
Mean Std. Error
Lower Bound
Upper Bound
95% Confidence Interval
Low Medium High
30.719 43.875 61.187
Performance
1.504 27.731 1.504 40.887 1.504 58.199
Pairwise Comparisons
33.707 46.863 64.176
Dependent Variable:
(I) Turbulence
(J) Turbulence
Lower Bound
-18.345 -35.658 7.967 -22.502
25.280 12.123
Upper Bound
-7.967 -25.280 18.345 -12.123
35.658 22.502
Mean Difference
(I-J) Std. Error
95% Confidence Interval for Differenceb
Low Medium High
High Low Medium
-13.156* -30.469* 13.156*
-17.312* 30.469* 17.312*
2.127 2.127 2.127 2.127 2.127 2.127
.000 .000 .000 .000 .000 .000
Based on estimated marginal means
*. The mean difference is significant at the .05 level. b. Adjustment for multiple comparisons: Bonferroni.
Univariate Tests
Dependent Variable: Performance
Sum of Squares df Mean Square F
The F tests the effect of Turbulence. This test is based on the linearly independent pairwise comparisons among the estimated marginal means.
3. Control_input * Turbulence
Dependent Variable: Performance
Mean Std. Error Lower Bound
27.562 2.127 23.337 46.313 2.127 42.087 71.312 2.127 67.087 33.875 2.127 29.649 41.437 2.127 37.212 51.062 2.127 46.837
Pairwise Comparisons
Control_input
Turbulence
Upper Bound
95% Confidence Interval
Joystick Low Medium
Medium High
Dependent Variable:
Performance
31.788 50.538 75.538 38.101 45.663 55.288
95% Confidence Interval for Differenceb
Turbulence
(J) Control_input
Difference (I-J) Std. Error
-6.312* 3.008 6.313* 3.008 4.875 3.008 -4.875 3.008 20.250* 3.008
-20.250* 3.008
.039 .039 .109 .109 .000 .000
Upper Bound
-.336 12.289 10.851
26.226 -14.274
(I) Control_input
Lower Bound