top of page

Quantitative Research Design

When you are collecting numerical data to answer your research question (not just for demographic purposes) then you are using a quantitative design. Often, this data is generated from surveys, tests, or from archival data of test scores, attendance data, or other information on your population of interest. This data will be examined using statistical analysis. This page will provide you with key information on collecting and cleaning data, and running the statistical analyses needed to answer your research questions. I encourage you to read each of the sections below in order to help you build a strong project that can produce meaningful data. 
​
If you have already read through all the information and are looking for something specific, you can click on a specific step below to take you directly to that section. 
 

​
Quantitative Research Questions
Quantitative research looks at relationships between variables, differences between groups, and changes or impacts that occur due to an intervention or other experience. Research questions for quantitative studies should reflect this. The terms "relationship", "difference", "change/impact", or "effect" will likely be in your research question.  
​
Here is a great video that highlights the differences between quantitative and qualitative research.
​
Statistics: The Bare Bones Background
Statistics really comes down to two things: probability and variability. Don't panic. You already know what these are. First, probability. How likely it is for something to happen.  You get data and look at how common something is. Lots of the time we actually use the average (or mean) for this.  It tells us what "score" was common.  
​
Second, variability.  This tells us what would happen if we got a second set of data.  Would we get the same average score? Of course not (well, if you are working with humans at least).  It would be weird to get the same scores. Humans are not robots. They, ready for it...VARY from each other. 

 

Okay, so how much would we expect one sample or group to differ from another?  A lot? A little?  We can answer these questions by collecting data for two groups.  We look at what the means for those two groups, and determine if the difference between the means exceeds the difference we would expect between two samples of data. If the two groups we are looking at differ more than what we would expect two groups to differ, well, that's a big deal.  We might even call it significant...
​
Keep re-reading this as you go through a stats course or analyzing your data. Yes, I know, I only talked about splitting data into two groups, but if you use your imagination you can apply this logic to many scenarios. What if it was three groups, or a relationship between variables?  What would you expect (based on standard variability/deviation) and what did you find with your actual data?  Compare them and you will be able to see whether the groups or variables you are comparing are significantly different from each other. 
​
For a quick explanation of what statistics are and what they are doing check out this video. 
​

​
For a more in-depth explanation of how statistics work, watch the following three videos in order:

​
Collecting and Cleaning Data
To collect data, you need to determine a few things. First, who are your participants, and importantly how many of them do you need? After this, you need to know how you will collect the data. This involves choosing instruments (e.g., surveys, tests, metrics) that will give you the quality data you need to answer your specific research question. Lastly, when you have the data, you need to 'clean' it to ensure it is numerically coded in an acceptable way to allow you to analyze it in a statistical software program. I provide details on each of these below. 
​
Sample Size
A good sample size is necessary for any research project. When doing quantitative research, the quality of the statistical results is influenced directly by your sample size. Therefore, it is important to know how many people you need in your sample. To determine this, you will run a power analysis.
​
To do a power analysis, you need to know the type of statistical test you will run. You can determine this by consulting your textbook and notes from your quantitative research course. You can also schedule a time to talk with me and we can determine the types of tests together. 

​
Instruments
It is tempting to make your own instrument or survey. Do not do this. Look in the literature to find surveys or tests that already exist and have validity and reliability information about them. If you make a new instrument it will not have any reliability or validity testing; doing that testing is a project in and of itself. The process for this includes generating questions, testing those questions with experts in the field to refine the language, then piloting the instrument with a small group of the target population to look for initial pitfalls. This is followed by a full-scale research study with a large sample to examine the reliability and validity. Those are measured with extensive statistical analysis. Refinements are made based on these statistical results. Only then do you have a final instrument that is considered to have some level of validity and reliability. It's a long process. Find an instrument that has already been developed this way. Here's how.
​
You can find instruments by looking in the methods sections of articles that have done research with the variable you are interested in examining. For example, if you are looking at student motivation, and you cite Eccles and Wigfield's research on this, go find the surveys they have developed (meaning they tested them for validity and reliability). You will need to find a copy of the survey, or a copy of the questions inside the article (sometimes they are in a table that shows average scores for each question). Then find the article that states statistics on the validity and reliability of the instrument. It takes some digging, but it is worth it as you will get data that is reliable. 
​
After collecting your data, you will need to check for reliability among the data you have.  This is straightforward process. Here is a video that will show you how to do this in JASP.
​
Download and Clean Your Data
Your data will need to be in spreadsheet form. Any questions you wish to run statistical analysis on will need to be in numeric form. Your survey platform (SurveyMonkey, GoogleForm, etc.) will not necessarily save everything in number form. For that reason, you will have to go into the spreadsheet and edit things to be numeric. This handout will walk you through that process. We can also schedule a meeting together where I can go through this process with you. 
​
Statistical Software
To do statistical analysis, you need appropriate software. I suggest you use JASP or JAMOVI. They are both similar and free.

I have several statistics tutorial videos that are all done with JASP, but you should use whatever is comfortable for you. You'll easily be able to translate between JASP and JAMOVI.

 

Learn about how to use JASP here. 

 

​
​Video Tutorials of Different Statistical Analyses
Now that you have data from a sufficient sample, using valid and reliable measures, downloaded and cleaned your data and reviewed how to use JASP, you are ready to analyze your data. You will need to follow the plan of statistical tests that you created before running your power analysis. If you would like to review that plan (they have a way of going in one ear and out the other) or work on developing a plan you can schedule a time to talk with me. 
​
Here is a collection of videos that you can reference to help you run different statistical tests. 


​Writing Up Your Results
Quantitative results can be written up many ways. The following gives a standard approach to this. 


Open with a brief summary of the study problem, purpose and research questions. Then give a brief summary of how the study was conducted. After this, you will present the findings. It will need to include (and I suggest in this order):

  • Results of your reliability analysis (see video above).

    • You do not need any tables in your document for this. Just a simple statement that you ran reliability, what questions were included, and what the statistic was ("alpha=.56". Find the "alpha" symbol in "insert symbols" in Word). You can conclude whether this was acceptable or reliable. If you are unsure of whether it is acceptable, send me an email so that we can discuss it. 

  • Descriptive statistics for demographic variables. This will include tables and/or graphs and a written summary.

  • Descriptive statistics for the key variables in your research questions.

    • The tables and graphs for this will be generated when you run the statistical test in JASP. For example, if you are examining motivation among students, and you have a survey with three subscales, make a table that has the sample size (N), mean, standard deviation, and 95% confidence interval for the total score and each subscale. This will results in a table that that has: 

      • five column: N, M, SD, Lower 95% CI, Upper 95% CI

      • and four rows of data: total scores, subscale 1, subscale 2, subscale 3, subscale 4​
         

You may want to include some graphs​ as well. For example, if you are comparing motivation scores by students' year in school, you could make an error bar chart of the subscale scores with four bars: one for each year in school (first year, sophomore, junior, senior).

 

Note that there is a difference between a table and figure. A table is a grid with numbers. A figure is everything else (bar chart, pie chart, image, etc.) Also, APA requires different title formatting for tables and figures. Please review the differences here. 


Now it's time to construct the written narrative of the descriptive statistics for each key variable​. A good technique for this is to start writing a short paragraph under each table or figure. Look at the table/figure and start explaining what you see. Remember, just the facts (YES: "This table shows that first-year students have the highest motivation M=4.67(.23) and it decreases each year ending at M=2.45(.43) in the senior year"). You do not have to get into detailed discussion about what this means (NO: "This means we need to help students not lose their motivation as they progress through college"). If those thoughts come to you, make a note of them for the discussion section. 

Notice how I wrote out the mean and standard deviation above, M=2.45(.43). The mean ALWAYS goes after "M=" and the standard deviation goes in parentheses "(.43)" immediately after it. There is no space between the mean and the parentheses. 
 

After you provide some narrative about the descriptive statistics you see in the tables and figures, write up narrative of each statistical test run (chi-squares, t-tests, ANOVAs, correlations, regressions). The key elements for each test write-up should include:

  • The type of test run ("I ran a one-way ANOVA...").

  • Why you ran it ("...to examine the differences in motivation between first year, sophomore, juniors, and seniors.")

  • What you found ("The tests was significant with a large effect size...")

  • The APA formatted test statistic (..."F(3, 245)=6.78, p<.01")  https://depts.washington.edu/psych/files/writing_center/stats.pdf

  • Summary. Remember, just the facts. Short and sweet here (one to two paragraphs). ​


​If you need help along this journey, please click "Get Help" above and reach out. 
​

Quantiative Research Questions
The Bare Bone Background
Collecting and Cleaning Data
Sample Size
Reliablity and Validity
Download and Clean Data
Statistical Software
Video Tutorials
Writing Up Your Results
bottom of page