Tamarah Smith, PhD

RMSA©- Research Methods Skill Assessment

Tamarah Smith, PhD

Request a copy at:  RMSA.V2014@gmail.com

 

        The Research Methods Skill Assessment (RMSA) was created to assess psychology majors’ performance with goal 2.0 of the American Psychological Association’s (APA) Guidelines for the Undergraduate Major. Goal 2.0 states that students should be able to “Understand and apply basic research methods in psychology…” (APA, 2007, p.13).  During creation of the RMSA, the second version of the APA Guidelines was published (APA, 2013) and the RMSA is consistent with the changes reflected in this later edition.

​

        The motivation for creating the instrument came from the research highlighting poor application of research methods in the psychological literature (see Cumming, 2014; Bakker & Wicherts, 2011; Farley, et al., 2009) and my intention to begin evaluating the role education has on this problem.  Research on coverage of APA quantitative objectives has shown that limited time in spent in the statistics classroom on these topics (Friedrich, Buday, & Kerr, 2002).  Further, evaluation of doctoral programs has shown that most students at this level are spending limited time in quantitative courses (Aiken, West, & Milsap, 2008). The RMSA was developed to help begin an assessment of quantitative skills, important to psychology (e.g., outlined in the Guidelines), and their potential connection to the problematic practices in the field today (e.g., Farely et al., 2009). 

​

        The RMSA has been validated with undergraduate students who completed either a paper-pencil or online versions utilizing both “open-book” and “closed-book” formats.  As such, the RMSA can be utilized in a number of ways such as embedded into a course exam, as a stand-alone exit exam, as well as be repeated over a period of time to assess changes in students’ skills.  Two versions, 21 questions to measure key statistical skills and 33 questions to measure both statistical and research design skills, allow additional flexibility in measuring key areas of interest.  For example, those interested in statistical skills may administer only the first 21 questions; those interested in research design may administer only the 12 related questions; or, all 33 questions can be administered.  This manual serves as a guide to understanding the RMSA, how to administer, and the reliability and validity of the instrument. 

 

​