Archive | NORMES Research

Current Evaluation Projects

College Ready in Mathematics and Physics Partnership

The College Ready (CR) in Mathematics and Physics Partnership is funded by the National Science Foundation (NSF). The project is administered through the University of Arkansas, Fayetteville. The partnership includes 38 school districts in Arkansas and Oklahoma. The overarching goal of CR is to impact the successful transition of students from high school to college, and facilitate students’ progress in attaining baccalaureate degrees. Ultimately CR will demonstrate an impact on student outcomes through decreases in college remediation rates in mathematics, increases in student proficiency scores on state assessments, increased numbers of students enrolling in AP mathematics and physics courses, increased number of students scoring 3 or better on AP exams, and increased graduation and college-bound rates.

Arkansas Mathematics Science Partnership (MSP)

In January 2002, the No Child Left Behind Act of 2011 (NCLB) became law. Title II, Part B of this legislation authorized the MSP competitive grant program. The purpose of the MSP is to increase the academic achievement of students in mathematics and science by enhancing the subject matter knowledge and teaching skills of classroom teachers. Partnerships between high-need school districts and the science, technology, engineering, and mathematics (STEM) faculty in institutions of higher education (IHE) are at the core of these improvement efforts. The U.S. Department of Education awards funds to the Arkansas Department of Education which are passed out to successful proposals submitted by Arkansas institutions of higher education, school districts, or nonprofit organizations that have formed partnerships which focus on the improvement of mathematics and/or science instruction in grades K-12.

Recent Prior Evaluation Projects

Springdale, Arkansas Public Schools

The Extended Year Learning Program, Westwood Elementary
The Extended Year Learning Program was an effort of Westwood Elementary School to remove academic barriers and to help children develop their creative, intellectual and interpersonal skills. The program included year-round after-school activities. Extending the school day is believed to help prevent crime, juvenile delinquency, and violent victimization, as well as improve academic achievement, reduce grade retention, and increase school attendance.

International Baccalaureate Primary Years Program

The International Baccalaureate Primary Years Program (PYP) was designed for students aged 3 to 12. It focuses on the development of the whole child as an inquirer, both in the classroom and in the world outside. It is a framework guided by six interdisciplinary themes of global significance, explored using knowledge and skills derived from six subjects, as well as interdisciplinary skills, with a powerful emphasis on inquiry. The PYP addresses students’ academic, social and emotional well-being; encourages students to develop independence and to take responsibility for their own learning; supports students’ efforts to gain understanding of the world and to function comfortably within it; and helps students establish personal values as a foundation upon which international-mindedness will develop and flourish.

The Middle Level Scholars Program

The Middle Level Scholars Program (MLSP) was designed to address the academic, social and behavioral needs of students who are at risk of dropping out of school before graduation. MLSP has three components; extended school day, extended school year, and extended curriculum. It is believed that students who stay connected have a better chance.

Spanish Enrichment Program

Since the 1960′s, studies have shown that the best time to begin the study of a foreign language is in elementary school. Because children at this age show better mental flexibility, more creativity, divergent thinking skills, and improved listening and memory skills, kids are more apt to acquire language early on. Additionally, early exposure affords children more years to be immersed in the language. The Springdale School District instituted programs for Spanish language development at five of its 16 elementary schools.

The New Arrival Center

The New Arrival Center (NAC) was an intensive, half-day language class that teaches English Language Learners (ELL) students to understand, speak, and write in English. The NAC also provides students the necessary academic and social skills necessary to complete the integration program in a six to twelve

Posted in Evaluation Projects

Current Research

The NORMES offices are continuously involved in research endeavors leading to improved academic and health initiatives throughout the nation. While each division of NORMES focuses on specialized initiatives, all divisions collaborate to draw on the collective expertise and strengths of the NORMES faculty, staff and graduate students.

Dunn, K. E., Rakes, G. C., & Behrand, B.
Can you help me? Selfregulation, age, and help-seeking in online graduate students. The Internet and Higher Education.

Dunn, K. E., & Rakes, G. C. (Fall 2011).
Teaching teachers: An investigation of beliefs in teacher education students. Learning Environments Research.

Dunn, K. E. (in press, 2010).
Primary cognitive aging. In M. A. Toner, B. B. Shadden, and M. Gluth (Eds.), Communication and aging: For clinicians by clinicians (2nd ed.). Austin, TX: Pro-Ed.

Rakes, G.C., Fields, V.S., & Cox, K. E. (2010)
The influence of teachers’ technology use on instructional practices. In L. Shrum (Ed.), Considerations on Technology and Teachers: The Best of JRTE. Eugene, Oregon:
International Society for Technology in Education.

Dunn, K. E., & Rakes, G. C. (2010).
Learner-centeredness and teacher efficacy: Predicting teachers’ consequence concerns regarding the use of technology in the classroom. Journal of Technology and Teacher Education, 18(1), 57-83.

Rakes, G. C., & Dunn, K. E. (2010).
The impact of online graduate students’ motivation and self-regulation on academic procrastination. Journal of Interactive Online Learning, 9(1), 78-93.

Dunn, K. E., & Rakes, G. C. (2010).
Producing caring qualified teachers: An exploration of the influence of pre-service teacher concerns on learner centeredness. Teaching and Teacher Education, 26, 516-521.

Crawford, A. V., Green, S. B., Levy, R., Lo, W. J., Scott, L, Svetina, D, & Thompson, M. S. (2010).
Evaluation of Parallel Analysis Methods for Determining the Number of Factors. Educational and Psychological Measurement.

Newgent, R. A., Behrend, B. A., Lounsbery, K. L., Higgins, K. K., & Lo. W. J. (2010).
Psychosocial Educational Groups for Students (PEGS): An Evaluation of the Treatment Effectiveness of a School-Based Behavioral Intervention Program. Counseling Outcome Research and Evaluation, 1, 80-94.

Dunn, K. & Mulvenon, S. (2009).
A critical review of research on formative assessments: The limited scientific evidence of the impact of formative assessments in education. Practical Assessment, Research and Evaluation, 14(7). Retrieved March 23, 2009 from .

AERA 2009

The 2009 annual conference will be held in San Diego on April 13-17.

AERA 2008

The 2008 annual conference was held in New York on March 24-28. The following papers were presented by the NORMES faculty, research associates, and graduate assistants:

Denise T. Airola, Sean W. Mulvenon, Charles E. Stegman, and Ronna L. Turner

The College Readiness Index: Translating a Complex Measure into a Practical Measure

Rebecca A. Newgent, Larry W. Featherston, and Charles E. Stegman

A School-Based Mental Health Program: A Collaboration Among Schools, Mental Health Agency, and University

Rebecca A. Newgent

A Lunch Buddy Mentoring Program for Elementary School Students Who Are Bullied

Rebecca A. Newgent

A New Model for Identifying Potentially At-Risk Children Who Need Mentoring

Kening Wang, Sean W. Mulvenon, Yanling Xia, and Charles E. Stegman

Modified Equation for Computing Performance Growth Index (PGI) When Expected Gain (EG) is Negative

AERA 2007

The 2007 annual conference was held in Chicago on April 9-13. The following papers were presented by the NORMES faculty, research associates, and graduate assistants:

Denise T. Airola

A Statewide Analysis of the Impact of Reading First on Economically Disadvantaged Students Using an Early Learning Indicator

Crystal Beshears, Calli A. Holaway-Johnson, Charles E. Stegman, and Sean W. Mulvenon

Arkansas Best Practices: Implications for School Improvement

Crystal Beshears, Charles E. Stegman, and Sean W. Mulvenon

Technology and Social Studies: Teacher Integration of Technology and Student Achievement

Sarah C. McKenzie

Pre-K in ECLS K-5: The Academic Achievement of Students from Various Prekindergarten Care Environments

Karen K. Morton, Sean W. Mulvenon, and Sarah Dillard

Highly Qualified Teachers and Teacher Examination Passing Scores

Sean W. Mulvenon, Michael Miller, Dan Ferritor, and Jon Johnson

Grade Inflation in Higher Education: Does an “A” Grade Still Mean Excellent?

Sean W. Mulvenon and Ross Santy

Improving the Quality of Use of Educational Achievement Data

Sean W. Mulvenon

Growth Models in the Age of Educational Reform: Understanding the Policy and Math

Ronna L. Turner and Amy H. Robertson

Impact of Inquiry Training and Scientist-K-12 Teacher Partnerships on Classroom Practices and Teacher Attitudes

AERA 2006

The 2006 annual conference was held in San Francisco on April 7-10. The following papers were presented by the NORMES faculty, research associates, and graduate assistants:

Denise T. Airola and Charles Stegman

From Placement to Proficiency: Finding a Balance in Appropriate Assessment and Accountability for English Language Learners

Sarah C. McKenzie, Nick T. Ogle, Charles E. Stegman & Sean W. Mulvenon

Does School Configuration Impact School Performance on AYP Assessments?

Sarah C. McKenzie, Sean W. Mulvenon & Charles E. Stegman

On Track – An Examination of the Patterns and Policy Implications: Associated with Matching Student Data

Posted in Current Publications, NORMES Research


Office for Clinical and Applied Research

The purpose of the Office for Clinical and Applied Research (OCAR) is to conduct targeted educational research in the clinical and applied arena. OCAR faculty, research associates, and graduate students have developed partnerships with local educational systems and mental health agencies as well as faculty and graduate students in the Counselor Education Program and other service-related departments at the University of Arkansas. The primary mission of OCAR is to conduct research on and provide services to children at-risk (academically, physically, and emotionally) through delivery-based grants. The outcomes of our research have made a practical impact on the delivery of services for children in Northwest Arkansas and beyond.

The educational and clinical/applied arenas are experiencing a paradigm shift toward more outcome-based empirically supported research. Further, grant funding agencies, departments of education, and other similar entities are now requiring outcome-based empirical research for the programs they support. As such, the efforts at OCAR have led to increased understanding of the psychometric properties of assessment tools and research models based in empirical findings that can be applied to the delivery of educational and clinical/applied services.

Current Programming

The Psychosocial Education and Prevention Series (PEPS) Program was implemented in the Fayetteville School District in 2008. The PEPS Program provides both universal and selective intervention and prevention programs. OCAR provides school-wide intervention programs for elementary school teachers on addressing the issues of bullying. The model is based on Bully Busters (Horne, Bartolomucci, & Newman-Carlson, 2003). OCAR also provides two selective intervention programs for elementary school students. First, the Psychosocial Educational Groups for Students (PEGS) Program is a 6-week series of traditional prevention psychoeducation topics that help elementary school students improve relationships with their peers and feel better about themselves. The majority of students who participate in the PEGS program have low self-esteem and exhibit problem behaviors. Second, the At-Risk Kids Program for Bullied Children (ARK) Program is a 6-week series of targeted prevention psychoeducation topics that help elementary school students who exhibit behaviors related to being a victim of peer harassment (bullied) as well as students exhibiting bullying behaviors. The PEPS Series Program or any of its individual components is available to additional school districts.

The PEPS Program is also in the process of examining the following issues: 1) the relationship between types of bullying and victimization and internalization and externalization, 2) indicators of teacher efficacy in relation to peer victimization, and 3) undiagnosed Autism vs. problem behavior.

Prior Programming

OCAR worked with the Counselor Education Program and the Center for Research on Aggression and Violence (CRAV) in the Department of Psychology, for approximately five years providing assessment and lunch buddy mentoring for elementary school children who are potential victims of peer harassment (bullying). The Peer Safety Project (PSP) was implemented in 2004 at four elementary schools and expanded to two additional elementary schools in the Springdale School District in 2007.

OCAR worked with Ozark Guidance, an area comprehensive mental health agency, for approximately seven years providing independent evaluation of social service and mental health intervention programs in 11 different elementary schools in Northwest Arkansas. Our partnership began in 2002 and consisted of three projects that are now in the self-sustainability phase.

Future Programming

Future programs that OCAR is developing will address the needs of children who are exposed to violence and reducing risk behavior by promoting positive youth development. Intervention and evaluation systems are currently being identified for this program.

Contact Information

Independent evaluation services provided by OCAR include:
• Consultation on the data collection design
• Suggestions for assessment measurement tools
• Data entry excel sheets for project personnel to collect and enter data
• Statistical design and SAS code upon receipt of research questions from contracting agency
• Annual report to assess the effectiveness of the program (statistical analysis)
• Recommendations for modifications needed in the program model

For additional information on programs OCAR provides or if you are interested in implementing one of our current programs or contracting with OCAR for an independent evaluation of your program, please contact Dr. Rebecca Newgent at or 479-575-5593.

Posted in Clinical

Technology Office

Mission Statement

The mission of the Office for technology and support (OTS) is to facilitate all technological requirements of the NORMES offices and its users. In addition, we provide computer training, systems and software support, graphic and web design, multi-language computer programming and data management. OTS houses and maintains the NORMES Web Application and Data Center (WADC).

Supported Computing Platforms

- Windows NT / 2000 / XP
- Linux / Unix

Programming Languages





Sun Solaris 10

Sun M4000

Attached to Storage Tek Array (10 TB)

SAS, WebApps, Portal, etc.

Windows Server 2003

Dell Poweredge 6850

Attached to 2TB Array (Powervault 220s)

SAS_Data, Programs repository (SAS, Office etc)

Windows server 2003

Dell Poweredge 1850

136 GB

SAS backend (SAS Remote Svcs, Tomcat, PostgreSQL)

Windows server 2003

Dell Poweredge 650

35 GB

Web Php Files, Apache

Ubuntu Server 6.10

Dell Poweredge 2650

133 GB

MySQL, XMPP Chat Server (Open fire)

Windows server 2003

Dell Poweredge 650

35 GB

Windows Domain Controller, DNS Server, WINS

Windows server 2003

Dell Poweredge 650

35 GB

Windows Active Directory, VPN

Ubuntu Server 6.10

Dell Poweredge 2650

132 GB

Old Public Webserver (External IP Address)

Windows server 2000

Dell Poweredge 4600



Windows server 2003

Dell Poweredge 1850

136 GB

Enterprise Guide (Accounts and Data, NORMES_AR)

Red Hat Linux 4

Dell Poweredge 1950

65 GB

Yosemite Software, Connection to Tape Backup


Dell Poweredge 1950

Apache Deamon for, website data (WordPress, etc.)

Posted in Technology Office

Ed Policy

Mission Statement

To develop and provide data driven, scientifically-based educational research, in hopes that it will result in educational policy decisions rooted in empirical evidence. The scientifically-based educational research will be disseminated through the following:

The Educational Research and Policy Studies Journal (ERPS)

The Educational Research and Policy Studies Journal has evolved from the Arkansas Educational Research and Policy Studies Journal to have a more national emphasis and be a publishing source for academics who are conducting scientifically based research in education. To date, eleven issues have been published and additional manuscripts are being sought. The ERPS journal can be viewed here.

Geographical Academic Policy Series or “GAPS”

Many questions and analyses that we are asked to conduct can be difficult for non-statisticians to understand, but all types of people are interested in educational issues in Arkansas. GAPS has been designed to provide district and school-level statistical information through a color-coded system. In short, GREEN is positive and RED is negative. A quick review of most graphs provides an interpretation of the research question and how it impacts different regions of Arkansas. Graphs can be viewed here.

“Ask the Researchers”: An Educational Policy Series

The goal of Ask the Researchers is to provide a brief (1-2 page) forum on various educational issues. It is our intent to invite faculty members from NORMES, the University of Arkansas and other universities to participate in this series. Currently, you can view our policy briefs here.

Posted in Ed Policy

Educational Statistics and Research Methods Program

In the News

UA Professor Named First Billingsley Chair in Educational Research and Policy Studies

ESRM Faculty

George Denny, Professor of Educational Statistics and Research Methods

Karee Dunn, Clinical Assistant Professor of Educational Statistics and Research Methods

Wen-Juo Lo, Assistant Professor of Educational Statistics and Research Methods

Christopher Lucas, Professor of Educational Statistics and Research Methods

Sean Mulvenon, Professor of Educational Statistics and Research Methods

Charles Stegman, Professor of Educational Statistics and Research Methods

Ronna Turner, Associate Professor of Educational Statistics and Research Methods

ESRM Students

Kelli Blackford, Doctoral Student

Denise Airola, Doctoral Student

Calli Holaway-Johnson, Doctoral Student

Elizabeth Keiffer, Doctoral Student

Jam Khojasteh, Doctoral Student

Sarah McKenzie, Doctoral Student

Karen Morton, Doctoral Student

Rachel Sutcliffe, Doctoral Student

ESRM programs

Master of Science (M.S.) in Educational Statistics and Research Methods

Doctor of Philosophy (Ph.D.) in Educational Statistics and Research Methods

Posted in Ed Stats Program



Office of Research, Measurement and Evaluation


The Office of Research, Measurement, & Evaluation (ORME) at the University of Arkansas was created in 1998 by Dr. Sean Mulvenon, Director, and Dr. Ronna Turner, Associate Director, for the purpose of conducting targeted educational research, consistent with the mission of the university as a land grant academic institution. Since ORME’s inception, its graduate students and faculty have developed partnerships with local educational agencies, state departments of education, local community groups, and faculty of other programs at the University of Arkansas. Past collaborative activities have included educational database development, management, and analysis; data assessment and analysis training; academic accountability model development; program and project evaluation; and theoretical research in the fields of education, statistics, and measurement. In 2003, Dr. Mulvenon secured legislative appropriations funding for the National Office of Research, Measurement, & Evaluation Systems (NORMES) which resulted in the development of a national office focused on large-scale database and systems analysis development for use by educators in Arkansas and other participating states for the purpose of conducting data-driven decision-making to facilitate academic reform.

Our Mission

Our mission is to conduct targeted educational research, consistent with the mission of a land grant academic institution. The faculty and graduate assistants of ORME will continue to seek external funding through grant writing, conduct targeted educational research that addresses educational issues in Arkansas, and work to develop journal publications and conference presentations to augment the visibility of the University of Arkansas.


In 2003, Dr. Turner became the Director of ORME, which functions as an office within NORMES, with a revised focus of continuing program and project evaluation, conducting targeted research in the fields of education and social science, and continuing theoretical research in the fields of education, testing, and measurement. Current grants include:

* Collaboration with Physics faculty on a National Science Foundation GK12 grant: K-12 I Can Do Science! (KIDS) partnering graduate students in the fields of science, engineering, and math with local middle school classrooms for the purpose of conducting inquiry-based collaborative activities in science and math classrooms;
* Collaboration with Physics faculty on a U.S. Department of Education FIPSE grant incorporating business skills training into a traditional physics graduate program; and
* Evaluation of the development of a Microelectronics-Photonics interdisciplinary science and engineering graduate degree program funded by a National Science Foundation IGERT grant: A New Era in Electronics Education. The interdisciplinary program also incorporates business skills training and innovative research facilities to provide an advanced technological degree with a focus on meeting the increased expectations of the science and engineering industry.

Selected research articles and presentations related to grant activities and ORME research are also available.

Research Services

ORME is a multifaceted educational and health research agency that is ready to provide multiple services to agencies doing research. The staff at ORME includes professors with expertise in statistics, measurement, evaluation, policy analysis, and health issues. Additional technical support staff members are available with expertise in computer programming, web-site development, database management, and data collection.

The following types of services are available:

  • Support for grants that are being written
  • Structuring research agendas within grants
  • Designing studies
  • Data collection and maintenance
  • Data analysis using the best statistical models
  • Conducting evaluation studies
  • Provide research for policy-makers
  • Developing reports for dissemination and publication
  • Utilize web-based technologies to support research

Posted in ORME

Research Office

Posted in NORMES Research

  • Latest News
  • Welcome To NORMES
  • Subscribe

Contact Us

University of Arkansas
346 North West Avenue
302 WAAX Fayetteville, AR 72701
E- mail:
Phone: (479)575-5593
Fax: (479)575-5185