Alliance for Organizational Psychology 

Memorandum of Understanding 

on Fostering Ethical, Relevant and Rigorous Research

 

This Memorandum of Understanding on fostering ethical, rigorous, and relevant research has been developed during the first Alliance for Organizational Psychology (AOP) Small Group Meeting held in Zürich, Switzerland on February 12 and 13, 2016, with several representatives of the three founding organizations of the AOP – Division 1 of IAAP, EAWOP, and SIOP – participating.

There currently are two very active debates in academia concerning (un)ethical scientific conduct on the one hand and (re)balancing academic rigor and relevance on the other. There are common root causes for the problems discussed in both. Agreement was reached that the most pressing problems that impede the conduct of ethical, rigorous and relevant research in Work and Organizational

Psychology specifically and in psychology and possibly other sciences more broadly are the following: 

1.      Research quality, impact and reputation are compromised by publication policies that

•    place too little emphasis on cumulative theory building and testing and by contrast place too much emphasis on empirical studies having to test novel theory.

•    place too little emphasis on real-world importance of the problems addressed in research.

•    encourage authors to produce Results sections that are flattering to Introduction sections. 

2.      Common academic systems share some of the blame by incentivizing quantity (rather than quality) of individual output, which leads to an overloaded review and publication process and encourages unethical behavior (among those so inclined). 

3.      Academic systems promote theory novelty rather than major innovation and the use of a small range of scientific methods, both of which hamper research relevance.

4.      There are few if any institutional mechanisms for influencing policy makers. 

There is also agreement that some effective solutions to these problems already exist:

1.      Regarding publication policies, we can begin by redefining theoretical contribution and empirical contribution so that there is room for theory trimming, theory refining, constructive replications, intervention studies, null findings, and multidisciplinary and multi-method research. 

2.      The burden on reviewers can be reduced by more desk rejections and by standardized reporting methods for empirical results.  

3.      Academic departments should use multiple quantitative and qualitative assessment criteria, including but going beyond publication/citation counts based on fixed journal lists, in hiring, tenure and promotion decisions.

Promotion of existing solutions and development and implementation of new measures will happen according to the following agreed upon action plan:

Task forces from among the participants of the Small Group Meeting (and possibly additional individuals) will

•      work on developing a repository of detailed checklists and practical guidance for reporting of quantitative and qualitative data analyses and results. 

•      write a position paper on redefining theoretical and empirical contributions to be submitted to a leading journal in psychology/management.

•      develop guidelines for multiple (and not solely quantitative) assessment criteria for hiring, tenure and promotion decisions. 

•      write a short statement based on this memorandum of understanding for publication in several of the leading psychology and management journals.

The governing boards of AOP, IAAP Division 1, EAWOP, and SIOP will be encouraged to take the necessary steps in their associations to

•      evaluate and facilitate access to online tools for methods training (e.g. CARMA) for the members of their organizations.

•      support building of a repository of detailed checklists and practical guidance for reporting of quantitative and qualitative analysis and results.

•      promote knowledge of and training for new methods through earmarked workshops at the associations' conferences.

•      promote training for advocacy through earmarked workshops at the associations' conferences.

•      promote multiple (and not solely quantitative) assessment criteria for hiring, tenure and promotion decisions through providing resources for academic departments/boards on their associations' webpages (e.g., collection of positive examples, proposal for balanced scorecard for academic performance, arguments against journal lists).

•      inform each other and where possible coordinate activities aimed at policy makers (e.g., contacts in Washington DC, Brussels and UN on cross-cutting topics; white papers on highly visible work practices; continuous overview of activities at the level of constituent organizations).

•      push for reference models for teaching in work and organizational psychology and inclusion of work and organizational psychology in curricula and textbooks at the Bachelor level.

•      lobby for adequate representation of work and organizational psychology in funding bodies and earmarked funding.

•      explore costs and benefits of collaboration with "behavioral insights" initiatives rooted in behavioral economics.

•      explore expansion of translational publication outlets. 

•      reach out to engineering and economics associations for possible collaboration.

The editors of Journal of Applied Psychology, Leadership Quarterly, European Journal of Work and Organizational Psychology, Journal of Business and Psychology (and possibly further journals) will be asked to

•      encourage replication, consolidation of constructs, intervention studies, and problem-driven research, possibly through special issues and special calls for participation from a wide range of countries.

•      promote an understanding of and encourage paradigmatic theory building and testing based on compelling and sound research designs (e.g., good use of the breadth of psychology as a natural and social science, sensible trade-offs regarding endogeneity problems).

•      encourage and incentivize continuous reviewer training, including specialized methods reviewers, and monitor and reward review quality in order to build and publicize reviewer credentials.

•      strengthen editors' role as educators rather than police with respect to academic misconduct.

•      establish accountability among authors to act as reviewers also (e.g., introduce "pubcoins", which would provide a transparent account of review activity and might be used to prohibit submission in case an author has not been active as a reviewer).

•      require proof of at least one friendly review before submission.

•      provide access to a repository of detailed checklists and practical guidance for reporting of quantitative data analyses and results for both authors and reviewers.

•      provide practical guidance for reporting of qualitative research.

All participants of the Small Group Meeting in their various roles (e.g., as heads of department, board members of funding agencies, executives of professional organizations) will

•      promote a broad set of assessment criteria (including expert assessment of the quality, relevance and impact of the candidate’s research and dissemination activities) for hiring, tenure and promotion decisions.

•      increase incentives in academic departments for creation and documentation of practical impact within and beyond single organizations.

•      lobby for funding of systematic reviews / white papers aimed at policy makers.

•      promote multi-disciplinary and problem-driven research and the systematic combination of descriptive, normative, and prescriptive research in own work, through training and mentoring of others and through thematic research centers ("think big").

•      promote open source methods, resources, and training.

•      devote time to connect with policy-makers and potential users of own research.

•      devote time to membership in professional and research decision-making bodies.

•      lobby for resources and opportunities for outreach (e.g., press work, media training, practitioner conferences).

•      explore new channels of communication.

•      explore cooperation with local /"grass-root" groups.

•      provide opportunities for PhD students for academic exchanges within and across disciplines and for internships in practice.

•      submit roundtables/symposia with content from this Memorandum of Understanding to relevant conferences.

The organizers of the Small Group meeting will set up a mechanism for regular communication among the participants of the Small Group Meeting and progress reports on the implementation of the action plan. 

The participants of the Small Group Meeting agree to this Memorandum and have declared their signature of the document.

 

Zürich, February 13, 2016

 

 

John Antonakis, University of Lausanne

 

 

John Arnold, Loughborough

University

 

 

Felix Brodbeck , LudwigMaximilians-Universität Munich

 

 

José Cortina, George Mason University

 

 

Kevin Daniels, University of East Anglia

 

 

Franco Fraccaroli, University of Trento

 

 

Gloria Gonzalez-Morales, University of Guelph

 

Vicente Gonzalez-Roma,

University of Valencia

 

 

Gudela Grote, ETH Zürich

 

 

Deanne den Hartog, University of Amsterdam

 

 

Gerard P. Hodgkinson, University of Warwick

 

 

Tine Köhler, University of

Melbourne

 

 

Steve Kozlowski, Michigan State University

 

 

Barbara Kozusznik, University of Silesia

 

 

Malgorzata Kozusznik, University of Valencia

 

 

Ron Landis, Illinois Institute of Technology

 

 

Ioannis Nikolaou, Athens

University of Economics and

Business

 

 

Ernest O'Boyle, University of Iowa

 

 

José María Peiró, University of Valencia

 

 

Ramon Rico, Autonomous

University of Madrid

 

 

Rosalind Searle, Coventry

University

 

 

Silvia Silva, ISCTE-IUL Instituto Universitário de Lisboa

 

 

Donald Truxillo, Portland State

University

 

 

Fred Zijlstra , Maastricht

University