“What rules do we play by?” is the question we’ve followed in our bibliometric study which has just been published in the renowned journal Research Policy. Given the growing importance of journal rankings in academic performance management, it is relevant to researchers and managers alike whether there are certain characteristics of publications that are more prevalent the higher a journal is ranked. Our paper examines how tangible and adaptable characteristics of papers vary between different rating categories of journals and what the drivers of publication in journals at the top of rankings are. We build on a bibliometric analysis of more than 85,000 papers published in 168 management and business journals as rated in 18 popular journal rankings. Results refute some often repeated but rarely substantiated criticisms of journal rankings. Contrary to many voices, we find that interdisciplinarity and innovativeness are positively associated with publication in highly ranked journals. In other respects, our results support more critical assumptions, such as a widespread preference for quantitative methods. By providing more evidence on the implicit standards of journal rankings, this study expands on the understanding of what intended or unintended incentives they provide and how to use them responsibly.
The results of an international web-based survey on journal editors in four disciplines were published in Research Evaluation. The article is titled “How innovative are editors?: evidence across journals and disciplines”. Journal editors play a crucial role in the scientific publication system, as they make the final decision on acceptance or rejection of manuscripts. Some critics, however, suspect that the more innovative a manuscript is, the less likely it will be accepted for publication. Especially top-tier journals are accused of rejecting innovative research. As evidence is only anecdotal, this article empirically examines the demand side for innovative research manuscripts. I assess journal editors’ innovativeness, i.e. their general predispositions for innovative research manuscripts. As antecedents to innovativeness, personal and contextual factors are taken into account. I differentiate the concept of innovativeness in research by distinguishing three dimensions: innovativeness in terms of research problems, theoretical approaches, and methodological approaches. Drawing on an international web-based survey, this study is based on responses of 866 journal editors. The article sheds light on editors’ inclination toward accepting different forms of innovative research for publication. Overall, findings indicate that individual characteristics, such as editorial risk-taking or long-term orientation, are more decisive than journal-related characteristics regarding innovativeness. However, editors of older journals turn out to be less open toward new research problems and a u-shaped relationship between a journal’s rating score and editor’s willingness to adopt new theoretical approaches exists. Most surprisingly, editors’ consensus orientation regarding reviewer recommendations is positively associated with methodological innovativeness.
I just came across this beautiful website on which Northeastern University’s Barabasi Lab created an interactive visualization of Roberta Sinatra and colleagues’ paper “Quantifying the Evolution of Individual Scientific Impact”. The paper argues that scientists’ most impactful publications (as measured by citations) could occur at any point in their career. The authors base their argument on a large-scale bibliographic dataset containing publications of more than 10,000 scientists in different disciplines.
The visualization looks like a life-line for an individual scientist’s work (picture on the left) or an ocean with wave peaks and valleys for whole disciplines (picture on the right). It is striking how the overall disciplinary patterns look very similar – at least when they are corrected for absolute citation counts. The authors found that impact is randomly distributed within a scientist’s career. So, if you haven’t published an impactful paper yet, don’t give up – it might just be the next one.
Pictures are screenshots from the website: http://scienceofsuccess.barabasilab.com/
There’s also a short clip about the project on youtube:
Scientometrics published our article Editorial governance and journal impact: a study of management and business journals. It examines how characteristics of editors, in particular the diversity of editorial teams, are related to journal impact. Our sample comprises 2244 editors who were affiliated with 645 volumes of 138 business and management journals. Results show that multiple editorships and editors’ affiliation to institutions of high reputation are positively related to journal impact, while the length of editors’ terms is negatively associated with impact scores. Surprisingly, we find that diversity of editorial teams in terms of gender and nationality is largely unrelated to journal impact. Our study extends the scarce knowledge on editorial teams and their relevance to journal impact by integrating different strands of literature and studying several demographic factors simultaneously.
Together with other selected contributions from the last workshop higher education management, the German journal “Hochschulmanagement” (higher education management) published our paper “Open post-publication-peer-review: an alternative to double-blind reviews in academic journals?”.
The study contributes to the discussion about alternative forms of scientific communication by evaluating the actual dissemination as well as the potential use of open post-publication-peer-review (OPR). The study is based on survey data with a sample of 2.800 authors of academic papers. Results show that only one third of respondents believe that OPR is useful for enhancing the operative reliability of review processes. The advantages of OPR discussed in the literature are only relevant for the general willingness of authors to publish with OPR in principal. However, when it comes to actual publication decisions (open vs. blind peer review), these potential advantages are only of minor importance for the selection of an appropriate journal (with the exception of heterodox research which indeed seems to benefit from OPR). Instead, the choice between the different channels of scientific communication is based on institutionalized aspects (legitimacy, quality, design of the systems) and behavioral considerations (expected negative group dynamics and increased workload of OPR). Within the limitations of our dataset, we conclude that the current potential of OPR to solve the problems of traditional double-blind proesses is limited.
I know, publication bias is not a new topic but it is still of high relevance. I found some very interesting results in a study from Annie Franco, Neil Malhotra, Gabor Simonovits published in Science (19 Sep 2014, Vol. 345, Issue 6203, pp. 1502-1505): Publication bias in the social sciences: Unlocking the file drawer. According to the authors, “only 10 out of 48 null results were published, whereas 56 out of 91 studies with strongly significant results made it into a journal.” The following figure summarizes the results:
The pattern is quite remarkable. The majority of evidence that does not support any hypothesized relationship is not even written-up in the first place. So there’s reason for doubt that special platforms or journals who publish papers with contrary findings – as it is regularly discussed for overcoming publication bias – will significantly increase the number of null results published.
The topic of this year’s EFMD Higher Education Research Conference was ‘Innovations in Higher Education’ – the focus was on innovations in forms of governance, management and organisation of higher education institutions and business schools, innovations in education and innovations in research.
The conference took place in Barcelona from 10 – 11 October 2016 and was hosted by the IESE Business School, University of Navarra, on the edge of the beautiful Barcelona. It was the fifth such conference since the beginning in 2012. Presentations span a range of research problems, all addressing in a way the topic of innovation, from different (disciplinary) perspectives and/ or in different research contexts, which raised lively discussions. Interesting keynotes coplemented the conference, e.g., by Della Bradshaw, former business education editor of the Financial Times and one of the architects of the Financial Times 45 list (since 2016 Financial Times 50 list), with her keynote on “What has management education achieved in the last twenty years? What is the future holding for business schools?”.
For more information on next year’s EFMD Higher Education Research Conference, please have a look here.
The closing conference of the program “Performances de la recherche en sciences humaines et sociales” takes place at the University of Bern from 3 – 4 November 2016.
The conference addresses topics such as impact and quality in research in the humanities and the social sciences, excellence in research and societal usefulness, conveying quality of research in political areas and preserving diversity of research in times of excellence categories and rankings. The closing conference presents the results of the program and opens the discussion. It is the conference’s aim to bring together researchers, project leaders and important stakeholders of the Swiss higher education landscape.
- Prof. Dr. Shalini Randeria, Rector of the Institute for Human Sciences (IWM) in Vienna and Research Director and Professor of Social Anthropology and Sociology at the Graduate Institute of International and Development Studies (IHEID) in Geneva
- Prof. Dr. Peter Dahler-Larsen, Department of Political Science, University of Copenhagen, former President of European Evaluation Society
The 19th Irish Academy of Management (IAM) Conference was held at the UCD Michael Smurfit Graduate School of Business in Dublin from 31 August – 2 September 2016. The main conference theme was “Re-imagining business and the role of ethics”.
The conference provided a platform for a broad range of papers and interesting plenary panels, such as the keynote presentation titled “Towards a pluriversal world of academies of management”, which was held by Professors Nic Beech and Bill Cooke who represented the British Academy of Management.
The presented conference papers were anchored in a wide range of topics, such as “Policy, Legitimacy and ideology”, “Ethics and discourse”, “Leadership and culture”, “Clusters and networks” or “Business School Education”. In the last-mentioned session, I presented my paper “How innovative are editors: Evidence across journals and disciplines”.
Attending this conference was a nice and exciting experience and provided me again with some valuable insights for my own research!
Governing universities is a multi-level as well as a highly paradoxical endeavor. The featured studies in this book examine critically the multifaceted repercussions of changing governance logics and how contradictory demands for scholarly peer control, market responsiveness, public policy control, and democratization create governance paradoxes. While a large body of academic literature has been focusing on the external governance of universities, this book shifts the focus on organizations’ internal characteristics, thus contributing to a deeper understanding of the changing governance in universities.
The book follows exigent calls for getting back to the heart of organization theory when studying organizational change and turns attention to strategies, structures, and control mechanisms as distinctive but interrelated elements of organizational designs. We take a multi-level approach to explore how universities develop strategies in order to cope with changes in their institutional environment (macro level), how universities implement these strategies in their structures and processes (meso level), and how universities design mechanisms to control the behavior of their members (micro level). As universities are highly complex knowledge-based organizations, their modus operandi, i.e. governing strategies, structures, and controls, needs to be responsive to the multiplicity of demands coming from both inside and outside the organization.
Volume 47 of Springer’s Higher Education Dynamics Series advances higher education research by gathering distinguished scholars with an academic background in management and organization studies and a research interest in the dynamics of university governance. Among them are JC Spender, Mats Alvesson, Alfred Kieser, and many more.
As empirical social scientists, we are always looking for relationships between different phenomena. However, we need to be very careful which variables we include in our empirical models. For example, Spector and Brannick (2011) point out that common control variables (e.g., age or size of an organization) are often rather included because of “methodological urban legends” than for theoretical reasons. Therefore, many significant findings denote unobserved relationships or just correlate by chance.
I just came across a website which collects spurious correlations – just in case anyone needs further arguments, why a relationship should be established conceptually first.
Source: Spector, P. E., & Brannick, M. T. (2011). Methodological urban legends: The misuse of statistical control variables. Organizational Research Methods.
The International Centre for Higher Education Research (INCHER) provides a useful information tool for researchers: a compilation of selected academic journals and abstracts in higher education research.
The recently published lists for 2013 and 2014 comprise 25 thematically relevant, especially international journals for higher education research as well as article titles, author names and abstracts for the volumes 2013 and 2014.
The compilations are provided annually and can be downloaded here.
We’ve presented our study on “Performance Indicators in Academic Research”, asking “Do they Improve Performance?” at the 20th Annual Conference of the International Research Society for Public Management in Hong Kong. We had the chance to discuss our study in a PMRA-sponsored panel. Our empirical findings’ implications that “after a thorough selection of researchers, the best way to enhance academic performance is to grant them autonomy and to govern them by expertise rather than by performance appraisals” activated great consent. Besides the conference, the journey was an intense and enriching cultural experience. We’re looking forward to next year’s conference in Budapest.
For those who are interested in learning R, I recommend this short learning tool: http://tryr.codeschool.com/
“Code School” covers basic R expressions and provides by this a first impression about the program.
Jetta Frost, Markus Reihlen, Ferdinand Wenzlaff and I just published our book on multi-level governance in universities. It summarizes the main results from our project “RePort” which ended in late 2014. In the book, we analyze how external governance of science determines internal coordination mechanisms. For that purpose, we detail university governance on a macro (strategies to cope with the institutional environment), a meso (structures and processes), and a micro level of analysis (behavior control). The book is in German and published by the Kölner Wissenschaftsverlag. You can order it here. Since the threefold framework (macro-meso-micro) proved to be very useful, we are also working on an edited volume in English using the same structure.
As reported before, I am currently visiting the Center for Higher Education and Science Studies at the University of Zurich. On October 7th, I will give a talk, titled “Governing universities between profession and organization”. It is concerned with the development of science studies and higher education research and reports on our latest research results. Attendance is free and visitors are welcome! You’ll find the announcement here (only in German).
Reproducibility is a core value of research. An open collaboration of more than 100 authors has recently conducted replications of empirical studies published in psychology journals and found that replication effects are considerably weaker than the original effects. While 97% of original studies had significant results, only 36% of replications had. The authors conclude that journal reviewers and editors may reject replication studies as unoriginal and prefer innovative studies instead. However, “innovation points out paths that are possible; replication points out paths that are likely; progress relies on both.” Read the full paper in Science here.
The following studies on the practical relevance of management research and on university governance have been accepted for publication:
Alfred Kieser, Alexander Nicolai, and David Seidl develop a research program to investigate how the results of scientific research are utilized in management practice. Forthcoming as “The Practical Relevance of Management Research. Turning the Debate on Relevance into a Rigorous Scientific Research Program” in The Academy of Management Annals.
Bernadette Bullinger, Alfred Kieser, and Simone Schiller-Merkens find rigour and relevance to be embedded in competing institutional logics. Their forthcoming article “Coping with Institutional Complexity: Responses of Management Scholars to Competing Logics in the Field of Management Studies” will be published in the Scandinavian Journal of Management.
Following upper echelon theory, Steffen Blaschke and Fabian Hattke analyze how diversity of top management teams in universities relates to improved performance in terms of academic reputation and grant acquisition. The article “Striving for Excellence: The Role of Top Management Team Diversity in Universities” is forthcoming in a special issue of Team Performance Management.
Peer reviewing is a widespread procedure for evaluating the quality of scholarly manuscripts before publication. As such, it is at the heart of academia. Yet, little is known how scholars percieve the peer-review process.
Today, we initiated our survey into the peer-review system from the perspective of those who submit papers to academic journals. The survey asks questions about general attitudes towards your job, your personal experiences with peer reviews, and possible alternatives to the common pre-publication peer review process. We’ll keep you posted on the results!