articles on research productivity

Once highly productive, forever highly productive? Full professors’ research productivity from a longitudinal perspective

• Open access
• Published: 24 March 2023
• Volume 87 , pages 519–549, ( 2024 )

Cite this article

You have full access to this open access article

• Marek Kwiek   ORCID: orcid.org/0000-0001-7953-1063 1 , 2 , 3 &
• Wojciech Roszka   ORCID: orcid.org/0000-0003-4383-3259 2 , 4  

7534 Accesses

7 Citations

41 Altmetric

Explore all metrics

This longitudinal study explores persistence in research productivity at the individual level over academic lifetime: can highly productive scientists maintain relatively high levels of productivity. We examined academic careers of 2326 Polish full professors, including their lifetime biographical and publication histories. We studied their promotions and publications between promotions (79,027 articles) over a 40-year period across 14 science, technology, engineering, mathematics, and medicine (STEMM) disciplines. We used prestige-normalized productivity in which more weight is given to articles in high-impact than in low-impact journals, recognizing the highly stratified nature of academic science. Our results show that half of the top productive assistant professors continued as top productive associate professors, and half of the top productive associate professors continued as top productive full professors (52.6% and 50.8%). Top-to-bottom and bottom-to-top transitions in productivity classes occurred only marginally. In logistic regression models, two powerful predictors of belonging to the top productivity class for full professors were being highly productive as assistant professors and as associate professors (increasing the odds, on average, by 179% and 361%). Neither gender nor age (biological or academic) emerged as statistically significant. Our findings have important implications for hiring policies: hiring high- and low-productivity scientists may have long-standing consequences for institutions and national science systems as academic scientists usually remain in the system for decades. The Observatory of Polish Science (100,000 scientists, 380,000 publications) and Scopus metadata on 935,167 Polish articles were used, showing the power of combining biographical registry data with structured Big Data in academic profession studies.

Similar content being viewed by others

Are Scientists Changing their Research Productivity Classes When They Move Up the Academic Ladder?

The Annus Mirabilis paper: years of peak productivity in scientific careers

How long do top scientists maintain their stardom an analysis by region, gender and discipline: evidence from italy.

Avoid common mistakes on your manuscript.

Introduction

This study explores persistence in research productivity at the individual level over academic lifetime. We examine the trajectories of the academic careers of 2326 Polish full professors, including their lifetime biographical histories and their lifetime publication histories. We studied the dates of their academic promotions and their publication output (79,027 articles) between promotions over a 40-year period across 14 science, technology, engineering, mathematics, and medicine (STEMM) disciplines. Our focus was on transitions between productivity classes throughout the professors’ academic careers, from the assistant professor stage to the full professor stage.

We hypothesized that the current placement of full professors in the productivity classes of top, middle, and bottom (i.e., top 20%, middle 60%, and bottom 20% of scientists in prestige-normalized productivity in each discipline) corresponds, to some degree, to their placement in productivity classes at earlier stages of their careers. We speculated that current highly productive full professors could have also been highly productive associate professors and highly productive assistant professors earlier in their careers.

Our starting point was the current distribution of full professors by productivity classes in the 4-year period from 2014 to 2017. They were classified as either highly productive, average productive, or low productive. We then examined the productivity classes to which they could be retrospectively assigned at earlier stages of their careers.

The guiding thread of the paper follows the central findings of studies of highly productive scientists and their attributes (e.g., Fox & Nikivincze, 2021 ; Yin & Zhi, 2017 ; Agrawal et al., 2017 ; Cortés et al., 2016 ; Abramo et al., 2017 ; Kwiek  2016 ). Our research addresses three parallel questions: To what extent does scientists’ research productivity change over their academic lifetime? Have currently highly productive scientists always been highly productive? Is it rare for radical changes (moving up or down) in productivity class to occur over academic careers? Most productivity studies focus on the individual traits of highly productive scientists, and many combine individual and organizational (environmental) features (Fox & Mohapatra, 2007 ; Fox & Nikivincze, 2021 ). Our approach to productivity analysis is longitudinal, relative (class based), and prestige normalized:

longitudinal : we trace the productivity of full professors in our sample for several decades (since they entered the higher education system)

relative : we do not examine publication numbers but focus on productivity classes, retrospectively assigning individuals to classes and comparing scientists to their peers in their disciplines and career stages

prestige normalized : more weight is given to articles in high-impact journals than to those in low-impact journals, recognizing the highly stratified nature of academic science, especially in STEMM disciplines

Beyond its scholarly implications (in theories of productivity), persistent high productivity over the course of individual scientists’ careers has implications for hiring and promotion policies. How can high productivity be maintained in departments, institutions, and national systems when, as this research shows, there is only marginal mobility between the lowest and highest productivity classes?

In this study, the unit of analysis was the individual researcher, not the individual publication. Although we used a combination of administrative, biographical, and bibliometric data, our research was not bibliometric in nature and belongs to academic profession studies. It was not possible to perform lifetime retrospective analyses of individual scientists without having full access to raw bibliometric metadata for all publications by all individual Polish scientists in the past 50 years. It was not possible to construct retrospective productivity classes for all scientists by discipline, career stage, and selected periods between promotions without having access to each scientist’s global publication metadata - without the ability to collect structured Big Data from Scopus, a commercial bibliometric database. Our study provides an example of combining structured Big Data and national registry data to conduct detailed analyses of academic careers within a national (i.e., Polish) academic science system.

Literature review

High research productivity.

For at least half a century, the sociology of science and of academic careers has addressed the theme of inequality in academic knowledge production (Hermanowicz, 2012 ), as a small percentage of scientists “contribute disproportionately to the advancement of science and receive a disproportionately large share of rewards and resources needed for research” (Zuckerman, 1988 : 526). Within the Mertonian sociological tradition, the priority of discovery is important (Merton, 1973 : 293), as one of the more salient motivations of scientists is “the desire for peer recognition” (Cole & Cole, 1973 : 10). The scientific community, therefore, is not a “company of equals,” and recognition of their work is “the only unambiguous demonstration that what [scientists] have done matters to science” (Zuckerman, 1988 : 526). The recognition afforded by publications and citations translates into funds for further research, and the distribution of output, citations, academic awards, and research funding is highly stratified. At an institutional level, inequality and stratification are pervasive features of higher education systems, with “endless competition” between “status-seeking” institutions (Taylor & Cantwell, 2019 ); at the individual level, research powerfully segregates the academic profession, and rewards are “distributed in a highly stratified fashion” (Marginson, 2014 : 107).

In any system of academic science, a small number of scientists publish the majority of papers and attract the most citations (Abramo et al., 2009 ; Ruiz-Castillo & Costas, 2014 ; Stephan, 2012 ). In every scientific community, highly productive scientists hold prestigious academic positions and are responsible for shaping the identity of academic disciplines (Cortés et al., 2016 ). How do highly productive scientists emerge in higher education? Scientific productivity is believed to result from (1) individual attributes, (2) organizational attributes (academic environment), and (3) features of the national academic science system, within which the allocation of rewards and recognition of research achievements play important roles. Science is a complicated social institution, and scientists must be systemically supported within their national edifice of science to sustain high productivity over time. Indeed, the efficient operation of science depends on how it “divides up the rewards and prizes it offers for outstanding performance, and structures opportunities for those who hold extraordinary talent” (Cole & Cole, 1973 : 15).

Access to resources is enjoyed by those held in high esteem within the scientific community, who are strongly motivated to publish because scientific esteem “flows to those who are highly productive” (Allison & Stewart, 1974 : 604). Highly productive scientists are those whose productivity persists over time (Abramo et al., 2017 ), they are the small group that maintains high productivity in their own work, supported or not by structural features of the science system, including mechanisms for accumulating advantages over time. Cumulative advantage is a broader process by which “small initial differences compound to yield large differences” (Aguinis & O’Boyle, 2014 : 5). In science, consequently, the Matthew effect leads to inequalities in access to financial and nonfinancial rewards (Xie, 2014 ).

Historically, the sociology of science shows that scientific recognition is rooted almost exclusively in research production (Cole and Cole, 1973 ), and the reward system is structured to benefit scientists who best perform their role. In Merton’s ( 1973 : 297) words, “The institution of science has developed an elaborate system for allocating rewards to those who variously live up to its norms.” In Merton’s reputation- and resource-based model of scientific careers, new resources are not simply rewards for past high productivity but serve the primary function of stimulating future high productivity: “The scientific community favors those who have achieved significant success in the past” (DiPrete & Eirich, 2006 : 282). In the past decade, studies have looked intensively at high research productivity (e.g., Yair et al., 2017 ; Aguinis & O’Boyle, 2014 ; Agrawal et al., 2017 ; Abramo et al., 2017 ; Yin & Zhi, 2017 ; Piro et al., 2016 ; Kwiek, 2016 , 2018 ). Most recently, Fox and Nikivincze ( 2021 ) studied highly prolific scientists from a social-organizational perspective that examined both individual characteristics and departmental features. They identified three predictors of high productivity: rank, collaborative span, and a favorable work climate ( work climate being a perceived departmental atmosphere that stimulates [or stifles] performance) (Fox & Mohapatra, 2007 ). Abramo et al. ( 2017 ), whose research most nearly resembles ours, examined the research performance of all Italian professors in the sciences over three consecutive 4-year periods (2001–2012). Their analyses demonstrate that 35% of top scientists retain their high productivity for three consecutive periods, and 55% do so for two periods. Higher percentages of male scientists than female scientists keep their stardom, with some cross-disciplinary differences (Abramo et al., 2017 : 793–794). Our research differs from theirs in time span (lifetime vs. 12 years), sample selection (full professors vs. all professors), and methodology (three productivity classes vs. top performers and unproductive scientists; prestige-normalized productivity vs. Fractional Scientific Strength).

Several key theories have emerged from the sociology and economics of science to explain dramatic differences in individual research productivity, which may be useful in studying the stratification of Polish scientists. The “sacred spark” theory (Cole & Cole, 1973 ) states that “there are important, predetermined differences among scientists regarding their ability and motivation for creative scientific research” (Allison & Stewart, 1974 : 596). Highly productive researchers “are motivated by an inner drive to create science and by a pure love of the work” (Cole & Cole, 1973 : 62). Productive scientists are a highly motivated group of researchers and have the necessary “capacity to work hard and persist in the pursuit of long-range goals” (Fox, 1983 : 287). Stephan and Levin ( 1992 : 13) hold a similar view, stating that “there is a general consensus that certain people are particularly good at doing science and that some are not just good but superb.” Cumulative advantage theory (Merton, 1973 ) holds that productive scientists will be even more productive in the future, while low-productivity scientists become even less productive over time. “Scientists who are rewarded are productive, while scientists who are not rewarded become less productive” (Cole & Cole, 1973 : 114). Finally, the utility maximization theory, which emerged from the economics of science, asserts that researchers reduce their research-oriented efforts with time because they believe that other tasks may be more personally rewarding for them. Discussing aging and productivity, Stephan and Levin ( 1992 : 35) argue that “later in their careers, scientists are less financially motivated to conduct research” (see Kyvik, 1990 ). These three main theories of research productivity complement one another and apply to varying degrees to the academic profession in Poland (Kwiek, 2019 : 27-32). The sacred spark and cumulative advantage theories explain high research productivity, while low productivity in Poland may be understood through both the cumulative advantage and utility maximization theories.

Admission to the class of the most productive scientists requires a strong research orientation and long hours spent on research (Kwiek, 2016 , 2018 ) in addition to the innate capacities highlighted by the sacred spark theory and the prior achievements stressed in the cumulative advantage theory. A high proportion of the most productive scientists will always be among the most productive—regardless of circumstances, location in the system, age, and career stage—while only a small proportion of low-productivity scientists ever become highly productive as shown in this study. In the process of accumulating advantages, exceptional research productivity early in a career translates into new resources and rewards that make it easier to sustain high research productivity in subsequent years and decades. Research resources are not rewards for past productivity but are designed to stimulate the productivity of the most productive in the future: “The scientific community favors those who have achieved the most in the past in terms of the additional resources and attention they have enjoyed” (DiPrete & Eirich, 2006 : 281–282).

Full professorship: international insights

Research on full professors in academia is important to our study, as our sample is specific (including only full professors) and the topic is rarely studied due to the limited availability of data on academic ranks. Thus, we briefly discuss selected recent papers on full professors. In the US context, Yuret ( 2018 ) analyzed the “paths to success” in obtaining full professorships and found that promotion to full professorship was related to high mobility and short duration of PhD studies. Kolesnikov et al. ( 2018 ) studied full professors in two fields at 10 research-intensive universities to test the hypothesis that productivity is inversely correlated with impact. However, higher productivity led to lower impact in one field and to higher impact in the other.

The extant literature also includes studies in Israel and Norway. Weinberger and Zhitomirsky-Geffet ( 2021 ) examined diversity in scholarly performance among tenured professors at Israeli universities by distinguishing between high-, average-, and low-impact scholars. The results of their linear regression analysis show that women outperformed men in terms of scientific impact, and these differences in performance reveal that scholarly success and promotion to full professorship may not be fully determined by productivity (Weinberger & Zhitomirsky-Geffet, 2021 : 2949). In Norwegian universities, Piro et al. ( 2016 ) studied the influence of prolific full professors on the citation impact of their university department. While productivity was skewed at the level of individuals, the influence of prolific professors on their departments’ citation impact was modest.

Fox ( 2020 : 1002) argues that gender predicts academic rank: “Women are less likely than men to hold higher ranks, and the gender disparity is especially apparent for the rank of professors.” Regarding access to full professorships by gender, the recent evidence is inconclusive. Using multilevel logistic regression, Marini and Meschitti ( 2018 ) found that men had a 24% higher probability of being promoted in Italy than women with the same scientific output. Madison and Fahlman ( 2020 ) analyzed all promotions to full professorships in Swedish institutions, however, and found no bias against women attaining full professorships in relation to publication metrics. Lerchenmueller and Sorenson ( 2018 ) studied gender gaps in career transitions in the life sciences in the USA and found that the gender gap largely emerges during the brief period when men and women move from working in another researcher’s lab to leading their own (Lerchenmueller & Sorenson, 2018 : 1015). In Germany, Lutter and Schröder ( 2016 ) show that women sociologists were likelier than men with the same number of publications to obtain a full professorship. Among men, the strongest predictor of success was publication in Social Science Citation Index (SSCI) journals. Among women, by contrast, the strongest predictor was the accumulated number of academic awards. Drawing upon the profiles of 2,528 scholars, Habicht et al. ( 2022 ) examined gender differences in obtaining tenured psychology professorships in Germany, and they reject the female devaluation theory, which holds that women’s career achievements are devalued in relation to those of men.

Puuska ( 2010 ) focused on ranks, productivity, and various types of publication by 1,417 Finnish professors. The results show that full professors were the most productive; the principle of “the higher, the more productive” applied to all academic ranks (Puuska, 2010 : 428–430). In several male-dominated fields, female full professors were more productive than male full professors, which may indicate that, in those fields, only exceptionally productive women win full professorships (Puuska, 2010 : 435). Abramo et al. ( 2011 ) examined the links between individual productivity and academic rank among Italian university researchers. The results reveal a uniform productivity distribution across the ranks that only slightly favors full professors. Full professors exhibited the highest productivity, but “top scientists” (i.e., the upper 10%) were evenly distributed among the three ranks (Abramo et al., 2011 : 927). Aksnes et al. ( 2011 ) show that Norwegian full professors have a lower-than-average citation index despite their high productivity index; by far, the highest indexes in both categories were obtained by postdocs (Aksnes et al., 2011 : 632). Those authors also studied the impact of mobility on productivity by comparing mobile and non-mobile Norwegian scientists, finding that the mobile full professors were the most productive group (Aksnes et al., 2011 : 219). Finally, in the US context, Fox and Nikivincze ( 2021 : 1250) show that, compared with the rank of assistant professor, the rank of full professor was a strong and positive predictor of being highly prolific.

Research questions and hypotheses

The six research questions in Table 1 were based on selected findings in the previous studies mentioned in the “ High research productivity ” section (RQ1, RQ2, RQ4, and RQ6) and in the “ Full professorship: international insights ” (RQ3 and RQ5). The hypotheses pertain to the persistence of high productivity (H1) and low productivity (H2) over time; the persistence of high productivity at the beginning and towards the end of academic careers (H4); disciplinary differentiation (H3) and gender differentiation (H5) in mobility between productivity classes; and (H6) the role of past productivity class membership in estimating (via logistic regression analysis) the odds ratio of belonging to top productivity classes. An overarching research question concerns changes in productivity from a life-cycle perspective: have currently top-performing full professors always been top performing, and have currently low-performing full professors always underperformed?

Context, data, methods, and sample

The national context.

With 1,218,000 students, Poland’s higher education system employs 88,416 full-time academics (48.04% women) distributed among the major ranks as follows: 8,990 full professors (27.49% women), 17,303 associate professors (39.84% women), and 38,978 assistant professors (50.36% women). Regarding promotion to higher ranks in the higher education sector, there were 597 new full professors in 2021 (220 women; 36.85%) and 490 new associate professors (212 women; 43.27%); furthermore, 3,431 doctoral degrees were awarded (1752 women; 51.06%) (GUS, 2022 : 30–38). Publications come predominantly from universities in several major academic cities, with the bulk of internationally visible academic knowledge production coming from the 10 research-intensive Excellence Initiative—Research Universities (IDUB) institutions selected to receive additional funding in 2020–2026. The growth in globally indexed publications has been substantial in the past decade, increasing about 100% (from 31,707 in 2010 to 62,131 in 2021).

More than 90% of new hires are in-house hires (i.e., academics with doctoral degrees from the same institution); cross-institutional mobility is of marginal importance, and promotion to higher ranks is almost exclusively within the same institution. Salaries and workloads are similar across the system and regulated at a national level. All academics in the three ranks are expected to be equally involved in teaching and research. Promotion to the ranks of associate and full professor are nationally governed, related to degrees obtained (the habilitation and professorship degrees), and based on research achievements assessed by peer committees. Full professorship, the crowning achievement in an academic career, is desired by many but available to few (about 600 new professorships annually in the past few years). There are no institutional or national limits on the number of new full professorships, which are awarded on the basis of successfully passing rigorous, research-based national promotion procedures. Only one dimension of the academic career matters in promotions: research output since earning the habilitation degree. All assistant professorships are tenure-track positions, and tenure is granted to associate professors (upon obtaining the habilitation degree), with long-term job contracts and job stability for the vast majority of academics. In the past few years, expectations of publication in globally indexed journals have notably risen (although they have always been high in STEMM disciplines). Reward structures are similar across the system, with higher ranks offering higher salaries and greater participation in university self-governance.

Dataset and sample

The data used in this study were collected from a national administrative and biographical register of all Polish scientists ( N  = 99,935) and from the Scopus bibliometric database (2009–2018, N  = 380,000 publications). The final number of articles was 158,743, and they were published by 25,463 unique authors with Polish affiliations. The database was then enriched with publication metadata collected from Scopus, which were obtained through a collaboration agreement with the ICSR Lab, which is a cloud-computing platform provided for research purposes by Elsevier ( N  = 935,167 articles from 1973–2021 by authors with Polish affiliations). We used information about the entire academic output of individual authors based on their Scopus Author IDs in the database. Our final sample included full professors in 14 STEMM disciplines ( N  = 2326) who authored or co-authored 79,027 articles.

Defining academic disciplines and academic age

We defined individual attributes for the sample of 23,543 scientists in all academic positions and disciplines, including every full professor in the 14 STEMM disciplines in our final sample. In the All Science Journal Classification (ASJC) system of disciplines used in Scopus, a journal publication has one or multiple disciplinary classifications. The dominant discipline of each full professor was determined based on all publications (type: article) included in their individual publication portfolios for the period from 2009–2018 (the modal value is the most frequently occurring value). When there was no single value, the dominant discipline was randomly selected from among the most frequently occurring disciplines.

Our dataset included the professors’ year of birth and the year in which every full professor achieved three scientific degrees—the doctoral degree, habilitation degree, and professorship—which were used as proxies for assistant, associate, and full professor, respectively. The three degrees are clear markers in their careers, and their details (date, title, institution, discipline, field, reviewers) are available in our dataset. Full professors entered the Polish equivalent of tenure-track positions when they obtained their doctoral degrees and received permanent employment (the equivalent of tenure) when they obtained their habilitation degrees. As an analytical approach, we chose a three-degree system rather than a multiple-rank system (with “university professors” and “ordinary professors”) because the latter system is not consistently applied across all institutions. For an international audience, the best way to discuss promotions in rank in Poland is through the three-degree system, with clearly defined dates for each scientist and with nationally determined, research-based requirements.

We obtained the year of the first publication indexed in Scopus using the application programming interface (API) protocol, which is a set of programming codes that enable data transmission between one software product and another provided by Scopus. The gender of all scientists with at least a PhD degree is included in the data provided by the national registry of scientists, and in this study, it was treated as a binary variable.

Full professors: discipline, age, and gender distribution

The distribution of our final sample was as follows: about three-fourths of full professors were men (see Table  2 ); about one-third worked in 10 research-intensive IDUB institutions. The three disciplines with the largest number of full professors were medical sciences (MED), agricultural and biological sciences (AGRI), and engineering (ENG). About half of all Polish full professors in our sample were publishing in these three disciplines. The largest share of female full professors in larger disciplines was in biochemistry (BIO), MED, and AGRI (about one-third). The lowest share was in physics and astronomy (PHYS) (5.5%), mathematics (MATH) (6.3%), and ENG (5.8%). About two-thirds were aged more than 60 years and about a half were aged from 65–70 years. In our sample, 16% were young (under 55 years) full professors, including 2% aged 40–44 years. The distribution by biological age and gender are presented in Fig.  1 . The distribution of female scientists was equal across ages, while the distribution of male scientists was steeper. The distribution by age was more similar than expected. The gender distribution of the full professors in our sample was close to their gender distribution in the population over the past five years (GUS, 2022 ).

Distribution of biological age: kernel density plot, full professors in 14 STEMM academic disciplines combined, by gender

Methodological approach

Constructing lifetime biographical and lifetime publication histories.

The Laboratory of Polish Science database constructed and maintained by the authors includes the complete publication histories of all Polish scientists working in the higher education sector as of November 2017, holding at least a PhD degree, and having at least one publication in the Scopus database. The database includes the publication and citation metadata on all publications by each scientist in each stage of their scientific career. The database included data on 14,271 assistant professors, 7,418 associate professors, and 3,774 full professors in STEMM and non-STEMM disciplines.

However, we focused only on full professors, which enabled us to trace their individual biographical histories and individual publication histories in the earlier stages of their careers. Only full professors could be compared in three earlier stages. The analysis of full professors included a long period of scientific activities lasting several decades. We retrospectively examined the academic career classes of full professors who had been working for 20–40 years. The compilation of complete lifetime biographical histories (i.e., years of birth and years of subsequent academic promotions) and complete lifetime publication histories (i.e., detailed data on publications, collaborations, mobility, and citations), spanning entire academic careers, allowed us to retrospectively analyze the transitions between productivity classes over time of all full professors.

In this study, we applied a longitudinal approach to analyzing the transitions between the productivity classes of the full professors over their careers, from the year in which they received their PhD degrees to 2017. We analyzed the productivity of individual scientists as they aged and moved up the academic ladder. Each publishing scientist within their unique biographical history (based on dates) and unique publication history (based on publication metadata) was characterized by transitions between productivity classes compared with their peers in the same discipline and at the same career stage.

Constructing prestige-normalized research productivity

The productivity of researchers at a given stage in their academic career was calculated as the number of all publications (publication type: article) produced in that stage divided by the number of years spent at the stage (and multiplied by 4 to maintain the comparability of productivity over 4-year reference periods). Productivity may vary during academic careers, with pre-promotion peaks and post-promotion pauses (Katz, 1973 ), so this approach reduced potential differences between the first years after each promotion (when productivity may decrease) and the years just before a new promotion (when productivity may increase). We divided the academic careers of the full professors into three stages based on distinct opening and closing dates (doctorate, habilitation, full professorship), and we constructed both lifetime productivity profiles and productivity profiles in their three distinct career stages. We used a full counting approach instead of a fractional counting approach in which single-authored and multiple-authored publications were counted equally. We used the prestige-normalized publication number rather than the raw publication number.

In prestige-normalized productivity, we combined the output indicator of research productivity with the indicator of scholarly impact on science (based on citations). Output indicators measure the knowledge produced, and impact indicators measure the ways in which scholarly work affects the research community (Sugimoto & Larivière, 2018 : 1). The weight of an article depends on its position in the global hierarchy of academic journals. In our approach, articles published in journals with, on average, a high impact on the academic community captured through the proxy of average citation numbers were given more weight in calculating productivity than articles in low-impact journals because they required, on average, more scholarly effort to write and get published. Our approach to productivity recognizes the highly stratified nature of academic science, in which both the quantity of publications and their standardized quality are important.

Measuring journal prestige is closely related to the Polish system of evaluating scientists and scientific units and to the indicators used in the IDUB national excellence program. Articles in highly prestigious journals require, on average, a greater workload and have, on average, greater resonance in the world of science, as captured through citations. In Scopus, the prestige rank of a journal is determined annually by the journal’s placement in the CiteScore ranking system, which is prepared annually for all journals indexed (e.g., 40,562 in 2022). Percentile ranks are based on values in a range from 1 to 99, in which the highest prestige is the 99th percentile. Highly prestigious journals in each field, with low acceptance rates, tend to be in the 90–99th percentile ( Higher Education and Studies in Higher Education are in the 96th percentile of Scopus journals). Publications in more prestigious journals count more in productivity calculations compared with publications in less prestigious journals within each discipline.

In a non-normalized approach to productivity (full-counting), an article published in any journal would receive a value of 1. In contrast, in the prestige-normalized productivity approach, an article in a journal with a percentile rank of 90 received a value of 0.90, while an article in a journal with a percentile rank of 40 received a value of 0.40. Articles published in journals with percentile ranks of and below 10 received a value of 0.1. A prestige-normalized approach to individual research productivity allows for a fair measurement of scholarly effort in STEMM disciplines in which vertical journal stratification is a fact of life. Counting all publications in the same way would disregard individual scholarly efforts invested in research. Each discipline has specific highly competitive top-tier journals, and “the tyranny of the top five” (Heckman & Moktan, 2018 ) is applicable far beyond economics.

Constructing academic career classes

This study draws upon the notion of climbing the academic ladder, which defines the decades-long academic careers of full professors. The current full professors (promotion date: full professorship awarded) were initially assistant professors (promotion date: doctoral degree awarded) and then associate professors (promotion date: habilitation degree awarded). They all remained for a specific number of years at previous stages of their academic careers. At each stage, they demonstrated specific productivity. We ranked all academics (segregated by discipline) by their 4-year prestige-normalized research productivity within specific career stages. For each full professor, we counted all articles published within the stages as defined by promotion dates: the first stage is between doctoral degree and habilitation degree (which we term assistant professorship to reflect internationally understandable career steps), the second stage is between the habilitation degree and the title of professor (which we term associate professorship ), and the third stage is between the professorship title and 2017 (which we term full professorship ). For instance, if a biographical history of full professor X shows that she obtained her doctoral degree in 1995, habilitation degree in 2002, and full professorship in 2012, then her assistant professorship stage was 1995–2001, associate professorship stage 2002–2011, and full professorship stage 2012–2017.

Central to our analysis was the current distribution of full professors by productivity classes in the 4-year period from 2014–2017. They were classified as either high productivity, average productivity, or low productivity full professors. We then examined the productivity classes to which they could be retrospectively assigned at earlier stages of their careers, that is, when they were assistant professors and associate professors.

We assigned seven academic career classes to each full professor (see Fig.  2 ): three productivity classes, two promotion age classes, and two promotion speed classes. The current and past productivity classes were the top, middle, or bottom—that is, the upper 20%, middle 60%, or lower 20%, respectively, in a prestige-normalized and discipline-normalized approach separately within each of the 14 STEMM disciplines. The promotion age classes were young, middle, or old associate professors and young, middle, or old full professors. That is, the upper 20%, middle 60%, or lower 20%, respectively, in terms of promotion age expressed in full years. The promotion speed classes were fast track, typical track, and slow track associate professor and fast track, typical track, and slow track full professor, that is, the upper 20%, middle 60%, and lower 20%, respectively, in terms of the transition time between subsequent promotions, also expressed in full years.

Classification scheme used for full professors: productivity, promotion age, and promotion speed classes

At each stage of their careers, the full professors were more productive or less productive. They changed their productivity classes in relation to their colleagues in the same discipline and remained at the same stage of their academic career and in the same academic position. Our study compared “apples with apples” rather than “apples with oranges” (Nygaard et al., 2022 ). The scientists were consistently compared at the same stage of their careers within the same discipline.

What would not be possible without using raw Scopus (or WoS) metadata in our case? (1) To massively define disciplines : we examined all lifetime publications to determine the modal discipline of every full professor. (2) To massively measure prestige-normalized productivity : all publications in the lifetime publication histories of all full professors were linked to the journal prestige expressed in the Scopus journal’s percentile rank, and 4-year productivity was calculated accordingly. (3) To link every article to the three stages of the academic careers of all full professors: only Scopus (or WoS) had all articles by all full professors during their lifetimes. (4) To establish academic age for all full professors: the date of the first publication, regardless of type, was necessary in regression models.

Limitations

The present study has several limitations related to the data and methodology. First, our sample included all Polish scientists who were internationally visible through their research in Scopus from 2009 to 2018 and were employed in Polish higher education system in November 2017; consequently, non-publishing (and non-publishing internationally) scientists were not included. However, the percentage of scientists in STEMM disciplines who published internationally was high; moreover, it increased over time, and it was much higher than in non-STEMM disciplines (Kwiek, 2020 ).

Second, this research combined (near perfect) administrative and biographical data collected from a national registry of scientists with (much less perfect) bibliometric data at the individual level. Therefore, we combined data on “real individuals” with national identification numbers with metadata on publications by individual Scopus Author IDs rather than “real scientists.” Our Observatory of Polish Science was constructed through a deterministic and probabilistic record linkage between two original data sets that differed in nature. For the past two decades, it has been widely debated to what extent bibliometric data are biased linguistically, geographically, and disciplinarily (Boekhout et al., 2021 ). However, sources other than raw Scopus (or the raw Web of Science Core Collection) datasets could not be used to construct full publication histories of all scientists within a whole national science system. No other source of publication metadata has been available about Polish scientists from the past 50 years. Finally, our study shows a “success bias”: its sample includes only full professors i.e. those who got to the top of academic hierarchies.

Mobility between productivity classes from a lifetime career perspective

In this subsection, we examine the persistence of productivity classes of full professors from a lifetime career perspective: Have current top-performing full professors always been top-performing? And have current low-performing full professors always been low performing?

Figure  3 shows the lifetime career trajectories of 2326 full professors in 14 STEMM disciplines combined (total). Their productivity was classified as top, middle, or bottom (20%, 60%, or 20%, respectively) in three periods: between becoming assistant professors and becoming associate professors (left column); between becoming associate professors and becoming full professors (middle column); and after becoming full professors (right column). Our focus was on the mobility of top productivity classes and bottom productivity classes in the three stages of an academic career. The results are shown in Sankey diagrams.

Sankey diagram of retrospectively constructed mobility between productivity classes in the three stages of an academic career. All STEMM disciplines (total) are combined, and only current full professors are shown. Top (upper 20%), middle (middle 60%), and bottom (lower 20%) productivity classes are shown in percentages of 100% (or rounded) in each of the three classes. The bottom class in the left column is larger than 20%, and the middle class is smaller than 60%; the cutting-off points did not permit a different division into classes. N  = 2326

The majority of highly productive scientists (top) remained highly productive compared with their peers in the same discipline and within the same academic position, which is shown in thick left-to-right horizontal flows (as shown in Fig.  3 ). More than half of the highly productive scientists moved from the top class to the top class in the first (52.6%) and second stages of their academic careers (50.8%). Only about 2.3% moved to the low-productivity class in the first period, and only about 5% moved to the low-productivity class in the second period. These exceptional cases of top-to-bottom mobility in productivity classes are shown as thin descending flows from the top classes to the bottom classes (Fig.  3 ). The mobility from the bottom productivity classes to the top productivity classes in the first and the second periods was limited. In Fig.  3 , upward mobility is shown as thin ascending flows from the bottom classes to the top classes: 8.5% and 2.9%, respectively. Extreme mobility between productivity classes (top-to-bottom and bottom-to-top) was characteristic of only 100 scientists of 2326.

The Sankey diagrams also show the ongoing mobility between middle-performing classes (Middle) and top-performing classes (top). Although the majority of professors assigned to the middle-performing class remained in the same class, some moved up, and some moved down. The data on possible combinations of mobility in this case are shown in Table  3 : the first panel shows the data on mobility from assistant professors to associate professors, the second panel shows mobility from associate professors to full professors, and the third panel describes the subsample used (all special cases can be identified at an individual level, and further discussed).

Mobility between productivity classes differed substantially between disciplines. We examined in detail the disciplines with the largest number of full professors (i.e., MED) and a discipline in which the patterns of top-to-top and bottom-to-bottom mobility were the most stable from a comparative cross-disciplinary perspective (i.e., MATH). MATH has been frequently studied because of its unique features, such as a low collaboration rate and a low share of female scientists (e.g., Mihaljević-Brandt et al., 2016 ).

The MED case (Fig.  4 ) presented a clear pattern of productivity class mobility: its top-to-top and bottom-to-bottom mobility was high, and its top-to-bottom and bottom-to-top mobility was limited over entire academic careers. More than half of highly productive assistant professors (top) became highly productive associate professors (top); and more than half of low-productive assistant professors (bottom) became low-productive associate professors (bottom) (55.1% and 50.6%, respectively; see thick flows in Fig.  4 ). The mobility pattern was similar for the two stages of academic careers. The majority of highly productive associate professors (top) became highly productive full professors (top), and almost half of the low-productive associate professors (bottom) became low-productive full professors (bottom; 50.6% and 46.1%, respectively). Extreme productivity class transitions (top-to-bottom and bottom-to-top) were rare, which is shown by very thin flows linking top and bottom productivity classes in both periods of their academic careers. Extreme transitions were experienced by 3.8% (downward) and 3.9% (upward) of assistant professors and by 5.2% (downward) and 1.3% (upward) of associate professors.

Sankey diagram of retrospectively constructed mobility between productivity classes in the three stages of academic careers. MED and current full professors only. N  = 379

In the MATH case (Fig.  5 ), the persistence of highly productive assistants and associate professors was very high. Two-thirds of scientists in the top productivity classes remained in these classes: 69% of highly productive assistant professors continued to be highly productive associate professors, and 65.5% of highly productive associate professors continued to be highly productive full professors. The likelihood that low-productive associate professors would enter the class of highly productive full professors was slim (3.4%).

Sankey diagram of retrospectively constructed mobility between productivity classes in the three stages of academic careers. MATH and current full professors only. N  = 142

Overall, cross-disciplinary differences were substantial. An aggregated picture holding for all disciplines combined hides behind it much more nuanced discipline-specific pictures. Disciplines were characterized by different intensities of upward and downward mobility (Fig.  6 ). In some disciplines, no highly productive assistant professor dropped to the bottom productivity class (e.g., CHEM chemistry, CHEMENG chemical engineering, COMP computer science, EARTH earth and planetary sciences, ENER energy, MATER materials science, and MATH mathematics). Upward mobility from a bottom class to a top class was nonexistent for associate professors in CHEMENG chemical engineering, EARTH earth and planetary sciences, ENVIR environment, and PHYS physics and astronomy. In other disciplines, no highly productive assistant professor and no highly productive associate professor dropped to the bottom productivity class, and upward mobility to a top class was nonexistent for associate professors (e.g., computer science [COMP] and earth and planetary sciences [EARTH]). In other disciplines, while no top-to-bottom mobility in productivity classes was observed, bottom-to-top mobility was notable (e.g., energy [ENER] and physics and astronomy [PHYS]). Finally, the biggest variability, as expected, was noted for the smallest disciplines, as the case of PHARM pharmacology, toxicology, and pharmaceutics clearly shows. Moreover, the results showed variations by gender within disciplines in which higher proportions of women than men remained in the top productivity classes, as shown in Tables 4 and 5 .

Overview: Sankey diagrams of retrospectively constructed mobility between productivity classes in the three stages of academic careers. Eleven STEMM disciplines and all disciplines combined (total), current full professors only

Figure  6 shows transitions in all disciplines not described above. The stability of top-performing classes was high and ranged from 34.4–69.0% for assistant professors who became associate professors and 20–65.5% for associate professors who became full professors. The share exceeded 50% in most disciplines in the first case and in half of the disciplines in the second case. Further details on mobility are shown in Table 4 .

We also conducted a comparison of productivity classes in the first and last stages of the academic career (Table 5 ): assistant professor and full professor. Almost half of the current highly productive full professors had been highly productive assistant professors 20–40 years earlier (46.8%). However, the results showed an interesting gender disparity: the percentage of female scientists who continued to be highly productive throughout their careers was considerably higher than the percentage of male scientists who continued to be highly productive throughout their careers (48.1% vs. 42.5%) (Table 5 ). Cross-disciplinary and gender differences were substantial: for instance, all (100%) highly productive male full professors were highly productive assistant professors 20–40 years earlier in the two most male-dominated disciplines, MATH and PHYS (compared with females at 46.4% and 44.4%, respectively). The principle “once highly productive, forever highly productive” held for all cases of Polish male mathematicians, physicists, and astronomers (current full professors). Footnote 1

Logistic regression models

This subsection presents the odds ratio estimates of belonging to top productivity classes for current full professors and, retrospectively, for current full professors at earlier stages of their academic careers (in the same disciplines) ( N  = 2326). The individual-level variables included gender, biological age, academic age (the number of years since the first publication, see Kwiek and Roszka, 2022b ), and the biological ages at which the doctorate, habilitation (or postdoctoral degree), and full professorship were awarded. Most importantly in the context of the two-dimensional analyses presented in the section “Mobility between productivity classes from a lifetime career perspective,” the individual-level variables also included classifications from our general classificatory scheme (Fig.  2 ): membership in current and past productivity classes, promotion-age classes, and promotion-speed classes (with the 20/60/20 divisions in each case). The only organization-level variable was the research intensity of the employing institution (IDUB vs. other institutions); other variables were tested (e.g., research budget, total budget, total number of scientists) but proved significantly correlated with the IDUB variable.

Crucially, the results of the logistic regression (Table 6 ) powerfully augment the result yielded by the descriptive statistics: being among the most productive full professors is dependent on having been in analogous groups of highly productive scientists at earlier stages of one’s academic career. Thus, belonging to the class of highly productive assistant professors increased the probability of becoming a highly productive full professor by, on average, from two up to almost four times (Exp(B) = 2.8; 95% confidence interval 2.1–3.6), while belonging to the class of highly productive associate professors increased the probability of success by, on average, from almost four to almost six times (Exp(B) = 4.61; 95% confidence interval 3.6–6). The only significant predictor indirectly related to age was belonging to the youngest 20% of full professors in terms of promotion age. Membership in this class increased the probability of success by, on average, almost twice (Exp(B) = 1.942; see the variables Top_assistant_prof_class, Top_associate_prof_class, and Young_full_prof_class).

Similarly, among current full professors when they were associate professors (Model 2), belonging to the class of highly productive assistant professors increased the probability of becoming a highly productive associate professor by, on average, from almost five to more than nine times (Exp(B) = 6.667; 95% confidence interval 4.7–9.4). The important determinants of membership in the top 20% of productive scientists were related to age, both biological and academic. Biological age had a negative effect, and it had a significantly stronger negative effect on associate professors than on assistant professors. An increase in biological age by one year reduced the probability of entering the class of highly productive assistant professors by 20–25%. Among associate professors, this 1-year increase reduced the likelihood by up to one-third – one-fourth. Among assistant professors, a 1-year increase in academic age (and thus publication experience or the number of years since first publication) resulted in an average increase of 10–15% in the probability of success, while, among associate professors, the average increase was only 0.2–4.1%.

Another age-related variable that significantly affected the probability of success was the promotion age of assistant professors. Among associate professors, an increase in doctoral promotion age (variable: Assistant_prof_promotion_age) had a negative effect, decreasing the probability of success by an average of 5.8% (with 95% confidence interval 0.5–10.8%), while, among assistant professors, the direction of change was positive and high; a 1-year increase in doctoral promotion age increased the probability of success by an average of 20.7% (14–27%). The age at promotion to a postdoctoral degree (variable: Associate_prof_promotion_age) significantly and strongly influenced the probability of success; a 1-year increase in the age at promotion increased the likelihood of entering the group of the 20% most productive associate professors by about half (on average, 47.5%; 40–55%). This variable could not be included in the model for assistant professors because they had not yet been promoted to this stage, having not yet earned their postdoctoral degrees. However, one variable (indirectly) related to age that was important to the likelihood of being among the 20% of most productive assistant professors was being among the 20% of the youngest scientists promoted to doctoral degrees. Membership in this group increased the probability of success by an average of 73.9% (although the confidence interval in this case was quite wide: 23.2–145.5%). Gender had a significant impact only among associate professors. Being male increased the probability of success by an average of 42.6%, but the range of the confidence interval (3%–97%) suggests that the significance of this predictor should be interpreted with caution (the role of gender differences, see Kwiek and Roszka, 2021a , b ; Kwiek and Roszka, 2022a ).

In summary, for current full professors, the most powerful predictors of belonging to the class of highly productive scientists are having belonged to that class while working as assistant professors and as associate professors; a third powerful predictor is membership in the class of full professors promoted early in their careers. Retrospectively, for current full professors in their past as associate professors, the single most powerful predictor is having belonged to the class of highly productive assistant professors; other predictors include belonging to the class of associate professors promoted early (Exp(B) = 1.475) and, possibly, being a male (Exp(B) = 1.426). Finally, also retrospectively for assistant professors, the single most powerful predictor of belonging to the class of highly productive assistant professors is belonging to the class of assistant professors promoted early (Exp(B) = 1.739).

Discussion and conclusions

Highly productive scientists have often been examined as a special academic class: as “eminent” and “highly prolific” scientists and as “stars,” “top scientists,” and “top performers” (Fox & Nikivincze, 2021 ; Agrawal et al., 2017 ; Kwiek, 2016 ; Cortés et al., 2016 ; Abramo et al., 2009 ). They are “motivated by an inner drive to do science and by a sheer love of the work” (Cole & Cole, 1973 : 62), and, while some scientists are particularly good at doing science, “some are not just good but superb” (Stephan & Levin, 1992 : 13). In that vein, some full professors in our sample were simply superb at doing science from the moment they entered academia through their late-career stages. About half the highly productive full professors had always been highly productive, regardless of the trajectories of their personal lives or their external circumstances (e.g., the post-communist transition period in the Polish economy, which severely affected the academic sector). Highly productive full professors in their 60s were also highly productive when they were assistant and associate professors in their 30s, 40s, and 50s.

The message of our regression analysis is simple: past productivity classes (i.e., publication history) powerfully determine current productivity classes, with much smaller roles played by the other predictors. Our regression models strongly support the results of our two-dimensional analyses, according to which scientists who have once been highly productive tend to remain highly productive and those who have once had poor productivity have little chance of moving to the high productivity classes (shown as thin upward flows between the bottom and top productivity classes across all disciplines) (Fig.  3 ).

There are only two powerful predictors of high productivity among full professors: membership in the class of highly productive assistant professors and membership in the class of highly productive associate professors, which increase the odds by, on average, almost three and five times, respectively (by 179% and 361%). The most powerful predictor of becoming a highly productive associate professor (in the sample of current full professors) was being a highly productive assistant professor as shown by the staggering increase in odds: almost seven times (or by 570%). For highly productive assistant professors, the most powerful predictor was obtaining a PhD early in their careers. Additionally, our results support previous findings that full professors appointed early tend to be more productive than full professors appointed later in their careers (Abramo et al., 2016 ). Membership in the class of young full professors increased the odds of belonging to the class of highly productive full professors by an average of 94.2%. Neither gender nor age (biological or academic) emerged as a predictor of membership in the class of highly productive full professors. The results did not directly support the claim that the productivity of top- and medium-performing scientists increases or remains stable with age (Costas et al., 2010 : 1578), as our study focused on changing productivity classes rather than on evolving productivity over time.

The results of our study revealed an unexpectedly high level of immobility in the system. Membership in the productivity class during assistant professorships and associate professorships, to a large extent, determined membership in the productivity class during full professorship and beyond. Does the “once highly productive, forever highly productive” principle hold across all STEMM disciplines? The results of this study indicate the affirmative. About half of the current full professors belonged to the same productivity class throughout their academic careers. They had remained for decades in the bottom or top productivity classes in relation to their peers and within their specific disciplines. About half of the current full professors had changed their productivity class membership by only one class in a tripartite division into top, middle, or bottom classes, with some discipline and gender differentiation. Cross-disciplinary and gender differences were substantial: for instance, all highly productive male full professors (100%) were highly productive assistant professors 20–40 years earlier in the two most male-dominated disciplines, MATH and PHYS. So the principle held for all cases of Polish male mathematicians, physicists, and astronomers.

More than half of the highly productive assistant professors became, on average, highly productive associate professors in relation to their peers in a similar period, the same academic position, and the same discipline. More than half of the highly productive associate professors became, on average, highly productive full professors (52.6% and 50.8%, respectively). Moreover, a study of direct start-to-end mobility shows that, on average, almost half of the highly productive assistant professors became highly productive full professors. They did not change their productivity class membership to a lower class throughout their academic careers (46.8%), with a large differentiation among disciplines. Similar processes of transition in productivity class membership included low-productive scientists.

The most radical changes in productivity class membership, that is, transitions from the very top to the very bottom of productivity, occurred at a marginal level; upward bottom-to-top transfers occurred on a similar small scale. In our sample, the 2326 full professors in the last four decades included 35 scientists who had radically changed their productivity classes downward and 65 who had moved upward (so, in total, only 4.3% of current full professors). Above-average mobility was observed in the disciplines of BIO, MATH, and PHYS, while the least mobility was observed in PHARM.

Perhaps the most interesting question is why the pattern of “once highly productive, forever highly productive” is so pervasive in Polish higher education. Among several possible explanations, one follows the lines of two traditional theories of productivity, sacred spark theory and cumulative advantage theory. The former holds that there is a small group of scientists who will always be superb in their achievements, as they have a spark that others lack, being inherently highly motivated, well organized, creative, and skillful. The latter theory identifies a group of scientists who, with or without that spark, keep accumulating advantages from the very beginning of their careers. Their advantages come from their socialization to internationalized work environments, specific work cultures, and work habits available mostly in elite institutions or departments; from doctoral advisors who were role models; and from resources available through research funding, including long-term international fellowships. The cumulative advantage theory explains high productivity by a set of reinforcing factors that, combined, continually push academic careers forward (with ever better access to resources of all kinds: research time, infrastructure, funding, international networks, publications in prestigious journals, externally funded doctoral and postdoctoral researchers, etc.).

Another useful theoretical line of explanation is the credibility cycle in academic careers (Latour & Woolgar, 1986 : 200–208), in which prestigious papers are converted into recognition that leads to successful individual grant applications that are converted into new equipment, data, software, arguments, and articles. Perhaps the credibility cycle is faster for scientists affected by this mechanism at an early stage in their careers: once funded, with excellent publications, they have better chances to be funded again and to be promoted sooner to higher ranks, reflecting the idea that each element of the credibility cycle in academic careers “is but one part of an endless cycle of investment and conversion” (Latour & Woolgar, 1986 : 200). Advantages already gained lead more quickly to future advantages, as in any positional competition having the nature of a zero-sum game: “what winners win, losers lose” (Hirsch, 1976 : 52). The above theoretical mechanisms have more powerful effects in resource-poor systems such as Poland’s, in which, historically, funding could be won or lost by a small margin due to the scarcity of public funding.

The patterns of mobility between productivity classes over the course of an entire academic career in national academic science systems may have far-reaching implications for science policies, especially regarding hiring and promotion. Hiring and tenure to both low-productivity and high-productivity scientists may have long-standing consequences for institutions and the national system in terms of the average productivity level. Research careers are usually long. After entering the system and achieving job stability, scientists in Poland (where attrition is very low) and elsewhere usually remain in the system for years, if not decades (see especially Abramo et al., 2017 discussing star scientists and unproductive scientists in Italy). The scientists included in this study, all of whom are currently full professors in the 14 STEMM disciplines present in the Scopus bibliometric database, have remained in the system for 20–40 years. Individual hiring and promotion decisions made at the departmental level thus have long-lasting implications for productivity at the national level, spanning two to four decades.

Our results may also imply the need to cultivate productivity, especially among young academics: entering productivity elites early on substantially increases the chances of belonging to productivity elites in later career stages. The importance of cultivating productivity goes beyond research-intensive universities and pertains to the whole higher education sector. Understanding persistent inequality in productivity matters especially in resource-poor systems in which research funding is highly competitive. Scientists in STEMM disciplines tend to be powerfully locked-in early on in their careers in their productivity classess and the chances of changing them radically from a long-term longitudinal perspective—becoming much more productive compared with their peers—are slim. It would be interesting to see whether similar mechanisms operate within social sciences and humanities; however, the character of the Scopus data (limited coverage for social sciences and humanities in the Polish case) does not allow us to go beyond STEMM disciplines in our research.

The results of our study indicate the opportunities provided by structured Big Data (in this case, the Scopus raw dataset). We examined all current Polish full professors in STEMM, but the data we used were collected from two large datasets. One was the Observatory of Polish Science, which included full biographical and administrative data on almost 100,000 Polish scientists and their 380,000 publications in Scopus from 2009–2018. The second dataset comprised Scopus metadata on almost a million (935,167) Polish publications in the past 50 years. The merger of several datasets made it possible to create not only current productivity classes to which all professors were allocated – but also retrospective productivity classes. Importantly, every full professor was compared in terms of research productivity as an assistant and associate professor with their exact peers (current full professors) when they were at the same earlier stages of academic careers in the same discipline. We retrospectively examined their academic careers as extensively as necessary to compare “apples with apples” rather than “apples with oranges” at all three stages of their academic careers.

Finally and more generally, structured Big Data offer fundamentally new opportunities to examine the academic profession, both nationally, cross-nationally, and globally. The Big Data collected and stored by others (e.g., governments and corporations) for other than academic purposes can be analyzed by students of the academic profession as a new, complementary data source to complement traditional sources, such as academic surveys and interviews. This could provide a better balance between small-scale (low-N) and large-scale (big-N) studies, with a fertilizing effect on the field (for an overview of the field, see Carvalho, 2017 ). The key word is complementarity : new data sources complement, rather than replace, traditional sources.

The new data must be repurposed (Salganik, 2018 ), and they come with their own limitations and biases, but the amount of data available and their longitudinal character (enabling the analysis of changes in academic careers over time) offer great promise. From datasets that are vast in both size and complexity, we can extract only the useful current and past information about academics and their output. We can examine huge amounts of data to discover patterns that would otherwise be imperceptible, looking at outliers, deviations, and special cases and performing analyses based on unprecedented numbers of observations. While Big Data dramatically deepen our insight into society generally (Selwyn, 2019 ), specific parts of structured, curated, and reliable Big Data (such as commercial bibliometric datasets) can radically sharpen our insights into the academic profession, allowing it to be examined with the use of new temporal (time), topical (themes), geographical (places), and network (connections) analyses (see Börner, 2010 : 62–63). The various dimensions of academic work can be studied with ever more precision and a remarkable level of detail.

The use of curated, large-scale data sources allows to study the academic profession over years, across countries (institutions, cities), across academic disciplines, at different levels of granularity and in terms of research teams and individuals, male and female scientists, and junior and senior scientists. The small observation numbers yielded by traditional surveys of the academic profession limit the analytical power of the datasets and weaken the ability to draw policy implications from the research. Small-scale studies are useful and theoretically inspiring but, in the current world, they may not convince policy makers and grant-making agencies. Several factors increase the pressure to study the academic profession using Big Data: first, the increasing availability of digital data on scholarly inputs and outputs at an individual level (funding, publications, collaboration, mobility); second, the growing availability of computing power to analyze the data; finally, the pressure to provide both the public and the scholarly community with a more quantified, data-based, sound, and convincing understanding of changes in higher education in general and in the academic profession in particular.

Data Availability

We used data from Scopus, a proprietary scientometric database. For legal reasons, data from Scopus received through collaboration with the International Center for the Studies of Research (ICSR) Lab cannot be made openly available.

Z -test statistics for the three stages of academic careers show that for both transitions between assistant professors, associate professors and full professors, as well as for direct transitions from assistant professorship to full professorship, in most domains no differences in percentages were observed for men and women. Exceptions included top associate professor to top full professor transitions, where a significant difference was observed for AGRI ( z  =  − 2.786, p- value = 0.008, Cohen’s d  = 0.333) and for all domains ( z  =  − 3.955, p -value < 0.001, Cohen’s d  = 0.182). A significant difference was observed for ENVIR for the transition from bottom assistant professors to bottom full professors ( z  =  − 2.384, p -value = 0.023, Cohen’s d  = 0.341). In all above cases, a significantly higher percentage was observed for women. The effect size of the differences is rather moderate.

Abramo, G., D’Angelo, C. A., & Caprasecca, A. (2009). The contribution of star scientists to overall sex differences in research productivity. Scientometrics, 81 (1), 137–156.

Article   Google Scholar  

Abramo, G., D’Angelo, C. A., & Di Costa, F. (2011). Research productivity: Are higher academic ranks more productive than lower ones? Scientometrics, 88 (3), 915–928.

Abramo, G., D’Angelo, C. A., & Murgia, G. (2016). The combined effects of age and seniority on research performance of full professors. Science and Public Policy, 43 (3), 301–319.

Abramo, G., D’Angelo, C. A., & Soldatenkova, A. (2017). How long do top scientists maintain their stardom? An analysis by region, gender and discipline: Evidence from Italy. Scientometrics, 110 , 867–877.

Agrawal, A., McHale, J., & Oettl, A. (2017). How stars matter: Recruiting and peer effects in evolutionary biology. Research Policy, 46 (4), 853–867.

Aguinis, H., & O’Boyle, E. (2014). Star performers in twenty-first century organizations. Personnel Psychology, 67 (2), 313–350.

Aksnes, D. W., Rørstad, K., Piro, F. N., & Sivertsen, G. (2011). Are female researchers less cited? A large-scale study of Norwegian researchers. Journal of the American Society for Information Science and Technology, 62 (4), 628–636.

Allison, P. D., & Stewart, J. A. (1974). Productivity differences among scientists: Evidence for accumulative advantage. American Sociological Review, 39 (4), 596–606.

Boekhout, H., van der Weijden, I., & Waltman, L. (2021). Gender differences in scientific careers: A large-scale bibliometric analysis.  https://arxiv.org/abs/2106.12624 . Accessed 10 Jan 2023.

Börner, K. (2010). Atlas of Science . The MIT Press.

Google Scholar  

Carvalho, T. (2017). The study of the academic profession—Contributions from and to the sociology of professions. In J. Huisman & M. Tight (Eds.), Theory and method in higher education research (pp. 59–76). Emerald.

Chapter   Google Scholar  

Cole, J. R., & Cole, S. (1973). Social stratification in science . University of Chicago Press.

Cortés, L. M., Mora-Valencia, A., & Perote, J. (2016). The productivity of top researchers: A semi-nonparametric approach. Scientometrics, 109 (2), 891–915.

Costas, R., van Leeuwen, T. N., & Bordons, M. (2010). Self-citations at the meso and individual levels: Effects of different calculation methods. Scientometrics, 82 , 517–537.

Article   CAS   PubMed   PubMed Central   Google Scholar  

DiPrete, T. A., & Eirich, G. M. (2006). Cumulative advantage as a mechanism for inequality: A review of theoretical and empirical developments. Annual Review of Sociology, 32 (1), 271–297.

Fox, M. F. (1983). Publication productivity among scientists: A critical review. Social Studies of Science, 13 (2), 285–305.

Fox, M. F. (2020). Gender, science, and academic rank: Key issues and approaches. Quantitative Science Studies, 1 (3), 1001–1006.

Fox, M. F., & Mohapatra, S. (2007). Social-organizational characteristics of work and publication productivity among academic scientists in doctoral-granting departments. Journal of Higher Education, 78 (5), 542–571.

Fox, M. F., & Nikivincze, I. (2021). Being highly prolific in academic science: Characteristics of individuals and their departments. Higher Education, 81 , 1237–1255.

GUS. (2022). Higher education and its finances in 2021. Main Statistical Office of Poland.

Habicht, I. M., Lutter, M., & Schröder. (2022). Gender differences in the determinants of becoming a professor in Germany. An event history analysis of academic psychologists from 1980 to 2019. Research Policy, 51 (6), 104506.

Heckman, J. J., & Moktan, S. (2018). Publishing and promotion in economics. The tyranny of the top five . NBER (Working Paper 25093).

Hermanowicz, J. (2012). The sociology of academic careers: Problems and prospects. In J. C. Smart & M. B. Paulsen (Eds.), Higher education: Handbook of theory and research 27 (pp. 207–248)

Hirsch, F. (1976). Social limits to growth . Harvard University Press.

Book   Google Scholar  

Katz, D. A. (1973). Faculty salaries, promotions, and productivity at a large university. American Economic Review, 63 (3), 469–477.

Kolesnikov, S., Fukumoto, E., & Bozeman, B. (2018). Researchers’ risk-smoothing publication strategies: Is productivity the enemy of impact? Scientometrics, 116 (3), 1995–2017.

Article   CAS   Google Scholar  

Kyvik, S. (1990). Age and scientific productivity: Differences between fields of learning. Higher Education, 19 , 37–55.

Kwiek, M. (2015). Academic generations and academic work: Patterns of attitudes, behaviors and research productivity of Polish academics after 1989. Studies in Higher Education , 40 (8), 1354–1376.

Kwiek, M. (2016). The European research elite: A cross-national study of highly productive academics across 11 European systems. Higher Education , 71 (3), 379–397.

Kwiek, M. (2018). High research productivity in vertically undifferentiated higher education systems: Who are the top performers? Scientometrics , 115 (1), 415–462.

Kwiek, M. (2019). Changing European academics. In A comparative study of social stratification, work patterns and research productivity . London and New York: Routledge.

Kwiek, M. (2020). Internationalists and locals: International research collaboration in a resource-poor system. Scientometrics , 124 , 57–105.

Kwiek, M., & Roszka, W. (2021a). Gender disparities in international research collaboration: A large-scale bibliometric study of 25,000 university professors. Journal of Economic Surveys , 35 (5), 1344–1388.

Kwiek, M., Roszka, W. (2021b). Gender-based homophily in research: A large-scale study of man-woman collaboration. Journal of Informetrics , 15 (3), 1–38.

Kwiek, M., Roszka, W. (2022a). Are female scientists less inclined to publish alone? The gender solo research gap. Scientometrics , 127 , 1697–1735.

Kwiek, M., Roszka, W. (2022b). Academic vs. biological age in research on academic careers: a large-scale study with implications for scientifically developing systems. Scientometrics , 127 , 3543–3575.

Latour, B., & Woolgar, S. (1986). Laboratory Life . Princeton University Press.

Lerchenmueller, M. J., & Sorenson, O. (2018). The gender gap in early career transitions in the life sciences. Research Policy, 47 (6), 1007–1017.

Lutter, M., & Schröder, M. (2016). Who becomes a tenured professor, and why? Panel data evidence from German sociology, 1980–2013. Research Policy, 45 (5), 999–1013.

Madison, G., & Fahlman, P. (2020). Sex differences in the number of scientific publications and citations when attaining the rank of professor in Sweden. Studies in Higher Education, 46 (12), 2506–2527.

Marginson, S. (2014). University research: The social contribution of university research. In J. C. Shin & U. Teichler (Eds.), The future of the post-massified university at the crossroads (pp. 101–118). Springer International Publishing.

Marini, G., & Meschitti, V. (2018). The trench warfare of gender discrimination: Evidence from academic promotions to full professor in Italy. Scientometrics, 115 (2), 989–1006.

Merton, R. K. (1973). The sociology of science: Theoretical and empirical investigations . University of Chicago Press.

Mihaljević-Brandt, H., Santamaría, L., & Tullney, M. (2016). The effect of gender in the publication patterns in mathematics. PLoS ONE, 11 (10), e0165367.

Article   PubMed   PubMed Central   Google Scholar  

Nygaard, L. P., Aksnes, D. W., & Piro, F. N. (2022). Identifying gender disparities in research performance: The importance of comparing apples with apples. Higher Education, 84 , 1127–1142.

Piro, F. N., Rørstad, K., & Aksnes, D. W. (2016). How do prolific professors influence the citation impact of their university departments? Scientometrics, 107 (3), 941–961.

Puuska, H.-M. (2010). Effects of scholar’s gender and professional position on publishing productivity in different publication types: Analysis of a Finnish university. Scientometrics, 82 (2), 419–437.

Ruiz-Castillo, J., & Costas, R. (2014). The skewness of scientific productivity. Journal of Informetrics, 8 (4), 917–934.

Salganik, M. J. (2018). Bit by bit. Social research in a digital age. Princeton University Press.

Selwyn, N. (2019). What is digital sociology? Polity Press.

Stephan, P. (2012). How economics shapes science . Harvard University Press.

Stephan, P. E., & Levin, S. G. (1992). Striking the mother lode in science: The importance of age, place, and time . Oxford University Press.

Sugimoto, C., & Larivière, V. (2018). Measuring research: What everyone needs to know . Oxford University Press.

Taylor, B. J., & Cantwell, B. (2019). Unequal higher education: Wealth, status, and student opportunity . Rutgers University Press.

Weinberger, M., & Zhitomirsky-Geffet, M. (2021). Diversity of success: Measuring the scholarly performance diversity of tenured professors in Israeli academia. Scientometrics, 126 , 2931–2970.

Xie, Y. (2014). ‘Undemocracy’: Inequalities in science. Science, 344 (6186), 809–810.

Article   ADS   CAS   PubMed   PubMed Central   Google Scholar  

Yair, G., Gueta, N., & Davidovitch, N. (2017). The law of limited excellence: Publication productivity of Israel Prize laureates in the life and exact sciences. Scientometrics, 113 (1), 299–311.

Yin, Z., & Zhi, Q. (2017). Dancing with the academic elite: A promotion or hindrance of research production? Scientometrics, 110 (1), 17–41.

Yuret, T. (2018). Path to success: An analysis of US educated elite academics in the United States. Scientometrics, 117 , 105–121.

Zuckerman, H. (1988). The sociology of science. In N. J. Smelser (Ed.), Handbook of sociology (pp. 511–574). Sage.

Download references

Acknowledgements

Marek Kwiek is grateful for the comments from the hosts and audiences of seminars at the University of Oxford (CGHE, Center for Global Higher Education), Stanford University (METRICS, Meta-Research Innovation Center at Stanford), both in June 2022, and DZHW (German Center for Higher Education Research and Science Studies), Berlin, in January 2023. We gratefully acknowledge the assistance of the International Center for the Studies of Research (ICSR) Lab and Kristy James, Senior Data Scientist. We also want to thank Lukasz Szymula from the CPPS Poznan Team for improving the visualizations. We gratefully acknowledge the support provided by the NDS grant no. NdS/529032/2021/2021.

Author information

Authors and affiliations.

Institute for Advanced Studies in Social Sciences and Humanities, Adam Mickiewicz University of Poznan, Poznan, Poland

Marek Kwiek

Center for Public Policy Studies, Adam Mickiewicz University, Poznan, Poland

Marek Kwiek & Wojciech Roszka

German Centre for Higher Education Research and Science Studies (DZHW), Berlin, Germany

Poznan University of Economics and Business, Poznan, Poland

Wojciech Roszka

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Marek Kwiek .

Ethics declarations

Conflict of interest.

The authors declare no competing interests.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Kwiek, M., Roszka, W. Once highly productive, forever highly productive? Full professors’ research productivity from a longitudinal perspective. High Educ 87 , 519–549 (2024). https://doi.org/10.1007/s10734-023-01022-y

Download citation

Accepted : 07 March 2023

Published : 24 March 2023

Issue Date : March 2024

DOI : https://doi.org/10.1007/s10734-023-01022-y

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Research productivity
• Academic profession
• Academic career
• Longitudinal study
• Full professors
• Find a journal
• Publish with us
• Track your research

• Open access
• Published: 28 August 2018

A systematic review on academic research productivity of postgraduate students in low- and middle-income countries

• E. A. Obuku   ORCID: orcid.org/0000-0002-2728-4942 1 , 8 , 9 ,
• J. N. Lavis 2 , 6 ,
• A. Kinengyere 5 ,
• R. Ssenono 8 ,
• M. Ocan 8 ,
• D. K. Mafigiri 4 , 7 ,
• F. Ssengooba 3 ,
• C. Karamagi 1 &
• N. K. Sewankambo 1  

Health Research Policy and Systems volume  16 , Article number:  86 ( 2018 ) Cite this article

8490 Accesses

12 Citations

11 Altmetric

Metrics details

While several individual studies addressing research productivity of post-graduate students are available, a synthesis of effective strategies to increase productivity and the determinants of productivity in low-income countries has not been undertaken. Further, whether or not this research from post-graduate students’ projects was applied in evidence-informed decision-making was unknown. Therefore, we conducted a systematic review of literature to identify and assess the effectiveness of approaches that increase productivity (proportion published) or the application (proportion cited) of post-graduate students’ research, as well as to assess the determinants of post-graduate students’ research productivity and use.

We conducted a systematic review as per our a priori published protocol, also registered in PROSPERO (CRD42016042819). We searched for published articles in PubMed/MEDLINE and the ERIC databases through to July 2017. We performed duplicate assessments for included primary studies and resolved discrepancies by consensus. Thereafter, we completed a structured narrative synthesis and, for a subset of studies, we performed a meta-analysis of the findings using both fixed and random effects approaches. We aligned our results to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement.

We found 5080 articles in the PubMed ( n  = 3848) and ERIC ( n  = 1232) databases. After excluding duplicates ( n  = 33), we screened 5047 articles, of which 5012 were excluded. We then retrieved 44 full texts and synthesised 14, of which 4 had a high risk of bias. We did not find any studies assessing effectiveness of strategies for increasing publication nor citations of post-graduate research projects. We found an average publication proportion of 7% (95% CI 7–8%, Higgins I-squared 0.0% and Cochran’s Q p < 0.01) and 23% (95% CI 17–29%, Higgins I-squared of 98.4% and Cochran’s Q, p < 0.01) using fixed effects and random effects models, respectively. Two studies reported on the citation of post-graduate students’ studies, at 17% (95% CI 15–19%) in Uganda and a median citation of 1 study in Turkey (IQR 0.6–2.3). Only one included study reported on the determinants of productivity or use of post-graduate students’ research, suggesting that younger students were more likely to publish and cohort studies were more likely to be published.

Conclusions

We report on the low productivity of post-graduate students’ research in low- and middle-income countries, including the citation of post-graduate students’ research in evidence-informed health policy in low- and middle-income countries. Secondly, we did not find a single study that assessed strategies to increase productivity and use of post-graduate students’ research in evidence-informed health policy, a subject for future research.

Peer Review reports

Writing a thesis is a fundamental step for post-graduate studies globally [ 1 ]. This process inculcates knowledge and skills for scientific enquiry, critical thinking, systematic problem-solving, and appraisal of scientific and lay claims. This if often followed by dissemination of the thesis results to the scientific community, of which publication in a peer-reviewed journal is the highest and most respected form [ 2 ]. However, how much of this thesis work appears only in theses, and its citation particularly in public policy-related work, in low- and middle-income countries is the subject of our systematic review.

Our first aim was to conduct a systematic review of literature that identifies and assesses the effectiveness of approaches that increase productivity (proportion published) or the application (proportion cited) of post-graduate students’ research. Our second aim was to assess the determinants of post-graduate students’ research productivity and use.

We registered our protocol a priori in PROSPERO (CRD42016042819) and thereafter published it in a peer-reviewed journal [ 3 ]. We thus present an overview of the methodological approach and highlight differences between the protocol and the actual conduct of this systematic review.

Search strategy

Electronic search.

We report the electronic search for the PubMed/Medline database only ( https://www.ncbi.nlm.nih.gov/pubmed/ ). We combined terms using Boolean logic ‘OR’ for synonyms and ‘AND’ across elements of PICOS (Population Intervention Comparison Outcome and Study design), as follows: ‘medicine’, ‘nursing’, ‘dentistry’, ‘pharmacy’ and ‘public health’ described the professional fields of interest; while ‘degree’, ‘doctor’, ‘post-doctor’, ‘master’, ‘fellow’, ‘resident’, ‘student’, ‘trainee’, ‘graduate’ and ‘post-graduate’ were intermediate transitional terms for the population of interest, setting or interventions. The terms describing the interventions of interest included were ‘mentor’, ‘grant’, ‘fund’, ‘supervise’, ‘workshop’, ‘seminar’, ‘conference’, ‘manuscript writing’, ‘scientific writing’, ‘academic writing’, ‘scholarly writing’, ‘grants writing’, ‘capacity-building’ and ‘research’.

We defined ‘productivity’ as the proportion of dissertations from which at least one manuscript was published and ‘use’ as the proportion of dissertations cited in peer-reviewed articles, technical reports or policy-related documents. Thus, the search terms for the outcome of interest were ‘abstract’, ‘thesis’, ‘dissertation’, ‘publication’, ‘poster session’, ‘poster presentation’, ‘book chapter’, ‘technical report’, ‘policy brief’, ‘policy dialog’, ‘evidence-informed policy’, ‘evidence-based policy’, ‘evidence-informed health policy’, ‘evidence-based health policy’, ‘decision-making’, ‘policy-making’ and ‘dissemination’.

We restricted these search terms to the title or abstract, and included terms for the outcomes to maximise relevance and efficiency. Further, in order to minimise the risk of an empty review, we did not enter specific terms for the study design as we intended to use all evidence types to describe the available range of interventions. See the full search string in Additional file  1 . We found articles in French, Persian and Spanish, and used Google translator ( https://translate.google.com/ ) for English translations during screening and full-text review.

Additional searches

In our targeted search, we screened the reference lists of included publications and retrieved full texts of articles likely to be eligible for inclusion. We contacted authors of included articles for any literature that they may be aware of.

Selection of studies

Data management.

Using EndNote software version X7 (Thomson Reuters, 2015) we imported all identified titles, excluded duplicates, and screened and grouped these into relevant eligibility categories as described in our Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) flow chart [ 4 ].

Minimising bias in study identification and selection

A second reviewer (RS), an Information Scientist, validated the electronic search in PubMed by performing an independent and duplicate search. Similarly, a third reviewer (MO), screened all full texts excluded by the first reviewer (EAO). We resolved differences by discussion and consensus.

Criteria for considering inclusion of studies

We included published studies reporting at least one outcome of interest, and reporting on post-graduate research conducted in a low- and middle-income country.

Exclusion criteria for ineligible studies

Our exclusion criteria were studies about research conducted by Bachelor’s degree or undergraduate students’ or established university faculty not identified as post-graduate students, studies conducted in high-income settings or in low-income settings but by students from high-income settings, qualitative designs, non-empirical studies, syntheses, editorials or perspectives, studies published earlier than 1990, or those that were totally irrelevant.

Data abstraction

We adapted a data extraction tool for observational studies we had developed for a previous systematic review [ 5 ]. We then abstracted administrative, study design and primary outcome data on productivity measured as proportion of publications. We further abstracted our secondary outcomes of use of the research as measured by citations, time to publication, conference abstracts and additional outcomes describing the nature of the post-graduate students’ research (predominant types of research, first authorship status of student).

Handling of missing data

We denoted variables that were missing as not reported. We did not employ any statistical methods for handling missing data, neither did we contact authors for additional information.

Risk of bias of assessment of included studies

We adapted a tool we used in a published systematic review to assess for the risk of bias in the included studies [ 5 ]. We considered the following seven aspects of bias [ 6 , 7 ]: selection bias due to sampling or proportion of responders or baseline characteristics (and confounding), detection bias due to reliability of measurements used, and bias due to method of data analysis used for overall outcome and reporting biases. We categorised risk of bias as high, moderate or low, guided by the descriptive assessment questions in our tool (Additional file  2 ).

Synthesis of included studies

We employed a structured synthesis in which the units of analysis were findings from a single primary study. First, we described the characteristics of the included primary studies. Using the command ‘metaprop’ in Stata version 14.1 (Stata, College Station, Texas, USA), we constructed forest plots for proportions of published post-graduate work for both fixed and random effects models. The kind of data we abstracted would not permit assessment of the measures of effect, comparing two groups. We thus evaluated single group prevalence of publications, conference abstracts and predominant types of studies of the post-graduate work. We visually explored heterogeneity by inspecting the forest plots and statistically quantifying this using the I-squared statistic and tested for significance using Cochran’s Q. As we found a high level of heterogeneity we conducted a meta-regression testing the variables of duration of study, period of study and geographical region, before conducting a sensitivity analysis by excluding the very large study contributing 93% of the overall combined information. Finally, we wrote a narrative synthesis for the results for which we were unable to perform quantitative meta-analysis.

Differences between the published protocol and the actual study

In conducting this review, we employed practical approaches to circumvent unanticipated methodological challenges. We did not contact heads of academic or research departments in target universities as key informants, nor did we search additional grey literature-specific databases due to resource and time constraints of completing this doctoral project. Secondly, we did not find a single article reporting on the effects of strategies to increase productivity or increase the use of post-graduate students research and therefore synthesis was based on other relevant outcomes to map the field for future studies. Third, we did not assess the overall quality of evidence, as we did not find effectiveness studies or studies with comparison groups testing interventions for increasing productivity or use of post-graduate students research [ 8 ].

Systematic review flow, screening and inclusion

Our results are illustrated in the PRISMA flow chart (Fig.  1 ) and in Table  1 , while Table  2 shows a summary of the risk of bias assessments. We retrieved 5080 titles and abstracts from two databases, PubMed ( n  = 3848) and ERIC ( n  = 1232). We updated the search in PubMed only and findings are as recent as July 17, 2017, as it provides the bulk of health-related literature. After excluding 33 duplicates, we screened all 5047 titles or abstracts, excluding 5012 mainly due to irrelevance ( n  = 4659) or various reasons not meeting eligibility criteria ( n  = 353). We retrieved a total of 44 full text articles, of which 9 were from additional targeted searching; we finally reviewed 14 of them, with only 12 in the statistical meta-analysis. The main reason for excluding the full texts was not containing at least one outcome of interest and the two studies were dropped in the meta-analysis either because they lacked the primary outcome [ 9 ] or the primary outcome was not reported in a way to permit synthesis [ 10 ].

Flow diagram for systematic review on productivity of post-graduate students’ research

Meta-analysis of productivity of post-graduate students’ research in low- and middle-income countries

Description of included studies

The studies we included were from Asia ( n  = 7), Africa ( n  = 5) and South America ( n  = 2), published between 2007 and 2014, including post-graduate students over a 30-year span (1974 to 2014). The smallest study had a sample size of 90, while the largest had 22,625. All studies were about post-graduate students pursuing Masters’ degrees, of which 4 included Doctoral students as well. Importantly, although many studies included cohorts of post-graduate students, all used a cross-sectional analysis. We included 1 study that did not report the primary outcome but described other secondary outcomes, and 7 studies that reported at least an additional outcome of interest.

Findings on the outcomes of interest

Publication proportion of post-graduate students’ research.

We included 12 of the 14 review studies in the meta-analysis for the primary outcome, all together contributing work of 27,477 post-graduate students for the meta-analysis (Fig. 2 ). The proportion of post-graduate students research published ranged from 6% in Turkey [ 11 ] to 41% in Togo [ 12 ]. We found an average publication proportion of 7% (95% CI 7–8%; Higgins I-squared 0.0% and Cochran’s Q, p < 0.01) and 23% (95% CI 17–29%; Higgins I-squared of 98% and Cochran’s Q, p < 0.01) using fixed and random effects models, respectively.

Meta-regression and sensitivity analyses

After excluding the study by Orzgen et al. [ 11 ] as it accounted for 92% of the review sample size, the results were 20% (95% CI 19–21%; Higgins I-squared and Cochran’s Q, p < 0.01) by fixed effects and 24% (95% CI 20–29%; Higgins I-squared of 92% and Cochran’s Q, p < 0.01) by random effects.

Our meta-regression results were not significant for duration of study span, period of the research or geographical region (data not shown).

Citation of post-graduate students’ research

Two studies reported on the citation of studies, which was our surrogate for use of post-graduate students’ research. In Uganda [ 13 ], this was 17% overall (95% CI 15–19%) and 4% specific for policy-related documents, while in Turkey [ 11 ], the median citation was 1 study (IQR 0.6–2.3).

Determinants of post-graduate students’ research

Only one included study reported on the determinants of productivity or use of post-graduate students’ research, and it suggested that younger students were more likely to publish and that cohort studies were more likely to be published [ 13 ].

Additional outcomes

Six studies reported the time to publication from completion of theses by post-graduate students. The earliest average time to publication was 2 years (Cameroon, Egypt) [ 14 , 15 ], while the rest were 2.3, 2.8 and 3 years in Uganda [ 13 ], India [ 2 ] and Iran [ 16 ], respectively. Only three studies reported the proportion of abstracts presented in conferences as 54% in Togo, 11% in Cameroon and 2% in Uganda [ 12 , 13 , 14 ]. Post-graduate students were first authors of their work in 23%, 54%, 62% and 80% in Cameroon, India, Egypt and Turkey, respectively, and in nearly all their papers in Uganda [ 2 , 12 , 13 , 14 , 17 ]. Cross-sectional studies were the predominant designs of post-graduate students’ research projects (44–80%), while randomised trials were few (3–23%) [ 2 , 9 , 13 ].

Principal findings

We report that the majority of post-graduate students in low-income countries infrequently publish their research theses. Secondly, the most published studies are cross-sectional in design with hardly any clinical trials, likely because of feasibility considerations with higher logistic demands, particularly for students. Third, it is apparent that post-graduate students are not the first authors in a significant proportion of their published work. Nevertheless, the evidence of citations suggests that post-graduate students’ research work is used in some form.

Findings in relation to other systematic reviews

Although we did not identify data to support our primary objective, we found a systematic review about interventions for increasing scholarly productivity among residents in the United States and Canada, which are high-income settings [ 18 ]. This review mapped the following approaches that were associated with increased scholarly productivity: protected research time, research curricula, research directors, dedicated research days, and research tracks, but with mixed effects on resident presentations or publications. It would be important to extend these single studies by explicitly testing the approaches found to be associated with productivity in low-income countries such as Uganda.

The proportion of publications in our review was relatively low. In a seminal paper by Dickersin et al. [ 19 ], students in John Hopkins University in the 1980s were found to publish less than the faculty. Although it is not clear why students published less, we found that students were not necessarily first authors of their work even when they published. Indeed, authorship would be an incentive for career advancement for which students and faculty would benefit. It is possible that power imbalances may explain this finding and, in some instances, discourage would be student authors from publishing.

With only a single study appropriately documenting the determinants of productivity and citation, we could hardly draw firm conclusions. The results herein suggested that younger students were more likely to publish and that cohort studies were more likely to be published. Bullen and Reeve [ 20 ] documented that Master of Public Health students in New Zealand mentioned that barriers to publication included a protracted publication process and a negative perception of the importance of the results. Future qualitative investigations in this setting would be informative.

Systematic review strengths and limitations

This is the first systematic review documenting the academic research productivity of postgraduate students in low- and middle-income countries. We have employed a robust and internationally agreed methodology [ 21 ] to conduct this systematic review with a sizeable sample of included studies and students.

Nonetheless, we report two main methodological limitations in our review. First, we did not find a single study assessing the effects of interventions to increase academic research productivity of post-graduate students. This could be explained by the identification of all the relevant studies or university reports, limited by grey literature beyond the reach of our review team. Nonetheless, we employed a robust and comprehensive search strategy beyond the electronic search, which enabled us access to articles for over a 40-year span and in languages other than English. Secondly, identifying the outcome of ‘use’ of students’ research in the policy process or decision-making in health remains a challenge, with citation being an imperfect proxy. An additional limitation was the fact that we did not explore grey literature more effectively, largely due to resources constraints.

Implications for future research and policy

It is clear that there is a sheer lack of evidence to effectively assess interventions to increase productivity and use of research conducted by post-graduate students in low- and middle-income countries. The cohort studies that exist did not analyse the data in a manner that permits comparisons of groups exposed to specific approaches, an area that can be strengthened. Future studies should be prospective, employing mixed methods to investigate interventions to increase productivity and citation of post-graduate students’ research as well as to identify unique aspects such as publishing in predatory journals. Additionally, it is becoming more common for early career researchers globally to publish during their candidature (PhD by publication), an area that can be examined further. Policy-makers and actors should invest in supporting research in this area.

We report a low productivity of post-graduate students’ research in low- and middle-income countries, including the use of post-graduate students’ research in evidence-informed health policy. Secondly, we did not find a single study that assessed strategies to increase productivity and use of post-graduate students’ research in evidence-informed health policy, a subject for future research.

Ogunyemi D, Bazargan M, Norris K, Jones-Quaidoo S, Wolf K, Edelstein R, et al. The development of a mandatory medical thesis in an urban medical school. Teach Learn Med. 2005;17(4):363–9. https://doi.org/10.1207/s15328015tlm1704_9 .

Article   PubMed   Google Scholar  

Dhaliwal U, Singh N, Bhatia A. Masters theses from a university medical college: publication in indexed scientific journals. Indian J Ophthalmol. 2010;58(2):101–4. https://doi.org/10.4103/0301-4738.60070 .

Article   PubMed   PubMed Central   Google Scholar  

Obuku EA, Lavis JN, Kinengyere A, Mafigiri DK, Sengooba F, Karamagi C, et al. Where is students’ research in evidence-informed decision-making in health? Assessing productivity and use of postgraduate students’ research in low- and middle-income countries: a systematic review. Health Res Policy Syst. 2017;15(1):18. https://doi.org/10.1186/s12961-017-0169-9 .

Article   PubMed   PubMed Central   CAS   Google Scholar  

Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. BMJ. 2009;339:b2535.

Ocan M, Obuku EA, Bwanga F, Akena D, Richard S, Ogwal-Okeng J, et al. Household antimicrobial self-medication: a systematic review and meta-analysis of the burden, risk factors and outcomes in developing countries. BMC Public Health. 2015;15:742. https://doi.org/10.1186/s12889-015-2109-3 .

Guyatt GH, Oxman AD, Vist G, Kunz R, Brozek J, Alonso-Coello P, et al. GRADE guidelines: 4. Rating the quality of evidence--study limitations (risk of bias). J Clin Epidemiol. 2011;64(4):407–15. https://doi.org/10.1016/j.jclinepi.2010.07.017 .

Sterne JA, Hernan MA, Reeves BC, Savovic J, Berkman ND, Viswanathan M, et al. ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions. BMJ. 2016;355:i4919. https://doi.org/10.1136/bmj.i4919 .

Guyatt G, Oxman AD, Akl EA, Kunz R, Vist G, Brozek J, et al. GRADE guidelines: 1. Introduction-GRADE evidence profiles and summary of findings tables. J Clin Epidemiol. 2011;64(4):383–94. https://doi.org/10.1016/j.jclinepi.2010.04.026 .

Ahmed Y, Kanyengo C, Akakandelwa A. Mapping postgraduate research at the University of Zambia: a review of dissertations for the Master of Medicine Programme. Med J Zambia. 2010;37(2):52–7.

PubMed   PubMed Central   CAS   Google Scholar  

Linch GF, Ribeiro AC, Guido LA. Graduate program in nursing at the Federal University of Santa Maria: trajectory and results. Rev Gaucha Enferm. 2013;34(1):147–54.

Özgen Ü, Egri M, Aktas M, Sandikkaya A, Özturk ÖF, Can S, Ozcan C. Publication pattern of Turkish medical theses: analysis of 22,625 medical theses completed in years 1980-2005. Turk Klin J Med Sci. 2011;31(5):1122-31.

Pitche PT, Onipoh DK, Tchangai-Walla KL. Scientific dissemination of medical dissertations at the University of Lome (Togo). Sante. 2007;17(2):117–20.

PubMed   Google Scholar  

Obuku EA, Lavis JN, Kinengyere A, Mafigiri DK, Sengooba F, Karamagi C, et al. Academic research productivity of post-graduate students at Makerere University College of Health Sciences, Uganda, from 1996 to 2010: a retrospective review. Health Res Policy Syst. 2017;15:30. https://doi.org/10.1186/s12961-017-0194-8 .

Munung NS, Vidal L, Ouwe-Missi-Oukem-Boyer O. Do students eventually get to publish their research findings? The case of human immunodeficiency virus/acquired immunodeficiency syndrome research in Cameroon. Ann Med Health Sci Res. 2014;4(3):436–41. https://doi.org/10.4103/2141-9248.133474 .

Nour-Eldein H, Mansour NM, Abdulmajeed AA. Master’s and doctoral theses in family medicine and their publication output, Suez Canal University, Egypt. J Fam Med Prim Care. 2015;4(2):162–7. https://doi.org/10.4103/2249-4863.154622 .

Article   Google Scholar  

Motamed-Jahromi M, Leila Dehghani S. Nursing MSc theses: a study of an Iranian College of Nursing and Midwifery in two decades (1990–2010). Glob J Health Sci. 2014;6(5):118–24. https://doi.org/10.5539/gjhs.v6n5p118 .

Sipahi H, Durusoy R, Ergin I, Hassoy H, Davas A, Karababa A. Publication rates of public health theses in international and national peer-review journals in Turkey. Iran J Public Health. 2012;41(9):31–5.

Stevenson MD, Smigielski EM, Naifeh MM, Abramson EL, Todd C, Li ST. Increasing scholarly activity productivity during residency: a systematic review. Acad Med. 2017;92(2):250–66. https://doi.org/10.1097/acm.0000000000001169 .

Dickersin K, Min YI, Meinert CL. Factors influencing publication of research results. Follow-up of applications submitted to two institutional review boards. JAMA. 1992;267(3):374–8.

Article   PubMed   CAS   Google Scholar  

Bullen CR, Reeve J. Turning postgraduate students’ research into publications: a survey of New Zealand masters in public health students. Asia Pac J Public Health. 2011;23(5):801–9. https://doi.org/10.1177/1010539511417998 .

Moher D, Shamseer L, Clarke M, Ghersi D, Liberati A, Petticrew M, et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Rev. 2015;4:1. https://doi.org/10.1186/2046-4053-4-1 .

Eftekhari Y, Rezaeian M, Zare Bidaki M, Arabshahi A. A survey on the status of article publication from defended medical theses in Rafsanjan University of Medical Sciences, School of Medicine during 1993–2007. J Rafsanjan Univ Med Sci. 2013;12(7):531–44.

Ogrenci A, Eksi MS, Ozcan-Eksi EE, Koban O. From idea to publication: publication rates of theses in neurosurgery from Turkey. Neurol Neurochir Pol. 2016;50(1):45–7. https://doi.org/10.1016/j.pjnns.2015.11.007 .

Koca K, Ekinci S, Akpancar S, Gemci MH, Ersen O, Akyildiz F. An analysis of orthopaedic theses in Turkey: Evidence levels and publication rates. Acta Orthop Traumatol Turc. 2016;50(5):562–66. https://doi.org/10.1016/j.aott.2016.03.001 .

Ticse R, Ygreda P, Samalvides F. [Publication of research projects for certification as medical specialists at a Peruvian university, 2007-2010]. Rev Peru Med Exp Salud Publica. 2014;31(2):292–6.

Download references

Acknowledgements

The authors acknowledge Benjamin Temper and Joanita Nangendo for initial review of the search strategy of the protocol.

The writing of this manuscript has been supported by the IDRC International Research Chair in Evidence-Informed Health Systems and Policies, jointly held by Nelson Sewankambo and John Lavis. Dr Ekwaro Obuku is a doctorate student at Makerere University College of Health Sciences, Kampala, Uganda. He is a beneficiary of this training grant number 104519–008, held by Prof Nelson K Sewankambo of Makerere University, College of Health Sciences, Kampala, Uganda, and Prof John N Lavis of McMaster University, Hamilton, Ontario, Canada.

Author information

Authors and affiliations.

Clinical Epidemiology Unit, Department of Medicine, School of Medicine, College of Health Sciences, Makerere University, PO Box 7072, Kampala, Uganda

E. A. Obuku, C. Karamagi & N. K. Sewankambo

McMaster Health Forum, Centre for Health Economics and Policy Analysis, Department of Health Research Methods, Evidence and Impact, and Department of Political Science, McMaster University, Hamilton, Canada

J. N. Lavis

Department of Health Policy and Planning, School of Public Health, College of Health Sciences, Makerere University, Kampala, Uganda

F. Ssengooba

Department of Social Work and Social Administration, College of Humanities and Social Sciences, Makerere University, Kampala, Uganda

D. K. Mafigiri

Sir Albert Cook Library, College of Health Sciences, Makerere University, Kampala, Uganda

A. Kinengyere

Department of Global Health and Population, Harvard School of Public Health, Harvard University, Cambridge, MA, United States of America

Center for Social Science Research on AIDS, Department of Anthropology, College of Arts and Sciences, Case Western Reserve University, Cleveland, OH, United States of America

The African Centre for Systematic Reviews and Knowledge Translation, Makerere University, Kampala, Uganda

E. A. Obuku, R. Ssenono & M. Ocan

Faculty of Epidemiology and Population Health, London School of Hygiene and Tropical Medicine, London, United Kingdom

E. A. Obuku

You can also search for this author in PubMed   Google Scholar

Contributions

EAO, NKS and JNL participated in developing the idea into a concept. EAO wrote the initial protocol, while AK developed the search strategy. RS and MO validated the review methods through independent and duplicate searching, abstraction and risk of bias assessments. JNL, NKS, AK, DKM, FS and CK appraised the draft protocols, reviewed and approved the final version of this manuscript for publication.

Corresponding author

Correspondence to E. A. Obuku .

Ethics declarations

Ethics approval and consent to participate.

As this is a systematic review of literature this section is not applicable. Nonetheless, the School of Medicine Research & Ethics Committee, the Uganda National Council for Science & Technology (HS 3268) and the Office of the President of Uganda (ADM/154/212/01) approved this study.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Additional files

Additional file 1:.

Supplement 1. Feasibility of yield of literature of pilot electronic search strategy for post-graduate students’ research. Supplement 2. Updated search strategy as at 17th July 2017 in PubMed ( https://www.ncbi.nlm.nih.gov/pubmed/ ). (DOCX 21 kb)

Additional file 2:

Risk of bias assessment tool: productivity of post-graduate students’ research in low- and middle-income countries. (DOC 59 kb)

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License ( http://creativecommons.org/licenses/by/4.0/ ), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Cite this article.

Obuku, E.A., Lavis, J.N., Kinengyere, A. et al. A systematic review on academic research productivity of postgraduate students in low- and middle-income countries. Health Res Policy Sys 16 , 86 (2018). https://doi.org/10.1186/s12961-018-0360-7

Download citation

Received : 19 February 2018

Accepted : 07 August 2018

Published : 28 August 2018

DOI : https://doi.org/10.1186/s12961-018-0360-7

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Productivity
• Publication
• Evidence-informed health policy
• Knowledge translation

Health Research Policy and Systems

ISSN: 1478-4505

• Submission enquiries: Access here and click Contact Us
• General enquiries: [email protected]

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• CAREER FEATURE
• 18 June 2018

How a hobby can boost researchers’ productivity and creativity

• Julia Rosen 0

Julia Rosen is a freelance writer in Portland, Oregon.

You can also search for this author in PubMed   Google Scholar

When Audrey Kelly isn’t catching toads and analysing their DNA to study how species hybridize, she makes bread. Kelly is a fifth-year PhD student at the University of North Carolina, Chapel Hill, and she learnt to bake from her father before she moved away for her undergraduate programme. “It’s kind of like a science experiment,” she says, “but you get to eat it at the end.”

Access options

Access Nature and 54 other Nature Portfolio journals

Get Nature+, our best-value online-access subscription

24,99 € / 30 days

cancel any time

Subscribe to this journal

Receive 51 print issues and online access

185,98 € per year

only 3,65 € per issue

Rent or buy this article

Prices vary by article type

Prices may be subject to local taxes which are calculated during checkout

Nature 558 , 475-477 (2018)

doi: https://doi.org/10.1038/d41586-018-05449-7

Newman, D. B., Tay, L. & Diener, E. J. Happiness Stud. 15 , 555–578 (2014).

Article   PubMed   Google Scholar  

Eschleman, K. J., Madsen, J., Alarcon, G. & Barelka, A. J. Occup. Organ. Psychol. 87 , 579–598 (2014).

Article   Google Scholar  

Sio, U. N. & Ormerod, T. C. Psychol. Bull. 135 , 94–120 (2009).

Baird, B. et al. Psychol. Sci. 23 , 1117–1122 (2012).

Prakash, R. S., Voss, M. W., Erickson, K. I. & Kramer, A. F. Annu. Rev. 66 , 769–797 (2014)

Download references

Related Articles

Tales of a migratory marine biologist

Career Feature 28 AUG 24

Nail your tech-industry interviews with these six techniques

Career Column 28 AUG 24

How to harness AI’s potential in research — responsibly and ethically

Career Feature 23 AUG 24

Binning out-of-date chemicals? Somebody think about the carbon!

Correspondence 27 AUG 24

Tenure-track Associate Professor [Female Only]

Seeking tenure-track assoc. professor for interdisciplinary research in nanoprobe life sciences or related fields at WPI Nano Life Science Institute.

Kanazawa, Ishikawa, Japan (JP)

Nano Life Science Institute, Kanazawa University

Assistant/Associate Professor

Center for Virology and Vaccine Research at Beth Israel Deaconess Medical Center is seeking Assistant or Associate Professor.

Boston, Massachusetts (US)

Beth Israel Deaconess Medical Center (BIDMC)

OPEN FACULTY POSITION-INSTITUTE OF MOLECULAR BIOLOGY, ACADEMIA SINICA, TAIWAN, ROC

One tenure-track faculty position is open to establish an active research program in all disciplines of molecular and cellular biology.

Taipei (TW)

The Institute of Molecular Biology, Academia Sinica, Taiwan

Global Faculty Recruitment of School of Life Sciences, Tsinghua University

The School of Life Sciences at Tsinghua University invites applications for tenure-track or tenured faculty positions at all ranks (Assistant/Ass...

Beijing, China

Tsinghua University (The School of Life Sciences)

Tenure-Track/Tenured Faculty Positions

Tenure-Track/Tenured Faculty Positions in the fields of energy and resources.

Suzhou, Jiangsu, China

School of Sustainable Energy and Resources at Nanjing University

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

• Research article
• Open access
• Published: 14 October 2015

A method for measuring individual research productivity in hospitals: development and feasibility

• Caterina Caminiti 1 ,
• Elisa Iezzi 1 ,
• Caterina Ghetti 2 ,
• Gianluigi De’ Angelis 3 &
• Carlo Ferrari 4  

BMC Health Services Research volume  15 , Article number:  468 ( 2015 ) Cite this article

8924 Accesses

13 Citations

2 Altmetric

Metrics details

Research capacity is a prerequisite for any health care institution intending to provide high-quality care, yet, few clinicians engage in research, and their work is rarely recognized. To make research an institutional activity, it could be helpful to measure health care professionals’ research performance. However, a comprehensive approach to do this is lacking.

We conducted a literature analysis to determine how best to assess research performance. Our method was not restricted to bibliometric and citation parameters, as is usually the case, but also including “hidden” activities, generally not considered in research performance evaluations.

A set of 12 easily retrievable indicators was used and corresponding points assigned according to a weighting system intended to reflect the effort estimated to perform each activity. We observed a highly skewed score distribution, with a minority of health care professionals performing well across the indicators. The highest score was recorded for scientific papers (768/1098 points, 70 %). Twenty percent of researchers at our institution generated 50 % of points.

Conclusions

We develop a simple method for measuring research performance, which could be rapidly implemented in health care institutions. It is hoped that the proposed method might be useful for promoting research and guiding resource allocation, although further evaluations are needed to confirm the method’s utility.

Peer Review reports

It is widely accepted that research plays an essential role in developing new health care services and improving healthcare quality. Research provides new knowledge that can be transferred into practice, helps create advanced care environments, that attract the best physicians contributes to learning among young Health Care Professionals (HCPs) and ensures continuous education among established professional. In fact, hospitals engaged in research have been recognized to as providing better patient care. Therfore, adequate research capacity is a prerequisite for any public health care system striving to provide high-quality care [ 1 ].

The goal of evidence-based practice increasingly requires research to be embedded within the health care setting, making clinician participation an essential component of its success. In fact, clinicians are well-placed to identify relevant research ideas, design and conduct innovative projects, ensure translation of research into improved health outcomes, and solicit patient enrollment in experimental trials. Nevertheless, the international literature shows that only a minority of clinicians participate in research aproblem common to many countries [ 2 – 5 ].

This issue is particularly relevant in teaching hospitals, which have a responsibility to provide leadership in conducting, supporting and supervising research [ 5 ]. There is no standardized method for measuring HCP’s research efforts and their results; this is a key obstacle to the incorporation of research in hospitals as an institutional activity [ 6 ]. Such a method would, among other benefits, inform resource allocation decisions, encourage research participation among increasingly busy clinicians, and create accountability to the community for research projects.

To this end, we developed a mechanism that attempts to measure as objectively as possible research productivity, and tested it at our institution to determine its feasibility and utility. In this study, research productivity is defined as the product of research activities. The terms “research productivity”, “output” and “performance”, are used interchangeably.

The University Hospital of Parma is a large health care facility located in Emilia-Romagna, a region in northern Italy with a population of 5 million served by four university hospitals, four health care research institutes and 12 community hospitals.

Since 2004, regional legislation formally identifies research as a fundamental institutional activity, equal to patient care and continuous training. This policy underlies several funding initiatives aimed to promote research, with special attention to young professionals [ 7 ]. Regional hospitals are required proactively support their researchers, through clinical governance actions aiming to track research activities already underway, identify priority areas for resource allocation and infrastructure, and provide adequate tracking and recognition of researcher efforts. However, no method for measuring research has been devised and implemented across health care institutions in this region, where hospital productivity is currently only being measured in terms of patient care activity.

This work pursues the following objectives:

develop a simple method to measure individual research productivity and analyze hospital department performance

determine its feasibility and describe its potential usefulness in a large University Hospital

Choice of indicator variables

Literature analysis was conducted to determine how best to assess research performance. The terms “research output”, “research productivity”, and “research performance” were used to retrieve potentially relevant studies published over the last 5 years. Articles that only analyzed bibliometric indices and those that did not report on an empirical setting were not considered.

Although the evaluation of research productivity would ideally include the assessment of impact, in practice this is extremely difficult to achieve, because the multifaceted nature of evaluation, the lack of standard terminology, and the heterogeneity of empirical experiences make it hard to identify a preferred model of impact measurement [ 8 ]. For this reason, the measure of research output is often used as a proxy for impact. A wide range of indicators and metrics are available for this purpose, and their choice depends on considerations of their strengths and limitations [ 8 ]. The most widely used research output indicators are bibliometric and citation parameters (e.g. number of publications in peer-reviewed journals, impact factor and H-index) [ 9 ]. These are very simple to calculate, but also exhibit various limitations that have been extensively described [ 9 – 11 ].

Wootton [ 12 ] recently proposed a simple method to measure research output, defining an indicator simple enough to be calculated and generalizable to other settings, but still able to capture the complexity of research productivity. The indicator, inspired by the analysis of 12 reports on research productivity, is constructed on the following three domains, based on data relating to individual researchers: (i) research grant income, (ii) peer-reviewed publications and (iii) PhD student supervision. Activity in each domain is converted to points, which are used to calculate a score for research output that allows comparisons: (a) within an organizational unit, for example from year to year, or (b) in the same year between organizational units (e.g. research teams, wards, departments, and hospitals). The proposed score was arbitrary, because no validated and widely accepted metrics exists, but it was compared with an independent assessment made by a group of expert researchers, which yielded a significant correlation of 71 %.

This indicator, however, neglects a range of “hidden” activities that are also relevant to research output assessment. There are described in a well-known editorial by former BMJ Editor Richard Smith [ 13 ]. These include, for example, participation in the preparation of guidelines, teaching activities in the field of research, and peer-reviewing.

In another interesting work- by Mezrich et al. [ 14 ] reported the development of a more complex system for the assessment of the productivity of a ward/academic department, which assigns points to research activities by considering the estimated effort required to perform each activity and its attributed academic value.

Development of the method

The approach we defined is an adaptation of the model proposed by Wootton [ 12 ], integrated with other types of activities indicated by Smith [ 13 ] and inspired by the metrics used by Mezrich et al. [ 14 ]. The choice of research activities was determined by the availability of required information. The weighting system was constructed considering the hypothesized effort for all indicators,. For some indicators, specific criteria were also applied.

For each HCP, a set of information easily retrievable from existing administrative sources (mostly the Parma Ethics Committee’s archive) and from bibliographic databases (e.g.,ISI Web of Knowledge) was collected. These includedcompetitive research funding, publications, students/collaborators supervised, commissioned studies and patent filing. Additionally, some information usually not recorded was gathered by means of a simple questionnaire adapted from the literature, a tool used by German researchers for the measurement of the effects of a training program designed to improve HCPs’ research skills [ 15 ]. This adapted questionnaire (see Additional file 1 ) has been employed at our institution since 2012 to gather data on self-reported participation in research activities [ 16 ] and contains the remaining seven indicators used in this study. To be included in the final score, research activities indicated in the questionnaire had have been previously documented.

The set of proposed indicators, the weighting system and the number of possible points assigned to each are depicted in Table  1 .

For the first indicator, concerning grants acquired by the Principal Investigator in competitive research funding programs, one point is assigned for every €24.000 awarded; this is the lowest award for a standard Italian research grant (Decree of the Italian Ministry of Education, Universities and Research, no. 102, 9 March 2011). This solution is suggested by Wootton as a scaling factor to facilitate comparisons between countries. Thus because on average the annual cost of a resident physician is three times greater than that of a research grant, one point is assumed to reflect about 1/3 of an HCP’s annual work (approximately 4 months). This assumption was used to assign scores taking -into account the estimated time required to carry out a given research activity relative to others.

For the publication indicator, each paper received a score weighted by the Normalized journal Impact Factor (NIF) and by author position (Table  2 ). The NIF is an adjusted method for calculating the impact factor that takes into account the diversity of citing behavior in different disciplines and is inteded intended to assess the relative position of journals, potential employers, and researchers within each field [ 17 ]. Weighting criteria for the publication score are based on the method developed by Tscharntke [ 18 ], whereby the first author is awarded the highest value, but the second and last authors also receive a higher score than the other coauthors.

For the remaining indicators, score assignment was straightforward, as shown in Table  1 . The calculation performed for this set of indicators allows us to obtain (for each HCP) a combined score resulting from the sum of non-dimensional values, which permits spatial and temporal comparisons.

Implementation

Overall, the time needed to create a single database, process indicators for each HCP and analyze data was about 9 weeks of work by one person. The analysis was performed using SAS version 8.2. Time for data collection and analysis may be significantly reduced with the use of web-based software into which pertinent data may be entered by HCPs themselves.

To allow for comparisons with other institutions, wards were grouped into the following six areas, which represent relatively homogeneous research activities: Surgery units; Diagnostic Services; Emergency Medicine; General Medicine, Geriatrics and Rehabilitation; Specialized Medicine; Pediatrics and Gynecology. To reduce variability (Coefficient of Variation = 38 %) due the different numbers of HCPs in each area, estimates were corrected by direct standardization. For each area, along with the sum and the weighted sum, the mean score (per capita output) and corresponding range are also provided.

Intra- and inter comparisons

Tables  3 and 4 summarize respectively raw data and calculated values for research activity relating to the year 2013, subdivided for each indicator. The most relevant findings with respect to this work’s objectives are the following:

When no score is assigned, prevailing activities are publications (597/1165, 51 %), projects not funded by competitive programs (108/1165, 9 %) and research proposals submitted to competitive programs but non awarded (89/1165, 8 %).

The highest score was recorded for scientific papers (768/1098 points, 70 %), followed by research grant income (15 %) and peer-reviewing (5 %). Together, these three items account for 90 % of the research output at our institution.

The area of specialized medicine exhibited the highest research productivity, even after standardization (111/222). This means that a mere 20 % of HCPs at our institution produced 50 % of points

The annual mean per-capita output score was 1.2 points, ranging from 0.4 to 2.2 points (indicated the the highest-scoring researchers were nearly six times more productive than lowest scoring researchers).

Figure  1 shows the research output for individual HCPs belonging to each area, for the following indicators: grant income, scientific publications, PhD students/collaborators supervised, and other activities. Our analysis indicated:

a (Research output for each HCP belonging to the area) – SURGERY UNITS. b (Research output for each HCP belonging to the area) – DIAGNOSTIC SERVICES. c (Research output for each HCP belonging to the area) – EMERGENCY MEDICINE. d (Research output for each HCP belonging to the area) – GENERAL MEDICINE, GERIATRICS AND REHABILITATION. e (Research output for each HCP belonging to the area) – SPECIALIZED MEDICINE. f (Research output for each HCP belonging to the area) – PEDIATRICS AND GYNECOLOGY

Within all areas, few individuals obtained high scores, whereas the majority received low scores or zero points

Only a few individuals performed well across multiple indicators, whereas for the majority, output mainly consisted of publications.

Figure  2 summarizes the individual score distribution for score classes, which shows even more clearly that high research productivity was only achieved by a small group of HCPs.

Score frequency distribution

We present a novel method for measuring research performance. The results obtained by our method’s implementation at a large Italian University Hospital highlight the simplicity of its implementation and describe its potential uses.

To our knowledge, this is the first comprehensive approach to measuring individual research output in hospitals that also includes “hidden” research activities, which are essential to ensure high-quality patient care, such as participation in the definition of guidelines, submission of research proposals to competitive funding programs regardless of funding acquisition, and teaching activities concerning one’s own research. This system exhibits many potential strengths and possible applications: it enables identification of identify which HCPs are highly productive in research, reveals of areas potentially in need of improvement, and provides indications for resource allocation.

Our work differs from Wootton’s study in many respects. First other things, implementation lasted 1 year and involved the entire institution, whereas for Wootton’s study lased 5 years and concerned two departments. Still, the two studies are similar enough to make a direct comparison of results. In both studies, score distribution is considerably skewed, and most points are earned by a small number of HCPs, mostly performing well on the publication indicator.

The chosen indicators and attributed scores still remain to be validated and widely shared. In fact, as evident in Tables  3 and 4 , the chosen weighting system leads to the dominance of publication output and grant income. Other hospitals may feel that a more balanced scorecard would be preferable. However, validation was not the aim of this work, also because a precise use of results has not yet been defined. In fact, as Mezrich et al. pointed out by [ 14 ], for some purposes, such as measuring change in activity or productivity from one year to the next or the relative productivity of individuals performing similar activities in a single division or at different institutions, the values chosen would not matter, as long as they were consistent for all HCPs. A validated weighting system may instead be used as a tool to guide and promote research. For instance, more points may be assigned to strategic research activities (e.g., supervision of young PhD students and research collaborators), or rankings may be used (e.g., reviewers for prestigious international journals could be awarded higher scores). However, such systems should be applied with caution, as pointed out in a recent systematic review [ 19 ] on the effects of strategies introduced in academic medical centers to assess productivity as part of compensation schemes. The results of the 9 study review demonstrate that these strategies improve research output and help to achieve the department’s mission, but may have unintended negative consequences; for instance, HCPs may assume that items not included in the evaluation are less important and may thus neglect them.

It must be emphasized that this study is based on secondary data not collected for the purpose of this research, which may have led to an underestimation of the score, particularly concerning “hidden” activities, which had to have been previously documented by HCPs.

Although further evaluations is needed, this work suggests that the proposed method may is feasible and may be useful to achieve different purposes, such as:

Guiding funding of health care facilities, as is done with patient care (for instance through Diagnosis-Related Groups - DRGs)

Including research activity in the assessment of a ward’s productivity, in the analysis of the workload and in subsequent allocation of necessary resources

Overcoming the current disparity observed in Italian university hospitals, where recognition for research activities is ensured to HCPs employed by the university but not to those employed by the hospital, though both groups work in the same institution

Highlighting the most productive and authoritative research centers, which may be qualified as centers of excellence for research worth being supported and enhanced

Providing information that could form the basis for a regional research network, according to a Hub and Spoke model, to increase research capacity in facilities that do not have research as their mission and to prevent study duplication and consequent waste of resources.

Abbreviations

Health-Care Professionals

Normalized journal impact factor

Diagnosis-Related Groups

Whitworth A, Haining S, Stringer H. Enhancing research capacity across healthcare and higher education sectors: development and evaluation of an integrated model. BMC Health Serv Res. 2012;12:287.

Article   PubMed   PubMed Central   Google Scholar  

Pfeiffer SI, Burd S, Wright A. Clinician and research - recurring obstacles and some possible solutions. J Clin Psychol. 1992;48(1):140–5.

Article   CAS   PubMed   Google Scholar  

Roxburgh M. An exploration of factors which constrain nurses from research participation. J Clin Nurs. 2006;15(5):535–45.

Article   PubMed   Google Scholar  

Dev AT, Kauf TL, Zekry A, Patel K, Heller K, Schulman KA, et al. Factors influencing the participation of gastroenterologists and hepatologists in clinical research. BMC Health Serv Res. 2008;8:1–11.

Article   Google Scholar  

Paget SP, Lilischkis KJ, Morrow AM, Caldwell PHY. Embedding research in clinical practice: differences in attitudes to research participation among clinicians in a tertiary teaching hospital. Intern Med J. 2014;44(1):86–9.

Embi PJ, Tsevat J. Commentary: the relative research unit: providing incentives for clinician participation in research activities. Acad Med. 2012;87(1):11–4.

Agenzia Sanitaria e Sociale Regione Emilia-Romagna. http://assr.regione.emilia-romagna.it/it/newsletter ASR [Access 10/13/2015].

Banzi R, Moja L, Pistotti V, Facchini A, Liberati A. Conceptual frameworks and empirical approaches used to assess the impact of health research: an overview of reviews. Health Res Policy Syst. 2011;9:26.

Pendlebury DA. The use and misuse of journal metrics and other citation indicators. Archivum Immunologiae Et Therapiae Experimentalis. 2009;57(1):1–11.

Durieux V, Gevenois PA. Bibliometric indicators: quality measurements of scientific publication. Radiology. 2010;255(2):342–51.

Elliott DB. The impact factor: a useful indicator of journal quality or fatally flawed? Ophthalmic Physiol Opt. 2014;34(1):4–7.

Wootton R. A simple, generalizable method for measuring individual research productivity and its use in the long-term analysis of departmental performance, including between-country comparisons. Health Res Policy Syst. 2013;11:2.

Smith R. Measuring the social impact of research - Difficult but necessary. Br Med J. 2001;323(7312):528–8.

Article   CAS   Google Scholar  

Mezrich R, Nagy PG. The academic RVU: a system for measuring academic productivity. J Am Coll Radiol. 2007;4(7):471–8.

Lowe B, Hartmann M, Wild B, Nikendei C, Kroenke K, Niehoff D, et al. Effectiveness of a 1-year resident training program in clinical research: A controlled before-and-after study. J Gen Intern Med. 2008;23(2):122–8.

Bilancio di Missione, Azienda Ospedaliero-Universitaria di Parma. http://www.ao.pr.it/chi-siamo/bilanci/bilancio-di-missione/ . [Access 10/13/2015].

Owlia P, Vasei M, Goliaei B, Nassiri I. Normalized impact factor (NIF): an adjusted method for calculating the citation rate of biomedical journals. J Biomed Inform. 2011;44(2):216–20.

Tscharntke T, Hochberg ME, Rand TA, Resh VH, Krauss J. Author sequence and credit for contributions in multiauthored publications. Plos Biology. 2007;5(1):13–4.

Akl EA, Meerpohl JJ, Raad D, Piaggio G, Mattioni M, Paggi MG, et al. Effects of assessing the productivity of faculty in academic medical centres: a systematic review. Can Med Assoc J. 2012;184(11):E602–12.

Download references

Acknowledgements

We thank Prof. Marco Vitale of the University of Parma, and Luca Sircana, Managing Director of the Parma University Hospital, for supporting this project, and for believing in the importance of promoting a culture of scientific research. We also thank Francesca Diodati for her help with the translation and editing of the manuscript.

Author information

Authors and affiliations.

Research and Innovation Unit, University Hospital of Parma, Parma, Italy

Caterina Caminiti & Elisa Iezzi

Medical Physics Unit, University Hospital of Parma, Parma, Italy

Caterina Ghetti

Gastroenterology Unit, University Hospital of Parma, Parma, Italy

Gianluigi De’ Angelis

Infectious Diseases and Hepatology Unit, University Hospital of Parma, Parma, Italy

Carlo Ferrari

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Caterina Caminiti .

Additional information

Competing interests.

The authors declare that they have no competing interests.

Authors’ contributions

CC was responsible for the design of the study, participated in data interpretation and was in charge of drafting the manuscript. EI was involved with data collection and analysis, and contributed to drafting the manuscript. CG conducted the review of policy documents used to plan the study, and critically revised the manuniscript. GDA participated in the design of the study and critically revised the manuscript. CF conceived the study, participated in its design, and contributed to drafting the manuscript. All authors read and approved the final manuscript.

Additional file

Additional file 1:.

Questionnaire to record participation in resarch activities. (DOC 32 kb)

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License ( http://creativecommons.org/licenses/by/4.0/ ), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Cite this article.

Caminiti, C., Iezzi, E., Ghetti, C. et al. A method for measuring individual research productivity in hospitals: development and feasibility. BMC Health Serv Res 15 , 468 (2015). https://doi.org/10.1186/s12913-015-1130-7

Download citation

Received : 24 February 2015

Accepted : 05 October 2015

Published : 14 October 2015

DOI : https://doi.org/10.1186/s12913-015-1130-7

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Research productivity
• Research output
• Capacity building
• Health care providers
• Productivity measurement
• Research institutions
• University hospitals

BMC Health Services Research

ISSN: 1472-6963

• General enquiries: [email protected]

Productivity in academia

• 23(2):184-195

• Makerere University

Abstract and Figures

Discover the world's research

• 25+ million members
• 160+ million publication pages
• 2.3+ billion citations

• Chunhuai Yang
• Meichen Meng

• Holly P. Jones
• Minhyuk Seo

• Bruno Ocaya
• STUD HIGH EDUC

• Lawrence C. Michels

• Moneef Zou 'bi

• Darrell R. Lewis
• J HIGH EDUC
• Carole J. Bland
• Bruce Center

• Justin Staples
• M.W. Browne

• William Luckey
• PATRICIA A. HELLAND
• Recruit researchers
• Join for free
• Login Email Tip: Most researchers use their institutional email address as their ResearchGate login Password Forgot password? Keep me logged in Log in or Continue with Google Welcome back! Please log in. Email · Hint Tip: Most researchers use their institutional email address as their ResearchGate login Password Forgot password? Keep me logged in Log in or Continue with Google No account? Sign up

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here .

Loading metrics

Open Access

Peer-reviewed

Research Article

How Many Is Too Many? On the Relationship between Research Productivity and Impact

* E-mail: [email protected]

Affiliations Université de Montréal, École de bibliothéconomie et des sciences de l'information, C.P. 6128, Succ. Centre-Ville, H3C 3J7 Montréal, Qc., Canada, Université du Québec à Montréal, Centre interuniversitaire de recherche sur la science et la technologie (CIRST), Observatoire des sciences et des technologies (OST), C.P. 8888, Succ. Centre-Ville, H3C 3P8 Montreal, Qc., Canada

Affiliation Leiden University, Centre for Science and Technology Studies (CWTS), Wassenaarseweg 62A, 2333 AL Leiden, The Netherlands

• Vincent Larivière, 
• Rodrigo Costas

• Published: September 28, 2016
• https://doi.org/10.1371/journal.pone.0162709
• Reader Comments

Over the last few decades, the institutionalisation of quantitative research evaluations has created incentives for scholars to publish as many papers as possible. This paper assesses the effects of such incentives on individual researchers’ scientific impact, by analysing the relationship between their number of articles and their proportion of highly cited papers. In other words, does the share of an author’s top 1% most cited papers increase, remain stable, or decrease as his/her total number of papers increase? Using a large dataset of disambiguated researchers (N = 28,078,476) over the 1980–2013 period, this paper shows that, on average, the higher the number of papers a researcher publishes, the higher the proportion of these papers are amongst the most cited. This relationship is stronger for older cohorts of researchers, while decreasing returns to scale are observed for recent cohorts. On the whole, these results suggest that for established researchers, the strategy of publishing as many papers as possible did not yield lower shares of highly cited publications, but such a pattern is not always observed for younger scholars.

Citation: Larivière V, Costas R (2016) How Many Is Too Many? On the Relationship between Research Productivity and Impact. PLoS ONE 11(9): e0162709. https://doi.org/10.1371/journal.pone.0162709

Editor: Pablo Dorta-González, Universidad de las Palmas de Gran Canaria, SPAIN

Received: April 27, 2016; Accepted: August 26, 2016; Published: September 28, 2016

Copyright: © 2016 Larivière, Costas. This is an open access article distributed under the terms of the Creative Commons Attribution License , which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Data Availability: Aggregated data are available from Figshare: 10.6084/m9.figshare.3795543 . However, restrictions apply to the availability of the bibliometric data, which is used under license from Thomson Reuters. Readers can contact Thomson Reuters at the following URL: http://thomsonreuters.com/en/products-services/scholarly-scientific-research/scholarly-search-and-discovery/web-of-science.html .

Funding: This study was funded by the Social Sciences and Humanities Research Council of Canada, as well as by the Canada Research Chairs program. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Competing interests: The authors have declared that no competing interests exist.

Introduction

Over the last few decades, evaluations have become widespread in various spheres of society [ 1 ]. Despite being assessed internally through peer review since the second half of the 20 th Century, research has, for most of its modern existence, been exempt from external evaluations—thanks to post-WWII economic and scientific growth, as well as the general idea, advocated by Vannevar Bush [ 2 ], that science should be free of external interventions. Means of evaluating research and scholars have, however, slowly changed during the 1980s and 1990s, with researchers, administrators, and policy makers gradually incorporating bibliometric indicators in the process. Such quantitative analyses of research activity and impact gained further importance in the 2000s [ 3 ], when an increasing set of tools and indicators for assessing individual researcher’s output and impact—such as Google Scholar and the h-index—were developed and made easily available. While in some cases bibliometric assessments were performed to complement peer review in the allocation of research funding—such as the BOF-key in Flanders (Belgium) [ 4 ] or the Research Assessment Exercise/Framework in the UK—they have, in other settings, become the only means through which research is assessed and funded [ 5 ]. Various publication-based and citation-based funding models can now be found in Australia, Norway, Denmark, Sweden and Finland—and translates as the ‘currency’ through which academic exchanges of tenure, promotion and salary raises are made [ 6 ].

The institutionalisation of these evaluations led many researchers to put large emphasis on the number of papers they published. This has led to adverse effects [ 7 – 10 ]. Indeed, like any social group, researchers might be prone to change their behaviour once the rules of the game or what is expected from them become explicit; a phenomenon that could be referred to as the Hawthorne effect [ 11 ], and that can be associated with Goodhart’s [ 12 ] and Campbell’s [ 13 ] laws. As most evaluations and rankings are based on numbers of papers published, this has created incentives for researchers to publish as many papers as possible . For instance, in Australia, where publication counts were used without any emphasis on publication venue or citations, researchers have been found to increase the numbers of publications in journals with high acceptance rates and lower impact [ 14 ].

In this research evaluation culture of quantity, researchers may have adopted different publication strategies. For example, some researchers might focus on publishing few, high-quality papers—e.g. being more ‘selective’ [ 15 ] or ‘perfectionist’ [ 16 ], while some others may aim at publishing as many papers as possible, irrespective of their quality—e.g. ‘mass producers’ [ 16 ] or ‘big producers’ [ 17 ]. The practice of publishing as many papers as possible—often referred to as ‘salami slicing’—has been long discussed in the literature [ 18 – 20 ]. However, only a few authors have analysed the effect of these publication behaviour on citations received, or more generally, the relationship between research output and scientific impact at the individual researchers’ level. For instance, using a sample of 99 male scholars at prestigious US universities, Feist [ 21 ] showed that eminence—defined as a mix of peer assessed creativity, visibility and honours received—was most likely associated with scholars who publish a lot of papers, rather than with selective scholars. Similarly, Hanssen and Jørgensen [ 22 ] analysed the effect of ‘experience’ on papers’ citations; experience being defined as the author’s previous number of publications. Drawing a sample of papers in transportation research (N = 779) they showed that experience is a statistically significant determinant of individual papers’ citations, although this increase becomes marginal once a certain threshold is met in terms of papers previously published. Supporting this, Bornmann and Daniel [ 23 ] have shown, for a small sample of PhD research projects in biomedicine (N = 96), that an increase in the number of papers associated with a project leads to an increase in the total citation counts of the set of papers. Finally, using 74,000 Swedish publications and 48,000 authors for the period 2008–2011, a strong relationship between scholars’ research output and their probability of producing highly cited publications was found, even when fractional counting of papers is used [ 24 ].

This paper expands on such previous work, with a larger dataset and over a longer time period. Indeed, using a large dataset of distinct disambiguated researchers (N = 28,078,476) who published at least one paper during the 1980–2013 period, this paper aims to better understand the relationship between publication activity and scholarly impact. More specifically, it aims to answer the following research question: what is the relationship between research productivity and scholarly impact? Is the share of scholars’ top papers increasing, stable, or declining, as their research productivity increases? In other words, can scholars be too productive? A good analogy for this is archery: if an archer throws one arrow, what is the probability that it hits the center of the target? Does an increase in the number of arrows thrown leads to an increase in the proportion arrows hitting the center of the target? Our working hypothesis is that authors with higher number of papers would also publish a higher proportion of top cited papers. Such hypothesis would be in agreement with Merton’s theory of cumulative advantage [ 25 ], and supported by the empirical work in the sociology of science [ 16 ]. Similarly, in a Bourdieusian framework, the main goal of a researcher is to increase its rank in the scientific hierarchy and gain more scientific capital [ 26 ]. If publishing a high number of scientific papers and being abundantly cited are the ways through which researchers can reach this goal, then they will adapt their behaviour to reach these evaluation criteria.

This paper uses Thomson Reuters’ Web of Science (WoS) database for the period 1980–2013. Only journal articles are included. Given that the unit analysed in this paper are individual researchers, we used the author disambiguation algorithm developed by Caron & van Eck [ 27 ] to identify the papers authored by individual researchers. On the whole, the algorithm managed to attribute papers to 28,078,476 individuals who have published their first paper between 1980 and 2013. In order to assess differences across disciplines, and to take into account scholars’ publication and citation patterns [ 28 ]—we categorized researchers into four disciplinary domains based on their publication venues: 1) law, arts, and humanities, 2) medical and life sciences, 3) natural sciences, and 4) social and behavioral sciences. Such categorization was based on a reclassification of WoS Subject Categories into four main disciplinary categories.

Three methods were tested to take into account the fact that researchers might publish in more than one of the four broad disciplinary domains. A first method was to consider author-domain combinations [ 29 ] as distinct entities, and to divide a researcher in as many entities (MAX = 4) as there are domains in which his or her papers were published. The disadvantage of such method for this paper is that it splits those who have published in more than one broad field into different “researchers,” which, in turn, reduces their total research output. It also increases the number of scholars analysed by 12.2% to 31,490,527. A second method was to use the broad discipline in which a scholar was the most active in terms of number of papers as his/her main discipline, and assign all of his/her papers into that discipline. In the case of a tie—an author who has published the same maximum number of papers in two fields—the field of the researcher was chosen randomly among the fields with the highest number of papers. Finally, a third method was only to count the researcher in the field in which he/she has the highest number of papers, and to restrict the papers analysed to those in that discipline. Although all three methods yielded similar results because of the large numbers involved, we chose the second method, as it includes all the research output of a scholar into the field that is most likely to be her/his main one.

As our aim is to assess researchers’ contribution to papers that have the highest impact, we isolated the top 1% most cited papers published each year for each discipline (normalised by WoS subject categories). Citations were counted until the end of 2013, and any overlap between the groups of citing and of cited authors (self-citations) were excluded. Hence, the paper only focuses on ‘external citations’ as these are the most relevant for evaluative purposes [ 30 ].

Fig 1 presents the number of scholars analysed in the paper, according to their year of first publication. The fact that the database starts in 1980 explains why we observe a decrease in the number of new scholars for the first few years of the database (as all 1980 scholars are de facto “new” scholars). For disciplines of the natural and medical sciences (Panel A), we observe a steep increase of the number of new scholars from the mid-1980s onwards. For the social and behavioral sciences (Panel B), the early 1980s decrease is followed by a relatively stable number of new authors, until the early 2000s when the number of new authors increase from about 40,000 to 70,000 annually. For law, arts, and humanities, the number of new authors is relatively stable, if not slightly decreasing, throughout the period.

• PPT PowerPoint slide
• PNG larger image
• TIFF original image

https://doi.org/10.1371/journal.pone.0162709.g001

S1 Table presents the main descriptive values for the dataset of scholars studied, focusing on the two main variables under study (i.e. the total number of papers and the share of top 1% articles). As one could expect, differences in research productivity are observed across disciplines, a pattern already documented by Ruiz-Castillo & Costas [ 29 ]. Similarly, some differences in the proportion of top 1% publications can also be observed across disciplines and cohorts. Interestingly, we observe an increase in the number of active scholars between cohort 1981–1985 and 2009–2013 in all disciplines but Law, arts and humanities, which suggest that the coverage of the WoS has, in those disciplines, actually decreased over the last 30 years.

Fig 2 presents, for the oldest cohort studied—researchers who have published their first paper between 1981 and 1985—the relationship between the number of papers throughout their career and the proportion of those papers that made it to the top 1% most cited. For any specific number of papers, the expected value of top 1% papers is, as one might expect, 1%. For each of the four broad domains, authors with very few papers are, on average, much less likely to publish high shares of top 1% most cited papers. But, even more importantly, we also see that for each domain, there is an increase in the proportion of top 1% most cited papers as researchers’ output increase. The strength of the relationship, however, varies by domain, with a stronger relationship in medical and life sciences (R 2 = 0.83) followed by natural sciences (R 2 = 0.73), social and behavioral sciences (R 2 = 0.68) and then by the social sciences and law, arts and humanities (R 2 = 0.57).

Only classes of numbers of papers with 30 researchers or more are shown. Power trendlines and R2 are were obtained using the Excel software.

https://doi.org/10.1371/journal.pone.0162709.g002

More specifically, medical and life sciences scholars with less than 10 papers generally publish less than 1% of their papers in the top 1% most cited groups of papers, the share of top papers increases with productivity, and reaches values between 2% and 3% for scholars who published more than 200 papers. In natural sciences, the increase is not as fast, with scholars who, roughly, publish less than 45 papers still being less likely to publish a high share of top papers. Along these lines, the results are also more scattered, with the group of scholars publishing 150 papers or more obtaining shares of top 1% most cited papers between 1% and 2.5%. In the social and behavioral sciences, very fast increase is seen in terms of shares of top cited papers: researchers with 4 papers or more already punch above their weight , and between 1.5% and 4% papers from researchers who published between 40 and 60 papers were in the top 1% most cited papers. Similar trends can be found in law, arts and humanities, with scholars who have published more than one paper having a share of top papers above average, and scholars with 10–20 papers obtaining a share of top papers between 1.3% and 2.6%.

When the subset of younger researchers who have published their first paper between 2009 and 2013 is considered, different patterns are observed ( Fig 3 ). For medical and life sciences, there is an increase in the share of highly cited publications—although with percentages that are lower than those observed for the oldest cohort—slightly after 15 publications, when decreasing returns to scale are observed. And contrary to the older cohort of scholars from this domain, it is those with higher numbers of papers (>30) that obtain lower shares of top papers. For natural sciences, the trend is similar to that obtained for the older cohort, with larger shares of top papers associated to higher levels of research production. In this domain, scholars with high levels of production (e.g. >40 papers) reach shares of top papers that are sometimes between 5 and 7 times world average. For both social and behavioral sciences, and law, arts and humanities, we observe increases in the proportion of top papers as output rises but, in a manner similar to medical and life sciences researchers, decreasing returns to scale (i.e. lower shares of top papers for higher levels of production) are observed.

Only classes of numbers of papers with 30 researchers or more are shown. Power trendlines and R2 are used for natural sciences and social and behavioral sciences, while 2nd order polynomials are used for medical and life sciences, and law, arts and humanities.

https://doi.org/10.1371/journal.pone.0162709.g003

An important characteristic of this cohort is that it got socialized to research recently—when the evaluation culture was more present—which might make them prone to publish as much as possible. However, the drop in the share of top papers observed in each domain except natural sciences suggests that these academically-younger scholars struggle to keep impact high once a certain threshold is met. This might be due to the fact that these scholars have not yet secured permanent or tenure positions and, thus, might feel that they cannot be as selective as older scholars

In order to take into account scholars’ various career paths, we compiled the proportion of top 1% most cited papers as a function of career length ( Fig 4 , panel A) for all cohorts combined ( Fig 4 , panel B). Career length analysis shows that longer careers are associated with a higher proportion of papers published in the top 1% most cited, irrespective of the domain. More specifically, authors with very short careers—which account for the majority of scholars and are likely to be occasional or transient authors who have performed master or doctorate degrees without remaining in research—have systematically lower shares of top papers. As career length increases, shares of top papers increase slightly for natural sciences and medical and life sciences, and at a faster pace for social and behavioral sciences as well as for law, arts and humanities. For all domains but the latter, the longest career length is associated with the highest proportion of top 1% most cited papers. Along these lines, annual numbers of papers are also associated with higher levels of scientific impact. While the positive relationship between the two variables is quite clear for social and behavioral sciences, we observe a flattening of the curve at about 10 papers for medical and life sciences and natural science followed, for the latter field, by an increase in top papers for annual numbers of papers between 30 and 50. Law, arts and humanities, however, follows a different pattern, with scholars having less than one paper/year obtaining the highest proportion of top papers. Although not shown, similar patterns were obtained when only looking at specific cohorts.

A) As a function of career length, B) As a function of their annual number of papers (rounded). Only groups with 30 researchers or more are shown.

https://doi.org/10.1371/journal.pone.0162709.g004

Discussion and Conclusion

Previous research has shown that, in many contexts, the focus on quantitative indicators for research evaluation has had adverse effects [ 7 ]. This paper aimed to provide an original analysis of one of these: to publish as much as possible . Our results have shown that, for older researchers, the higher the number of papers published, the more likely those ended up being amongst the most cited papers of their discipline. For younger scholars, however, such relationship could only be observed in the natural sciences and, to a certain extent, in the social and behavioral sciences. For the two other broad domains, higher scientific output was associated with decreasing shares of highly cited publications.

Several factors can contribute to the explanation of this pattern. For instance, authorship criteria [ 31 ] and co-authorship patterns [ 32 ] are different in disciplines of the natural sciences and of medical and life sciences than in the social and behavioral sciences and law, arts and humanities, which might explain why senior researchers are more likely to contribute to many papers (and many top cited ones). Similarly, while this study takes age of academics into account—which has been shown to be a key driver of scholarly output and impact [ 33 , 34 ]—there are several other factors that have been shown to affect research output and impact, such as interdisciplinary, gender, funding, country, and database used [ 28 , 30 , 35 – 38 ]. The thorough exploration of all these factors is obviously beyond the objectives and possibilities of this paper, and further research should help to clarify how these factor interact and play a role in the patterns observed here.

From a theoretical point of view, these results conform to the Mertonian theory of cumulative advantages [ 25 ]: the higher the number of papers an author contributes to, the more he/she gets known and, hence, is likely to further attract citations. In Bourdieusian terms [ 26 ], the more an author publishes and accumulates citations in a domain, the more this capital will yield additional papers and citations. The relationship could also be in the other direction, as authors with a lot of scientific capital might have more opportunities to contribute to papers (e.g. through collaboration, funding, etc.). Still, the results show that highly productive authors, on average, also contribute to more highly cited papers; the fact that this is not consistently observed for younger researchers might be due to the fact that this age group has not had enough time to stratify [ 16 ] itself into different categories of scholars. Finally, from a practical point of view, the interdependencies between absolute indicators—such as number of publications—and relative indicators—such as the proportion of top cited papers—reinforce the idea of research performance as a multidimensional concept [ 39 ], difficult to measure by a reduced set of indicators. This supports the idea of contextualize research performance as a complex ecosystem of different types of scholars, activities, abilities and relationships [ 40 ], as well as the idea raised in the Leiden Manifesto [ 41 ] that individuals are best assessed on a qualitative judgment of their portfolio, combining several indicators and information about their activities.

Supporting Information

S1 table. main descriptive values for scholars covered by the analysis..

https://doi.org/10.1371/journal.pone.0162709.s001

Acknowledgments

This paper is an extended version of a proceeding presented at the 2015 conference of the International Society for Scientometrics and Informetrics. The authors thank the referees for their useful comments and suggestions.

Author Contributions

• Conceptualization: VL RC.
• Formal analysis: VL RC.
• Funding acquisition: VL.
• Investigation: VL RC.
• Methodology: VL RC.
• Software: VL RC.
• Validation: VL RC.
• Visualization: VL RC.
• Writing – original draft: VL RC.
• Writing – review & editing: VL RC.
• 1. Dahler-Larsen P. The evaluation society. Palo Alto: Stanford University Press; 2011
• 2. Bush V. Science: The Endless Frontier: A Report to the President on a Program for Postwar Scientific Research. Washington, DC: National Science Foundation; 1945. Available: https://www.nsf.gov/od/lpa/nsf50/vbush1945.htm
• View Article
• Google Scholar
• PubMed/NCBI
• 7. Binswanger M. How Nonsense Became Excellence: Forcing Professors to Publish. In: Welpe IM, Wollersheim J, Ringelhan S, Osterloh M, editors. Incentives and Performance. Springer International Publishing; 2015 p. 19–32.
• 8. Frey BS, Osterloh M. Evaluations: Hidden costs, questionable benefits, and superior alternatives. Institute for Empirical Research in Economics, University of Zurich. 2006
• 9. Haustein S, Larivière V. The use of bibliometrics for assessing research: possibilities, limitations and adverse effects. In: Welpe IM, Wollersheim J, Ringelhan S, Osterloh M, editors. Incentives and Performance. Springer International Publishing; 2015 p. 121–39
• 11. Gillespie R. Manufacturing knowledge: a history of the Hawthorne experiments. Cambridge University Press; 1993.
• 12. Goodhart CAE. Problems of Monetary Management: The U.K. Experience. Papers in Monetary Economics (Reserve Bank of Australia); 1975.
• 16. Cole JR, Cole S. Social stratification in science. Chicago: University of Chicago Press; 1973.
• 24. van den Besselaar P, Sandstrom U. Does Quantity Make a Difference? The importance of publishing many papers. 2015;arXiv:1510.01871.
• 26. Bourdieu P. Science of science and reflexivity. Cambridge, UK: Polity Press; 2004.
• 27. Caron E, van Eck NJ. Large scale author name disambiguation using rule-based scoring and clustering. In 19th International Conference on Science and Technology Indicators. Context counts: Pathways to master big data and little data. CWTS-Leiden University Leiden; 2014 p.79–86.
• 31. Pontille D, Fraenkel B. La signature scientifique: Une sociologie pragmatique de l'attribution. Paris: CNRS; 2004.
• 40. Costas R, Noyons E. From publications to people: bibliometric benchmarking of a selection of countries in the Life Sciences based on individual-level bibliometrics. STI 2014 Leiden, 2014;114–22.

Boosting productivity

Research identifies small changes that lead to big improvements in performance

By Heather Stringer

September 2017, Vol 48, No. 8

Print version: page 54

• Learning and Memory
• Neuropsychology
• Perception and Attention

When Larry Rosen, PhD, talks to people who want to improve their productivity, he zeroes in on the importance of minimizing interruptions. Rosen, professor emeritus of psychology at California State University, Dominguez Hills, goes as far as to suggest that people put up a "do not disturb" sign when they need to focus on a task.

While this may not be plausible for everyone, Rosen's studies have shown how being distracted can become a bad habit that ultimately decreases our effectiveness at work or in school.

Fortunately, he and other psychology researchers have identified new ways to help people overcome the hurdles that stand in the way of their productivity, whether they are personal habits or environmental challenges. Here are some of those findings.

Grow your attention span

Even though technology can empower us to accomplish things faster, Rosen has found that those benefits can disappear when digital distractions are so readily available.

While it can be tempting to think that dealing with these messages is productive, Rosen says this is a false sense of effectiveness. "We may think we are multitasking, but we are really task-switching," he says. "These interruptions take us away from the task at hand." The original task becomes less salient in our brains, and when we return, we waste time trying to remember what we were thinking when we left, Rosen explains.

To increase attention span and productivity, one of Rosen's solutions is the "technology break." He encourages students and workers to give themselves a couple of minutes to check alerts, texts and other messages after 15 minutes of undistracted work. The best way to stay focused is silencing the phone, turning it face down to avoid seeing visual notifications, turning off email alerts and closing distracting websites, Rosen says.

"Once you learn how to work for 15 minutes, start increasing the time before taking a technology break," Rosen says.

Taking short breaks not only satisfies the technology fix, but it also allows us to maintain focus, according to a study conducted by Alejandro Lleras, PhD, a psychology professor at the University of Illinois at Urbana–Champaign. He found that participants who took a short break while focusing on a visual task maintained the same level of performance for 40 minutes, but performance declined for those who didn't take any breaks ( Cognition , Vol. 118, No. 3, 2010).

"We know that after about 30 minutes, concentration starts to decrease, so it's important to take small breaks to stay focused on your main task."

The results also showed that the breaks can be surprisingly short—only a couple of seconds for some tasks—to achieve this effect.

Write out your goals

Many people who work are familiar with the idea of setting goals for themselves, but achieving those goals can be elusive. Research is showing that establishing a habit of writing about goals can boost performance.

Cheryl Travers, PhD, a professor at the School of Business and Economics at Loughborough University in Leicestershire, England, asked students to identify areas where they needed to improve, such as raising a grade in a class or increasing concentration while studying. The students were asked to visualize desired outcomes and outline how they could put their goals into practice.

Then the students kept diaries for three months to reflect on their goal progress. For example, students could write down what happened as they attempted to make a change in a particular situation, what worked well or not well, what could have been done better and actions they could take going forward. Travers found that the reflective goal-related writing had a significant impact on their ability to perform better academically ( British Journal of Educational Psychology , Vol. 85, No. 2, 2015).

"The act of writing something down seems to make us accountable to a goal," Travers says. "It also helps people to write their way through a problem when they encounter barriers."

By writing about successes and failures and thinking about strategies to overcome difficulties, students gained confidence in themselves and developed academic self-efficacy, Travers says. What was particularly interesting was the evidence showing that academic performance improved even for students who set nonacademic, "softer" goals, such as "increase my assertiveness" or "decrease stress."

Travers is now collecting diaries for managers in organizations, and she will be studying whether this reflective goal setting improves their effectiveness as leaders. "This process allows people to essentially become self-coaches because they are continually evaluating goal outcomes and becoming more self-aware about leader behaviors."

Get together

The idea of fitting in another meeting may seem counter-productive for people working in group settings, but research ­suggests that taking time to debrief as a team can improve productivity in the long run.

Michaela Schippers, PhD, a professor of behavior and performance management at the Rotterdam School of Management at Erasmus University, studied teams working in a health-care environment. She found that the groups that met regularly to evaluate work processes were much more likely to come up with innovative solutions to problems than groups that did not meet regularly. Her work has shown how this team reflexivity (reflecting on team functioning) can significantly improve work performance levels ( Journal of Management , Vol.41, No. 3, 2015; Journal of Organizational Behavior , Vol. 34, No. 1, 2013).

"Workers should have regular debriefings, like in the military, but the purpose is not to point out what people are doing wrong," Schippers says. "Instead, the group can brainstorm how to improve as a team, ideally with a facilitator who is leading the meeting."

In the study, the reflexive teams talked about issues such as decreasing waiting times for patients, improving patient record systems and developing a more effective appointment system.

"Our work showed that it was very important for teams that are particularly busy to meet regularly to debrief, because these teams benefited most from the innovative improvements," Schippers says. "The meetings gave them space to think collectively about what could be changed."

Get out of the chair

Researchers are finding that employees with stand-­capable workstations may be more productive than their seated counterparts. Mark Benden, PhD, a professor in the department of environmental and occupational health at Texas A&M School of Public Health, studied two groups of call center employees over six months. One group sat at traditional desks and the other group at stations that enabled workers to elevate their tables whenever they wanted to stand. Benden found that those with stand-capable workstations stood about 1.5 hours longer per day and were 42 percent more productive than those who worked at seated desks. Productivity was measured by how many successful calls the workers completed per hour ( IIE Transaction on Occupational Ergonomics and Human Factors , Vol. 4, No. 2-3, 2016).

"By being up more of the time, we improve blood flow to the brain and circulation to the body, and these things combine to make the brain more active and engaged," Benden says.

Research also suggests that it is important for people to avoid "static standing" in one place, he says. The best stand-­capable workstations have foot rails that allow workers to take weight off of one side of the body. If it's not possible to get this type of workstation, workers should take breaks to walk around and get out of the chair, Benden says.

Further reading

The Distracted Mind: Ancient Brains in a High-Tech World Gazzaley, A., & Rosen, L., 2016

Managing Motivation: A Manager's Guide to Diagnosing and Improving Motivation Pritchard R.D., & Ashwood, E.L., 2008

Evidence-Based Productivity Improvement: A Practical Guide to the Productivity Measurement and Enhancement System Pritchard, R.D., Weaver, S.J., & Ashwood, E.L., 2012

Future Time Perspective and Promotion Focus as Determinants of Intraindividual Change in Work Motivation Kooij, D.T., Bal, P.M., & Kanfer, R., Psychology and Aging , 2014

Letters to the Editor

• Send us a letter

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List

Workplace Stress and Productivity: A Cross-Sectional Study

1 University of Oklahoma at Tulsa, Tulsa, OK

Rosey Zackula

2 Office of Research, University of Kansas School of Medicine-Wichita, Wichita, KS

Katelyn Dugan

3 Department of Population Health, University of Kansas School of Medicine-Wichita, Wichita, KS

Elizabeth Ablah

Introduction.

The primary purpose of this study was to evaluate the association between workplace stress and productivity among employees from worksites participating in a WorkWell KS Well-Being workshop and assess any differences by sex and race.

A multi-site, cross-sectional study was conducted to survey employees across four worksites participating in a WorkWell KS Well Being workshop to assess levels of stress and productivity. Stress was measured by the Perceived Stress Scale (PSS) and productivity was measured by the Health and Work Questionnaire (HWQ). Pearson correlations were conducted to measure the association between stress and productivity scores. T-tests evaluated differences in scores by sex and race.

Of the 186 participants who completed the survey, most reported being white (94%), female (85%), married (80%), and having a college degree (74%). A significant inverse relationship was observed between the scores for PSS and HWQ, r = −0.35, p < 0.001; as stress increased, productivity appeared to decrease. Another notable inverse relationship was PSS with Work Satisfaction subscale, r =−0.61, p < 0.001. One difference was observed by sex; males scored significantly higher on the HWQ Supervisor Relations subscale compared with females, 8.4 (SD 2.1) vs. 6.9 (SD 2.7), respectively, p = 0.005.

Conclusions

Scores from PSS and the HWQ appeared to be inversely correlated; higher stress scores were associated significantly with lower productivity scores. This negative association was observed for all HWQ subscales, but was especially strong for work satisfaction. This study also suggested that males may have better supervisor relations compared with females, although no differences between sexes were observed by perceived levels of stress.

INTRODUCTION

Psychological well-being, which is influenced by stressors in the workplace, has been identified as the biggest predictor of self-assessed employee productivity. 1 The relationship between stress and productivity suggests that greater stress correlates with less employee productivity. 1 , 2 However, few studies have examined productivity at a worksite in relation to stress.

Previous research focused on burnout, job satisfaction, or psychosocial factors and their association with productivity; 3 – 7 all highlight the importance of examining overall stress on productivity. Other studies focused on self-perceived stress and employer-evaluated job performance instead of self-assessed productivity. 8 However, most studies examining this relationship have been occupation specific. 8 , 9 Larger studies examining this relationship were performed in other countries. 1 , 5 , 9 , 10

The purpose of this study was twofold. First, the study sought to elucidate the relationship between stress and productivity in four worksites in Kansas. Second, the study sought to examine potential differences in stress and productivity by sex and race.

Recruitment and Sampling Procedures

The target population was employees from four WorkWell KS worksites. WorkWell KS is a statewide worksite initiative in Kansas that provides leadership and resources for businesses and organizations to support worksite health. Because access to employee emails was unavailable, a URL link to an online survey was sent to the worksite contact, who was responsible for ensuring the distribution of the URL link to a cross-section of employees at the worksite. Following a WorkWell KS workshop (held in Topeka, Kansas on November 6, 2017) attendees from the four worksites were recruited to distribute a link to an online survey to their employees. Workshop attendees were members of wellness committees or were worksite representatives. Employee responses to the online survey were collected through mid-December 2017. No compensation was given for disseminating the survey link or for participating in the study. This study was approved by the University of Kansas School of Medicine-Wichita’s Human Subjects Committee.

Online Survey

The online survey comprised demographic items with two instruments, the Perceived Stress Scale (PSS), 11 and the Health and Work Questionnaire (HWQ). 12 Demographic items included employee, sex, race, age, marital status, and highest level of education completed.

Perceived Stress Scale

Stress was measured by the PSS, a 10-item questionnaire designed for use in community samples. The purpose of the instrument is to assess global perceived stress during the past month. Each item is measured with a Likert-type scale (0 = Never, 1 = Almost Never, 2 = Sometimes, 3 = Fairly Often, 4 = Very Often). This scale is reversed on four positively stated questions. Scoring of the PSS is obtained by summing all responses. Results range from zero to 40, with higher PSS scores indicating elevated stress: scores of 0 – 13 are considered low stress, 14 – 26 moderate stress, and 27 – 40 are high perceived stress. The results for perceived stress were used by this study as an indication of psychological well-being.

Health and Work Questionnaire

The HWQ is a 24-item instrument that measures multidimensional worksite productivity. Productivity is assessed by asking respondents how they would describe their efficiency, overall quality of work, or overall amount of work in one week. All items are scaled with Likert-type response anchors, each ranging from 1 to 10 points. Most are positively worded items with response scales from least (scored as a 1) to most favorable (scored as a 10). Exceptions are items 1 and 16 through 24, which are negatively worded and reversed scored. Items are divided into six sub-scales: productivity, concentration/focus, supervisor relations, non-work satisfaction, work satisfaction, and impatience/irritability. As part of the HWQ, employees assessed productivity two ways: on themselves and how their supervisor or co-workers might perceive it. Accordingly, productivity is stratified into a self-assessed sub-score and perceived other-assessed sub-score. HWQ scores are tallied and averaged for each sub-scale, with higher scores generally indicating greater productivity.

The Consent Process

Representatives who participated in the WorkWell KS workshop sent an e-mail to their employees with a request to click on the link and complete the online survey. The link opened the electronic consent, which was the opening remark, followed by the two assessment instruments and the demographic items. Consent was implied by participation in the survey. To encourage survey participation, representatives also sent employees a few e-mail reminders at their own discretion.

Statistical Analysis

The statistical analysis included descriptive statistics, measures of association, and comparisons of survey responses by sex and race. Descriptive statistics comprised response summaries; means and standard deviations were used for continuous variables, while frequency and percentages were used for categorical responses. The relationship between stress and productivity measures were assessed using Pearson correlations. Sex and race comparisons for PSS and HWQ subscales were evaluated using two-sided t-tests; alpha was set at 0.05 as the level of significance. Study participants with missing values were excluded pairwise from the analysis.

Response Rates

Four of nine worksites participated in the study, including two health departments (89 participants), one school district (76 participants), and one non-profit for the medically underserved (21 participants). A total of 188 employees opened the survey link, 186 employees answered the first question of the survey, and 174 employees completed the survey items. The 12 study participants with missing values were excluded from the pairwise analysis. The response rate, defined as those participants who completed the survey, was 58.6% (n = 174). To protect the confidentiality of respondents, data were aggregated and no other comparisons were made by location.

Participants who completed the survey included 174 employees from four worksites in Kansas. Of those who responded, 94% (155 out of 165) reported being white, 85% (142 of 167) reported being female, 81% (124 of 153) reported being between 30 and 59 years, and 60% (99 of 166) reported having a bachelor’s degree or higher ( Table 1 ).

Participant demographics.

With regard to measures of stress, the mean PSS was 16.4, with a standard deviation of 6.2, suggesting that employees have moderate levels of stress at these locations. This result was consistent with the HWQ question regarding “overall stress felt this week”, with a mean score of 4.7 (SD 2.5; 10 is “very stressed”). Regarding measures of productivity, the mean overall HWQ was 6.3 (SD 0.7). With the exception of reverse items, as noted below, scores of 10 indicated high levels of productivity. Mean scores by scale were: 7.3 (SD 1.0) for overall productivity, with 7.5 (SD 1.3) for own assessment, and 7.5 (SD 1.2) for perceived other’s assessment; 7.1 (SD 2.7) supervisor relations, 7.8 (SD 1.8) for non-work satisfaction, and 7.3 (SD 1.7) for work satisfaction. The mean scale for the reverse items scores were concentration/focus at 3.4 (SD 2.0), and impatience/irritability 3.2 (SD 1.6).

Correlations between the PSS and the HWQ subscales ranged from −0.61 to 0.55 ( Table 2 ). A negative association was observed between the PSS and the overall HWQ, r(177) = − 0.35, p < 0.001. While each of the positively-coded HWQ subscales was associated negatively with the PSS, the strongest correlation occurred between work satisfaction and PSS, r(177) = −0.61, p < 0.001, suggesting that as stress increases work satisfaction declines.

Measures of correlation within and between the PSS and HWQ.

HWQ: Health and Work Questionnaire mean score; PSS: Perceived Stress Scale mean score

In evaluating differences by sex, mean scores were significantly higher for males compared with females for the HWQ Supervisor Relations subscale (8.4 (SD 2.1) versus 6.9 (SD 2.7), respectively; p < 0.005; Table 3 ). No other sex differences were observed for either instrument. Similarly, there were no significant differences by race.

Comparing results of the PSS and the HWQ by sex.

Findings suggested there is an inverse association between overall stress and productivity; higher PSS scores were associated with lower HWQ scores. These findings are consistent with other cross-sectional studies comparing productivity and other measures of psychological well-being. 1 , 8 , 9 , 10 Thus, employer efforts to decrease stress in the workplace may benefit employee productivity levels.

In addition, males scored higher for supervisor relations in the HWQ than females. This finding may suggest that males have stronger relationships with their supervisors. Indeed, there is compelling evidence to suggest the main factor affecting job satisfaction and performance is the relationship between supervisors and employees. 13 Although, this relationship may be mitigated by employee-supervisor interactions of sex, race/ethnicity, status, education, age, support systems, and other factors, none of which were evaluated in the current study.

For example, Rivera-Torres et al. 14 suggested that women with support systems, defined as co-workers and supervisors, experienced less work stress than males. Results from this study seemed to support Rivera-Torres et al. 14 in that females tended to report higher levels of stress compared with males (although not significant) and reported weaker relationships with their supervisors. In addition, Peterson 15 evaluated what employee’s value at work and found that males and females differed significantly. When asked to rank work values, men valued pay/money/benefits along with results/achievement/success most, whereas women valued friends/relationships along with recognition/respect. Perhaps, more research is necessary to understand the nuances between co-worker and supervisor regarding work satisfaction and productivity.

The study contributes to the literature in the use of different metrics for psychological well-being, defined as stress. Multiple organizations within Kansas were evaluated for both productivity and stress. To our knowledge, the PSS and HWQ have never been used together to measure the relationship between stress and productivity. Results suggested that overall productivity (HWQ) was associated with the HWQ “work satisfaction” subscale. Perceived stress also had the strongest inverse relationship with HWQ sub-scale “work satisfaction” when compared with HWQ sub-scale “productivity”.

This study suggested that productivity, stress, and job satisfaction were correlated, therefore, additional research needs to include each of these variables in greater detail as the current literature has been mixed on their relationships and potential collinearity. For example, one study examining two occupations suggested psychological well-being (defined as psychological functioning) was associated with productivity, whereas job satisfaction did not. 7 In contrast, another study suggested that psychological well-being has been a bigger factor in job productivity than work satisfaction alone, but both are associated with job productivity. 9 This current study was able to examine this relationship by using the PSS and the HWQ together.

More research is needed to understand these differences by standardizing terminology. In this study, psychological well-being was defined as stress. However, other studies have defined psychological well-being as happiness or as one’s psychological functioning. 7 , 8 This study also expanded the relationship between psychological well-being and stress. Previous research focused more on the relationship between productivity and burnout or job satisfaction.

This study had limitations such as a small sample size (in number of organizations and number of employees). The sample size assessed small organizations in the United States, whereas many other large scale studies on stress occurred over multiple large organizations in other countries. 1 , 10 There was limited racial diversity in the current study, as 6.1% (10 of 165) reported being non-white. The population studied was also primarily female, limiting the strength of comparisons made between sexes. Furthermore, because worksites often share computers, questionnaires may have been completed using the same IP address; thus, we were unable to prevent multiple entries from the same individual.

The current study did not detect a difference in productivity or stress by race. This differed from other research. For instance, non-whites experience greater overall stress than whites potentially attributable to poorer employment status, income, and education. 16 Non-whites experience stress secondary to racial discrimination. 17 , 18 In one study, when examining productivity among university faculty, non-whites reported greater stress and produced less research (productivity) compared to whites. 16 Further research needs to be conducted on productivity and stress by race and ethnicity, and associated variables, such as employment status, income, education, and occupation, need to be accounted for in analysis. Differences between other research and the current study regarding race may be attributed to the fact that only 6% of respondents who answered race reported being non-white, making racial diversity in this study limited, although representative of the population sampled.

CONCLUSIONS

This study suggested there is a negative correlation between overall stress and productivity: higher stress scores were significantly associated with lower productivity scores. This negative association was observed for all HWQ subscales, but was especially strong for work satisfaction. This study also suggested that males may have better supervisor relations compared to females, although no differences between sexes were observed by perceived levels of stress. There was no difference in productivity or stress by race. The results of this study suggested that employer efforts to decrease employee stress in the workplace may increase employee productivity.