Keywords
impact actors, research quality, information systems, finance, management science, journal demographics, acceptance rates, bibliographic measures, JCR, SJR, SNIP, citation rates
Abstract
Purpose: Faculty research is frequently the basis of pay, tenure, and promotion decisions in the university arena. Meanwhile, perceptions regarding the quantity and quality of the research produced by a faculty is often the basis of departmental, college, and university reputation. The journal in which research findings are published is often used to assess the overall research quality. In order to better benchmark journal quality, this report provides findings of a meticulous investigation of leading journals in the finance, information systems and management science disciplines. It examines four different citation-based measures of quality and four journal characteristics that are exogenous to the quality of any individual piece of research. In unison, these investigative paths provide a clearer understanding of journal quality across the business realm, and hence of the quality of research appearing in business journals. Design: This study assists in the development of an accurate perception regarding business research through a careful analysis of the popular Journal Citation Reports (JCR) impact factor across leading journals in three diverse business disciplines. By considering three newer journal quality metrics, a.) SCImago Journal Rank (SJR), b.) Source Normalized Impact per Paper (SNIP), and c.) Percentage of articles cited, this research builds on past research. Top-tier journals in finance, information systems, and operations research and management science (referred to here as “management science ”) are compared to evaluate the consistency of these measures across disciplines. The differences in journal characteristics and their impact on the citation-rate based measures of quality are also analyzed. Further, the potential impact of a discipline-based variation in the acceptance rate, issue frequency, the time since journal inception, and total reviewers are put forth as additional potential exogenous factors that may influence the perception of the overall journal quality. T-tests are applied for discipline comparisons, while correlation and multiple regression are employed in the analysis of journal characteristics.
Findings: There is a significant difference in the JCR impact measures of high-quality finance and management science journals versus high-quality information systems journals. However, only the JCR measures for finance journals correlate with a variety of journal-specific factors, including the journal's acceptance rate and frequency of issue. The SJR measures for finance and management science journals are, on the other hand, consistently higher than information systems journals, though the SJR value of any individual journal can be quite volatile. Most importantly, finance and management journals also report significant relations between the SJR measures and the journal's acceptance rate and year of initial issue. By comparison, the SNIP metric rates suggest that information systems and management science journals have higher quality. Moreover, underscoring the SNIP metrics for both the base years of the current study, articles in leading information systems and management journals are cited over twelve percentage points more than those in finance journals. Overall, results show that given the metric, the measured variance in the quality of finance, information systems, and management science journals is correlated with the identified journal-specific factors. Research limitations: The present research is limited to three business disciplines, making the examination of journals in other business disciplines a logical extension of it. Whereas this research takes journal quality as fixed, one could also evaluate a quality measures reaction to a variation in journal characteristics (i.e. changes in acceptance rates). Furthermore, one could include other measures of journal quality, comprising the h-index or the more recently-released CiteScore metric. Such research would not only build on the present research, but also improve the accuracy of scholarly outlets and consequently the research quality. Practical implications: Discipline-specific traits should be considered, and adjusted for, when making inferences about the long-term value of recently-published research. Our investigation demonstrates that citation-based research measures and journal-specific factors vary systematically across disciplines, which is why discipline-specific differences in journal characteristics, leading to the differences in citation-based quality measures, need to be considered, when making inferences about the long-term value of recently-published research. As a result, this research has significant implications for the basis upon which recommendations regarding salary adjustments, retention, and promotion are made. Social implications: Research quantity and quality are two hallmarks of leading research institutions. Assessing research quality is very problematic, because its definition has changed from being based on the review process (i.e. “blind refereed”) to currently standing on acceptance rates and impact factors. Furthermore, the impact factor construct has been a lightning rod of controversy among researchers and administrators. Even journals themselves argue over which metric to employ, in the end supporting those putting them in the best light. This research assesses how impact factors and journal characteristics, which may influence the impact factors, vary by business discipline. The research is especially important and relevant to the authors who separately chair faculty departments that include finance, information systems, and management science, and are therefore in roles requiring an assessment of faculty research productivity, including quality.
Originality/value: This study is a detailed analysis of bibliographic aspects of the top-tier journals in three quantitative business areas. In addition to the popular JCR, SJR, and SNIP measures of performance, our analysis studies the seldomexamined percentage of articles cited metric. articles-citation metrics. A deeper understanding of citation-based measures is obtained through an evaluation of changes in how journals have been rated on these metrics over time. Our research shows firstly, that there are discipline-related systematic differences in both citation-based research measures and journal-specific factors, and secondly, that these discipline-specific traits should be considered when making inferences about the long-term value of recently published research. Furthermore, discipline-specific differences in journal characteristics, leading to the differences in citation-based quality measures, should in any case be considered when making personnel and remuneration decisions.
Recommended Citation
Krueger, T. M., Shorter, J. D., & Colvin, R. G. (2021). Bibliographic Measures of Top-Tier Finance, Information Systems, and Management Science Journals. Economic and Business Review, 23(1), 1-14. https://doi.org/10.15458/2335-4216.1001
Creative Commons License
This work is licensed under a Creative Commons Attribution-Noncommercial-No Derivative Works 4.0 License.
