Semantic Clustering of Research Papers Using SBERT and  K-Means to Support the Discovery of Potential Academic Collaborations

Werayuth Charoenruengkit

doi:10.53848/ssstj.v13i2.1445

Authors

Werayuth Charoenruengkit Srinakharinwirot University

DOI:

https://doi.org/10.53848/ssstj.v13i2.1445

Keywords:

SBERT, K-Means clustering, Authorship network, Topic modeling

Abstract

Identifying potential research collaborations is critical for fostering interdisciplinary innovation and academic productivity. Traditional approaches, such as co-authorship network analysis, are limited in scope because they rely solely on past collaborations and fail to capture latent thematic connections among researchers. This paper proposes an integrated analytical framework that applies semantic content analysis to research publications to support the discovery and exploration of potential academic collaborations. We compare SBERT-based K-Means clustering with a traditional TF-IDF K-Means baseline and observe superior performance in capturing semantic relationships, as evidenced by higher Silhouette Scores and lower Davies-Bouldin Index values. The framework was evaluated using a text corpus comprising the titles, abstracts, and keywords of 1,335 articles authored or co-authored by researchers affiliated with a university over the past five years. Using the SBERT-based configuration, the approach identified twelve thematic clusters that correspond to interpretable research domains within the institution. Representative terms extracted for each cluster provide clear thematic boundaries and indicate the primary research focus of each domain. Furthermore, multi-dimensional analysis reveals that departmental affiliation patterns exhibit both expected disciplinary concentrations and cross-departmental research overlaps, while network visualization illustrates the interconnected research landscape and highlights authors who act as bridges between clusters.

References

Bastian, M., Heymann, S., & Jacomy, M. (2009). Gephi: An open source software for exploring and manipulating networks. In Proceedings of the Third International AAAI Conference on Weblogs and Social Media, 3, 361–362.

https://doi.org/10.1609/icwsm.v3i1.13937

Cao, M. D., & Gao, X. (2005). Combining contents and citations for scientific document classification. Lecture Notes in Computer Science, 3809, 143–152.

Davies, D. L., & Bouldin, D. W. (1979). A cluster separation measure. IEEE Transactions on Pattern Analysis and Machine Intelligence, PAMI-1(2), 224–227.

https://doi.org/10.1109/TPAMI.1979.4766909

de Paula Fonseca e Fonseca, B., Sampaio, R. B., de Araújo Fonseca, M. V., & Zicker, F. (2016). Co-authorship network analysis in health research: Method and potential use. Health Research Policy and Systems, 14, Article 34. https://doi.org/10.1186/s12961-016-0104-5

Fagan, J., Eddens, K. S., Dolly, J., Vanderford, N. L., Weiss, H., & Levens, J. S. (2018). Assessing research collaboration through co-authorship network analysis. Journal of Research Administration, 49(1), 76–99.

Fantechi, A., Gnesi, S., Livi, S., & Semini, L. (2021). A spaCy-based tool for extracting variability from NL requirements. In Proceedings of the 25th ACM International Systems and Software Product Line Conference — Volume B (p. 32–35).

Färber, M., & Jatowt, A. (2020). Citation recommendation: Approaches and datasets. International Journal on Digital Libraries, 21(4), 375–405. https://doi.org/10.1007/s00799-020-00288-2

Ferligoj, A., Kronegger, L., Mali, F., Snijders, T. A., & Doreian, P. (2015). Scientific collaboration dynamics in a national scientific system. Scientometrics, 104(3), 985–1012.

https://doi.org/10.1007/s11192-015-1585-7

Glänzel, W., & Schubert, A. (2005). Analysing scientific networks through co-authorship. In H. F. Moed, W. Glänzel, & U. Schmoch (Eds.), Handbook of quantitative science and technology research (pp. 257–276). Springer. https://doi.org/10.1007/1-4020-2755-9_12

Griffiths, T. L., & Steyvers, M. (2004). Finding scientific topics. Proceedings of the National Academy of Sciences, 101(Suppl. 1), 5228–5235. https://doi.org/10.1073/pnas.0307752101

Grootendorst, M. (2022). BERTopic: Neural topic modeling with a class-based TF-IDF procedure. arXiv. https://doi.org/10.48550/arXiv.2203.05794

Gropp, C., Herzog, A., Safro, I., Wilson, P. W., & Apon, A. W. (2019). Clustered latent Dirichlet allocation for scientific discovery. In 2019 IEEE International Conference on Big Data (Big Data) (pp. 4503–4511). IEEE. https://doi.org/10.1109/BigData47090.2019.9005964

Ikonomakis, E. K., Tasoulis, D. K., & Vrahatis, M. N. (2019). Density based text clustering. In Recent Progress in Computational Sciences and Engineering (2 vols, pp. 218–221). CRC Press.

Kim, J., Tao, L., Lee, S.-H., & Diesner, J. (2016). Evolution and structure of the scientific co-publishing network in Korea between 1948–2011. Scientometrics, 107(1), 27–41.

https://doi.org/10.1007/s11192-016-1878-5

Kim, S.-W., & Gil, J.-M. (2019). Research paper classification systems based on TF-IDF and LDA schemes. Human-centric Computing and Information Sciences, 9, Article 30.

https://doi.org/10.1186/s13673-019-0192-7

Kumar, S. (2015). Co-authorship networks: A review of the literature. Aslib Journal of Information Management, 67(1), 55–73. https://doi.org/10.1108/AJIM-09-2014-0116

Moghadasi, M. N., & Zhuang, Y. (2020). Sent2Vec: A new sentence embedding representation with sentimental semantic. In 2020 IEEE International Conference on Big Data (Big Data) (pp. 4672–4680). IEEE. https://doi.org/10.1109/BigData50022.2020.9378337

Mu, W., Lim, K. H., Liu, J., Karunasekera, S., Falzon, L., & Harwood, A. (2022). A clustering-based topic model using word networks and word embeddings. Journal of Big Data, 9, Article 38. https://doi.org/10.1186/s40537-022-00585-4

Newman, M. E. J. (2001). The structure of scientific collaboration networks. Proceedings of the National Academy of Sciences, 98(2), 404–409. https://doi.org/10.1073/pnas.98.2.404

Probierz, B., Kozak, J., & Hrabia, A. (2022). Clustering of scientific articles using natural language processing. Procedia Computer Science, 207, 3443–3452. https://doi.org/10.1016/j.procs.2022.09.403

Radu, R.-G., Rădulescu, I.-M., Truică, C.-O., Apostol, E.-S., & Mocanu, M. (2020). Clustering documents using the document to vector model for dimensionality reduction. In 2020 IEEE International Conference on Automation, Quality and Testing, Robotics (AQTR) (pp. 1–6). IEEE. https://doi.org/10.1109/AQTR49680.2020.9129967

Reimers, N., & Gurevych, I. (2019). Sentence-BERT: Sentence embeddings using Siamese BERT-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing (pp. 3982–3992). Association for Computational Linguistics. https://doi.org/10.18653/v1/D19-1410

Rousseeuw, P. J. (1987). Silhouettes: A graphical aid to the interpretation and validation of cluster analysis. Journal of Computational and Applied Mathematics, 20, 53–65.

https://doi.org/10.1016/0377-0427(87)90125-7

Schäfermeier, B., Hirth, J., & Hanika, T. (2023). Research topic flows in co-authorship networks. Scientometrics, 128, 5051–5078. https://doi.org/10.1007/s11192-022-04529-w

Song, K., Tan, X., Qin, T., Lu, J., & Liu, T.-Y. (2020). MPNet: Masked and permuted pre-training for language understanding. In Proceedings of the 34th International Conference on Neural Information Processing Systems (pp. 16857–16867).

Subakti, A., Murfi, H., & Hariadi, N. (2022). The performance of BERT as data representation of text clustering. Journal of Big Data, 9, Article 15. https://doi.org/10.1186/s40537-022-00564-9

Ullah, M., Shahid, A., ud Din, I., Roman, M., Assam, M., Fayaz, M., Ghadi, Y., & Aljuaid, H. (2022). Analyzing interdisciplinary research using co-authorship networks. Complexity, 2022, Article 2524491. https://doi.org/10.1155/2022/2524491

Wang, J. (2010). Consistent selection of the number of clusters via cross-validation. Biometrika, 97(4), 893–904. https://doi.org/10.1093/biomet/asq061

Wang, W., Wei, F., Dong, L., Bao, H., Yang, N., & Zhou, M. (2020). MiniLM: Deep self-attention distillation for task-agnostic compression of pre-trained transformers. In Proceedings of the 34th International Conference on Neural Information Processing Systems (pp. 5776–5788).

Yau, C.-K., Porter, A., Newman, N., & Suominen, A. (2014). Clustering scientific documents with topic modeling. Scientometrics, 100(3), 767–786. https://doi.org/10.1007/s11192-014-1321-8