A  model for converting data into NoSQL data warehouse for developing a real-time financial data warehouse system

Thanin Muangpool; Klaokanlaya Silachan; Sanya Kuankid

doi:10.53848/ssstj.v11i2.762

Authors

Thanin Muangpool Creating Innovation for Sustainable Development Research Center, Nakhon Pathom Rajabhat University
Klaokanlaya Silachan Faculty of Science and Technology, Nakhon Pathom Rajabhat University
Sanya Kuankid Faculty of Science and Technology, Nakhon Pathom Rajabhat University

DOI:

https://doi.org/10.53848/ssstj.v11i2.762

Keywords:

API, NoSQL, MongoDB node, MongoDB serverless, Cassandra

Abstract

This research introduces a novel model, the Financial Data Warehouses API (FDW-API), developed using PHP, Node.js, and Express.js. The model is designed to transform banking credit dataset information into a data warehouse format using a Non-Only SQL (NoSQL) database, stored in JSON format. Three types of databases were employed: MongoDB Node, MongoDB Serverless, and Cassandra. The study includes a comparative analysis of the data retrieval speed from all three databases. The model's applicability was tested in a real-time credit approval web application, demonstrating its effectiveness in transforming and storing data. Testing involved loading datasets ranging from 200, 300, 400, 500, 600, 800, and 1000 entries. Results indicate that the MongoDB serverless database outperformed others in terms of efficiency. Additionally, the FDW-API model streamlines data transformation and storage, facilitating real-time analysis and decision-making for financial institutions and data-driven businesses. Its flexibility integrates seamlessly with existing systems, reducing development time and costs, while its scalability accommodates growing data volumes and evolving business needs, providing a valuable tool for strategic insights and competitive advantage.

References

Alfred, R., & Kazakov, D. (2006). Pattern-based transformation approach to relational domain learning using dynamic aggregation for relational attributes. Proceedings of the 2006 International Conference on Data Mining (pp. 118-124). Las Vegas, Nevada.

Al-Mamory, S., & Jassim, F. S. (2013). Evaluation of different data mining algorithms with KDD CUP 99 data set. Journal of University of Babylon, 21.

Barahama, A. D., & Wardani, R. (2021). Utilization extract, transform, load for developing data warehouse in education using Pentaho Data Integration. Journal of Physics: Conference Series, 2111. doi:10.1088/1742-6596/2111/1/012030

Boonhao, P. (2020). NewSQL databases and usability trends. Mahidol R2R e-Journal, 8(2), 38-52.

Bouaziz, S., Nabli, A., & Gargouri, F. (2019). Design a data warehouse schema from document-oriented database. Procedia Computer Science, 159, 221-230. doi:10.1016/j.procs.2019.09.177

Chauhan, A. (2019). A review on various aspects of MongoDB databases. International Journal of Engineering Research & Technology (IJERT), 8(05), 90-92.

Garani, G., Chernov, A., Savvas, I., & Butakova, M. (2019). A data warehouse approach for business intelligence. Proceedings of the 2019 IEEE 28th International Conference on Enabling Technologies: Infrastructure for Collaborative Enterprises (WETICE) (pp. 70-75). Napoli, Italy.

Harvy, I., Matitaputty, G. A., Girsang, A. S., Michael, S., & Isa, S. M. (2019). The use of book store GIS data warehouse in implementing the analysis of most book selling. Proceedings of the 7th International Conference on Cyber and IT Service Management (CITSM) (pp. 1-5). Jakarta, Indonesia.

Hassan, C. A. U., Hammad, M., Uddin, M., Iqbal, J., Sahi, J., Hussian, S., & Ullah, S. S. (2022). Optimizing the performance of data warehouse by query cache mechanism. IEEE Access, 10, 13472-13480.

doi:10.1109/ACCESS.2022.3148131

Jaratsantijit, Y. (2022). Comparative study of query performance between relational database and NoSQL database for information system: A case study of the asset database of information technology service center. Chiang Mai: Chiang Mai University.

Jose, B., & Abraham, S. (2020). Performance analysis of NoSQL and relational databases with MongoDB and MySQL. Materials Today: Proceedings, 24(3), 2036-2043. doi:10.1016/j.matpr.2020.03.634

Nizzad, A. R. M., & Irshad, M. B. M. (2021). Data warehouse implementation: Cost effective approach for small businesses. Journal of Information Systems & Information Technology (JISIT), 6(2), 62-71.

Oditisi, I., Bicevska, Z., Bicevskis, J., & Karnitis, G. (2018). Implementation of NoSQL-based data warehouses. Baltic J. Modern Computing, 6(1), 45-55.

Petricioli, L., Humski, L., & Vrdoljak, B. (2021). The challenges of NoSQL data warehousing. Proceedings of International Conference on E-business Technologies (pp. 44-48). Serbia.

Singsanit, K. (2021). The development of executive information system for managing research in university by data integration techniques on ontology on business intelligence. Journal of Buddhist Education and Research: JBER, 7(1), 157-174.

Songsiri, K., & Tamee, K. (2022). Development of data warehouse for financial report in Faculty of Science, Naresuan University. Journal of Applied Informatics and Technology, 4(2), 99-113. doi:10.14456/jait.2022.8

Tavallaee, M., Bagheri, E., Lu, W., & Ghorbani A. A. (2009). A detailed analysis of the KDD CUP 99 data set. Proceedings of the 2009 IEEE Symposium on Computational Intelligence in Security and Defense Applications (CISDA 2009) (pp. 1-6). Ottawa, Canada.

Wang, D., Li, Q., Xu, C., Wang, P., & Wang, Z. (2021). Research of data warehouse for science and technology management system. Proceedings of the International Conference on Service Science (ICSS) (pp. 65-69). Xi'an, China.

Yulianto, A. A. (2019). Extract transform load (ETL) process in distributed database academic data warehouse. Journal on Computer Science and Information Technologies, 4(2), 64-71. doi:10.11591/APTIKOM.J.CSIT.36