Development ETL (Extract, Transform and Load) Module in Indonesian Agricultural Commodities OLAP System


Aditia Yudhistira(1*); Imas Sukaesih Sitanggang(2); Hari Agung Adrianto(3);

(1) Institut Pertanian Bogor
(2) Institut Pertanian Bogor
(3) Institut Pertanian Bogor
(*) Corresponding Author

  

Abstract


The SOLAP system for Indonesian Agricultural Commodities is a successful development based on previous studies. Agricultural commodity data are managed in a data warehouse with a galactic schema, which has 7 fact tables, namely cut flower horticulture, ornamental plant horticulture, horticulture, food crops, plantation, livestock population, and livestock production, as well as 3 dimensional tables, namely location, time, and commodity. The results of SOLAP operations on the system can be visualized in the form of crosstabs, graphs and maps. The system uses a web platform so that it can be accessed by the public. However, the SOLAP system cannot update data in real time. This study aims to develop a data warehouse for Indonesian Agricultural Commodities SOLAP in real time by creating a scraping system. This study has succeeded in developing a data warehouse in real time on the indonesian agricultural commodity SOLAP system by creating a real time scraping system that is applied to the SOLAP server and has succeeded in making the ETL process run in real time on the SOLAP server and optimizing polygon-based spatial data visualization using the Douglas-Peucker. This study has also carried out functional testing of OLAP features and functions on the Indonesian Agricultural Commodity SOLAP system using the black box testing method. The results of this study provide accurate and real-time data on the SOLAP of Indonesian Agricultural Commodities, with the results of SOLAP feature testing achieving 100 percent pass and the data conformity test results of OLAP function as expected. In addition, the results of this study make it possible to automatically update the data according to a predetermined schedule to provide real-time information.


Keywords


Agricultural Commodities; Douglas-Peucker; Functional Testing; Real-Time; SOLAP

  
  

Full Text:

PDF
  

Article Metrics

Abstract view: 302 times
PDF view: 136 times 		     

Digital Object Identifier

doi  https://doi.org/10.33096/ilkom.v15i2.1758.335-343 		  

Cite

How to cite item

References


A. I. Putri and I. S. Sitanggang, “Data Cubes Integration in Spatial OLAP for Agricultural Commodities,” IOP Conf. Ser. Earth Environ. Sci., vol. 58, no. 1, 2017, doi: 10.1088/1755-1315/58/1/012034.

N. P. M. Ardiyanti, A. S. Kusuma, and I. K. B. Sandika, “OLAP (On-Line Analytical Processing) Implementation for Web-Based Sales Transaction Processing (Case Study: Lilola Boutique),” J. Teknol. Inf. dan Komput., vol. 4, no. 1, pp. 80–89, 2018, doi: 10.36002/jutik.v4i1.398.

M. Hilman and D. Djamaludin, “Factor Analysis of ETL Process Optimization in Data Warehouses to Support Management Decision Making with Business Intelligence,” Fakt. Exacta, vol. 11, no. 1, p. 24, 2018, doi: 10.30998/faktorexacta.v11i1.2325.

A. R. Ginanjar, I. S. Sitanggang, and Annisa, “Optimization of spatial visualization module in solap for indonesian agricultural commodities,” Int. J. Geoinformatics, vol. 16, no. 1, pp. 9–19, 2020.

I. S. Sitanggang, R. Trisminingsih, F. Fuady, and H. Khotimah, “Extract, Transform, Load Module in SOLAP for Indonesia Agricultural Commodity,” 3rd Int. Conf. Sustain. Inf. Eng. Technol. SIET 2018 - Proc., pp. 101–105, 2018, doi: 10.1109/SIET.2018.8693152.

F. Ghita and R. Trisminingsih, “SOLAP Data Warehouse Testing for Indonesian Agricultural Commodities,” J. Ilmu Komput. dan Agri-Informatika, vol. 8, no. 1, pp. 42–56, 2021, doi: 10.29244/jika.8.1.42-56.

I. S Sitanggang, R. Trisminingsih, H. Khotimah, and M. Syukur, “Usability testing of SOLAP for Indonesia agricultural commodity,” IOP Conf. Ser. Earth Environ. Sci., vol. 299, no. 1, 2019, doi: 10.1088/1755-1315/299/1/012054.

W. S. Fana, R. Sovia, R. Permana, and M. A. Islam, “Data Warehouse Design With ETL Method (Extract, Transform, And Load) for Company Information Centre,” Int. J. Artif. Intell. Res., vol. 5, no. 2, pp. 132–137, 2021, doi: 10.29099/ijair.v5i2.215.

A. Rahma Salsabila, M. Daffa, M. Kandias, H. Maulana, and E. D. Wahyuni, “Proceedings Of The National Seminar On Technology And Information Systems (Sitasi) 2022 Implementation Of Web Scraping Technique To Display Data English Premier League Team Implementation Of Web Scraping Technique To Showing English Premier League Team Team Data,” no. September, pp. 10–11, 2022, [Online]. Available: http://sitasi.upnjatim.ac.id/%7C40.

Fatmasari, Y. N. Kunang, and S. D. Purnamasari, “Web Scraping Techniques to Collect Weather Data in South Sumatera,” Proc. 2018 Int. Conf. Electr. Eng. Comput. Sci. ICECOS 2018, pp. 385–390, 2019, doi: 10.1109/ICECOS.2018.8605202.

D. Kaiying, C. Senpeng, and D. Jingwei, “On optimisation of web crawler system on Scrapy framework,” Int. J. Wirel. Mob. Comput., vol. 18, no. 4, pp. 332–338, 2020, doi: 10.1504/IJWMC.2020.108530.

X. Tian, T. Zhang, X. Zhuang, and X. He, “Research and Implementation of Campus Network Search Engine Based on Scrapy Framework and Elasticsearch,” Proc. 32nd Chinese Control Decis. Conf. CCDC 2020, pp. 4193–4198, 2020, doi: 10.1109/CCDC49329.2020.9164582.

X. Barillot, J. Hangouët, and H. Kadri-dahmani, “Generalization of the ‘ Douglas and Peucker ’ Algorithm for Cartographic Applications,” pp. 2–4, 1975.

S. Lakhera and T. Praveena, “Visual Analysis using Modified Ramer-Douglas – Peucker Algorithm on Time Series Data,” Int. Res. J. Eng. Technol., vol. 7, no. May, pp. 4225–4229, 2020.

B. A. Priyaungga, D. B. Aji, M. Syahroni, N. T. S. Aji, and A. Saifudin, “Black Box Testing on Library Applications Using Equivalence Partitions Techniques,” J. Teknol. Sist. Inf. dan Apl., vol. 3, no. 3, p. 150, 2020, doi: 10.32493/jtsi.v3i3.5343.


Refbacks



Copyright (c) 2023 Aditia Yudhistira, Imas Sukaesih Sitanggang, Hari Agung Adrianto

Creative Commons License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.