Enhancing Data Quality through Automated Data Profiling
DOI:
https://doi.org/10.36676/jrps.v15.i4.17Keywords:
Data quality, Auto data profiling, data management, anomaly detectionAbstract
Data profiling is now a popular solution for automating data accuracy and data quality and is characterized by increased reliability of datasets. This paper briefly discusses the difficulties of achieving high data quality, the importance of automation in overcoming these difficulties, and the methods and procedures of data profiling. Automated profiling through data validation thus leads to improved decision making, especially through the unearthing of gaps and contradiction as well as supporting data management as a critical component of compliance. The paper also demonstrates through the use of interesting case examples and illustrating applications how profiling can open up the full utility of organisational data resources.
Downloads
References
Articles, Z., & Articles, Z. (2024, January 26). UNDERSTANDING PROFILING AND AUTOMATED DECISION-MAKING UNDER GDPR: IMPLICATIONS AND PRACTICAL APPLICATIONS - Zedroit. Zedroit - Ensuring Privacy, Securing Business. https://www.zedroit.com/understanding-profiling-and-automated-decision-making-under-gdpr-implications-and-practical-applications/
Ehrlinger, L., & Wöß, W. (2022). A survey of data quality measurement and monitoring tools. Frontiers in big data, 5, 850611. https://doi.org/10.3389/fdata.2022.850611
Jakubik, J., Vössing, M., Kühl, N., Walk, J., & Satzger, G. (2024). Data-centric artificial intelligence. Business & Information Systems Engineering, 1-9. https://doi.org/10.1007/s12599-024-00857-8
Jang, W. -J., Lee, S. -T., Kim, J. -B., & Gim, G. -Y. (2019). A Study on Data Profiling: Focusing on Attribute Value Quality Index. Applied Sciences, 9(23), 5054. https://doi.org/10.3390/app9235054
Mitropoulos, P., Patroumpas, K., Skoutas, D., Vakkas, T., & Athanasiou, S. (2021). BigDataVoyant: Automated Profiling of Large Geospatial Data. In EDBT/ICDT Workshops. http://star.informatik.rwth-aachen.de/Publications/CEUR-WS/Vol-2841/BigVis_1.pdf
Radivojević, T., Costello, Z., Workman, K., & Garcia Martin, H. (2020). A machine learning Automated Recommendation Tool for synthetic biology. Nature communications, 11(1), 4879. https://doi.org/10.1038/s41467-020-18008-4
Scarcella, L. (2019). Tax compliance and privacy rights in profiling and automated decision making. Internet Policy Review, 8(4). https://ssrn.com/abstract=3933264
Taleb, I., Serhani, M. A., Bouhaddioui, C., & Dssouli, R. (2021). Big data quality framework: a holistic approach to continuous quality management. Journal of Big Data, 8(1), 76. https://doi.org/10.1186/s40537-021-00468-0
W. Epperson, V. Gorantla, D. Moritz and A. Perer, (2024). "Dead or Alive: Continuous Data Profiling for Interactive Data Science" in IEEE Transactions on Visualization & Computer Graphics, vol. 30, no. 01, pp. 197-207. 10.1109/TVCG.2023.3327367
Yayik, A., Aybar, V., APIK, H. H., Içöz, S., Bakar, B., & Güngör, T. (2022). Deep learning-aided automated personal data discovery and profiling. Turkish Journal of Electrical Engineering and Computer Sciences, 30(1), 167-183. https://doi.org/10.3906/elk-2102-54
Downloads
Published
Issue
Section
License
Copyright (c) 2024 International Journal for Research Publication and Seminar
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.