Designing traceability into big data systems

McClatchey, Richard; Branson, Andrew; Shamdasani, Jetendr; Kovacs, Zsolt

Abstract

Providing an appropriate level of accessibility and traceability to data or process elements (‘Items’) in large volumes of data, often Cloud-resident, is an essential requirement in the Big Data era. Enterprise-wide data systems need to be designed from the outset to support usage of such Items across the spectrum of business use rather than from any specific application view. The design philosophy advocated in this paper is to drive the design process using a so-called ‘description-driven’ approach which enriches models with meta-data and description and focuses the design process on Item re-use, thereby promoting traceability. Details are given of the description-driven design of big data systems at CERN, in health informatics and in business process management. Evidence is presented that the approach leads to design simplicity and consequent ease of management thanks to loose typing and the adoption of a unified approach to Item management and usage.

Citation

McClatchey, R., Branson, A., Shamdasani, J., & Kovacs, Z. (2015, July). Designing traceability into big data systems. Paper presented at 5th Annual International Conference on ICT: Big Data, Cloud and Security

• Presentation Conference Type: Conference Paper (unpublished)
• Conference Name: 5th Annual International Conference on ICT: Big Data, Cloud and Security
• Start Date: Jul 27, 2015
• End Date: Jul 28, 2015
• Publication Date: Jul 1, 2015
• Peer Reviewed: Peer Reviewed
• Keywords: description-driven systems, big data, object design
• Publisher URL: http://bigdataclouds.org/index.html
• Additional Information: Title of Conference or Conference Proceedings : 5th Annual International Conference on ICT: Big Data, Cloud and Security (ICT: BDCS 2015)