A description driven approach for flexible metadata tracking

Branson, Andrew; Shamdasani, Jetendr; McClatchey, Richard

Abstract

Evolving user requirements presents a considerable software engineering challenge, all the more so in an environment where data will be stored for a very long time, and must remain usable as the system specification evolves around it. Capturing the description of the system addresses this issue since a description-driven approach enables new versions of data structures and processes to be created alongside the old, thereby providing a history of changes to the underlying data models and enabling the capture of provenance data. This description-driven approach is advocated in this paper in which a system called CRISTAL is presented. CRISTAL is based on description-driven
principles; it can use previous versions of stored descriptions to define various versions of data which can be stored in various forms. To demonstrate the efficacy of this approach the history of the project at CERN is presented where CRISTAL was used to track data and process definitions and their associated provenance data in the construction of the CMS ECAL detector, how it was applied to handle analysis tracking and data index provenance in the
neuGRID and N4U projects, and how it will be matured further in the CRISTAL-ISE project. We believe that the CRISTAL approach could be invaluable in handling the evolution, indexing and tracking of large datasets, and are keen to apply it further in this direction.

Citation

Branson, A., Shamdasani, J., & McClatchey, R. (2013, November). A description driven approach for flexible metadata tracking. Paper presented at 7th ESA International Conference on Ensuring Long-Term Preservation and Adding Value to Scientific and Technical Data (PV 2013), Frascati, Italy

• Presentation Conference Type: Conference Paper (unpublished)
• Conference Name: 7th ESA International Conference on Ensuring Long-Term Preservation and Adding Value to Scientific and Technical Data (PV 2013)
• Conference Location: Frascati, Italy
• Start Date: Nov 4, 2013
• End Date: Nov 6, 2013
• Publication Date: Nov 4, 2013
• Publicly Available Date: Jun 7, 2019
• Peer Reviewed: Peer Reviewed
• Pages: 355-364
• Keywords: CMS, metadata, schema evolution, computer science, provenance, semantics
• Public URL: https://uwe-repository.worktribe.com/output/926077
• Publisher URL: http://www.congrexprojects.com/2013-events/pv2013/welcome
• Related Public URLs: http://arxiv.org/abs/1402.5953v1; http://www.congrexprojects.com/docs/default-source/13c17_docs/pv-2013-final-posters-4-11.pdf?sfvrsn=2
• Additional Information: Additional Information : Imported from arXiv; Title of Conference or Conference Proceedings : 7th ESA International Conference on Ensuring Long-Term Preservation and Adding Value to Scientific and Technical Data (PV 2013)