Heterogeneous-race-free memory models

Hower, Derek R.; Hechtman, Blake A.; Beckmann, Bradford M.; Gaster, Benedict R.; Hill, Mark D.; Reinhardt, Steven K.; Wood, David A.

Abstract

Commodity heterogeneous systems (e.g., integrated CPUs and GPUs), now support a unified, shared memory address space for all components. Because the latency of global communication in a heterogeneous system can be prohibitively high, heterogeneous systems (unlike homogeneous CPU systems) provide synchronization mechanisms that only guarantee ordering among a subset of threads, which we call a scope. Unfortunately, the consequences and semantics of these scoped operations are not yet well understood. Without a formal and approachable model to reason about the behavior of these operations, we risk an array of portability and performance issues. In this paper, we embrace scoped synchronization with a new class of memory consistency models that add scoped synchronization to data-race-free models like those of C++ and Java. Called sequential consistency for heterogeneous-race-free (SC for HRF), the new models guarantee SC for programs with "sufficient" synchronization (no data races) of "sufficient" scope. We discuss two such models. The first, HRF-direct, works well for programs with highly regular parallelism. The second, HRF-indirect, builds on HRF-direct by allowing synchronization using different scopes in some cases involving transitive communication. We quantitatively show that HRF-indirect encourages forward-looking programs with irregular parallelism by showing up to a 10% performance increase in a task runtime for GPUs. Copyright © 2014 ACM.

• Presentation Conference Type: Conference Paper (Published)
• Conference Name: International Conference on Architectural Support for Programming Languages and Operating Systems - ASPLOS
• Start Date: Mar 1, 2014
• End Date: Mar 5, 2014
• Publication Date: Mar 14, 2014
• Deposit Date: Aug 17, 2015
• Peer Reviewed: Peer Reviewed
• Pages: 427-440
• Book Title: Proceedings of the 19th international conference on Architectural support for programming languages and operating systems - ASPLOS '14
• ISBN: 9781450323055
• DOI: https://doi.org/10.1145/2541940.2541981
• Keywords: memory models, programming languages, theory, formal models
• Public URL: https://uwe-repository.worktribe.com/output/820562
• Publisher URL: http://dx.doi.org/10.1145/2541940.2541981
• Additional Information: Title of Conference or Conference Proceedings : Proceedings of the ACM International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS)
• Contract Date: Apr 15, 2016