Patterns of Practice and Design: Towards an Agile Methodology for Educational Design Research

. This paper proposes a framework which attempts to address the barriers to the development of successful educational design research through a process which identifies gaps in current practices and devises innovations to target them. Educational design research assumes an ambitious position: a dual commitment to understand and contribute to both theory and practice. This task is confounded by the complexity of the domain and the inherent multi-stake-holder nature of most initiatives. Three barriers to success are identified: the shortage of mechanisms for cross-stakeholder dialogue, the failure to account for existing practices and contexts, and the rigid processes dictated by the dynamics of research projects. We report early findings from an attempt to address these barriers, based on an extension and adaptation of the Participatory Pattern Work-shop (PPW) methodology. A pilot implementation suggests that this framework optimises the exchange of ideas and methods, leading to a richer and more nuanced conceptualisation of educational innovations.


Introduction
Educational design research (EDR) [13], or Design-based research [21] emerged over a decade ago as an alternative paradigm for education science [5,9]. EDR is a changeoriented paradigm: its emphasis goes beyond understanding the world as it is, to ask "how do we make it better?" This entails a dual commitment to advance theory and practice simultaneously. It dictates a highly interventionist, inherently multi-disciplinary, iterative and situated methodology, which holds the promise of producing theory relevant to practice and practical innovations informed by theory. It is interventionist in the sense that researchers introduce innovations into the environment they study to observe their effects, iterative because these innovations evolve in tandem with their theoretical underpinnings, situated meaning that interventions are introduced into reallife settings, rather than laboratory conditions. The relation to theory is opportunistically eclectic: rather than maintaining a zealous allegiance to a monolithic theoretical tradition, researchers will draw on multiple sources as befits the challenges at hand. Research questions are unashamedly value-driven: when asking "how do we make the world better?" researchers are compelled to take a stance on what is "better". EDR shares some of the features of the learning design approach [18]. However, the latter is much more focused on improving practice -without an equal commitment to advancing theory. On the other hand, research in learning design has yielded a wealth of tools [28] and representations [25], which could potentially benefit EDR.

The Layers project and the theory of hybrid social networks
The Learning Layers Project (http://learning-layers.eu/), funded by the EU FP7 programme, is developing technologies to support informal learning in the workplace, specifically in the healthcare and construction sectors. A central construct in the projects conceptualisation of this domain is the idea of Hybrid Social Learning Networks. This refer to situations where learners' predominant mode of learning is social, where they rely on a network of activity systems to sustain their learning practices, and where this network is manifested in both physical and virtual connections.
The project team identified EDR as an appropriate approach, given the commitment to advance both the professional practice in hybrid social networks of healthcare practitioners and the theoretical understanding of emergent learning in such networks. Specifically, we identified the Participatory Pattern Workshop methodology (described below) as appropriate to our work. However, we found it necessary to extend and elaborate this methodology for two reasons. First, we noticed parallels and potential synergies with agile software development methodologies, and wanted to leverage these. Second, we observed a need for a closer account of the existing (pre-intervention) professional practices, which would allow our designs to blend into the current situation.

SNaP!, PPW and LDS
The Participatory Pattern Workshop (PPW) methodology [24] is a framework for engaging multi-disciplinary communities in collaborative reflection on educational innovation in a given domain. This methodology leads participants through a process of articulating their experience in the form of Design Narratives, eliciting from those Design Patterns, and using these to generate testable future design conjectures, in the form of Design Scenarios [23]. This methodology is mirrored by the Learning Design Studio approach, which guides practitioners through a reflective process of designing, prototyping and evaluating new innovations [20]. The synergies between the two are discussed in [18].
Design patterns originate in Christopher Alexander's work in theory of architecture [3]. They have been widely adopted as a form for sharing design knowledge in software engineering, and from there made their way into educational design [6,7,11,31]. Yet most efforts in this direction focus solely on the patterns themselves, without addressing their empirical and theoretical substantiation, or the mechanisms for applying them to novel challenges. The PPW methodology responds to both issues. It suggests design narratives -semi structured accounts of a design experience -as a means for organising empirical evidence for patterns and design scenarios as a form of testable design propositions which can combine patterns to formulate new solutions.

The Patterns of Practice and Design (PPD) Approach
When considering the PPW approach for the Layers project, we observed several limitations of the methodology: 1. Design patterns enable a trajectory from practice through theory back to practice. Patterns encode practitioner experiences in a form that can be calibrated with theory and then re-applied to new situations. Yet, they do not provide a pathway directly from theory into practice. A pattern always originates in experience. How do we represent directives for design derived from theory? 2. In order to affect change in a socio-cultural situation, we need to first construct a detailed conceptualisation of the current state of affairs. Any innovation we introduce will need to blend in and then modulate existing practices. How do we describe these practices, and how do we bind them into the design cycle? 3. The PPW methodology has been shown to be effective in establishing cross-disciplinary design-level discourse. However, when developers proceed to translate the outcomes of such a discourse to a development plan, they need to represent them in a suitable language. We needed boundary objects to bridge the research and reflection dynamics and the development processes.
Consequently, we extended the PPW methodology by including design principles [13] as boundary objects translating theory into practice, and agile user stories as boundary objects bridging the EDR language with that of software engineering. The resulting methodology ( Fig. 1), which we call "Patterns of Practice and Design" (PPD), leads practitioners and researchers through a cycle in which they: • Understand existing epistemic practices.
• Identify gaps in those practices.
• Consider relevant theories as well as existing / previous attempts to address these gaps.
• Conceptualise a novel solution.
• Define the evaluation protocols for this solution. To support this process, we use a platform called ILDE (Integrated Learning Design Environment), developed by Universitat Pompeu Fabra, Barcelona [13,14]. ILDE already had built-in support for design narratives and design patterns, as well as other learning design representations. It also supports a simple yet powerful mechanism for ad-hoc templates, which allowed us to add support for all the other constructs we used. In order to enhance the rigor of the process, we added evaluation rubrics for the various representations. These rubrics provided a standard key for peer review: for each representation, we defined a list of evaluation criteria, and for each criteria -a set of indicators for red (requires revision) amber (acceptable) and green (adequate).

Kernel theories
As with any scientific inquiry, our investigation starts with the articulation of a theoretical framework. In our case, the primary framework is the theory of learning in hybrid social networks, described above. This framework builds on post-Vygotzkian theory [9] and the theory of social machines [29]. Yet, as we noted, design research deals with messy real world situations, where a monolithic theoretical perspective may be too constraining and counter-productive. Indeed, to guide our work we drew in insights from networked working [25], communities of practice and their relation to social networks [35], the mathematics of complex networks [26], and communities of inquiry [11].34 A full review of this framework is the subject of a separate paper. Here, we wish to demonstrate that the theoretical framework was not an a-priori dogma we imposed on the problem domain, but a dynamic structure we constructed to help us explain our observations and make valid predictions. To fine tune this structure, we needed to calibrate it with a detailed image of the domain. This image emerged through the construction of narratives of practice.

Share narratives of practice
A 'Practice' (as opposed to just "practice"), or 'patterned practice', is a recurring pattern of behaviour manifesting certain intentions in a given context. Arguably, the aim of (educational) design is to enhance practices: to allow people to revise their intentions and manifest their intentions in more effective and efficient ways.
Understanding current practices is a prerequisite to any attempt to enhance practices. After all, as Latour says [16], design is always re-design.
How do we capture, and make sense of, human experience? Bruner [7] argues that our primary instrument for this task is narrative. A narrative provides an account of 'someone doing something in some context'.
Based on ethnographic data, or any other sources, construct a narrative that describes a current practice. In order to do so, we need to: • Define the "actors'' in the narrative. These should be fictitious personas based on real people.
• Define the "context'' of the narrative. Describe the material, social and intentional forces that shape your problem space.
• Note the intentions that your narrative focuses on, and list the actions that actors performed to satisfy them.
An example practice narrative from our collection follows a group of clinic managers working in different small clinics within the same administrative region. These clinic managers wanted to work together to improve professional aspects of their clinics, by sharing, experiences, knowledge and workload. To do this they have set up a network and use monthly meetings and email to discuss their common interests and identify potential joint work. The actual practice narrative would ideally include the full details of the characteristics of the clinics, and the clinic managers, details of any additional professionals involved or influenced by the network, the relations between the network and the administrative region, and so forth. In order to ensure the realism and precision of the narrative yet adhere to ethical standards, the people involved would be substituted with personas: fictitious characters which representative the real protagonists.

Identify patterns of practice
The narratives we collected are "thick descriptions" of single instantiations of (presumably more general) practices. Next, we need to distil the essence of these narratives in a concise form that can serve us as a building block in our design process. A "pattern of practice'' is a claim of the form: In a context C, actors of type T have intentions I and perform actions {A1, A2, A3…} to satisfy them.
For example, the narrative above would be abstracted into a pattern such as: in areas served by small clinics, professionals performing a similar function do not have sufficient support in-house. To redress this issue, they form informal cross-clinic networks of peers in a similar role. These networks will self-organise using a combination of physical and virtual interactions, and define a set of operational routines.
Ideally, we should derive such patterns by systematically comparing multiple narratives, noting their overlaps and eliminating idiosyncrasies. Due to pragmatic constraints, and the dynamics of developing the methodology in tandem with the actual research, this was done in a semi-structured manner, where some narratives were analysed through a formal process supported by the ILDE, and some where pre-processed by researchers and only the resulting patterns recorded in the ILDE. In cases where our documented data includes singular narrative which we felt were representative of a broader phenomenon, we attempt to identify the salient features of the narrative and seek circumstantial evidence to support our choices. For example, if we have a narrative of a particular middle-aged Asian nurse attending CPD, we would assume that it is her role that is pertinent to the pattern, rather than her gender, race or age.

Domain specific theories of practice
The kernel theories provided the lens by which we analysed and interpreted the narrative and patterns of practice. Yet this lens is still too abstract. We needed to project the kernel theories into the settings in which we operate, making them more specific and concrete, bringing them closer to the narratives and patterns of practice. To do this, we translated the kernel theories into domain specific theories of practice. These are theoretical constructs framed in the language, constrains and potentials of the practical domain under question.

Identify gaps / issues / challenges
Design is always oriented at change. Herbert Simon [31] defines design as "devising courses of action aimed at transforming existing states of the world into desired ones. This is what Latour [16] calls design's "remedial intent"; a designer identifies an aspect of the world around her which she thinks can be made better, and then creates things which she intends to assist in that change.
Before we undertake an act of design, we need to specify the change we wish to engender. This will be the objective of our design, and the measure of its success. Our focus is not on physical change in the world, but on a change in practice. This may include allowing actors to satisfy intentions that are currently unsatisfied, or allowing them to satisfy their intentions in a more effective and efficient manner.
In Agile software development methodologies, such objectives are often phrased in the form of "user stories'' [11] [12] [13]. Despite the risk of confusion (stories vs. narratives), we use this term because of its prevalence.

Share design narratives
Once we identify a challenge and analyse it, our instinct is usually to confront it and do our best to resolve it. Yet, if we want to be rigorous and efficient in our design -we should first study precedents. Any challenge we identify, the chances are that someone tried to resolve before. If not, there are surely other very similar challenges which have been solved. Again, as Latour [16] states, design is humble.
A "design narrative" is a semi-structured story that illuminates a particular attempt to address a complex challenge [23,24]. It can be told from the view (and in the voice) of the designer, or the user (the learner or teacher, in the case of learning design). A design narrative recounts an incident where the designer attempted to bring about a change in the user's world. It details the context of this incident, the objectives of the design and the user, the actions they took and their result, the obstacles they encountered and how they dealt with them.

Identify design patterns
In order to form our understanding of the context, we generalised, or abstracted, from practice narratives to patterns of practice. Similarly, we need to identify critical issues and corresponding transferable methods of solution. Such a description of a recurring problem, or design challenge, the characteristics of the context in which it occurs, and a possible method of solution is called a "design pattern" [23,24,22]. The design patterns paradigm was proposed by Christopher Alexander [3] as a form of design language within architecture. This was done with the explicit aim of externalizing knowledge to allow the accumulation and generalization of solutions and to allow all members of a community or design group to participate in discussions relating to design. These patterns were organized into coherent systems called pattern languages where patterns are related to each other. "Each pattern describes a problem that occurs over and over again in our environment, and then describes the core of the solution to that problem, in such a way that you can use this solution a million times over, without ever doing it the same way twice." [3].
Educational Design patterns (or Pedagogical Patterns) and pattern languages have been proposed as effective means to facilitate rigorous pragmatic discourse, bridging theory and practice in educational innovation [6,8,11,24,32]. The SNaP! Methodology provides a pathway from design narratives through design patterns to design scenarios (discussed below). This pathway starts from acts of innovation and ends in acts of innovation, and uses theory in a slightly ad-hoc manner to substantiate the patterns. Our extension of the framework introduced design principles as a more structured link feeding in from (domain specific) theories to the patterns.

Propose Design Principles
Whereas Design Patterns are derived from practice and then explained by theory, Design Principles [15] are derived from theory and then validated by evidence from practice. Thus, having marked the goals of our design in the form of agile user stories, we proceed with a two thronged attack to seek their resolutions: on one hand, drawing on relevant pragmatic examples (through design narratives and design patterns) and on the other hand bringing the theory to bear through design principles. The proposed innovations derived from this process are represented as scenarios and storyboards.

Construct design scenarios (as storyboards)
A Design Scenario [23,24] takes the form of a Design Narrative and projects it into the future, i.e. rather than recounting past events, it suggests a possible trajectory of future events. The result is a testable design propositions, stating: "if we introduce innovation I in context C, we will achieve objectives O and (possibly) see additional results R". The power of a scenario is in describing the proposed innovation in terms of its interaction with and effects on current practice. Rather than providing a structural specification of what the "thing" we will create is, we focus on what it will do. The exact features, and consequently the structural elements of the innovation will follow from the functional definition. A scenario can be represented in many ways. We chose storyboards as the primary form, as they proved to be effective for brainstorming and user validation.

Validating and iterating: prototypes and field tests
We began this paper by presenting educational design research as a paradigm with a dual commitment, to advance both theory and practice. Fulfilling this commitment requires that the scenarios be put to test in the domain of practice, and their effects observed and analyzed. The outcomes of this analysis will validate, update or challenge the underlying theory. At the same time, it will feed back into the next cycle of development to improve the efficacy of the innovations.

Applying the Methodology
To test this methodology, we conducted two design workshops and then validated the outputs from the workshops at a meeting with a group of healthcare practitioners. The workshops where conducted in Bristol in January and February 2015. They were attended by representatives of the various roles in the project: learning scientists, field researchers, computer scientists and software developers. The first workshop ran for one full day and included nine participants on site and four remote (via skype). The second workshop ran for two days and included seven participants on site. The healthcare practitioners meeting was conducted in early March 2015 in Leeds. A developer and a researcher who participated in both design workshops represented the design research team, and four practitioners represented the target group. The researcher present at the practitioner meeting is also the coordinator of the healthcare pilot. The meeting was positioned as a design consultation with prospective users of the proposed innovation.
The first workshop focused on articulating the practice narratives, eliciting practice patterns from these, and extracting agile user stories from the narratives and patterns. The second workshop proceeded to analyse the relevant theory alongside examples of past innovations, to derive design patterns and principles, which in turn where used to develop new scenarios that respond to the user stories from the first workshop. The workshops used a combination of paper and digital representations. Sketches and handwritten notes were transcribed or scanned and posted on the ILDE. Before and in-between workshops, we used the ILDE to collaboratively collate evidence, analyse it and propose design conjectures.
The final output of the second workshop were a set of hand-drawn storyboards representing proposed techno-pedagogical innovations which would address the needs we identified in the learning context. That is, these were proposals for new practices and software tools to support them, which would seamlessly interface with existing professional practices and enhance practitioners knowledge-building by allowing them to invoke their network and seek help when needed in an effective and efficient manner. Three of these storyboards were selected to be reviewed by practitioners. They we reproduced at a higher graphical standard and printed.

Results
The workshops yielded ten practice narratives, six personas, five design principles, seven collections of agile user stories, seven design narratives, eleven design patterns and five scenario storyboards. We did not generate any practice patterns, because we found that some of the practice narratives were in fact generalisations over prior data.
Most of these outputs are available for viewing at: http://ilde.upf.edu/layers/ (requires free registration). Three of the storyboards from the second design workshop were selected for discussion at the practitioner meeting, and re-produced to a higher graphical standard. Thse were presented and discussed at the practitioner meeting with four healthcare practitioners from the healthcare pilot region within which Learning Layers is working. These practitioners represented healthcare networks 1 with whom the project has been working or plans to work. The results of the discussions with the healthcare practitioners are summarized below.
Bridging between meetings -maintaining flow and focus. The users reported that they recognised the scenario set out in storyboards 1 and 2 and that support for keeping activities moving/flowing and focused between meetings was very important to them. The Practice Manager stated that the described tool would fit well with the way that the Practice Manager's network wanted to work, with individuals and working groups taking on specific areas of work/learning, developing and sharing expertise in these areas and thus managing the workload across the network effectively. They also discussed how having a clear process such as this could help those network members who are currently less active to feel more confident about getting engaged and taking on a particular role or area of work. Although storyboards 1 & 2 were set within the context of the Practice Managers' network, the other network representatives reported that they felt the tool would be equally useful in their networks and again they emphasised the value in maintaining the flow/momentum of activities between meetings and the focus of the discussions. They all discussed how email is currently the main tool used to help bridge between meetings, but that this has considerable limitations (information overload, difficulty tracking discussions, loss of focus, indiscriminate targeting of information). However, it was noted that any new tool would need to be easy to use, any difficulties encountered would lead to people reverting to email. The main changes they suggested to storyboards 1 and 2 were to include a suggestions area into which everyone could send information (even those outside the working group) and to highlight that they may want to ask for feedback from the wider group before the next meeting. So it should be possible for them to share the material, which the working group has developed, for comment (not editing) by the wider group. The GP was also interested in whether the tool could produce a summary/report of the process followed by the working group, so that this could then be presented by a doctor as evidence (in their revalidation portfolio) of their engagement in quality improvement activities. Storyboards 1 and 2 prompted lively and engaged discussions, clearly working well as a boundary object supporting cross-stakeholder dialogue. The healthcare practitioners' recognition of the bridging meeting meetings scenario and their belief that the tool depicted in the scenario would fit into their working practices suggests that the PPD approach has, at this stage, ensured that existing practices and context have been taken into account. It also appears that the proposed design (in storyboards 1 and 2) is likely to blend into the current situation.
Help Seeking -More discussion, less Q&A. Storyboard 3 received a less enthusiastic response. The participants felt that this scenario (clinical help seeking using an online tool) would not happen within a GP practice context. In fact a comment was made that they would wonder about the competence of someone who chose to use such a system in this situation. They emphasised that when help was needed with a specific patient case (a clinical issue) and/or when a rapid response was needed, then they would not consider using an online tool. In these case they would refer to someone within their own GP practice (a colleague), someone who they knew, who worked for the same organisation and who had a shared responsibility in the patient's treatment. Their request for help/advice in these cases would usually be face to face or on the phone.
Although the scenario was rejected by the group, they did then explore how such a tool could be useful for more process type of questions and advice -such as "How do you encourage patients to come in for smoking cessation groups?" (areas such as those that are taken up and explored by the working groups in storyboards 1 and 2). All the participants felt that the tool would need to be more like a forum, allowing open and flexible discussion, rather than a Q&A system. Comments were also made about how such a forum tool could help to foster a sense of connection between staff who may otherwise feel quite isolated due to their working patterns (e.g. part-time workers, staff who are the only person working in their role within the GP Practice, GPs who are in patient consultations for long periods of the day). Finally the group also discussed that they would like the tool to point them towards discussions and resources, rather than individuals.
Storyboard 3 received a more mixed reaction, with the healthcare practitioners rejecting some parts of it (the idea that an online tool could be used for patient-specific clinical questions that require a quick answer), but then exploring how the tool could be adapted to provide the support they need (more open discussions on process type questions and issues). By presenting the proposed solution as a storyboard, this accessible representation allowed users to comment on and refine the proposed solution before any implementation was started. The feedback provided by the users will not only influence the development but will also allow some clarification and revisions of the patterns previously identified, in order to ensure they better represent the users' experience. Therefore the PPD approach has helped the developers and researchers to work in a more agile fashion, checking and clarifying their understanding of the current context at an early stage, and then adjusting the designs accordingly.

Discussion
Prior to adopting this methodology, the team experienced the myriad of tensions and communication barriers typical to such multi-disciplinary teams. It often seemed that each party had its own language, agenda and frame of mind. The representations we adopted, we well as the procedures for generating them and moving from one to another, where crucial to the transformation of our loose professional network into a community of practice. These representation served as boundary objects, mediating the discourse between different stakeholders. The also provided checkpoints in the complex and non-linear process of design. The sequence of design workshops with the outcomes being introduced to the users worked extremely well. The workshop participants were able to take the more generalised findings from the data, and to break this down into levels of granularity, a powerful tool for learning. The users, reflecting upon the findings, were able to grasp the essence of the proposed innovations, associate them to their work setting and practices, and evaluation their potential effect.
The bottom line is that using this methodology we were able to produce a blueprint for a techno-pedagogical innovation which will address the critical learning needs of practitioners in their authentic settings.
This journey was not without challenges. The methodology, along with its wealth of representations, was somewhat complex to grasp at first and caused some confusion. However, once these initial hurdles where surmounted, we found that the methodology offered us a common language and focus our discourse on positive change for the target audience.

Summary
This paper started by presenting the Educational Design Research paradigm and some of the challenges is poses, within the context of the Learning Layers project work on hybrid social learning networks of healthcare networks.
The Participatory Pattern Workshop methodology was considered as a response to some of these challenges. The team acknowledged the value of this methodology, and at the same time identified some limitiations. Consequently, the team proceeded to extend the framework to offer a mechanism to include a better representation of current practices and user needs and to strengthen the link between theory and practice.
The enhanced framework, identified as "Patterns of Practice and Design" was used in a study exploring the potential of social and mobile technologies to improve on-thejob learning in small healthcare clinics. The methodology proved to be highly productive, producing a high volume of quality representations in ILDE, and a set of solutions which user appreciated.