Bridging e-Health and the Internet of Things: The SPHERE Project

There's a widely known need to revise current forms of healthcare provision. Of particular interest are sensing systems in the home, which have been central to several studies. This article presents an overview of this rapidly growing body of work, as well as the implications for machine learning, with an aim of uncovering the gap between the state of the art and the broad needs of healthcare services in ambient assisted living. Most approaches address specific healthcare concerns, which typically result in solutions that aren't able to support full-scale sensing and data analysis for a more generic healthcare service, but the approach in this article differs from seamlessly linking multimodel data-collecting infrastructure and data analytics together in an AAL platform. This article also outlines a multimodality sensor platform with heterogeneous network connectivity, which is under development in the sensor platform for healthcare in a residential environment (SPHERE) Interdisciplinary Research Collaboration (IRC).

The current challenge for the AAL interdisciplinary research community is discovering how to build integrated, useful, and deployable systems that can close the loop from data sensing through processing engines to end users.Such systems will rely on sensor datasets and deliver in (quasi) real time relevant contextual information to specific users, from clinicians and care providers to the individuals whose data are being collected and processed.The premise of the work in this area is that building profiles of activities of daily living (ADLs) will lead to useful datasets that can scale to very large populations, supporting early diagnosis, tracking the progress of chronic diseases, informing personalized treatment, and encouraging healthy behavior changes.
These systems are composed of several subsystems and rely on various technologies, some of which are fit for this purpose and some of which are still under development.Additional challenges arise when these technologies have to cope with multiple end users, as is typical in a domestic environment.A household often comprises several people who might have differing healthcare needs and individual preferences.For smart home systems to be effective in detecting and managing health conditions, they must provide meaningful clinical data but also be desirable to their domestic users.
This article provides an overview of current developments in the fields of sensing, networking, and machine learning, with an aim of underpinning the vision of the sensor platform for healthcare in a residential environment (SPHERE) project.The main aim of this interdisciplinary work is to build a generic platform that fuses complementary sensor data to generate rich datasets that support the detection and management of various health conditions.
Sensing Technologies in Ambient Assisted Living Sensing technologies are used in AAL for a range of applications.Some existing solutions include physiological, environmental, and vision sensors that frequently assist in T oday's aging population and the rise in chronic health conditions is precipitating a shift toward empowering people to manage their care and well-being at home.In particular, advances in ambient assisted living (AAL) are providing resources that improve patients' lives, as well as informing necessary a m b i e n t a S S i S t e d l i v i n g health monitoring in the home.They help characterize users' everyday activities by providing long-term sensing data that, in combination with ambient intelligence algorithms, contribute to behavior pattern recognition. 1 In the following subsections, we discuss the distinctions frequently adopted in the literature and caused by the different data outputs and approaches.However, one of SPHERE's aims is to integrate these various sensing modalities into an Internet of Things (IoT) solution for AAL.

Physiological signal Monitoring
Physiological signals provide health evidence directly from the human body via diverse biosensors that measure various physiological parameters. 2 These biosensors are deployed through an implantable (in-body), wearable (onbody), portable (off-body), or environmental modality.Of these, implantable sensors are the most intrusive and are included in this list merely for completeness-the aspiration in AAL has been to foster comfort through unobtrusive technology.Further research into telecare, telehealth, and telemedicine systems has improved biomedical sensing efficacy.
Recently, the advent of personal wearable devices for self-monitoring has pushed research outputs and fashionable electronic gadgets into the consumer space.Different conceptto-prototype systems have been proposed and implemented in response to individual healthcare issues.4][5] A typical biomedical sensing system is composed of a data acquisition module that collects various biomedical signals, a signal-processing module, a communication gateway (normally a computer or smartphone) that forwards data over the Internet, and a monitoring center.Mobile healthcare (m-health) uses smartphones and handheld devices for biomedical signal monitoring. 6lmost all of these personal physiological signal-monitoring systems can use various embedded sensors to synchronize data automatically from the device to the network gateway or monitoring center.Although it's promising that these wearable selfmonitoring gadgets perform activity tracking to collect ADL data, the case analysis in Figure 1 shows that several issues must be addressed for them to truly support AAL systems, including • lack of long-term, continuous, easy-toaccess raw data that contains rich details of clinically relevant information; • lack of interoperability with other healthcare systems; and • limited expandability to adapt to new sensing data.
The technology is moving toward more comfortable and desirable wearable devices and should build on users' real-life attitudes and experiences.Two big challenges are first, the absence of ambient information related to the physiological data and second, energy consumption (battery life).The former could introduce sensing cognition difficulty or even bias, and the latter is actually a bottleneck to wearable device proliferation.

Home Environment Monitoring
A smart home is a system of pervasive information and communication technologies by which both the home environment and residents' interactions with it are unobtrusively monitored. 2A list of sensors appears elsewhere, 2 but it's by no means comprehensive and has the potential to grow as the field of sensor technologies matures.Most AAL research projects use a diverse sensor portfolio rather than single sensors in various applications, as discussed later.
The fusion of ambient monitoring and signal processing techniques assists with the accurate recognition of activities or events in the home environment.Figure 2 provides an overview of some existing sensors and technology types, as well as their applications in a healthcare setting. 7It isn't an exhaustive summary, but it demonstrates different sensor modality usage.
One prominent area of application is fall detection 8 via wearable, ambient, and camera-based approaches.Accurate localization within the home environment is another important component in AAL applications. 9Many ambient sensor systems have been applied to address different health issues, such as mental health, emotional state, sleep measures, diabetes, and Alzheimer's disease, 10 monitoring individual daily activities for health assessments and to detect deviation from a user's behavioral patterns.A good example is the Casas project, 11 which treats environments in a smart home as intelligent agents and uses technologies from machine learning and pervasive computing.Different versions of smart home systems serve different purposes, including managing energy consumption, healthcare, home automation, and home entertainment.They all provide rich ADL datasets, but even though the data is available and can help identify behavior profiles, they've been relatively underexplored and integrated as indicators of health and well-being.The major challenge in this space is system and data integration for different commercially available devices to support user-friendly configuration. 12The datasets generated from those different smart home systems are disaggregated or less efficiently manipulated by advanced machine learning algorithms.A truly generic AAL system of systems that creates knowledge-based, context-aware services for AAL is yet to be realized. 13

Vision-Based Monitoring
Intelligent visual monitoring has received a great deal of attention in the past decade, especially because of increased interest in smart healthcare systems in home environments. 14lthough a wide variety of sensing technologies can be used for in-home assistive systems, visual sensors have the potential to address several limitations-specifically, they don't require the user to wear them, and they can simultaneously detect multiple events.Analysis of human motion via visual information has been achieved through the use of multicamera architectures in indoor and outdoor environments 15 and centralized or distributed platforms predicated on processing requirements and scalability issues.Recently, human motion analysis algorithms have dramatically improved through the combination of color cameras and depth sensors, the main advantages of the latter being their low cost and ability to provide real-time, dense depth data without intensive processing.These devices allow the extraction of detailed 3D information from a scene, boosting their effectiveness in detecting the human shape.However, they suffer from several limitations, such as interference from natural light, scattering, and limited range.Although visual data provides rich information, most of it affects individuals' privacy, prompting researchers to develop different methods for ensuring privacy protection in videos and images. 14Intelligent video analysis lends itself well to many application areas in health monitoring.Systems for daily-life assistance have been designed to monitor people with dementia, measure sleeping correlated with respiration, and track medication habits, 16 with the vision systems in infotainment gadgets fueling research interest in their use for healthcare applications. 17However, works based solely on computer vision techniques for monitoring and clinical evaluation of movement disorders are still in their infancy.
Fall detection is a major challenge in healthcare for the elderly, with videobased technologies offering many advantages over popular wearable alarms because they don't require user action and they're always active.Recently, RGB-depth (RGB-D) devices have successfully outperformed other sensing technologies for fall detection. 8In addition, the use of sensors on staircases can reflect musculoskeletal problems and recovery progress, with researchers recently proposing a general method for online estimation of quality of movement on stairs. 18he primary limitations of videobased systems come from cluttered environments, occluded scenes, and unstable lighting conditions.But even if these issues can be reduced by using multiple cameras or complementary devices, such as depth sensors, they're still open problems that must be tackled by integrating the information provided from different environmental sensors.Moreover, in most real-world applications, analyzing and processing data in real time is paramount, but existing methods can fail due to computational demands.The lack of a comprehensive and realistic dataset is also an issue.

Networking Technologies for Smart Homes
Existing networking technologies play an increasingly prominent role in modern AAL designs.In-home communications are well supported, and their performance, from a communication system perspective, is relatively well understood.These technologies are stable and mature, with current research focusing on incorporating different communication technologies into clinical applications that feature heterogeneous devices with diverse communication protocols.What gives sensing platforms the functionality of remote monitoring is ubiquitous network connectivity to close the loop between residents and clinicians.
Due to existing in-home infrastructures, wired technologies commonly provide high data-transmission rates.Among these, power line communication technologies are evolving in the field of smart home applications, especially advanced metering infrastructure and automated home energy management.Widely adopted systems use X10, KNX, and ITU-T G.Hn, IEEE 1901. 2 Various wireless networking technologies and communication protocols are summarized elsewhere 19 ; Figure 3 lists some of the typical shortrange wireless options.WiFi has the significant advantage of being Internet Protocol (IP) enabled.However, hardware with WiFi connectivity is still relatively power hungry and less suitable for battery-powered sensor motes in applications anticipating long-term deployment.To break down this barrier, an adaptive sublayer 6LoWPAN enables IPv6 for low-power, processing-limited, embedded hardware over low-bandwidth wireless networks.
Adopting these networking technologies requires guaranteeing the necessary communications throughput, power consumption, and hardware costs.Beyond this, to fully underpin a multimodality sensor system in a smart home, the IoT infrastructure must provide ubiquitous connectivity and interaction to all the sensing devices in a heterogeneous network circumstance.Additional advantages can be gained through IP-|enabled sensing networks because they remove  the need for translation gateways or proxies in hardware and software, thereby creating more seamlessly integrated AAL systems.However, datacollecting points in wireless sensor networks must have identifiers to be manipulated.If a unique "name" for a sensor is defined by an IP (more likely, an IPv6) address, the data-collecting point can be addressable through the whole end-to-end system.
Other important standards such as UPnP/DLNA, ECHONET, Open Service Gateway initiative (OSGi), and Continua Health Alliance are reviewed in detail elsewhere, 20 but for this article, it suffices to say that a significant barrier to their widespread use is their limited compatibility.Overlay networking protocols and metadata technologies aim to solve this problem.All AAL systems are built around a gateway device that provides remote access to sensor data to connect and bridge diverse networks.The home gateway implements multiple functions, such as a local monitoring/controlling center, intelligent agents, and network management.Nobuo Saito reviewed the home gateway from a broad, practical perspective and proposed an architecture suitable for better implementation and management. 21In the context of a sensing platform for healthcare and well-being in a smart home, middleware solutions embedded in the gateway address the fusion of different clusters of sensors, coordinating and managing highly heterogeneous systems.Several works address AAL systems specifically because middleware solutions are often designed for different application domains-for example, openAAL middleware defines a framework on top of the OSGi specification to facilitate integration and communication among services, including the context manager, procedural manager, and composer. 22 key factor for the IoT infrastructure to successfully enable AAL systems is to provide loosely coupled functionality, allowing autoconfiguration and dynamic interoperability among not only all devices but all end users as well.

Pattern Analysis and Machine Learning
The performance of different sensor technologies, in terms of reliability, discriminative ability, and monetary and energy costs, is context-dependent.Readings from individual sensors must be preprocessed, integrated, and mined to provide the most likely model of activity that maximizes information content in the given health monitoring context.Moreover, the decision-making process must be implemented and fine-tuned-in particular, it must consider the contextual knowledge of sensors and individuals.
Although we've seen some advances in applying machine learning techniques to ADLs, an end-to-end system doesn't currently exist in this space.What follows is a discussion on the state of the art of such a system's individual elements.

Quantification of uncertainty
Multiple heterogeneous sensors in a real-world environment introduce different sources of uncertainty.At a basic level, we might have sensors that simply aren't working or that are giving incorrect readings.More generally, a given sensor will at any given time have a particular signal-to-noise ratio; the types of noise corrupting the signal might also vary.
Consequently, we need to handle quantities whose values are uncertain, and we need a principled framework for quantifying uncertainty that will let us build solutions in ways that can represent and process uncertain values.A compelling approach is to build a model of the data-generating process that directly incorporates each sensor's noise models.Probabilistic (Bayesian) graphical models, coupled with efficient inference algorithms, provide a principled and flexible modeling framework. 23

Feature Construction, selection, and Fusion
Given an understanding of data-generating processes, sensor data can be interpreted to identify meaningful features, so it's important that it's closely coupled to the development of individual sensing modalities. 24Sensors might have strong spatial or temporal correlations, or specific combinations of sensors might be particularly meaningful.A key hypothesis underlying the SPHERE project 25 is that once calibrated, many weak signals from particular sensors can be fused into a strong signal, allowing meaningful health-related interventions. 26ased on the calibrated and fused signals, the system must decide whether intervention is required and which intervention to recommend-interventions will need to be information gathering as well as health providing.This is known as the exploration-exploitation dilemma, which must be extended to address the challenges of costly interventions and complex data structures. 27

adapting to Context and Domain Knowledge
Data mining and decision making must be contextualized and situated within a wide body of nontrivial, health-related background knowledge, which in turn requires highly explanatory models. 28he operating context will vary from training to deployment among different applications, residents, and households, so the incorporation of methods that are robust to these variations is critical.
Continuous data streams can be mined for temporal patterns that vary among individuals; these patterns can be directly built into the model-based framework and additionally learned on both group-wide and individual levels to glean context-sensitive and -specific patterns.A recent review of methods for dealing with multiple heterogeneous data streams appears elsewhere. 29inally, interfaces will need to communicate data and predictions.Such communications must be informative and assist in decision makingincluding the ability to communicate uncertainty and conflicts within that data, which is increasingly becoming an important issue. 30

Real-World Implementations
The premise for the SPHERE project is that we don't know what data is necessary to drive analytics for ADL identification and standardization across different homes and that single-modality sensing platforms can't answer this question fully.To address this, SPHERE has developed a multimodality sensing platform to collect data from 100 houses in the Bristol area.Figure 4 shows the overall architecture, which follows a clustered-sensor approach and is currently installed and running in a real house in Bristol.
The SPHERE system uses three sensing technologies: environment, video, and wearable sensing.The environment sensors include humidity, temperature, air quality, noise level, luminosity, occupancy, door contacts, and utility consumption (water, gas, and electricity), centrally and at the appliance and faucet level.The currently deployed system uses 40 nodes, providing more than 90 data points, all structured and time stamped to establish context and temporal relationships.The video sensors are RGB-D devices placed in various locations, such as the living room, kitchen, hall, and staircases.The video sensors let us gather information about residents' cadence, gait, and 3D trajectory throughout the smart environment.The wearable sensors are Bluetooth low energy (BLE) devices with dual accelerometer data; they support dual-operation mode (connection-oriented and extra-low energy connectionless communication modes) to provide full 50-Hz accelerator measurements in addition to localization services.
The data from each sensor cluster is collected in a SPHERE home gateway that maintains time synchronization in the system and, in addition, controls data access to ensure user privacy.The data from the gateway is collected by a heterogeneous data management platform (SPHERE's data hub), which manages data access and makes a dynamic library of data analytics services available for registered end users.The current system is operational and undergoing scripted validation experiments, with multiple sensor domain data processed to establish ADLs against external (manual or automatic) activity tagging.On deployment, the data from the environment, wearable, and camera sensing subsystems are fused and processed in real time for activity and health monitoring in longitudinal and focused studies.A key objective of the SPHERE project is to deliver datasets with a strong focus on the richness of metadata annotations, as well as the experimental and user contexts to provide to the wider research community a platform for improved understanding of their roles in behavioral trends for healthcare.
E ven at a high level, this overview reveals certain gaps and challenges caused by the multidisciplinary nature of the systems required to provide AAL data and applications.Some of these challenges aren't unique to ehealth, but they're happening in fields where researchers want data collected in multiple domains from multiple technological systems-not necessarily designed or even deployed together-to bring together a cohesive, stable, and reliable view of the measured activities.
Although individual technologies will continue to be developed, whether wireless or wired, the main challenge remains the design of analytics-driven, data-gathering platforms that provide a rich set of data efficiently, reliably, and on-demand.SPHERE is addressing this by building a multimodality sensing system as an infrastructure platform fully integrated, at design stage, with intelligent data processing algorithms driving the data collection.

t h e a u t h o r S
Ni Zhu is a postdoctoral researcher working on the SPHERE project at the University of Bristol.His research interests include the design and development of Internet-based embedded system, next-generation wireless sensor networks, and telemedicine systems.Zhu has a PhD in telemedicine from the University of Manchester.Contact him at ni.zhu@bristol.ac.uk.
tom Diethe is a research fellow on the SPHERE project at the University of Bristol.His research interests include probabilistic machine learning, computational statistics, learning theory, and data fusion.Diethe has a PhD in machine learning applied to multivariate signal processing from University College London.Contact him at tom.diethe@bristol.ac.uk.
Massimo Camplani is a research associate on the SPHERE project at the University of Bristol.His research interests include areas in computer vision.Camplani has a PhD in electronic and computer engineering from the Università degli Studi di Cagliari.Contact him at massimo.camplani@bristol.ac.uk.lili tao is a postdoctoral researcher on the SPHERE project at the University of Bristol.Her research interests include human motion analysis, 3D deformable objects reconstruction, and facial expression analysis.Tao has a PhD in computer vision from the University of Central Lancashire.Contact her at lili.tao@bristol.ac.uk.
alison Burrows is a postdoctoral researcher in User-Centred Design on the SPHERE project at the University of Bristol.Her research interests include inclusive design, user experience, ethnography, and design research methodologies.Burrows has a PhD in inclusive design from Loughborough University.Contact her at alison.burrows@bristol.ac.uk.
Niall twomey is a postdoctoral researcher on the SPHERE project at the University of Bristol.His research interests include data mining, fusion of environmental sensors in smart home environments, and the use of digital signal processing, machine learning, and application-centric decision making for objective health and wellness assessments.Twomey has a PhD in machine learning applied to signal processing from University College Cork, Ireland.Contact him at niall.twomey@bristol.ac.uk.
Dritan Kaleshi is a senior lecturer of communication networks at the University of Bristol.His research interests are in network protocols, distributed systems design and performance evaluation, and data interoperability for sensor/actuator systems, including new networking architectures, protocols, and algorithms for large-scale, loosely coupled systems such as the Internet of Things.Kaleshi represents the UK in various international standardization bodies (ISO/IEC, CEN/CENELEC) related to home networking and the smart grid.Contact him at dritan.kaleshi@bristol.ac.uk.
Majid Mirmehdi is a professor of computer vision in the Department of Computer Science at the University of Bristol, and graduate dean and faculty graduate education director in its faculty of engineering.His research interests include natural scene analysis and medical image analysis.Mirmehdi is editor in chief of IET Computer Vision Journal and an associate editor of the Pattern Analysis and Applications Journal.He's a senior member of IEEE and a fellow of the International Association for Pattern Recognition.Contact him at m.mirmehdi@bristol.ac.uk.
Peter Flach is a professor of artificial intelligence at the University of Bristol.His research interests include mining highly structured data and the evaluation and improvement of machine learning models using receiver operating characteristic analysis.Flach is editor in chief of the Machine Learning journal and author of Machine Learning: The Art and Science of Algorithms That Make Sense of Data (Cambridge Univ.Press, 2012).Contact him at peter.flach@bristol.ac.uk.
Ian Craddock is director of the SPHERE project at the University of Bristol.He serves on the steering board of the university's flagship Elizabeth Blackwell Health Research Institute, and is also Toshiba's managing director for its Telecommunications Research Lab in Bristol, responsible for a portfolio of both internal and collaborative communications, healthcare, and smart city research.Contact him at ian.craddock@bristol.ac.uk.
Selected CS articles and columns are also available for free at http://ComputingNow.computer.org.
Bridging e-Health and the Internet of Things: The SPHERE Project Ni Zhu, Tom Diethe, Massimo Camplani, Lili Tao, Alison Burrows, Niall Twomey, Dritan Kaleshi, Majid Mirmehdi, Peter Flach, and Ian Craddock, University of Bristol interventions from relatives, caregivers, and healthcare professionals.

Figure 1 .
Figure 1.Physiological signal-monitoring systems.Different concept-to-prototype systems have been proposed and implemented in response to various healthcare issues.

Figure 2 .
Figure 2. Sensor taxonomy of environment, wearable, and video sensing modalities used in ambient monitoring and activity detection.This isn't an exhaustive summary, but it demonstrates different sensor modality usage.

Figure 3 .
Figure 3. Short-range wireless networking technologies in the home.WiFi has the significant advantage of being Internet Protocol (IP) enabled.

Figure 4 .
Figure 4.The proposed system scenario of the sensing platform.The clustered-sensor approach is currently installed and running in a real house in Bristol.
PIR), light, microphone, television IR sensor, weather conditions, internal and external light levels, temperatures, pressure, humidity, smart meters Accelerometer, blood pressure sensor, blood glucose monitors RGB camera, depth sensor, time-of-flight camera, thermal infrared imagery Tri-axial accelerometer, blood pressure sensors, blood glucose sensors RGB camera, depth sensor, infrared imagery PIR, microphone, IR sensor, weather conditions, internal and external light levels, temperatures, pressure, humidity, solar average rate, wind speed and direction