MEMS sensors can help make healthcare technology sensitive to patients’ needs and abilities, significantly enhancing the user experience.
With the Baby Boomer generation reaching retirement age, a wave of elderly citizens is about to hit the United States. As people get older, they become more likely to develop serious health conditions. The CDC reports that 1.5 million people currently reside in skilled nursing homes. The spike in elderly citizens will likely result in major shortages of caregivers and assisted care facilities. Worldwide, there were 357 million people aged 65 and older in 1990; that number is expected to more than double by 2025, to 761 million.1
How will we care for all these people, and what sorts of lives will they lead?
We know that real quality of life is directly tied to independence. We like to do what we can for ourselves. Intelligent systems can help to maintain or enhance one’s ability to live independently longer. However, it is important that such systems strike a careful balance. Quality of life technologies (QoLTs) are intelligent systems that take into account the abilities, needs, and intentions of the user. These systems can adapt to an individual’s unique circumstances without forcing the individual to accept more assistance or automation than they desire.
In essence, QoLTs must enhance, rather than replace, natural human capabilities. For this reason, QoLT systems must often be aware of the people around them and the contexts and environments in which they are used. They must be capable of responding quickly to changing conditions. Sensor technologies are therefore integral to the QoLT approach. Microelectromechanical systems (MEMS) provide a foundation for the continual awareness, recognition, and learning that is required by efficient and effective QoLT design.
“Providing intelligent sensing and actuation, which can be combined with electronics processing ‘muscle’ like ASICs, microprocessors, and even DSPs, MEMS enable a high degree of interactivity with the environment,” says Karen Lightman, managing director of the MEMS Industry Group. “MEMS pack this intelligence into a small footprint, making it the ideal companion for resource-constrained applications.”
There are several projects in development at the Quality of Life Technology Center, an engineering research center run by Carnegie Mellon University and the University of Pittsburgh and sponsored by the NSF, that provide examples of how MEMS technology can improve a user’s quality of life. These projects enhance quality of life in relation to the user’s mind, body, and community.
Tracking technologies are now prevalent, but they are often reactionary, inflexible, and context-insensitive. QoLTs must go beyond monitoring and activity tracking to account for highly complex and personalized factors, such as user intent, engagement, preference, and motivation. Devices developed under the emerging virtual coach concept leverage sensor technologies to provide assistance based on awareness of a person’s cognitive or physical activities and abilities. These devices facilitate a person’s functional needs by monitoring what they do, providing appropriate support as circumstances change, and giving timely feedback to users and clinicians. Such devices can prompt the steps and order in which to perform a specific task or provide clinical guidance on therapy or rehabilitation routines. One goal is to help people understand their behaviors and motivate them to make positive changes.
The MemExerciser, for example, enables people to engage in reminiscence therapy. The system captures the user’s experience with wearable sensors (a camera, audio recorder, and GPS logger). The system then uses content and context to analyze the information, the best of which is made available to the caregiver, who then selects the most meaningful photos, sounds, and annotations to cue the user’s memory. The MemExerciser has been shown to improve cognitive recall among people with memory impairments.2 The device facilitates a self-guided review method for exercising memory, freeing caregivers from having to repeatedly answer questions about events or activities that the user cannot recall.
Similarly, tracking technologies that capture experiential information through first-person or inside-out perspectives can improve user-friendly devices’ ability to understand, recognize, and respond to human intent. A prototype of such a device, First-Person Vision, consists of two on-board cameras and audio and movement sensors embedded into a pair of eyeglasses. The sensors record information about the activity being performed as well as the gaze and focus of the user. An analysis of the concurrent dataset can provide insight as to user intent. As a QoLT device, First-Person Vision helps a user recognize faces, gestures, actions, and locations, enabling a range of assistive products for people with various impairments.
A new offshoot of this area of development tackles emotion recognition. Research in sensing and machine learning has advanced enough that researchers can classify user activity and make key inferences about user intent. Detection of human emotion will be instrumental in modulating the system responses of virtual coach devices. Studies are underway to determine what visual and audio sources can reveal about the role of facial muscles in conveying emotion. As this type of work advances, it is expected that different levels and kinds of emotion-recognition strategies will be heavily integrated with virtual coaching to help optimize the assistance capabilities of other emerging QoLT systems.
Clinicians’ needs have driven the development of many virtual coach QoLTs. Partners of the QoLT Center have cited frustration with their inability to affect the behaviors of patients outside of the clinical setting. Obstacles include simple usability issues, such as a difficulty getting patients to remember how to operate assistive devices, and more complicated user-engagement concerns, such as motivating patients to perform prescribed exercises. The virtual coach concept can potentially meet these various challenges, resulting in a positive impact on patients’ physical well-being.
Rehabilitation gaming is one area of increasing interest. Researchers are exploring adaptations of the Microsoft Kinect gaming system to create a personal exercise coach that would help patients correct their position and posture during physical therapy routines practiced at home. Gaming elements can help motivate patients to regularly maintain their therapy, resulting in fewer accidental injuries and higher overall success rates.
But virtual coaches in the physical realm go well beyond the traditional mode of avatar-based systems. Wearable sensors are also surfacing in commercial markets. Devices like BodyMedia’s FIT armband are equipped to track caloric intake, monitor sleep quality, detect mood changes, or alert individuals to physical stress thresholds.
In addition to their applications as healthcare products, QoLT devices can have mainstream appeal. Consider VibeAttire. Users plug an MP3 audio player into a sensor-lined vest. When the music plays, the vest vibrates to the beat. The product’s sensory substitution-based approach is especially important for people with hearing impairments, who will be able to feel and respond to music, possibly the first time in their lives. It is also something that anyone, hearing or not, can enjoy.
Among wearable QoLTs, eWatch is an activity monitor designed to optimize ergonomics and wearability. The multisensor platform resembles a normal wristwatch. It monitors a user’s physiological states, runs the data it gathers through machine-learning algorithms, and provides analysis-based feedback and advice to the user. eWatch is versatile and has been integrated with a range of QoLT prototype products, including different virtual coach devices that help patients with a range of tasks, such as dealing with carpal tunnel syndrome, managing personal wellness, and using manual wheelchairs.
Some devices interface intimately with end users as a natural extension of the users’ physical space. The Seating Coach falls into this category. It is a robust system that helps wheelchair users comply with clinician recommendations, such as limiting use to an appropriate duration and keeping a posture that will prevent pressure ulcers from developing. Functions include maintaining clinician-prescribed tilt, recline, and leg elevation; monitoring seat pressure; and delivering reminders about overdue activities. The system also helps clinicians train clients to properly use power-seat functions, reducing the likelihood of an adverse event.
QoLTs can be employed in group home settings to minimize some of the burden on our healthcare system and caregivers. A communal health kiosk enables senior citizens to track their health and vital functions on a regular basis without having to visit the doctor. The integrated multisensor system can perform physical, physiological, cognitive, and behavioral monitoring of a patient, thus tracking their total well-being. A remote clinician arranges for appropriate tests to be taken at the kiosk; the results and compliance reports are then recorded and returned to clinicians electronically. Tests include those that can measure blood pressure, blood oxygen levels, handgrip, weight, hearing, visual acuity, and stress levels. The kiosk displays historical data, so patients can review their progress regularly, via carefully designed visualizations.
Another way to detect changes in health is to monitor daily activities through embedded assessment. This approach involves integrating sensors directly within the individual’s personal space or in frequently used items. A current research study examines how well a person performs routine tasks like making coffee, taking medicine, and answering phone calls.3 Embedded sensors detect and log successful and unsuccessful attempts to use devices around the home. The logged data is then presented back to the study subjects with visualizations that foster self-reflection about the possible causes for deviations from routine behavior. For example, logged data may show an increase in misdialed or incomplete phone call attempts. The data could alert users unaware that they are frequently dialing the phone incorrectly to potential changes in their cognitive abilities.
A company is making strides in whole-home sensing with a pilot program. Each cottage in Blueroof Technologies’ Smart Cottage Community features a discrete sensor network that spans standard residential appliances and infrastructural elements, capturing information about a broad range of daily activities. Sensors throughout the environment can monitor several things, including whether the stove, oven, washer, and dryer are on or off; whether the refrigerator, cabinets, and drawers have been accessed; or whether a shower, faucet, or toilet is running. Routinely used furniture items, such as sofas, chairs, televisions, and phones, can also be monitored. A smart medicine cabinet can assist with medication management and reminders.
Recent experimentation focuses on advancing the smart home idea to facilitate high levels of sensor-based assistance. The Cueing Kitchen, which is being developed by the QoLT Center as part of a different project, explores the integration of lighting and projector systems to prompt patients with cognitive impairments to complete basic cooking tasks. For example, the system might detect that a user has started to make a peanut butter sandwich. It would then cue the user to remember the location of the items needed by sequentially illuminating the areas where the plates, silverware, bread, and peanut butter are stored. Exploratory work also includes personalized social coaching (PSC) software and applications. One example of PSC considers the use of conditional and context-specific feedback as an alternative means of interacting with smart home devices. PSC technology could be designed to supplement the user’s interactions with their smart home by leveraging learned preferences and data collected from smartphone sensors.
These QoLT systems help people to remain active, productive and employable well into their golden years. They also instill greater confidence, safety and mobility across a wide range of environments. A QoLT design process has been used to develop a wide array of highly application-specific virtual coach systems. In fact, more than 15 systems currently exist at different stages of commercialization readiness. Among them are Navigation Assistant, Medical Device Usage Coach, Head Coach, Building Navigation Coach, IADL Coach, “Who’s That?” Coach, Ergobuddy, Fitness Coach, Personal Safety Coach, QoLT Usage Coach, Manual Wheelchair Coach, and a Carpal Tunnel Coach. These systems are often built by graduate and undergraduate students working with the QoLT Center. The systems’ development is made possible by a growing set of reusable components, including sensor data processing, sensor fusion, context-aware sensing, machine learning, ecological momentary assessment, data visualization, and computer vision. These components must prove stable and be sufficiently documented, so engineers at varying levels of proficiency can use them to build a complete system quickly.
Assessment is hardly an afterthought. The methods and metrics for assessing quality of life outcomes are adapted to the life cycle development process of these emerging technologies. Early in the development process, small sample studies take place involving focus groups and targeted end-users who provide qualitative feedback on the potential of a given technology to improve both general and targeted quality of life outcomes for specific individuals. These focus groups also provide design feedback, which guides further development of the technology.
As systems or devices evolve, the user studies become larger and more structured, and the data becomes more quantitative. Several technologies (e.g., First Person Vision, Health Kiosk) are now at the field-testing stage, where standardized quality of life measures are used. Three levels of measures are addressed: one assessing impact on general quality of life (e.g., the Euroqual, a qualitative research methodology for social sciences used in Europe); another assessing domain-specific quality of life (e.g., the burden imposed on the family caregiver by the First Person Vision system); and a third outcome that is specific to the targeted goals of the given technology (e.g., improving the ability of caregivers to manage difficult patient behaviors).
QoLTs represent a fast-approaching era of ubiquitous computing and advanced human-computer interfaces that extend and enhance both our lives and our selves. Virtual coach systems, in particular, provide memory and reasoning support wherever a person goes and whatever he or she does. At home or in the community, these systems integrate with all aspects of daily living to provide support for our physical, mental, and emotional states. They help us capture context for learning about how different aspects of our lives contribute to overall patterns of behavior and activity. They arm us with information that we need to take steps to reduce, overcome, or eliminate obstacles to healthy living. And they promote a sense of self-awareness that is critical for motivating and maintaining positive changes as we age into a healthier future.
1. NR Hooyman and HA Kiyak., Social Gerontology: A Multidisciplinary Perspective, 6th ed. (Boston: Allyn and Bacon, 2002).
2. ML Lee and AK Dey, “Lifelogging Memory Appliance for People with Episodic Memory Impairment,” in Proceedings of UbiComp 2008 (Seoul, Korea: ACM, 2008), 44–53.
3. ML Lee and AK Dey, “Reflecting on Pills and Phone Use: Supporting Self-Awareness of Functional Abilities for Older Adults,” in Proceedings of CHI 2011 (Vancouver, Canada: ACM, 2011).
Daniel P. Siewiorek is a Buhl University professor of electrical and computer engineering and computer science at Carnegie Mellon University’s Quality of Life Technology Center.