Designing sensor-based strategies to address workplace efficiency and improve care delivery in a nursing home community.
My Role:
My Role: HCI Researcher
Project Duration: 2023 - Present
Skills:
Rapid ethnography
Survey
Narrative design & Storyboarding
AI prompt engineering
Tools: Nvivo, ChatGPT4 (DallE)
I led the execution of the project at all phases. My contribution ranged from defining recruitment protocol for data collection, data analysis, writing, and research dissemination at a high-impact conference venue and journal. I designed and implemented the culminating design artifact— I designed the two sensor-based narratives in the form of a video-based storyboard to address the stress and workload burden of direct care staff in the participating nursing home and prototyped that story using generative AI prompt engineering.
Collaborators: 5 researchers
Context
National nursing shortage
High reports of stress among healthcare workers
Concerns about the use of sensors in the workplace
Determining Community Goals
Determining Priority Goals
Over two consecutive days, two pairs of research team members conducted four-hour observation sessions during the morning and afternoon shifts to understand the sociotechnical context of the community. Following the field study, supported by two researchers, I purposively recruited participants using convenience sampling and administered a 5-minute survey. Participants responded to questions asking them about potential goals for the nursing home community, suggested other goals through a free-form response, and ranked the top three priority areas from most to least important for the nursing home community.
Overall, 25 respondents were recruited through purposive and convenience sampling, including direct care staff (nurses), administrative leaders, residents, and family care partners of residents.
Defining Problem Statement
Staff mental health (72%, 18/25) was ranked high among direct-care staff, administrative leaders, and residents, followed by Residents' safety (52%, 12/25), ranked high among family, staff, and leadership. However, our findings showed that staff shortages impinge on resident care and increase staff stress, highlighting the interconnectedness of both goals.
While sensors cannot directly address staffing shortages, staff stress – the next popularly mentioned issue related to mental health to an extent can be addressed through sensors, and because participants considered that staff stress is paramount to resident safety. We reasoned that by improving the former, we also might improve the former. Therefore our new problem statement became
What are the perceptions of nursing home leaders, care staff, residents,
and family care partners about tailored sensorization plans to reduce
staff stress?
Ideation & Prototyping
Identifying Use Cases and Sensors
We took a two-pronged to sensor plan development.
In this phase, led by Lewis Davis II, we explored possible use cases and sensors that could be used to address stress following the workflow created by Roberto Yus, Ph.D.
We focused our ideation on two stakeholder groups– nurses and residents since both of these stakeholder relationships directly impact the quality of care and the extent of stress. to achieve this, we focused on two broad use cases
Monitor residents to reduce staff burden and overall residents physical or emotional needs.
Monitor staff and identify an “optimal time” to prompt someone and/or whether an intervention was effective. For example, prompt staff to engage in behaviors to reduce stress (e.g., forcing to take breaks) or prompt staff to improve their time management while caring for residents.
After achieving team consensus through voting, we settled on 2 plans– a) to monitor staff for signs of stress and alert them, and b) reduce staff workload burden by also targeting a painpoint affecting residents—pressure ulcers.
We then identified existing sensors in the consumer market. We weighed the use cases against the sensor performance, cost, reliability, operational power consumption, and portability. We also evaluated the sensors based on data security, level of invasiveness, implementation, and monitoring assessment. In the end, we settled on using wearable and environmental sensors.
Narrative Design & Storyboard Prototyping with ChatGPT4
In stage 2, I collaborated with three researchers ( Tera Reynolds, Yoon Kim, and LSDII) to implement the final prototype. This process involved developing the storyline, scenes, and characters, then using these design elements, such as the scene descriptions and keywords in the scripts as inputs for chatGPT4 to create the storyboards before recording the video. Please read more about this process <<medium article link>>
A scene describing Tamara a direct care staff who works in a nursing home
Tamara is seen interacting with a wearable sensor that monitors her electrodermal activities for signs of stress
Tamara is seen here with a resident, Ms Johnson who is wearing a wearable chest sensor that tracks and documents residents turn activities in order to lower the risk of pressure sores
Results & Reflection
Reflection on Designing AI-based Storyboard
Using ChatGPT for storyboarding was a new and challenging experience but a process worth embarking on. I remember when I started to think about the interview phase, my team members were unsure of how to demonstrate the plans most effectively in the shortest time possible to the participants. It was then my design mind kicked in– honestly, I was itching for a design challenge after such a long hiatus. As the only designer on the team, I suggested and felt confident that we present the sensor-based strategies with narrative storytelling.
I don’t consider myself a fantastic artist, but over the years, I have experimented with hand-drawn storyboard prototypes and even done them with high-fidelity tools such as Illustrator and Photoshop. I felt that a high-fidelity prototype would be most effective, but I was worried that implementing it with Illustrator would be time-consuming due to the research timeline. Alas! I opted for my mentor’s (Kelly Doran) suggestions to try using AI to create the storyboard. Boy, that was a challenge because rather than use stock images, I opted to go the prompt engineering route to generate my images. Plus I have always wanted to create a comic strip and this was the right challenge to do it. I also wanted to know if using AI was an efficient approach in comparison to using applications in the Adobe suite.
Overall, the learning curve was steep. I faced a couple of challenges: 1) Figuring out how to generate consistent character depiction using text was a kung fu experience, and of course, ChatGPT had a mind of its own, but I was up for the fight. I combined the consistent character and storyboarding plugins, including the image editing features in ChatGPT 4, to refine my images. I did not achieve a consistent-looking image throughout my storyboard, but ChatGPT was, to a great extent, helpful and consistent with the scene depiction and the description of the character (e.g., race, color of the nurses scrubs). I soon found that creating consistent images and scenes involved a great deal of specificity and detail, or else ChatGPT would make its suggestions, which would sometimes deviate from my intended goal for the scenes.
2) Being locked out from generating images wasn’t fun despite using a paid version. After a couple of iterations, ChatGPT paused my account from generating images for several hours (up to 12hrs). The frustrating part was when I resumed ChatGPT, doesn’t remember my history nor reused my images. Several times, I had to start from scratch to create the character. Thankfully, it helped that I was saving all the prompts I used to generate my best images and had a good variety to select from when it was time to record my narrative video
Would I use ChatGPT as a prototyping tool in the future? Certainly! I feel confident with the acquired skill – prompt engineering, my next venture into AI–based storyboarding would prove more efficient as I do plan to use it more in my healthcare research and possibly recording an actual video.
Preliminary Findings
The final prototypes were converted to a video narrative and used as the central probes by the research team during the interview phase of our data collection. We used the probes to elicit participants’ opinions on a sensorized nursing home environment.
In summary, participants shared their opinions about their preferences, acceptance, and feasibility of both sensor-based strategies. Overall, sensor plan B turned out to be the most preferred strategy among participants, although they did not recognize its direct impact in reducing staff stress; rather, they felt that it helped with task management and documentation. This was so because several participants felt that tasks might be completed for residents assigned to direct care staff, yet they may still feel stressed due to staff shortage; thus, many questioned the feasibility of Plan B in their community. Finally, both plans were widely accepted by participants because the data collected was commensurate with the purpose and was not invasive, although many felt that it should be anonymized due to concerns about job security.