Digital Emotion: Designing AI with Empathy

UX design is all about understanding your users and designing to enhance their lives and help them achieve their goals. A key part of this is using empathy to understand the nuances of a user’s life - how the actions they might take in your product are influenced by health, finances, family etc.

This is, in a nutshell, what design empathy is about. Admittedly, it is an ideological school of design, but as new technologies develop, we start to see many of the same systems implemented elsewhere.

This is a good thing, really: new digital technologies need design empathy. Moreover, there is a strong positive correlation between empathy and a buyer’s level of trust as well as satisfaction.

The importance of AI-based design empathy

There are few technologies where the empathy-first approach is as apparent as in the field of AI. Unfortunately, without good UX designers with empathy-driven ideas on hand, AI-based apps and software have the potential to destroy customer trust and completely mismanage experiences.

Although AI has a long way to go before most businesses need to worry about full-scale implementation, many businesses already utilise algorithms, big data and other AI-driven practices to interact with or sell to customers. Interestingly, studies found that customers are more forgiving of algorithms making errors than humans - as they perceive algorithms to have a lack of agency compared to humans.

Despite the power of AI-driven software, some risks cause significant issues. As designers and creators, we are responsible for building many forms of Machine Learning (ML) or AI-powered experiences. We have to practice empathetic thinking to help guide the utilisation and implementation of AI.

To build empathy, you must first do research.

It’s that empathy that drives much of our focus on user research. Here at KOMODO, we know that to really understand your users, you can’t just rely on a catch-all user persona. Instead, you need to understand them on an emotional level to design features that have a contextual understanding of how a user will interact with them.

Target gets it wrong

Many businesses have been stung by implementing AI-based software or techniques without building empathy into the experience. The US brand Target, for example, used an algorithm based on purchase patterns to predict when customers were pregnant, which was found to have a strong success rate. They would then send baby vouchers to customers whose high scores indicated that they were pregnant.

However, the lack of empathy-driven thinking in this process is clear: what if a customer doesn’t know they are pregnant themselves? What if they are surrogating their baby? What if they plan to give it up for adoption? Target found themselves in hot water back in 2021 when a man complained that his high-school-aged daughter was being sent baby vouchers. Target apologised, but the man later called back to inform them that she was, in fact, pregnant.

Target, luckily, recognised this as a moment of change. So they stopped sending baby-specific vouchers to highly scoring customers and instead littered baby ads in amongst other products so that the audience didn’t feel spied on. It worked, with revenue shooting up and fewer complaints coming in.

While this example is more of an issue with privacy, it’s also a clear sign of a lapse in empathetic thinking.

IBM really gets it wrong

In 2013, IBM partnered with The University of Texas MD Anderson Cancer Centre to develop a new oncology advice system. Its goal was to cure cancer. IBM’s data engineers, unfortunately, built the AI-driven software tool on hypothetical cancer patients rather than on real patient data (again, poor user research as well as lacking empathy).

This led to a product that was delivering unsafe cancer treatment recommendations to patients - with one example stating the system advised a patient who had severe bleeding from taking a drug that would actually make the bleeding worse. With $62 million invested in the project, Watson for Oncology was cancelled by 2017 and marked as a colossal failure.

The poor user research is a problem here, but it also lacks empathetic design thinking. If the engineers had worked with a good UX designer who valued empathy-driven processes, they might have realised that the system had to really understand genuine patients and their needs.

KOMODO practising empathy

Datatrial and Healthcare

Our team understands the value of user research and empathy-driven design ideology. Take, for example, our work with a HealthTech company named Datatrial. They needed a patient booking app that would make it easier for patients to book their appointments and monitor their progress.

However, the patients were at various stages of life-changing illness and treatment cycles. We needed to really get to know the user base to design an app that helped them, could accommodate for periods where the patient was feeling more ill than usual and didn’t cause unnecessary stress or frustration.

But remember: the app was also built for a client, not solely their patients. It needed to be able to gather useful data on the effectiveness of trials - which was a challenge when patients were feeling low and ill, making them more inclined to respond negatively. We recognised this and devised a dot-based feedback system rather than a ‘positive/negative’ one, allowing users to rate their experience without having to opt for binary yes/no style answers.

Empathy in housing arrears

Another example of bringing empathy to a data-driven process occurred when we were asked by Orchard, a software company in the social housing sector, for help tackling arrears.

Social housing arrears are a sensitive issue - some occupants fall into arrears accidentally, or very rarely, whilst others are more neglectful of their responsibilities. But no software system supported that level of nuance - all any existing system could do was monitor who was in arrears or who was not.

We performed detailed in-person user research for this task, interviewing officers and shadowing them as they visited tenants. We also interviewed stakeholders, as the software had to also solve their challenges.

However, most relevant to this article is the fact that we used occupant data in a GDPR-compliant and sensitive way to build AI-based prediction models that could spot when tenants might go into arrears and report on the factors that led up to it.

This gave officers far more visibility over tenant situations. They could see, for example, if a tenant that had just gone into arrears had reported multiple repairs but not had them carried out, meaning they were justifiably refusing to pay.

This entire process was one of empathy-driven design. We worked hard to understand who the users were, their situations, and how we could solve the problem without negatively impacting tenants. In the end, it was Orchard’s most successful product launch and helped reduce instances of arrears, give more insight to the housing organisations and ultimately, make tenants' lives better.

Do you practice empathy in design?

Design empathy and AI empathy go hand-in-hand. AI-driven software demands that developers understand nuance and user emotion. But it’s the role of UX designers to give users an empathy-driven interface and features that allow them to interact with the AI.

At KOMODO, our team can help you understand your users through real-world user research and data-driven analysis, then build systems that can combine AI and UX to deliver unrivalled personalisation that actually helps users, rather than accidentally alerting their families to undisclosed pregnancies…that’s right, Target.

Get in touch if you’d like to know more.