How medical symptom checkers can shine a light on robo-advice

While running in the great outdoors this weekend, I was listening to the Skeptics Guide to the Universe podcast, which explores and de-bunks pseudo-scientific claims. It got me thinking about the delivery of financial advice, and automated advice in particular.

After discussing Covid-19, co-host Steven Novella, a medical doctor, went on to question the accuracy of online symptom checkers (SCs). In a recent blog post, he summarized the findings of an Australian study that evaluated the accuracy of 27 online SCs.

Novella characterized the study’s results as “pretty disappointing.” On average, the correct diagnosis was listed first 36% of the time.

Not bad at first blush to hit it on the head more than one-third of the time, but the correct diagnosis was only listed among the top 10 possible diagnoses 58% of the time. For practical purposes, that means the correct diagnosis was missed 42% of the time, as most patients likely don’t look beyond the top 10.

More interesting to me, though, were the potential reasons why the diagnoses were so poor.

Parallels with automated financial advice

As the panellists shared their thoughts, I couldn’t help but see the parallels with automated tools in the personal finance and investment fields. These include do-it-yourself online brokerages and robo-advisors that offer online investment portfolios, but also advisors’ own tools: from the reference sources that clients never see, to side-by-side processes, to that solo interaction between the client and the evaluation tool.

Wherever the technology lies on that spectrum, it can’t be a proxy for the advisor.

As with the doctor-patient relationship, a financial advisor has to understand the client in order to deliver personalized advice. The less the advisor interacts with the client, the more difficult it is to deliver.

And even with the benefit of direct contact, it remains the advisor’s responsibility to apply and explain the tools and their output so the client’s interests are adequately served.

Points to ponder

Here are some of the podcast panellists’ thoughts on symptom checkers, and how I see them applying to financial advice:

Quality of input

Patients are often not very good at describing symptoms. People may say numbness when they mean weakness, or they may treat the two as the same, Novella said.

A client may use similarly imprecise language to describe their personal goals or their feelings about risk.

Alternatively, a client may adopt words offered by the document they’re filling out or the platform they’re using without appreciating nuances. Absent the advisor probing further, the premises for a client’s action (or inaction) may not properly reflect their intentions.

People come to you with a narrative

Both medical patients and financial clients can be biased, frequently responding with their relative feelings anchored on recent experiences. One has to ask questions in multiple ways to deconstruct the objective facts from the narrative, and then reconstruct those facts in the medical or financial arena where they are to be applied.

It’s a dynamic investigative process that takes a lot of insight into how people think and communicate. A focused professional can then use this knowledge to move the relationship forward.

Body language

An experienced physician reads all the signs, including the patient’s body language. Advisors can observe a client’s posture and tone of voice, and how a couple interacts — things you don’t experience through checks in boxes or even the most eloquent text summaries.

Triage function

This is the “Do I go to the hospital?” moment. The SCs reviewed in the Australian study scored 49% on this measure, though they erred more on the side of sending people when it wasn’t necessary.

While there’s no life or death counterpart in financial advice, the accumulated harm of unverified guidance could make it difficult for someone to recover should problems materialize in later years.

The “why” of the output

It’s not enough to produce an output and expect that it will speak for itself. Newer SCs that use artificial intelligence give reasons for why they predict certain things, as well as how sure they are about the output provided.

That’s a large part of the financial advisor’s value to the client: explaining where things come from and where they are headed. It’s a check against the quality and thoroughness of the inputs that led to the output, and an opportunity to reinforce the client’s confidence and commitment to the plan you are creating together.

As a final point, it should be noted that SCs have no common regulation, if any at all. Comparatively, there is plenty of regulation in the financial field, and compliance departments help advisors stay vigilant and within boundaries.

Still, financial tools can be very complex. It’s incumbent on advisors to fully understand what is at their disposal so that digital platforms are used as tools of the advisor, and not in place of the advisor.

Doug Carroll is tax and estate counsel to Aviso Wealth in Toronto