Helping Physicians with AI, The Data Science of Health

Promising AI powered physician assistance tools are exciting because they change work models clinicians use to treat patients, improve medical outcomes, and save lives.


New medical technologies can seem like science fiction. For instance, if you've ever watched Star Trek, you likely saw characters use a “tricorder,” a device that can ‘scan’ individuals for signs of disease or conditions, interpret diagnostic information, and sometimes take corrective action -- all in more or less in real time.

The average Star Trek fan might not realize that many capabilities of the tricorder actually exist now.

New physician assistance tools use a similar model, based on the ability to combine various functions to streamline or automate medical work.

One way to think of this is in terms of three broad functions – the first one is the collection and aggregation of information, often through sensors. Sensor-based technologies have been around for a while, but they're quickly taking off in healthcare while being paired with other tools. They’re also evolving in how they collect health data. One example is the abundance of current tools that record physiological functions like heart rate in real time. Just a few years ago, these technologies were not widely available. Their recent emergence has brought vast change to healthcare, in the treatment and diagnosis of conditions like atrial fibrillation, and in general efforts to figure out whether a patient is experiencing either tachycardia or bradycardia, whether immediate or chronic.

The second type of broad function takes this data and transforms it into insights. In the medical world, this is often focused on diagnosis. Data by itself is not inherently meaningful, unless it assists a pattern of comprehension. Artificial Intelligence models excel at pattern recognition and are built to understand, and often to present, patterns for easier recognition to humans.

The third function transforms these insights into actions – by orienting or focusing clinical decisions and clinical work. This can mean training machines to provide relevant results – and training doctors to harness these results.

When both humans and machines are trained effectively, the collaborative results can be impressive.

Today's technologies don't look just like what's on Star Trek, but they have some of the functionality built in, and there's always the potential to enhance and improve on these capabilities over time.

The advent of machine learning and artificial intelligence, and the progress made over the last few years, has the potential to contribute to better medical outcomes for millions of patients. These new tools are based on a very different fundamental philosophy of care -- the idea that capable decision support tools can help doctors to improve their accuracy, and enhance what they can do in the exam room and in the operating room.

One of the best ways to judge how important new AI-driven medical systems are is to look at the numbers in terms of dollars spent. A study from Accenture shows current spending estimates of $40 billion for robot-assisted surgery, $20 billion for “virtual nursing assistants,” and $18 billion for “administrative workflow assistance.”

As the presentation of the study points out, these segments are generating this kind of capital for a reason; they’re bringing in revenue for adopters. That’s because they’re driving superior outcomes, advancing what clinicians are able to do in their fields.

Teamwork in Medicine

In some ways, the new use of machine learning in physician assistance programs echoes other types of progress that practice administrators are making in the medical world.

Today, when you visit a specialist, you're more likely to meet with a physician assistant than you would have been ten or twenty years ago (as seen in this resource from Barton Associates). These PAs are credentialed and qualified for specific kinds of clinical work, to assist the primary medical doctor.

Using this care model frees up valuable resources -- it enables the specialist office to see more patients, and to treat and consult with patients in more specific ways. For example, a skilled surgeon may spend more time in surgery. Meanwhile, the practice is typically able to provide a comparable level of care to patients – or in many cases, an elevated standard of care. Machine learning tools like those offered by Xyonix further enhance this process.

AI systems often serve as additional ‘team members’ of the practice structure -- this team member just happens to be extremely good at assimilating vast stores of knowledge and delivering insights extremely fast without ever tiring.

If human PAs are part of the doctor's team, so are the physician assistance software AI models that are doing more in the clinical world. The AI systems may be checking x-rays or scans to look for key indicators of a particular diagnosis. They may inspect skin for signs of cancer. They may listen for symptoms and signs of disease in audio streams. Whatever the AI systems are doing, they are contributing to the specific way a practice has set up its services to triage patient care -- to make sure that each particular patient gets exactly what he or she needs at a particular moment.

In addition, ML/AI tools like those we make at Xyonix are made to enhance human teamwork processes as well. Think of a surgeon who can get reviews from experts and others beyond the hospital walls, or a specialist who can converse with other specialists to figure out a tough diagnosis.

If you watch a doctor making the rounds and observe their interactions with patients and the patient's extended care group, you see that a physician operates in a team. Physicians consult one another and other care staff in myriad ways -- these team interactions contribute heavily to a physician's clinical decisions. Typically, however, a doctor is limited to the team that's physically in the hospital at the time. Physicians can refer to notes from other teams of doctors, but they often can't conveniently converse or discuss things with absent doctors. In traditional medicine, communications have been delayed for the purposes of consulting teams – and the speed of clinical decision can move quite slowly.

Many physician assistant tools help crowdsource input in sophisticated ways – they not only provide broad medical opinions based on large data analysis and statistics, but they often incorporate feedback a doctor gets while examining a patient, medical record or during the course of a clinical interaction. This crowdsourcing increasingly provides instructional training examples that power AI systems.

EMRs/EHRs and Beyond – Not Just a Template

Not too many years ago, the healthcare world was abuzz over electronic health record and electronic medical record technologies. There was a lot of excitement about how these digital platforms could help improve clinical care and treatment.

These technologies essentially provide digital interfaces for documenting patient information. In some small ways, they started to assist doctors, but often not on an insight-driven basis. Some of the smartest features of electronic health records were templates that would help doctors to input a common diagnosis -- or auto chart fillers that could help doctors choose the language and dictation content that they needed to fill out a patient chart.

The key is that none of this was driven by anything particularly intelligent. The templates and automation tools were all geared toward rote data entry. They did help doctors to streamline care documentation -- but that's mostly where their utility ended.

Nonetheless, through the HITECH act and related initiatives, the government promoted the use of these new digital tools as one of the first steps toward fully modern and futuristic medicine.

AI powered physician assistance software is transcending EHR tools -- artificial intelligence increasingly helps doctors better understand an individual patient's condition and treatment options.

In addition to improving individual patient care, machines also help physicians effectively treat a broader community of patients. There are different ways to affect AI driven progress, and some rest on particular approaches that match a given task. One common approach involves the technology of natural language processing.

You might call this the “physician talk” model – but it applies to both voice and data, although mining natural language for information can be easier with text than it is with voice. In voice, there’s the extra step of transcribing the audio to determine meaning and intent -- recent deep learning models trained on increasingly large volumes of data have made remarkable progress in accuracy.

NLP models, or parsers, can learn to understand what physicians are saying in dictation, as well as when they are writing into a chart. Machines are increasingly able to listen – and make use of what they hear. This passive data aggregation is a very important part of what’s behind some of these technologies – for instance, physician assistance tools can report insights to doctors, based on what they've said in the past. That might sound like a simple task, but it’s actually a powerful way to authenticate clinical work. Doctors are only human, and work according to their perceptions in linear time, typically seeing many patients in a given day. These technologies, on the other hand, can present wide aggregated data that condense fields of study into a unified perceptive model. For example, physicians might state whether a patient is exhibiting particular symptoms many different ways. Normalizing these permutations into a single representation enables a higher level aggregation fundamental to gaining a deeper understanding of symptom rates across a wide population of patients and physicians. 

Another model could be described as a "records-based” model – think about electronic medical records and the types of information they contain. How do you mine that information effectively?

Machine learning programs can tag bits of natural language for classification – by building highly complex classifications, they can see, for example how prescription drugs are prescribed to patients, what doctors find in examinations and consultations, and other key bits of information that can be replicated across an enormous number of charts.

Any discussion of physician-assisted tools wouldn’t be complete without image analysis and computer vision – diagnostic radiology is  an enormous and growing field within the medical industry. Doctors are relying on different types of images and scans for all sorts of clinical work, and AI methodologies that can help are going to be vitally important to new healthcare workflow models.


When machines are applied to the scans and images, new technologies can be immensely effective in reading them in detail. A convolutional neural network, or CNN, can often provide excellent results that can, again, be extended across an infinite numbers of cases – this is the type of technology that's often in use when assisting physicians in assessing some visual items found in the scan – a cancerous lesion, or an outcome from invasive surgery, or anything else that can show up in CT scans, MRIs, x-rays or other types of diagnostic imagery.

Yet another model is a memory model that can be used to track clinical care. When nurses perform important interventions on patients, from tests to IVs and central lines, these actions could be recorded accurately in a comprehensive care narrative. Machine learning systems with memory, such as an LSTM setups of a recurrent neural network can be taught to “know” what has happened in a patient’s room, and deliver that on a timeline to clinicians and other stakeholders.

The Bedside Manner

Used correctly, new machine learning healthcare tools can provide a source of assurance for patients.

Patients need to trust their physicians, and many have an emotional connection with their doctor. Patient's also trust their doctors to harness modern technologies and evidenced driven care practices. People don't want to hear a diagnosis from a machine, but many of them might like to know their doctor has consulted an AI model trained by millions of top rate physicians proven to markedly outperform the average physician.

So when the medical provider can show that they have this kind of resource, it gives the patients and their families more peace of mind. It impresses on them that the medical business has the skill and ability to help their loved one through whatever condition he or she is facing.

Effective Use Cases

One important component of creating the best artificial intelligence PA applications is understanding when these tools can be the most helpful.

Think of it this way: when are doctors most like machines? When are physicians engaged in machine-like activities or behaviors? These are areas ripe for AI health innovation.

Clinical work is highly variable. Think of all of what a doctor might do in an average patient visit. Some of the core work is inherently “social” – doctors are explaining complex medical information to patients. That’s really not something that AI technologies should dominate -- at least, not yet, and perhaps, never.

On the other hand, when a doctor makes her way to the patient’s side and takes out a stethoscope – she gets quiet – and listens. At that particular moment, the work model switches from social to analytical. The doctor is then acting in a way that is “machine-like” – quantifying noise and substance in a signal pattern.

Those are the kinds of tasks to which artificial intelligence medical tools are well suited to assist. That’s a big part of what Xyonix is doing in the medical space – looking at these tasks, and automating them with a knowledge base and increasingly evolved AI.

Into the Future

Our new machine learning technologies are, by today's standards, pretty amazing -- but in many ways, they're really just the start.

There are all sorts of additional ways we can build on these ideas to give doctors new valuable insights -- we just haven't built them yet. We at Xyonix are contributing daily to this rapid progress -- this large leap into the future that will cause us to look back on the care of prior decades and marvel at what we've achieved and how far we’ve come.

These new care models will improve quality of life – they’ll increase longevity. They'll bring loved ones back to their families. They’ll do this, in general, by leveraging the power of distributed networks, the power of the medical community in general, and the resources that exist to fight disease, and bring them all of the way down to the individual point of care, the “front lines” where medical outcomes are created.

They’ll also help doctors to do more in a shorter period of time, which will help with the pressures and burdens put on top clinicians in the medical community. It's a win-win for the world, and we feel good about the work we’re doing to carry medicine into the future.

Need Help with Your AI Needs? CONTACT us -- we might be able to help.