‘ChatGPT, what is the Markov Process?’: See how a 19th-century trick helps AI ‘decide’
If we can ‘talk’ to AI programs today, it’s in part because of a Russian from the 1800s. Markov’s approach to data in flux changed how we navigate our world.
There’s an odd little trick to how AI programs, including large language models such as ChatGPT, learn and make decisions. The oddest thing about it is that it can be traced to a Russian born in 1856.
Andrey Andreyevich Markov was a child with serious health problems; he walked with the help of crutches until he was 10. Perhaps because of this, he spent more time with books than he otherwise would have, in his early years. He fell in love with math, so in love that he excelled at it and more or less ignored the other subjects in school, leading to him being tagged stubborn and “a rebel”.
His work in his chosen field would eventually alter how planners, statisticians and programmers view our world. As a professor of math and statistics at his alma mater, St Petersburg University, he combined number theory and probability theory to devise what is now known as the Markov Process.
It is an approach to information processing that factors in constant change. In an era long before the dawn of computing, his approach — which is relatively simple when broken down for each application — embraces the idea of constant flux, and uses the potential scenarios of change to the advantage of the system as a whole.
We’ll come to exactly how in a bit, but consider, first, the far-reaching implications of such a system. It has applications in traffic management, hospital administration, gaming and finance — in any field, in fact, that is variable by nature.
One field in which the Markov Process (and its components, Markov Chains) have become pivotal, in the 21st century, is the training of artificial intelligence (AI) programs.
Here’s how it works. A Markov Chain allows a live system — whether this be a set of traffic signals or a chatbot’s conversation — to use its current state to loosely predict what state will follow next and chart its next steps accordingly.
So, to use a simplistic example, a system of traffic lights can respond to the specific number of cars waiting at a signal by adjusting the duration of the green light to follow. Once the green light is turned on, it begins to use only its current data set — the number of cars passing through — to estimate its next step.
In the age of artificial intelligence, this can seem like an obvious thing to do, but it helps to remember that the key building blocks in the world of software remain formats such as the decision tree, which run on the “if-then” principle. The decision tree is used, for example, by the software programs in basic ATM machines. Here, only a limited number of predetermined options can be accommodated. Each cue, in terms of options a user is likely to choose, is laid out for the machine in advance; each step choreographed.
Where a decision-tree software program performs something akin to a well-practised ballet, the Markov-driven AI model is like a beautiful impromptu jazz performance, among strangers, at a street corner.
Rather than demanding that a software program essentially read its cue cards and perform the mandated next step, the Markov Process gives the program the tools, and thereby the capability, to recalculate and recalibrate as a process unfolds, and “decide”.
And so it is that ChatGPT, Bing and other AI chatbots, can conduct an interaction in the tone of the user, and provide data and creative output (stories, poems, explainers).
Try the ChatGPT prompt “Write me a poem about 2 am in the style of nursery rhyme” vs “Write me a poem about 2 am in the style of Robert Frost” and here’s a sampling of what one gets (we leave you to determine which is which).
Sample 1
The night is dark, the air is chill,
But all is calm and all is still.
The world outside is fast asleep,
As I quietly count some sheep.
Sample 2
The woods are dark and the stars are bright,
As I ponder the meaning of this solitary night.
The silence is broken only by my deep breaths,
And the rustling of leaves in the cool night air.
Setting aside the quality of the verse, which is debatable as all verse is, ChatGPT couldn’t provide two distinct styles, at random, quickly and seamlessly, without the mid-19th-century math lover Markov’s approach to input and response.
Millions of random and unpredictable data sets like the two poems have formed the basis of what this AI language model has learnt; the Markov process has formed the basis of how it uses that information.
How does the Markov Process compare with machine learning, another method widely used to “teach” AI programs? With machine learning, all data must be present, in order to be classified, examined, scrutinised. A machine learning program can only adjust within a limited and predetermined range of contexts, and cannot deviate from its set responses based on real-time cues.
The one thing that cannot be avoided, however, when one uses the Markov Process, is the margin of error. Each error leads to a new learning point, with each newly encountered scenario factored in for future use. But this makes human involvement not just recommended but essential.
In a study on artificial intelligence in medicine by researchers from US-based Indiana University and Centerstone Research Institute in 2013, the Markov Process was used as the basis for AI-driven decision-making in health care. The study deployed AI to think like a doctor amid variables such as costs and complexity, policies and payment methods, to develop a computational framework for non-disease-specific scenarios. It was also in use to monitor healthcare devices used by patients, and could predict an oncoming potential failure.
Such a program could greatly supplement human intervention, the study found.
Ask ChatGPT what the jobs of the future are likely to be, as AI takes over more functions, and it offers, unprompted, at #3: “Healthcare Professionals -- The healthcare industry is already making use of AI and machine learning... there will be a growing need for healthcare professionals who can work alongside them.” (The rest of its top 5 is made up of data scientists, software developers, cybersecurity experts and “as the world becomes more aware of the impact of human activity on the planet”, environmental scientists.)
Partly because of the Markov Process, AI is learning, growing, and “knows” that it is.
Over time, could the process help AI take over? Not if the quality of the “Robert Frost” verse is any indication.
There’s an odd little trick to how AI programs, including large language models such as ChatGPT, learn and make decisions. The oddest thing about it is that it can be traced to a Russian born in 1856.
Andrey Andreyevich Markov was a child with serious health problems; he walked with the help of crutches until he was 10. Perhaps because of this, he spent more time with books than he otherwise would have, in his early years. He fell in love with math, so in love that he excelled at it and more or less ignored the other subjects in school, leading to him being tagged stubborn and “a rebel”.
His work in his chosen field would eventually alter how planners, statisticians and programmers view our world. As a professor of math and statistics at his alma mater, St Petersburg University, he combined number theory and probability theory to devise what is now known as the Markov Process.
It is an approach to information processing that factors in constant change. In an era long before the dawn of computing, his approach — which is relatively simple when broken down for each application — embraces the idea of constant flux, and uses the potential scenarios of change to the advantage of the system as a whole.
We’ll come to exactly how in a bit, but consider, first, the far-reaching implications of such a system. It has applications in traffic management, hospital administration, gaming and finance — in any field, in fact, that is variable by nature.
One field in which the Markov Process (and its components, Markov Chains) have become pivotal, in the 21st century, is the training of artificial intelligence (AI) programs.
Here’s how it works. A Markov Chain allows a live system — whether this be a set of traffic signals or a chatbot’s conversation — to use its current state to loosely predict what state will follow next and chart its next steps accordingly.
So, to use a simplistic example, a system of traffic lights can respond to the specific number of cars waiting at a signal by adjusting the duration of the green light to follow. Once the green light is turned on, it begins to use only its current data set — the number of cars passing through — to estimate its next step.
In the age of artificial intelligence, this can seem like an obvious thing to do, but it helps to remember that the key building blocks in the world of software remain formats such as the decision tree, which run on the “if-then” principle. The decision tree is used, for example, by the software programs in basic ATM machines. Here, only a limited number of predetermined options can be accommodated. Each cue, in terms of options a user is likely to choose, is laid out for the machine in advance; each step choreographed.
Where a decision-tree software program performs something akin to a well-practised ballet, the Markov-driven AI model is like a beautiful impromptu jazz performance, among strangers, at a street corner.
Rather than demanding that a software program essentially read its cue cards and perform the mandated next step, the Markov Process gives the program the tools, and thereby the capability, to recalculate and recalibrate as a process unfolds, and “decide”.
And so it is that ChatGPT, Bing and other AI chatbots, can conduct an interaction in the tone of the user, and provide data and creative output (stories, poems, explainers).
Try the ChatGPT prompt “Write me a poem about 2 am in the style of nursery rhyme” vs “Write me a poem about 2 am in the style of Robert Frost” and here’s a sampling of what one gets (we leave you to determine which is which).
Sample 1
The night is dark, the air is chill,
But all is calm and all is still.
The world outside is fast asleep,
As I quietly count some sheep.
Sample 2
The woods are dark and the stars are bright,
As I ponder the meaning of this solitary night.
The silence is broken only by my deep breaths,
And the rustling of leaves in the cool night air.
Setting aside the quality of the verse, which is debatable as all verse is, ChatGPT couldn’t provide two distinct styles, at random, quickly and seamlessly, without the mid-19th-century math lover Markov’s approach to input and response.
Millions of random and unpredictable data sets like the two poems have formed the basis of what this AI language model has learnt; the Markov process has formed the basis of how it uses that information.
How does the Markov Process compare with machine learning, another method widely used to “teach” AI programs? With machine learning, all data must be present, in order to be classified, examined, scrutinised. A machine learning program can only adjust within a limited and predetermined range of contexts, and cannot deviate from its set responses based on real-time cues.
The one thing that cannot be avoided, however, when one uses the Markov Process, is the margin of error. Each error leads to a new learning point, with each newly encountered scenario factored in for future use. But this makes human involvement not just recommended but essential.
In a study on artificial intelligence in medicine by researchers from US-based Indiana University and Centerstone Research Institute in 2013, the Markov Process was used as the basis for AI-driven decision-making in health care. The study deployed AI to think like a doctor amid variables such as costs and complexity, policies and payment methods, to develop a computational framework for non-disease-specific scenarios. It was also in use to monitor healthcare devices used by patients, and could predict an oncoming potential failure.
Such a program could greatly supplement human intervention, the study found.
Ask ChatGPT what the jobs of the future are likely to be, as AI takes over more functions, and it offers, unprompted, at #3: “Healthcare Professionals -- The healthcare industry is already making use of AI and machine learning... there will be a growing need for healthcare professionals who can work alongside them.” (The rest of its top 5 is made up of data scientists, software developers, cybersecurity experts and “as the world becomes more aware of the impact of human activity on the planet”, environmental scientists.)
Partly because of the Markov Process, AI is learning, growing, and “knows” that it is.
Over time, could the process help AI take over? Not if the quality of the “Robert Frost” verse is any indication.
