We’ve been running algorithms in our businesses for ages. We just do not depend on it entirely.
Covered as best practices, business models, or Excel formulae, algorithms show us how well our business is progressing. And with data availability, more companies are harnessing advanced analytics and AI to navigate global markets, pushing these algorithms to work harder for the business.
So, are we ready to have algorithms run the business? The short answer is no. Companies still want to use the experience to validate data-driven insights and still admire gut-driven business decisions.
But now, the future is looking very different. COVID-19 made established business models redundant. We became more reliant on data-driven algorithms to keep us in touch with our business, our customers, and even our competitors. As bots and AI become part of the new business normal, and as data science takes mega leaps forward, algorithms are about to shape the business landscape.
Why we’re ready
In a Gartner article, the research firm wrote that “algorithms define action.” They are now able to take on very complex tasks that only humans did before. Digital natives Amazon and Google, who use algorithms as their core business value propositions, showed that you can run some types of business as algorithms today.
While Adam Mayer, senior manager of technical product marketing at Qlik, feels algorithms are not ready to take over the corner office, he adds that algorithms are learning faster. AI algorithms are ingesting vast amounts of data that were previously locked in silo data stores. This means we’re running our companies on increasingly smarter business models.
Also, algorithms have already taken over mundane tasks. Robotic process automation is already sweeping aside manual tasks, while predictive maintenance and other data science advances are helping us to stay proactive. Many of today’s factories are becoming dark factories, where robots run the entire warehousing unit.
For Mayer, it boils down to these issues: having up-to-date data that is business- or analytics-ready, not having data siloed (which means the algorithms end up working only part of the problem), human-machine harmony, having the right culture to take advantage of algorithms, and complete data trust.
Why we’re not
For algorithms to run businesses, they need to prove that they can manage outliers. Traditional algorithms and models tend to normalize data sets. “They take the outliers out,” says Mayer.
The problem is that past performance is not a determinator of the future one — as COVID-19 showed us. And in some cases, an outlier may indicate an opportunity or a far more favorable result.
Mayer highlighted this point with the U.K. examinations. With COVID-19 spoiling in-person examination plans, the Office of Qualifications and Examinations Regulation turned to an algorithm to predict student examination scores backed by predictions made by teachers.
It fell into a familiar nightmare. An MIT Technology Review article showed that 40% of students got downgraded results because of human (i.e., teacher) bias. The algorithm also “disproportionally” hurt disadvantaged communities while inflating the scores for students hailing from well-funded private schools.
The article called this “algorithmic discrimination.” The Government reversed its decision, choosing to score students based on the highest value either from the teacher’s predictions or the algorithm.
The main problem is “because they locked out the outliers,” says Mayer. He describes the initiative penalized exceptional performing students coming from poorer performing schools. “Removing outliers really does cause quite big mistakes. So definitely we need to level up on our analytic approaches there and focus on those outliers.”
Dr. Koh Noi Sian, a data science practitioner in Singapore, adds that there are always specific tasks that can’t be coded into an algorithm, “such as those that require creativity and judgment which AI algorithms cannot do.”
Mayer adds that running an algorithm does not mean we need to take the human out of the decision-making. Also, there are certain things where humans need humans. Explaining the results of your health diagnosis is an apt example.
“If I have a critical illness, I don’t want to get an email. I want a human doctor to sit down and tell me what the results and the next steps are,” says Mayer.
Future is hopeful
Despite the challenges and issues, Mayer feels future algorithms may be ready to manage entire businesses in the future. But to achieve this level of business sophistication, data science needs to advance further.
One area that will be key is data lineage. “Which goes back to data trust,” says Mayer. “Data lineage plays a huge part in building this trust as you can then show the quality of data sources that the algorithm is using.”
Mayer points to advances in metadata management as a way forward. “You can put more metadata in so you can have a real sort of rich view of all the data that you’ve got and make that available to the right people in secure governed ways. And that really helped to build up trust in the data.”
Data modernization and democratization also play a role. This allows the algorithm to make “unified” queries across the organization for a more accurate picture of the business's health.
Data governance needs to become part of the corporate culture — not just a compliance department mandate. By making all employees “data governance aware,” data misuse or bad data can be eliminated. And data science teams can spend less time preparing the data and focus on finetuning the business model.
Also, questions about liability (what happens when an algorithm makes erroneous outcomes) and algorithm ownership are others that companies need to consider. Algorithm Councils or an empowered Center of Excellence can also provide the stewardship.
“Are the algorithms still relevant? Are they doing what we want them to do? Do they need to be tweaked? Do we need more data or different types? I think these would be perfect questions for the Algorithm Council,” says Mayer.
Eventually, algorithms will create other algorithms to manage different business areas while managing their relevance and accuracy. Companies need to prepare for such outcomes as they give more business autonomy to algorithms.
“The building blocks are definitely there, and we have lots of examples of successful algorithms. As humans, we’re just not ready yet,” adds Mayer.
Winston Thomas is the editor-in-chief of CDOTrends, HR&DigitalTrends and DataOpsTrends. He is always curious about all things digital, including new digital business models, the widening impact of AI/ML, unproven singularity theories, proven data science success stories, lurking cybersecurity dangers, and reimagining the digital experience. You can reach him at [email protected].
Image credit: iStockphoto/Alexandr Dubovitskiy