Real-time AI Is No Longer a Pipe Dream
- By Winston Thomas
- February 22, 2023
For AI to be useful, it needs to do two things really well: it needs to give you the right answer, and the answer needs to come at the time of asking.
Today, AI has gotten very good at the first aspect. Generative AI, like ChatGPT and Bard, is taking this further by adding context. While the answers can sometimes be questionable, with enough reinforcement learning that we are giving it voluntarily, it will eventually get there.
Real-time AI is a different animal. It requires data scientists to stream data for real-time AI outcomes. With IoT devices coming online and every vehicle and physical machine transformed into a data node, it makes sense.
However, real-time AI is hard. You need to create or engineer features fast and do it as and when you need them.
One company is thinking differently. DataStax, which used to be a data platform player, is reinventing itself as a real-time AI player. And its acquisition of Kaskada, an ML company that has a niche play in time-based data to train ML models, shows that it’s serious.
The reality of real-time
Layman's terms, with broad brush strokes of intent and purpose, do poorly with science. Science demands accurate definitions.
The term “real-time” is one such term whose definition depends on who you ask. So what real-time means to a CEO or a Board is very different from that of a line of business leader whose needs to make operations decisions the next second.
“Is it a matter of speed? Is it a matter of source? I believe you need to look at the outcome. Because ultimately, what defines the time that matters is the person behind the application or the type of transaction you need to do,” says Thomas Been, chief marketing officer at DataStax.
Been noted that it comes down to the opportunity window. For an AI model to catch a fraudulent transaction, it has only a millisecond. But making a play for a new market opportunity can be done within days.
Discovering new real-time opportunities
If you characterize real-time as the opportunity window, it is easy to see why the real business world is hurtling toward business models where decisions are made in milliseconds.
Take Priceline, for example. The travel business giant, which competes with Expedia and Travelocity. Pricing is everything, and here the company is working with DataStax to do real-time analytics in the cloud and marrying it with Starburst’s real-time analytics platform for a data mesh along with Google’s BigQuery AI capabilities.
Such a setup allows Priceline to personalize services and target potential customers with the right products. It opens your world to new opportunities.
“When you start looking at things with this real-time perspective, you also start understanding the impact that real-time data can have,” says Been.
The State of the Data Race 2022 study by DataStax validated Been’s claim. 71% of respondents said they can tie their revenue growth directly to real-time data; 78% agreed that real-time data is a “must have” and not a “nice to have.”
“So I would encourage any organization to not look at where the real-time data is. It's really about the outcome and the impact you're going to make on your service and your applications or the way you operate. And I think then it will drive a very different perspective and a bigger view on the business impact,” Been says.
Real-time meets the data engineer
It’s simple to say we want real-time data. But bring the problem to the data engineer, who’s already under pressure to create new pipelines from a growing number of sources, and he, she or they may balk.
But Been argues for real-time AI to work, data engineers shift their mindsets.
“I think we're still inspired by decades of running analytics. And we still have this image in our mind taking data from one source and bringing it to machine learning,” Been points out.
“This might work fine for prototyping, but the model starts to get hungry when you hit production. Then it becomes a Herculean task to move these data around and give data scientists and models the granularity they want,” he continues.
Instead, Been argues that data engineers need to start with the premise that there are going to have to deal with huge data volumes from various sources at high speeds. He feels that we already have technologies like Cassandra, serverless architectures, and NoSQL databases.
“But they are still not been applied to the data pipeline; there's kind of a gap there that we need to close,” says Been. Companies like Netflix and Uber are examples of those who work with such scenarios daily.
“And when you look at how they operate, it is very interesting because it's very different. What they do is actually capture this real-time data and leverage streaming storage while keeping the data almost raw. And they can give access very easily to the application to the models at scale with very low latency to ensure this data becomes available,” Been describes.
“In a nutshell, what they're doing is bringing machine learning to the data, where the data is,” says Been.
This simplifies the pipelines but also streamlines the costs. “And we're here to help organizations adopt without investing the seven years and big budget that Netflix put into it,” says Been.
Walking the talk in real-time
One way DataStax is looking to help is by adding the core Kaskada technology to its cloud services. It bought Kaskada in January this year.
The new technology will join DataStax’s Astra DB (a DBaaS offering built on Apache Cassandra) and Astra Streaming. The aim is to help companies easily and cost-effectively with real-time AI applications.
The Kaskada technology will process copious amounts of event data as streams or stored in databases. Its unique time-based capabilities will create and update features for ML models based on sequences of events or over time. This means customers can adapt to rapidly changing content. It also asynchronously creates features, allowing applications to use millions of predictions based on unique contexts.
For Been, he believes that adding Kaskada builds on the original promise of helping data engineers “bring ML to data and not bring data to ML.”
“Now they have a very efficient way to drive features important for machine learning. They also now have a very efficient way to fully leverage the value of these operational workloads they have put in Cassandra or under management in Astra,” says Been.
“We are absolutely simplifying feature engineering. Most data scientists spend their time waiting for a feature. Now they can create features on the fly,” he adds.
Essentially, it means the processing time is reduced, which is what you need when you are working with real-time data. “I could draw a parallel between food and data; if it's being processed too much, it kind of loses its value,” Been says.
Winston Thomas is the editor-in-chief of CDOTrends and DigitalWorkforceTrends. He’s a singularity believer, a blockchain enthusiast, and believes we already live in a metaverse. You can reach him at [email protected].
Image credit: iStockphoto/mustafaU
Winston Thomas
Winston Thomas is the editor-in-chief of CDOTrends. He likes to piece together the weird and wondering tech puzzle for readers and identify groundbreaking business models led by tech while waiting for the singularity.