Interpreting The Internet Using Machine Learning
- By Paul Mah
- September 07, 2022
Businesses are constantly striving to better understand their customers. The Internet is buzzing with public data that represents a wealth of information across customer demographics, but connecting the dots on this data to build an actionable strategy remains a challenge.
“There are three questions to address when building a strategy on the back of big data”, says Dr. Angad Singh Chowdhry, the co-founder and chief of product at Quilt.AI. “Who are my customers? What is the competitive ecosystem for my product? What cultural landscape do we find ourselves in today?”
The pressure to stay informed about customer behavior has increased in recent years, especially for fast-moving consumer goods (FMCG) and consumer packaged goods (CPG) companies. “Preferences are fluid, and cultural dynamics have an inordinate impact on these preferences,” says Chowdhry. Brands are scrambling to keep up.
Ask everyone
The traditional way for businesses to engage with customers is to invest in focus groups, which are time- consuming and limited, or social listening, which struggles with context and cultural frameworks.
“What if we could leverage publicly-available data while preserving the complexity of human emotions and expressions across cultures and age groups?” asks Chowdhry. “A customer talking about apples might be speaking literally of fruits, sub-culturally about a brand, or culturally about being a teacher's pet. The identity of the speaker also impacts the meaning of what is being said – Gen Z’s worldviews are radically different from millennials’, and this worldview impacts behavior, as well as how they self-express.”
To cope with the avalanche and richness of online data and conversations while maintaining a sense of nuance, Quilt.AI turned to machine learning (ML). “Machine learning allowed us to build culturally specific interpretations and models on all these datasets. It's allowed us to translate and interpret without losing too much of the information. It allows us to group and segment consumer data in meaningful ways,” continues Chowdhry.
Chowdhry notes that his ML team is interdisciplinary – experienced not only in technology but also in the humanities. Chowdhry, who holds a PhD in Anthropology, thinks of the Quilt.AI team as “fundamentally both an engineering team and an insights team. This interdisciplinary nature allows us to innovate in imaginative ways.”
“Building insight on the data that is available on the Internet is very, very difficult,” admitted Chowdhry. “The field of ML is rapidly evolving, and iterating on cutting edge models and deploying them for our own work has been a key part of how we have grown.”
Building and scaling the technology
Generating empathetic insights demands fluidity. “The exploratory nature of Quilt.AI’s work requires us to build pipelines on the fly by chaining data sources and analytics models”, says Yash Joshi, Vice-President of Product and Engineering at Quilt.AI. “Depending on the problem at hand, we may identify the values conveyed by a set of video advertisements, or segment consumers of a particular product into subcultures, or identify social influencers and map their networks and relevance for a product category”.
“Quilt.AI’s tools are powered by the Quilt Platform, which hosts these models and pipelines”, continues Joshi. “The team has adopted a microservices-oriented approach with each unit of functionality encapsulated in an independently deployable and scalable service. Our pipelines chain these microservices together in a flexible way.”
As Quilt.AI grew, it required partners who knew how to extract the maximum value from cloud infrastructure. As Joshi says, “Our technology is fundamentally cloud-native. We need to make sure that our offerings are well-engineered, scalable, easy to use, secure, and performant. And we do that by leveraging Google's Cloud Platform (GCP) with the help of Searce.”
“The biggest challenge for firms such as Quilt.AI has been making sense of the many platform vendors and technologies available,” says Raffi Ismail, Lead Cloud Consultant at cloud solutions and technology services provider Searce. “The challenge for clients like Quilt.AI has always been making the underlying platform work for them while focusing on their core expertise.”
“Searce takes the difficulty out of the equation. Our partners focus on developing their products using the cloud tools that Google has provided; we help our partners understand the tools and leverage them in the most efficient way possible. As advisors and consultants, we help our partners realize their roadmap, vision, and objectives to achieve the best outcomes,” summed up Ismail.
Steering brands towards empathy
Rather than treating customers as rows on a spreadsheet, Dr. Chowdhry believes in understanding them as contradictory and culturally layered beings. “Interpreting the internet, and using that to understand people authentically, is our mission. This authenticity is what creates empathy.”
Paul Mah is the editor of DSAITrends. A former system administrator, programmer, and IT lecturer, he enjoys writing both code and prose. You can reach him at [email protected].
Image credit: iStockphoto/Andrii Chagovets
Paul Mah
Paul Mah is the editor of DSAITrends, where he report on the latest developments in data science and AI. A former system administrator, programmer, and IT lecturer, he enjoys writing both code and prose.