Orient Overseas Container Line (OOCL) was facing a tsunami of data.
The Hong Kong container shipping and logistics service company, founded in 1969, had successfully implemented a data-driven strategy. Operating 59 vessels of different classes, it was an early adopter of technology. While the investments improved efficiency, they also created a boatload of data.
So, when OOCL wanted to enrich its data sets with big data sources, it needed a smarter way to tame the data-laden seas it was operating in. It decided to migrate the data to the cloud.
“Data integration and replicating data across cloud and on-premises can cause data integrity, security, cost issues,” said Carol Choi, assistant general manager for OOCL.
The company found its answers with data virtualization.
“Data virtualization solutions can address this to keep a single source of truth and minimize integration efforts. It can act as an abstract layer to apply governance policy with complete audit capability to fulfill regulation,” added Choi.
Data virtualization ahoy!
One familiar issue that OOCL faced was the proliferation of data silos. The complexity made data governance challenging.
“Besides, we could not apply the same set of governance policies (i.e., masking sensitive data) across data sources because each technology had its own way to do governance,” said Choi.
The company saw data virtualization as a means to “breakthrough these silos to abstract data to apply governance, audit, metadata management at a centralized location,” she added.
The IT team set up a TIBCO Data Virtualization cluster at OOCL for high availability and load balancing. These scalable clusters featured cache servers that stored frequently queried data and improved performance. Better data management control and auditing were other intangible benefits.
“The biggest challenge was to convince OOCL that TIBCO Data Virtualization could solve their pain points out of the box. And we achieved it by a two-month detailed on-site POC,” said Nick Lim, general manager for APJ at TIBCO Software.
The data virtualization setup resolved challenges in data replication and movement. Data science teams got near real-time data for analysis and allowed business leaders to make quick, accurate decisions using machine learning. Overall, it made OOCL more responsive to dynamic market shifts.
“We can now provide more real-time analytics and provide data for data scientist analysis with less data leakage risk. Self-service data requests can reduce lead time while enforcing predefined data governance policies,” said Choi.
Choosing the partner
OOCL was careful when selecting TIBCO Data Virtualization.
“We conducted more than 100 functional and non-functional test cases on the TIBCO Data Virtualization solution to confirm that it was able to fulfill our requirements while [offering] performance [that] is good and scalable,” said Choi.
The tight integration with other TIBCO products, such as Spotfire, and the solution providers’ familiarity with OOCL environments were equally essential.
“TIBCO is a strategic partner to OOCL. We have worked together for over 10 years. We understand the current business environments as well as their project priorities. These are very important for our success,” said Lim.
With TIBCO Data virtualization on board, OOCL does not need to move data between hybrid cloud environments.
“It is complex to handle those data movements, especially downstream users, need near real-time data from the data warehouse. Schedule data clone/refresh cannot fulfill this case as the data volume is big, and data integrity issues happen across sources,” said Choi.
“In the big data world, customers are facing challenges in volume, variety, and velocity. And TIBCO Data Virtualization is uniquely positioned to address these challenges,” added Lim.
More importantly, data virtualization kept the data at the source. It improved the accuracy of data queries as it accessed the most current data in near real-time.
Collaboration is crucial
While TIBCO Data Virtualization helped to solve a lot of data-related pains, it is not a cure-all.
Choi was clear that having an abstract data layer for virtualization is “not a magic fix for everything.” “TIBCO Data Virtualization is able to query everything, but most of the queries will execute at the data source.”
This makes performance tuning vital, and not just for the TIBCO Data Virtualization layer. Equally important is a tight collaboration with the data source owner to ensure that the virtualization is usable.
“You really need to spend efforts to align expectations with the user and let them understand how data virtualization works,” said Choi.
For OOCL, the data virtualization journey has only just begun. The next port of call will be fully automating data requests and approvals, identifying the sensitive data from the requests, and applying the right data masking and filtering at the individual request level.
“This will minimize the manual efforts and provide a more standardized approach for downstream users to request data from different data sources,” said Choi.
It also allows OOCL to go full speed ahead on other data-driven initiatives, such as IoT, blockchain, and AI.
Winston Thomas is the editor-in-chief of CDOTrends and HR&DigitalTrends. He is always curious about all things digital, including new digital business models, the widening impact of AI/ML, unproven singularity theories, proven data science success stories, lurking cybersecurity dangers, and reimagining the digital experience. You can reach him at [email protected].
Image credit: iStockphoto/AvigatorPhotographer