Netflix Releases Polynote for Data Scientists

Netflix has announced the release of Polynote, an open-sourced multi-language programming notebook environment that integrates with Apache Spark and is designed with robust support for Scala, Python and SQL.

According to Netflix, Polynote provides data scientists and machine learning researchers with a notebook environment that allows them the freedom to seamlessly integrate its JVM-based ML platform with the Python ecosystem’s popular machine learning and visualization libraries.

Polyglot system

The polyglot system of Polynote allows each cell to be written in a different language, with variables shared between them. When code is executed, the resulting output values are passed back to the computational engine, which can make them available to other cells for further processing.

The tool offers an integrated development environment (IDE) that offers interactive autocomplete, parameter hints and error highlighting. Each cell in a notebook can be written in a different language with variables shared between them.

Developers working in a cluster computing environment will also enjoy the ability to save configuration and dependency setup within the notebook itself. Elsewhere, native data exploration and visualization are also built-in, with integration with matplotlib and Vega.

In the official announcement, Netflix notes that Polynote has already seen substantial adoption among the organization’s personalization and recommendation teams, and it is now being integrated with the rest of its research platform.

“On the Netflix Personalization Infrastructure team, our job is to accelerate machine learning innovation by building tools that can remove pain points and allow researchers to focus on research. Polynote originated from a frustration with the shortcomings of existing notebook tools, especially with respect to their support of Scala,” said the company.

“Plenty of exciting work lies ahead. We are very optimistic about the potential of Polynote and we hope to learn from the community just as much as we hope they will find value from Polynote.”