We are thrilled to announce that Datalayer, a SaaS platform for data analysis, has officially been awarded ISO 27001 certification, a significant milestone in our commitment to ensuring the highest levels of information security and data protection for our customers.

In the realm of AI, data science, and machine learning, Jupyter Notebooks are highly valued for their interactive capabilities, enabling users to develop with immediate feedback and iterative experimentation.

However, as models grow in complexity and datasets expand, the need for powerful computational resources becomes critical. Traditional setups often require significant adjustments or sacrifices, such as migrating code to different platforms or dealing with cumbersome configurations to access GPUs. Additionally, often only a small portion of the code requires GPU acceleration, while the rest can run efficiently on local resources.

What if you could selectively run resource-intensive cells on powerful remote GPUs while keeping the rest of your workflow local? That's exactly what Datalayer Cell Kernels feature enables. Datalayer works as an extension of the Jupyter ecosystem. With this innovative approach, you can optimize your cost without disrupting your established processes.

We're excited to show you how it works.

As a startup, we have been using the OVHcloud services those last 10+ years for DNS domain name registration. After all, this is what OVH started with and become famous (BTW this blog is hosted under datalayer.blog which we bought from OVH, as most of the other domain names we are using). During that time, we have seen more and more services being launched at OVHcloud (note the name change where cloud is added), starting with the dedicated bare metal offering.

Because we were looking more to the virtual machine and Kubernetes world, we have been naturally tempted to use the Azure, GCP or AWS offerings at their very early stage. All those leading cloud providers offer credits for startups combined with technical support and go-to-market offering.

At some point, Kubernetes at OVH has been around for some time (see this annoucement back in 2019) and we heard from other companies that those new services were very flexible and potentially cheaper than others. So it became clear to us that we wanted to try them and we naturally applied to their Startup program. Guess what, we received a positive response 2 days after 🎉 The journey was ready to start!

We are thrilled to announce the 0.0.6 release of Datalayer. This release improves the data analytics user and developer experience with Jupyter React, a javascript library to ensure React.js is a first-class citizen in the Jupyter ecosystem.

Jupyter React is built on top of JupyterLab which aims to be the next default notebook for Python data scientists and is actively developed. However, some users sill prefer the classic notebook and JupyterLab is not yet mainstream... The following points can be the identified as the source of the shadow:

1. The user interface is intimidating and quite complicated. An initiative to strip-down the user interface has been taken with Retrolab, but the result still looks pretty much like JupyterLab without visible value compared to the classic notebook. Users will even loose some beloved features like their preferred keyboard shortcuts, VIM mode, performance...
2. The extensions ecosystem is rich but breaking changes in the core of JupyterLab have made the overall ecosystem fragile and subject to failures on installation.
3. The overall performance (startup time, load large notebook, switch tabs...) is know to be degraded on JupyterLab.
4. The recently merged realtime collaboration feature is solely not usable with a real multi-user authentication and authorization system.
5. As developer, the Lumino widget toolkit which backs JupyterLab user interface is hard to use and looks pretty much like a Qt toolkit rather than like a modern javascript e.g. React.js, Vue.js, Svelte...

All Data Scientists know that story... Install the well-known Jupyter Classic or JupyterLab Notebook on their local PC/laptop, pip install some python libraries like pandas..., download some datasets and finally start analysing with a notebook in isolation. There are a few pain points there:

1. Setting up the tools is hard and time consuming. You have to install Python, Jupyter and add the libraries you need. Conda environments or Docker containers can help mitigate the pain at some point, but finally these are yet additional tools to setup and manage.
2. At some point, they want to collaborate with teammates, or want to share some results. The Data Scientist is just on his island and has no easy way to break the silo. The recent Realtime collaboration features have been merged into JupyterLab but it is just the permises and miss fundamental building blocks like identity, authorization...
3. The analysis is not easily reproducible. The setup you have done on a particular Windows platform is completely different from the setup another collaborator may have done on macOS.

More Cloud-native​

There comes the need for an better solution. At Datalayer we think that a more Cloud-native Jupyter can help remove those pain points. In other words, we embrasse the infrastructure provided by cloud providers like GCloud, AWS, Azure... and build on top to provide more power to the Data Scientist.

Cloud native computing is an approach in software development that utilizes cloud computing to "build and run scalable applications in modern, dynamic environments such as public, private, and hybrid clouds.

Crossplane is an open source Kubernetes add-on that enables platform teams to assemble infrastructure from multiple vendors, and expose higher level self-service APIs for application teams to consume, without having to write any code. It allows you to compose cloud infrastructure and services based on XRD (cross resource definitions) that extends the existing Kubernete CRD (Custom Resource Definition). To achieve this awesome goal, you have to use various repositories that reside in the GitHub crossplane, crossplane-contrib and upbound organisations. As adaptor of that new technology, you can rely the official documentation where a lot of details are gathered.

To ease our understanding and document our experiments, we have created a crossplane-example repository that will take you step-by-step to use Crossplane to deploy your infrastructure on top of Google Cloud and also develop a user interface and Helm chart that access a database created by Crossplane.

Crossplane community is welcoming, just like the Crossplane logo is fun!

Since our last blog post on January 2018, we have changed a lot the Datalayer architecture. Back in 2018, we had chosen for Apache Zeppelin for its good integration with Big Data frameworks like Apache Spark and competely replaced the existing Angular.js user interface with a home-brewed React.js implementation to integrate with the Kubernetes Control Plane. While rolling out more and more features on top of our former version 0.0.1, we have been intrigued in February 2018 by JupyterLab being announced to be ready for users. Back in time, in July 2016, JupyterLab was positioned as the next generation of the Jupyter Notebook.