Skip to main content

Your submission was sent successfully! Close

Thank you for signing up for our newsletter!
In these regular emails you will find the latest updates from Canonical and upcoming events where you can meet our team.Close

Thank you for contacting us. A member of our team will be in touch shortly. Close

An error occurred while submitting your form. Please try again or file a bug report. Close

  1. Blog
  2. robgibbon
robgibbon

robgibbon

25 posts

Rob has 20+ years' industry experience building, scaling, managing and serving the teams, technology and environments behind around 50+ commercial web properties and data hubs across all major industries in varied roles. Rob brings both deep commercial and technical expertise to the product leadership team at Canonical. When he's not busy making great product, Rob is out running, reading and thinking, or just being.

robgibbon
15 October 2024

Apache Spark 4.0 beta release – try it now

Data Platform Article

Apache Spark is a popular framework for developing distributed, parallel data processing applications. Our solution for Apache Spark on Kubernetes has made significant progress in the past year since we launched, adding support for Apache Iceberg, a new GPU accelerated image using the NVIDIA Spark-RAPIDS plugin, and support for the Volcan ...

robgibbon
15 July 2024

Deploying and scaling Apache Spark on Amazon AWS EKS

Data Platform Article

Move over Hadoop, it’s time for Spark on Kubernetes Apache Spark, a framework for parallel distributed data processing, has become a popular choice for building streaming applications, data lake houses and big data extract-transform-load data processing (ETL). It is horizontally scalable, fault-tolerant, and performs well at high scale. H ...

robgibbon
23 May 2024

Can it play Doom? Running an AI LAN party on a Spark cluster with ViZDoom

AI Article

It’s all about AI these days, so I decided to try and answer the important question: can you make a Spark cluster run AI agents that play a game of Doom, in a multiplayer LAN party? Although I’m no data scientist, I was able to get this to work and I’ll show you how so ...

robgibbon
14 May 2024

Deploy an on-premise data hub with Canonical MAAS, Spark, Kubernetes and Ceph

AI Article

Download the Spark reference architecture guide In this post we’ll explore deploying a fully operational, on-premise data hub using Canonical’s data centre and cloud automation solutions MAAS (Metal as a Service) and Juju. MAAS is the industry standard open source solution for provisioning and managing physical servers in the data centre. ...

robgibbon
22 February 2024

Migrating from Cloudera to a modern data hub architecture

Data Platform Article

In the early 2010s, Apache Hadoop captured the imagination of the tech community. A free and powerful open source platform, it gave users a way to process unimaginably large quantities of data, and offered a dazzling variety of tooling to suit nearly every use case – MapReduce for odd jobs like processing of text, audio ...

robgibbon
12 December 2023

Announcing the Charmed Kafka beta

Data Platform Article

Charmed Kafka is a complete solution to manage the full lifecycle of Apache Kafka. The Canonical Data Fabric team is pleased to announce the first beta release of Charmed Kafka, our solution for Apache Kafka®. Apache Kafka® is a free, open source message broker for event processing at massive scale. Kafka is ideal for building ...

robgibbon
17 October 2023

Why we built a Spark solution for Kubernetes

Data Platform Article

We’re super excited to announce that we have shipped the first release of our solution for big data – Charmed Spark. Charmed Spark packages a supported distribution of Apache Spark and optimises it for deployment to Kubernetes, which is where most of the industry is moving these days. Reimagining how to work with big data ...

robgibbon
10 August 2023

Write a Spark big data job with ChatGPT

AI Article

I’ve read and watched more than a few articles about ChatGPT in the last couple of months. It seems the large language model AI hype machine just can’t stop.  As somebody with a passion for music production, some of the more interesting things I’ve seen included a guy using ChatGPT to build a virtual effect ...

robgibbon
3 July 2023

Charmed Spark beta release is out – try it today

AI Article

The Canonical Data Fabric team is pleased to announce the first beta release of Charmed Spark, our solution for Apache Spark. Apache Spark is a free, open source software framework for developing distributed, parallel processing jobs. It’s popular with data engineers and data scientists alike when building data pipelines for both batch an ...

robgibbon
3 May 2023

Big data security foundations in five steps

Data Platform Article

We’ve all read the headlines about spectacular data breaches and other security incidents, and the impact that they have had on the victim organisations. And in some ways there’s no place more vulnerable to attack than a big data environment like a data lake. ...

robgibbon
16 November 2022

Apache Kafka service design for low latency and no data loss

Apps Article

Designing a production service environment around Apache Kafka that delivers low latency and zero-data loss at scale is non-trivial. Indeed, it’s the holy grail of messaging systems. In this blog post, I’ll outline some of the fundamental service design considerations that you’ll need to take into account in order to get your service arch ...

  1. Previous page
  2. 1
  3. 2
  4. 3
  5. Next page