Charmed Apache Spark documentation

Charmed Apache Spark solution is a set of Canonical supported artefacts (including charms, rocks and snaps) that makes operating Apache Spark workloads on Kubernetes seamless, secure and production-ready.

Charmed Apache Spark solution includes the:

• Charmed operators to deploy and manage Apache Spark-related components, such as Apache Kyuubi and Spark History Server, using Juju

• OCI Images that benefit from security patching and bug fixing support to run Apache Spark securely

• Client tools, either in the form of a snap or python package to simplify the user experience of running Apache Spark on K8s

For an overview of all components in the solution, see the Components overview page.

Apache Spark is a free, open-source software project by the Apache Software Foundation.

The solution helps ops teams and administrators automate Apache Spark operations from Day 0 to Day 2 with additional capabilities, such as: user management, encryption, password rotation, easy-to-use application integration, and monitoring.

In this documentation

Tutorial	Introduction • Step 1: Environment setup
Deployment	Environment setup • Charmed Apache Spark • Charmed Apache Kyuubi • Requirements
Service account management	Integration hub • Python • Spark-client
Operations	Monitoring • Spark History Server: Auth and web GUI • Use K8s pods • Streaming jobs • Use GPUs
Apache Kyuubi	External connections • Integrate • Metastore • Backups • Upgrades • GPU support
Security	Overview • Enable encryption (Apache Kyuubi) • Cryptography • Self-signed certificates

How the documentation is organized

Tutorial: For new users needing to learn how to use Charmed Apache Kafka
How-to guides: For users needing step-by-step instructions to achieve a practical goal
Reference: For precise, theoretical, factual information to be used while working with the charm
Explanation: For deeper understanding of key Charmed Apache Kafka concepts

Project and community

Charmed Apache Spark is a distribution of Apache Spark. It’s an open-source project that welcomes community contributions, suggestions, fixes and constructive feedback.

• Read our Code of Conduct

• Join the Discourse forum

• Contribute and report issues

• Explore Canonical Data Fabric solutions

• Contact us for all further questions

Apache®, Apache Spark, Spark®, and the Apache Spark logo are either registered trademarks or trademarks of the Apache Software Foundation in the United States and/or other countries.

License and trademarks

Apache®, Apache Spark, Spark™, Apache Kyuubi, Kyuubi™, and their respective logos are either registered trademarks or trademarks of the Apache Software Foundation in the United States and/or other countries.

The Charmed Apache Spark solution is free software, distributed under the Apache Software License, version 2.0. See LICENSE for more information.