Minimum system requirements¶
The following are the minimum system requirements for Charmed Apache Spark:
8GB of storage.
2 CPU threads per host.
Access to the internet for downloading the Canonical Apache Spark image.
Access to a Kubernetes cluster, e.g. MicroK8s or Charmed Kubernetes.
Recommended versions¶
This section lists the recommended versions for deploying Charmed Apache Spark, plus additional versions validated during testing.
Kubernetes¶
We recommend deploying Charmed Apache Spark on a Canonical Kubernetes cluster running version 1.32 LTS.
The following table lists the validated combinations of substrate, architecture, and Kubernetes version:
K8s Distribution |
K8s versions |
Architecture |
GPU Acceleration |
|---|---|---|---|
Canonical K8s |
|
AMD, ARM |
Yes |
MicroK8s |
|
AMD |
No |
Azure AKS |
|
AMD |
No |
Amazon EKS |
|
AMD |
No |
Note
Others combinations of substrate, architecture, and Kubernetes versions may also work, although they are not continuously validated. To validate an architecture, you can deploy the terraform solutions as suggested in the How-to deploy guide, and then run the Charmed Spark UATs.
Juju¶
We recommend deploying Charmed Apache Spark on Juju 3.6.14.
Note
Juju controller versions from 3.9.10 to 3.6.13 are affected by a regression in Juju that has been fixed in 3.6.14.
Support for Juju 3.6.14+ has been addressed by this issue. When using Juju controller versions greater or equal to 3.6.14, make sure you use a charm revision above 107 for spark-integration-hub-k8s. On Juju controller versions lower or equal to 3.6.9, use charm revisions below 107.