# Deploying Sourcegraph executors on Kubernetes (docker-in-docker)

<Callout type="warning">
	Docker-in-Docker Kubernetes executors are in beta. For production workloads where you
	want the strongest reliability and support guarantees, you can also deploy via
	[Terraform](/self-hosted/executors/deploy-executors-terraform) or the
	[Linux binary](/self-hosted/executors/deploy-executors-binary).
</Callout>

[Kubernetes manifests](https://github.com/sourcegraph/deploy-sourcegraph-k8s) are provided to deploy Sourcegraph Executors on a running Kubernetes cluster. If you are deploying Sourcegraph with helm, charts are available [here](https://github.com/sourcegraph/deploy-sourcegraph-helm).

## Deployment

Executors on Kubernetes require privileged access to a container runtime daemon in order to operate correctly. To ensure maximum capability across Kubernetes versions and container runtimes, a [Docker in Docker](https://www.docker.com/blog/docker-can-now-run-within-docker/) sidecar is deployed with each executor pod so that the executor does not have to access the host container runtime directly.

### Step-by-step Guide

Ensure you have the following tools installed:

-   [kubectl](https://kubernetes.io/docs/tasks/tools/#kubectl)
-   [Helm](https://helm.sh/) if you're installing Sourcegraph with `helm`.

#### Deployment via Helm

Please refer to the [Sourcegraph Helm docs](/self-hosted/deploy/kubernetes#quickstart) for the latest instructions.

To specifically deploy Executors,

1. Create an overrides file, `override.yaml`, with any other customizations you may require.

    1. See [details on configurations](/self-hosted/deploy/kubernetes#configuration)
    2. See [here](/self-hosted/executors/executors-config) for a full list of executor environment variables

2. Run the following command:
    ```bash
    helm upgrade --install --values ./override.yaml --version <your Sourcegraph Version> sg-executor sourcegraph/sourcegraph-executor-dind
    ```
3. Confirm executors are working by checking the _Executors_ page under **Site admin > Executors > Instances** .

## Security considerations

Docker-in-docker executors require the Docker sidecar to run as a [privileged](https://kubernetes.io/docs/tasks/configure-pod-container/security-context/) container. `privileged: true` understandably raises concerns, so it's worth understanding the threat model before evaluating it — and seeing why, with the right deployment practices, it is acceptable for the large majority of self-hosted Sourcegraph instances.

### Executors run untrusted code by design

Every executor deployment method exists to run **arbitrary, untrusted code**. Auto-indexing invokes language indexers and package-manager hooks to resolve dependencies, and batch changes run user-defined tooling against the contents of a repository. See [Executors](/admin/executors/) and [Firecracker](/self-hosted/executors/firecracker) for the full sandboxing model.

This means the risk of a malicious job attempting to break out of its sandbox, consume excessive compute, or exfiltrate code and credentialsis is inherent to running executors. It is not unique to Docker-in-Docker:

-   **Firecracker** provides the strongest per-job isolation (a MicroVM boundary), but it is not a complete security control on its own. Even with Firecracker, Sourcegraph additionally configures `iptables` to stop jobs from reaching [private IP ranges](/self-hosted/executors/firecracker#known-caveats) and the cloud metadata service.
-   **Native Kubernetes** and **Docker-in-Docker** run jobs as containers without a MicroVM boundary, so they rely more heavily on the surrounding isolation and network controls.

The runtime you choose changes the isolation boundary, but they do not negate the need for additional controls to furthur limit the blast radius.

### Security hardening and risk management

A privileged container shares the host kernel and can access host devices, so in principle a successful container breakout could affect the node. The mitigations below contain that blast radius to a pool of disposable nodes that run *only* untrusted executor jobs and that cannot reach your internal network — which is the same posture Sourcegraph applies to Firecracker-isolated executors.

We strongly recommend applying the following controls when running docker-in-docker executors:

1. **Run executors on a dedicated, isolated node pool.** Schedule executor pods onto their own node pool using taints/tolerations and node selectors or affinity, kept separate from the rest of your Sourcegraph workloads and any other sensitive services. A breakout is then contained to nodes that run only untrusted executor jobs, and those nodes can be recycled aggressively.

2. **Sandbox the executor nodes where supported.** Run the executor node pool on a sandboxed runtime such as [gVisor](https://gvisor.dev/) (for example, [GKE Sandbox](https://cloud.google.com/kubernetes-engine/docs/concepts/sandbox-pods)). We offer built-in support for `gVisor` in the docker-in-docker helm chart. Learn more from the documentation.

3. **Restrict network access.** Apply Kubernetes [NetworkPolicies](https://kubernetes.io/docs/concepts/services-networking/network-policies/) and/or cloud firewall rules to the executor node pool to:
    - Block access to internal and private resources and services.
    - Block access to the cloud instance metadata service at `169.254.169.254` (AWS / GCP).