Use of private X.509 machine identities to secure internal “east-west” traffic between workloads running in Kubernetes clusters is on the rise. At the same time, a common constraint for security teams with a policy to enforce protection of private key material, is to avoid using Kubernetes Secrets and instead centrally store and protect private key material for signing X.509 certificates using dedicated secrets management solutions, typically from HashiCorp Vault or Venafi Trust Protection Platform (TPP). In this scenario, enterprise security policy insists that root Certificate Authorities (CAs) or intermediate CAs must issue and sign mutual TLS (mTLS) certificates for Kubernetes workloads, with the private keys held securely outside the cluster in a “hardened” environment. In highly regulated sectors, such as banking and defence, these security scenarios are especially important.
The challenges for security policy using Kubernetes Secrets
- Kubernetes Secrets provides an acceptable level of security for most applications, but can be prone to leakage through human error, or maliciously intercepted via the Kubernetes API server or the etcd database directly.
- Using a hardened environment requires that highly sensitive private keys never be stored in Kubernetes Secrets and thus not be accessible by the API.
Enterprise security that builds on cert-manager
The good news is that cert-manager is already well-established in production clusters and it’s been built to serve use cases that need public trusted or private X.509 machine identities. Security teams can use this favoured solution to enforce stricter methods for signing private certificates with HashiCorp Vault, Venafi TPP, as well as others, acting as or on behalf of the Certificate Authority.
Platform teams will continue to drive scale and automation from Kubernetes using cert-manager to secure all internal workloads across production clusters. Security teams recognise this and have signalled their desire for Jetstack Secure to enable cert-manager to integrate with their preferred private key storage systems.
In response to this need from enterprise security teams, Jetstack Secure incorporates an add-on component for cert-manager that is designed to be run in a hardened environment that is external to the Kubernetes cluster. This issuer is isolated from workloads, and automatically requests an intermediate Certificate Authority from your chosen service; this could be either a Venafi TPP instance, HashiCorp Vault, from disk, or even another cert-manager. The component can then securely sign Kubernetes workloads off cluster, whilst preventing any way for its intermediate Certificate Authority or credentials to be leaked or stolen through the Kubernetes API. As part of Jetstack Secure, the issuer is designed to support high request throughput, and maintain high availability.
- Jetstack Secure provides a consistent and automated means of using a hardened security environment for out of cluster certificate signing.
- Developer teams now maintain the high levels of workload automation, whilst giving security teams control over Certificate Authority key material which is essential to sustaining an enterprise-wide strong security posture.
Jetstack Secure delivers a hardened security posture
Jetstack Secure’s issuer for cert-manager is designed to be used outside of the cluster and integrate with external Certificate Authority systems. Security teams can rely on this Jetstack Secure capability to sign certificates for many use cases, such as inter-pod mTLS using the csi-driver and inter-service mTLS with a service mesh. It will protect both private key and CA credentials from being exposed, even in total Kubernetes database breach scenarios.
Integrated to work with popular service mesh solutions
Many larger deployments of Kubernetes are now running service mesh solutions which provide workload security by encrypting inter-service traffic using mTLS. The need to deploy “mTLS everywhere” using service mesh solutions such as Istio or Linkerd have become popular because they have the capacity to secure and enforce strict mTLS, using machine identities that are automated in the cluster.
From a security standpoint, the highest threat prevention levels are achieved with a security policy that stops bad actors from maliciously requesting certificates for any valid identity in the mesh. Jetstack Secure enforces this strict policy by controlling access and preventing the credentials to be signed via the Kubernetes API. Using Jetstack Secure, the enterprise security posture is strengthened by:
- Integrating private enterprise PKI within the service mesh solution using signed builds of cert-manager that are FIPS compliant.
- Providing observability across all mesh workloads.
- Securely signing all mTLS certificates using private key material directly from the enterprise’s preferred hardened environment.
Automating Venafi TPP security policy for Kubernetes environments
Configuring Venafi TPP to store the private key material for each cluster can become a difficult task to manage for Kubernetes environments that are growing quickly and the number of clusters is increasing. Enterprise security policy will require each cluster to use the TPP solution for secrets management but even within a modern DevSecOps team environment, security engineers responsible for managing the enterprise PKI controls will not be directly involved in creating new production clusters.
Jetstack Secure provides a set of powerful configuration and control capabilities that can be instantly and consistently deployed to each new cluster:
- Jetstack Secure implements security “policy-as-code” meaning platform teams can deploy the out-of-cluster issuance capability as standard for every cluster.
- Each new cluster is now consistent with and adherent to enterprise security policy with enforced controls.
- Jetstack Secure provides observability and monitoring and will proactively detect misconfiguration and configuration drift, helping to ensure rapid remediation of potential threats to the system.
Ready to bring the power of Jetstack Secure to your Kubernetes infrastructure?
Speed with security can only come with policy consistency. Platform teams deploy cert-manager as standard for new production clusters and service mesh environments, and now with Jetstack Secure, each new production cluster can be bootstrapped to integrate directly with private key store systems to sign mTLS certificates for internal workloads. The solution is built to run on cert-manager and comes directly from Jetstack - the team that created the cert-manager open source project. This removes the burden from developer teams to manage secrets directly, and allows the security team to easily automate and enforce policy for all clusters.
Sign-up to use Jetstack Secure and connect your first cluster for free, or reach out to us directly to discuss hardening your cluster environments using cert-manager to integrate directly with external Certificate Authorities such as Hashicorp Vault and Venafi TPP
Cover photo by John Moore on Unsplash