Adobe details open-source approach for secure Kubernetes metrics isolation

Adobe engineers have detailed an open-source strategy to enable secure, self-service access to Prometheus metrics within multi-tenant Kubernetes environments. The architecture is specifically designed for GPU-heavy clusters, allowing teams to monitor power usage and resource utilization without compromising the privacy of other groups.

In large-scale deployments, a central Prometheus instance often contains metrics from thousands of namespaces. Providing unrestricted access to all tenants poses security risks, as users could potentially access rival data, and high query volumes can lead to performance bottlenecks or ‘noisy-neighbor’ issues.

Adobe’s solution utilizes a tenant-aware proxy positioned between users and the central Prometheus instance. The system leverages existing Kubernetes and CNCF technologies, employing NGINX and kube-rbac-proxy for authentication and authorization. A critical component, the ‘prom-label-proxy’, modifies incoming PromQL queries to enforce namespace constraints, ensuring that users cannot bypass security boundaries by manually constructing queries to access other namespaces. Additionally, the platform introduces a Kubernetes custom resource named ‘MetricAccess’, which allows teams to declare specific metric requirements using exact names, regular expressions, or selectors.