Add instrumentation section and update GA criteria

Signed-off-by: Dixita <ndixita@google.com>

Author: ndixita

keps/prod-readiness/sig-node/2837.yaml

@@ -2,4 +2,6 @@ kep-number: 2837
 alpha:
   approver: "@jpbetz"
 beta: 
+  approver: "@soltysh"
+stable:
   approver: "@soltysh"  

keps/sig-node/2837-pod-level-resource-spec/README.md

@@ -39,6 +39,17 @@
     - [[Scoped for Beta] HPA](#scoped-for-beta-hpa)
     - [Cluster Autoscaler](#cluster-autoscaler)
     - [VPA](#vpa)
+  - [Instrumentation](#instrumentation)
+    - [Admission & API Validation](#admission--api-validation)
+      - [<code>pod_level_resources_admission_total</code>](#pod_level_resources_admission_total)
+      - [<code>pod_level_resources_defaulting_total</code>](#pod_level_resources_defaulting_total)
+      - [<code>pod_level_resources_validation_errors_total</code>](#pod_level_resources_validation_errors_total)
+    - [The Kubelet (Execution Phase)](#the-kubelet-execution-phase)
+      - [<code>kubelet_pod_level_oom_kills_total</code>](#kubelet_pod_level_oom_kills_total)
+    - [Resource State (State Metrics)](#resource-state-state-metrics)
+      - [<code>kube_pod_level_resources_requests</code>](#kube_pod_level_resources_requests)
+      - [<code>kube_pod_level_resources_limits</code>](#kube_pod_level_resources_limits)
+    - [Regression Monitoring (Existing Metrics)](#regression-monitoring-existing-metrics)
   - [Test Plan](#test-plan)
     - [Unit tests](#unit-tests)
     - [Integration tests](#integration-tests)
@@ -69,6 +80,7 @@
     - [[Future KEP Consideration in 1.35] Topology Manager](#future-kep-consideration-in-135-topology-manager)
     - [[Future KEP Consideration in collaboration with sig-autoscaling] VPA](#future-kep-consideration-in-collaboration-with-sig-autoscaling-vpa)
     - [[Scoped for GA] User Experience Survey](#scoped-for-ga-user-experience-survey)
+    - [[Scoped for GA] User Experience Survey](#scoped-for-ga-user-experience-survey-1)
 <!-- /toc -->
 
 
@@ -78,17 +90,17 @@
 Items marked with (R) are required *prior to targeting to a milestone / release*.
 
 - [ ] (R) Enhancement issue in release milestone, which links to KEP dir in [kubernetes/enhancements] (not the initial KEP PR)
-- [ ] (R) KEP approvers have approved the KEP status as `implementable`
-- [ ] (R) Design details are appropriately documented
+- [x] (R) KEP approvers have approved the KEP status as `implementable`
+- [x] (R) Design details are appropriately documented
 - [ ] (R) Test plan is in place, giving consideration to SIG Architecture and SIG Testing input (including test refactors)
   - [ ] e2e Tests for all Beta API Operations (endpoints)
   - [ ] (R) Ensure GA e2e tests meet requirements for [Conformance Tests](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/conformance-tests.md) 
   - [ ] (R) Minimum Two Week Window for GA e2e tests to prove flake free
-- [ ] (R) Graduation criteria is in place
+- [x] (R) Graduation criteria is in place
   - [ ] (R) [all GA Endpoints](https://github.com/kubernetes/community/pull/1806) must be hit by [Conformance Tests](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/conformance-tests.md) 
 - [ ] (R) Production readiness review completed
 - [ ] (R) Production readiness review approved
-- [ ] "Implementation History" section is up-to-date for milestone
+- [x] "Implementation History" section is up-to-date for milestone
 - [ ] User-facing documentation has been created in [kubernetes/website], for publication to [kubernetes.io]
 - [ ] Supporting documentation—e.g., additional design documents, links to mailing list discussions/SIG meetings, relevant PRs/issues, release notes
 
@@ -1272,6 +1284,77 @@ type RecommendedPodResources struct {
 Note: Detailed KEP design is owned and being worked on by 
 sig-autoscaling: [#7571](https://github.com/kubernetes/autoscaler/issues/7571)
 
+### Instrumentation
+
+This section outlines the final list of metrics for the Pod-Level Resources feature, excluding Resource Manager extensions. These metrics are designed to provide deep observability into admission control, scheduling efficiency, and Kubelet-level execution.
+
+#### Admission & API Validation
+These metrics track feature adoption, user intent, and validation friction within the control plane.
+
+##### `pod_level_resources_admission_total`
+Total number of pods processed during Kubelet admission, categorized by resource configuration strategy.
+
+Labels: 
+- `config_mode` - Possible values: `container_level`, `pod_level_only`, `pod_and_container_level`.
+- `status` - Possible values: `admitted`, `rejected`.
+- `qos_class` - Possible values: `guaranteed`, `burstable`, `best_effort`.
+
+This metric is recorded as a counter.
+
+##### `pod_level_resources_defaulting_total`
+Total number of times pod-level resource fields were automatically populated by API server defaulting logic.
+
+Labels:
+- `resource` - Possible values: `cpu`, `memory`, `hugepages`.
+- `strategy` - Possible values: `request_from_container_requests`, `request_from_pod_limit`, `hugepage_limit_from_container_limits`.
+
+This metric is recorded as a counter.
+
+##### `pod_level_resources_validation_errors_total`
+Total number of pods rejected specifically due to validation errors in pod.spec.resources.
+
+Labels:
+- `reason` - Possible values: `sum_mismatch`, `negative_value`, `unsupported_resource`, `request_limit_mismatch`, `forbidden_field`.
+
+This metric is recorded as a counter.
+
+#### The Kubelet (Execution Phase)
+Tracks operation failures during the container lifecycle that are specific to the shared pod-level resource pool.
+
+##### `kubelet_pod_level_oom_kills_total`
+Total number of OOM kills triggered specifically because the shared pod-level memory pool was exhausted. This metric is crucial for identifying cases where a container was killed even if it was under its own individual limit, but the pod's aggregate limit was reached.
+
+This metric is recorded as a counter.
+
+#### Resource State (State Metrics)
+These metrics expose the current state of Pod-level resource requests and limits, primarily for consumption by kube-state-metrics and observability dashboards.
+
+##### `kube_pod_level_resources_requests`
+Exposes the value of `pod.spec.resources.requests` for each resource type.
+
+Labels:
+- `pod` - Name of the pod.
+- `namespace` - Namespace of the pod.
+- `uid` - Kubernetes UID of the pod.
+- `resource` - The resource type (e.g., `cpu`, `memory`).
+- `unit` - The unit of the resource (e.g., `core`, `bytes`).
+
+This metric is recorded as a gauge.
+
+##### `kube_pod_level_resources_limits`
+Exposes the value of `pod.spec.resources.limits` for each resource type.
+
+Labels: Same as `kube_pod_level_resources_requests`.
+
+This metric is recorded as a gauge.
+
+#### Regression Monitoring (Existing Metrics)
+While not new, the following metrics must be monitored to ensure no regressions in scheduling or node stability occur after adopting pod-level resource specifications.
+
+- `schedule_attempts_total{result="error|unschedulable"}`: Monitored to detect spikes in unschedulable pods due to potential bugs in the new resource requirement calculation.
+- `node_collector_evictions_total`: Monitored to ensure that the new pod eviction ranking logic based on pod-level requests behaves as expected and does not lead to an increase in unintended evictions.
+- `started_pods_errors_total` / `started_containers_errors_total`: Monitored to detect failures in the Kubelet's ability to create and configure the pod-level cgroups and container sandboxes.
+
 ### Test Plan
 
 [X] I/we understand the owners of the involved components may require updates to
@@ -1343,7 +1426,6 @@ feature gate and by setting the new `resources` fields in PodSpec at Pod level.
 
 #### GA (stable)
 
-* VPA integration of feature moved to beta.
 * No major bugs reported for 3 months.
 * Pod Level Resources Support With In Place Pod Vertical Scaling KEP is past alpha.
 * User feedback (ideally from at least two distinct users) is green
@@ -1718,6 +1800,12 @@ Pick one more of these and delete the rest.
 
 - [X] Metrics
   - Metric name:
+  - `pod_level_resources_admission_total`: Total number of pods processed during Kubelet admission, categorized by resource configuration strategy.
+  - `pod_level_resources_defaulting_total`: Total number of times pod-level resource fields were automatically populated by API server defaulting logic.
+  - `pod_level_resources_validation_errors_total`: Total number of pods rejected specifically due to validation errors in pod.spec.resources.
+  - `kubelet_pod_level_oom_kills_total`: Total number of OOM kills triggered specifically because the shared pod-level memory pool was exhausted.
+  - `kube_pod_level_resources_requests`: Exposes the value of `pod.spec.resources.requests` for each resource type.
+  - `kube_pod_level_resources_limits`: Exposes the value of `pod.spec.resources.limits` for each resource type.
   - `apiserver_rejected_requests` will indicate any failures (`Bad Request` code=400) related to translation of new `resources` field in PodSpec. 
   - `schedule_attempts_total{result="error|unschedulable"}`
   - `node_collector_evictions_total`: to check if a pod level resource setting is causing to evict more pods than normal
@@ -1927,6 +2015,7 @@ resource specs.
   (#4678)[https://github.com/kubernetes/enhancements/pull/4678]
 - **2025-06-18:** Revised KEP for Beta
 - **2026-01-27:** Revised KEP for 1.36 to include fixes for issues 135082 and 136120.
+- **2026-06-08:** Revised KEP for GA in 1.37.
 
 ## Drawbacks
 
@@ -2089,6 +2178,21 @@ recommendations have been proposed and require further discussion.
 
 #### [Scoped for GA] User Experience Survey
 
+Before promoting the feature to GA, we plan to conduct a UX survey to
+understand user expectations for setting various combinations of requests and
+limits at both the pod and container levels. This will help us gather use cases
+for different combinations, enabling us to enhance the feature's usability. If we
+identify the need for significant changes to the defaulting logic based on this 
+feedback, we'll release another Beta version of Pod-Level Resources to
+incorporate those adjustments.boration with sig-autoscaling] VPA 
+Pod-Level Resources allows pod-level limits to be greater than aggregated container
+limits to allow the containers to share idle resources among each other. 
+Integrating this functionality with VPA necessitates the development of a complex
+new recommendation algorithm. Concepts such as proportionate pod and container level
+recommendations have been proposed and require further discussion.
+
+#### [Scoped for GA] User Experience Survey
+
 Before promoting the feature to GA, we plan to conduct a UX survey to
 understand user expectations for setting various combinations of requests and
 limits at both the pod and container levels. This will help us gather use cases

keps/sig-node/2837-pod-level-resource-spec/kep.yaml

@@ -1,4 +1,4 @@
-title: KEP Template
+title: Pod Level Resource Specifications
 kep-number: 2837
 authors:
   - "@ndixita"
@@ -25,17 +25,18 @@ see-also: []
 replaces: []
 
 # The target maturity stage in the current dev cycle for this KEP.
-stage: beta
+stage: stable
 
 # The most recent milestone for which work toward delivery of this KEP has been
 # done. This can be the current (upcoming) milestone, if it is being actively
 # worked on.
-latest-milestone: "v1.36"
+latest-milestone: "v1.37"
 
 # The milestone at which this feature was, or is targeted to be, at each stage.
 milestone:
   alpha: "v1.33"
   beta: "v1.34"
+  stable: "v1.37"
 
 # The following PRR answers are required at alpha release
 # List the feature gate name and the components for which it must be enabled