Add instrumentation section and update GA criteria

Signed-off-by: Dixita <ndixita@google.com>

Author: ndixita

keps/prod-readiness/sig-node/2837.yaml

@@ -2,4 +2,6 @@ kep-number: 2837
 alpha:
   approver: "@jpbetz"
 beta: 
-  approver: "@soltysh"  
+  approver: "@soltysh"
+stable:
+  approver: "@jpbetz"

keps/sig-node/2837-pod-level-resource-spec/README.md

@@ -39,6 +39,13 @@
     - [[Scoped for Beta] HPA](#scoped-for-beta-hpa)
     - [Cluster Autoscaler](#cluster-autoscaler)
     - [VPA](#vpa)
+  - [Instrumentation](#instrumentation)
+    - [Admission & API Validation](#admission--api-validation)
+      - [<code>kubelet_pod_level_resources_admission_total</code>](#kubelet_pod_level_resources_admission_total)
+    - [The Kubelet (Execution Phase)](#the-kubelet-execution-phase)
+      - [<code>kubelet_pod_oom_kills_total</code>](#kubelet_pod_oom_kills_total)
+      - [<code>pod_cpu_cfs_throttled_seconds_total</code>](#pod_cpu_cfs_throttled_seconds_total)
+    - [Regression Monitoring (Existing Metrics)](#regression-monitoring-existing-metrics)
   - [Test Plan](#test-plan)
     - [Unit tests](#unit-tests)
     - [Integration tests](#integration-tests)
@@ -69,6 +76,7 @@
     - [[Future KEP Consideration in 1.35] Topology Manager](#future-kep-consideration-in-135-topology-manager)
     - [[Future KEP Consideration in collaboration with sig-autoscaling] VPA](#future-kep-consideration-in-collaboration-with-sig-autoscaling-vpa)
     - [[Scoped for GA] User Experience Survey](#scoped-for-ga-user-experience-survey)
+    - [[Scoped for GA] User Experience Survey](#scoped-for-ga-user-experience-survey-1)
 <!-- /toc -->
 
 
@@ -78,17 +86,17 @@
 Items marked with (R) are required *prior to targeting to a milestone / release*.
 
 - [ ] (R) Enhancement issue in release milestone, which links to KEP dir in [kubernetes/enhancements] (not the initial KEP PR)
-- [ ] (R) KEP approvers have approved the KEP status as `implementable`
-- [ ] (R) Design details are appropriately documented
+- [x] (R) KEP approvers have approved the KEP status as `implementable`
+- [x] (R) Design details are appropriately documented
 - [ ] (R) Test plan is in place, giving consideration to SIG Architecture and SIG Testing input (including test refactors)
   - [ ] e2e Tests for all Beta API Operations (endpoints)
   - [ ] (R) Ensure GA e2e tests meet requirements for [Conformance Tests](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/conformance-tests.md) 
   - [ ] (R) Minimum Two Week Window for GA e2e tests to prove flake free
-- [ ] (R) Graduation criteria is in place
+- [x] (R) Graduation criteria is in place
   - [ ] (R) [all GA Endpoints](https://github.com/kubernetes/community/pull/1806) must be hit by [Conformance Tests](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/conformance-tests.md) 
 - [ ] (R) Production readiness review completed
 - [ ] (R) Production readiness review approved
-- [ ] "Implementation History" section is up-to-date for milestone
+- [x] "Implementation History" section is up-to-date for milestone
 - [ ] User-facing documentation has been created in [kubernetes/website], for publication to [kubernetes.io]
 - [ ] Supporting documentation—e.g., additional design documents, links to mailing list discussions/SIG meetings, relevant PRs/issues, release notes
 
@@ -1272,6 +1280,45 @@ type RecommendedPodResources struct {
 Note: Detailed KEP design is owned and being worked on by 
 sig-autoscaling: [#7571](https://github.com/kubernetes/autoscaler/issues/7571)
 
+### Instrumentation
+
+This section outlines the final list of metrics for the Pod-Level Resources feature, excluding Resource Manager extensions. These metrics are designed to provide deep observability into admission control, scheduling efficiency, and Kubelet-level execution.
+
+#### Admission & API Validation
+These metrics track feature adoption, user intent, and validation friction within the control plane.
+
+##### `kubelet_pod_level_resources_admission_total`
+Total number of pods processed during Kubelet admission, categorized by resource configuration strategy.
+
+**Note:** This metric is **ALPHA** and temporary. It is intended to track feature adoption while the feature is new and is scheduled to be removed 2-3 releases after the Pod-Level Resources feature reaches General Availability (GA).
+
+Labels: 
+- `config_mode` - Possible values: `container_level`, `pod_level_only`, `pod_and_container_level`.
+- `status` - Possible values: `admitted`, `rejected`.
+- `qos_class` - Possible values: `guaranteed`, `burstable`, `best_effort`.
+
+This metric is recorded as a counter.
+
+#### The Kubelet (Execution Phase)
+Tracks operation failures during the container lifecycle that are specific to the shared pod-level resource pool.
+
+##### `kubelet_pod_oom_kills_total`
+Total number of OOM kills triggered specifically because the shared pod-level memory pool was exhausted. This metric is crucial for identifying cases where a container was killed even if it was under its own individual limit, but the pod's aggregate limit was reached.
+
+This metric is recorded as a counter.
+
+##### `pod_cpu_cfs_throttled_seconds_total`
+Total time in seconds that containers in a pod were throttled due to exceeding pod-level CPU limits. This metric helps identify pods that are consistently hitting their aggregate CPU limits.
+
+This metric is recorded as a counter.
+
+#### Regression Monitoring (Existing Metrics)
+While not new, the following metrics must be monitored to ensure no regressions in scheduling or node stability occur after adopting pod-level resource specifications.
+
+- `schedule_attempts_total{result="error|unschedulable"}`: Monitored to detect spikes in unschedulable pods due to potential bugs in the new resource requirement calculation.
+- `node_collector_evictions_total`: Monitored to ensure that the new pod eviction ranking logic based on pod-level requests behaves as expected and does not lead to an increase in unintended evictions.
+- `started_pods_errors_total` / `started_containers_errors_total`: Monitored to detect failures in the Kubelet's ability to create and configure the pod-level cgroups and container sandboxes.
+
 ### Test Plan
 
 [X] I/we understand the owners of the involved components may require updates to
@@ -1343,7 +1390,6 @@ feature gate and by setting the new `resources` fields in PodSpec at Pod level.
 
 #### GA (stable)
 
-* VPA integration of feature moved to beta.
 * No major bugs reported for 3 months.
 * Pod Level Resources Support With In Place Pod Vertical Scaling KEP is past alpha.
 * User feedback (ideally from at least two distinct users) is green
@@ -1718,7 +1764,9 @@ Pick one more of these and delete the rest.
 
 - [X] Metrics
   - Metric name:
-  - `apiserver_rejected_requests` will indicate any failures (`Bad Request` code=400) related to translation of new `resources` field in PodSpec. 
+  - `kubelet_pod_level_resources_admission_total` (**ALPHA, Temporary**): Total number of pods processed during Kubelet admission, categorized by resource configuration strategy. Scheduled for removal 2-3 releases after GA.
+  - `kubelet_pod_oom_kills_total`: Total number of OOM kills triggered specifically because the shared pod-level memory pool was exhausted.
+  - `pod_cpu_cfs_throttled_seconds_total`: Total time in seconds that containers in a pod were throttled due to exceeding pod-level CPU limits.
   - `schedule_attempts_total{result="error|unschedulable"}`
   - `node_collector_evictions_total`: to check if a pod level resource setting is causing to evict more pods than normal
   - `started_pods_errors_total`: exposed by kubelet to check if large number of pods are failing unusually
@@ -1927,6 +1975,7 @@ resource specs.
   (#4678)[https://github.com/kubernetes/enhancements/pull/4678]
 - **2025-06-18:** Revised KEP for Beta
 - **2026-01-27:** Revised KEP for 1.36 to include fixes for issues 135082 and 136120.
+- **2026-06-08:** Revised KEP for GA in 1.37.
 
 ## Drawbacks
 
@@ -2089,6 +2138,21 @@ recommendations have been proposed and require further discussion.
 
 #### [Scoped for GA] User Experience Survey
 
+Before promoting the feature to GA, we plan to conduct a UX survey to
+understand user expectations for setting various combinations of requests and
+limits at both the pod and container levels. This will help us gather use cases
+for different combinations, enabling us to enhance the feature's usability. If we
+identify the need for significant changes to the defaulting logic based on this 
+feedback, we'll release another Beta version of Pod-Level Resources to
+incorporate those adjustments.boration with sig-autoscaling] VPA 
+Pod-Level Resources allows pod-level limits to be greater than aggregated container
+limits to allow the containers to share idle resources among each other. 
+Integrating this functionality with VPA necessitates the development of a complex
+new recommendation algorithm. Concepts such as proportionate pod and container level
+recommendations have been proposed and require further discussion.
+
+#### [Scoped for GA] User Experience Survey
+
 Before promoting the feature to GA, we plan to conduct a UX survey to
 understand user expectations for setting various combinations of requests and
 limits at both the pod and container levels. This will help us gather use cases

keps/sig-node/2837-pod-level-resource-spec/kep.yaml

@@ -1,4 +1,4 @@
-title: KEP Template
+title: Pod Level Resource Specifications
 kep-number: 2837
 authors:
   - "@ndixita"
@@ -25,17 +25,18 @@ see-also: []
 replaces: []
 
 # The target maturity stage in the current dev cycle for this KEP.
-stage: beta
+stage: stable
 
 # The most recent milestone for which work toward delivery of this KEP has been
 # done. This can be the current (upcoming) milestone, if it is being actively
 # worked on.
-latest-milestone: "v1.36"
+latest-milestone: "v1.37"
 
 # The milestone at which this feature was, or is targeted to be, at each stage.
 milestone:
   alpha: "v1.33"
   beta: "v1.34"
+  stable: "v1.37"
 
 # The following PRR answers are required at alpha release
 # List the feature gate name and the components for which it must be enabled