Streaming Encoding for LIST Responses

Enhancement Description

One-line enhancement description (can be used as a release note): Streaming Response Encoding
Kubernetes Enhancement Proposal: https://github.com/kubernetes/enhancements/blob/master/keps/sig-api-machinery/5116-streaming-response-encoding/README.md
Previous discussion: Don't copy whole response during response marshalling kubernetes#129304, Streaming json list encoder kubernetes#129334
Primary contact (assignee): serathius@
Responsible SIGs: api-machinery
Enhancement target (which target equals to which milestone):
- Beta release target (x.y): 1.33
- Stable release target (x.y): 1.34

Milestones:

Beta

Stable
- KEP (k/enhancements) update PR(s): Move KEP-5116 to GA #5324
- Code (k/k) update PR(s):
  - Graduate streaming list encoding feature gates kubernetes#132648
- Docs (k/website) update(s): Document KEP-5116 website#51457
Optional improvements being worked on that are not blocking for graduation:
- [optional] Stabilize benchmark list by limiting QPS kubernetes#130869
- [optional] Implement checking apiserver memory limit below threshold in list benchmarks kubernetes#131885
- [optional] Implement missing LIST benchmarks kubernetes#130169

/sig api-machinery

@yulongfang

I'm glad to see this proposal. We have also implemented similar capabilities in our inner repo and are preparing to push this part to upstream. We have submitted a CFP for the upcoming KubeCon China conference.

In our implementation, we use sync.Pool to efficiently manage memory allocation and cache the serialized results of each item. When the buffer reaches a certain size, we execute a flush operation to parallelize the serialization processing and write to http2.

Additionally, we have added support for gzip compression, which is only enabled when the first batch of cached data reaches 128 * 1024.

For json serialization, we have customized the StreamMarshal method for unstructuredList.

As for protobuf, we generate code through a generator to ensure reverse protobuf marshalling compatibility.

type StreamMarshaller interface {
	// return the object size and the item size slice
	StreamSize() (uint64, []int)

	StreamMarshal(w stream.Writer, itemSize []int) error
}

And it has conducted extensive testing with large datasets and have obtained comparative results. @yulongfang Can you share some benchmark results?

@chenk008

Thank @chenk008 for your introduction. We have many large-scale clusters in Alibaba Cloud. When the controllers of these large-scale clusters are restarted, they will initiate a full list request to the apiserver, which will have a certain impact on the stability of the cluster. We have to use larger machines to run the apiserver, resulting in a waste of resources.

In this context, we adopted the method to carry out relevant optimization and achieved the following results.

list json format return data stress test scenario description:

apiserver version: 1.30
apiserver specification: 32c 128GB
apiserver replica number: 1 replica
number of stock resources: build 10,000 100kb cr information
stress test scenario: increase pressure according to the gradient of qps 0.1 / 0.5

list json format return data related stress test data:

qps 0.05

before optimization: cpu 35.7 c mem 89Gb
stream json after optimization: cpu 6.22 c mem 60 Gb

qps 0.1

before optimization: cpu 11 c mem 146Gb
stream json after optimization: cpu 7.45 c mem 97 Gb

list protobuf Format Returned data Stress test scenario description:

apiserver version: 1.30
apiserver specification: 32c 128GB
apiserver replica number: 1 replica
Number of existing resources: Build 10,000 configmaps information of size 100kb
Stress test scenario: Increase pressure according to the gradient of qps 0.1 / 0.5

list configmaps format Returned data Related stress test data:

qps 0.05

Before optimization: cpu 16.8 c mem 54.3Gb
After stream json optimization: cpu 16.8 c mem 16.1 Gb

qps 0.1

Before optimization: cpu 42 c mem 122Gb
After stream json optimization: cpu 42 c mem 18 Gb

@chenk008

FYI: Technical details are usually discussed in KEP PRs or elsewhere, with the KEP issue serving as a place to link back work.

@chenk008 @yulongfang you might consider reviewing #5119

@chenk008

Hey @chenk008 @yulongfang please see the previous discussion in kubernetes/kubernetes#129304 and kubernetes/kubernetes#129334.
We also have already done a performance analysis of our changes in kubernetes/kubernetes#129304 (comment).

We also added running a automatic benchmark of list requests. You can see the results in https://perf-dash.k8s.io/#/?jobname=benchmark%20list&metriccategoryname=E2E&metricname=Resources&Resource=memory&PodName=kube-apiserver-benchmark-list-master%2Fkube-apiserver

We currently run it in JSON + configmap with RV="" configuration, hope to expand it to include Proto, Pods, CustomResources and other types of LIST request. Would be awesome if you can contribute.

/milestone v1.33

@jpbetz

@jpbetz @dipesh-rawat this is target to v1.33 and the KEP was merged.
Should the lead-opted-in and tracked label be added and tracked by release team?

@serathius

@serathius @pacoxu Unfortunately, the enhancement freeze deadline has passed, and this KEP issue was not lead-opted-in, so it wasn’t added to the tracking board for the v1.33 release. Post-freeze, we've disabled the automated sync job for KEP issues to the tracking board.

To move forward, we’ll need a short exception request filed so the team can add the lead-opted-in label and manually include this in the tracking board.

If you still wish to progress this enhancement in v1.33, please file an exception request as soon as possible, within three days. If you have any questions, you can reach out in the #release-enhancements channel on Slack and we'll be happy to help. Thanks!

(cc v1.33 Release Lead @npolshakova)

@jpbetz

Ups, @jpbetz is OOO. @deads2k can you take a look?

Sent https://groups.google.com/g/kubernetes-sig-release/c/fDI9FdlClnA

@serathius

Hello @serathius 👋, 1.34 Docs Shadow here.
Please take a look at Documenting for a release - PR Ready for Review to get your PR ready for review before Tuesday July 29th 2025 18:00 PST.
Thank you!

@fykaa

Hi @fykaa, SIG API Machinery lead here. I just wanted to confirm this is code complete for code freeze. There are a couple PRs that were listed as "optional" that were listed under the stable graduation criteria. I've confirmed that there are not blocking and moved them out of the graduation criteria. This should not be good to go for code freeze.

@jpbetz

@jpbetz your response makes it eligible to be tracked for code freeze!
Thanks for the update!

@serathius

Hi @serathius 👋, v1.35 Enhancements Lead here.

I am closing the v1.34 milestone now.

If you'd like to work on this enhancement in v1.35, please have the SIG lead opt-in by adding the lead-opted-in label, which ensures it gets added to the tracking board. Also, please set the milestone to v1.35 using /milestone v1.35.

Thanks!

/remove-label lead-opted-in
/remove-label tracked/yes
/milestone clear

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

Mark this issue as fresh with /remove-lifecycle stale
Close this issue with /close
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

Let's close and track followup separatly
/close

@serathius

@serathius: Closing this issue.

Details

In response to this:

Let's close and track followup separatly
/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

Navigation Menu

Repository navigation

Streaming Encoding for LIST Responses #5116

Enhancement Description

58 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Streaming Encoding for LIST Responses #5116

Description

Enhancement Description

Activity

serathius commented on Jan 31, 2025

chenk008 commented on Feb 7, 2025

yulongfang commented on Feb 8, 2025

BenTheElder commented on Feb 12, 2025

serathius commented on Feb 13, 2025

serathius commented on Feb 13, 2025

pacoxu commented on Feb 14, 2025

dipesh-rawat commented on Feb 14, 2025

serathius commented on Feb 14, 2025

serathius commented on Feb 14, 2025

58 remaining items

yujen77300 commented on Jul 21, 2025

jpbetz commented on Jul 22, 2025

fykaa commented on Jul 23, 2025

rayandas commented on Sep 16, 2025

k8s-triage-robot commented on Dec 15, 2025

serathius commented on Dec 16, 2025

k8s-ci-robot commented on Dec 16, 2025

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions