Skip to content

storage: Delete mvlog#30798

Open
StephanDollberg wants to merge 3 commits into
devfrom
stephan/rm-mvlog
Open

storage: Delete mvlog#30798
StephanDollberg wants to merge 3 commits into
devfrom
stephan/rm-mvlog

Conversation

@StephanDollberg

Copy link
Copy Markdown
Member

It's unused dead code and no plan of using it.

Remove to save build time and context.

Backports Required

  • none - not a bug fix
  • none - this is a backport
  • none - issue does not exist in previous branches
  • none - papercut/not impactful enough to backport
  • v26.1.x
  • v25.3.x
  • v25.2.x

Release Notes

  • none

It's unused dead code and no plan of using it.

Remove to save build time and context.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR removes the unused experimental storage::experimental::mvlog implementation (and its associated Bazel targets + tests) to reduce build time and eliminate dead code.

Changes:

  • Deleted the src/v/storage/mvlog library sources/headers (including versioned_log, segment read/write helpers, and supporting types).
  • Deleted the src/v/storage/mvlog/tests test suite and its Bazel BUILD file.
  • Removed the Bazel BUILD file for the mvlog package itself.

Reviewed changes

Copilot reviewed 40 out of 40 changed files in this pull request and generated no comments.

Show a summary per file
File Description
src/v/storage/mvlog/versioned_log.h Removed unused versioned log public API/types.
src/v/storage/mvlog/versioned_log.cc Removed unused versioned log implementation.
src/v/storage/mvlog/tests/skipping_data_source_test.cc Removed mvlog test coverage for skipping data source.
src/v/storage/mvlog/tests/segment_reader_test.cc Removed mvlog segment reader tests.
src/v/storage/mvlog/tests/segment_io_test.cc Removed mvlog segment IO tests.
src/v/storage/mvlog/tests/segment_appender_test.cc Removed mvlog segment appender tests.
src/v/storage/mvlog/tests/file_test.cc Removed mvlog file wrapper tests.
src/v/storage/mvlog/tests/entry_stream_utils_test.cc Removed mvlog entry stream utils tests.
src/v/storage/mvlog/tests/entry_stream_utils_death_test.cc Removed mvlog death test target.
src/v/storage/mvlog/tests/BUILD Removed Bazel test targets for mvlog.
src/v/storage/mvlog/tests/batch_collector_test.cc Removed mvlog batch collector tests.
src/v/storage/mvlog/tests/batch_collecting_stream_utils_test.cc Removed mvlog batch collecting stream utils tests.
src/v/storage/mvlog/tests/active_segment_test.cc Removed mvlog active segment/versioned log tests.
src/v/storage/mvlog/skipping_data_source.h Removed skipping data source API.
src/v/storage/mvlog/skipping_data_source.cc Removed skipping data source implementation.
src/v/storage/mvlog/segment_reader.h Removed segment reader API.
src/v/storage/mvlog/segment_reader.cc Removed segment reader implementation.
src/v/storage/mvlog/segment_identifier.h Removed segment id type.
src/v/storage/mvlog/segment_appender.h Removed segment appender API.
src/v/storage/mvlog/segment_appender.cc Removed segment appender implementation.
src/v/storage/mvlog/reader_outcome.h Removed internal reader outcome enum.
src/v/storage/mvlog/readable_segment.h Removed readable segment API.
src/v/storage/mvlog/readable_segment.cc Removed readable segment implementation.
src/v/storage/mvlog/logger.h Removed mvlog logger declaration.
src/v/storage/mvlog/logger.cc Removed mvlog logger definition.
src/v/storage/mvlog/file.h Removed mvlog file + file_manager wrapper API.
src/v/storage/mvlog/file.cc Removed mvlog file + file_manager wrapper implementation.
src/v/storage/mvlog/file_gap.h Removed file gap type.
src/v/storage/mvlog/errc.h Removed mvlog error codes.
src/v/storage/mvlog/entry.h Removed mvlog entry types.
src/v/storage/mvlog/entry.cc Removed mvlog entry formatting implementation.
src/v/storage/mvlog/entry_stream.h Removed mvlog entry stream API.
src/v/storage/mvlog/entry_stream.cc Removed mvlog entry stream implementation.
src/v/storage/mvlog/entry_stream_utils.h Removed mvlog entry stream utils API.
src/v/storage/mvlog/entry_stream_utils.cc Removed mvlog entry stream utils implementation.
src/v/storage/mvlog/BUILD Removed Bazel library target for mvlog.
src/v/storage/mvlog/batch_collector.h Removed batch collector API.
src/v/storage/mvlog/batch_collector.cc Removed batch collector implementation.
src/v/storage/mvlog/batch_collecting_stream_utils.h Removed batch collecting stream utils API.
src/v/storage/mvlog/batch_collecting_stream_utils.cc Removed batch collecting stream utils implementation.

dotnwat
dotnwat previously approved these changes Jun 12, 2026
@dotnwat

dotnwat commented Jun 12, 2026

Copy link
Copy Markdown
Member

@StephanDollberg i think you can also remove the "src/v/io" directory.

@andrwng

andrwng commented Jun 12, 2026

Copy link
Copy Markdown
Contributor

There are some other areas of code that aren't used (were aspirationally worked on but then deprioritized). The page cache comes to mind, and I also see some v8 engine stuff in the tree.

Alternatively, I wonder if there's a way to get agents to ignore these parts of the tree, and remove them from the full build, in case the work is ever reprioritized. I suppose it's always possible to revert a removal, but I guess it's also possible people just forget about it and start from scratch (which isn't the end of the world, just duplicate work, if it matters).

@dotnwat

dotnwat commented Jun 12, 2026

Copy link
Copy Markdown
Member

page cache

is that the src/v/io or something else (either way, nuke it)

v8

this one is like 99.999% removed. i think the only thing remaining was some constant used to decode a record that might exist in the controller log. i don't think it makes any sense to keep around the v8 files in the tree, so maybe we coudl just move that constant to where it's used.

@StephanDollberg

Copy link
Copy Markdown
Member Author

Cool, will remove those as well.

I think it's definitely good to remove that stuff for build/CI times, human context size, LLM context size and avoid having to update it constantly.

@vbotbuildovich

Copy link
Copy Markdown
Collaborator

Retry command for Build#85753

please wait until all jobs are finished before running the slash command

/ci-repeat 1
skip-redpanda-build
skip-units
skip-rebase
tests/rptest/tests/cluster_linking_e2e_test.py::ShadowLinkingReplicationTests.test_auto_prefix_trimming@{"source_cluster_spec":{"cluster_type":"redpanda"},"storage_mode":"tiered","with_failures":true}

@vbotbuildovich

vbotbuildovich commented Jun 12, 2026

Copy link
Copy Markdown
Collaborator

CI test results

test results on build#85753
test_status test_class test_method test_arguments test_kind job_url passed reason test_history
FLAKY(FAIL) ShadowLinkingReplicationTests test_auto_prefix_trimming {"source_cluster_spec": {"cluster_type": "redpanda"}, "storage_mode": "tiered", "with_failures": true} integration https://buildkite.com/redpanda/redpanda/builds/85753#019ebd6b-4f15-4423-ae98-b61575876931 26/31 Test FAILS after retries.Significant increase in flaky rate(baseline=0.0095, p0=0.0002, reject_threshold=0.0100) https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=ShadowLinkingReplicationTests&test_method=test_auto_prefix_trimming
FLAKY(PASS) NodeWiseRecoveryTest test_recovery_local_data_missing {"wait_for_final_manifest_uploads": false} integration https://buildkite.com/redpanda/redpanda/builds/85753#019ebd6b-e9c2-4a71-8c42-ec36f3029722 10/11 Test PASSES after retries.No significant increase in flaky rate(baseline=0.0176, p0=1.0000, reject_threshold=0.0100. adj_baseline=0.1000, p1=0.3487, trust_threshold=0.5000) https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=NodeWiseRecoveryTest&test_method=test_recovery_local_data_missing
test results on build#85758
test_status test_class test_method test_arguments test_kind job_url passed reason test_history
FLAKY(FAIL) AuditLogTestsAppLifecycle test_recovery_mode {"audit_transport_mode": "kclient"} integration https://buildkite.com/redpanda/redpanda/builds/85758#019ebdc8-a1e3-48a8-bc76-30b573248c16 16/21 Test FAILS after retries.Significant increase in flaky rate(baseline=0.0212, p0=0.0007, reject_threshold=0.0100) https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=AuditLogTestsAppLifecycle&test_method=test_recovery_mode
FLAKY(FAIL) ShadowLinkingReplicationTests test_auto_prefix_trimming {"source_cluster_spec": {"cluster_type": "redpanda"}, "storage_mode": "cloud", "with_failures": false} integration https://buildkite.com/redpanda/redpanda/builds/85758#019ebdc8-a1e1-44a0-ba5a-dd1838ca3703 24/31 Test FAILS after retries.Significant increase in flaky rate(baseline=0.0095, p0=0.0000, reject_threshold=0.0100) https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=ShadowLinkingReplicationTests&test_method=test_auto_prefix_trimming
FLAKY(FAIL) ShadowLinkingReplicationTests test_auto_prefix_trimming {"source_cluster_spec": {"cluster_type": "redpanda"}, "storage_mode": "tiered", "with_failures": false} integration https://buildkite.com/redpanda/redpanda/builds/85758#019ebdc8-a1e2-4c50-9bcd-4a5f8542dc2e 8/11 Test FAILS after retries.Significant increase in flaky rate(baseline=0.0095, p0=0.0038, reject_threshold=0.0100) https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=ShadowLinkingReplicationTests&test_method=test_auto_prefix_trimming
FLAKY(PASS) NodeWiseRecoveryTest test_recovery_local_data_missing {"wait_for_final_manifest_uploads": false} integration https://buildkite.com/redpanda/redpanda/builds/85758#019ebdc5-b23e-486f-9b34-2abf7198d046 10/11 Test PASSES after retries.No significant increase in flaky rate(baseline=0.0184, p0=1.0000, reject_threshold=0.0100. adj_baseline=0.1000, p1=0.3487, trust_threshold=0.5000) https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=NodeWiseRecoveryTest&test_method=test_recovery_local_data_missing
FLAKY(PASS) ShadowLinkingRandomOpsTest test_node_operations {"failures": true, "workload_set": "cloud_combos"} integration https://buildkite.com/redpanda/redpanda/builds/85758#019ebdc8-a1e5-4836-9cf4-14e891cc24f5 10/11 Test PASSES after retries.No significant increase in flaky rate(baseline=0.0220, p0=1.0000, reject_threshold=0.0100. adj_baseline=0.1000, p1=0.3487, trust_threshold=0.5000) https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=ShadowLinkingRandomOpsTest&test_method=test_node_operations

@vbotbuildovich

vbotbuildovich commented Jun 12, 2026

Copy link
Copy Markdown
Collaborator

Retry command for Build#85758

please wait until all jobs are finished before running the slash command

/ci-repeat 1
skip-redpanda-build
skip-units
skip-rebase
tests/rptest/tests/cluster_linking_e2e_test.py::ShadowLinkingReplicationTests.test_auto_prefix_trimming@{"source_cluster_spec":{"cluster_type":"redpanda"},"storage_mode":"tiered","with_failures":false}
tests/rptest/tests/cluster_linking_e2e_test.py::ShadowLinkingReplicationTests.test_auto_prefix_trimming@{"source_cluster_spec":{"cluster_type":"redpanda"},"storage_mode":"cloud","with_failures":false}
tests/rptest/tests/audit_log_test.py::AuditLogTestsAppLifecycle.test_recovery_mode@{"audit_transport_mode":"kclient"}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants