[opentelemetry-util-genai] Remove experimental sem conv flag, usage, and references. Cleanup tests

[DylanRussell]

Description

This PR removes the experimental sem conv flag, usage, and references from opentelemetry-util-genai. Also does cleanup on some of the associated tests..

I also removed the last vestiges of the flag from the Google GenAi instrumentation which were blocked on it being removed from the utils package first..

The other instrumentation packages need to migrate off some of the deprecated code before this can be submitted..

Fixes # (issue)

Type of change

Please delete options that are not relevant.

• [ x] Bug fix (non-breaking change which fixes an issue)

How has this been tested?

Unit tests

Checklist

See CONTRIBUTING.md
for the style guide, changelog guidance, and more.

• [x ] Followed the style guidelines of this project
• [ x] Changelog updated if the change requires an entry
• [ x] Unit tests added
• [ x] Documentation updated

[Copilot]

Pull request overview

This PR removes the GenAI experimental semantic-conventions opt-in flag path from opentelemetry-util-genai, and updates util + Google GenAI instrumentation tests/docs to rely solely on the GenAI content-capture and event-emission environment variables.

Changes:

• Removed semconv “experimental” flag gating and related test helpers, consolidating content/event behavior around OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT and OTEL_INSTRUMENTATION_GENAI_EMIT_EVENT.
• Refactored util invocation code paths to use the new “should_*” helpers instead of semconv stability mode checks.
• Updated/cleaned up util and Google GenAI instrumentation tests and local dev env configuration accordingly.

Reviewed changes

Copilot reviewed 18 out of 18 changed files in this pull request and generated 10 comments.

Show a summary per file

File	Description
util/opentelemetry-util-genai/src/opentelemetry/util/genai/utils.py	Removes semconv stability checks and redefines content/event decision helpers (currently introduces unset-env crashes; see comments).
util/opentelemetry-util-genai/src/opentelemetry/util/genai/handler.py	Removes semconv-driven capture logic (currently drops TelemetryHandler.should_capture_content() API used by other instrumentations).
util/opentelemetry-util-genai/src/opentelemetry/util/genai/_workflow_invocation.py	Switches span message capture gating to should_capture_content_on_spans().
util/opentelemetry-util-genai/src/opentelemetry/util/genai/_retrieval_invocation.py	Switches span content attribute gating to should_capture_content_on_spans().
util/opentelemetry-util-genai/src/opentelemetry/util/genai/_invocation.py	Removes experimental-mode early return so content capture is controlled solely by content mode.
util/opentelemetry-util-genai/src/opentelemetry/util/genai/_inference_invocation.py	Emits events based on should_emit_event() only (no longer semconv-gated).
util/opentelemetry-util-genai/tests/test_utils.py	Moves semconv/env helper coverage into this file and adds direct tests for content/event helpers.
util/opentelemetry-util-genai/tests/test_utils_events.py	Replaces the shared env patch helper with explicit patch.dict usage (currently mis-tests defaults by explicitly setting "false").
util/opentelemetry-util-genai/tests/test_handler_retrieval.py	Removes semconv stability setup/teardown logic from retrieval tests.
util/opentelemetry-util-genai/tests/test_handler_completion_hook.py	Removes semconv stability setup and deletes tests tied to the removed should_capture_content() behavior.
util/opentelemetry-util-genai/tests/test_handler_agent.py	Removes is_experimental_mode() patching from agent tests.
util/opentelemetry-util-genai/tests/test_events_options.py	Deletes the old semconv-stability-specific test module (coverage moved/rewritten elsewhere).
util/opentelemetry-util-genai/README.rst	Removes instructions to set OTEL_SEMCONV_STABILITY_OPT_IN for GenAI behavior.
instrumentation/opentelemetry-instrumentation-google-genai/tests/generate_content/test_e2e.py	Removes semconv stability env setup and relies on content-capture env only (fixture currently leaks env var).
instrumentation/opentelemetry-instrumentation-google-genai/tests/generate_content/test_config_span_attributes.py	Ensures tests explicitly set content capture mode to avoid relying on removed stability flag.
instrumentation/opentelemetry-instrumentation-google-genai/tests/common/base.py	Removes semconv stability initialization; now sets content-capture env directly (currently not restored).
instrumentation/opentelemetry-instrumentation-google-genai/dev/main.env	Removes the semconv stability opt-in env var from local dev env.

[lmolkova]

Thanks! Just some questions and suggestions to keep public API

[lzchen]

If we are removing all experimental flags, why keep this? I thought now was the time to make breaking changes?

[lmolkova]

because it's a public API used by libs in the contrib.

The plan:

• release this repo
• release the last version on libs in the contrib (they should still work with these utils) and deprecated them along with the release
• after that we're free to party on and break runtime compatibility with them.

You can argue that everything is experimental and not expected to be compatible, but really this version of util here is likely the only thing every lib in contrib repo is compatible with.

[lzchen]

Nit: Why not keep the patch instead of changing real env vars?