Skip to content

Add HuggingFace media tasks for image, audio, and video output rendering in result panel #5674

@juliethecao

Description

@juliethecao

Task Summary

Feature Summary

The HuggingFace inference operator is being landed as a sequence of focused PRs from parent issue #5041. The backend operator and task families were introduced in earlier PRs; the property editor components were added in the preceding PR. This issue covers rendering media outputs (images, audio, video) produced by HuggingFace tasks directly in the workflow result panel.

HuggingFace text-to-image, text-to-speech, and text-to-video tasks return data URLs or CDN-hosted media URLs as result fields. Without this PR, those URLs are displayed as raw text. This issue adds inline media rendering in both the result table and the row detail modal, so media results are immediately visible without leaving the workflow editor.

Concretely, landing this would add:

  • isImageUrl / isAudioUrl / isVideoUrl — URL detection helpers that identify media data URLs and file extension URLs, used to determine how to render a result cell
  • Inline media rendering in the result table — image cells render as <img>, audio cells as <audio>, video cells as <video>; non-media cells fall through to existing text rendering
  • Media rendering in the row detail modal — the row detail view renders media fields inline with a copy-to-clipboard fallback for the raw URL
  • Backend string truncation scoped by output mode — the 100-char truncation is disabled for SetSnapshotMode (used by visualization/HF outputs) so full data URLs are not cut off; regular table pagination mode retains truncation

Proposed Solution or Design

Add new files:

Path Purpose
frontend/.../common/util/media-type.util.ts isImageUrl, isAudioUrl, isVideoUrl detection helpers
frontend/.../common/util/media-type.util.spec.ts Unit tests for all three helpers across data URLs, file extensions, CDN hosts, and rejection cases

Modify:

File Change
result-table-frame.component.{ts,html} Add isImageCell / isAudioCell / isVideoCell methods; render <img>, <audio>, <video> tags conditionally in the table cell template
result-table-frame.component.spec.ts Tests for the three cell-type detection methods
result-panel-modal.component.{ts,html} Build rowEntries with per-field media metadata; render media inline in the row detail view
result-panel-model.component.scss Add modal-toolbar and row-detail styles
ExecutionResultService.scala (+ spec) Gate string truncation on isVisualization; pass true when mode is SetSnapshotMode so HF data URLs are never truncated

Design constraints:

  • Media detection is purely client-side and stateless — no backend calls required.
  • Non-media cells fall through to existing rendering unchanged — no regressions for regular table operators.
  • String truncation is preserved for PaginationMode and SetDeltaMode; only SetSnapshotMode (visualization/HF) disables it.
  • The fal.media CDN host is explicitly recognized as a video source in isVideoUrl since fal.ai returns hosted URLs rather than data URLs for video outputs.

References


Impact / Priority

(P2) Medium — required for HuggingFace media task results to be usable in the UI. Does not affect existing operators or non-HuggingFace result panels.

Affected Area

Frontend (Angular) — result panel components and shared utility.
Backend (Scala) — ExecutionResultService string serialization.

Task Type

  • Refactor / Cleanup
  • DevOps / Deployment / CI
  • Testing / QA
  • Documentation
  • Performance
  • Other

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No fields configured for Task.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions