fishjam-composition

Imported from software-mansion-labs/skills (skills/fishjam/references/composition/SKILL.md). Install upstream with npx skills add software-mansion-labs/skills --skill composition. Copyright stays with the author (MIT).

Fishjam Compositions

A composition takes several live streams, lays them out in a scene, and pushes the result out as new streams in real time. Rendering happens on Fishjam's servers, built on Software Mansion's Smelter.

Because the layout is rendered once on the server, every viewer receives one ready-made stream: it looks the same on every device, and a low-end phone only has to play a single video instead of decoding every participant.

Read ../platform/SKILL.md first. It defines rooms, peers, tracks, management tokens, livestream rooms, and notifications that this skill builds on.

What you can build

• A video call turned into a broadcast. The room's participants in a grid, speaker, or picture-in-picture layout, with names, speaking highlights, and camera-off placeholders, streamed to a livestream audience in your app.
• Restreaming. A call, an OBS feed, or any other input sent to YouTube, Twitch, or another RTMP platform.
• Overlays and broadcast graphics driven by your backend. A LIVE badge, captions, lower thirds, sponsor banners, or a score bug, toggled as things happen in your product.
• A standby or placeholder channel. Graphics only, or a looping video, switched to the live layout when the show starts. An output shows one scene at a time; events or updates switch it.

Mental model

inputs                       composition                 outputs
room tracks (forwarded) ──┐                            ┌─▶ WHIP → Fishjam livestream → viewers
WHIP publisher ───────────┼─▶ scene per output ────────┤
RTMP encoder ─────────────┤   video tree + audio mix   └─▶ RTMP → YouTube / Twitch / other
MP4 file URL ─────────────┘

A composition produces video; it does not serve it. There is no URL a player can open. Every output must push to a destination that viewers can reach.

Designing a composition

Answer three questions: where the streams come from, where the result goes, and what draws the layout.

Where do the streams come from?

Where does the result go?

What draws the layout?

Use a template unless the layout is fixed. Most real compositions react to something: peers joining a call, someone speaking, a caption or "live" badge toggled from the backend. With static scenes your backend has to track all of that state and rebuild and resend the scene on every change; a template does it inside the composition.

Templates are written in TypeScript and React whatever the backend language; a Python backend uploads the built bundle the same way. Read templates.md for templates and scenes.md for the component tree both approaches share.

Flow and SDK calls

Both server SDKs ship a CompositionClient, authenticated with the same management token as FishjamClient (no Fishjam ID needed).

Register in dependency order: assets and inputs before the scenes that reference them, and the destination (for example a livestream room) before the output that pushes to it.

Minimal flow

A looping MP4 composed into a Fishjam livestream room. It uses a static scene to stay self-contained with no build step; for a template version see templates.md. Python equivalent: ../python-server-sdk/composition.md.

import { CompositionClient, FishjamClient } from '@fishjam-cloud/js-server-sdk';
import type { InputId, OutputId } from '@fishjam-cloud/js-server-sdk';

const managementToken = process.env.FISHJAM_MANAGEMENT_TOKEN!;
const fishjamClient = new FishjamClient({ fishjamId: process.env.FISHJAM_ID!, managementToken });
const compositionClient = new CompositionClient({ managementToken });

const livestream = await fishjamClient.createRoom({ roomType: 'livestream' });
const { token } = await fishjamClient.createLivestreamStreamerToken(livestream.id);

const { compositionId } = await compositionClient.createComposition();
await compositionClient.registerMp4Input(compositionId, 'movie' as InputId, {
  url: 'https://smelter.dev/videos/template-scene-race.mp4',
  loop: true,
});
await compositionClient.registerWhipOutput(compositionId, 'main' as OutputId, {
  endpointUrl: fishjamClient.livestreamWhipUrl(),
  bearerToken: token,
  video: {
    resolution: { width: 1280, height: 720 },
    initial: { root: { type: 'rescaler', child: { type: 'input_stream', inputId: 'movie' } } },
  },
  audio: { initial: { inputs: [{ inputId: 'movie' }] } },
});

await compositionClient.deleteComposition(compositionId);
await fishjamClient.deleteRoom(livestream.id);

Between registering the output and deleting, viewers watch livestream.id like any Fishjam livestream. Full signatures: ../js-server-sdk/composition.md and ../python-server-sdk/composition.md.

Lifecycle and billing

• Billed per minute for each registered input and output, whether media flows or anyone watches. Inputs created by forwarding a Fishjam room are not billed. Rates: https://fishjam.swmansion.com/pricing.
• Unregister inputs and outputs you no longer need; they bill for as long as they stay registered.
• Starts automatically by default. Create with autostart: false when inputs arrive later (room forwarding), then call start.
• Idle cleanup: by default a composition is removed after about 5 minutes in which none of its inputs carry media. A composition with no inputs at all counts as idle.
• cleanup_without_inputs: false makes cleanup wait for inputs and outputs to go silent. Needed for late-joining room peers or input-less templates (captions only). An output that keeps publishing is then never cleaned up for you.
• Delete explicitly when finished, even after a test. Deleting removes all its inputs and outputs.

Key rules

• Destinations must exist before you register. An output connects to its endpoint during registration, and an MP4 URL is fetched during registration. Unreachable targets fail the register call, not later.
• One output per livestream room. A Fishjam livestream accepts one streamer (../platform/room-types.md); push each output to its own livestream room.
• updateOutput mirrors registration. An output registered with video and audio needs both in every update; one registered with only video accepts only video.
• Template outputs sound like what they render. Give the output an audio config, or it has no audio track. The mix is every <InputStream> currently rendered; an input that is not rendered, such as a peer with no tile, is silent. Detail: templates.md.
• Resolution must be even on both sides.
• Composed rooms use H.264. Create them with videoCodec: "h264" explicitly, even though it is the default, so a changed default cannot break the composition.
• Casing differs by API. Composition API bodies are snake_case and reject unknown fields. Server API bodies are camelCase (compositionURL, outputId). The SDKs handle both.

References