Skip to content

Custom Pipelines

IP Camera streams can be configured with their own custom GStreamer pipelines, allowing for rich configuration of how the stream is processed. This section will not explain the intricacies of GStreamer pipelines as the official website provides excellent documentation on how these work. Instead, included are a few pipeline examples.

BrainFrame does quite a bit of work in the background to ensure that many different IP camera types are supported seamlessly. When using custom pipelines, more intimate knowledge of the IP camera stream is required compared to using BrainFrame normally.

Note that all custom pipelines:

  • Must include a {url} template field. This is where the specified IP camera URL will be inserted into the pipeline.
  • Must have an appsink element named "main_sink". This is where frames will be extracted from the pipeline for processing.
  • May optionally include an element named "buffer_src". This is required for frame skipping to work with custom pipelines. This name should be given to an element in the pipeline that sections off frame data from the network before decoding, like rtph264depay.

To specify a custom pipeline, check the "Advanced Options" checkbox in the stream creation window and enter your pipeline into the "Pipeline" textbox.

Example Pipelines

Cropping the Video Stream

For composite video streams or for scenes that contain uninteresting sections, one may want to crop the video stream before processing. Here is an example of a custom pipeline to accomplish this for an H264 RTSP stream:

rtspsrc location="{url}" ! rtph264depay name="buffer_src" ! decodebin ! videocrop top=x left=x right=x bottom=x ! videoconvert ! video/x-raw,format=(string)BGR ! appsink name="main_sink"

This pipeline uses the videocrop element to crop the video by some configurable value. The "x" values should be replaced with the amount in pixels to crop from each side of the frame.

Lower Latency Streaming

By default, BrainFrame will “buffer” frames in order to ensure a more stable streaming experience. In order to prevent that, try using the pipeline below:

rtspsrc location="{url}" latency=X ! rtph264depay name="buffer_src" ! decodebin ! videoconvert ! video/x-raw,format=(string)BGR ! appsink name="main_sink"

Replace the "X" in latency=X with 0 for no buffering at all. The unit X is in milliseconds.

Rotating the Video Stream

rtspsrc location="{url}" ! rtph264depay ! avdec_h264 ! videoconvert ! videoflip video-direction=x ! videoconvert ! video/x-raw,format=(string)BGR ! appsink name="main_sink"

The "x" value should be the number of degrees to rotate. Try numbers such as 0, 90, 180, and so on.

Hardware Decoding with Multiple Nvidia GPUs

BrainFrame automatically detects when an Nvidia GPU is available and attempts to do hardware video decoding on it. Currently, video decoding is only done on the first available device. This means that if your machine has multiple Nvidia GPUs installed, only one of them will be utilized.

GStreamer dynamically creates decoder elements that allow you to choose which Nvidia GPU the work will be done on. Using H.264 as our example format:

  • nvh264dec uses device 0
  • nvh264device1dec uses device 1
  • nvh264device2dec uses device 2
  • ... and so on

By using different decoder elements for each stream's custom pipeline, you can distribute decoding work across multiple GPUs. For example, if you had fifteen video streams and three GPUs, you might consider having the first five use nvh264dec, the next five use nvh264device1dec, and the final five use nvh264device2dec.

The device IDs referenced here are CUDA device IDs. By default, CUDA orders devices from fastest to slowest, device 0 being the fastest. It is possible to change the way CUDA orders devices via an environment variable. See the official documentation for details.

Here is an example pipeline that uses device 1 to decode an H.264 RTSP stream:

rtspsrc location="{url}" ! rtph264depay name="buffer_src" ! h264parse ! nvh264device1dec ! glcolorconvert ! video/x-raw(memory:GLMemory),format=(string)BGR ! gldownload ! video/x-raw,format=(string)BGR ! appsink name="main_sink"

Warning

Current releases of the BrainFrame Client do not support using NVCODEC hardware decoding. Using these custom pipelines will cause streaming errors in the client as a result. We only recommend these pipelines for advanced use cases.