Notes

Posted by lane

I needed a GstController to control the alpha channel on a video crossfade. The gstreamer documentation makes it look easy, but it actually tripped me up for quite some time. I am using the python bindings, and they have some subtlies that took me some time to figure out.

First, I got the general usage out of the Pitivi source code. To create a controller, you do the following:

        controller = gst.Controller(alpha, "alpha")
        controller.set_interpolation_mode("alpha", gst.INTERPOLATE_LINEAR)
        controller.set("alpha", 0, 0.0)
        controller.set("alpha", 5*gst.SECOND, 1.0)

In the above example, alpha is the element to be controlled. This causes the alpha property on the alpha element to linearly change from 0.0 to 1.0 over the course of 5 seconds.

After adding the above code snippet to my application, however, it did not work. To debug it, I ended up checking out the gstreamer source code and after tracing things through using printf()’s, I finally discovered that my controller was getting attached to the alpha as I wanted but then it was getting detached almost immediately thereafter. That made me realize that in python I needed to hold a reference to the controller for the duration of the stream. I was letting the controller variable in the above example go out of scope, at which point it had no more references and therefore got deleted and removed from the alpha element. So the solution was to hold a reference to it as shown below in the example of a video crossfade:

import os, time
import gobject
import pygst
pygst.require("0.10")
import gst

class Main:
    def __init__(self):
        src1 = gst.element_factory_make("filesrc")
        src2 = gst.element_factory_make("filesrc")

        src1.set_property("location", "/home/lane/test1.mp4")
        src2.set_property("location", "/home/lane/test2.mp4")

        dec1 = gst.element_factory_make("decodebin2")
        dec2 = gst.element_factory_make("decodebin2")

        alpha1 = gst.element_factory_make("alpha")
        alpha2 = gst.element_factory_make("alpha")

        self.controller = gst.Controller(alpha2, "alpha")
        self.controller.set_interpolation_mode("alpha", gst.INTERPOLATE_LINEAR)
        self.controller.set("alpha", 0, 0.0)
        self.controller.set("alpha", 5*gst.SECOND, 1.0)

        mixer = gst.element_factory_make("videomixer")

        queue = gst.element_factory_make("queue")
        color = gst.element_factory_make("ffmpegcolorspace")
        sink  = gst.element_factory_make("autovideosink")

        pipeline = gst.Pipeline("pipeline")
        pipeline.add(src1, src2, dec1, dec2, alpha1, alpha2, mixer, queue, color, sink)

        def on_pad(comp, pad, data, element):
            sinkpad = element.get_compatible_pad(pad, pad.get_caps())
            pad.link(sinkpad)

        dec1.connect("new-decoded-pad", on_pad, alpha1)
        dec2.connect("new-decoded-pad", on_pad, alpha2)
        src1.link(dec1)
        src2.link(dec2)
        alpha1.link(mixer)
        alpha2.link(mixer)
        mixer.link(queue)
        queue.link(color)
        color.link(sink)

        self.pipeline = pipeline

    def start(self):
        self.running = True
        bus = self.pipeline.get_bus()
        bus.add_signal_watch()
        bus.enable_sync_message_emission()
        bus.connect("message", self.on_message)
        self.pipeline.set_state(gst.STATE_PLAYING)

    def on_exit(self, *args):
        self.pipeline.set_state(gst.STATE_NULL)
        self.running = False

    def on_message(self, bus, message):
        t = message.type
	if t == gst.MESSAGE_EOS:
            self.on_exit()
	elif t == gst.MESSAGE_ERROR:
            err, debug = message.parse_error()
	    print "Error: %s" % err, debug
            self.on_exit()

loop = gobject.MainLoop()
gobject.threads_init()
context = loop.get_context()

m = Main()
m.start()

while m.running:
    context.iteration(True)

Some things to note in the above code:

• In order to prevent the controller from going out of scope and getting deleted, I added it as an attribute to the Main object (i.e. self.controller).
• I am manually iterating the mainloop. This is so that I do not have to create any gtk stuff. I recently discovered this technique and I use it in command line tools that do not require X11 or a $DISPLAY environment variable set.

This entry was posted on Tuesday, January 4th, 2011 at 4:10 pm and is filed under python. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.