Capturing video with Petalinux – Part 1 – The hardware design

In one of the previous articles, I showed you how you could build a basic design on Zynq SoC and boot Petalinux over it. This article would be an extension to the previous one. Our goal for this time would be capturing video frames and saving them as raw files. Sounds simple when you have the entire Linux already set up for you. When it comes to building one from scratch, it won’t be very easy. Especially, when you are doing it all for the first time. It took me several week’s worth of effort that included reading documentation, forums posts and trying out various things. For some, the amount of struggle might be lower than what I had to endure throughout the process. Nevertheless, it was an interesting project that I would be presenting here in multiple parts. Well, multiple parts, because I am still not done yet.

The hardware design

The hardware design consists of the Zynq Processing System block configured to output an additional clock of 144.9 MHz to support full HD resolution. You could work with lower clock speeds but that might reduce your frame rate as you go for higher resolution frames.

Next, we add the Video test pattern generator (TPG) and video frame buffer writer blocks. If you are struggling with this part, I highly recommend you read my previous article and then come back here.

You will also need to add the video timing controller (VTC) block. As such, this block isn’t very useful if you aren’t pushing your video out through any display interface. The TPG Linux driver looks for the timing controller and fails if it isn’t in the design. This reason forces us to include the VTC block in spite of not pushing the video out over any interface.

Also, add the AXI GPIO block in your design and this is required by the Video4Linux2 (V4L2) API. This AXI GPIO block will connect to the reset pins of the test pattern generator and the frame buffer writer that we have in the system. When we finally have our system ready for use, we would be using V4L2 for configuring our video pipeline. It’s during the configuration stage when V4L2 asserts a reset signal to reset the components of the video pipeline.

Reset block

Initially, while building the design and trying out things, I assumed that I don’t need a controllable reset system. The whole thing gets reset at power-on anyway, so what’s the need for a controllable reset? When I tried booting the system, the driver threw an error saying that it wasn’t able to find the reset property for the device. While going through the driver source code, I realized that the V4L2 tries to reset the component and eventually fails when it’s unable to do that. Therefore, we need some sort of reset-ability which could come either from AXI GPIO or the Zynq’s hard GPIO peripheral.
You can find the final block design of our system below.

.

I am using the Zybo 7010 board for this project and the board files helped me auto-configure most of the Zynq PS system. Make sure you enable UART0 in the PS because that’s how we will communicate with the board when we boot it up. I have uploaded the board design tcl file which will help you recreate the whole project quite easily.

Export the hardware file and start building petalinux

The basic steps to build petalinux were shown in my previous article where we built Petalinux from scratch. Most of those steps will remain untouched except for the addition of a few extra steps. Therefore, I will only be showing the added extra steps here to avoid making the article too long.

petalinux-config -c rootfs

This should open the following window where you need to select certain features.

You need to select the v4l2utils. Additionally, you may also select openCV and the rest of the features that you might consider useful.

Usually, the next step would be to build petalinux but we are going to wait because there’s something more we need to do. That something is defining the video pipeline in the device tree.

Defining the device tree

Fortunately, the base system device tree gets auto-generated by the device tree generator built scripts built into Petalinux. Nevertheless, Petalinux puts the job of defining video pipeline interconnection upon the user.

Automatically generated device tree resides in the following path:

/<project_dir>/components/plnx_workspace/device-tree/device-tree

We won’t be touching any of the files in this directory. All the files in this directory are auto-generated by Petalinux every time you build it. Edit the following file to add our video pipeline.

/<project_dir>/project-spec/meta-user/recipes-bsp/device-tree/system-user.dtsi

The system-user.dtsi file tells Petalinux about manual device tree entries. The V4L2 expects you to specify what component connects to what. In addition to that, we also specify the video format, pixel width, reset pin, port direction and so on.

I have practically copy-pasted my entire system-user.dtsi file for your reference. When you read carefully, the TPG has one output port that’s mentioned as tpg_out and it connects to its ‘remote-endpoint’ vcap_tpg_in mentioned in the next block of code just below it.

You must read the pl.dtsi that resides alongside system-conf.dtsi in the auto-generated device tree directory. There are times when the auto-generator script fails to identify all the properties in your hardware. For example, when I was building this system, the pl.dtsi did not mention any reset property for the TPG. Hence, I added it manually in the system-user.dtsi file.

reset-gpios = <&axi_gpio_0 0 0 1>;

Now, its time to build the petalinux system by running,

petalinux-build

This may take quite a while depending on your machine. Sit back and let it finish. Finally, when the build completes, we need to package it with the following command.

Incoming search terms:

• https://nuclearrambo com/wordpress/capturing-video-with-petalinux-part-1-the-hardware-design/