• 0
Esti.A

OpenCV and Pcam5-c

Question

Hi everyone,

I am working in teh demo that was created by Digilent to get images from teh Pcam5-c that is connected by a MIPI CSI-2 interface to teh zybo z7-020 board. In this case, I was wondering if I decide to do the processing of the image (edge detection, enhancement of light,...) I would need to use SDx. I have seen that there are plenty of solutions in OpenCv but I dont know how the hardware proyect and the SDx project can be linked. Note, in this initial design I have teh initiallization od teh camara and platform in a C++ application file that I dont know how if I should export to sdx file and how does this interact with the platform.

Anothr queation I have is that for interacting with the openCv set-up do I must use Linux ?

Kind regards

Esti

 

Share this post


Link to post
Share on other sites

20 answers to this question

Recommended Posts

  • 0

Thank you @jpeyron for your quick answer. I see that I should install linux to work with these softwares as most of them are only suported by Linux. Another thing I dont understand is if I can use Vivado 2018.2 for the hardware design but a SDx 2017.4 for the same proyect or not.

Thank you very much

Esti

Share this post


Link to post
Share on other sites
  • 0

Hi @Esti.A,

Projects are version specific.

It is my understanding that SDSoC is a wrapper for HLS.

HLS IP blocks are imported into Vivado/SDK.

I believe you need to use the same version of Vivado/SDK as you are using for SDx. 

best regards,

Jon

Share this post


Link to post
Share on other sites
  • 0

Hi @jpeyron,

And consequently the SDx version should match with the petalinux version. So how I am supossed to build projects in other vivado versions? cause I am getting quite a lot of error to configure the petalinux project.  Can this be due to the bsp file definition?

As I have read, I have to create a hdf file and try to match it withe the created project, so what is not making sense?

Thanks

Esti

 

Screenshot from 2019-04-05 15-56-20.png

Share this post


Link to post
Share on other sites
  • 0

Hi @Esti.A,

I would suggest using the version of Vivado/SDK/Petalinux that the project was made in and for. If you are typing to update an existing project to an newer version there are alot of potential issues that could occur. We do not have a tutorial on how to do this. I would suggest looking through the xilinx petalinux documentation like the petalinux wiki. Here is a forum thread that might be helpful as well.

best regards,

Jon

Share this post


Link to post
Share on other sites
  • 0

Hi @jpeyron,

My question is how sdx is helping me to create a proyect managing all the libraries of Open CV, because as far as I know I could use OpenCv without xfopenCv whic is integrated in SDx. Another thing that is not clear is how should the set-up done to get a connection with teh actual PCam and OpenCV.

 

Thanks

 

Esti

 

 

Share this post


Link to post
Share on other sites
  • 0
23 hours ago, Esti.A said:

Hi @jpeyron,

My question is how sdx is helping me to create a proyect managing all the libraries of Open CV, because as far as I know I could use OpenCv without xfopenCv whic is integrated in SDx. Another thing that is not clear is how should the set-up done to get a connection with teh actual PCam and OpenCV.

 

Thanks

 

Esti

 

 

Hi @Esti.A,

SDx, which includes SDSoC (Software Defined System on Chip), is a development environment that allows you to develop a computer vision application, in your case, using C/C++ and OpenCV library. The target of SDx-built applications are Xilinx systems on chip (SoC) (Zynq-7000 or Zynq Ultrascale+). Xilinx SoC architecture has two main components: ARM processor (single or multi core) named Processing System (PS) and FPGA, named Programmable Logic (PL). Using SDx to build an application for SoC allows you to choose which functions from your algorithm are executed in PS and which ones are executed in PL. SDx will generate all data movers and dependencies that you need to move data between PS, DDR memory and PL. The PL is suitable for operations that can be easily executed in parallel. So if you are going to choose a median filter function to be executed in PL, instead of PS, you will obtain a better throughput from your system.

As you said, you can use OpenCV to develop your application. You have to take into account that OpenCV library was developed with CPU architecture in mind. So the library was designed to obtain the best performance on some specific CPU architectures (x86-64, ARM, etc.). If you are trying to accelerate an OpenCV function in PL using SDx you will obtain a poor performance. To overcome this issue, Xilinx has developed xfopencv, which is a subset if OpenCV library functions. The functionalities of xfopecv functions and OpenCV functions  are the same but the xfopencv functions are implemented having FPGA architecture in mind. xfopencv was developed in C/C++ following some coding guideline. When you are building a project, the C/C++ code is given as input to Xilinx HLS (High Level Synthesis) tool that will convert it to HDL (Hardware Description Language) that will be synthetized for FPGA. The above mentioned coding guideline provides information about how to write C/C++ code that will be implemented efficiently in FPGA. To have a better understanding on xfopencv consult this documentation.

So SDx helps you to obtain a better performance by offloading PS and by taking advantage of parallel execution capabilities of PL. Have a look on SDSoC documentation. For more details check this.

An SoC is a complex system composed by a Zynq (ARM + FPGA), DDR memory and many types of peripherals. Above those,  one can run a Linux distribution (usually Petalinux, from Xilinx) and above the Linux distribution, the user application will run. The user application may access the DDR memory and different types of peripherals (PCam in your case). Also, it may accelerate some functions in FPGA to obtain a better performance. To simplify the development pipeline Xilinx provides an abstract way to interact with, named SDSoC platform. SDSoC platform has two components: Software Component and Hardware Component that describes the system from the hardware to the operating system. Your application will interact with this platform. You are not supposed to know all details about this platform. This was the idea, to abstract things. Usually, the SDSoC platforms are provided by the SoC development boards providers, like Digilent.

All you have to do is to download the last SDSoC platform release from github. You have to use SDx 2017.4. You don't have to build your own SDSoC platform. This is a complex task. You can follow these steps in order to build your first project that will use PCam and Zybo Z7 board. The interaction between PCam and the user application is done in the following way: there is an IP in FPGA that acquires live video stream from the camera, the video stream is written into DDR memory. This pipeline is abstracted by the SDSoC platform. The user application can access the video frames by Video4Linux (V4L2). The Live I/O for PCam demo shows you how to do this.

I suggest you to read the proposed documentation to obtain a basic knowledge needed for SDSoC projects development.

Best regards,

Bogdan D.

Share this post


Link to post
Share on other sites
  • 0

Thank you for your great answer @bogdan.deac . I previously have tried with this project but I get the same problem as with Vivado 2018.2 which is that I am not able to write teh dsa file because its not able to get the properties of the design.

So I dont know what is not working in there but I get the same result for both projects. This one, has been directly ran and loaded without any changes on it.

1265011604_Screenshotfrom2019-04-1718-21-21.thumb.png.1e9d6fecf590beb80af93c971a4f2feb.png

 

Share this post


Link to post
Share on other sites
  • 0

Hi @bogdan.deac,

1.The version I am using is 2017.4.

2. I cloned from git directly from the link.

 

I resolved the problem by manually defining the location of the bd. loading it again and introducing every line in the TCL console.

Share this post


Link to post
Share on other sites
  • 0

Hi @Esti.A,

If you clone the repo you obtain the "source code" for the platform and you have to generate the platform by yourself. This is a time consuming and complicated task and is not recommended if you do not understand SDSoC very well.

I advise you to download the last SDSoC platform release from here. You will obtain a zip file that contains the SDSoC platform already build. After that, you can follow these steps to create your first project.

Share this post


Link to post
Share on other sites
  • 0

Hi @bogdan.deac,

I followed your instructions but I found a error message while booting from the sd card the reVision platform that says the following:

 


U-Boot 2017.01 (Aug 05 2018 - 22:17:14 -0700)

Model: Zynq Zybo Z7 Development Board
Board: Xilinx Zynq
I2C:   ready
DRAM:  ECC disabled 1 GiB
MMC:   sdhci@e0100000: 0 (SD)
Using default environment

In:    serial
Out:   serial
Err:   serial
Net:   ZYNQ GEM: e000b000, phyaddr 1, interface rgmii-id
SF: Detected s25fl128s_64k with page size 256 Bytes, erase size 64 KiB, total 16 MiB

Warning: ethernet@e000b000 using MAC address from ROM
eth0: ethernet@e000b000
U-BOOT for Zybo Z7
ethernet@e000b000 Waiting for PHY auto negotiation to complete......... TIMEOUT !
Hit any key to stop autoboot:  0
Device: sdhci@e0100000
Manufacturer ID: 1d
OEM: 4144
Name: SD    
Tran Speed: 50000000
Rd Block Len: 512
SD version 3.0
High Capacity: Yes
Capacity: 58.9 GiB
Bus Width: 4-bit
Erase Group Size: 512 Bytes
reading image.ub
3910868 bytes read in 340 ms (11 MiB/s)
## Loading kernel from FIT Image at 10000000 ...
   Using 'conf@2' configuration
   Verifying Hash Integrity ... OK
   Trying 'kernel@0' kernel subimage
     Description:  Linux Kernel
     Type:         Kernel Image
     Compression:  uncompressed
     Data Start:   0x100000d4
     Data Size:    3878632 Bytes = 3.7 MiB
     Architecture: ARM
     OS:           Linux
     Load Address: 0x00008000
     Entry Point:  0x00008000
     Hash algo:    sha1
     Hash value:   4b23816e227252b7549419997f26b3edbd525a7e
   Verifying Hash Integrity ... sha1+ OK
## Loading fdt from FIT Image at 10000000 ...
   Using 'conf@2' configuration
   Trying 'fdt@0' fdt subimage
     Description:  Flattened Device Tree blob
     Type:         Flat Device Tree
     Compression:  uncompressed
     Data Start:   0x103b30b0
     Data Size:    30941 Bytes = 30.2 KiB
     Architecture: ARM
     Hash algo:    sha1
     Hash value:   6bda90ed3c9361add0bd7bb38aeb560c25288661
   Verifying Hash Integrity ... sha1+ OK
   Booting using the fdt blob at 0x103b30b0
   Loading Kernel Image ... OK
   Loading Device Tree to 07ff5000, end 07fff8dc ... OK

Starting kernel ...

Uncompressing Linux... done, booting the kernel.
Booting Linux on physical CPU 0x0
Linux version 4.9.0-xilinx-v2017.4 (digilent@ubuntu) (gcc version 6.2.1 20161016 (Linaro GCC 6.2-2016.11)) #1 SMP PREEMPT Mon Jul 9 19:13:02 PDT 2018
CPU: ARMv7 Processor [413fc090] revision 0 (ARMv7), cr=18c5387d
CPU: PIPT / VIPT nonaliasing data cache, VIPT aliasing instruction cache
OF: fdt: Machine model: Zynq Zybo Z7 Development Board
bootconsole [earlycon0] enabled
OF: graph: no port node found in /amba_pl/xilinx_drm
EXT4-fs (mmcblk0p2): couldn't mount as ext3 due to feature incompatibilities
INIT: version 2.88 booting
Starting udev
udev: Not using udev cache because of changes detected in the following files:
udev:     /proc/version /proc/cmdline /proc/devices
udev:     lib/udev/rules.d/* etc/udev/rules.d/*
udev: The udev cache will be regenerated. To identify the detected changes,
udev: compare the cached sysconf at   /etc/udev/cache.data
udev: against the current sysconf at  /dev/shm/udev.cache
Populating dev cache
ALSA: Restoring mixer settings...
No state is present for card ZyboZ7SoundCard
Found hardware: "Zybo-Z7-Sound-C" "" "" "" ""
Hardware is initialized using a generic method
/usr/share/alsa/init/default:26: value write error: Input/output error
/usr/share/alsa/init/default:26: value write error: Input/output error
/usr/share/alsa/init/default:263: value write error: Input/output error
/usr/share/alsa/init/default:263: value write error: Input/output error
/usr/share/alsa/init/default:265: value write error: Input/output error

No state is present for card ZyboZ7SoundCard
hwclock: can't open '/dev/misc/rtc': No such file or directory
Tue Jul 10 02:14:48 UTC 2018
hwclock: can't open '/dev/misc/rtc': No such file or directory
Starting internet superserver: inetd.
INIT: Entering runlevel: 5
Configuring network interfaces... udhcpc (v1.24.1) started
Sending discover...
Sending discover...
Sending discover...
No lease, forking to background
done.
Starting system message bus: dbus.
Starting Dropbear SSH server: dropbear.
hwclock: can't open '/dev/misc/rtc': No such file or directory
Starting syslogd/klogd: done
Starting tcf-agent: OK

 

as you can see it is not finding this device and I dont know if this causes not to display any image in the hdmi. I am using filter2d demo, as is suggested for PCAM-5c set-up.

 

Share this post


Link to post
Share on other sites
  • 0

It's an error from the audio driver which we haven't gotten to work in Petalinux.

It does not affect video processing.

Share this post


Link to post
Share on other sites
  • 0
Posted (edited)

The thing is that actually I am not able to watch any image in the monitor, and the port tty1 is still opened. I have tried to check the drivers but no image is displayed, and when initializing the stream it gets blocked in the process.

image.thumb.png.9c009274c3b0a363e2df6ea631e89e0b.png

Edited by Esti.A

Share this post


Link to post
Share on other sites
  • 0

Please answer the following questions:

1. Did you follow these steps?

2. Have you got any error message on one of them?

3. Have you mount /dev/mmcblk0p1? How do you launch the opencv application?

 

Share this post


Link to post
Share on other sites
  • 0

Hi @bogdan.deac,

 

1. Yes, I did follow the steps defined in the link

2. I didn't get any error while the project creation but I get the following when running the .elf file after mounting the mmcblk0p1

image.thumb.png.74ff7d0c51ab78c0cb549afd98df28ba.png

all the video initializing is performed correctly:

image.thumb.png.702002f692381a4fed82099455146c67.png

 

And I also checked the raw test creation :

image.thumb.png.4168937c52b405309c0e19e98c9659df.png

And if I try to run again the app (Demo.elf) it starts running but I get the following message:

image.thumb.png.eb52366501f36257ae21415dfa5a863f.png

 

 

 

 

 

 

image.png

Edited by Esti.A

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now