Computer Vision on FPGA

[research student]

Hi All,

I'm starting a project for a class.  I'd like to do a real-time computer vision project on an FPGA.  It seems the Nexys Video Artix-7 FPGA has HDMI IN and OUT and am thinking of purchasing it for my project.

However, I am getting hung-up on some of the details I'm hoping the forum can help a novice out (apologies for the lacking in correct terminology ahead of time):

1. What is being licensed here? In other words, The Video Board has ethernet on it but there is a separate license for the ethernet "IP Core"?  There is a license for the Vivado software (but it's free for the webpack - what are the limitations?  What does device locked mean?)  Are there IP Cores for the HDMI ports that will expire in 180 days too?  What if my research project takes longer then 180 days - how much will this cost me on day 181 to use the SD card slot or a keyboard?

2. Am I correct in assuming that I can do some computer vision (object recognition / classification) on the Nexys Video Board, feeding HDMI live stream in, processing on the FPGA, and viewing HDMI out on monitor simultaneously?  Do I need a special HDMI source for this or will any HDMI signal work (like from a computer or DVD player)?

3. What does one use "MicroBlaze" for?  Is there a license for "MicroBlaze" too?

4. Are there licenses for accessing the RAM or any other peripherals?

5. With Vivado, how long does it take too compile a small project and load it onto the FPGA?  Are there certain ways that are quicker than others?  I've programmed micro controllers and the whole compile/upload to device cycle is pretty quick.

6. If I purchase the Video Board and plan to do research with it (no commercial use), how long, from the day I start, will I have full access to all the developer tools, "IP cores", etc...

Thanks in advance for any assistance or knowledge you can share!

Regards,

RS

 

 

[D@n]

@research student,

I'll do my best to answer your questions:

1. Xilinx offers a free webpack license that you can use to build IP for a Nexys Video board.  I have used it quite successfully on several projects, and have not had a problem with it.  While there are many companies out there that sell IP, there are also open source controllers for both SD cards and keyboards.
2. The hardware should be able to support any HDMI source/sink.  However, be aware that there can be a rather complex negotiation that takes place over HDMI between the source and the sink regarding what formats the sink will accept.  I'm told that there is an example of this negotiation that can be examined within the Linux kernel.  Most users, however, of this board simply force the HDMI output into a format they know the receiver can accept, and so avoid this difficulty.  While the result works, it can be hardware specific.  You might even manage to bypass this negotiation by running it through the FPGA as a pass-through .... I just haven't done it.
3. MicroBlaze is used for ... anything you wish to use a CPU for.  FPGA's are *really* good at parallel processing--doing everything at once.  CPU's are really good at doing one thing at a time.  If you choose to place a soft-core CPU, such as MicroBlaze, into your design, then you can handle complex sequential tasks while using less FPGA logic--or at least that's the idea.  One purpose you might have for such a CPU would be to handle the HDMI negotiation discussed above.  Another might be to handle user modes or interactions within your design.  CPUs are programmed in C, or C++, whereas the rest of the FPGA will be programmed in Verilog or VHDL.  Keep in mind, there are many soft-core CPU's available out there.  MicroBlaze gets all the attention, but there are others such as OpenRISC, RISC-V, or even (my favorite) the ZipCPU.
4. Are there licenses for accessing peripherals?  Not that I am aware of!  You will need to use the Xilinx Memory Interface Generator, though, if you wish to access the SDRAM on the board.  (Yes, I am working on an alternative ... but I wouldn't bet your project that I'd be successful
5. Configuring an FPGA is usually fairly quick.  Loading it into a flash can take a coupe minutes, should you need to do that.  Building the configuration, though, can take longer.  With some of the development cycles I have been working with, the build can take five minutes.  I know of the development cycles of others that have taken fifteen minutes or even a half an hour.  You can also set your parameters to have the build process take all night in order to try to do better, although I personally have never found any reason to do so.  Still, I've been working with the 35T chips, and the Nexys Video uses a 200T chip--a bigger chip, so you'd expect the build to take longer as your design becomes more complex.
6. As I understand Xilinx licenses, the official answer is as long as you would like.  The unofficial answer would be to notice that they just cut off support for their pre-series seven FPGA lines.  Sure, you can still use the tools with them, but they are no longer supported.  There will come a day when a new version of windows or linux makes it impossible to program for those devices.  There's no reason to believe this will not happen for the seven series devices as well, but Vivado is a new platform.  It's reasonable to assume that it's  going to be around for a while.

At any rate, welcome to the community, and welcome to the forum!  I look forward to hearing about how you move forward with your project.

Dan

[Sergiu]

Hello,

There are a few things I want to make clear (with the risk of repeating some of what D@n already said)

First of all there is the development platform a.k.a. the Xilinx Vivado which will be used to create projects with. This comes in several packages out of which the Webpack Edition is free and quite powerful in recent versions. It will allow you to write code, simulate, debug, program the boards and integrate IP Cores. You can download the software from the Xilinx website.

Then there are the IP Cores and other reference material. Some are provided by Xilinx for free (microblaze, MIG, ethernet etc) and some are only available for purchase/trial. Digilent also provides reference material, manuals and demo projects, all for free, available on our wiki or github channel. A good place to start would be the Nexys Video wiki page.

In short, at this level, most of the stuff is free and available. Demo projects like the one for the Nexys Video are created to use as many peripherals as posible. In any case if you run into any trouble just come to the forum.

Sergiu