Jump to content
  • 0

How to send Video stream using USB protocol?


core2explore

Question

Hello Everyone,

We have develop simple imaging system based on Nexys Video development board. We have interfaced Sensor with VGA/QVGA resolution using PMoD ports which provides Monochrome 14-bit data and we are sending this data on Ethernet using UDP packets on PC. We have designed our custom UDP protocol for sending data. 

Currently we are having Nexys Video board but as per data-sheet this USB port is used for storing programming file. 

Now we would like start development of similiar imaging system with USB output port. We are new to USB development on FPGA. Can you please share us how to proceed.? How to get started with USB design in Verilog? Is this possible on this board?

 

 

 

 

Link to comment
Share on other sites

6 answers to this question

Recommended Posts

  • 0
1 hour ago, core2explore said:

How to get started with USB design in Verilog? I

If you want to use USB as a data transport the place to start is with the specifications. There are some interfaces where the specifications are unavailable unless you pay to become a developer. Fortunately, for Ethernet and USB 2.0 and USB 3.0 at least this is not the case. If you don't have a basic understanding of how the USB protocol and physical layers work you are in for some tough sledding.

Fortunately, the Nexys Video is a pretty good platform for experimenting with USB. It has an FT2232H that can use FIFO mode and an FMC connector that accepts USB 3.0 development mezzanine boards form Cypress or FTDI. I've used all of those options on the Nexys Video. Digilent has a Adept library for using the FT2232H in what they refer to as DPTI mode. I've seen up to 30 MB/s on it ( I modified the eeprom to do synchronous 245 FIFO mode and do NOT recommend this; read all of the pleas for help on this web site from those who did this foolishly ). With USB the 'up to' is always a required modifier when talking about data rates. There are a lot of factors that affect sustained or average data rates in the software layer of USB before you even consider OS kernel behavior. You don't have to use Adept to use the Nexys Video or Genesys2 FT2232H, though I'd advise starting with that; you can write PC applications using FTDI libraries. You will need elastic storage, usually provided by FIFOs, if you want to guarantee some sort of baseline sustained data rate,

One tip when working with USB is to avoid sending partial data frames.

Understand that streaming data is different than transmitting data.

Cypress and FTDI both have application notes that are a good adjunct to the specifications. Both offer inexpensive USB 3.0 evaluation kits that work on the Nexys Video using the LPC FMC connector. The Cypress FX3 uses an embedded ARM processor so it is a lot more flexible, and a lot more complicated to develop with. FTDI tends to be a little less helpful than I've wanted it to be when running into problems.

1 hour ago, core2explore said:

Is this possible on this board?

Yes. a better question would be 'Is this likely to be a good approach?' That answer is a bit harder to come by. It depends on how much time and effort you are willing to put into it. You can find all of the relevant Digilent demo projects... if you you come across one let me know. If taking the time to read and grasp the basics of the USB specification seems like a big ask, then the likelihood of success is low.  Beyond the specifications and software behavior it takes a lot of experience to understand. I've done quite a bit of experimentation along these lines with no object other than to get a sense of what might work.

Edited by zygot
Link to comment
Share on other sites

  • 0

I'd say the biggest problem with using USB for these designs is the fact that USB is a shared bus which gives no guarantees regarding bandwidth, and it can vary wildly depending on how exactly your device is connected to the root port, and what else is connected to the same logical bus. 

Typically this is not a problem for video streaming applications as even if some pixel data is lost in transit, it's not a problem because human eye is unlikely to notice few odd-colored pixels within a single frame which is shown for just a fraction of a second, but it might very well be enough for the neural network or CV algorithm to trip over. Also this will pretty much require implementing some sort of port controller yourself from the ULPI level and upwards, because I'm not aware of any COTS solution that supports isochronous endpoints.

That said, I have personal experience with achieving over 300 MBytes/s of sustained transfer rate over USB 3.0 using FT601 USB3-FIFO bridge IC on a custom PCB, so it's definitely possible in right circumstances.

Edited by asmi
Link to comment
Share on other sites

  • 0
17 hours ago, asmi said:

I have personal experience with achieving over 300 MBytes/s of sustained transfer rate over USB 3.0 using FT601 USB3-FIFO bridge

So have I. I've also had experiences with the same device giving me about 10 KB/s. The difference is due to a number of factors. My point is that assuming that the best measured data rates are typical or guaranteed for any application is a bad idea. This is due to the protocol, the drivers, the OS, the software application, etc. A great way to kill your data rate is having to require packet termination. USB isn't overly complex but I've run into more than my share of surprises since I've been developing with it.

Link to comment
Share on other sites

  • 0
On 12/22/2020 at 11:44 AM, asmi said:

Typically this is not a problem for video streaming applications as even if some pixel data is lost in transit, it's not a problem because human eye is unlikely to notice few odd-colored pixels within a single frame which is shown for just a fraction of a second, but it might very well be enough for the neural network or CV algorithm to trip over. Also this will pretty much require implementing some sort of port controller yourself from the ULPI level and upwards, because I'm not aware of any COTS solution that supports isochronous endpoints.

^^It likely will even have less of an impact on neural networks, well designed networks are trained with noise added/removed during development. Also the number of input layers for analyzing images generally reduce image quality even further. I wouldn't worry about a little bit of missing information. Generally what is going to trip an algorithm up is latency of information. 

I am curious how you sustain 200 MBps transfer rate over USB 3.0. What application level protocols are you using?

Link to comment
Share on other sites

  • 0
On 10/13/2022 at 5:44 PM, EMV said:

^^It likely will even have less of an impact on neural networks, well designed networks are trained with noise added/removed during development. Also the number of input layers for analyzing images generally reduce image quality even further. I wouldn't worry about a little bit of missing information. Generally what is going to trip an algorithm up is latency of information. 

I  don't have much experience with CNNs, but I do know that some CV algorightms definitely trip over spurious noise, for example edge detection will most likely fail. How much it is a problem depends on your application though, that's why I said that it might be a problem (but maybe not).

On 10/13/2022 at 5:44 PM, EMV said:

I am curious how you sustain 200 MBps transfer rate over USB 3.0. What application level protocols are you using?

I used FTDI's F3xx driver and API library which is freely available to user's of FT60x bridges, I did not dive deep into technical details, but they use one or several of bulk endpoints at the USB level to transfer the data, which guarantee delivery but don't guarantee bandwidth, and actual achieved bandwidth heavily depends on specific circumstances (like if device is directly connected to a root hub port, or via hub, if there are any other active devices on the same bus). So this works great if you are in a position to provide the best case scenario (direct connection, no other devices on a bus), but can cause quite a bit of headaches for general purpose use.

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...