Sign in to follow this  
xc6lx45

busbridge3: High-speed FTDI/FPGA interface

Recommended Posts

Hi,

as I may not have time for FPGA work for a while - just started in a fascinating new role related to high-speed digital diaper changing - I decided to post this now.

Here's the Github repo (MIT-licensed)

  • The project provides a very fast (compared to UART) interface via the ubiquitous FTDI chip to a Xilinx FPGA via JTAG. Most importantly, it achieves 125 us response time (roundtrip latency), which is e.g. 20..1000x faster than a USB sound card.
  • It also reaches significantly higher throughput than a UART, since it is based on the MPSSE mode of the FTDI chip.
  • Finally, it comes with a built-in bitstream uploader, which may be useful on its own.
  • I implemented only the JTAG state transitions that I need but in principle this can be easily copy-/pasted for custom JTAG interfacing.

So what do you get:

  • On the software side an API (C#) that bundles transactions, e.g. scattered reads and writes, executes them in bulk and returns readback data
  • On the RTL side a very basic 32 bit bus-style interface that outputs the write data and accepts readback data, which must be provided in time. See the caveats.
  • In general, a significant increase in complexity over a UART. The performance comes at a price. In other words, if a UART will do the job for you, DO NOT use this project.

For more info, please see the repo's readme file.

For CMOD A7-35, it should build right out-of-the-box. For smaller FPGAs, comment out the block ram and memory test routines, or reduce the memory size in top.v and Program.cs.

I hope this is useful.

When I talked to the FTDI guys at Electronica last week I did not get the impression that USB 3.0 will make FT2232H obsolete any time soon  for FPGA: They have newer chips and modules but it didn't seem nearly as convenient, e.g. the modules are large and require high density connectors. In FPGA-land, I think USB 2.0 is going to stay...

Cheers

Markus

Share this post


Link to post
Share on other sites

Thanks for the scoop before you move on to slogging poop... First glance looks nice and definitely worth anyone's time to check it out with the aroma of solder wafting in the air. I have high expectation and intend to, just for fun, verify your latency claims whatever they mean. It's very heartening to see worthwhile contributions that can motivate a desire to learn.

For performance I'll stick with Cypress USB solutions... even the old ALTYS FX2 design keeps on giving...

THANKS, 

Share this post


Link to post
Share on other sites

Thanks - if you manage to get it running, the latency number should be printed to the console from this part in Program.cs:

sw2.Reset(); sw2.Start();
int nRep = 1000;
m.memTest32(memSize: 1,baseAddr: 0x87654321,nIter: nRep);
Console.WriteLine("roundtrip time "+((double)sw2.ElapsedMilliseconds/(double)nRep)+" ms (should be close to 0.125 ms from 8 kHz USB 2.0 microframe rate)");

Throughput is another matter - I didn't profile it today but if I recall correctly, it reaches 20+ MBit/s out of the box. There is an FTDI whitepaper on optimal buffer settings, there may be some opportunity for tweaking at FTDI driver level and in the device manager.

Starting from Visual Studio disables some of the JIT optimizations (I think). For profiling, run the .exe straight from windows explorer.

There was a missing code line in bb3_lvl2_io.cs that shows only in low-level JTAG read access, e.g. if I use it as plain JTAG interface to connect straight to BSCANE2, or something like that. It's fixed now.

Share this post


Link to post
Share on other sites

@xc6lx45,

After second, more through look, my reaction has gone to really really nice! Excellent work! For anyone wanting a one step PC application to FPGA application this is a fantastic tutorial.

[edited] And by the way best wishes in your new endeavour. When you need a change of pace stop by and drop off more of worthwhile your insight. We will all be poorer off without them.

Edited by zygot

Share this post


Link to post
Share on other sites

@xc6lx45

Hi,

I had no issues creating a bitstream in Vivado 2018.2 using your (imported) project file.

While trying to recreate your project I ran into a snag with the C# code. I have VS2010 tools so of course your project files and solutions are unusable. I did manage to use the sharpDevelop tool you suggested. The executable configured the board and then threw an unhandled exception... (Arithmetic operation resulted in an overflow)  but still terminated gracefully. I'm afraid that my C# skills have become rusty.

I had to modify the path for the top.bit since I was running out of a different directory than you did ( I had to create a new solution project); simple enough to resolve.

Let me make a few suggestions for such projects:

  • Identify exactly what tools (versions) you use to create components
  • Since Microsoft tools are notorious for not playing nice with other versions of itself you might want to anticipate that most users will have to do a work-around; not a complaint, just a thought. Assume that you audience might have a different development trajectory, especially for PC software, than you did when developing the project.

I got too close to quit but so far haven't accomplished a verification that your project can be recreated. I know that you are interested an any feedback, as I am for any projects that I've posted.

regards

Edited by zygot

Share this post


Link to post
Share on other sites

Thanks. Please check gitHub again, I added a fourth folder "sharpDevelop_build" (5.1 from sourceforge).
This should run out of the box, if a CMOD A7 35 is found, and no other Digilent USB interface is present.

If more debugging is needed, my exception handler might get into the way. I'd comment out the try/catch. Alternatively, enable "pause on exception", as shown in the screenshot here: http://community.sharpdevelop.net/blogs/siegfried_pammer/archive/2014/08/27/customizing-quot-pause-on-handled-exceptions-quot-in-sharpdevelop-5.aspx

When porting to sharpDevelop I ran into an issue with type casting and bit shifting: Running the older code on sharpDevelop will cause an overflow error. Don't know why, probably different interpretations (or versions?) of the standard. Whatever, it is fixed now by stating the problematic cast explicitly.

To get rid of the relative link to top.bit (it's ugly), change this line in Program.cs
bitstreamFile = @"..\..\..\busBridge3_RTL\busBridge3_RTL.runs\impl_1\top.bit"; Console.WriteLine("DEBUG: Trying to open bitstream from "+bitstreamFile);
e.g. to
bitstreamFile = "top.bit";
and copy top.bit into the folder where the .exe file is generated.

Share this post


Link to post
Share on other sites
2 hours ago, xc6lx45 said:

I'd comment out the try/catch.

I created an executable using your sharpDevelop_build directory but got the exact same results as the project that I created. Commenting out the try/catch lines just created an exception in the memif without the graceful termination. I'll try and debug but it may take awhile. I'm running in WIN7 (far far away from the internet )

Edited by zygot

Share this post


Link to post
Share on other sites

There's a problem with your memTest32 function... haven't found out where the overflow is but possibly a cast issue ?

Share this post


Link to post
Share on other sites

Sorry but it works on both my Windows 8 and Windows 7 machine (both 64 bit, though).

Does one of the changed lines from this commit look familiar? https://github.com/mnentwig/busbridge3/commit/887d1d3984dbf69ab5a7b72398d90ddde98ab0a9

If so, could you please try a clean checkout? Otherwise, the error message would be helpful (memTest is almost "toplevel", everything happens in there).

Share this post


Link to post
Share on other sites

The committed lines seem to be there. If I uncomment the memTest32 calls and comment out the try/catch I get the following:

System.OverflowException: Arithmetic operation resulted in an overflow.
   at busbridge3.memIf_cl.getUInt32(Int32 offset) in c:\Projects\busbridge3-master\busBridge3\bb3_lvl4_memIf.cs:line 362
   at busbridge3.memIf_cl.getUInt32(Int32 offset, Int32 num) in c:\Projects\busbridge3-master\busBridge3\bb3_lvl4_memIf.cs:line 373
   at busbridge3.memIf_cl.memTest32(Int32 memSize, UInt32 baseAddr, Int32 nIter) in c:\Projects\busbridge3-master\busBridge3\bb3_lvl4_memIf.cs:line 505
   at Program.Main2(String[] args) in c:\Projects\busbridge3-master\busmasterSw\Program.cs:line 162
   at Program.Main(String[] args) in c:\Projects\busbridge3-master\busmasterSw\Program.cs:line 8

I'm running WIn7 64-bit as well.

Edited by zygot

Share this post


Link to post
Share on other sites
10 hours ago, zygot said:

   at busbridge3.memIf_cl.getUInt32(Int32 offset) in c:\Projects\busbridge3-master\busBridge3\bb3_lvl4_memIf.cs:line 362

OK that is exactly where I made the change for sharpDevelop.

Could you please double-check that the line reads
retVal |= (UInt32)this.jtag.io.readData[offset++] << 24;
in the old version it was (note the brackets!)
retVal |= (UInt32)(this.jtag.io.readData[offset++] << 24);

The red one works in VS but not sharpDevelop.

Share this post


Link to post
Share on other sites

The "problem" with sharpDevelop was one of default settings. It had arithmetic overflow protection enabled, whereas my Visual Studio project disables it in the "advanced" options.

================================================================================

I added a 2nd, simplified example, on github. It runs independently in the same design, on USER1- and USER2 JTAG opcodes.

The new variant is "almost" like a UART, except that the data stream is continuous (e.g. may design the application FPGA code to send zero bytes if there is no return data)

Share this post


Link to post
Share on other sites
8 hours ago, xc6lx45 said:

retVal |= (UInt32)this.jtag.io.readData[offset++] << 24;

Yeah, my eyesight isn't that great to spot the parens on my own... the source for my last build (what was the latest source from GIT) was not the updated version. The fixed lines stopped throwing exceptions after I put the rest of the code back to yours to run everything.

I did download your latest this morning and installed it into a fresh directory. Built the bitstream and C# application. I don't get any error but am not sure what I am looking at. Your USER2_demo runs twice but doesn't provide any notification of success... so I guess that it passes. The application repeatedly prints out the lines:

  • configured test register delay...
  • round trip time...
  • margin 1:
  • margin 2:

the reported round trip times seem to be around 0.260 ms if the CMOD is plugged into a hub and around 0.06ms if I plug the CMOD directly into a PC USB port. The one LED does blink at a 1 sec interval.

I never do get to the "All tests pass..." declaration that I'm expecting from the program.cs source.

For what it's worth, last night as I was playing around with the SharpDevelop debugger it appeared that the second call to  getUInt32 at line 505 in bb3_lvl4_memif.cs was where the exceptions were happening.

I'm going to try an see if I can make the CMOD Verilog do something a bit more interesting than blink the LED...

Sorry for the troubles...

 

Share this post


Link to post
Share on other sites

>> I never do get to the "All tests pass..." declaration that I'm expecting from the program.cs source.

True. It's edited to run forever (counts up to minus 1 => the loop never terminates). Once you see repetitions in the console, it has passed all tests once.

I sometimes leave it running overnight but haven't yet witnessed any random bit errors.

With the USER2 "demo", it's just the same: It checks correctness of the received data - as long as there are no exceptions, it passes.
This could be used as a full-duplex throughput test, since there is no protocol overhead.

 

Share this post


Link to post
Share on other sites

@xc6lx45, Well, thanks for the support. Again, it's a fantastic tutorial and highly usable. As I  mentioned I intend to massage it a bit and perhaps add my own twist to the interface for a future project.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this