Genesys 2 10GbE Availability

[dlgeng]

Hi,

I'm trying to build an FPGA cluster for vector processing. I'm interested in the Genesys 2 board. According to the specification, it should be able to provide 10GbE access via the FMC interface by, .for example, adding another fibre transceiver board. So that I can run 10GbE for the inter-board communication. 

However, apart from the theory, has anyone actually succeeded in getting a 10G link from the Genesys 2 FMC connector? or put it another word, has the transceiver channels in the FMC been tested to achieve the 10G performance? Please feel no offence, just there are list of reasons that a 10G link could fail and for many times I saw the high-end specifications came with small footprints to limit its availability. After all I do not want to end up with a pile of boards going no where.

Many thanks,

David

 

[dlgeng]

@sbobrowicz

Thanks for the information. The NetFPGA SUME board is particularly interesting.
 

[dlgeng]

@jpeyron thanks for the 3D data, it's just in time for me to integrate multiple boards into a chassis. 

Can I briefly say the 10GbE fiber interface from the FMC connector works. The IBERT runs over 10 min without logging any error from the fiber channels (SFP+) in the Vivado Serial I/O Links panel. Also I put two SATA connectors for cheap internal connection, those SATA connections also run stable under 10GbE until the SATA cable been touched. So the fiber is a lot more reliable and I'll probably run half of the speed for the SATA based internal connection.  

 

[jpeyron]

Hi @dlgeng,

Sorry for the delay. I have attached the 3D cad model for the Genesys 2. We will also be getting the 3D cad model up on the Genesys 2 wiki page next week as well.

thank you for your patience,

Jon

Genesys2.zip

[jpeyron]

HI @dlgeng,

I just heard from our co-worker that has been working on the 3D CAD model for the Genesys 2. There had been a hold up with the silkscreen but we should have the 3D CAD model done Friday. Sorry for the delay.

thank you,

Jon

[dlgeng]

@Biancaan early version of the STEP file would be very useful, as I'm designing a non-standard FMC interface board, I can put them together before getting the real PCB. regards

[Bianca]

Hi @dlgeng,

I can do that but since we want a standardization of all our step files and publish them on the wiki, we will send you a CAD model of the board and publish it afterwards on the reference page. If you really are in a hurry I will PM you with a step exported from Altium but I guess that in one or two days the other one will be ready.

Best regards,

Bianca

[dlgeng]

@jpeyron There is probably no need to bother your mechanical engineer. Try Garrett A, Open the PCB file, from the Altium Designer, File->Export->STEP 3D, the job done. regards

[jpeyron]

Hi @dlgeng,

Thank you for clarifying about the DP connection. We have reached out to our mechanical engineer that makes the 3D CAD Models for us. It will be a few days for us to get it to you. When we do we will post the step file on this thread.

[dlgeng]

Hi, Jon,

I did consider the NetFPA-SUME and have contacted the netfpga team. With the academic discount, as Garrett suggested, the performance to cost ratios of the two boards are quite comparable. However, the idea of my FPGA cluster is to have light nodes, so that at later stage I can install hundreds of them, and each node can have a simple firmware. For example, I'll only need to handle a 32bit DDR interface instead of two 64bit DDR plus QDR as with the NetFPGA SUME board.

The question about the DP connection was actually trying to split a DP cable and solder it directly to a SFP+ connector for a quick 10G test. However, since that is not going to be my end solution, I'll skip it for the moment. I'll just route a simple FMC adaptor PCB with a few SFP+ connections to test the 10G performance.

BTW, do you think there is chance to get a copy of the 3D shape of the Genesys 2 board? e.g. a STEP file. So that I can integrate it to the CAD system when designing a chassis to hold many of these boards. 

Regards,

David

[jpeyron]

Hi @dlgeng,

The gerber file for the Genesys 2 board is not public. I am not quite sure what you mean by DP connectors? Are you wanting spacing between different pins on the FMC? Is your question suggesting trying to connect a SFP+ directly to the FMC?

thank you,

Jon

[jpeyron]

Hi @dlgeng,

I also reached out to our senior layout engineer and their response was as follows:

As Elod has said, the Genesys 2 DP lanes have been validated to 5Gbps, but should run much faster than that considering the layout. Unfortunately, we cannot guarantee any mezzanine card we did not design.  You will have to research that for yourself. However, in light of what you are asking for, I highly recommend the NetFPGA-SUME.  It has four 12.5Gps SFP+ connectors, and the FMC high speed pairs have been tested up to 10Gbps.  Which would seem to be an exact fit for your needs.  I realize the SUME is quite a bit more money, but if you can get the academic price or in with the NetFPGA.org team, it is much more comparable. 

Cheers,

Garrett A

 

 

 

[dlgeng]

Another thought. As I was checking the schematics for the GTX to FMC connections, I realized there were also four lanes to each DP connector. Also, since DP 1.3 allows each lane up to 8Gbps, any chance I could just hook a SFP+ transceiver to the DP port and use 1 lane for a 10GbE interface?

PS, is the gerber file for the Genesys 2 board available to the public? just want to know the physical space of the two DP connectors, especially when I haven't got the board yet.

[zygot]

@sbobrowicz

I didn't realize that my comments were "side chatter", off topic or even a waste of @dlgeng's time. I'm particularly disappointed in your reaction as I'd hoped that someone from Digilent would address the possibility of creating affordable FMC mezzanine boards when a potential customer mentions a need. I'd purchase an affordable GbE mezzanine card for my Genesys2 if you made one.

 

[sbobrowicz]

@zygot  valid discussion, but move it to another thread in the General Discussion section of this forum. dlgeng has a good technical question and shouldn't be forced to sift through all this side chatter to get to relevant information. This is a warning for this and future posts in the technical forum section. 

@dlgeng Here are some boards from some other companies, but they are pricey or require direct quotes:

http://www.hitechglobal.com/FMCModules/FMC_4SFP+_Module.htm

http://www.hitechglobal.com/FMCModules/x10SFP+_FMC_Module.htm

http://www.vadatech.com/product.php?product=292&catid_prev=0&catid_now=0

http://www.fastertechnology.com/products/fmc/fm-s14.html

I would recommend contacting companies directly and asking if they will provide you with an academic discount based on your work. Another option you could consider is contacting the Xilinx University Program, they might sponsor you and help you get some better pricing. 

You should also try reaching out to the NetFPGA group and seeing if they will sponsor your work. They provide preferential pricing on the SUME to projects that they feel contribute to their community. More info on this here: http://store.digilentinc.com/netfpga-sume-virtex-7-fpga-development-board/ .

[zygot]

@dlgeng,

This is wild conjecture on my part but I suspect that Digilent isn't the kind of operation to invest much money into in-house engineering. And, I doubt that you will find what you are looking for ( if you do please do tell...).

Might I suggest that, given my previous post you and Digilent have a common interest? The biggest problem would be your time schedule. I would think that a GbE or XAUI LPC mezzanine board with an SPF+ connector would be a market opportunity given the alternatives.

 

[zygot]

@elodg,

Both of your posts have been very helpful and informative. If I were a product vendor using a common high density connector I'd want customers and potential customers to be apprised of level of effort that went into it's design.

Frankly, I'm baffled at a vendor offering 2 not inexpensive FPGA boards sporting an FMC connector and not exploiting what's arguably both boards' best and most valuable feature. I understand the argument that Digilent can't verify every possible FMC mezzanine board but isn't it a bit short-sighted to put all that work into it and then act as if they don't exist? Is money really that tight that purchasing a few carefully selected FMC boards for test and advertising is beyond reason? Given the price point of Digilent products ( ignoring the occasional high-end "Net" FPGA board ) the overall capability of the Nexys Video and Genesys2, and the likely budget of their customers there really aren't that many FMC boards out there to choose from. For the most part FMC mezzanine boards are in a far more rarefied universe cost-wise than most Digilent customers likely live in.

 

 

[dlgeng]

Hi, thank you all for the inputs. Let's do not worry too much about the standards. Given the cost for an academic user, I think I'll just buy one and give it a test. If it works at 6Gbps which means there is no vital fault in the board, it has good chance to run at 10Gbps. BTW, anyone knows where to get a decent (in terms of price) FMC to SFP+ adaptor? I searched the internet but the solution I could find is more expensive than the Genesys2 board. I prefer not to build one myself. It does not only take time, also there is risk in new hardware. Regards.

[elodg]

@zygot,

Standards that specify layers from mechanical all the way up to protocol like HDMI, USB, SATA are strict and do not leave much wiggle room. VITA.57 (FMC) is an interconnect standard that makes little effort to specify anything other than the mechanical layer. The electrical requirements are intentionally open-ended. The protocol layer is not even mentioned. It makes sense, since it is heavily geared towards FPGA. The interconnect needs to stay flexible to cover the wide range of applications (including data rates possible).

Every effort was made for the Genesys 2 to be VITA.57-compliant. Differential impedance is 100ohm and pair mismatch is < 1ps. Unfortunately, @dlgeng's request is outside the scope of the standard and requires a separate validation effort.

Thank you both for your input and we will strive to provide better specifications in the future.

[zygot]

@Digeng,

You question raises an important issue about FMC connectors.

The FMC connector, both the high-pin count and low-pin count versions, are used by the VITA standards committee and are covered by (I believe) the VITA.57 standard. As far as I know there's no law prohibiting use of a connector developed by a standards committee whether HDMI, USB, VITA, SATA or any other connector for a purpose other than the one involving a particular standard. I suspect ( I don't know as I haven't purchased the relevant standard ) that saying that your board meets VITA57 standards implies a level of performance for the transceiver IO. It may be that claiming to meet VITA.57 requires membership in that committee. I do know that some vendors specifically state that their FMC connector is NOT VITA.57 compliant. This seems to be a honourable and  reasonable way to address this. When a vendor simply ignores the relevant standard it leaves its customers in limbo as to the motivation. I'd want to remove any such confusion if I were a vendor. 

I don't know what the topology of your cluster is. I don't know if you use GbE switches and require all the layers of XAUI or GbE. I am pretty well convinced that a bunch of Genesys2 boards might  be candidates for such a project using the mDP connectors. You can look over the Transceiver Bootcamp project in the Project Vault if you're still reading this after my previous disclaimer.

best of luck

[zygot]

elodg,

Seems reasonable. Can I suggest that you make the case to whomever it is that makes such decisions that providing this kind of information up front could enhance Digilent's reputation as a vendor in the market space that it live in?

Even for your more technically "informed" and "sophisticated" customers there's only so much due diligence that one can do with a limited amount of published data.

It seems that "Let the buyer beware" is the central core of US law ( I'm not a lawyer and don't provide legal opinions ) ( and this applies to purchasing products and services as well as selecting public officials...) and there are lot's of vendors who reply on that principal as a business model. Given that many or perhaps most of Digilent's customers are, from a technical  perspective, at a heavy disadvantage in this regard might I suggest that Digilent set itself apart from those kinds of vendors by making a better effort at helping people make this analysis before a purchase?

[zygot]

I forgot to mention that the power supply on the Genesys2 might need some analysis. I mention this as there was a user trying to fill the Genesys2 FPGA with a bit-coin miner design....

For close to the cost of the Genesys2 Xilinx has the KC705. I bought the Genesys2 for its HDMI and specifically for its mDP interfaces which happen to both be quite nice for my interests.

[elodg]

Hello @dlgeng,

The Genesys 2 manufacturing test runs the Aurora protocol @5Gbps on the ten FMC DP lanes through a loopback board. By design the lanes should be able to reach 10Gbps. We do not sell any mezzanine cards that actually use GT lanes, so we have not validated it for specific applications like 10GbE. It may be cynical, but we simply cannot cover all use cases with a development board.

Contact sales to see if they are willing to lend you a board for short-term testing.

[zygot]

Well now...

You ask a perfectly valid, and useful question that should have been asked a long time ago.

It seems to me that asking users to test and qualify a board for a particular application is a bit misguided. It is the responsibility of a product's vendor to do this. The Genesys2 board is both expensive and poorly supported. It's quite cynical to sell stuff that that advertises an interface which should be appropriate for a particular application and let customers find out for themselves after a purchase.

I can only say that I've used the Genesys2 board with a number of FMC mezzanine boards with success; but none of them used the transceivers. From just looking at the schematic Digilent seems to have done a pretty good job with the Genesis2 HPC FMC connector... but did they get the layout right for XAUI or 10GBE? I can't answer that.

Perhaps Digilent will be willing to take back a board purchased by a customer, if it doesn't perform to expectations for an application that should work with an interface like the FMC.