FMC & SFP+ link (Aurora) on Nexys Video

[Bert_ALSE]

Hi,

We own a Nexys Video board on which which try to hook an optical coupler (3.125G) to perform some tests using Aurora 8b/10b.

We use a 3rd party FMC adapter. I think we've been careful to correctly set Vadj (to 1.8V), to adjust the adapter's PLL to 125MHz, etc.
We even set a 1s delay between setting Vadj mux and starting the regulator (Vadj_enable).
As typical, we lost JTag (this is known, and an overlook of the FMC adapter), but that's not my point here.

The issue is we have lots of pain to get a LinkUp, and when we get it, the data integrity is not good.

The documentation is very scarce, it mentions this link as "gigabit" (why ?).

So anyone who has used the FMC connector and in particular the transceiver is very welcome to share his experience.
We also are looking for an Artix 7 board with an SFP+ connector, or a working LPC FMC SFP+ adapter but we failed to locate anything affordable/available.

Thanks in advance,

Bert

[zygot]

For your needs there's absolutely nothing in the Nexys Video implementation of the FMC connector or FMC transceivers preventing success. I prefer the Genesys2 method of setting Vadj but a workable solution isn't too hard for the IO controlled one on the Nexys Video. Confidence in of those statements leaves me questioning your adaptor or HDL implementation. I suppose that you don't have very good support from the adapter vendor to verify basic communications via loopback. Delays in the optical interface will complicate timing and training the interface. I suggest introducing instrumentation into your HDL source to help understand where the issues are.  I suspect that you will have to do some self-education to make progress. That isn't a bad thing unless you have a overly optimistic deadline or near-sighted manager making decisions for you.

It's unfortunate but the FMC mezzanine universe is not kind to the financially challenged. I think that you are on the right track in looking for an FPGA board with an SFP type connector if time is the chief concern. In general, I don't know of any FMC mezzanine vendors that provide acceptable levels of support regardless of the price point. Are there any FMC development boards from optical IC vendors?

Opal Kelly has the Syzygy crowd funded project that has a transceiver interface it claims works with via fiber. I haven't tested it. While the main board has very nice interfaces I'm not convinced about the choices in other areas of the overall system design. I've been meaning to work on it for a while now. Normal support for the project is pretty much non-existent so far. Xilinx does put SFP+ connectors on a lot of their development boards and the price isn't so bad if the free Vivado license can help justify the cost. In fact, if I need to refresh my full Xilinx tools to a newer version I generally do this by buying a Xilinx development board. I'm stuck with a particular part for generating bitstreams but this isn't usually a big issue in the larger context.

[Bert_ALSE]

More precisely my question was about anyone having used this transceiver, through FMC, on this board.

Using optical cables simply means you zero the SI factor which kicks in with DAC cables (at higher speeds).

When we plug to our VC707 kit, we have no problem. We're well past debugging our application (which is essentially Xilinx' Aurora application).

We have a problem with this [Board / FPGA / FMC Adapter] combination and I try to figure out at which tree to bark.
The FMC adapter works at 10G no problem so it's probably out of the loop.

Having lost JTag just makes debugging a little bit more complicated (we use our GB Ethernet IP).

I looked at the Opal Kelly future board but I don't see the interface we need.
Any other suggestions ?

BR

[zygot]

So, let's start at the simple questions since you clearly have some experience with the transceivers.

Does the Aurora interface work using loopback in the FPGA?

I imagine that selecting the Artix family might pose some challenges to any designs that you already have under you belt using a higher end part. The external clock source may also be an issue.

I believe that I read in a post that the engineer who implemented the FMC connectors for Digilent's boards implemented an Aurora interface as part of the testing @elodg may have some useful help in this regard.

 

[zygot]

1 hour ago, Bert_ALSE said:

I looked at the Opal Kelly future board but I don't see the interface we need.

One of the PODs that they sell for the HUB  is a dual SFP board. There's no support so far so you'll have to find a supportive ear somewhere in that organization if you want to pursue that as a possibility. I do suspect that it might not be worth your while. Opal Kelly has posted data claiming to prove that the SFP board works.. but I seem to remember that it's at a lower data rate.

[zygot]

1 hour ago, Bert_ALSE said:

When we plug to our VC707 kit, we have no problem

After moving on to other things I was interrupted by having remembered this comment.

I assume that your knowledge that different device families have different transceiver types (GTP, GTH, etc) has been worked out as part of your analysis.

In general vendors of Artix development boards don't use the highest speed grades as they are trying to make money. Xilinx kits tend to be different as they want to sell silicon.

[zygot]

1 hour ago, Bert_ALSE said:

We're well past debugging our application

Gee wish I got a buck for every time I heard that...

[elodg]

@Bert_ALSE, to work around the JTAG issue on the FMC adapter, you can try either
shunting FMC_TDI to FMC_TDO on the FMC adapter OR
interrupting the PRSNT_M2C_L connection to GND by pulling the pin from the FMC connector, for example. It is not for the faint-hearted, in any case.

You mention Aurora and gigabit, so I assume you wish to use the GTP transceivers in the Artix-7 on the the Nexys Video. VADJ has absolutely no effect on the GTP transceivers. Just make sure you set it appropriately for any other control pins or interfaces using regular user I/O (non-GT) pins.
You also mention VC707, which has GTX (GTZ?) transceivers. The data rates supported are obviously different, but there might be other functional differences as well. Make sure all the pre-emphasis and voltage swing settings are correct.

DPx and GBTCLK pins go directly to the FPGA. The PCB traces are length-matched and designed for 100-ohm characteristic impedance. Via stub elimination was not considered necessary. The GT power supplies are within spec. The interface was validated using IBERT with an FMC loopback board at the maximum data rate supported by the FPGA.

[zygot]

Perhaps this post should also go to the Sales or Suggestions area but it certainly is germane to this thread.

I realize that Digilent's FPGA boards are not meant for commercial products. I also realize that, and understand why, Digilent might be loath to provide performance or test data for it's FMC connectors and transceivers where they exist. 

Since Digilent does its due diligence in testing these things it seems to me that making this information available to those considering purchasing a particular board for a particular purpose would be in the best interest of all involved. You're going to end up providing it anyway sooner or later though with perhaps some angst or unhappiness on the part of a customer who has invested time and money; time being the most costly investment. When a company can do something that's perceived as being honest, having the customer's interests as a priority, and generally just being a good partner then why wouldn't it want to do that. I don't see a downside to being considerate here.   

[Bert_ALSE]

On 7/5/2018 at 2:12 PM, elodg said:

@Bert_ALSE ...

Thanks a pile @elodg for the very detailed and pertinent answer ! Exactly what I needed ?

It shows your board has been carefully designed, with good SI and verification (eye diagram at 3.7G is comfortable).
Our experience is that things get nasty above 8G, 5G isn't that difficult,. Our Stratix V board with 100G connectivity wasn't easy (we ended up using Megtron6).

As a result of your post, we have insisted, and we now end up with a fully operational 3.125G bi-directional link between Nexys running the Xilinx Aurora IP and an Arria10 board using our Aurora IP (so far we had not been able to find a suitable Artix 7 board to test). We use the FMC adapter from our friends at ReflexCES, and they overlooked the JTAG issue. Since we now have a way to work around the missing JTag, I didn't launch into scavenging the FMC connector, but it's definitely in my plans ? 
However, for your future boards I think a jumper for FMC JTag would be most welcome...

There still is an issue about reliability of getting a Link Up following a reset on Xilinx side (works ~50%), but that's not our code on this side so we care only moderately (this issue often shows up in the various forums). btw : we need the 1.8V Vadj because we use the PLL located on the FMC adapter that we indeed program & configure from the Artix7. It means there is a sequence and some delays to respect after power up (or reset) : shut down the V converter, select 1.8V, wait a bit, turn on, wait, program the external PLL etc. It might have an influence. We have not resolved this reset link up reliability issue, but we've exhausted our time budget for this mission, and we can live with this (the link is 100% reliable when UP after reset).

At this stage, our mission is accomplished : we have an Artix7 platform (and a cheap one at that !) which allows us to test our various Aurora IP ports (only in 1 lane at 3.125, but the other configurations are well tested on many other platforms). It gives us another checkmark in our tested interoperability matrix.

It also demonstrates that your board is well designed and usable (at least) at 3.125G through the LPC FMC connector.

Thanks again for your help,

Bert