Examples of using the DDR3 on ARTY S7 board

[Mathias]

Hi,

I have this board and need to use DDR3. To my surprise, there are no examples whatsoever of how to use it. I have tried with MIG a couple of months back but was so disappointed about the complexity that I gave up on my project. Now I got some strength to try again.

Why are there no examples of how to use the DDR3 memory on your site?

Regards,
Mathias

 

[zygot]

I agree with you.

In fact, I think that every externally connected device on Digilent's FPGA boards should have an easy to understand HDL example tutorial. Since every FPGA board that Digilent sells has some sort of external memory that customers want to use this is a glaring omission. In fact I've recently been thinking about the same question ( again... ) and it's on my ToDO list.

My personal opinion is that Xilinx wants to sell the notion that every FPGA design has to have a MicroBlaze and AXI busses to do anything useful. Unfortunately, this approach can use up most of a devices resources. For educational purposes this is shameful. For commercial reasons this is naughty. But no one has to 'drink the lemonade' to use a memorable quote.

The MIg tools is odd, can be unpleasant to work with, hard for beginners to work with, and isn't complete. Fortunately, unlike some other free IP, it provides unencrypted source code and is useful for HDL designs without a soft processor. I do miss the old days when low cost FPGA devices had nice multi-channel external memory controllers. These were higher performance than Series 7 single-channel soft controller IP and in most cases easier to work with. How odd that even the lowly Artix 15T, like virtually every Artix device, has a PCIe hard block and no external memory block. Is this really what Xilinx customers want?

Anyway... stay tuned. I aim to fix this if no one else wants to. Using the capabilities that you pay for shouldn't be so hard to figure out, especially for products geared to the educational market.

Edited September 29, 2021 by zygot

[Mathias]

Hi Zygot and thanks for answering this question. Never before did I have this kind of problem, perhaps only in the FPGA world this can happen.

I simply want to read and write from/to memory. Don't need to be that fast either.

Although I love FPGA and such, I feel that I cannot spend months and months only to make the memory work. Microblaze is nothing I need nor want to use.
I and many others would certainly appreciate any help we can get with this !

[JColvin]

Hi @Mathias,

If you are only wanting simple example that has the DDR memory being tested, you can follow this guide which walks through setting up DDR memory and MIG on the Arty S7 board in Vivado and exporting it into Vitis: https://digilent.com/reference/programmable-logic/guides/getting-started-with-ipi. When choosing an application project in Vitis, there is a premade Memory Test which then does a test write and read back of data to the DDR.

As for a more pragmatic design that doesn't rely on the processor (or one that does) Digilent does not currently have one that I know of, though I will ask for one.

Thanks,
JColvin

[Mathias]

Hi JColvin,

Thanks for the information. Would appreciate any files of any kind that I can use to better understand how to use the dram. Seems vitis needs almost 170GB to install, yikes. The instruction you refer to is about 50 steps to follow, I wish it was more simple, but thanks for the help anyway :)

Regards,
Mathias

[JColvin]

Hi @Mathias

The instructions are long mostly because they are written to be fairly verbose and (ideally) clear.

Vitis should not be that big to install; or at least when I use the self-extracting web installer and only leave board options that I need checked, the final disk installation size of both Vivado and Vitis 2021.1 for me is just shy of 70 GB. Makes me yearn for the less than 15 GB size that 2015.4 was to be sure, but that's neither here nor there.

I asked another co-worker to make a DDR demo and they're hoping to get it completed in the next couple of weeks (in between all of their other tasks and making sure the documentation/tutorial aspect is clean).

Let me know if you have any questions.

Thanks,
JColvin

[zygot]

You don't need need a MicroBlaze, Vitis or the SDK to use DDR in a FPGA application. Digilent has a few demos scattered among the support for various boards that they sell; but they are all board design/ MicroBlaze based. This includes the one that is being offered here.

One problem with creating a comprehensive tutorial involving the MiG is that for different Xilinx families, devices, and memory devices you will not have a consistent experience using the MiG IP. That is settings for some functionality will appear for some combinations of FPGA/Memory devices and not for others. Some options, like 2:1 or 4:1 clock to PHY options will be greyed out for some devices and not others. The official Xilinx documentation for using the MiG is confusing and has errors to add to the confusion. In short, no tutorial that anyone would want to slog through will provide a step by step recipe for creating a DDR design for any particular board. I know this because I've started on such a tutorial and have done HDL (no soft processor) designs for many of the boards sold by Digilent and others.

Just trying to find the correct datasheet for a particular DDR device can be an expedition. There are just too many details to understand.

Providing a project that includes DDR is highly platform dependent. Presenting a tutorial that helps understand how to create a MiG DDR controller from scratch is a lot more complicated due to the reason stated above.

If you want a short, easy to implement, 10 step guide to creating and using a DDR controller for any version of Vivado and any board then I'm sorry to disappoint you... it cant be done; it's either going to be complicated or short on details, not particularly useful to everyone and suitable for a particular platform.

I've just finished creating a MiG controller for the NetFPGA-1G-CML from scratch and connecting it to PCIe and USB 3.0 interfaces for testing in separate projects. While writing to and reading data from DDR in a PC application can be a step, it isn't particularly useful. Usually, designs need to have have at least two channels that can access DDR. Such is the case where one might want to capture ADC samples, or write long DAC samples. Unfortunately, though Xilinx has deemed the hard multi-channel external memory controller to be unnecessary, it hasn't provided the tools with the ability to do multi-channel external memory interface designs, at least for the all HDL design flow. I find this to be curious but not accidental.

Anyway, I aim to post such a tutorial but will be more than pleased to see Digilent do so as well, even if it's specific for one of its boards. Other FPGA board vendors make the effort at providing a demo for their boards that at least do the trivial example... that is providing access to a DDR memory for one channel.

And here lies the problem with the MicroBlaze/board design flow that Digilent and Xilinx are committed to. For smallish devices, after you put all of the extraneous junk into your design there just aren't enough resources left to do what you want to do in your design. And if there isn't any IP to accomplish a design you are stuck. That and of course, you need to use the Xilinx software development tools. And virtually every new release of the tools will break your old designs. This is why this flow is almost unheard of in industry.

[edit] The issues mentioned here are worse than stated. You can't even post a tutorial for a particular FPGA board that Digilent sells because some, like the Arty A7, have variants that uses different speed grades. The 'official' Xilinx documentation for their own IP and tools is not up to date and users have to hunt down every advisory for a particular IP to see what's been officially recognized and addressed, though that doesn't mean fixed.
The best option is to help people understand how to sidestep potholes or work themselves out of them. Personally I think that DDR deserves a special forum spot to itself. If no one else wants to do it then I will. I'm still working out how to present my tutorial, but I'm pretty sure at this point that it won't be a simple to read, one solution for all, tutorial. And that's a good thing if people have problems and post questions about them. The problems that users have implementing DDR is really just a microcosm of the larger FPGA tool/documentation dis-functionality, so understanding how to navigate through DDR implementation is useful for everything else.

Edited October 3, 2021 by zygot

[Mathias]

On 10/2/2021 at 12:02 AM, JColvin said:

Hi @Mathias

The instructions are long mostly because they are written to be fairly verbose and (ideally) clear.

Vitis should not be that big to install; or at least when I use the self-extracting web installer and only leave board options that I need checked, the final disk installation size of both Vivado and Vitis 2021.1 for me is just shy of 70 GB. Makes me yearn for the less than 15 GB size that 2015.4 was to be sure, but that's neither here nor there.

I asked another co-worker to make a DDR demo and they're hoping to get it completed in the next couple of weeks (in between all of their other tasks and making sure the documentation/tutorial aspect is clean).

Let me know if you have any questions.

Thanks,
JColvin

Hi,

What is the status of this? Any demo I can take a look at yet?

In the meantime I have tried to use the MIG to generate code and I think I've got something to work with. I looked at "Arty S7™ FPGA Board Reference Manual" in the "DDR3L Memory section":

https://digilent.com/reference/_media/reference/programmable-logic/arty-s7/arty-s7_rm.pdf

and in xilinx documenation in "DDR3 SDRAM Memory Interface Solution":

https://www.xilinx.com/support/documentation/ip_documentation/ug586_7Series_MIS.pdf

With a lot of effort and after a lot of trail and error I can now make implementation and meeting timing constraints (Worst negative slack: 0.341 ns) for the DRAM only project. If I look at the "device" tab I see that DRAM implementation takes up a lot of space on the fpga already.

 

And the thing is, as soon as I start to use the DRAM I cannot meet the timing constraints any more. Not by a long shot (Worst negative slack: -8.982). I have attached the timing report.

It seems that I am forced to use the 100Mhz clk on the board to the DRAM implementation and I cannot use it for anything else. If I try to, I get "sub optimal" errors and/or placement errors. So for the implementation that is actually using the DRAM part must use another clock. So I use the 12Mhz for that. I think this makes things worse, timing wise...

Any suggestions? An example would of course be best so I can have a look if I have done something wrong.

Regards,
Mathias

timing_report.zip

[zygot]

A sure way to introduce timing issues is by failing to properly handle signals crossing clock domains.

Did you see my DDR tutorial ?
https://forum.digilentinc.com/topic/22197-a-guide-to-using-ddr-in-the-all-hdl-design-flow/

Edited January 22, 2022 by zygot

[Mathias]

8 hours ago, zygot said:

A sure way to introduce timing issues is by failing to properly handle signals crossing clock domains.

Did you see my DDR tutorial ?
https://forum.digilentinc.com/topic/22197-a-guide-to-using-ddr-in-the-all-hdl-design-flow/
 

No, I did not see your tutorial. Thanks a lot for taking the time to create it!
I will check it out, it will help me a lot for sure.