ZYBO - downloaded program data is wrong

[hwant]

Hi guys, I bought a zybo board and did a simple hello world project to test the functionality, but it didn't work. Here's what I've done:

After exporting hardware and creating sdk projects, I downloaded the bitstream & program into zybo as usual. But the board wouldn't run the program normally(It doesn't terminate and doesn't print helloworld). So I debug the board using xsdb, step by step, and find out that disassembly result is not the same as elf file displayed in SDK.

xsd shows that data at 0x100000 is 0xea020049; however, in the sdk, the data should be 0xea000049, as shown in the second picture. If I keep on stpi, since it'll go to the wrong place, CPU will finally goto infinite loop.

xsdb% connect
tcfchan#0
xsdb% targets
  1  APU
     2  ARM Cortex-A9 MPCore #0 (Running)
     3  ARM Cortex-A9 MPCore #1 (Running)
  4  xc7z010
xsdb% fpga -f "design_1_wrapper_hw_platform_0/design_1_wrapper.bit"
100%    1MB   1.8MB/s  00:01
xsdb% targets 2
xsdb% source "design_1_wrapper_hw_platform_0/ps7_init.tcl"
xsdb% rst; ps7_init; ps7_post_config;
Info: ARM Cortex-A9 MPCore #0 (target 2) Stopped at 0xffffff28 (Suspended)
Info: ARM Cortex-A9 MPCore #1 (target 3) Stopped at 0xffffff34 (Suspended)
xsdb% dow "hello_world/Debug/hello_world.elf"
Downloading Program -- C:/Xilinx/Zybo/project_2/project_2.sdk/hello_world/Debug/hello_world.elf
        section, .text: 0x00100000 - 0x001016eb
        section, .init: 0x001016ec - 0x00101703
        section, .fini: 0x00101704 - 0x0010171b
        section, .rodata: 0x0010171c - 0x00101733
        section, .data: 0x00101738 - 0x00101bab
        section, .eh_frame: 0x00101bac - 0x00101baf
        section, .mmu_tbl: 0x00104000 - 0x00107fff
        section, .init_array: 0x00108000 - 0x00108003
        section, .fini_array: 0x00108004 - 0x00108007
        section, .bss: 0x00108008 - 0x0010802f
        section, .heap: 0x00108030 - 0x0010a02f
        section, .stack: 0x0010a030 - 0x0010d82f
100%    0MB   0.4MB/s  00:00
Setting PC to Program Start Address 0x00100000
Successfully downloaded C:/Xilinx/Zybo/project_2/project_2.sdk/hello_world/Debug/hello_world.elf
xsdb% mrd 0x100000 16
  100000:   EA020049
  100004:   EA040025
  100008:   EA00002B
  10000C:   EA00003B
  100010:   EA000032
  100014:   E320F000
  100018:   EA000000
  10001C:   EA00000F
  100020:   F92DD91F
  100024:   ED3F1FBB
  100028:   ED6D0B20
  10002C:   EEF11A10
  100030:   00001004
  100034:   00001A10
  100038:   FFFF1004
  10003C:   EFF1019E

elf file contents:

Disassembly of section .text:

00100000 <_vector_table>:
  100000:   ea000049    b   10012c <_boot>
  100004:   ea000025    b   1000a0 <Undefined>
  100008:   ea00002b    b   1000bc <SVCHandler>
  10000c:   ea00003b    b   100100 <PrefetchAbortHandler>
  100010:   ea000032    b   1000e0 <DataAbortHandler>
  100014:   e320f000    nop {0}
  100018:   ea000000    b   100020 <IRQHandler>
  10001c:   ea00000f    b   100060 <FIQHandler>

00100020 <IRQHandler>:
  100020:   e92d500f    push    {r0, r1, r2, r3, ip, lr}
  100024:   ed2d0b10    vpush   {d0-d7}
  100028:   ed6d0b20    vpush   {d16-d31}
  10002c:   eef11a10    vmrs    r1, fpscr
  100030:   e52d1004    push    {r1}        ; (str r1, [sp, #-4]!)
  100034:   eef81a10    vmrs    r1, fpexc
  100038:   e52d1004    push    {r1}        ; (str r1, [sp, #-4]!)
  10003c:   eb00019e    bl  1006bc <IRQInterrupt>

So, the problem is, WHY is the DRAM data partially wrong??? I almost doubt DRAM works normally, but I just bought the board a week ago lol.

Edited March 6, 2017 by hirayaku
Typos

[jpeyron]

Hi @hirayaku,

Could you please post a screen shot of the block design in vivado and console messages and code in sdk as well. when you run what selection did you choose i.e. launch on hardware system debug?  what version of vivado? what OS are you using?

cheers,

Jon

[hwant]

Hi @jpeyron!

Since I'm only testing the board, the block design and software are super simple. I'm using Vivado 2016.04 webpack on Windows 10. 

Block design

The code is as in the example "hello_world" project. I only pasted the main function here.

#include <stdio.h>
#include "platform.h"
#include "xil_printf.h"

int main()
{
    init_platform();

    print("Hello World\n\r");

    cleanup_platform();
    return 0;
}

SDK logs after I selected "System Debugger using xxx.elf on local":

14:42:35 INFO	: Disconnected from the channel tcfchan#1.
14:42:36 INFO	: 'targets -set -filter {jtag_cable_name =~ "Digilent Zybo 210279778233A" && level==0} -index 1' command is executed.
14:42:36 INFO	: 'fpga -state' command is executed.
14:42:36 INFO	: Connected to target on host '127.0.0.1' and port '3121'.
14:42:37 INFO	: Jtag cable 'Digilent Zybo 210279778233A' is selected.
14:42:37 INFO	: 'jtag frequency' command is executed.
14:42:37 INFO	: Sourcing of 'C:/Xilinx/Zybo/project_2/project_2.sdk/design_1_wrapper_hw_platform_0/ps7_init.tcl' is done.
14:42:37 INFO	: Context for 'APU' is selected.
14:42:39 INFO	: Hardware design information is loaded from 'C:/Xilinx/Zybo/project_2/project_2.sdk/design_1_wrapper_hw_platform_0/system.hdf'.
14:42:39 INFO	: Context for 'APU' is selected.
14:42:39 INFO	: 'stop' command is executed.
14:42:40 INFO	: 'ps7_init' command is executed.
14:42:40 INFO	: 'ps7_post_config' command is executed.
14:42:40 INFO	: Context for processor 'ps7_cortexa9_0' is selected.
14:42:41 INFO	: Processor reset is completed for 'ps7_cortexa9_0'.
14:42:41 INFO	: Context for processor 'ps7_cortexa9_0' is selected.
14:42:41 INFO	: The application 'C:/Xilinx/Zybo/project_2/project_2.sdk/hello_world/Debug/hello_world.elf' is downloaded to processor 'ps7_cortexa9_0'.
14:42:41 INFO	: ----------------XSDB Script----------------
connect -url tcp:127.0.0.1:3121
source C:/Xilinx/Zybo/project_2/project_2.sdk/design_1_wrapper_hw_platform_0/ps7_init.tcl
targets -set -nocase -filter {name =~"APU*" && jtag_cable_name =~ "Digilent Zybo 210279778233A"} -index 0
loadhw C:/Xilinx/Zybo/project_2/project_2.sdk/design_1_wrapper_hw_platform_0/system.hdf
targets -set -nocase -filter {name =~"APU*" && jtag_cable_name =~ "Digilent Zybo 210279778233A"} -index 0
stop
ps7_init
ps7_post_config
targets -set -nocase -filter {name =~ "ARM*#0" && jtag_cable_name =~ "Digilent Zybo 210279778233A"} -index 0
rst -processor
targets -set -nocase -filter {name =~ "ARM*#0" && jtag_cable_name =~ "Digilent Zybo 210279778233A"} -index 0
dow C:/Xilinx/Zybo/project_2/project_2.sdk/hello_world/Debug/hello_world.elf
----------------End of Script----------------

14:42:41 INFO	: Memory regions updated for context APU
14:42:41 INFO	: Context for processor 'ps7_cortexa9_0' is selected.
14:42:41 INFO	: 'con' command is executed.
14:42:41 INFO	: ----------------XSDB Script (After Launch)----------------
targets -set -nocase -filter {name =~ "ARM*#0" && jtag_cable_name =~ "Digilent Zybo 210279778233A"} -index 0
con
----------------End of Script----------------

14:42:41 INFO	: Launch script is exported to file 'C:\Xilinx\Zybo\project_2\project_2.sdk\.sdk\launch_scripts\xilinx_c-c++_application_(system_debugger)\system_debugger_using_debug_hello_world.elf_on_local.tcl'

 

Edited March 6, 2017 by hirayaku

[jpeyron]

Hi @hirayaku,

I made your project the same in vivado as your block design. Exported the hardware including the bitstream and launched sdk. In sdk,I click on new application. I named it and hit next. Then I clicked on hello world template and finish. I next set up the serial terminal. I used tera term to connect as my serial terminal. It must be set to 115200 for the baud rate. I leave it set at the default settings for everything else. Next you click on program fpga in sdk. Next right click on the application you made and choose run as-> launch on hardware(system debugger). The word hello world should print in the serial terminal. I have attached screen shots of what i am talk about as well as a picture of the zybo. Make sure to have the mode JP5 set as shown. I also took screen shots of what the zybo should look like in the device manager. 

cheers,

Jon

zybo.zip

[hwant]

@jpeyron,

Thank you for your demo. But the problem remains unsettled.

All the settings, including SDK settings, USB drivers and jumpers, are exactly the same as what you post. But my zybo just **doesn't work.** Like what I posted first, I can program FPGA with the bitstream file, after that DONE led would turn on; I can download the program onto the board, but when it comes to executing the program, everything is wrong.

This is the system debugger debugging window in SDK. I set the Debug configuration to be "Stop at program entry" so zybo wouldn't execute a single instruction until I continue. The first instruction at program entry is WRONG: it should jump to 0x0010012c (i.e. _prestart), but it jumps to 0x0098012c. It's because the instruction data at 0x00100000 has several bits flipped, as shown in my first post. BTW, the second instruction is also wrong.

I think that's why I cannot execute hello_world normally: I cannot even enter main function because the data in the memory are all messed up. I don't know what causes that, but my best guess is that it has something to do with the memory controller, that is, I doubt the memory controller or DRAM in my board is broken. Not totally broken, since you can see that 3rd to 8th instructions are right. But the instruction at 0x00100020 is wrong again.

Key point: the data are only **partially**(not totally) wrong. I am wondering what causes that and how to fix that. Your board probably cannot reproduce this because I think it's highly possible to be board-specific.

Edited March 7, 2017 by hirayaku

[jpeyron]

Hi @hirayaku,

I did a compare on the SDK log files between my project and yours.I attached my SDK log below. The major difference I found is at the begining of you log it states

4:42:35 INFO    : Disconnected from the channel tcfchan#1.

and mine does not. Looking into tcf here looks like this is how the SDK communicates to the hardware. I believe that the reason you are having issues is that your project is in the Xilinx folder. I would suggest to use a different place for your project.  Please let me know if this fixes your issues. Also have you tried running any of the projects from our github here?

cheers,

Jon

SDK_log.txt

Edited March 7, 2017 by jpeyron

[hwant]

@jpeyron,

Still, no luck.

I tried but nothing happens, and data in the memory is still wrong. It's impossible that the project folder position has anything to do with the downloaded program data, after all.

I have some clues, however. I tried to lower the frequency of memory: from 533Hz, which is claimed to be supported by Digilent, to 200Hz. What about now? Most data are right, including the first and second instruction at 0x00100000. However, there are still a few wrong instructions, so that I still cannot execute the program completely. No "hello world", for sure.

DDR frequency:

:

Instructions at 0x00100000 (they are right; compare them to my previous posts)

Some instructions are wrong, e.g. 0x00100064:

 

Therefore, I'm almost sure it has something to do with the memory controller. If you know the answer, please directly tell me. Don't tell me about the irrelevant information (e.g. tcf. tcf will mess program data up? Are you kidding me?), because I've spent too much time on this possibly malfunctional board. If you do not know the answer, I request you to ask other Digilent employees. Or I have no choice but to request refund.

[D@n]

@hirayaku,

It looks like you may have changed or adjusted the parameters of the memory controller.  For example, the Zybo runs with a 525MHz clock according to the board support files, not a 200MHz clock or a 533 MHz clock.  Can you please check your memory controller configuration against the one in the board support files here, to check that it matches?

Thanks,

Dan

[hwant]

Hey @D@n,

I tried new configurations, but still it doesn't work.

My memory configurations are in the attachments, the same as in the preset.xml file. Frequency is set to 525MHz. The instruction data is still wrong: same as the first post.

hirayaku

 

 

Edited March 8, 2017 by hirayaku

[D@n]

We're getting closer.  Give me another holler on #digilent-fpga when you're ready to tackle this again,

Dan

[jpeyron]

Hi @hirayaku,

I have not seen this issue before. I have reached out to a more experience engineer about you issue. Hopefully you will get a response soon. I am sorry for the inconvience that this has caused.

thank you,

Jon

[hwant]

Hi @jpeyron,

It has been one day, but... no one is coming to tackle this problem.

hirayaku

[jpeyron]

Hi @hirayaku,

I am sorry that you have not received a response yet. This issue has been escalated. Hopefully they will be able to respond in the near future. 

thank you,

Jon

[sbobrowicz]

Hello,

I agree that your symptoms strongly indicate an issue with DDR3 communication or general system instability. 

There are many settings inside of the preset.xml file that are very important for the ZYBO to operate correctly. I recommend that you start by using the vivado board files (thanks Dan) and run block automation on your processing system block after it has been inserted into the block diagram. This will make all of the important settings match what they need to for the ZYBO. Then you can make any application specific changes that you need. 

The reason I am recommending this is because I see from your screenshots that you are specifying 33.333MHz as your input clock. The ZYBO has a 50MHz input clock (see PS_CLK on the schematic), so basically all internal PLLs are going to be generating clocks that are far from correct. This could result in highly unexpected behavior. There are other settings that are likely to cause you problems further down the road beyond the input clock, which is why I recommend just starting over and using the board files.

You mention we call out the ZYBO as supporting 533MHz. I apologize for this, as it is not correct. The ZYBO only works at 525MHz due to PLL restrictions when using a 50MHz input clock. If you point out where we made this mistake I will get it fixed.

[Spencer]

On 3/16/2017 at 10:02 AM, sbobrowicz said:

Hello,

I agree that your symptoms strongly indicate an issue with DDR3 communication or general system instability. 

There are many settings inside of the preset.xml file that are very important for the ZYBO to operate correctly. I recommend that you start by using the vivado board files (thanks Dan) and run block automation on your processing system block after it has been inserted into the block diagram. This will make all of the important settings match what they need to for the ZYBO. Then you can make any application specific changes that you need. 

The reason I am recommending this is because I see from your screenshots that you are specifying 33.333MHz as your input clock. The ZYBO has a 50MHz input clock (see PS_CLK on the schematic), so basically all internal PLLs are going to be generating clocks that are far from correct. This could result in highly unexpected behavior. There are other settings that are likely to cause you problems further down the road beyond the input clock, which is why I recommend just starting over and using the board files.

You mention we call out the ZYBO as supporting 533MHz. I apologize for this, as it is not correct. The ZYBO only works at 525MHz due to PLL restrictions when using a 50MHz input clock. If you point out where we made this mistake I will get it fixed.

Hi, all.

I have exactly same problem that hirayaku had. This problem is so painful.

Is this problem solved and if someone can, pls summarize that solution ?

Thanks in advance.

 

[JColvin]

Hi @Spencer,

This is an old thread, but could you let me know the following details:

- Which Zybo board are you using? The original Zybo, the Zybo Z7-10, the Zybo Z7-20?
- What version of Vivado and SDK/Vitis are you using?
- What OS are you using?
- Are you using the new board files from Digilent that are available for download from our GitHub here, https://github.com/Digilent/vivado-boards?
- Presuming you are using a newer version of Vivado (as opposed to 2016.4), Digilent has a guide to setting up and running a block design for our boards available here: https://reference.digilentinc.com/programmable-logic/guides/getting-started-with-ipi.

Let me know if you have any questions,
JColvin