Setting up AXI Quad SPI on Arty

[UncleSlug]

Is there a tutorial for setting up the Quad SPI flash on the Arty for use by a Microblaze processor? I found one that uses the Quad SPI to store the FPGA image (https://reference.digilentinc.com/learn/programmable-logic/tutorials/arty-programming-guide/start#programming_the_arty_using_quad_spi) and one for the software development (https://reference.digilentinc.com/learn/programmable-logic/tutorials/htsspisf/start). I seem to be missing the step where you configure the Quad SPI in the block design. I added a Quad SPI AXI block connected it up to the bus/clocks like the existing GPIO and UARTlite I had (as done in https://reference.digilentinc.com/vivado/getting-started-with-ipi/start). Then I configured the block to use XIP and tried to change things around in the address editor. The address of the flash seems to be stuck in the data section, though I would have thought using XIP it should be able to go in the instruction section somehow.

So I must be doing something very wrong. While I'm here, if the FPGA image gets stored in the Quad SPI flash how does it coexist with executable code for a Microblaze. I'm not at the PC I'm using so I will post some pics soon.

 

Thanks

 

 

 

[UncleSlug]

So here is what I had in my block design. I tried re-configuring the Microblaze to 'Enable Peripheral AXI Instruction Interface' but I can't connect either of the AXI ports on the quad spi block. I think that is because they are both masters?

[jpeyron]

Hi @UncleSlug,

I am not familiar with the XIP feature.  Here is an older tutorial that runs through setting up microblaze on the Arty A7.  Here is a tutorial the covers using the QUAD SPI FLASH IP Core. When adding the QUAD SPI FLASH IP Core add a 50 MHz to the Clocking wizard and attach it to the EXT_SPI_CLK pin on the QUAD SPI FLASH IP Core.

thank you,

Jon

[UncleSlug]

I might try not using XIP and just use the SPI flash for code storage. Then copy the code into RAM before executing it.

[D@n]

@UncleSlug,

I've now written several SPI controllers ,for several parts.  XIP has been an important feature to those controller.  Other than trouble shooting, I can't think of any reason not to use XIP.

Dan

[UncleSlug]

I've made some progress but I don't seem to be able to save the FPGA bit stream complete with the bootloader program and load from reset (as done in https://reference.digilentinc.com/learn/programmable-logic/tutorials/htsspisf/start). I can configure the hello world program to load from QSPI flash at 0x00c00000 into RAM and run with the bit stream loaded through the USB/JTAG. The problem comes if I write the bit stream to the flash (with offset 0x00000000) in the SDK and push the reset button, it doesn't work (I've tried with and without the jumper).

As far as I can tell, when you program the FPGA from the SDK it creates another bit stream which is the same as that from Vivado (the non-SDK part) except with the block RAM initialised (in this case with a spi bootloader. Neither of these seem to work for me. I've tried opening the implementation/synthesis going into edit device and ticking the master spi x4 configuration option but that doesn't seem to help.

I am curious to know where the offsets come from, I can't find any other reference to 0x00c00000. Does the QSPI flash just start at 0x00000000 and a full image/bit stream will be smaller than 0x00c00000? Then is the remaining space available for programs or whatever else you want?

[D@n]

@UncleSlug,

A couple questions:

1. Can you tell how far it gets when loading?  Does the design load, but the microblaze fails to start up?  Or does the design not load at all?
2. If you query the ICAPE2 interface, what reason does it give for your design not loading?  Does the boot status register offer any useful feedback at all?
3. If you do a hex dump of the flash, does it match what it should look like?  For example, do the first several bytes match the bytes of a .bin file?  (Not the .bit, the .bin file)

Dan

[UncleSlug]

@D@n,

I think I have some of the info.

1. I can't quite tell just yet, I think maybe I should change the design to use some of the LEDs in programmable logic.
2. I assume these are the registers that you can look at in the hardware manager? The boot status has the value 'REGISTER.BOOT_STATUS    00000000000000000000000000000001' and the config status has the value 'REGISTER.CONFIG_STATUS    01010000000100000101100111111100'. I can't see any error bits set and it looks like it should have loaded correctly.
3. I've read back all of the flash from the hardware manager and compared it to the BOOT.bin file in projectname.sdk\system_wrapper_hw_platform_0\cache and they appear to be exactly the same except the flash image is longer and has data at 0x00c00000 (must be hello world in srec format). Comparing these to the system_wrapper.bin from Vivado results in some differences, but I think that is probably because the block ram isn't initialized in that file?

Thanks

[D@n]

@UncleSlug,

If I'm reading it right, then the boot status register just tells you what you already know: the FPGA was unable to read a valid design from the flash.  That doesn't really give us anymore information than you outlined initially.

Thinking through this some more, I had almost forgotten that I had struggled with something similar on my own Arty-A7 board.  Background: Once the flash is put into XIP mode, it treats the first 8-clocks of any transaction, the clocks that would otherwise issue a command to the flash in non-XIP mode, as having an address instead.  If, while the flash is in this state, you do a warm boot, the Xilinx chip appears not to know that it needs to take the flash out of XIP mode.  The command then sent to the flash, such as a read command, is ignored and treated instead like an address.  This is why the hard boot works, but the warm boot fails.

Sometimes these flash chips have an external reset you can use for that purpose.  I don't think the Arty's flash has such a reset though. 

If you read from the flash specification (p62 for me), you'll read that the suggested way of dealing with this is to send a SPI command to the device where D0 (MOSI in SPI mode) contains nothing but ones for a specific number of clock periods.  This will put the chip back into a mode where it will accept commands again.

So how could you fix this?  1) You could ensure that the flash is taken out of XIP mode prior to any FPGA reload.  In my own design, this was accomplished by rewriting the flash using my own controller, since the controller kept track of what mode the flash was in.  Then, I'd probably use the IPROG command of the ICAPE2 to control the reload in this case, rather than the on-board button.  (I personally never did much with the on-board button.)  2) You could avoid using the soft reboot feature.  3) You could make certain that the CPU reset button doesn't reset the flash controller, etc.  (This is difficult if the reset button resets the DDR SDRAM driver, since that driver also generates the clock used by the rest of the design.)  4) If you just need to reboot the CPU and not reload the FPGA, you could adjust the flash controller so that  either it doesn't respond to the reboot, or it issues the proper get-out-of XIP sequences before starting.  (Of course, once it starts the first command would be to enter into the XIP mode )

This isn't a very satisfying answer I'm sure.  I'd be curious what Digilent's engineers might say regarding this issue.

Dan

P.S. Do be aware, there is a feature of this flash chip that allows it to start from power up into XIP mode.  (See bits 9-11 in the nonvolatile configuration register.) Do not enable this feature, or your hard reboot (power cycle) will stop working as well.

[jpeyron]

Hi @UncleSlug,

I have not used the XIP mode in the QUAD SPI IP Core. I would look at the AXI Quad SPI v3.2 LogiCORE IP Product Guide for more information on using this mode. Here is XAPP1176 that should also be helpful with using XIP and the QUAD SPI IP Core. I have also reached out to some of my co-workers about this thread to see if they have any additional input.

thank you,

Jon

[jpeyron]

Hi @UncleSlug,

I tried using the XIP mode with the basic hello world without success. Looking into the information in the AXI QUAD SPI IP core the read function is only implemented. I believe that not having the write function with the XIP mode interferes with uploading the project on the flash. I would suggest to use the How To Store Your SDK Project in SPI Flash tutorial to load your project in flash. Make sure to give the EXT_SPI_CLK 50 Mhz from the clocking wizard. I would also suggest to compress the bitstream. The Flash loads the project much quicker on power on with a compressed bitstream.

thank you,

Jon

[D@n]

@UncleSlug,

Sorry, @jpeyron,, I'm going to disagree here.

1. It's not write vs read that's the problem.  XIP write doesn't make sense in the first place.
2. Let's take a moment to understand XIP so we know what we are talking about.  Normal flash transactions require an 8-bit read command followed by an address and then a mode byte before they can read.  In XIP mode, once the read is accomplished the chip stays in a special mode waiting for the next read command.  The next read command then starts not with the 8-bit read command, but rather directly with the address.  That's what makes XIP special: reads can be accomplished 8-SPI clocks faster then they would be accomplished otherwise.
3. There is no XIP write mode because ... writes are so rare, and take so long, that there's no reason to optimize them with some kind of XIP mode in the first place.
4. The problem with the XIP interface mode is getting out of it once its been turned on.  In order to send a flash command again, you have to send a read request with a "leave-XIP" mode byte following the address.  (Remember, in XIP mode, a read request consists of an address followed by a mode byte--nothing more, whereas normal flash requests start with an 8-bit command.)  Once you leave XIP mode, you can then issue any 8-bit flash command you want--such as erase sector, program page, program page with quad-I/O, read ID, read the non-volatile configuration status register, etc  Until then, your 8-bit command will be interpreted as bits of the address--leaving the flash chip and the FPGA controller for it out of synch with each other.
5. Whether or not the Quad IP core you are mentioning above supports it or not I don't know, but the flash chip on the Arty A7 can handle an SPI SCK that toggles as fast as 100MHz.  I think the data sheet says 108MHz.  I know I've personally tested it up to 100Mhz.  (Yes my core was running at 200MHz at the time.  Were I to build the core again, I'd use the ODDR modes and toggle the SPI clock with the system clock.)
6. Compressing the bitstream is always a good idea--independent of XIP mode.
7. Loading from flash in QSPI x4 mode is also a good idea, though I doubt you'd notice the difference without a good measurement device.

Hence, if you want to use XIP mode (I use it all the time), I would recommend:

1. Using a flash controller that can start itself from a known state--regardless of whether or not it was started while the flash chip thought it was still in XIP mode .  Such a controller might need to send the flash a very carefully crafted command to leave XIP (if it is in XIP mode), or ignore the command if not.  You are welcome to use my controller if you wish, only it has a  wishbone interface instead of an AXI interface.  Alternatively, you could start your design from a cold power up every time.
2. Integrating a write capability into your controller, and writing your own configuration to flash.  You can then command your design to load a new configuration, following the writes to the flash, by writing to the FPGA's IPROG resgister.  (Be careful when writing to the flash--you *cannot* read from it while it is busy writing)
3. Restarting your design either from power up, or following a write (or other command) to the flash that would take the flash out of XIP mode.  That will insure that the FPGA starts with a clean slate (i.e. not in XIP mode) when it tries to issue a read command to read from the flash.  (i.e. the button won't work.)
4. While not necessarily related to XIP, do be careful when setting or adjusting the flash configuration registers.  There are several non-volatile options that won't clear on boot, and will render this roadmap non-working.

Dan

[jpeyron]

Hi @UncleSlug,

Also the offset of 0x00c00000 is the space needed for the .elf file for the Arty-A7 and the rest is used by the download.bit file when loading an SDK project into flash.  

thank you,

Jon

 

[UncleSlug]

Hi @jpeyron, @D@n,

In case it wasn't clear in my last two posts I stopped trying to use XIP mode and just tried to make everything the same as the examples (to eliminate some variables) which I haven't quite got working yet. I haven't had much time to play around with it in the past few days. I guess I could try to read the qpsi registers to make sure I haven't accidentally configured it to start up in the wrong mode. As I mentioned I will probably change my design so that some of the LEDs are triggered by the buttons in programmable logic in case somehow the image loads but the processor doesn't start or somethings weird.

Thanks