Arty A7-35 Ethernet Rev D Ethernet Mii Interface

[wolfgangmeyerle]

Hi,

I'm currently experimenting with the implementation of an Ethernet Stack in RTL logic. Nothing too complicated, just receiving and transmitting bare metal Ethernet packages.

However I'm struggling a little bit how to receive the correct traffic. For sending packages on the PC side I'm using Ostinato. I'm Debugging the received traffic via an Uart Interface. For some strange reason the FPGA seems to swallow Bytes. 

A couple of Questions:

Regarding the preamble and start frame delimiter. Is it true that Always 7 Bytes have to be sent with 0x55 as Content following the 8th Byte with 0x5D or can the number of preamble Bytes also be shorter ?

During the Transmission of the data from the PC to the FPGA I would expect that the eth_rx_dv signal is always  set to '1' as long as the message is. Is this correct?

What might it be that bytes are lost? Any ideas to track down the bug? 

I'm using the 100Mbit Connection which was connected to a 25Mhz Clock.

I used the following Code from an FPGA UDP send example found online to connect the Clocks. Is anything missing or is the Receiver Clock recovered from the PC?:

   -- Clocks ...
   ----------------------------------------------------
   -- Correctly forward the clock out,so rising edge 
   -- will be in the middle of the valid data 
   ----------------------------------------------------
clock_fwd_ddr : ODDR
   generic map(
      DDR_CLK_EDGE => "SAME_EDGE", 
      INIT         => '0',
      SRTYPE       => "SYNC")
   port map (
      Q  => eth_ref_clk,
      C  => clk25MHz,
      CE => '1', R  => '0', S  => '0',
      D1 => '0', D2 => '1'
   );

   -------------------------------------------------------
   -- Generate a 25MHz and 50Mhz clocks from the 100MHz 
   -- system clock 
   -------------------------------------------------------
i_bufg: bufg port map (i => CLK100MHz, o => CLK100MHz_buffered);
clocking : PLLE2_BASE
   generic map (
      BANDWIDTH          => "OPTIMIZED",
      CLKFBOUT_MULT      => 8,
      CLKFBOUT_PHASE     => 0.0,
      CLKIN1_PERIOD      => 10.0,

      -- CLKOUT0_DIVIDE - CLKOUT5_DIVIDE: Divide amount for each CLKOUT (1-128)
      CLKOUT0_DIVIDE     => 32,  CLKOUT1_DIVIDE     => 16, CLKOUT2_DIVIDE      => 16, 
      CLKOUT3_DIVIDE     => 16,  CLKOUT4_DIVIDE     => 16, CLKOUT5_DIVIDE      => 16,

      -- CLKOUT0_DUTY_CYCLE - CLKOUT5_DUTY_CYCLE: Duty cycle for each CLKOUT (0.001-0.999).
      CLKOUT0_DUTY_CYCLE => 0.5, CLKOUT1_DUTY_CYCLE => 0.5, CLKOUT2_DUTY_CYCLE => 0.5,
      CLKOUT3_DUTY_CYCLE => 0.5, CLKOUT4_DUTY_CYCLE => 0.5, CLKOUT5_DUTY_CYCLE => 0.5,

      -- CLKOUT0_PHASE - CLKOUT5_PHASE: Phase offset for each CLKOUT (-360.000-360.000).
      CLKOUT0_PHASE      => 0.0, CLKOUT1_PHASE      => 0.0, CLKOUT2_PHASE      => 0.0,
      CLKOUT3_PHASE      => 0.0, CLKOUT4_PHASE      => 0.0, CLKOUT5_PHASE      => 0.0,

      DIVCLK_DIVIDE      => 1,
      REF_JITTER1        => 0.0,
      STARTUP_WAIT       => "FALSE"
   )
   port map (
      CLKIN1   => CLK100MHz_buffered,
      CLKOUT0 => CLK25MHz, CLKOUT1 => CLK50Mhz, 
      CLKOUT2 => open,     CLKOUT3  => open,
      CLKOUT4 => open,     CLKOUT5 => open,
      LOCKED   => open,
      PWRDWN   => '0', 
      RST      => '0',
      CLKFBOUT => clkfb,
      CLKFBIN  => clkfb
   );
 

 

[zygot]

6 hours ago, wolfgangmeyerle said:

Is it true that Always 7 Bytes have to be sent with 0x55 as Content following the 8th Byte with 0x5D or can the number of preamble Bytes also be shorter ?

This is a good question to figure out by experimentation. Before you can do that you need to have your Ethernet interface working perfectly. I suggest that you don't mess with the synchronization bytes that your PHY depends on until you are receiving good data and appear to be capturing packets.

The rx_dv signal indicates that the RxD data is valid; not necessarily correct. Even if you can't find a data sheet for your PHY there is often enough information available describing the signals and initialization for your device. Fortunately, documentation on the IEEE 802.3 standard including the physical layer is freely available.

It seems to me that what you need to do is figure out a way to debug your HDL. You say that you are losing bytes; that's a good place to start. The first thing I'd do is try and capture RxD and Rx_dv and Rx_err signals while the PHY is receiving a packet. The clock domain is the rx_clk supplied by the PHY. Here's where the Vivado ILA can be real handy.  The ideal test setup would be having a board connected to yours that would send the same simple packet over and over at some interval, like 1 second. The ARP packet is nice and short. You shouldn't have to worry about PHY initialization but the ILA should help figure out timing issues. Of course you will need to assemble bytes from the MII interface nibbles properly; so capturing bytes instead of PHY RxD pin data in your ILA might be helpful. Once you are capturing good data you can move on to parsing packets and calculating CRC check words.

[wolfgangmeyerle]

Hm. I just started with Vivado. What I thought would be still difficult to debug with the ILA is that the PC is not stopping it's communication to the FPGA when he send out packages even if the ILA stops the Hardware. So it will be difficult for me to grab every nibble from the physicial interface. Can you Point out some Guidelines or tutorials which I can follow to ramp Knowledge how to use the physicial interface? It's a Little bit frustrating that you have the Hardware on the desk but no tutorials around to follow how to use it.

I've written so far some logic to send out data correctly to the PC, so Building up the crc and swapping the Bytes is already done and working from a Sender Point of view. It's just the Receiver site which looks odd. 

I tried to narrow it down just Building up a std_logic_vector where I shift the nibble data through until it's full and send that out via the uart interface and I still saw that some Bytes are missing. I double checked the uart interface and it sends out every byte. 

Next thing I tried today was wiring Input and Output ports together. If the PC sends out data the data seems to be replicated 1:1 on the Ethernet, so clocks setup seems to be right. Could it be that maybe some processes "block". I have just a basic idea how processes work in vhdl but I assume that everything runs in parallel. I only have to make sure that a process doesn't run forever by including a rising Edge condition on the Clock Signal is that correct?

[zygot]

13 hours ago, wolfgangmeyerle said:

What I thought would be still difficult to debug with the ILA is that the PC is not stopping it's communication to the FPGA when he send out packages even if the ILA stops the Hardware

I have frequently used the ILA or SignalTap (Quartus) to debug the Ethernet PHY interface of new projects. As long as the ILA is clocked with the appropriate clock you can capture a packet by triggering off of the appropriate signal. You can instantiate two instances of ILA; one for the transmit and one for the receive clock domains and capture packets going both directions. Just make sure that the ILA depth is large enough. This is one application that is ideal for Vivado ILA debug IP. For debugging the receiver you just need to capture data over the period where eth_rx_dv is true. A limitation of the ILA is that you trigger a capture and once a buffer with samples is filled anything that happens after that  is ignored. For this application this behavior is a strength. Just trigger on eth_rx_dv. The trigger conditions are set in the Vivado Hardware manager every time you do a capture.

A problem with using a PC as a stimulus for your testing is that there will likely be a lot of packet types sent over some period of time. The only way to overcome this is by using a custom Ethernet driver so that the OS doesn't see the interface as a normal Ethernet device. By far the best test setup is to have two FPGA boards connected as a point to point system. That way you can control exactly what is being sent and at what interval. You don't need to worry about CRC, packet structures, packet lengths, or any of the things that are unimportant as to whether or not your Ethernet receive logic is working. You just need a preamble, SDF, and some reasonable number of payload data.. 16 is easy. Debugging is always easier if you eliminate irrelevant parts of a system. 

It is also possible to test your Ethernet PHY interface logic by using the loopback feature in most PHY devices. This is complicated when you don't have access to the register set for your PHY when it's made by a vendor like Marvell who chooses not to share this information without s signed NDA. It's also complicates things because if your PHY transmit logic is wrong then you are debugging two things at the same time. More unknown is never better when trying to figure out why your design doesn't work.

It is possible to create an application using Python or C on a Linux OS ( well at least a lot easier on Linux ) that will form and send packets from a PC. This isn't trivial either.

I can think of a number of reasons for implementing a limited Ethernet stack in logic using an HDL. As you develop your project keep in mind that your network design will have a big impact on how complicated your HDL will need to be. Point to point Ethernet between two FPGA boards is one fairly straight-forward. Adding, switches, routers, fragmentation, out of order packets and other wrinkles will add considerable complexity. You might find that logic that works for the simple case isn't easily ported to a more complicated case.  

 

[zygot]

16 hours ago, wolfgangmeyerle said:

I have just a basic idea how processes work in vhdl but I assume that everything runs in parallel. I only have to make sure that a process doesn't run forever by including a rising Edge condition on the Clock Signal is that correct?

Somehow I missed this the first time I read it.

Yikes. No, I don't think that conceptually you are grasping VHDL. It's very easy to confuse computer programming concepts with logic design/simulation concepts.

All processes in VHDL run concurrently. This means that two processes can't manipulate the same signal. Processes can be clocked or combinatorial. Clocked processes imply storage (memory) but combinatorial logic can have storage. Clocked processes can use either edge of a clock. There are also concurrent statements. You cannot imply an evaluation order, in time, to multiple concurrent elements. Some statements, like if-then-else imply priority over which parts of the statement will be evaluated or even if they are evaluated. Some do not. Logic design using an HDL requires a different approach, conceptually, than programming a fixed structure like a microprocessor. Even though modern FPGA devices use LUTs to implement logic there are a lot more fine-grained details to worry about with FPGA design such as delay, levels of logic, multiple clock domains, etc. If that wasn't enough there's logic design concepts and then there's concepts related to synthesis of logic. The synthesis aspect of Verilog and VHDL were added after the languages were used for simulation. Often textbooks consider each separately. The last "joy" of FPGA design are the tools themselves. Synthesis and place and route are dependent on timing analyses based on the FPGA architecture and is largely clock based. When there are multiple clocks they can be derived or unrelated making this analysis complicated, and usually requiring some help from the user in the form of constraints.

I strongly suggest stepping back a moment and working on the basic concepts of digital design using an HDL. You need to debug you before you can debug your designs. 

 

[JColvin]

Hello,

We (Digilent) do not have any formal examples or materials to be use Ethernet in RTL; we have some material that uses the Ethernet with Microblaze for a different Artix chip here that you could reference, though it will not be quite so straightforward to convert into RTL. Taking one of zygot's statements out of context, there are a lot more fine-grained details to worry about.

Thanks,
JColvin