Video processing in HDMI in project

[Testo]

Dear everybody,

I'm new for this field. Currently, I have zybo zynq 7000 development board.

My project is to receive video stream from PC and send back the processed video.

The tuttorial HDMI in is useful. https://reference.digilentinc.com/learn/programmable-logic/tutorials/zybo-hdmi-demo/start

I want to add Custom IP, for example changing color space from RGB to HSV.

 

I found that, in sdk, the stream video for invert image is retrieved for a moment, invert it, and write it back.

Thus, my idea is if I send the retrieved data to my custom IP, instead of invert it by PS. Then, read data back from custom IP and write it back.

It should solve my project. I connect my custom IP via VDMA. I try to figure it out how to use VDMA but i don't understand it.

Could you give me any suggestions how to do this?

 

[jpeyron]

Hi Testo,

I have not worked with the VDMA so i would need to reach out to the engineer that created the project. He is on vacation for the next 2 weeks. When he is back I will be able to help you with this project. Sorry for the inconvience.

cheers,

Jon

[Testo]

Dear Jon,

Is there any update here? I still want any advice if it possible.

I tried to figure it out by myself but it still confuss.

 

[jpeyron]

Hi Testo,

The engineer I reach out to is back and we should be able to get you a response within the next day or two. Sorry about the wait.

cheers,

Jon

[sbobrowicz]

The interface that your custom IP core will need to implement to connect to the VDMA core is called AXI Stream. This interface is a pretty simple uni-directional FIFO style interface. In your case, your IP core will need to implement one slave AXI stream interface (for the input video stream from memory) and one master AXI stream interface (for the output video to memory).

The first place you should go to learn how to implement a custom video IP core is Xilinx's AXI4-Stream Video IP and System Design. That doc will outline a lot of the different things you will need to consider about your design. You may also need to refer to the VDMA IP documentation here. If you need additional information on the AXI stream interface, check out the AXI Reference Guide from Xilinx. Specifically page 94 will have information on the Video "flavor" of AXI stream.

To get started, I would recommend that you use the create custom IP wizard in Vivado and check the boxes indicating that you will have an AXI stream master and slave. This will provide you with a good starting template and some control logic for the AXI stream handshaking signals. You can also look at the source for Xilinx's AXI-Stream to Video Out and Video In to AXI-stream IP cores as examples. The source code for those cores should be included with Vivado.

 

[Testo]

Dear Sirs,

Thank you for your reply. I aleady implemented my IP with 24bits stream input and output type.

Moreover, I tried the simplest design with VDMA api example from Xilinx and it doesn't work even it doesn't use HLS IP.

In block design, I modified VDMA as 1 frame buffer, 24 data width write and read channels.

In SDK, I modified the VDMA api example in Write channel setup. I relocated the address of write channel to the next possible address instead of the same address as read channel because I want to inspect that is it write to array or not. Another modified is that I don't have any ddr in my desgined, so the api caller must malloc memory address, which are 2 dimension array[2], array[0] for read channel, array[1] for write channel. The result printed that it transferd data success but I inspected at arra[1], which is wrtie channel address. The data doesn't changed. What do I do wrong? Even use api created by Xilinx I cannot use VDMA.

int main()
{
	int Status;
	XAxiVdma InstancePtr;

	int frameSize = 100*200*3;
	u8 frame_buffer[2][frameSize];

	xil_printf("\n--- Entering main() --- \r\n");
	xil_printf("Starting the first VDMA \n\r");

	srcBuffer = (int)&frame_buffer[0];

	//Check Address of buffer
	xil_printf("SrcBuffer: %x\n\r", srcBuffer);
	xil_printf("Frame[%d]: %x\n\r", 0, frame_buffer[0]);
	xil_printf("Frame[%d]: %x\n\r", 1, frame_buffer[1]);
	xil_printf("WriteAddr: %x\n\r", srcBuffer + (100 * 200 * 3));

	//Initial data in read buffer
	int j;
	for (j = 0; j < FRAME_SIZE; j = j+3){
		frame_buffer[0][j] = (u8)255; //r
		frame_buffer[0][j+1] = (u8)200; //g
		frame_buffer[0][j+2] = (u8)150; //b
	}

	/* Calling the API to configure and start VDMA without frame counter interrupt */
	Status = run_triple_frame_buffer(&InstancePtr, 0, 100, 200, srcBuffer, 100, 0);
	if (Status != XST_SUCCESS) {
		xil_printf("Transfer of frames failed with error = %d\r\n",Status);
		return XST_FAILURE;
	} else {
		xil_printf("Transfer of frames started \r\n");
	}
    return 0;
}

static int WriteSetup(vdma_handle *vdma_context)
{
	/*
	 * Same code
	 */ 
	 
	Addr = vdma_context->buffer_address + (100 * 200 * 3);	// modifed to address another index of array
	
	/* 
	 * Same code 
	 */
}

Sorry for bothering you but I really want your help.

I really really appriciate your help.

[rappysaha]

Hi Testo,

I have initialized the VDMA with DDR3 and Microblaze processor. I don't know why you don't use DDR3 directly. I have a little knowledge about ZYNQ processor.

What is the size of the malloc memory? Is it enough to hold your frame size?

I am just asking because I am also working with same thing.

Thanks

Rappy Saha

[Testo]

Hi Rappy Saha,

 

Doesn't VDMA provide memory inside itself? Do we need DDR3 for storage?

I really new at here. Then, I'm not sure my design is correct.

 

The malloc memory is exactly the same size of frame size.

 

Thank

[rappysaha]

Hi Testo,

I don't know ZYNQ processor includes DDR3 or not but in my case, when I used Microblaze I added MIG IP to use DDR3.

Thanks

Rappy

[sbobrowicz]

On 1/10/2017 at 11:42 PM, rappysaha said:

Hi Testo,

I don't know ZYNQ processor includes DDR3 or not but in my case, when I used Microblaze I added MIG IP to use DDR3.

Thanks

Rappy

The reason you don't need the MIG is because the Zynq processing system includes a DDR3 controller. Your VDMA gets write/read access to the DDR3 via the HP port that is connected to the axi_mem_intercon. The ARM processor also has read/write access to the DDR3, but the complexity of this is hidden by the processing system block.

I've got a couple things to try. First, you need to invalidate your cache after you have the processor write the framebuffer. This is because the data can remain in cache and never get flushed to the actual memory. Add this line to main after your for loop that writes the frame_buffer.

Xil_DCacheFlushRange((unsigned int) frame_buffer[0], frameSize);

Another thing I noticed is that you seem to be assuming that the destination buffer is sequential in memory directly after the source framebuffer. I might be wrong here, but I don't think the C language guarantees that multi-dimensional arrays are sequentially aligned in memory. I would recommend declaring a global (u8 *) variable called destBuffer and initializing it the same way you did with srcbuffer (I'm assuming srcbuffer is declared as a u8* too). After you initialize the two buffers, you should never need to access the frame_buffer variable directly again (though declaring it is required to make sure the memory is allocated). You can use the array style access syntax ( [ ] ) with srcbuffer and destbuffer to get at individual pixels, but you should not assume the two have any positional relationship within memory

[sbobrowicz]

Testo got back to me and let me know that his problem was caused by using a VDMA address that was not in 0x0000_0000-0x4000_0000 DDR memory space