Replacing HDMI input with OV7670 camera in Zybo HDMI demo

[dzabakh]

Hello everyone!

I'm not sure whether this forum is the right place to ask this question but still.

I have connected a low-cost OV7670 camera to this Digilent example:

https://reference.digilentinc.com/learn/programmable-logic/tutorials/zybo-hdmi-demo/start?redirect=1

Here is what I've done. I took the OV7670 - > AXI4Stream core from here (link below) and attached it instead of HDMI input. I changed this module to have not 32 bit RGBA output but 24 bit RGB input

https://lauri.xn--vsandi-pxa.com/hdl/zynq/xilinx-video-capture.html

and also I took the OV7670 Controller from here (link below) and also attached it to the design

https://lauri.xn--vsandi-pxa.com/hdl/zynq/zybo-ov7670-to-vga.html

The system works o'k.

What I would like to do is to remove the HDMI part from this design. I just want the image to be captured by the camera and be shown on VGA screen. If I understand it right the axi_gpio_video and the v_tc_1 ip-cores send some interrupt essential for the stream to start. I am interested and I have no understanding of what I have to do to remove the HDMI part from the design so that I always saw the image from my OV7670. Do I have to somehow simulate the interrupts? Can I do this in C code?

Thank you very much for response in advance.

[dzabakh]

I guess, I have to improve the explanation.

Here is what I want to do. My aim is to make a design that takes image from OV7670 camera, puts it to DDR through VDMA, then takes it from DDR and shows it on the screen through VGA. I want the design to be baremetal, without Linux.

I took a Digilent example for Zybo that takes AXI HDMI input, puts it to DDR though VDMA and then shows the image through VGA. I replaced the HDMI input of the VDMA with OV7670 input. Like this (image 1).

If I understand it correct from the design and the C code provided with it the processor gets some interrupt when some HDMI input is detected. Right now my design works only when some HDMI source is plugged in the board. The system works and the image is being shown on the screen. I want to make it work without any HDMI (image 2).

As I understand, I have to somehow simulate the highlighted interrupts. What can I do with it?

[jpeyron]

Hi @dzabakh,

I am not very experienced with the HDMI demo. I would guess that one of the IP's is looking for for something like a HDMI hand shake verification as part of its state machine.  I will reach out to my co-workers to look into your forum thread. 

thank you,

Jon

[dzabakh]

Hi @jpeyron,

I will describe what I've tried to do, maybe this will help your specialist give a good recommendation.

If I understand it correct, the processing system waits for two interrupts: 1) an interrupt from v_tc_1 that would show that a signal is detected on HDMI input, 2) an interrupt from axi_gpio_video that occurs when aPixelClkLckd signal from dvi2rgb becomes '1' and axi_gpio_video sends an interrupt. I tried to connect vtiming_in of v_tc_1 to v_sync of my camera so that the interrupt happened when the signal from the camera is detected. And also I connected gpio2_io_i of axi_gpio_video to locked signal of clk_wiz to be sure the interrupt from axi_gpio_video comes. But as I can see nothing happens, the picture doesn't switch to the image from my camera, I can only see the color stripes which is a pre-programmed pattern of your demo. Does this mean that the needed interrupts do not happen?

Also, I tried to do this from C code. I disconnected those two interrupts from the PS and tied them to 0. In the C code, I commented the DemoRun() function call in main() (I've checked that nothing changed) and added this code to the end of DemoInitialize() function. 

	VideoStop(videoPtr);
	if (videoPtr->callBack != NULL && videoPtr->state != VIDEO_DISCONNECTED)
		videoPtr->callBack(videoPtr->callBackRef, (void *) videoPtr);
	videoPtr->state = VIDEO_DISCONNECTED;
	VideoStart(videoPtr);
	if (videoPtr->callBack != NULL)
		videoPtr->callBack(videoPtr->callBackRef, (void *) videoPtr);

I took this code from interrupt handler VtcIsr(). I thought this code will switch the image to the stream from my camera but it didn't. The code becomes executed but nothing happens. 

Also, when I attached the camera input instead of HDMI input to vdma and plugged in an HDMI input to the board, everything worked o'k because the interrupts worked. 

The preferable way for me is to do what I want in C code and remove the interrupts from the design. Please tell me if you can help.

[elodg]

Looking at the block design and the interrupt handlers here is what is happening in the example design: when an HDMI clock is detected, the GPIO interrupt is triggered and gets handled by GpioIsr. This in turn enables the vtc (Video Timing Controller) interrupt. If a known resolution is detected on the timing signals, the vtc interrupt is triggered and gets handled by VtcIsr. This reads in the detected timing information (videoPtr->timing) and calls VideoStart.

You either need to tie all camera sync signals (not just v_sync) to vtc, if the camera outputs at a VESA standard timing, or replace the timing detection mechanism altogether. A quick and easy way is to hard-code the timing information, since it is you who is configuring the camera to a set resolution. In this case, remove the interrupts altogether and call VideoStart when the camera is transmitting. Mismatch in number of pixels and lines between the camera and VDMA might cause errors or lock-up.

Use Vivado to insert logic analyzer probes and follow the video stream as it flows from the camera to the VDMA. Since the pattern is shown I expect the output stream (from the VDMA to the VGA) to be working.

[dzabakh]

@elodg thank you!

I've removed the interrupts as I don't need them, I'm not going to change resolution. So everything works o'k. For those who will face the same problem:

1) Change this in VideoStart function:

	int Status;
	int i;

	//xil_printf("Video start entered\n\r");
	//if (videoPtr->state == VIDEO_DISCONNECTED)
		//return XST_NO_DATA;
	//if (videoPtr->state == VIDEO_STREAMING)
		//return XST_SUCCESS;

	/*
	 * Configure the VDMA to access a frame with the same dimensions as the
	 * current mode
	 */
	videoPtr->vdmaConfig.VertSizeInput = 480;//videoPtr->timing.VActiveVideo;
	videoPtr->vdmaConfig.HoriSizeInput = 640*3;//videoPtr->timing.HActiveVideo * 3;
	videoPtr->vdmaConfig.FixedFrameStoreAddr = videoPtr->curFrame;

2) Comment DemoStart in main function. Add this to the end of DemoInitialize function:

	VideoChangeFrame(&videoCapt, 1) ;
	while(1)
		VideoStart(&videoCapt) ;

Thank you for your help!

[jpeyron]

Hi @dzabakh,

Glad to hear you were able to get your project going. Thank you for posting what you did along with code.

thank you,

Jon

[TDB]

Hey,

I tried to build your project as you explained, but all i see on my monitor are grey stripes. Could you share your project? That would hellp me very much! Thanks!

[dzabakh]

On 11/3/2017 at 7:53 PM, TDB said:

Hey,

I tried to build your project as you explained, but all i see on my monitor are grey stripes. Could you share your project? That would hellp me very much! Thanks!

Hello TDB!

Sorry for the late answer - I was out of Internet.

I have written a tutorial but have no time to publish it. Here are the webpage and the files. The link leads to the archive mentioned in the tutorial. Hope this helps!

https://www.dropbox.com/s/xuw1qfcvnj5720l/Zybo.zip?dl=0

hdmi_in.pdf

webpage.zip

[TDB]

Hey, sorry for the late answer. I had to rebuild your project because i use Vivado 2016.4. Now i can see motion from an object in front of the camera on the screen but just in Red, Green and Blue pixels. Also I have to reset the camera continuously to see motion and dont get stuck in one frame. You have any Idea what i did wrong?

[dzabakh]

11 hours ago, TDB said:

 Also I have to reset the camera continuously to see motion and dont get stuck in one frame.

Hello TDB!

Something is wrong with the C source code you are running as it is C code who is responsible for refreshing frames. Please double-check you are doing everything correctly. The result I got was a continuously refreshing picture turned upside down and I didn't repair that. Also I had problems with kinda color inversion in the final result and I didn' t repair that either.

12 hours ago, TDB said:

Now i can see motion from an object in front of the camera on the screen but just in Red, Green and Blue pixels.

Didn't quite understand this one.

[TDB]

10 hours ago, dzabakh said:

Hello TDB!

Something is wrong with the C source code you are running as it is C code who is responsible for refreshing frames. Please double-check you are doing everything correctly. The result I got was a continuously refreshing picture turned upside down and I didn't repair that. Also I had problems with kinda color inversion in the final result and I didn' t repair that either.

Didn't quite understand this one.

Can you tell me which part of the code is responsible for refreshing frames? In my opinion there is no refreshing possible without any input from the uart in the code you provided. Maybe i miss something.

 

Also i attached an image to show you my current output from the monitor.

Thanks a lot for your patience and helpful messages!

[dzabakh]

10 hours ago, TDB said:

Maybe i miss something.

I was doing this half a year ago so I don't clearly remember the details and also I don't have any opportunity to use Vivado where I am now.

From the photo, I don't understand whether the picture is turned upside down. My one surely was. The problem with turning upside down is somewhere in programming the camera (ov7670_controller_0). There is a register that is responsible for turning image upside down. It was 0x1e37 if I remember it correctly. You have to check the datasheet for the camera. Programming the registers didn't work well for me. I don't know whether the problem was with my camera or code but I finally dropped the idea of tuning the image.

The colors look similar to what I had. I am colorblind so tuning the colors is a huge problem for me. I always had to ask someone whether the colors are okay or not. I think the trouble with the colors is somewhere in splitting the v_axi4s_vid_out_0.vid_data into 3 parts (vga_b, vga_g, vga_r). I checked it carefully but maybe not carefully enough. You need to check the color bus all the way through (ov7670_d -> ov7670_axi_stream_capture_1.d  ->  ov7670_axi_stream_capture_1.m_axis -> axi_vdma_0 -> axis_subset_converter_0 -> v_axi4s_vid_out_0). Check that all the bits are arranged in correct order and the merges are done correctly. This is the only advice I can give.

About refreshing. Yes, I was wrong. The v_axi4s_vid_out_0 core should give an output image when it gets an input (video_in, vtiming_in). Did I write about this in tutorial? Please double-check that you have connected and tuned everything correctly, maybe this is the problem.

10 hours ago, TDB said:

Thanks a lot for your patience and helpful messages!

Well, I did nothing for you

Please, write here about all the troubles you meet and the solutions you find, if you find any. I want the tutorial to be good and complete as using this camera is popular at the moment.

Thank you!