LWIP:itoa Function for FPGA- How to increase the buffer size ?

[Thausikan]

static char buf[32] = {0}; // Globally declared
char* itoa(int val, int base)
{
	
	int i = 30;
	
	for(; val && i ; --i, val /= base)
	
		buf[i] = "0123456789abcdef"[val % base];
	
	return &buf[i+1];	
}

I am working with Xilinx Ethernetlite (lwip) design. With LWIP Echo template, I am able to make communication between FPGA board and PC (hercules). The issue is in hercules or in itoa Function. I am unable to make sure.

In Hercules, to print the data i am doing itoa conversion Function in echo file. Only if buf size is 32. I am able to read the properly. If i increase the buf size is 1024. I am not able to read the data in Herceuls.

 

err_t recv_callback(void *arg, struct tcp_pcb *tpcb,
                               struct pbuf *p, err_t err)
{
	int i,j,Status;
	Status=aurora_rx_main();         ///FUNCTION CALL

	for(i=0;i<100;i++)
     {
		xil_printf("%d\n", DestinationBuffer[i]);
	}

    int base=10; // here 10 means decimal
    char *result={0};

	if (!p) {
		tcp_close(tpcb);
		tcp_recv(tpcb, NULL);
		return ERR_OK;
	}

	/* indicate that the packet has been received */
	tcp_recved(tpcb, p->len);

   if (tcp_sndbuf(tpcb) > 100)
	//if (tcp_sndbuf(tpcb) <= p->len)
	{
		 for (j=0;j<=100;j++)
		    {
		    result= itoa(DestinationBuffer[j],base);
		    err = tcp_write(tpcb,",",1,1);
		    err = tcp_write(tpcb,result,100,1);          // Unable to read the 100 values in Hercules. 
		   tcp_sent( tpcb, NULL );
          tcp_output(tpcb);
		    }


	} else
		//xil_printf("no space in tcp_sndbuf\n\r");

	/* free the received pbuf */
	pbuf_free(p);

	return ERR_OK;
}

 

[tcmichals]

err = tcp_write(tpcb,",",1,1); err = tcp_write(tpcb,result,100,1); // Unable to read the 100 values in Hercules.

Need to check the tcp_write return code, if ERR_OK is not returned this indicates that the write was not successful.   lwIP TCP raw API has a callback function when a TCP write operation completes, this makes the code more complex, but allows using limited memory.

lwip, has several examples, showing of to configure callbacks, download the lwip code and go to the src/apps/ 

Also, use wireshark to see the traffic between to devices.   Wireshark helps you see what is on Ethernet, thus having a way to see what is being received and transmitted. 

 

 

[Thausikan]

Sorry, I can`t understand your point. I am stuck in this problem for past 15 days.

How to increase TCP send buffer size ?? Please follow the below i have posted same thing with Screen shots.

https://forums.xilinx.com/t5/Embedded-Processor-System-Design/LWIP-echo-Example-data-printing-in-Hercules-with-Junk-values/m-p/900266#M42254

https://forums.xilinx.com/t5/Embedded-Development-Tools/KINTEX-KC705-LWIP-Ethernetlite-recv-callback-Function/td-p/895443

Actually, Data is generated from DAQ board and passed to KC705 board through optical cable. In KC705, received data is stored in destination buffer. Now, i am trying to read that data stored in destination buffer in PC through Ethernetlite (LWIP Echo template). Please clarify me step by step. Thanks

[tcmichals]

OK, here are two options:

• Fixing the code:

There all several callback functions that can be registered with the lwip raw TCP interface for a  struct tcp_pcb *tpcb.  One of them is:tcp_sent(newpcb,  the callback function ); 

When the recv data comes in, process the packet, then start transmitting, if the transmitter gets full, then save where the last successful transmit was done.  Continue the writing the results in the tcp_sent callback function.   Using the RAW TCP interface,  there is only one thread, so, accessing variables is thread safe.  The main issue, is, another recv_callback is invoked while still processing the previous request. 

FYI, you can download the lwIP code, there are several examples of using RAW TCP callbacks, i.e http, mqtt etc.   The reason for using callbacks, is there might not be enough RAM so, this should work for most applications.  

int howMuchToStillWrite=0;


/* arg is passed what was assigned to the call back functions */
err_t sentFunction(void * arg, struct tcp_pcb * tpcb, u16_t len)
{

   char buffer[20];
   if ( tcp_sndbuf(tpcb) > 100 &&  howMuchToStillWrite )
   {
        /* the best thing to do is use one tcp_write*/
	   int len = sprintf(outbuffer, ", %x", parameters);
        
	   err = tcp_write(tpcb,outbuffer,len, 1);
     	   if (err != ERR_OK)
          {
     	   //How much still needs to be written out.. 
     		howMuchToStillWrite = 100 -j;
     	  }
	 else
     	 {
     		tcp_sent( tpcb, NULL );
        	tcp_output(tpcb);
     	
	}
   }

 

}


err_t recv_callback(void *arg, struct tcp_pcb *tpcb,
                               struct pbuf *p, err_t err)
{
..... 

/* indicate that the packet has been received */
	tcp_recved(tpcb, p->len);

   char buffer[20];
   for (int j=0; j <= 100 && (tcp_sndbuf(tpcb) > 100)))
   {
		result= itoa(DestinationBuffer[j],base);
        /* the best thing to do is use one tcp_write
	   int len = sprintf(outbuffer, ", %x", parameters);
        
		err = tcp_write(tpcb,outbuffer,len, 1);
     

     	if (err != ERR_OK)
        {
     	   //How much still needs to be written out.. 
     		howMuchToStillWrite = 100 -j;
     	}
	else
     	{
     		tcp_sent( tpcb, NULL );
        	tcp_output(tpcb);
     	
	}




	/* free the received pbuf */
	pbuf_free(p);

 

• Adding more TCP memory (may not actually fix your problem)   There is an h file that has several lwip configuration, here are some recommended values:

/* TCP Maximum segment size. */

#define TCP_MSS                 1460

/* TCP sender buffer space (bytes). */
#define TCP_SND_BUF             (8 * TCP_MSS)

/* TCP sender buffer space (pbufs). This must be at least = 2 *
   TCP_SND_BUF/TCP_MSS for things to work. */
#define TCP_SND_QUEUELEN        (4 * TCP_SND_BUF/TCP_MSS)

I have not used lwIP on a microblaze.  Used lwIP on several other projects.

 

 

[Thausikan]

Thanks for your response. I have changed my code according to your format.

Edited code

static char buf[35] = {0};
int howMuchToStillWrite=0;

char *itoa(long int val, int base){

 int i = 34;
for(; val && i ; --i, val /= base)
buf[i] = "0123456789abcdef"[val % base];
return &buf[i+1];
}



err_t recv_callback(void *arg, struct tcp_pcb *tpcb,
                               struct pbuf *p, err_t err)
{
	int i,j,Status;
	int len;
	//char outbuffer[20];
	Status=aurora_rx_main();         ///FUNCTION CALL

	for(i=0;i<35;i++)
     {
		xil_printf("%d\n", DestinationBuffer[i]);
	 }

    int base=10; // here 10 means decimal
    char *result={0};

	if (!p)
	 {
		tcp_close(tpcb);
		tcp_recv(tpcb, NULL);
		return ERR_OK;
	 }

	/* indicate that the packet has been received */
	tcp_recved(tpcb, p->len);

	for (j=0; j <= 35 && (tcp_sndbuf(tpcb) > 35);j++)
	{
			    result= itoa(DestinationBuffer[j],base);
			   // err = tcp_write(tpcb,",",1,1);
			    err = tcp_write(tpcb,result,35,1);
			  //  tcp_sent( tpcb, NULL );
	         //  tcp_output(tpcb);
			    }
	     	if (err != ERR_OK)
	        {
	     	   //How much still needs to be written out..
	     		howMuchToStillWrite = 35 -j;
	     	}
		else
	     	{
	     		tcp_sent( tpcb, NULL );
	        	tcp_output(tpcb);

		    }

		/* free the received pbuf */
		pbuf_free(p);
	 return ERR_OK;
}

Hercules Window, I have attached. I am getting some constant ASCII values. Again i have pasted your modifies for cross verification.

 

 

[tcmichals]

Oh, the sprintf is using %x, was printing the address of the char * from itoa.  You can use sprintf function to convert the DestinationBuffer into a string.  I think DestinationBuffer is an array of integers? 

Also, added some debug statements  it is possible to see how the code works.  I think xil_printf is a function that is able to print messages out to the serial port.  If so, I typically use a simple DEBUG, __PRETTY_FUNCTION__ prints the current name of the function, __LINE__ is what line the code is at.   OK, so, the if statement loops 0, to 100 and if the TCP buffer larger then the buffer, then keep sending.  

I did not compile or test the code.  

 int howMuchToStillWrite = 0;
/* arg is passed what was assigned to the call back functions */

err_t sentFunction(void * arg, struct tcp_pcb * tpcb, u16_t len)
{
   char buffer[20];

    //debug, what is sending out the tcp buffer I think this allows you to see messages on the serial port? 
   xil_printf("DEBUG: %s %d len=%dn", __PRETTY__FUNCTION, __LINE__, len);

   if ( tcp_sndbuf(tpcb) > sizeof(buffer) &&  howMuchToStillWrite )
   {
     /* if DestinationBuffer is an array of integers, then use %d, this will format into a string
     ** using an %x was printing the address of the string 
     */
      int len = sprintf(outbuffer, "%d,", DestinationBuffer[j]);
      err = tcp_write(tpcb,outbuffer,len, 1);
      //debug, what is sending out the tcp buffer I think this allows you to see messages on the serial port? 
       xil_printf("DEBUG: Keep sending %s %d err=%d outbuffer:%s\n", __PRETTY__FUNCTION, __LINE__, err, outbuffer );
       if (err == ERR_OK)
       {
     	   //How much still needs to be written out.. 
     		--howMuchToStillWrite;
       }
       else
      {
        tcp_sent( tpcb, NULL );
        tcp_output(tpcb);
	 }
   }
}

err_t recv_callback(void *arg, struct tcp_pcb *tpcb,
                               struct pbuf *p, err_t err)
{
......
char buffer[20];

 for (int j=0; j <= 100 && (tcp_sndbuf(tpcb) > sizeof(buffer)); ++j)
 {
     /* if DestinationBuffer is an array of integers, then use %d, this will format into a string
     ** using an %x was printing the address of the string 
     */
	   int len = sprintf(outbuffer, "%d,", DestinationBuffer[j]);
   	   err = tcp_write(tpcb,outbuffer,len, 1);
   
       //debug, what is sending out the tcp buffer I think this allows you to see messages on the serial port? 
       xil_printf("DEBUG: %s %d %s\n", __PRETTY__FUNCTION, __LINE__, outbuffer );
   
       if (err != ERR_OK)
       {
     	   //How much still needs to be written out.. 
            
            howMuchToStillWrite = 100 -j;
            //debug, what is sending out the tcp buffer I think this allows you to see messages on the serial port? 
       		xil_printf("DEBUG: %s %d %d\n", __PRETTY__FUNCTION, __LINE__, howMuchToStillWrite );
            break;  //exit for loop, cannot send any more, having tcp_sendbuf, check
       }
 }
 ....
 }
     
     

Also, you may want to zero out howMuchToStillWrite if there is room in the buffer. 

[Thausikan]

Thanks . I am getting the data in Hercules only upto length 90. Afterthat in XSDK console, i am getting message :"no space in tcp_sndbuf".

I am unable to add this line code "xil_printf("DEBUG: %s %d %s\n", __PRETTY__FUNCTION, __LINE__, outbuffer )". I am getting an error message "undeclared". So, i have modified the code little bit.

static char buffer[50];

err_t recv_callback(void *arg, struct tcp_pcb *tpcb,
                               struct pbuf *p, err_t err)
{

	int i,j,Status;

	int len;
	Status=aurora_rx_main();         ///FUNCTION CALL
   
     for(i=0;i<50;i++)
        {
    		xil_printf("%d,", DestinationBuffer[i]);
    	}
	if (!p)
	{
		tcp_close(tpcb);
		tcp_recv(tpcb, NULL);
		return ERR_OK;
	}


	/* indicate that the packet has been received */
	tcp_recved(tpcb, p->len);

   for (j=0; j <= 50 && (tcp_sndbuf(tpcb) > sizeof(buffer)); ++j)
 {
    
	   len = sprintf(buffer, "%d,", DestinationBuffer[j]);
	   /*
	   xil_printf("\n");
	   xil_printf(" length : %d", len);
	   xil_printf(" buffer : %s", buffer);
	   */
   	   err = tcp_write(tpcb,buffer,len, 1);
   	    tcp_sent( tpcb, NULL );
   	   tcp_output(tpcb);

      
 }
   if(j<50)
     	   {
  	 		xil_printf("no space in tcp_sndbuf\n\r");
  		  //break;
     	   }

	/* free the received pbuf */
	pbuf_free(p);

	return ERR_OK;
}

err_t accept_callback(void *arg, struct tcp_pcb *newpcb, err_t err)
{
	static int connection = 1;

	/* set the receive callback for this connection */
	tcp_recv(newpcb, recv_callback);

	/* just use an integer number indicating the connection id as the
	   callback argument */
	tcp_arg(newpcb, (void*)connection);

	/* increment for subsequent accepted connections */
	connection++;

	return ERR_OK;
}

 

last value i am getting in negative. Why can clarify me ?

 

[tcmichals]

#define MAX_COUNT 50 
/* Note that MAX_COUNT if 50, so all indexing must be 0..49, which is 50 */
static char buffer[MAX_COUNT];

int howMuchToStillWrite = 0;
/* arg is passed what was assigned to the call back functions */

err_t sentFunction(void * arg, struct tcp_pcb * tpcb, u16_t len)
{

   if ( howMuchToStillWrite < MAX_COUNT  )
   {
      int len = sprintf(outbuffer, "%d,", DestinationBuffer[howMuchToStillWrite]);
      int bytesFree = tcp_sndbuf(tpcb);
      if (len > bytesFree)
      {
         xil_printf("DEBUG: Keep sending need bytes %d only have %d\n", __PRETTY__FUNCTION, __LINE__, len, bytesFree );
         return ERR_OK;
	  }

      err = tcp_write(tpcb,outbuffer,len, 1);
      //debug, what is sending out the tcp buffer I think this allows you to see messages on the serial port? 
       xil_printf("DEBUG: Keep sending %s %d err=%d outbuffer:%s\n", __PRETTY__FUNCTION, __LINE__, err, outbuffer );
       if (err == ERR_OK)
       {
     	   //How much still needs to be written out.. 
     		howMuchToStillWrite++;
       }
   }

  return ERR_OK;
}



err_t recv_callback(void *arg, struct tcp_pcb *tpcb,
                               struct pbuf *p, err_t err)
{

	int i,j,Status;

	int len;
	Status=aurora_rx_main();         ///FUNCTION CALL
   
    for(i=0;i<50;i++)
    {
    		xil_printf("%d,", DestinationBuffer[i]);
    }
                      
	if (!p)
	{
		tcp_close(tpcb);
		tcp_recv(tpcb, NULL);
        /* NEW */
       tcp_sent(newpcb, NULL);

		return ERR_OK;
	}
	/* indicate that the packet has been received */
	tcp_recved(tpcb, p->len);
   /* NEW */
      
   int bytesFree = tcp_sndbuf(tpcb);
   xil_printf("ERROR, %s:%d TCP xmit max size %d\n", __PRETTY_FUNCTION__, __LINE__, bytesFree);
      
   for (j=0; j < MAX_COUNT; ++j)
   {
	  int len = sprintf(buffer, "%d,", DestinationBuffer[j]);
                    
      /* sprintf will return the length of the string in buffer, will this fit into the tcp buffer? */
      
      if (len > bytesFree )
      {
         /* buffer will not fit into tcp buffer, is it the first attempt ? */
         if (j ==0)
         {
             /* if this is happening, now, need to create several global variables, one is a string buffer, then offset and length
             ** then in the sent_function keep sending the remainder of the string etc. */
             xil_printf("ERROR, cannot the string tcp buffer len %d buffer len %d\n",  bytesFree, len);
      		 howMuchToStillWrite = 0;
         }
      	 else
         {
      		howMuchToStillWrite = j;
        }
       /* exit the for loop because no room in tcp  */
        break;
      }
      
      /* should have room send, copy the buffer into a lwip buffer */
   	  err = tcp_write(tpcb,buffer,len, TCP_WRITE_FLAG_COPY);
      /* always check the return code */
      
      if ( err != ERR_OK)
      {
       		howMuchToStillWrite = 0; //clear this, but, this can cause issues if another read comes before this is cleared, use a queue */ 
   	   		tcp_output(tpcb); 
       }
       else
       {  
           /* this will indicate that no more can be sent */
            xil_printf("no space in tcp_sndbuf\n\r");
       		howMuchToStillWrite = j;
            /* exit the for loop */
       		break;
       }
  }
	/* free the received pbuf */
	pbuf_free(p);

	return ERR_OK;
}

err_t accept_callback(void *arg, struct tcp_pcb *newpcb, err_t err)
{
	static int connection = 1;

	/* set the receive callback for this connection */
	tcp_recv(newpcb, recv_callback);

	/* just use an integer number indicating the connection id as the
	   callback argument */
	tcp_arg(newpcb, (void*)connection);
       
    /* NEW, must register a callback when transmit is done */
    tcp_sent(newpcb, sentFunction);

	/* increment for subsequent accepted connections */
	connection++;

	return ERR_OK;
}

>Thanks . I am getting the data in Hercules only upto length 90. Afterthat in XSDK console, i am getting message :"no space in tcp_sndbuf".

Must register the sentFunction with lwIP newpcb using the tcp_sent function.  This, lwip will always do a callback when a transmit is complete.   

>, i am getting message :"no space in tcp_sndbuf".

Can be two issues, (1) the (tcp_sndbuf(tpcb) > sizeof(buffer)) is always failing (2) sending a few string then the tcp window is full.  

To debug issues like this always break up a complex conditional into one check.  See new code, the for statement will always loop 0, to 50.  the first if statement checks the result of the sprintf will fit into the tcp window, if not, check to see if it is the first loop, if it is, then no string can fit, and it must be broken into smaller chunks,this will require more code.  The current code assumes the sprintf will not overrun the 50 byte buffer.  There is a safe version of sprintf called snprintf, that checks to size of the buffer so an overflow does not occur.   The prior code was checking the tcp window buffer size against the size of buffer, i.e the tcp buffer must have at least 50 bytes free.  

Also, I would suggest to code the basics and debug on a PC (Windows or Linux)  This way you can single step and debug most of the code, yes, the functions might be a little different, but, you can get general code done.  Yes, there is no lwIP stack, but again, get the basics coded and tested.  

 

>I am unable to add this line code "xil_printf("DEBUG: %s %d %s\n", __PRETTY__FUNCTION, __LINE__, outbuffer )". I am getting an error message "undeclared". So, i have modified the code little bit.

To find out the error just remove one variable at a time, maybe it is outbuffer,  the printf uses the string with %s %d %s must match the variable arguments. ie, the first %s is __PRETTY_FUNCTION__ %d is __LINE__ %s is outbuffer.