Optimize the Arduino Webserver
Some time ago I showed already my version of an Arduino webserver. In this tutorial I would like to shed a light on how to optimize the Arduino webserver.
I will cover following targets:
- Reduction of used program memory (Flash Memory or PROGMEM) - the value shown by the compiler under "This sketch uses "
- Low usage of Static Memory in SRAM - the value shown as "globals"
- reduced TCP/IP overhead to ensure fast transmission times
Where to start...
Let's start with a simple HTTP page in plain text. The payload is 2 x 30 characters. If you include the trailing CR/LF the server has to transmit a payload of 64 bytes. This function will serve as basic example for all further tests.
void page1(EthernetClient &client)
{
client.println("HTTP/1.0 200 OK");
client.println("Content-Type: text/plain");
client.println();
client.println("123456789a123456789b123456789c");
client.println("Output with 5 println.........");
}
Depending on your LAN speed the transfer will last approx. 50ms. If you check the communication in Wireshark you will see that the server will transmit 14 packets:
If you enlarge the picture you will see, that I only display the sourcecode to avoid that the browser requests a favicon.ico. In Wireshark you see only the packets transmitted from the Arduino Webserver to the browser for the transmitted text.
Reduce client.print()
Each client.print initiates a separate communication between client and server. Therefore you should reduce the used client.print(). If you need separated lines, insert the \r\n manually:
void page2(EthernetClient &client)
{
client.print ("HTTP/1.0 200 OK\r\n"
"Content-Type: text/plain\r\n"
"\r\n"
"123456789a123456789b123456789c\r\n"
"Output with 1 print ..........\r\n");
}
The transmission needs only 35ms and uses 6 packets:
Remember: The less client.print() you use, the less program memory you will need. Additionally you reduce TCP traffic. If you check the numbers very carefully you will see a slightly higher demand of SRAM (globals). This is due to the additional \r\n which were needed as otherwise this is done by each client.println(). Further on, we will cover that aspect also.
The F-Macro and why you should NOT use it with client.print
The F-Macro is used heavily for fix text. This works perfect for the serial outputs, for a LCD or similar. The F-Macro prevents that the text will be copied into static memory (globals). Unfortunately this doesn't work well with the Ethernet client.print() on the Arduino. Due to implementation of client.print() each character within a F-Macro text will be sent as individual TCP packet. This will raise the traffic overhead and with larger pages you will recognize longer loading times in your browser.
// just as demonstration
// don't use the F-Makro with client.print in production!
void page3(EthernetClient &client)
{
client.print(F("HTTP/1.0 200 OK\r\n"
"Content-Type: text/plain\r\n"
"\r\n"
"123456789a123456789b123456789c\r\n"
"Output with F-Makro...........\r\n"));
}
You still send 64 bytes in the payload, but you can see already the longer transmission time in your browser. Wireshark reports 114 packets from the Arduino to the browser, and this explains the longer response time.
Remember: Don't use the F-Macro for client.print()!
The F-Macro still reduces the global SRAM (as the text isn't copy into SRAM) but has huge disadvantages regarding transmission overhead and speed. Let's see, if we have other options.
strcpy_P, PSTR and a temporary Buffer
So how can we avoid the permanent block of SRAM? Just introduce a temporary buffer! strcpy_P allows you to copy data from program memory to you variable:
void page4(EthernetClient &client)
{
char buffer[128] {'\0'};
strcpy_P(buffer, PSTR("HTTP/1.0 200 OK\r\n"
"Content-Type: text/plain\r\n"
"\r\n"
"123456789a123456789b123456789c\r\n"
"Buffer in one Line............\r\n"));
client.print(buffer);
}
As you still have only one client.print the transmission is done in 6 packets:
If you need to combine the output in several lines of code you can use strcat_P to concatenate (append) data from program memory to your buffer variable:
void page5(EthernetClient &client)
{
char buffer[128] {'\0'};
strcpy_P(buffer, PSTR("HTTP/1.0 200 OK\r\n"
"Content-Type: text/plain\r\n"
"\r\n"));
strcat_P(buffer, PSTR("123456789a123456789b123456789c\r\n"));
strcat_P(buffer, PSTR("Buffer with several lines.....\r\n"));
client.print(buffer);
}
The transmission time is still short, because you are using just one client.print():
but...
Pay attention to the size of your buffer:
- During runtime (when the function is processing) you need enough free stack memory in SRAM. Nevertheless - it will not block static memory (globals).
- You must not override the size of your buffer (including the final NULL terminator!). If you need to serve larger pages you either have to increase the buffer (in the example 128 bytes) or you have to send if you reach the limit of your buffer, do a client.print() and then start again collecting data into the buffer.
Remember:
A temporary buffer is a very efficient way to transmit data with the Arduino
Ethernet client, but needs special care regarding size of the buffer. It is the version with
minimalistic SRAM (globals) usage.
The StreamLib Library
If you have larger pages the manual buffer might get cumbersome. For larger pages I recommend to use the StreamLib library from Juraj Andrássy, which can be installed via the IDE library manager. This StreamLib library gives a simple access to a buffer. You can "print" to this buffer like you are used to with client/serial/LCD before. You can use all known formats which are supported by print including the F-Macro. If you have finished your output, you finalize with the .flush() method and your output will be sent to your destination. If the buffer reaches the buffer size, the library will initiate the send, empties the buffer and will be ready to receive more characters.
An example function could look like:
#include <StreamLib.h>
void page6(EthernetClient &client)
{
const size_t MESSAGE_BUFFER_SIZE = 64;
char buffer[MESSAGE_BUFFER_SIZE]; // a buffer needed for the StreamLib
BufferedPrint message(client, buffer, sizeof(buffer));
message.print(F("HTTP/1.0 200 OK\r\n"
"Content-Type: text/plain\r\n"
"\r\n"
"123456789a123456789b123456789c\r\n"
"Streamlib with F-Makro........\r\n"));
message.flush();
}
You define a buffer. Then you create an object (in the example message). The constructor takes 3 parameters:
- a reference to your Ethernet client. This Ethernet client will be used if the buffer gets full or when you manually end with .flush()
- a reference to your buffer
- the size of your buffer (use a simple sizeof for that)
Now you can use several steps to "print" into your object like you are used to with client.print(). The StreamLib will collect your data and will send if the buffer reaches its limits or if you finalize your output with .flush().
If you had already large pages on your Arduino webserver with client.print(), than it's very easy to replace the "client" by "message". Just don't forget to finalize the transmission with message.flush().
void page7(EthernetClient &client)
{
const size_t MESSAGE_BUFFER_SIZE = 64;
char buffer[MESSAGE_BUFFER_SIZE]; // a buffer needed for the StreamLib
BufferedPrint message(client, buffer, sizeof(buffer));
message.print(F("HTTP/1.0 200 OK\r\n"
"Content-Type: text/plain\r\n"
"\r\n"));
message.print(F("123456789a123456789b123456789c\r\n"));
message.print(F("StreamLib with several print..\r\n"));
message.flush();
}
The StreamLib library will need some additional program memory for the object, but as you can save a tons of SRAM using the F-Makro it's worth to implement.
Summary
Most of the Arduino Webservers can be optimized. There is plenty of space on an Arduino UNO and you don't have to waive big HTML pages, CSS, JavaScript. Reduce lines of code, reduce client.print. This not only helps you to reduce Flash/PROGMEM but also reduces TCP/IP traffic and you will gain increased transmission time.
Even if the F-Macro comes with lot of advantages - don't use it with Ethernet client.print(). Either use a manual buffer before printing or use the StreamLib library which makes handling of the buffer very easy.