A reliable Arduino Webserver and how to POST data

Examples of  how to use the  POST method with an Arduino are seldom. Most of the examples try to send data with GET. But data shouldn't be send with GET. Use POST to post data!

First we will check why the minimal sketch in the IDE Ethernet | Webserver is just a start and what we can do better. Than we check the different methods to bring data to the webserver (GET vs. POST) and finally we learn how to serve more pages and even images on the Arduino. In total you will need approx. 45 minutes to learn how to write a proper webserver for the Arduino.

One more thing ahead: This webserver works well for the Arduino UNO / Nano / Pro Mini and Arduino MEGA. If you are using an ESP8266 / ESP32 you don't have to do that all manually - there are existing libraries for ESP processors. Nevertheless - the basics are worth reading.

Requirements for our Arduino UNO Webserver

For our Arduino UNO Webserver we formulate following requirements:

  • be able to handle pages (handle GET Requests)
  • non existing pages should bring a HTTP 404 page
  • be able to receive data from the user (handle POST requests)
  • handle the browser favicon.ico correctly
  • as we are on a Arduino UNO: avoid any usage of Arduino-Strings
  • keep the loop() as tidy as possible, handle everything in separate functions

The IDE example Ethernet | Webserver

... and why it's a problematic ...

The example Ethernet | Webserver  provided in the IDE is only a minimal variant of a webserver. Every method, every input is accepted and leads to the output of a page. It uses the GET method to accept user data transferred from the browser to the Arduino.

I also recommend that you check the network traffic caused by the example (e.g. in Firefox under Tools | Web Developer | Network). The browser (like Firefox and others) not only requests the page from the address bar, but also a favicon.ico - a small graphic that (if available) is shown on the page tab in the browser.

Let's see what's happening:

The server simple answers each page request (GET) with the output of the HTML. When you check the serial monitor you see the two incoming requests:

A webserver should handle all requests accordingly.

Differences between POST and GET

Long story short:

GET requests append parameters to the URI/URL after an question mark (?). The user will see the data in the browser address field. GET requests are for retrieving data, like "GET a resource from the server". For example "GET the page 1.htm". Yes you can handover parameters, but these should be related to the resource.

The http-header for this kind of requests looks like:

GET /1.htm?content=1&anotherParameter=aText HTTP/1.1
HOST: 172.18.67.93
Content-Length: 0

Now let's talk about POST requests: these are used to transfer data from the user (browser) to the server.

In a POST request the data will be transmitted in the http-body of the request. The http-header ends with an empty line before the http-body starts:

GET /form.htm HTTP/1.1
HOST: 172.18.67.93
Content-Length: 24

content=1&anotherParameter=aText

You see, the browser transmits the same kind of parameters. There is no question mark is needed at the beginning.

Therefore a webserver must check the whole message: Check if it is a GET or a POST request, retrieve the requested URI and any GET parameters. In case of a POST request, read through the http-header up to the empty empty line and than read the message for any POST parameters.

How to implement an Arduino UNO Webserver accepting POST data

To send POST data, we use the Arduino UNO itself and host an input form that sends data to the Arduino using POST. In theory, another client could also send data to our Arduino web server using POST. For an example implementation, we make a simple Arduino webserver with a POST form that can switch some LEDs on and off using form buttons. That's very similar to what the IDE example does.

The setup() can essentially be taken from the IDE example Ethernet | webserver. The loop() only contains an new function call which is responsible for our webserver:

#include <SPI.h>
#include <Ethernet.h>

byte mac[] { 0x52, 0x64, 0x75, 0x69, 0x6E, 0x6F }; 
IPAddress ip( 172, 18, 67, 93 );
EthernetServer server(80); // Port 80 is HTTP port
const byte ledPin[] {2, 3, 4, 5, 6, 7, 8, 9}; 

void setup()
{
  Serial.begin(115200);
  Serial.println(F("\nsimple webserver"));
  for (auto &pin : ledPin) {
    pinMode(pin, OUTPUT);  // Set the digital pins ready to write to
  }
  Ethernet.begin(mac, ip); // Start the Ethernet shield
  printHardwareInfo();
  server.begin();          // Start the webserver
  Serial.print(F("server available at http://"));
  Serial.println(Ethernet.localIP());
}

void loop()
{
  checkForClient();
}

That's all. You have to define MAC and IP, an array with the GPIO pins for the LEDs, define them as output and use the begin methods for the network-shield.

We put all the server code in a separate tab. So use "New Tab" and name the tab "server". Create a new function and put all the previous loop() code from the example to your function

void checkForClient();
{
  // put all the loop() code from the IDE example in here
  ...
}

For first testing, you can compile the sketch an upload it to your Arduino - it should work like the original sketch.

Designing our own HTML page on the Arduino UNO/NANO/MEGA

Our HTML page - keep it small and simple - will be in a separate function. First, we still need the http-header which returns a http-response code 200. Since we do not send any content length, we use HTTP / 1.0. After the blank line, we send the http-body - that is the HTML page we want see in the browser window. It is a simple HTML5 page with an input form and two buttons for each pin from the ledPin array. Don't use client.print for each line as this will split the transmitted packages to the browser. If you want to add an linefeed in your HTML source, add \r\n to your line. Use as less client.print as possible:

void sendPage(EthernetClient &client)
{
  // Serial.println("[server] 200 response send");
  client.println(  "HTTP/1.0 200 OK\r\n" 
                   "Content-Type: text/html\r\n" 
                   "\r\n" // a blank line to split HTTP header and HTTP body
                   "<!doctype html>\n" // the HTTP Page itself
                   "<html lang='en'>\n"
                   "<head>\n"
                   "<meta charset='utf-8'>\n"
                   "<meta name='viewport' content='width=device-width'>\n"
                   "<title>Webserver as pin controller</title>\n"
                   "</head>\n"
                   "<body style='font-family:Helvetica, sans-serif'>\n" 
                   "<h1>Webserver as pin controller</h1>\n"
                   "<p>Buttons turn pins on or off</p>\n"
                   "<form method='post' action='/' name='pins'>");
  for (auto &pin : ledPin)
  {
    //client.print(  "<p>" );
    client.print(pin);
    client.print(  " <input name='pinD" );
    client.print(pin);
    client.print(  "' type='submit' value='On'>" );
    client.print(  "<input name='pinD" );
    client.print(pin);
    client.print(  "' type='submit' value='Off'>" );
    if (digitalRead(pin)) client.print( " active" );
    client.print(  "<br>\n" );
  }
  client.print ( "</form>\n" );

  client.print ( "</body>\n</html>" );
  client.stop();
}

Note: Do not use the F-Makro in combination with client.print because it will slow down your server response. I will show later an option how too save precious RAM and get a fast server.

The body-style is optional, it just uses fonts without serifes.

The sketch will not work currently, but as a teaser, we will get something like this:

To read the http-header and http-body you will use a state machine. To avoid magic numbers in the code you use an enum class with this four states:

  • REQUEST ... read the method (GET or PRINT), the URI and the GET parameters
  • CONTENT_LENGTH ... read the request length
  • EMPTY_LINE ... await the empty line
  • BODY ... read data from the the HTTP-body

This is a simple "enumeration class" and you need a status variable to store the actual status, and you start with the status "REQUEST":

enum class Status {REQUEST, CONTENT_LENGTH, EMPTY_LINE, BODY};
Status status = Status::REQUEST;

So adopt the IDE example. Read line by line. When you have received an "end of line" handle according your state machine.

while (client.connected()) {
  while (client.available()) {
    char c = client.read();
    Serial.print(c); 
    if ( c == '\n' )
    {
      if (status == Status::REQUEST)
      {
        status = Status::EMPTY_LINE; 
      }
      else if (status == Status::CONTENT_LENGTH)
      {
        status = Status::EMPTY_LINE;
      }
      else if (status > Status::REQUEST && i <= 2) // check if we have an empty line
      {
        status = Status::BODY;
      }
      else if (status == Status::BODY)
      {
        strlcpy(postParameter, lineBuffer, smallbuffersize);
        break; // we have received one line payload and break out
      }
      i = 0;
      strcpy(lineBuffer, "");
    }
    else
    {
    if (i < buffersize)
    {
      lineBuffer[i] = c;
      i++;
      lineBuffer[i] = '\0';
    }
  // MISSING: for future use: read content length
  }
}
if (status == Status::BODY) // status 3 could end without linefeed, therefore we takeover here also
{
  strlcpy(postParameter, lineBuffer, smallbuffersize);
}

The first line is very complex. The HTTP request begins with the METHOD, a blank separates the URI with any parameters and finally a blank again and the distinction whether it is a HTTP/1.0 or HTTP/1.1 request. You are only interested in the METHOD, the requested URI and any parameters. Use strtok to break the lines in it's elements.

if (status == Status::REQUEST) // read the first line
{
  //Serial.print(F("lineBuffer="));Serial.println(lineBuffer);
  // now split the input
  char *ptr;
  ptr = strtok(lineBuffer, " ");          // strtok willdestroy the newRequest
  strlcpy(method, ptr, smallbuffersize);
  Serial.print(F("method=")); Serial.println(method);
  ptr = strtok(NULL, " ");
  strlcpy(uri, ptr, smallbuffersize);     
  if (strchr(uri, '?') != NULL)
  {
    ptr = strtok(uri, "?");               // split URI from parameters
    strcpy(uri, ptr);
    ptr = strtok(NULL, " ");
    strcpy(requestParameter, ptr);
    Serial.print(F("requestParameter=")); Serial.println(requestParameter);
  }
  Serial.print(F("uri=")); Serial.println(uri);

The empty line can be recognized with an LF (line feed) in the first 2 characters. Be carefully: HTTP should send CR+LF after each header-request field.

      else if (status > Status::REQUEST && i < 2) // check if we have an empty line
      {
        status = Status::BODY;
      }

The parameters in the body can be splitted with strtok. If you are using very simple unique names, you can even split them with a simple strncmp:

if ( strncmp( postParameter, "pinD", 4) == 0 ) {
  byte pin = postParameter[4] - 48; // simple convert the 5 character (index 4) from ASCII to int
  if ( strncmp( postParameter + 5, "=On", 3) == 0 ) {
    digitalWrite(pin, 1);
  }
  else if ( strncmp( postParameter + 5, "=Off", 4) == 0 ) {
    digitalWrite(pin, 0);
  }
}

Depending on the button you press, the GPIO pin will be switched on or off.

Don't forget - the browser is still waiting for an answer of the webserver. In the example the server should response to the browser a new - updated - page. For usual the comparison for the page "/" should be enough, but we could also add "/index.htm" as valid page. In other words, the webserver will send the same page regardless if / or index.htm was requested:

 // send back a response

if (!strcmp(uri, "/") || !strcmp(uri, "/index.htm"))
  sendPage(client);

Perfect! But... if the user enters a wrong URI, the browser will get no answer and will run into a timeout. Therefore we have to handle this case with a simple HTTP-Error 404 page. Let's add an else to our if:

else 
  send404(client);

and generate a simple page: sending a response code 404 (for the browser) and an explanation for the user:

void send404(EthernetClient &client)
{
  client.println("HTTP/1.0 404 Not Found\r\n"
                   "Content-Type: text/plain\r\n"
                   "\r\n"
                   "File Fot Found\n");
  client.stop();
}

As an example - we are sending a simple text/plain page - without any HTML tags. But it is up to you to send HTTP also.

More pages on the Arduino UNO Webserver

If you want to serve more pages on your webserver just repeat the steps:

  • define a function serving the page, remember to use the F-Makro to save RAM
  • add more pages to your if statement which handels the URI
else if (!strcmp(uri, "/1.htm") 

In the end you will get a nice working webserver on your Arduino UNO.

Handling the favicon.ico with the Arduino Webserver

Don't forget the favicon.ico. Currently our Arduino webserver responses requests for the favicon.ico like following:

The server will get a valid 404-error. Let's see how we could do improve this. Its already a lot less transfered bandwith than sending the same page twice.

As we learned already, the first step is to add a new handler in our URI if:

else if (!strcmp(uri, "/favicon.ico")) // a favicon
  send204(client); 

The HTTP-Response 204 informs the browser, that our webserver has received the request successfully, but doesn't return any data. You just need two lines of code:

void send204(EthernetClient &client)
{
  client.println("HTTP/1.0 204 no content\r\n");
  client.stop();
}

And now you even see a perfect green OK in your browsers netmonitor:

IMHO is response code 204 a very good solution for the limited resources on an Arduino UNO webserver. Again smart and simple while keeping an eye on RAM and flash usage.

If you want to read how to serve a favicon.ico on the Arduino webserver, you might read my page Arduino Webserver with favicion.ico

Summary

You have learned how to set up a minimalist webserver on an Arduino UNO, how to serve different kind of data on the UNO and how to POST data to the Arduino. Your webserver handles requests correctly and answers wrong requests with HTTP-Code 404 or any other code you define in the code. Finally you can even handle the favicon.ico correctly for the browser.

If you have any questions to the Arduino Webserver - best place to ask is in the Arduino.cc forum.

Links

Protocol

First upload: 2020-03-09 | Version: 2020-11-14