Rothschopf Werner

rowex ganz privat

Angebote

A reliable Arduino Webserver and how to POST data

Examples of  how to use the  POST method with an Arduino are seldom. Most of the examples try to send data with GET. But data should'nt be send with GET. Use POST to post data!

First we will check why the minimal sketch in the IDE Ethernet | Webserver is just a start and what we can do better. Than we check the different methods to bring data to the webserver (GET vs. POST) and finally we learn how to serve more pages and even images on the Arduino. In total you will need aprox. 45 minutes to learn how to write a proper webserver for the Arduino.

One more thing ahead: This webserver works well for the Arduino UNO / Nano / Pro Mini and Arduino MEGA. If you are using an ESP8266 / ESP32 you don't have to do that all manually - there are existing libraries for ESP processors. Nevertheless - the basics are worth reading.

Requirements for our Arduino UNO Webserver

For our Arduino UNO Webserver we formulate following requirements:

  • be able to handle pages (handle GET Requests)
  • non existing pages should bring a HTTP 404 page
  • be able to receive data from the user (handle POST requests)
  • handle the browser favicon.ico correctly
  • as we are on a Arduino UNO: avoid any usage of Arduino-Strings
  • keep the loop() as tidy as possible, handle everything in separte functions

The IDE example Ethernet | Webserver

... and why it's a problematic ...

The example Ethernet | Webserver  provided in the IDE is only a minimal variant of a webserver. Every method, every input is accepted and leads to the output of a page. It uses the GET method to accept user data transfered from the browser to the Arduino.

I also recommend that you check the network traffic caused by the example (e.g. in Firefox under Tools | Web Developer | Network). The browser (like Firefox and others) not only requests the page from the address bar, but also a favicon.ico - a small graphic that (if available) is shown on the page tab in the browser.

Let's see what's happening:

The server simple answers each page request (GET) with the output of the HTML. When you check the serial monitor you see the two incoming requests:

A webserver should handle all requests accordingly.

Differences between POST and GET

Long story short:

GET requests append parameters to the URI/URL after an question mark (?). The user will see the data in the browser adress feld. GET requests are for retriving data, like "GET a resource from the server". For example "GET the page 1.htm". Yes you can handover parameters, but these should be related to the resource.

Der http-header for this kind of requests looks like:

GET /1.htm?content=1&anotherParameter=aText HTTP/1.1
HOST: 172.18.67.93
Content-Length: 0

Now let's talk about POST requests: these are used to transfer data from the user (browser) to the server.

In a POST request the data will be transmitted in the http-body of the request. The http-header ends with an empty line before the http-body starts:

GET /form.htm HTTP/1.1
HOST: 172.18.67.93
Content-Length: 24

content=1&anotherParameter=aText

You see, the browser transmits the same kind of parameters, the question mark is not needed.

Therefore a webserver must check the whole message: Check if it is a GET or a POST request, retrive the requested URI and any GET parameters. In case of a POST request, read through the http-header up to the empty empty line and than read the message for any post parameters.

How to implement an Arduino UNO Webserver accepting POST data

To send POST data, we use the Arduino UNO itself and host an input form that sends data to the Arduino using POST. In theory, another client could also send data to our Arduino web server using POST. For an example implementation, we make a simple Arduino webserver with a POST form that can switch some LEDs on and off using form buttons.

The setup() can essentially be taken from the IDE example. The loop() only contains the current function call which is responsible for our webserver:

#include <SPI.h>
#include <Ethernet.h>

byte mac[] { 0x52, 0x64, 0x75, 0x69, 0x6E, 0x6F }; // you can change the MAC and IP addresses to suit your network
IPAddress ip( 172, 18, 67, 93 );
EthernetServer server(80); // Port 80 is HTTP port
const byte ledPin[] {2, 3, 4, 5, 6, 7, 8, 9}; // define the LED pins. Don't use 0 (RX), 1(TX), 10 (SS ETH), 11 (MOSI), 12(MISO), 13(CLK). Be carefull with 4(SS SD).

void setup()
{
  Serial.begin(115200);
  Serial.println(F("\nsimple webserver"));
  for (auto &pin : ledPin) {
    pinMode(pin, OUTPUT);  // Set the digital pins ready to write to
  }
  Ethernet.begin(mac, ip); // Start the Ethernet shield
  printHardwareInfo();
  server.begin();          // Start the webserver
  Serial.print(F("server available at http://"));
  Serial.println(Ethernet.localIP());
}

void loop()
{
  checkForClient();
}

That's all. You have to define MAC and IP, an array with the GPIO pins for the LEDs, define them as output and use the begin methods for the network-shield.

We put all the server code in a separte tab. So use "New Tab" and name the tab "server". Create a new function and put all the previous loop() code from the example to your function

void checkForClient();
{
  // put all the loop() code from the IDE example in here
  ...
}

For first testing, you can compile the sketch an upload it to your Arduino - it should work like the original sketch.

Designing our own HTML page on the Arduino UNO/NANO/MEGA

Our HTML page - keep it small and simple - will be in a separate function. First, we still need the http-header which returns a http-response code 200. Since we do not send any content length, we use HTTP / 1.0. After the blank line, we send the http-body - that is the HTML page we want see in the browser window. It is a simple HTML5 page with an input form and two buttons for each pin from the ledPin array. To save precious RAM we store the page in flash/program memory. An easy way to do this is using the F-Makro. You can use the F-Makro for several lines. Don't use client.print for each line as this will split the transmitted packages to the browser. If you want to add an linefeed in your HTML source, add an \n to your line. Use as less client.print as possible:

void sendPage(EthernetClient &client)
{
  // Serial.println("[server] 200 response send");
  client.println(F("HTTP/1.0 200 OK\r\n" // \r\n Header Fields are terminated by a carriage return (CR) and line feed (LF) character sequence.
                   "Content-Type: text/html\r\n" // The Media type of the body of the request (used with POST and PUT requests).
                   "\r\n" // a blank line to split HTTP header and HTTP body
                   "<!doctype html>\n" // the HTTP Page itself
                   "<html lang='en'>\n"
                   "<head>\n"
                   "<meta charset='utf-8'>\n"
                   "<meta name='viewport' content='width=device-width'>\n"
                   "<title>Webserver as pin controller</title>\n"
                   "</head>\n"
                   "<body style='font-family:Helvetica, sans-serif'>\n" // a minimum style to avoid serifs
                   "<h1>Webserver as pin controller</h1>\n"
                   "<p>Buttons turn pins on or off</p>\n"
                   "<form method='post' action='/' name='pins'>"));
  for (auto &pin : ledPin)
  {
    //client.print(F("<p>"));
    client.print(pin);
    client.print(F(" <input name='pinD"));
    client.print(pin);
    client.print(F("' type='submit' value='On'>"));
    client.print(F("<input name='pinD"));
    client.print(pin);
    client.print(F("' type='submit' value='Off'>"));
    if (digitalRead(pin)) client.print(F(" active"));
    client.print(F( "<br>\n"));
  }
  client.print (F("</form>\n"));

  client.print (F("</body>\n</html>"));
  client.stop();
}

The body-style is optional, it just uses fonts without serifes.

The sketch will not work currently, but as a teaser, we will get something like this:

To read the http-header and http-body we will use a state machine. To avoid magic numbers in the code we use an enum class with this four states:

  • REQUEST ... read the method (GET or PRINT), the URI and the GET parameters
  • CONTENT_LENGTH ... read the request length
  • EMPTY_LINE ... await the empty line
  • BODY ... read data from the the HTTP-body

This is a simple "enumeration class" and we need a status variable to store the actual status, and we start with the status "REQUEST":

enum class Status {REQUEST, CONTENT_LENGTH, EMPTY_LINE, BODY};
Status status = Status::REQUEST;

So let's adopt the IDE example. We read line by line. When we have received an "end of line" we handle according your state machine.

while (client.connected()) {
  while (client.available()) {
    char c = client.read();
    Serial.print(c); 
    if ( c == '\n' )
    {
      if (status == Status::REQUEST)
      {
        status = Status::EMPTY_LINE; 
      }
      else if (status == Status::CONTENT_LENGTH)
      {
        status = Status::EMPTY_LINE;
      }
      else if (status > Status::REQUEST && i <= 2) // check if we have an empty line
      {
        status = Status::BODY;
      }
      else if (status == Status::BODY)
      {
        strlcpy(postParameter, lineBuffer, smallbuffersize);
        break; // we have received one line payload and break out
      }
      i = 0;
      strcpy(lineBuffer, "");
    }
    else
    {
    if (i < buffersize)
    {
      lineBuffer[i] = c;
      i++;
      lineBuffer[i] = '\0';
    }
  // MISSING: for future use: read content length
  }
}
if (status == Status::BODY) // status 3 could end without linefeed, therefore we takeover here also
{
  strlcpy(postParameter, lineBuffer, smallbuffersize);
}

The first line is very complex. The HTTP request begins with the method, a blank separates the URI with any parameters and finally a blank again and the distinction whether it is an HTTP / 1.0 or HTTP / 1.1 request. We are only interested in the method, the URI and any parameters. We use strtok to break the lines in it's elements.

if (status == Status::REQUEST) // read the first line
{
  //Serial.print(F("lineBuffer="));Serial.println(lineBuffer);
  // now split the input
  char *ptr;
  ptr = strtok(lineBuffer, " ");          // strtok willdestroy the newRequest
  strlcpy(method, ptr, smallbuffersize);
  Serial.print(F("method=")); Serial.println(method);
  ptr = strtok(NULL, " ");
  strlcpy(uri, ptr, smallbuffersize);     
  if (strchr(uri, '?') != NULL)
  {
    ptr = strtok(uri, "?");               // split URI from parameters
    strcpy(uri, ptr);
    ptr = strtok(NULL, " ");
    strcpy(requestParameter, ptr);
    Serial.print(F("requestParameter=")); Serial.println(requestParameter);
  }
  Serial.print(F("uri=")); Serial.println(uri);

The empty line can be recognized with an LF (line feed) in the first 2 characters. Be carefully: HTTP should send CR+LF after each header-request field.

      else if (status > Status::REQUEST && i < 2) // check if we have an empty line
      {
        status = Status::BODY;
      }

The parameters in the body can be splitted with strtok. If you are using very simple unique names, you can even split them with a simple strncmp:

if ( strncmp( postParameter, "pinD", 4) == 0 ) {
  byte pin = postParameter[4] - 48; // simple convert the 5 character (index 4) from ASCII to int
  if ( strncmp( postParameter + 5, "=On", 3) == 0 ) {
    digitalWrite(pin, 1);
  }
  else if ( strncmp( postParameter + 5, "=Off", 4) == 0 ) {
    digitalWrite(pin, 0);
  }
}

Depending on the button you press, the GPIO pin will be switched on or off.

Not to forget - the browser ist still waiting for an answer of the webserver. In our example the server should response to the browser a new - updated - page. For usual the comparison for the page "/" should be enough, but we could also add "/index.htm" as valid page.

 // send back a response

if (!strcmp(uri, "/") || !strcmp(uri, "/index.htm"))
  sendPage(client);

Perfect... but...: If the user enters a wrong URI, the browser will get no answer and will run into a timout. Therefore we have to handle this case with a simple HTTP-Error 404 page. Let's add an else to our if:

else 
  send404(client);

and generate a simple page: sending a response code 404 (for the browser) and an explanation for the user:

void send404(EthernetClient &client)
{
  client.println(F("HTTP/1.0 404 Not Found\r\n"
                   "Content-Type: text/plain\r\n"
                   "\r\n"
                   "File Fot Found\n"));
  client.stop();
}

As an example - we are sending a simple text/plain page - without any HTML tags. But it is up to you to send HTTP also.

More pages on the Arduino UNO Webserver

If you want to serve more pages on your webserver just repeat the steps:

  • define a function serving the page, remember to use the F-Makro to save RAM
  • add more pages to your if statement which handels the URI
else if (!strcmp(uri, "/1.htm") 

In the end you will get a nice working webserver on your Arduino UNO.

Handling the favicon.ico with the Arduino Webserver

Don't forget the favicon.ico. Currently our Arduino webserver responses requests for the favicon.ico like following:

The server will get a valid 404-error. Let's see how we could do improve this. Its already a lot less transfered bandwith than sending the same page twice.

As we learned already, the first step is to add a new handler in our URI if:

else if (!strcmp(uri, "/favicon.ico")) // a favicon
  send204(client); 

The HTTP-Response 204 informs the browser, that our webserver has received the request successfully, but doesn't return any data. You just need two lines of code:

void send204(EthernetClient &client)
{
  client.println(F("HTTP/1.0 204 no content\r\n"));
  client.stop();
}

And now you even see a perfect green OK in your browsers netmonitor:

IMHO is response code 204 a very good solution for the limited resources on an Arduino UNO webserver. Again smart and simple while keeping an eye on RAM and flash usage.

Serving a favicon.ico on the Arduino Webserver

Ok, but just in case we want to serve a fancy favicon.ico on the Arduino...

One option is to convert the picture into Base64 and serve the picture within the HTML-Header. There are already examples existing - and can be googled.

Let's try a different approach: serve the picture as separete "file" in the Arduino flash (program) memory:

Again, we add an "else if" for our new resource:

else if (!strcmp(uri, "/favicon.ico"))
  sendFavicon(client);

Then, we need a new function sendFavicon(). The HTTP-header "Content-Type" for a favicon.ico is

Content-Type: image/x-icon

To get the proper picture I'm using this workflow:

  • design the favicon and store it on the local hard disk
  • use an online hexeditor to convert the image into hex picture data
  • copy the hex data into a good editor (like notepad++) and replace all blanks with commas

At the end of the page you will find links to tools I'm using.

The hex data can be used in our Arduino sketch. Just be very carefully, that even for a 16x16 favicon you will need 257 bytes of memory. This will be to much for the limited RAM of the Arduino UNO. Therefore we put the whole array into PROGMEM (flash) and are reading the data if needed.

void sendFavicon(EthernetClient &client)
{
  const static byte tblFavicon[] PROGMEM = {
  0x00, 0x00, 0x01, 0x00, 0x01, 0x00, 0x10, 0x10, 0x10, 0x00, 0x01, 0x00, 0x04, 0x00, 0x28, 0x01,
  0x00, 0x00, 0x16, 0x00, 0x00, 0x00, 0x28, 0x00, 0x00, 0x00, 0x10, 0x00, 0x00, 0x00, 0x20, 0x00,
  0x00, 0x00, 0x01, 0x00, 0x04, 0x00, 0x00, 0x00, 0x00, 0x00, 0x80, 0x00, 0x00, 0x00, 0x00, 0x00,
  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x10, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
  0x00, 0x00, 0xa8, 0xa8, 0xa8, 0x00, 0x68, 0x64, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01, 0x11,
  0x11, 0x11, 0x11, 0x11, 0x11, 0x10, 0x11, 0x11, 0x11, 0x11, 0x11, 0x11, 0x11, 0x11, 0x11, 0x11,
  0x12, 0x22, 0x12, 0x22, 0x11, 0x11, 0x11, 0x11, 0x11, 0x12, 0x11, 0x12, 0x11, 0x11, 0x11, 0x11,
  0x12, 0x22, 0x11, 0x22, 0x11, 0x11, 0x11, 0x11, 0x12, 0x12, 0x11, 0x12, 0x11, 0x11, 0x11, 0x11,
  0x12, 0x22, 0x12, 0x22, 0x11, 0x11, 0x11, 0x11, 0x11, 0x11, 0x11, 0x11, 0x11, 0x11, 0x11, 0x11,
  0x11, 0x11, 0x11, 0x11, 0x11, 0x11, 0x11, 0x22, 0x11, 0x21, 0x12, 0x11, 0x22, 0x11, 0x12, 0x11,
  0x21, 0x21, 0x12, 0x12, 0x11, 0x21, 0x12, 0x11, 0x21, 0x21, 0x22, 0x12, 0x11, 0x21, 0x12, 0x11,
  0x21, 0x22, 0x12, 0x12, 0x11, 0x21, 0x12, 0x11, 0x21, 0x21, 0x12, 0x11, 0x22, 0x11, 0x11, 0x11,
  0x11, 0x11, 0x11, 0x11, 0x11, 0x11, 0x01, 0x11, 0x11, 0x11, 0x11, 0x11, 0x11, 0x10, 0x80, 0x01,
  0x00
  };

  client.print(F("HTTP/1.0 200 OK\r\n"
                 "Content-Type: image/x-icon\r\n"
                 "\r\n"));
  for (uint16_t i = 0; i < sizeof(tblFavicon); i++)
  {
    byte p = pgm_read_byte_near(tblFavicon + i);
   client.write(p);
  }
  client.stop();
}

Using this method, the favicon (or any other small picture) will even fit into an Arduino UNO. If you check your browser: the favicon.ico will now be displayed:

und also check the Network Monitor everything is working ok now:

Serving the icon as separate file has the advantage, that the browser could cache it locally and the favicon.ico could be re-used for all pages you are serving on your Arduino. The HTML pages might be smaller and therefore transmitted faster from the UNO to the browser.

Summary

We have learned how to set up a minimalistic webserver on an Arduino UNO, how to serve different kind of data on the UNO and how to POST data to the Arduino. Our webserver handels requests correctly and answers wrong requests with HTTP-Code 404 or any other code we define in the code. In the end we can even handle the favicon.ico for the browser.

If you have any questions to the Arduino Webserver - best place to ask is in the Arduino.cc forum.


Links


Protokoll

First upload: 2020-03-09 | Version: 2020-06-23