Arduino Webserver with SD Card Logging and NTP
I haven't found so many examples with an Arduino Webserver and the parallel usage of the SD Card, so I want to show my Version of this Sketch.
The basic idea is to display on the Arduino UNO web interface some data and log data on the SD Card. As we would need a time stamp for the log file, we will let the Arduino retrieve the current time from a NTP server with SNTP. As we have already a web interface it would be nice if we could download the collected data from the Arduino with the web browser.
Beside the main program, the sketch is organized in several tabs: NTP, SD and Server.
The Main Tab
There is nothing special in the main tab. We need some includes. If not already done, install the StreamLib library and the TimeLib library.
The Ethernet shield needs some definitions. Adopt the IP address to your home network.
The arrays for the digital pins and analog pins are just for this example. You can modify them to your need.
The setup() starts the Ethernet shield, initializes the SD Card and starts the UDP for NTP.
The loop() is short - it has only 3 functions - but it describes very well what the sketch is doing:
void loop()
{
checkForClient(); // listen for incoming clients - the "webserver"
timerSD(); // checks if it is time to write data to SD Card
timerNTP(false); // checks if it is time to sync with NTP
}
The Webserver
The Arduino webserver uses the StreamLib library as it makes the usage of F-Makro possible. You can read more about the StreamLib here.
- checkForClient() handles incoming client data in a finite state machine. It analyses the request (GET or POST), the requested URI and reads possible parameters - from the GET and from POST (when transmitted in the HTTP-Body). Finally the functions calls the requested page.
There are several examples of webpages:
- analogPage(): displays the value of the analog ports. It shows the basic principle how to use the Arduino webserver with the StreamLib library.
- timePage(): is a simple page showing the current system time and a marker, if the time was already synched with NTP.
- sendPage(): shows the state of digital ports and offers buttons to switch on/off the ports and send the command to the Arduino
- filePage(): offers a download link of the file and a button to delete the log file. As mentioned on the page - the Arduino will delete the file without further warnings.
- send404(): a small "Error 404" page to indicate, that the requested URI is not available
- send204(): notifies the browser, that the request was accepted but doesn't send back any further information. This comes very handy for the favion.ico or commands which should not send back data.
- fileDownload(): is not a "page" but it will download the logfile. One specific HTTP-Header Field will advise the browser that this is a download.
There are two helper functions: head() and navigation(). They are used because some parts of the webpages are similiar and I don't want to waste program memory for duplicate code.
The pages come without a stylesheet - but you are free to add your own stylesheet:
The digital page offers buttons to change the state of pins:
The logfile page is either empty or offers the possibility to download or to delete the file
If you chose to download the logfile, the additional HTTP Header field will
force the browser to switch to "Download" mode.
The SD Card on the Arduino Ethernet Shield
The Arduino Ethernet Shield uses pin 4 as chip select for the SD Card. I'm using a noname 2GB Micro SD Card - therefore it is formatted in FAT16 as recommended by the writer of the SD library and the results are reliable.
The SD tab has only two functions:
- timerSD(): handles the periodically logging of data. It will call the writeSD() only, if at least one NTP call was successful and the time is set accordingly. Otherwise we wouldn't have a valid timestamp for the Log file.
- writeSD(): demonstrates how to write data on the SD Card. There
are just a lot of prints to the data file, each value separated by a
delimiter.
As alternative you could composite the output into a buffer variable and finally print the buffer once to the file. But don't use an Arduino String Object (String with capital letter) as it will add a big overhead to your sketch. You will find an example using a C++ string buffer, how to convert numbers into C++ strings and how to concatenate them. The variable buffer[60] needs to be large enough to capture all needed data.
Retrieving time with SNTP
Mainly to get an accurate timestamp for the log file, the Arduino UNO retrieves the current time via SNTP. SNTP needs two UDP communications:
- Step 1: the Arduino sends a UDP packet to a NTP Server
- Step 2: the NTP Server sends a UDP packet to the Arduino
As the protocol is UDP the client (the Arduino) has to keep track of the communication logic.
- timerNTP() is a simple finite state machine with two states. IDLE just checks if it is time to retrieve a new NTP request. If a request was send, the timer expects for 1500ms an answer of the NTP server. There is some logic for the timer interval: on startup the timer will wait 4000..15.000ms, after the first request, it will wait 1 minute. If the time was already set once, the interval will be expanded to 1 hour.
- sendNTPpacket() is written very similar in lot of NTP examples. It just sends the SNTP request to the defined NTP server pool
- checkIncommingUdp() if we receive a UDP packet the Arduino will extract time information. It's also based on several examples and doesn't contain any checks if the time is valid.
Currently the system time (and therefore the logging time) is in UTC. If you want to implement local time you have to modify the returned time and/or implement the logic for Daylight Saving Time also.
It's not Working at all!?!
First of all, the demo works fine on an Arduino UNO. I would not publish the sketch if it weren't working. The sketch compiles with Arduino IDE 1.8.13 with two warning from the Ethernet Library. To fix this warning you have to patch the Ethernet library (or just accept the warnings).
Sketch uses 29540 bytes (91%) of program storage space. Maximum is 32256 bytes. Global variables use 1378 bytes (67%) of dynamic memory, leaving 670 bytes for local variables. Maximum is 2048 bytes.
Issues might occure, if you expand the sketch to much. If you add a lot more fields to the SD File, or any other components which will need more SRAM the Arduino UNO might reach its limit. Even if the reported usage of the global variables is not dramatic, the SD.h will need most of the remaining SRAM during runtime (dynamic memory). Also the webserver needs several buffers. If you encounter crashes ("reboots") - you might have run out of SRAM. If you need more hardware - (sensors, displays) I recommend to upgrade to an Arduino MEGA.
Summary
This sketch demonstrates that an Arduino UNO has enough resources for a Webserver, can do NTP requests and log data to a SD Card. There is even some program memory left to add a (small) stylesheet.