Temperature probes (part 3&4): graph rendering and scripting

Posted on by

Icone: screenshot temperatures
Now that we have installed the analog to numeric converter (refer to previous article), we are going to setup the rendering part of the project, taking benefice of the server mode of OWFS.
The main idea is to use the famous RRD-tools on my internal web server (MICKEY, running Ubuntu 10.04 Desktop edition). And to create specific perl scripts to interface OWFS with RRD database, and, at last, generate the graphs out of RRD database.

Software install

Assumption: you already have a running web server on the system (with PHP if you want to generate dynamic HTML pages).

RRD Tools installation

Very straightforward:
sudo apt-get install rrdtool
Note that it depends on other packages (librrd4, libpng12-0,…) that your favorite software packages manager will propose you to install if not already done.

Perl modules

Additional Perl modules are needed there. The ones I used in my scripts are:

  • libdatetime-perl
  • libdatetime-event-sunrise-perl
  • libmath-round-perl
  • librrds-perl

All of them can be installed in a single line of code:
sudo apt-get install libdatetime-perl libdatetime-event-sunrise-perl libmath-round-perl librrds-perl

I also had to copy the OWFS Perl modules from my OWFS server (be careful if, like me, you are not running on the same kernel architecture: you will have to compile OWFS on the appropriate architecture of course !).
You just need one file (OWNet.pm) and I decided to install it in the share PERL modules directory:
sudo cp OWNet.pm /usr/share/perl5/.

Scripting

Description of the environment

As you should know now, from previous articles, I have a set of 7 probes that are capturing temperatures, and that I want to display.
HTML files and graphs should be created in a specific directory on my web server: /var/www/html/temps

Initialize RRD database

I’m planning to run data polling every 5 minutes (this is an important parameter in the RRD database creation)
As I wanted to be able to re-create the RRD database file easily, I scripted it:

#!/bin/bash

DIR="/var/www/html/temps"
FILE="${DIR}/temps_vmc.rrd"

echo "Creating rrdtool DB for 7 temp sensors"

#   48 hours (2 days)   of 5 minutes data  : (1:576)
#  336 hours (2 weeks)  of 30 minutes data : (6:672)
# 1488 hours (2 months) of 2 hours data    : (24:744)
#  730 days  (~2 years) of 12 hours data   : (144:1460)
# 1830 days  (~5 years) of 24 hours data   : (288:1830)

rrdtool create $FILE \
	--step 300 \
	DS:Basement1:GAUGE:600:-50:60 \
	DS:Basement2:GAUGE:600:-50:60 \
	DS:Basement3:GAUGE:600:-50:60 \
	DS:Basement4:GAUGE:600:-50:60 \
	DS:Basement5:GAUGE:600:-50:60 \
	DS:Basement6:GAUGE:600:-50:60 \
	DS:Basement7:GAUGE:600:-50:60 \
	RRA:AVERAGE:0.5:1:576 \
	RRA:AVERAGE:0.5:6:672 \
	RRA:AVERAGE:0.5:24:744 \
	RRA:AVERAGE:0.5:144:1460 \
	RRA:AVERAGE:0.5:288:1830 \
	RRA:MAX:0.5:1:576 \
	RRA:MAX:0.5:6:672 \
	RRA:MAX:0.5:24:744 \
	RRA:MAX:0.5:144:1460 \
	RRA:MAX:0.5:288:1830 \
	RRA:MIN:0.5:1:576 \
	RRA:MIN:0.5:6:672 \
	RRA:MIN:0.5:24:744 \
	RRA:MIN:0.5:144:1460 \
	RRA:MIN:0.5:288:1830

It should be quite self-explanatory if you understand rrdcreate parameters. If not, don’t hesitate to look at rrdcreate help page, or the rrd for beginners page (which is much easier to understand, I think…).

You will notice that I created very generic RRD names for my probes (at this point of time, I didn’t know which one was which one…). Matching with real locations will be done in the next step.

The polling script

Now, it’s time to script (PERL) the 5-minutes polling of data which is storing its information in the RRD database, and which creates the graphs.
I must thank Ronin Technologies who published the script that I just had to adapt to my needs ! Thanks guys…
For the same reason, you can download my own version here.

Let’s look at some details:

############################
#
# Parameters to adjust
#
############################
my %sensors = (
        "28.E4B5E0020000" => { name => "Basement1", temp => 99, order => 5 , legend => "insufflation air neuf" , color => "#FF0000"},
        "28.34CEE0020000" => { name => "Basement2", temp => 99, order => 4 , legend => "entree PC            " , color => "#FF00FF"},
        "28.3ECFE0020000" => { name => "Basement3", temp => 99, order => 3 , legend => "sortie air vicie     " , color => "#00FFFF"},
        "28.6DD3E0020000" => { name => "Basement4", temp => 99, order => 2 , legend => "aspiration air vicie " , color => "#00FF00"},
        "28.BFCAE0020000" => { name => "Basement5", temp => 99, order => 1 , legend => "entree air neuf      " , color => "#0000FF"},
        "28.94B1E0020000" => { name => "Basement6", temp => 99, order => 6 , legend => "entree air ext       " , color => "#000000"},
        "28.BCC4E0020000" => { name => "Basement7", temp => 99, order => 7 , legend => "grenier-cagibi       " , color => "#0033AA"}
        );

my $owsystem = "geo:4304";
my $dir = "/var/www/html/temps";
my $width=600;
my $height=200;
my $LAT = "45.321925";
my $LON = "5.929002778";
my $watermark="© Thierry Hugue - 2011";

All is said in this section !
You can see:

  • the mapping between the “name” of each probe in the RRD database, and its OWFS identifier. Note that it is also the place to setup legend, color,… used later.
  • name and port of the OWFS server: my $owsystem = "geo:4304";. Simple, isn’t it ?
  • directory to store the data in RRD database, and to create graphs: my $dir = "/var/www/html/temps";
  • I used LAT & LON, to calculate sunrise and sunset. Objective is to set a specific pattern on the graphs during the night (night temperature can be very different from daily temperature). Note that sunrise and sunset are calculated when creating the graphs. This is fine on daily graph, a little bit less accurate on weekly graphs, and completely innacurate on monthly/year graphs (that’s why I’m not displaying them)

When you look at the polling part of the script, you will notice:

  • my $owserver = OWNet->new($owsystem.' -v -C');:
    this is the key of the communication with the OWFS server
  • sensors{$sensor}{temp} = $owserver->read("/$sensor/temperature");:
    I poll the “/temperature” attribute (which is the instant value)
  • once all data is polled, it is submitted to RRD database:
    RRDs::update( "$db", $chaine);

Last section is about graphs creation:

  • same routine for each graph, just the frequency (daily, weekly,…) is changing
  • pay attention to the parameter (variable @par) used to create the RRD graph. More information is available on the RRD:graphs page, the rrdgraph details page, and the rrdgraph examples page (last one most useful)

HTML pages

In order to nicely display the results, I developed an index.php page, which is dynamically generating a small HTML page to display the appropriate graph.
It is based on a style.css file and a few png, that you can download here.

Nothing specific to say about those HTML pages, except that they are W3C compliant (both HTML and CSS 😉 ): that’s something important I think, if you want your result to be viewable nicely in any browser… I really hate proprietary HTML code (but that’s another story here again…)

Crontab

All those scripts are scheduled using the cron:
thierry@mickey:~$ crontab -l
# m h dom mon dow command
*/5 * * * * $HOME/bin/temps.pl >$HOME/var/temps.log 2>>$HOME/var/temps.err
#

I like to keep track of what happens in my scripts. This is useful for forensic analysis (it could happen, as we will see in a future post…). That’s why I’m using LOG and ERR files.
Moreover, I used the advance notation (“*/5”), which is much more convenient than explicitly enter : 0,5,10,15,…. Nice to have feature !

Bonus script

In case I would need to look at certain graphs from the past, and as I didn’t want to generate manually the information out of the RRD database (especially as it is “summarizing” old data), I created a small script to rotate the graph files (and store them with specific name).

I generated a small shell script, very simple (all the “logic” is done in the crontab):

#!/bin/bash
#
#
DIR=/var/www/html/temps
ROOT="temp"
DATE=$(/bin/date "+%Y%m%d") 

# Input variable (mandatory)
VAR=$1

if [ "X$VAR" == "X" ]; then
	echo "Error : I need a param to select the frequency to rotate"
	exit 1
fi

if [ $VAR != "day" -a $VAR != "week" -a $VAR != "month" -a $VAR != "year"  ]; then
	echo "Error : I need a param in list (day, week, month, year) "
	exit 2
fi

# Action
cd $DIR 
/bin/cp -p ${ROOT}${VAR}.png ${ROOT}${VAR}_${DATE}.png

Nothing to say here, as it is very very very simple….

Now, look at the crontab:
# Rotate png files
# Everyday for the daily graph
# Every Monday for the weekly graph
# Every first day of the month, for the monthly graph
# Every first day of the year, for the yearly graph
30 0 * * * $HOME/bin/store_graph.sh day
31 0 * * Mon $HOME/bin/store_graph.sh week
32 0 1 * * $HOME/bin/store_graph.sh month
33 0 1 Jan * $HOME/bin/store_graph.sh year

Here it is !
You should now have very nice web pages, just like this one:
The HTML main page

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.