Understanding your data with graphs

Line Chart Example

Often, the machines and systems built at SwitchDoc Labs collect data and place it in a database for further research and analysis. In this example, the data originated from a SunAirPlus Controller/Data Collector [10]. Listing 3 is an example of Matplotlib code for one of the graphs created for Project Curacao.

Listing 3

Power Graph

001 # power graph generation
002 # filename: powersupplygraph.py
003 # Version 1.3
004 #
005 #
006
007 import sys
008 import time
009 import RPi.GPIO as GPIO
010
011 import gc
012 import datetime
013
014 import matplotlib
015 # Force matplotlib to not use any Xwindows backend.
016 matplotlib.use('Agg')
017
018 from matplotlib import pyplot
019 from matplotlib import dates
020
021 import pylab
022
023 import MySQLdb as mdb
024
025 sys.path.append('/home/pi/ProjectCuracao/main/config')
026
027 # if conflocal.py is not found, import default conf.py
028
029 # Check for user imports
030 try:
031   import conflocal as conf
032 except ImportError:
033   import conf
034
035 def  powersystemsupplygraph(source,days,delay):
036
037   print("powesystemsupplygraph running now")
038
039   # now we have to get the data, stuff it in the graph
040
041   try:
042     print("trying database")
043         db = mdb.connect('localhost', 'root', \
              conf.databasePassword, 'ProjectCuracao');
044
045         cursor = db.cursor()
046
047     query = "SELECT TimeStamp, SolarOutputCurrent, \
          BatteryOutputCurrent, PiInputCurrent FROM \
          powersubsystemdata where now() - interval %i hour \
          < TimeStamp" % (days*24)
048
049     cursor.execute(query)
050     result = cursor.fetchall()
051
052     t = []
053     s = []
054     u = []
055     v = []
056
057     for record in result:
058       t.append(record[0])
059       s.append(record[1])
060       u.append(record[2])
061       v.append(record[3])
062
063     print ("count of t=",len(t))
064
065     fds = dates.date2num(t) # converted
066     # matplotlib date format object
067     hfmt = dates.DateFormatter('%m/%d-%H')
068
069     fig = pyplot.figure()
070     fig.set_facecolor('white')
071     ax = fig.add_subplot(111,axisbg = 'white')
072     ax.vlines(fds, -200.0, 1000.0,colors='w')
073
074     ax.xaxis.set_major_locator(dates.HourLocator(interval=6))
075     ax.xaxis.set_major_formatter(hfmt)
076     ax.set_ylim(bottom = -200.0)
077     pyplot.xticks(rotation='vertical')
078     pyplot.subplots_adjust(bottom=.3)
079     pylab.plot\
        (t, s, color='b',label="Solar",linestyle="-",marker=".")
080     pylab.plot\
        (t, u, color='r',label="Battery",linestyle="-",marker=".")
081     pylab.plot(t, v, color='g',label="Pi Input",linestyle="-",marker=".")
082     pylab.xlabel("Hours")
083     pylab.ylabel("Current ma")
084     pylab.legend(loc='upper left')
085
086     if (max(v) > max(s)):
087       myMax = max(v)
088     else:
089       myMax = max(s)
090     pylab.axis([min(t), max(t), min(u), myMax])
091     pylab.figtext(.5, .05, ("Pi System Power Last \
                      %i Days" % days),fontsize=18,ha='center')
092
093     pylab.grid(True)
094
095     pyplot.show()
096     pyplot.savefig("/home/pi/RasPiConnectServer/static\
                       /systempower.png",\
                       facecolor=fig.get_facecolor())
097
098   except mdb.Error, e:
099
100         print "Error %d: %s" % (e.args[0],e.args[1])
101
102   finally:
103     cursor.close()
104             db.close()
105
106     del cursor
107     del db
108
109     fig.clf()
110     pyplot.close()
111     pylab.close()
112     del t, s, u, v
113     gc.collect()
114     print("systempower finished now")

The program retrieves the current (i.e., milliamp) values for the battery, the solar panels, and the Raspberry Pi A+ (i.e., how much it is consuming) from the MySQL database on the Raspberry Pi; then, it builds a graph and stores it on the Pi for use by RasPiConnect and for later transfer to the Project Curacao Live webpage.

The y axis of graph in Figure 3 is current in milliamps (mA), and the x axis is UTC time in six-hour increments. The Raspberry Pi wireless device is the source of variation in the green line. In the red and blue lines, you can see the diurnal pattern of solar power collection and the various peaks of current, depending on the brightness of the day and cloud cover. Looking at this data closely, you could estimate the percent cloudiness and get a good idea of the thickness of the cloud cover.

Figure 3: Project Curacao Raspberry Pi solar power data.

As before, you can take each statement in the code and refer to the manual to understand it fully, or you could just modify the code and see what happens. However, a few lines of code require explanation.

MySQL Database

The use of a database is a critical part of building any kind of large project that gathers data. Storing large sets of data in a database allows you to ask all sorts of questions and perform analyses. You can find a great resource for learning MySQL online [11]. Once learned, you will find applications for databases everywhere.

That said, I'll look at the query in line 47:

query = "SELECT TimeStamp, \
         SolarOutputCurrent, \
         BatteryOutputCurrent, \
         PiInputCurrent \
         FROM powersubsystemdata \
         where now() - interval %i hour < \
         TimeStamp" % (days*24)

This query selects the time and three measurements from the database over the course of the 10 days shown in the graph in Figure 3. Another non-obvious part of the code is what happens with the database query results in lines 50-61:

result = cursor.fetchall()
t = []
s = []
u = []
v = []
for record in result:
  t.append(record[0])
  s.append(record[1])
  u.append(record[2])
  v.append(record[3])

First, I have to split the query results for use by Matplotlib, so in lines 52-55, I set up four new arrays. For each result retrieved in line 50, I split the records into array t for TimeStamp (used for the x axis), array s for SolarOutputCurrent, and so on, which are then used to build the graph.

Buy this article as PDF

Express-Checkout as PDF

Pages: 6

Price $2.95
(incl. VAT)

Buy Raspberry Pi Geek

SINGLE ISSUES
 
SUBSCRIPTIONS
 
TABLET & SMARTPHONE APPS
Get it on Google Play

US / Canada

Get it on Google Play

UK / Australia

Related content