Volume Number: 21 (2005)
Issue Number: 1
Column Tag: Programming

Database Applications in REALbasic

by Will Leshner

REALbasic as a rapid database application development environment.

REALbasic Does Databases

Perhaps you've heard that REALbasic makes a great RAD tool, but maybe you haven't heard that it also makes a great database RAD tool. Not only does REALbasic have great support for a number of popular database engines, including MySQL, and PostrgreSQL, it also has a built-in single-user database engine that is quite useful. REALbasic also supports other database engines with its flexible plugin architecture, allowing third-party vendors to build support for other database engines that you can add to your projects simply by dropping their plugins into REALbasic's Plugins folder.

The purpose of this article is to get you to consider using REALbasic the next time you are looking for a powerful, yet fast, development environment for a database application you are planning to develop. I will introduce you to REALbasic's database support by building a front end to a database of election polling results. The application will support multiple database engines with one code base. Furthermore, thanks to REALbasic's graphics capabilities, I'll be able to add data graphing to the application as well with almost no trouble.

PollCat: A Polling Data Project

By the time this article is published, the U.S. elections will either be long over or, more likely, mired in litigation. You may be so sick of the election you don't want to hear anything more about it. However, I'm writing this article four days before the election and it's pretty much all I think about. So I decided that it might be interesting to put together a quick database application to display election polling data in a variety of formats. I'm calling the application PollCat, and its purpose is to display polling data for the 2004 Presidential election in both tabular and graphical formats. By the way, the data PollCat uses comes from data provided at the website www.electoral-vote.com, where you can download a great deal of polling information in CSV format. I'll have more to say about the specific data later.

Besides PollCat, we also need to write a utility that takes the raw data files and pours them into a database. I have included the REALbasic project for that utility, which I am calling PollCat Database Builder, with the source you can download for this article. PollCat Database Builder creates databases for several different database engines, including the single-user database built into REALbasic, as well as MySQL. I have also included a pre-built REAL database in the download, so you don't have to run PollCat Database Builder at all if you don't want to.

Prerequisites

In order to following along in this article, I strongly recommend that you download the source files that accompany it from the MacTech website. You will also need REALbasic. If you aren't a REALbasic user, you can download a trial version from the REAL Software website (www. realsoftware.com). I used the most recent version of REALbasic (5.5.3) to create PollCat and PollCat Database Builder. If you have an earlier version, the projects may not work. Also, you will need the Professional version of REALbasic in order to build projects that use the Database classes.

In addition to REALbasic itself, you will also need to download the database plugins from the REAL Software website. You can find them on the download page (http://realsoftware.com/download/). From the folder of plugins you will need the MySQL Plugin and the CSV Plugin. Place both of those plugins in REALbasic's Plugins folder (next to the REALbasic application). If you are interested in trying out the SQLite support in PollCat, you will need the SQLitePluginPro plugin, which you can download from the SQLabs website (http://www.sqlabs.net).

Finally, you should take a few minutes to check out the layout of the source folder that accompanies this article. You should find the PollCat.rb and PollCat Datbase Builder.rb projects, as well as a pre-built pollingdata.rdb file of polling data. There will also be a "raw data" folder, that contains several months of polling data in CSV format. Those are the same files you can download from the http://www.electoral-vote.com website.

REALbasic Databases: A Quick Tutorial

REALbasic is an objected-oriented programming language and it provides support for databases through classes. At the center of these classes is the Database class, which is responsible for managing one database instance. Different database engines, such as MySQL and PostgreSQL, subclass the Database super class in order to provide engine-specific functionality. Therefore, there is a MySQLDatabase for MySQL databases, a PostgreSQLDatabase, for PostgreSQL databases, and a REALDatabase for instances of the built-in REALbasic database.

SQL is the language we use to communicate with a Database subclass. Every Database subclass implements a SQLExecute method for executing SQL commands, and a SQLSelect function for querying a database. SQLSelect returns a RecordSet, which provides a convenient API for accessing the data returned by the query. Think of a RecordSets as a kind of array of records. Each record represents one row of query results. Unlike an array, however, you can't randomly access just any record in a RecordSet. Instead, you have to move through the RecordSet one record at a time using methods such as MoveNext, MovePrev, MoveFirst, and MoveLast. Here is an example of how RecordSets are commonly used:

dim rs as RecordSet
rs = db.SQLSelect("SELECT * FROM atable")
if rs <> nil then
   while not rs.EOF
      [do something with rs]
   wend
end if

RecordSets have two functions, Field() and IdxField(), for accessing the individual fields of one record. Field() takes the name of a field (a column header, in other words) and returns a DatabaseField. IdxField() takes a column index (column indexes start at 1) and also returns a DatabaseField. DatabaseFields wrap one field of data, and they can coerce their data into a number of different formats through accessor functions (such as StringValue, IntegerValue, DateValue, etc.).

REALbasic also provides a DatabaseRecord class for putting data into a database. A DatabaseRecord is a record that isn't yet in a database. You add data to the columns of a DatabaseRecord with Column accessors, of which there are several different kinds, depending on what kind of data you are putting into the database. Here is an example of using a DatabaseRecord to put data into a database:

dim dr as DatabaseRecord
dr = new DatabaseRecord
dr.Column("sometext") = "this is a string"
dr.IntegerColumn("anumber") = 3
dr.DoubleColumn("areal") = 10.123
db.InsertRecord "atable", dr

Note that there also exist a few database solutions that do not support the REALbasic database API. Instead, they provide their own database API. One example of such a database product is Valentina, a particularly fast database engine.

The Data And The Database

Each CSV file in the "raw data" folder represents a snapshot of the poll results for one day. But polls aren't taken every day. In fact, days can go by with no polls being taken. Some states aren't polled that often, especially if they aren't battleground states. So what, exactly, is in each CSV file? The answer is that each CSV file has the most recent poll conducted for each state. In other words, different files can have the same polling information in them for a particular state, if there is no new polling results to report for that state. Also, the votemaster at http://www.electoral-vote.com has his own formula for determining exactly what poll to use for a state in the event that there are multiple polls to choose from. If you are interested in the whole process, you can check out the website for yourself.

As for the structure of the PollCat database itself, it will consist of two tables: states and pollingdata. The states table will contain information about each state, including its population, the number of electoral votes it has, and its name. The pollingdata table will contain the actual polling data. The schema for the states table is as follows:

CREATE TABLE states (
   code         TEXT,
   name         TEXT,
   votes        INTEGER,
   population   DOUBLE)

The code field column contains the code for each state (e.g. OH, CA, TX, etc.), and the votes column contains the number of electoral votes. The schema for the pollingdata table is as follows:

CREATE TABLE pollingdata (
   currentdate      DATE,
   code             TEXT,
   polldate         DATE,
   bush             INTEGER,
   kerry            INTEGER,
   pollname         TEXT)

Each record of the pollingdata table contains the polling data for one date and state. If we want to figure out the average popularity of either candidate on any particular day, we simply need to average all the poll data for that day across all the states. If we want to calculate the electoral vote counts for each candidate for a particular day, we simply need to sum the votes for all the states a candidate is leading in for that day. Note that some states are a tie, so the votes may not add up to the total for every day.

By breaking up our data into two tables, we can avoid duplicating the electoral vote count and populating data in every row of the pollingdata table. If we need that information, we can just look it up in the states table by state code. It turns out you can write SQL that uses that technique to add up all the electoral votes for a particular candidate on a particular day.

The PollCat Interface

PollCat is an application in one window, as you can see in Figure 1. That window contains popups to choose a range of dates for which to calculate polling data, a popup to choose whether to show polling results or electoral votes, a popup to choose which database to get the data from, a tab panel that allows the user to switch from viewing the calculated data in either a tabular form, in a ListBox, or in a graph, and a refresh button.

The PollCat code

All of the PollCat code of interest is located in PollCatWindow. If you open PollCatWindow in REALbasic's code editor (double-click the PollCatWindow in the project to show the window itself, and then double-click the window to show the code editor) you will see all of PollCat's methods under the Methods heading in the list on the left.

When the user clicks the Refrsh button, we need to issue a SQL query to the database to get the data, and then fill the ListBox with the results. For example, to calculate the electoral votes for each candidate for a range of dates, we will issue a SQL query to the database to get a sum of electoral votes for each candidate for all the dates in the range of dates we are calculating. I have created a DataPoint class in order to store the data for each date. Each DataPoint has a date, a value for Kerry and a value for Bush. I plan to store all of the DataPoints for one calculation into a REALbasic Dictionary keyed by the date. Once the Dictionary has been created, it can be used to rebuild both the data table and the graph.

It is possible to write one SQL statement to get both the Bush and Kerry data in one query, but the REAL database can't handle such a query. Instead, we will need to perform two queries, one for each candidate. What we do is build the SQL query as a String, based on the states of the various popups in the PollCat window. Then we pass the query to the appropriate database and iterate over the results to build our Dictionary of DataPoints. The method responsible for building the SQL query is GetQueryFor, which takes both a candidate and an opponent and returns the query as a String. Here is the code for GetQueryFor:

Private Function GetQueryFor(aCandidate as String,
   aOpponent as String) As String

   // build a SQL query based on the values of the various popups
   // on the main window and return it as a string

   dim sql as String
   dim fromDate as new Date
   dim toDate as new Date

   // build the from date
   fromDate.year = 2004
   fromDate.month = mMonths.Value(FromMonthPopup.Text)
   fromDate.day = Val(FromDayPopup.Text)

   // build the to date
   toDate.year = 2004
   toDate.month = mMonths.Value(ToMonthPopup.Text)
   toDate.day = Val(ToDayPopup.Text)

   // construct the query based on which kind
   // of query we are to perform
   if TypePopup.Text = "Electoral Votes" then
      sql = "SELECT currentdate,sum(votes) as " + _
         aCandidate + _
         " FROM pollingdata,states WHERE" + _
         " currentdate >= '" + fromDate.SQLDate + "'" + _
         " AND currentdate <= '" + toDate.SQLDate + "'" + _
         " AND " + aCandidate + " > " + aOpponent + _
         " AND pollingdata.code = states.code " + _
         "GROUP BY currentdate"
   else
      sql = "SELECT currentdate, avg(" + _
         aCandidate + ") as " + aCandidate + _
         " FROM pollingdata,states WHERE" + _
         " currentdate >= '" + fromDate.SQLDate + _
         "' AND currentdate <= '" + toDate.SQLDate + _
         "' GROUP BY currentdate"
   end if

   return sql

End Function

GetQueryString is called within a method called AddToResults, which takes a candidate, an opponent, and a Dictionary to which DataPoint objects are added. We will need to call AddToResults twice, once for each candidate. Here is the code for AddToResults:

Private Function AddToResults(aCandidate as String,
   aOpponent as String, aResults as Dictionary) As Boolean

   // add the results for the given candidate and
   // opponent to the aResults Dictionary

   dim dp as DataPoint
   dim db as Database
   dim rs as RecordSet
   dim day as String

   // get the database
   db = GetDatabase
   if db = nil then
      MsgBox "Couldn't open database."
      return false
   end if

   // query the database
   rs = db.SQLSelect(GetQueryFor(aCandidate, aOpponent))
   if rs <> nil then
      while not rs.EOF
         // get the date of this data point
         day = rs.Field("currentdate").DateValue.SQLDate

         // if the data point isn't in the dicationary
         // then add it, otherwise use the existing
         // data point
         if not aResults.HasKey(day) then
            dp = new DataPoint
            dp.Day = rs.Field("currentdate").DateValue.SQLDate
            aResults.Value(dp.Day) = dp
         else
            dp = aResults.Value(day)
         end if

         // this part is a little icky: based on the name of
         // the candidate, set either the "bush" value or
         // the "kerry" value; a better approach would
         // be not to depend on the names
         select case aCandidate
         case "bush"
            dp.Bush = rs.Field("bush").DoubleValue
         case "kerry"
            dp.Kerry = rs.Field("kerry").DoubleValue
         end select

         // go to the next record
         rs.MoveNext
      wend
   end if

   return true

End Function

Once we have created our Dictionary of DataPoints, we can use it to build both a data table and a graph. I won't waste time on the data table. That's simply a matter of iterating over the data in the Dictionary and storing it in a REALbasic ListBox. The graph, however, is much more interesting and really shows why REALbasic is such a great choice for this kind of application.

There are a couple of ways we could handle drawing the graph and displaying it to the user, but I have chosen to draw the graph in a REALbasic Picture. A Picture is an object into which we draw lines, polygons, and text, using REALbasic's drawing primitives. We also need a way to draw the Picture in the PollCat window. A quick-and-dirty way to get the Picture to draw itself in a window, is to place a Canvas on the window and then set that Canvas' backdrop to point at the Picture. A Canvas is a control that has a graphical representation. A Picture is not a control, but you can make it the backdrop of a Canvas, thus making it visible in a window. If you check out the PollCatWindow in REALbasic, you will see a TabPanel with two tabs. You can switch between the two tabs in the window editor by clicking directly on them. In the first tab is a ListBox for the tabular data. In the second tab is a GraphCanvas, which will hold our Picture.

The method responsible for drawing our graph into a Picture is called RebuildGraph. The code is a bit long, so I won't show it all here. The first thing we do is sort all of our DataPoints into an array by date. Then, we iterate over the array of DataPoints, drawing lines from one point on the graph to the next, for both candidates. When we are finished we will have two lines, one blue for Kerry and one red for Bush. To accomplish this, we will need to keep track of a total of six points: both the previous and current y points for both Bush and Kerry, and the previous and current x points.

Drawing the graph itself is as easy as drawing a line from the previous point to the current point for each candidate, alternating the color between red and blue for each line. The code to do that is presented here. Note that the orgY and orgX are the x and y origin of the graph. Also, keep in mind that the y-coordinates grow down from the top, as they do in QuickDraw, so to calculate y coordinates for our lines we need to be subtract from the origin, not add to it.

// start by calculating the "previous" x and y points
prevX = orgX
bushPrevY = orgY - ((data(0).Bush - lo) * scaleY)
kerryPrevY = orgY - ((data(0).Kerry - lo) * scaleY)

// iterator over all the DataPoints,
// drawing lines from one to the other
for i = 1 to Ubound(data)
   // calculate the next x point, which
   // is the same for both lines
   nextX = orgX + (i*scaleX)

   // change the color to red
   p.Graphics.ForeColor = &cFF0000
   // calculate the next (scaled) Bush point
   bushNextY = orgY - ((data(i).Bush - lo) * scaleY)
   // draw the line
   p.Graphics.DrawLine prevX, bushPrevY, nextX, bushNextY

   // change the color to blue
   p.Graphics.ForeColor = &c0000FF
   // calculate the next (scaled) Kerry point
   kerryNextY = orgY - ((data(i).Kerry - lo) * scaleY)
   // draw the line
   p.Graphics.DrawLine prevX, kerryPrevY, nextX, kerryNextY

   // make our "next" points our previous points
   // in preparation for the next iteration
   prevX = nextX
   bushPrevY = bushNextY
   kerryPrevY = kerryNextY
next

Switching Databases

The last thing I'd like to show you is how easy it is to switch from one database engine to another in REALbasic. This assumes that the database engines you are switching to also implement REALbasic's database classes. The database I'll be using is a MySQL database. I'll assume you have access to a MySQL server, and I won't go into the details of how to set one up here. MySQL runs on Mac OS X, so if you don't have access to a server, you can always install it and run it locally.

PollCat Database Builder already supports creating a PollCat database on a MySQL server. If you choose MySQL from the BuildDatabase popup you will be prompted for your MySQL login information. Assuming you enter it correctly, PollCat Database Builder will create a pollcat database and populate that database with the states and pollingdata tables. Now you are ready to run PollCat and have it connect to the MySQL server for its data.

As it turns out, PollCat is written such that virtually no code needs to change in order to support MySQL. Wherever we talk to a database, we are talking to a Database superclass, and not to a specific subclass of Database. The only place we need to refer to specific Database subclasses is in the GetDatabase function, which returns a reference to a Database subclass depending on the current value of the DatabasePopup control. GetDatabase simply creates either a REALDatabase or a MySQLDatabase, depending on the value of DatabasePopup. If the database is to be a MySQLDatabase, then the user is prompted for the server login information, and the MySQLDatabase instance is initialized accordingly.

Private Function GetDatabase() As Database

   // get a database object depending on which kind
   // of database is selected in the database popup
   //
   // note that we cache the database so that we can
   // avoid having to reopen the MySQL database every
   // time the user clicks Refresh

   dim rdb as REALDatabase
   dim mdb as MySQLDatabase

   // see if we have a cached database and return that
   // if we do

   if mCachedDatabase = nil then
      if DatabasePopup.Text = "REAL" then
         // create a REALDatabase
         rdb = new REALDatabase
         rdb.DatabaseFile = GetFolderItem("pollingdata.rdb")
         if rdb.Connect then
            mCachedDatabase = rdb
         end if
      elseif DatabasePopup.Text = "MySQL" then
         // create a MySQLDatabase
         mdb = new MySQLDatabase
         // prompt the user for the login info
         if MySQLInfoDialog.GetConnectionInfo(mdb) then
            mdb.DatabaseName = "pollcat"
            if mdb.Connect then
               mCachedDatabase = mdb
            end if
         end if
      end if
   end if

   return mCachedDatabase
	
End Function

We cache the current database in an mCachedDatabase property so that the user will be not be presented with the MySQL server login dialog every time we call GetDatabase. By caching a reference to the database, we can leave it open as long as the user doesn't change database types. If the user does change database types, we just set the value of mDatabase to Nil so that GetDatabase will be required to create a new database subclass and initialize it anew.

The projects that accompany this article also have support for SQLite databases through the SQLitePluginPro plugin. You will need that plugin if you want to test SQLite databases out. A SQLite database works exactly like the built-in REALbasic database in that it is a single-user database in one file on the user's local hard drive. There is no server. If you want to try SQLite out, you will need to change the USE_SQLITE_DATABASE constants in both PollCat Database Builder and PollCat itself. You will need to use PollCat Database Builder to build the SQLite database before you can use it in PollCat. You might want to test the speed difference between running a query with the built-in REALbasic database and the SQLite database.

Conclusion

I hope this article has convinced you to consider REALbasic the next time you have a database project you are preparing to tackle. Not only do you have access to a number of popular database engines (we only looked at a couple, but there is also support for ODBC, OpenBase, PostgreSQL, and Oracle databases), you also have the power of a complete OOP language and development environment. REALbasic has a rich set of programming tools, including drawing primitives, networking, OS support, and, of course, user interface support, to help you finish any project in record time.

Will Leshner has been a computer scientist for twenty years and a software engineer for ten. Although he is a big fan of C and C++, he considers REALbasic to be the most fun he has ever had as a software developer. He currently works full time as a Software Engineer.