QueryDef - a complete implementation of a dynamic recordset

This article tries to explain the class library QueryDef intended to be used instead of the classic MFC class CRecordset. CRecordset is a great class with one single exception. You have to generate/write too much code. In a project I was involved there were over 700 recordsets defined. Soon working with CRecordset became a real hassle. I wanted a simpler method to access the data from an ODBC data source.

So the design goals of this library were:

1. Eliminate the need for one recordset-one class tuple.
2. Eliminate the need for bindings and the infamous list of data members henceforth.
3. Eliminate the need to know in advance the database schema.
4. Blur the distinction between a recordset and a stored procedure with output parameters.
5. Ease the access to the recordset column information.
6. Provide automatic conversion of data wherever is needed with easy formatting.
7. Simplifying the test with the SQL null value.
8. Provide a simple notification mechanism to bind visual controls with certain events related to the recordset.
9. Compatibilty with the CRecordset class (aka support for positioned updates and inserts).

The cornerstone of the library is the class CQueryDef. This class is designed to be used directly (w/o derivation) although there is nothing stopping you using it as a base class. Basically this class represents a collection of rows and columns returned from a SQL request. The table can be as small as no rows/no columns or as large as milions of rows with 255 columns. CQueryDef uses an internal representation of the column data making it possible to hold the following data types:

• String or text (this includes numeric and decimal SQL types)
• Int, long or short
• Float and double
• Datetime, smalldatetime, timestamps
• Binary data

To accommodate this data type spectrum the library makes use of the CQueryVar class. This class offers the necessary data type conversions by defining a series of conversion constructors and operators as well as assignment operators. Also provides several functions to format the data as output strings (human readable). CQueryVar holds only one data value.

CQueryVariant on the other hand holds a variable array of CQueryVar values. Basically it represents an entire row of SQL data. This class offers access to individual CQueryVar values (column data) and can manipulate an entire row as a whole.

CQueryCol describes an entire SQL column. All the information is retrieved from the ODBC data source once and contains:

• The name of the column as returned from the data source
• The SQL type
• The scale
• The precision
• Whether accepts SQL null values

CQueryCol information is returned as a collection or as individual values from the CQueryDef class (see the reference).

The notification mechanism relies upon the abstract sink IQueryDefEventSink that defines the following events:

struct IQueryDefEventSink;      // CQueryDef event sink
{
  // possible notifications - returning 0 means proceed with the next notification otherwise stop
  virtual LPARAM RSNotifyOpen(EVNHANDLE) = 0;            // the recordset has been opened
  virtual LPARAM RSNotifyClose(EVNHANDLE) = 0;           // the recordset has been closed
  virtual LPARAM RSNotifyMove(EVNHANDLE) = 0;            // the recordset has modified the current position
  virtual LPARAM RSNotifyAddNew(EVNHANDLE) = 0;          // recordset is about to insert a new record (called after CQueryDef::AddNew)
  virtual LPARAM RSNotifyEdit(EVNHANDLE) = 0;            // the current record will be modified to be updated (called after CQueryDef::Edit)
  virtual LPARAM RSNotifyUpdate(EVNHANDLE) = 0;          // the current record has beeen updated (called after CQueryDef::Update)
  virtual LPARAM RSNotifyDelete(EVNHANDLE) = 0;          // the current record has been deleted
  virtual LPARAM RSNotifyCancelUpdate(EVNHANDLE) = 0;    // the update of the current record has been canceled (called after CQueryDef::CancelUpdate)
  virtual LPARAM RSNotifyRequery(EVNHANDLE) = 0;         // the recordset has been refreshed
  virtual LPARAM RSNotifyFormatChanged(EVNHANDLE,BYTE nFormat) = 0;   // number of decimal digits or date format has been changed
}

A class object may be notified with these 10 events implementing a sink based on IQueryDefEventSink. To make things easier, the developer may choose to use CQueryDefDefaultSink instead and override only the events hes interested in.

#include <QueryDef.h>

USE_QUERYDEF;
class CEditDB : public CEdit,public CQueryDefDefaultSink
{
  ...

  CQueryDef*   m_pqset;            // querydef to bind to
  LPARAM RSNotifyMove(); 
}; 

LPARAM CEditDB::RSNotifyMove() 
{
  SetWindowText(m_pqset->Field("CustomerName"));
  return 0;
}

class CADialog : public CDialog
{
  ...
  CQueryDef		m_qDef;
  EVNHANDLE		m_eH;
  CEditDB		m_custName;
};

BOOL CADialog::OnInitDialog()
{
  ...
  // remember to call Unadvise when you no longer need notifications
  m_eH = m_qDef.Advise(static_cast<IQueryDefEventSink*>(&m_custName));
}

The library has two other helper classes. CTabbedString holds an entire row of data as TAB separated string values. It gives access to individual column values using the convenient operator []. Very useful when you want to fill a listbox or a list control with data from a SQL request.

...
CQueryDef        qDef(&theApp.db);   // uses application wide database connection
CTabbedString    sqlRow;
CListBox         listAllBadgeInfo;       // a listbox control

qDef.Open(CRecordset::snapshot,"select * from BADGE",CRecordset::none);
while (!qDef.IsEOF())
{
  sqlRow = qDef;

  if (sqlRow[0] != "111")
    listAllBadgeInfo.AddString(sqlRow);
  qDef.MoveNext();
}

Class CSQLNull is used to define a representation for the SQL null value. This class defines the library wide constant QueryDef::CSQLNull SQL_NULL. This value can be used to test for SQL null various returned results from CQueryDef and CTabbedString objects.

...
qDef.Open(CRecordset::snapshot,"select * from BADGE",CRecordset::none);
while (!qDef.IsEOF())
{
  if (qDef["CustID"] != SQL_NULL)
  listBadgeInfo.AddString(qDef["CustName"]);
  qDef.MoveNext();
}

Reference by example

This section shows only the most important functions and operators of the QueryDef library, the header file speaks for itself for the others. Typical database programs will make use of the QueryDef library in the following manner:

1. Declare a CQueryDef variable.
2. Connect it with a database connection (MFC CDatabase)
3. Open a recordset based on an ad-hoc query or stored procedure with or without parameters.
4. Navigate thru the recordset with the CRecordset navigational functions testing for the ending or beginning of the data set.
5. Process data contained in the CQueryDef variable in some manner (read or update the recordset).
6. Possibly process the notifications received from the CQueryDef object.

The following examples assume you already have an open database connection defined at the application level. Let it be theApp.m_db.

int TotalSales = 0;
CQueryDef     qDef(&theApp.m_db);
qDef.Open(CRecordset::forwardOnly,"select ID,Name,SaleAmount from CUSTOMERS order by Name");

while (!qDef.IsEOF())
{
  if (qDef["ID"] == SQL_NULL)
    continue;
  m_listbox.AddString(qDef["Name"]+ ": " + qDef["SaleAmount"]);
  TotalSales += qDef["SaleAmount"];

  qDef.MoveNext();
}

int nClientID;
...
int          TotalSales = 0;
CQueryDef    qDef(&theApp.m_db);

do
{
  qDef.Param(0) = nClientID;        // set the parameter as introduced by the user

  if (!qDef.IsOpen())
    qDef.Open(CRecordset::forwardOnly,"select ID,Name,SaleAmount from CUSTOMERS where ID = ? order by Name");
  else
    qDef.Requery();

  while (!qDef.IsEOF())
  {
    TotalSales += qDef["SaleAmount"];
    qDef.MoveNext();
  }
} while (...);     // code stripped away for brevity

You can access the column values on the current row of a CQueryDef object using several methods:

qDef["SaleAmount"] or
qDef[2] or
qDef.Field("SaleAmount") or
qDef.Field(2)

The first and the third are used when you dont know or dont care about the position of this column in the recordset. The second and the fourth are used when you know the position of the column in the recordset or you dont know the name of the column (select * might be one case). Using numbers instead of column names is faster too since there is no search involved. Using an invalid index or column name will result in an integer exception (CQueryVariant::eBoundary).

On the other hand the parameters are accessed always using the function CQueryDef::Param():

Param(int nPos)   // parameters are numbered starting with 0 from left to right

This functions/operators return CQueryVar& making it possible to use them on both sides of the assignment operator. It is legal to write:

qDef["ID"] = nClientID; or
nClientID = qDef["ID"];
...
qDef.Param(0) = 10;
qDef.Param(1) = "Books%";
CString strCategory = qDef.Param(0);

CQueryDef qDef(&theApp.m_db);

qDef.Param(1) = "AMOCO%";
qDef.Open(CRecordset::forwardOnly,"{?L = CALL rpt_CustomersByName ?}");
// rpt_CustomersByName is "select ID,Name,SaleAmount from CUSTOMERS where Name like ? order by Name"

while (!qDef.IsEOF())
{
  m_listbox1.AddString(qDef[0]);    // ID is SQL long
  m_listBox2.AddString(qDef[1]);    // Name is varchar
...
}

Each ? is counted so the input parameter to the stored procedure is number 1 (0 being the stored procedures returned value). Notice the suffix following the first parameter. This is used to specify the output parameters type. The table below shows the meaning of every type suffix:

You dont need to provide values (only types) for the output parameters. They will be dynamically allocated by CQueryDef and populated with the stored procedures output parameters. In a similar way you can use input/output parameters:

qDef.Param(1) = 0L;
qDef.Param(2) = "CMPT";
qDef.Open(CRecordset::forwardOnly,"{?L = CALL rpt_TotalSalesPerCustomer ?L,?}");
...
if (qDef.Param(0) != 0)    // error
  ThrowException();
AfxMessageBox(qDef.Param(1));

In the case you dont know the database schema at design time, CQueryDef can help with its Columns() member function:

CQueryColArray& Columns();

This returns an array of CQueryCol objects describing each column.

CQueryDef qDef(&theApp.m_db);
qDef.Open(CRecordset::snapshot,"select * from COMPANIES");

for (int i = 0; i < qDef.Columns().GetSize(); ++i)
{
  strColName[i] = qDef.Column(i).Name();
  dwColType[i] = qDef.Column(i).Type(); 
}

while (!qDef.IsEOF())
{
  for (int j = 0; j < i; ++j)
    strRowValues += qDef[j] + "\t";

  qDef.MoveNext();
}

CQueryDef qDef(&theApp.db);
try
{
  qDef.Open(CRecordset::snapshot,"select * from BADGE",CRecordset::none);

  qDef.AddNew();
  qDef["ID"]           = "XXX";
  qDef["CustID"]       = "  58";
  qDef["LockedStatus"] = 1L;
  qDef["StartDate"]    = COleDateTime(1998,02,05,10,30,0);
  qDef["ExportStatus"] = 0l;
  qDef.Update();

  qDef.AddNew();
  qDef["ID"]           = "YYY";
  qDef["CustID"]       = "  58";
  qDef["LockedStatus"] = SQL_NULL;
  qDef["StartDate"]    = COleDateTime(1998,02,05,10,30,0);
  qDef["ExportStatus"] = 1l;
  qDef.Update();

  qDef.MoveFirst();

  qDef.Edit();
  qDef["CustID"]    = "  57";
  qDef["StartDate"] = COleDateTime::GetCurrentTime();
  qDef.Update();
}
catch (CDBException* e)
{
  ...
}

AfxMessageBox(CQueryDef(&theApp.m_db,"select max(ID) from CUSTOMERS"));

CQueryDef qDef(&theApp.m_db); 
qDef.SetDateFormat("%Y-%m-%d"); 
qDef.SetDecimalDigits(3) 

qDef.Open(CRecordset::forwardOnly,"select ID,Name,StartDate,BalanceAmount from CUSTOMERS"); 

while (!qDef.IsEOF()) 
{ 
  if (qDef[0] == SQL_NULL) 
  continue; 

  m_list1.AddString(qDef["Name"]); 
  m_list2.AddString(qDef["StartDate"]);        // format is 1999-01-18 
  m_list3.AddString(qDef["BalanceAmount"]);    // format is 4552.923 

  qDef.MoveNext(); 
} 

SetDateFormat() and SetDecimalDigits() can be applied to the whole CQueryDef object or to individual columns. They have effect only on appropriate data types. When you apply one of these functions to the CQueryDef object level, it will overwrite individual column settings. An "EndingDate" column may be formatted different in the example above using:

qDef["EndingDate"].SetDateFormat("%m/%d/%y");

The string parameter for the SetDateFormat() function is documented in strftime() C runtime function.

CQueryDef  qDef(&theApp.m_db); 

q.Open(CRecordset::snapshot,"select a,b from t",CRecordset::none); 

try 
{ 
  while (!qDef.IsEOF()) 
  { 
     ... 
     qDef.MoveNext(); 
  } 
} 
catch (CDBException* e) 
{    // test the exception type 
  ...          // reestablish the link with the data source 
  qDef.ReOpen(newpDB); 
} 

The moment the link with the data source is lost (due to a communication problem) the framework will close all the recordsets opened on that connection. Using ReOpen() you may open again the CQueryDef object with the same attributes (w/o preserving the cursor position).

The notification to a user implemented sink, is started calling the function Advise():

EVNHANDLE CQueryDef::Advise(IQueryDefEventSink* pSink);

More than a sink may be connected to a CQueryDef object. The notification of all the sinks will be done in the order of the advise. One event is sent to a particular sink only and only if the sink just before it in the advise loop doesnt respond with a non 0 value. This means that you may choose to stop the event to "bubble" by returning a non 0 value from the yours implemented notification function. The handle returned from the Advise() function must be saved to be passed as the parameter to the Unadvise() function

void CQueryDef::Unadvise(EVNHANDLE evnHandle);

A disconnected sink will no longer receive any notification from the CQueryDef object. To temporarily stop the receiving of notifications you can call the FreezeEvents() function:

void CQueryDef::FreezeEvents(EVNHANDLE evnHandle,BOOL bFreeze=TRUE);

To resume receiving notifications youll have to call the same function with bFreeze FALSE. Just before a CQueryDef object is closed a Close notification will be sent and all the sinks are disconnected. After the CQueryDef object is closed the Advise handle becomes invalid and its use must be discontinued.

1. The CRecordset::SetFieldNull() and CRecordset::SetParamNull() functions dont fit in the QueryDef architecture. Their usage is forbidden and a call to any of them will result in a linker error. This behavior is by design. Instead use the assingment operator and the SQL_NULL constant.
2. CRecordset and CQueryDef henceforth, doesnt handle multiple TIMESTAMP columns. Furthermore such a column must be used at the end of the SQL query otherwise the "Invalid descriptor index" will be fired.
3. CQueryDef is not directly derived from CRecordset. Instead CRecordset2 is used to correct a problem related to binding string values w/o specifying the length of the string variable. Due to this problem QueryDef.DLL is related to RecordSetPatch.DLL. This secondary DLL must be shipped with your executable alongside QueryDef.DLL.

1. Include QueryDef.H file in your project.
2. Specify the QueryDef namespace by using USE_QUERYDEF macro in the files where you refer the library.
3. Use the CQueryDef class directly (you dont need to derive from it).
4. Have the QueryDef(D).LIB and RecordsetPatch(D).LIB files in your library path (D is for the debug version your executable will link with the correct version).
5. Put QueryDef(D).DLL and RecordsetPatch(D).DLL in your path.

1. It doesnt support multiple queries/batch of commands (maybe in a next version).
2. It doesnt have a good exception handling mechanism (although the CDBException can be used in relation with SQL exceptions) and requires care in manipulation due to automatic conversions that may slow down data processing.
3. The current version doesnt handle new UNICODE data types introduced by ODBC 3.51 (and present in SQL 7.0).

The example accompanying this document assumes the presence of an ODBC data source defined against a SQL Server 6.5 (7.0). The initial query works on the pubs database (make an ODBC DS with pubs the default database). Experiment with different queries and see the results. Call the formatting functions. Use stored procedures. Modify the source code to accommodate parameters. The TEST project was compiled with VC++ 6.0. The QueryDef library in the zip file is bult also with this compiler. Youll have to rebuild the library with VC++ 5.0 if you intend to use it with this compiler (see QueryDef.h file for special steps to take in library rebuild).

You may use this class library, even modify it w/o restriction if you specify the source. However you understand you do that it at your own risk.
The code in this class library represents my personal work and does not imply Computer Associates in any way.

Download source - 86 KB

Update.
The source files needed to build RecordsetPatchD.dll is here - 11 KB.