CTokenEx – A String Tokenizer Class

CodeGuru content and product recommendations are editorially independent. We may make money when you click on links to our partners. Learn More.

Environment: VC++ 6.0 SP3 (using Warning Level 4 in RELEASE Mode), Environments Tested: WinNT 4.0 SP4, Win95

Basically, I saw other string tokenizers here (CodeGuru) and they lacked the functionality I was
looking for.  Therefore, I created one for myself using the KISS method.  This is a
VERY SIMPLE sample!!!!

Here is a summary of the functionality in the CTokenEx class:

  • Can use “SplitPath” to break-up the path into sections (Drive/Sharename, Directory,
    Filename, Extention).  Also, recognizes UNC Names (which _tsplitpath doesn’t).

  • Can use “Join” to create a CString from a CStringArray with delimiters of your choice.
  • Can use “Split” to break-up a CString into a CStringArray (according to the delimiter).
  • Can use “GetString” to get the first sub-string in a CString (according to the delimiter).

NOTE:

The “Split” and “GetString” functions recognize multiple delimiters as an empty string so that
it will NOT add blanks to an Array.  See example code below: :

Say you have a CString that contains: “abc,def,,,ghi,,jkl,,”


//********************************************************
// Split Function
//********************************************************
//
// Split will fill an array with:
//
// String Position
// ====== ========
// abc 0
// def 1
// ghi 2
// jkl 3
//
//********************************************************
void CTokenDlgDlg::OnSplit()
{
CTokenEx tok;

// CString for the Split Function
CString csSplit = “abc,def,,,ghi,,jkl,,”;

// CStringArray to fill
CStringArray SplitIt;

// Call Split
tok.Split(csSplit, “,”, SplitIt);
}

/********************************************************
// GetString Function
//********************************************************
//
// GetString will return a string:
//
// abc
// …and more calls to GetString will return a strings:
// def
// ghi
// jkl
//
//********************************************************
void CTokenDlgDlg::OnGetstring()
{
CTokenEx tok;
char Buf[254];

CString csRef = “abc,def,,,ghi,,jkl,,”;
do {
CString csRet = tok.GetString(csRef, “,”);
// Do something with the returned value.
} while (!csRef.IsEmpty());
}

I hope that others find this class useful.

Downloads

Download demo project – 37 Kb
Download source – 99 Kb

More by Author

Get the Free Newsletter!

Subscribe to Developer Insider for top news, trends & analysis

Must Read