CTokenEx – A String Tokenizer Class

Environment: VC++ 6.0 SP3 (using Warning Level 4 in RELEASE Mode), Environments Tested: WinNT 4.0 SP4, Win95

Basically, I saw other string tokenizers here (CodeGuru) and they lacked the functionality I was
looking for.  Therefore, I created one for myself using the KISS method.  This is a
VERY SIMPLE sample!!!!

Here is a summary of the functionality in the CTokenEx class:

• Can use “SplitPath” to break-up the path into sections (Drive/Sharename, Directory,
  Filename, Extention).  Also, recognizes UNC Names (which _tsplitpath doesn’t).
• Can use “Join” to create a CString from a CStringArray with delimiters of your choice.
• Can use “Split” to break-up a CString into a CStringArray (according to the delimiter).
• Can use “GetString” to get the first sub-string in a CString (according to the delimiter).

NOTE:

The “Split” and “GetString” functions recognize multiple delimiters as an empty string so that
it will NOT add blanks to an Array.  See example code below: :

Say you have a CString that contains: “abc,def,,,ghi,,jkl,,”

  //********************************************************
  // Split Function
  //********************************************************
  //
  // Split will fill an array with:
  //
  // String  Position
  // ======  ========
  // abc     0
  // def     1
  // ghi     2
  // jkl     3
  //
  //********************************************************
  void CTokenDlgDlg::OnSplit()
  {
     CTokenEx tok;
     // CString for the Split Function
     CString csSplit = “abc,def,,,ghi,,jkl,,”;
     // CStringArray to fill
     CStringArray SplitIt;
     // Call Split
     tok.Split(csSplit, “,”, SplitIt);
  }
  /********************************************************
  // GetString Function
  //********************************************************
  //
  // GetString will return a string:
  //
  // abc
  //     …and more calls to GetString will return a strings:
  // def
  // ghi
  // jkl
  //
  //********************************************************
  void CTokenDlgDlg::OnGetstring()
  {
     CTokenEx tok;
     char Buf[254];
     CString csRef = “abc,def,,,ghi,,jkl,,”;
     do {
        CString csRet = tok.GetString(csRef, “,”);
        // Do something with the returned value.
     } while (!csRef.IsEmpty());
  }

I hope that others find this class useful.

Downloads

Download demo project – 37 Kb
Download source – 99 Kb