The Joy of Rotor

A funny thing happened to me as I was writing my first new column for CodeGuru. After a few months off writing "Visual C# .NET" for Microsoft Press, I was about half-way through an article on the .NET Framework's XSL namespace. Then on March 26, Microsoft released their shared-source version of the .NET Framework, commonly known as Rotor. My half-finished article on XSL now sits abandoned like last-week's pizza, as I have suddenly found myself unable to focus on anything but Rotor. In this week's article I'll discuss Rotor, cover some of the things I've discovered while installing Rotor on FreeBSD and Windows, and examine part of the Rotor source.

The announced intent of Rotor is to create a shared-source implementation of the .NET Framework that conforms to the ECMA standards. Rotor implements the portions of the .NET Framework that were submitted to ECMA, and are now known as ECMA 334 and ECMA 335. In addition, Rotor includes a remoting implementation that was not part of the ECMA submission.

But Rotor is much more than just a .NET Framework port to a Unix variant. It includes the entire source code required to build the .NET Framework, the C# and JScript compilers, and the associated tools. The source code and associated documentation includes a wealth of information about the .NET Framework. Although this code isn't the source code for the commercial .NET implementation for Windows, it does provide a look at one .NET implementation.

Rotor—The Good

You can have multiple installations of Rotor on a single machine. I'm currently running two separate installs on my BSD box (one is the plain vanilla version supplied by Microsoft, and one is a rapidly diverging version of the framework that I'm hacking on.) Since Rotor is designed to be cross-platform, there's no assumption made about the presence of the Win32 system registry. You can have multiple versions of the runtime installed, and you can select from the version you want to execute by changing your path. This also allows you to have Rotor installed at the same time as the commercial .NET Framework on Windows.

A great thing about debugging with Rotor is the SOS debugging library. Somehow the Rotor team has managed to build debugging extensions that work with windbg on Windows, and GDB on FreeBSD. I haven't had time to unravel the Kung-Fu required to make this happen, but a good starting point for more info about SOS is the documentation at sscli/docs/techinfo/sostips.html.

But probably the best thing about Rotor is the source code. Piles and piles of it—over 1.3 million lines of source. The source for the C# and JScript compilers, along with the source for ILDASM and similar tools should keep me busy and out of trouble for quite a while.

Rotor—The Bad

Not much to complain about, except that when running on BSD you can't debug with the Visual Studio .NET GUI debugger. In self-defense, I'm currently trying to convince my friends (and myself) that I actually like using a command-line debugger, but it doesn't seem to be working. If you're running on Windows, you can use the Visual Studio .NET debugger to step through the runtime or tools such as the C# compiler. However, you need to turn off debugging for managed code; otherwise Visual Studio thinks that you're trying to debug the commercial version of the framework.

And there are a few things that are missing and just won't be implemented by Microsoft. Of course you have the source for the ECMA portions of the framework, and there's no reason why a few geeks with time on their hands couldn't write a CLI host for another web server. I suspect that there will be quite a lot of activity with people extending the Microsoft bits.

Rotor—The Ugly

So there's over a million lines of source, and a complete implementation of the ECMA specs—what's the catch? For starters, the license is for non-commercial use. Microsoft has setup an email alias for commercial inquiries, but the standard license is for non-commercial use. Another limitation is that Rotor implements the ECMA 334 and ECMA 335 specifications, which don't include some parts of the commercial .NET Framework. For example, there's no Rotor equivalent of ASP.NET, ADO.NET, or Windows Forms. Nor is there any framework support for COM interop. And finally, there's no support for Visual Basic .NET with Rotor.

Personally, when I'm writing Windows Forms applications or targeting ASP .NET, I'll be using Visual Studio .NET anyway. For me, the positives for Rotor far outweigh the negatives. Rotor gives me a chance to see and use the internals of a CLI implementation, and it seems like I'm discovering cool stuff every hour.

Getting Rotor

So how do you get Rotor? The best approach is to download Rotor from MSDN. At the current time, the Rotor main page is here . However, the MSDN links are subject to change, so if that link is out-of-date, your best bet is to start at the MSDN page, here .

The build process for Rotor is amazingly consistent on both Windows and FreeBSD. There are three build options for Rotor:

  • The checked build has optimizations turned off, includes debug code, and includes debug symbols.
  • The fastchecked build has optimizations enabled, includes debug code, and includes debug symbols.
  • The free build has optimizations enabled, doesn't include debug code, and doesn't include debug symbols.

By default, the build process will create a fastchecked build.

To build Rotor on Windows XP or Windows 2000, you must have Visual C++ installed—the build process uses Visual C++ to build portions of the Rotor runtime. UseWinZip or a similar utility to expand the distribution into a subdirectory on your computer. Open a command console window, and execute the env.bat batch file, which will configure your environment for building the sources. Next, launch the build process in the same command console window by executing the buildall.bat batch file.

To build Rotor on FreeBSD, you'll need to expand the distribution tarball. If you're familiar with FreeBSD, you'll know what that means. For the rest of you (and you know who you are) I suggest reading the excellent FreeBSD user documentation at www.freebsd.org. For the extremely impatient, copy the distribution into a subdirectory and use these commands to expand the source:

   gunzip sscli200020326.tgz
   tar xf sscli200020326.tar

Next, source the environment by executing this script:

   source env.sh

If this script doesn't work with your current shell (for example if you're using bash or tcsh), you'll be promtped to execute this script instead:

   source env.csh

After the script has successfully executed, you'll see this message:

   Fastchecked Environment

Next, start the build process by starting the buildall script:

   sh < buildall

The build process is rather long—many of the build components are C and C++ source files that are built using the GNU C++ compiler. As the GNU compiler is quite resource hungry, be prepared for a long build. My personal approach to building the Rotor sources on FreeBSD employs a three-prong effort:

  1. Install Rotor on a Windows machine as well as the FreeBSD box
  2. During compilation, dig through my computer graveyard looking for additional RAM for the BSD box
  3. During compilation, refer to the Windows machine source

After the build has completed, and assuming you haven't reached mandatory retirement age, you can start exploring the sources. However, before you modify any sources you should execute the test script discussed in the next section. If you're you're just browsing, feel free to postpone this step for now—it takes longer than the compilation, and will degrade performance while the tests are executing.

Testing

Perl scripts are used to run automated tests that verify the framework and class libraries. You can use these test scripts to help ensure that any changes that you make to the Rotor source don't break the framework. To kick off the tests, execute the rrun.pl Perl script found in the sscli/tests subdirectory. Depending on the machine, testing can take a long time, or a very long time—my underpowered FreeBSD box initially took 10 hours to process all of the tests before I increased the RAM and recompiled the kernel to take advantage of the second processor.

After tests have been completed, a summary of the test results is displayed. You can also view the detailed results in the logfile at sscli/test/rrun.log.

Building Example Programs

Rotor comes with several example programs, including the ubiquitous Hello World program, which can be found at sscli/tools/hello. To compile a C# program under Rotor, just invoke the compiler, much like when compiling for the commercial .NET Framework:

   csc hello.cs

To execute the program, you must use clix to host your program, like this:

   clix hello

The next section discusses clix in more detail.

Let's Look at Some Code

So enough talk—let's do something with the Rotor source. An interesting part of the framework is the clix command. Rather than supply extensions to FreeBSD to enable launching managed applications, the Rotor team supplied a single host—clix, which is responsible for loading and executing your managed code. The source code for clix can be found in the sscli/clr/src/tools/clix subdirectory (Feel free to exchange forward slashes and backward slashes depending on your current OS context).

The main method inside clix.cpp parses the command-line arguments passed to it, and calls the launch method in order to actually start your application. I won't show the source here, but the key line of code is:

    nExitCode = Launch( pRuntimeName,
                        pModuleName,
                        pActualCmdLine);

The Launch method is composed of code that looks like it comes straight out of a program written for the Platform SDK. The first few lines of the Launch method are shown below, and are used to load the managed executable:

DWORD Launch( WCHAR* pRunTime,
              WCHAR* pFileName,
              WCHAR* pCmdLine)
{
    HANDLE hFile = NULL;
    HANDLE hMapFile = NULL;
    PVOID pModule = NULL;
    HINSTANCE hRuntime = NULL;
    DWORD nExitCode = 1;
    DWORD dwSize;
    DWORD dwSizeHigh;
    IMAGE_DOS_HEADER* pdosHeader;
    IMAGE_NT_HEADERS32* pNtHeaders;
    IMAGE_SECTION_HEADER*   pSectionHeader;
    WCHAR exeFileName[MAX_PATH + 1];

    // open the file & map it
    hFile = ::CreateFile(pFileName, GENERIC_READ,
                         FILE_SHARE_READ,
                         0, OPEN_EXISTING, 0, 0);
    if (hFile == INVALID_HANDLE_VALUE)
    {
        // If the file doesn't exist, append
        // a '.exe' extension and
        // try again.
        nExitCode = ::GetLastError();
        if (nExitCode == ERROR_FILE_NOT_FOUND)
        {
           const WCHAR *exeExtension = L".exe";
           if (wcslen(pFileName) + wcslen(exeExtension) <
                   sizeof(exeFileName) / sizeof(WCHAR))
           {
                wcscpy(exeFileName, pFileName);
                wcscat(exeFileName, exeExtension);
                hFile = ::CreateFile(exeFileName, 
                                     GENERIC_READ,
                                     FILE_SHARE_READ,
                                     0,
                                     OPEN_EXISTING,
                                     0, 0);
                if (hFile != INVALID_HANDLE_VALUE)
                {
                    pFileName = exeFileName;
                }
            }
        }
        if (hFile == INVALID_HANDLE_VALUE)
        {
            DisplayMessage( MSG_CantOpenExe,
                            nExitCode,
                            pFileName);
            goto Error;
        }
    }

When built on a Windows operating system, the calls to CreateFile and GetLastError predictably invoke the Windows API. When built for FreeBSD, the CreateFile, GetLastError, and other methods are implemented by the Portable Adaptation Layer, or PAL. This layer of code is responsible for providing a portability layer so that basic functionality is available on all platforms. The interface for the PAL can be found in the rotor_pal.h header file at sscli/pal. I'll dig deeper into the PAL in a future column, but don't wait for me—the source is available at sscli/pal/unix.

Meanwhile, the Load method is busy doing some housekeeping work, checking the file to make sure that it includes the proper headers and is in the correct format. As part of this process, the file is memory mapped using this code:

    hMapFile = ::CreateFileMapping( hFile,
                                    NULL,
                                    PAGE_WRITECOPY,
                                    0, 0, NULL);
    if (hMapFile == NULL)
    {
        nExitCode = ::GetLastError();
        DisplayMessage(MSG_OutOfMemory);
        goto Error;
    }

    pModule = ::MapViewOfFile( hMapFile,
                               FILE_MAP_COPY,
                               0, 0, 0);
    if (pModule == NULL)
    {
        nExitCode = ::GetLastError();
        DisplayMessage(MSG_OutOfMemory);
        goto Error;
    }

    dwSize = GetFileSize(hFile, &dwSizeHigh);
    if (dwSize == INVALID_FILE_SIZE)
    {
        nExitCode = ::GetLastError();
        if (nExitCode == 0)
            nExitCode = ERROR_BAD_FORMAT;
        DisplayMessage(MSG_BadFileSize);
        goto Error;
    }

Again, the PAL provides the necessary functionality on FreeBSD. If the managed executable appears to be valid, the runtime library is loaded. On Windows, this file is named sscoree.dll. On FreeBSD, the library is named sscoree.so. The code used to load the runtime is shown here:

    // load the runtime and go
    hRuntime = ::LoadLibrary(pRunTime);
    if (hRuntime == NULL)
    {
        nExitCode = ::GetLastError();
        DisplayMessage(MSG_CantLoadEE, nExitCode, pRunTime);
        goto Error;
    }

On Windows, this call falls through to the Win32 API. On FreeBSD, the PAL uses the code found in sscli/pal/unix/loader/module.c. This module includes methods that mimic LoadLibrary, FreeLibrary, and similar functions exposed by the Win32 API.

The code used to pass the managed code to the runtime is shown here:

  __int32 (STDMETHODCALLTYPE * pCorExeMain2)(
       PBYTE   pUnmappedPE,        // -> memory mapped code
       DWORD   cUnmappedPE,        // Size of memory mapped code
       LPWSTR  pImageNameIn,       // -> Executable Name
       LPWSTR  pLoadersFileName,   // -> Loaders Name
       LPWSTR  pCmdLine);          // -> Command Line

    *((VOID**)&pCorExeMain2) = ::GetProcAddress(hRuntime,
                                                "_CorExeMain2");
    if( pCorExeMain2 == NULL)
    {
        nExitCode = ::GetLastError();
        DisplayMessage( MSG_CantFindExeMain,
                        nExitCode,
                        pRunTime);
        goto Error;
    }

    nExitCode = (int)pCorExeMain2((PBYTE)pModule,
                       dwSize,
                       pFileName,    // -> Executable Name
                       NULL,         // -> Loaders Name
                       pCmdLine);    // -> Command Line

Error:
    ::UnmapViewOfFile(pModule);
    ::CloseHandle(hMapFile);
    ::CloseHandle(hFile);

    return nExitCode;
} 

This code declares a function pointer named pCorExeMain2 that's used to invoke the the _CorExeMain2 method in the runtime. Information about the managed exe is passed to this method, and the runtime executes the managed code. The _CorExeMain2 method is part of the runtime's virtual machine, and can be found in ceemain.cpp, which can be found at sscli/clr/src/vm. _CorExeMain2 starts up the CLI's execution engine, and executes the managed code. After execution, the return value is passed back to the runtime.

Whew! That's about it for this week. My next column will focus on the type system in the .NET Framework, and we'll do a bit more spelunking into the Rotor source.

About the Author

Mickey Williams is the founder of Codev Technologies, a provider of tools and consulting for Windows Developers. He is also on the staff at .NET Experts (www.dotnetexperts.com), where he teaches the .NET Framework course. He has spoken at conferences in the USA and Europe, and has written nine books on Windows programming. He has recently completed "Visual C# .NET" for Microsoft Press, which will be available in May. Mickey can be reached at mw@codevtech.com.



Comments

  • There are no comments yet. Be the first to comment!

Leave a Comment
  • Your email address will not be published. All fields are required.

Top White Papers and Webcasts

  • The first phase of API management was about realizing the business value of APIs. This next wave of API management enables the hyper-connected enterprise to drive and scale their businesses as API models become more complex and sophisticated. Today, real world product launches begin with an API program and strategy in mind. This API-first approach to development will only continue to increase, driven by an increasingly interconnected web of devices, organizations, and people. To support this rapid growth, …

  • Live Event Date: August 19, 2014 @ 11:00 a.m. ET / 8:00 a.m. PT You deployed your app with the Bluemix PaaS and it's gaining some serious traction, so it's time to make some tweaks. Did you design your application in a way that it can scale in the cloud? Were you even thinking about the cloud when you built the app? If not, chances are your app is going to break. Check out this upcoming webcast to learn various techniques for designing applications that will scale successfully in Bluemix, for the …

Most Popular Programming Stories

More for Developers

Latest Developer Headlines

RSS Feeds