.NET Nuts '& Bolts - The Joy of Rotor

A funny thing happened to me as I was writing my first new column for CodeGuru. After a few months off writing "Visual C# .NET" for Microsoft Press, I was about half-way through an article on the .NET Frameworks XSL namespace. Then on March 26, Microsoft released their shared-source version of the .NET Framework, commonly known as Rotor. My half-finished article on XSL now sits abandoned like last-weeks pizza, as I have suddenly found myself unable to focus on anything but Rotor. In this weeks article Ill discuss Rotor, cover some of the things Ive discovered while installing Rotor on FreeBSD and Windows, and examine part of the Rotor source.

The announced intent of Rotor is to create a shared-source implementation of the .NET Framework that conforms to the ECMA standards. Rotor implements the portions of the .NET Framework that were submitted to ECMA, and are now known as ECMA 334 and ECMA 335. In addition, Rotor includes a remoting implementation that was not part of the ECMA submission.

But Rotor is much more than just a .NET Framework port to a Unix variant. It includes the entire source code required to build the .NET Framework, the C# and JScript compilers, and the associated tools. The source code and associated documentation includes a wealth of information about the .NET Framework. Although this code isnt the source code for the commercial .NET implementation for Windows, it does provide a look at one .NET implementation.

Rotor The Good

You can have multiple installations of Rotor on a single machine. Im currently running two separate installs on my BSD box (one is the plain vanilla version supplied by Microsoft, and one is a rapidly diverging version of the framework that Im hacking on.) Since Rotor is designed to be cross-platform, theres no assumption made about the presence of the Win32 system registry. You can have multiple versions of the runtime installed, and you can select from the version you want to execute by changing your path. This also allows you to have Rotor installed at the same time as the commercial .NET Framework on Windows.

A great thing about debugging with Rotor is the SOS debugging library. Somehow the Rotor team has managed to build debugging extensions that work with windbg on Windows, and GDB on FreeBSD. I havent had time to unravel the Kung-Fu required to make this happen, but a good starting point for more info about SOS is the documentation at sscli/docs/techinfo/sostips.html.

But probably the best thing about Rotor is the source code. Piles and piles of it over 1.3 million lines of source. The source for the C# and JScript compilers, along with the source for ILDASM and similar tools should keep me busy and out of trouble for quite a while.

Rotor The Bad

Not much to complain about, except that when running on BSD you cant debug with the Visual Studio .NET GUI debugger. In self-defense, Im currently trying to convince my friends (and myself) that I actually like using a command-line debugger, but it doesnt seem to be working. If youre running on Windows, you can use the Visual Studio .NET debugger to step through the runtime or tools such as the C# compiler. However, you need to turn off debugging for managed code; otherwise Visual Studio thinks that youre trying to debug the commercial version of the framework.

And there are a few things that are missing and just wont be implemented by Microsoft. Of course you have the source for the ECMA portions of the framework, and theres no reason why a few geeks with time on their hands couldnt write a CLI host for another web server. I suspect that there will be quite a lot of activity with people extending the Microsoft bits.

Rotor The Ugly

So theres over a million lines of source, and a complete implementation of the ECMA specs - what's the catch? For starters, the license is for non-commercial use. Microsoft has setup an email alias for commercial inquiries, but the standard license is for non-commercial use. Another limitation is that Rotor implements the ECMA 334 and ECMA 335 specifications, which dont include some parts of the commercial .NET Framework. For example, theres no Rotor equivalent of ASP.NET, ADO.NET, or Windows Forms. Nor is there any framework support for COM interop. And finally, theres no support for Visual Basic .NET with Rotor.

Personally, when Im writing Windows Forms applications or targeting ASP .NET, Ill be using Visual Studio .NET anyway. For me, the positives for Rotor far outweigh the negatives. Rotor gives me a chance to see and use the internals of a CLI implementation, and it seems like Im discovering cool stuff every hour.

Getting Rotor

So how do you get Rotor? The best approach is to download Rotor from MSDN. At the current time, the Rotor main page is here . However, the MSDN links are subject to change, so if that link is out-of-date, your best bet is to start at the MSDN page, here .

The build process for Rotor is amazingly consistent on both Windows and FreeBSD. There are three build options for Rotor:

  • The checked build has optimizations turned off, includes debug code, and includes debug symbols.
  • The fastchecked build has optimizations enabled, includes debug code, and includes debug symbols.
  • The free build has optimizations enabled, doesn't include debug code, and doesn't include debug symbols.

By default, the build process will create a fastchecked build.

To build Rotor on Windows XP or Windows 2000, you must have Visual C++ installed - the build process uses Visual C++ to build portions of the Rotor runtime. UseWinZip or a similar utility to expand the distribution into a subdirectory on your computer. Open a command console window, and execute the env.bat batch file, which will configure your environment for building the sources. Next, launch the build process in the same command console window by executing the buildall.bat batch file.

To build Rotor on FreeBSD, you'll need to expand the distribution tarball. If you're familiar with FreeBSD, you'll know what that means. For the rest of you (and you know who you are) I suggest reading the excellent FreeBSD user documentation at www.freebsd.org. For the extremely impatient, copy the distribution into a subdirectory and use these commands to expand the source:

   gunzip sscli200020326.tgz
   tar xf sscli200020326.tar

Next, source the environment by executing this script:

   source env.sh

If this script doesn't work with your current shell (for example if you're using bash or tcsh), you'll be promtped to execute this script instead:

   source env.csh

After the script has successfully executed, you'll see this message:

   Fastchecked Environment

Next, start the build process by starting the buildall script:

   sh < buildall

The build process is rather long - many of the build components are C and C++ source files that are built using the GNU C++ compiler. As the GNU compiler is quite resource hungry, be prepared for a long build. My personal approach to building the Rotor sources on FreeBSD employs a three-prong effort:

  1. Install Rotor on a Windows machine as well as the FreeBSD box
  2. During compilation, dig through my computer graveyard looking for additional RAM for the BSD box
  3. During compilation, refer to the Windows machine source

After the build has completed, and assuming you haven't reached mandatory retirement age, you can start exploring the sources. However, before you modify any sources you should execute the test script discussed in the next section. If you're you're just browsing, feel free to postpone this step for now - it takes longer than the compilation, and will degrade performance while the tests are executing.

Testing

Perl scripts are used to run automated tests that verify the framework and class libraries. You can use these test scripts to help ensure that any changes that you make to the Rotor source don't break the framework. To kick off the tests, execute the rrun.pl Perl script found in the sscli/tests subdirectory. Depending on the machine, testing can take a long time, or a very long time - my underpowered FreeBSD box initially took 10 hours to process all of the tests before I increased the RAM and recompiled the kernel to take advantage of the second processor.

After tests have been completed, a summary of the test results is displayed. You can also view the detailed results in the logfile at sscli/test/rrun.log.

Building Example Programs

Rotor comes with several example programs, including the ubiquitous Hello World program, which can be found at sscli/tools/hello. To compile a C# program under Rotor, just invoke the compiler, much like when compiling for the commercial .NET Framework:

   csc hello.cs

To execute the program, you must use clix to host your program, like this:

   clix hello

The next section discusses clix in more detail.

Lets Look at Some Code

So enough talk lets do something with the Rotor source. An interesting part of the framework is the clix command. Rather than supply extensions to FreeBSD to enable launching managed applications, the Rotor team supplied a single host clix, which is responsible for loading and executing your managed code. The source code for clix can be found in the sscli/clr/src/tools/clix subdirectory (Feel free to exchange forward slashes and backward slashes depending on your current OS context).

The main method inside clix.cpp parses the command-line arguments passed to it, and calls the launch method in order to actually start your application. I wont show the source here, but the key line of code is:

    nExitCode = Launch( pRuntimeName,
                        pModuleName,
                        pActualCmdLine);

The Launch method is composed of code that looks like it comes straight out of a program written for the Platform SDK. The first few lines of the Launch method are shown below, and are used to load the managed executable:

DWORD Launch( WCHAR* pRunTime,
              WCHAR* pFileName,
              WCHAR* pCmdLine)
{
    HANDLE hFile = NULL;
    HANDLE hMapFile = NULL;
    PVOID pModule = NULL;
    HINSTANCE hRuntime = NULL;
    DWORD nExitCode = 1;
    DWORD dwSize;
    DWORD dwSizeHigh;
    IMAGE_DOS_HEADER* pdosHeader;
    IMAGE_NT_HEADERS32* pNtHeaders;
    IMAGE_SECTION_HEADER*   pSectionHeader;
    WCHAR exeFileName[MAX_PATH + 1];

    // open the file & map it
    hFile = ::CreateFile(pFileName, GENERIC_READ,
                         FILE_SHARE_READ,
                         0, OPEN_EXISTING, 0, 0);
    if (hFile == INVALID_HANDLE_VALUE)
    {
        // If the file doesn't exist, append
        // a '.exe' extension and
        // try again.
        nExitCode = ::GetLastError();
        if (nExitCode == ERROR_FILE_NOT_FOUND)
        {
           const WCHAR *exeExtension = L".exe";
           if (wcslen(pFileName) + wcslen(exeExtension) <
                   sizeof(exeFileName) / sizeof(WCHAR))
           {
                wcscpy(exeFileName, pFileName);
                wcscat(exeFileName, exeExtension);
                hFile = ::CreateFile(exeFileName, 
                                     GENERIC_READ,
                                     FILE_SHARE_READ,
                                     0,
                                     OPEN_EXISTING,
                                     0, 0);
                if (hFile != INVALID_HANDLE_VALUE)
                {
                    pFileName = exeFileName;
                }
            }
        }
        if (hFile == INVALID_HANDLE_VALUE)
        {
            DisplayMessage( MSG_CantOpenExe,
                            nExitCode,
                            pFileName);
            goto Error;
        }
    }

When built on a Windows operating system, the calls to CreateFile and GetLastError predictably invoke the Windows API. When built for FreeBSD, the CreateFile, GetLastError, and other methods are implemented by the Portable Adaptation Layer, or PAL. This layer of code is responsible for providing a portability layer so that basic functionality is available on all platforms. The interface for the PAL can be found in the rotor_pal.h header file at sscli/pal. I'll dig deeper into the PAL in a future column, but dont wait for me the source is available at sscli/pal/unix.

Meanwhile, the Load method is busy doing some housekeeping work, checking the file to make sure that it includes the proper headers and is in the correct format. As part of this process, the file is memory mapped using this code:

    hMapFile = ::CreateFileMapping( hFile,
                                    NULL,
                                    PAGE_WRITECOPY,
                                    0, 0, NULL);
    if (hMapFile == NULL)
    {
        nExitCode = ::GetLastError();
        DisplayMessage(MSG_OutOfMemory);
        goto Error;
    }

    pModule = ::MapViewOfFile( hMapFile,
                               FILE_MAP_COPY,
                               0, 0, 0);
    if (pModule == NULL)
    {
        nExitCode = ::GetLastError();
        DisplayMessage(MSG_OutOfMemory);
        goto Error;
    }

    dwSize = GetFileSize(hFile, &dwSizeHigh);
    if (dwSize == INVALID_FILE_SIZE)
    {
        nExitCode = ::GetLastError();
        if (nExitCode == 0)
            nExitCode = ERROR_BAD_FORMAT;
        DisplayMessage(MSG_BadFileSize);
        goto Error;
    }

Again, the PAL provides the necessary functionality on FreeBSD. If the managed executable appears to be valid, the runtime library is loaded. On Windows, this file is named sscoree.dll. On FreeBSD, the library is named sscoree.so. The code used to load the runtime is shown here:

    // load the runtime and go
    hRuntime = ::LoadLibrary(pRunTime);
    if (hRuntime == NULL)
    {
        nExitCode = ::GetLastError();
        DisplayMessage(MSG_CantLoadEE, nExitCode, pRunTime);
        goto Error;
    }

On Windows, this call falls through to the Win32 API. On FreeBSD, the PAL uses the code found in sscli/pal/unix/loader/module.c. This module includes methods that mimic LoadLibrary, FreeLibrary, and similar functions exposed by the Win32 API.

The code used to pass the managed code to the runtime is shown here:

  __int32 (STDMETHODCALLTYPE * pCorExeMain2)(
       PBYTE   pUnmappedPE,        // -> memory mapped code
       DWORD   cUnmappedPE,        // Size of memory mapped code
       LPWSTR  pImageNameIn,       // -> Executable Name
       LPWSTR  pLoadersFileName,   // -> Loaders Name
       LPWSTR  pCmdLine);          // -> Command Line

    *((VOID**)&pCorExeMain2) = ::GetProcAddress(hRuntime,
                                                "_CorExeMain2");
    if( pCorExeMain2 == NULL)
    {
        nExitCode = ::GetLastError();
        DisplayMessage( MSG_CantFindExeMain,
                        nExitCode,
                        pRunTime);
        goto Error;
    }

    nExitCode = (int)pCorExeMain2((PBYTE)pModule,
                       dwSize,
                       pFileName,    // -> Executable Name
                       NULL,         // -> Loaders Name
                       pCmdLine);    // -> Command Line

Error:
    ::UnmapViewOfFile(pModule);
    ::CloseHandle(hMapFile);
    ::CloseHandle(hFile);

    return nExitCode;
} 

This code declares a function pointer named pCorExeMain2 thats used to invoke the the _CorExeMain2 method in the runtime. Information about the managed exe is passed to this method, and the runtime executes the managed code. The _CorExeMain2 method is part of the runtimes virtual machine, and can be found in ceemain.cpp, which can be found at sscli/clr/src/vm. _CorExeMain2 starts up the CLIs execution engine, and executes the managed code. After execution, the return value is passed back to the runtime.

Whew! Thats about it for this week. My next column will focus on the type system in the .NET Framework, and well do a bit more spelunking into the Rotor source.

About the Author

Mickey Williams is the founder of Codev Technologies, a provider of tools and consulting for Windows Developers. He is also on the staff at .NET Experts (www.dotnetexperts.com), where he teaches the .NET Framework course. He has spoken at conferences in the USA and Europe, and has written nine books on Windows programming. He has recently completed "Visual C# .NET" for Microsoft Press, which will be available in May. Mickey can be reached at mw@codevtech.com.



Comments

  • Rotor and Forms ?

    Posted by Legacy on 06/17/2002 12:00am

    Originally posted by: Lloyd

    is the Sstem.Windows.Forms namespace present in Rotor ?
    
    I mean is it possible to build an application ?

    Reply
  • We'll take technology, please preach elsewhere

    Posted by Legacy on 05/15/2002 12:00am

    Originally posted by: Roshan James

    Mickey I agree with you completely.
    Dear Ganesh if you need to preach please take it elsewhere, there are pleanty of preach websites out there. Take a look at this one, this is for the technology enthusiasts, not for the stereotype preachers or propogandists.

    All of us, any of us, seriously at heart love out programming, the way of the art first. We take sides and pick enemies second. I am not going to comment on the open source much less the fsf, enough has been said, but please, we would appreaciate some quiet mindspace here.

    You can take a look at the work we do, use the code we write, anyone if free here to sell or not sell, to buy or not buy, to chose to be free or not to be, and thats freedom for us. By imposing someone brand of political freedom on anyone doesnt make things any free-er and dividing people didnt make things any more lucid.

    You are free to join us to learn or may please graciously let us be. If you have noticed, that is why the community, as we see it, dont sling jokes or insult back at every ms or windows programmer joke. We simply have better things to do and if you want to partake of it, please you are only welcome. If you dont, please go your way silently and let us be.

    Just as a footnote: we also appreciate Miguel of Mono. Atleast that man shuts up and codes, and does a good job of it. You could learn something from that. Somewhere i came across this vote on a linux page that beacuse he started on the .net venture he should be 'pecked to death by a thousand chickens'. Sorry Our Community doesnt appreaciate that kind of talk.

    That much time wasted.

    hey some some of the gc algo's are pretty fab arent they ?

    Roshan

    Reply
  • Open Source but not GPL

    Posted by Legacy on 05/13/2002 12:00am

    Originally posted by: Brian Pribis

    Great article. The co I work for is thinking of using .NET for some of its solutions, so the more info the better. Having access to .NET and the source is a big help as I ramp up to the task.
    It is interesting that MS went open source with this and appears to show a certain level of maturity on their part. Over a million lines of code is a lot to just throw out there. I just hope that the OS community will rise to the challenge and show MS and the rest that we aren't out there to steal code and get everything for free. The fact is Open Source works as a development/deployment model.

    Reply
  • Debugging Rotor

    Posted by Legacy on 04/20/2002 12:00am

    Originally posted by: Lee


    Mickey,

    Firstly thank you for an excellent article. Now that you have got many of us up and running with Rotor I think that it would be a sound idea to follow up your previous article with a more detailed survey of the Rotor landscape but also some debugging tips. I for one would particularly appreciate the latter; I miss the IDE of VisStudio.net when debugging say the hello.cs source. One work around I have found is to use the Rotor cs.exe to generate the hello.exe, launch hello.exe using clix from a DOS box using NTSD. An attempt can be made to perform an attach to the clix process. Due to NTSD no threads inside hello.exe will be running and you can then proceed to set breakpoints within the clix source. To some extent this is helpful because you can see where the JIT kicks in but obviously you cannot set breakpoints in the target hello.cs: this needs to be debugged separately using cordbg.

    What is your prefered Rotor debugging technique?

    Reply
  • Has anyone compared Rotor with Mono?

    Posted by Legacy on 04/16/2002 12:00am

    Originally posted by: Neelesh Pandit

    http://www.go-mono.net is the open source initiative at implementing a "clean" version of .NET (i.e. without looking at any MS code to avoid any potential copyright/licence infringments)

    Reply
  • indeed microsoft is goin open at last

    Posted by Legacy on 04/15/2002 12:00am

    Originally posted by: tarundeep singh kalra

    A really informative article;good efforts
    at last microsoft realized the power of open source and has realeasd the source codes;
    but still lot has to be waited for;
    hail!! open source

    Reply
  • Interesting

    Posted by Legacy on 04/14/2002 12:00am

    Originally posted by: Steven Carleton

    Thanks for exploring this issue for us!
    Hope to hear lots more from you about Rotor.

    I learned alot by studying the Microsoft MFC
    and C++ runtime files. I'm sure Rotor will
    be a big help in learning about C# & how to
    use the .NET framework.

    Eventhough my company only does Windows, it
    would be cool if we could get Rotor contributors
    from other platforms.

    Hopefully, the Rotor source is a good demo
    of .NET "best practices".

    Reply
  • Trusting techo

    Posted by Legacy on 04/09/2002 12:00am

    Originally posted by: Ganesh Prasad

    Poor guy, he really seems to believe that it is possible to build 
    
    a cross-platform version of .NET, even though Microsoft have
    exposed their hand by only releasing some parts of the spec,
    and even that only as Shared Source (a "look but don't
    touch" license), not Open Source.

    One thing is for sure. Microsoft are in the Microsoft platform
    business. They are not in the cross-platform business, even if
    they say they are. They care for their bottom line.

    If free .NET implementations on FreeBSD become very popular
    and threaten Microsoft revenues, the lawyers will appear and
    revoke all rights to such implementations. There is no point in
    accepting source code when it can be reeled back in. Only
    Open Source *guarantees* that code will perpetually be
    available to the public. Would you be willing to build your
    house on "donated" land that the donor can take back
    anytime?

    The author appears to be a pure techo dazzled by technology
    and unable to see the business imperatives behind the
    technology. I wouldn't waste a minute of my time on this
    project if I were him.

    Ganesh


    Reply
  • x86 and Win2K

    Posted by Legacy on 04/06/2002 12:00am

    Originally posted by: Alok Govil

    Hi all,

    I know nothing about C# or .Net

    And my aim is to extract source code of C# compiler/linker/assembler, and modify it to make it a regular compiler that would "itself" compile with any standard C++ compiler and for any platform.

    Finally, I wish to modify the compiler source code so that it compiles given program to native x86 instruction set.

    Does somebody with some insights into .Net and C# can help me guess how much effort that would be and what would be the major issues to be handled.

    I am guessing that a CLI run-time does not uses a virtual machine, and so a CLI executable would have native machine code. If that is not the case, then I will have to dig into this virtual machine source also.

    MS says that this system will work on only Windows XP or FreeBSD.

    What is it that will stop it from running on say Win2K. The pieces of WinXP-specific code would be the first thing I want to modify.

    Any help will be appreciated.

    Regards - Alok

    Reply
Leave a Comment
  • Your email address will not be published. All fields are required.

Top White Papers and Webcasts

  • As all sorts of data becomes available for storage, analysis and retrieval - so called 'Big Data' - there are potentially huge benefits, but equally huge challenges...
  • The agile organization needs knowledge to act on, quickly and effectively. Though many organizations are clamouring for "Big Data", not nearly as many know what to do with it...
  • Cloud-based integration solutions can be confusing. Adding to the confusion are the multiple ways IT departments can deliver such integration...

Most Popular Programming Stories

More for Developers

RSS Feeds

Thanks for your registration, follow us on our social networks to keep up-to-date