DLL Confusion

[MrDoomMaster]

Currently, my library is a static library, but I am going to convert it into a DLL file.

I have indeed gone to codeproject.com and read some tutorials on DLL files, but I still am very confused on a few things:

1) What does Export mean in terms of DLL Libraries?
2) What does Import mean in terms of DLL Libraries?
3) Why are both header files AND import libraries required to use DLL files?
4) When would you Import and when would you Export? (This may be part of questions #1 and #2)
5) What is the difference in the content located in the header files and the import library?
6) What content is located in the import library?
7) Using Visual Studio .NET 2003, must you create the import library and DLL file with two seperate projects? If no, how are both created at once? To me this seems like a project creating two EXE files. Very unusual.
8) What content is required to be inside DllMain()?
9) Would the project would require a seperate source file for the DllMain function, as well as the other H/CPP file pairs that consist of your library?

It may seem as if I didn't read any tutorials at all, but I think I've actually read a very poor tutorial. Maybe someone can post a good tutorial. In the meantime I'll be reading some more.

I thank anyone for answers they may provide.

[kirants]

1) What does Export mean in terms of DLL Libraries?


You can view the dll as a bunch of services. Now, to use the servioces, you need to make the services public. You do that by exporting functions from the DLL. Here is how you can look at it. A dll has a bunch of sections. Onen for code, one for resources, one for metadata for .Net apps etc.. These sections are defined by what is called a Portbale executable file format for executables for Windows platforms.

1 of the section is called an export data section. This section is essentially a table. Look at this as a table with as many entries as the no. of functiosn the dll exports. The table's entry will have the fucntion name and the location of the code in the dll's binary file where that code is present.

So, when you add a new fucntion in a dll and specify that you need to export it, a new entry is created for it by the linker when u link the dll.

2) What does Import mean in terms of DLL Libraries?

Similar to export.. The imports are those functions that you have specified that your component should look for in other components.

Say, you develeop a Win32 app. Most likely you will use something like CreateWindow. This is a function that you haven't written, but is a fucntion that user32.dll of windows provides. When you link your code, linker will look for all of your code to see if it finds it. If it , it uses that. If it doesn't you need to tell linker where to look at. You do this by specifying what libraries you want to look in by including user32.lib in your linker settings. So, the linker will now read this .lib. Now, this lib can be a static lib or an import lib. If it is a static lib, the code in the .lb becomes part of your component. If it is an import lib, the linker now starts manipulating a section called import data section of your final output to be generated.

It adds an entry for the dll which user32.lib is mentioning, and it adds a list of the functions ( services ) that you are importing in your own component. IN your case, CreateWindow .. It adds this literal string in there.

So, when compiled and linked, the component will have an import data section which will look something like
[section start] ["dll1.dll"]["CreateWIndow"]["FindWIndow"] .... ["dll2.dll"]["func1"]...[section end]

Now, when the OS is loading an application to be execute, it has to do some processing..

It will first load the exe ( a simple example ) , read through the exe's import libraries and tries to find them [in a specific search order ] .. if it finds them, it is going to load them to the process' space and tries to fix up the calls to actually go to the specific dll ( which in itself is an interesting topic ).

So, if for some reason it doesn't find the dll or the dll doesn't export that function ( an older version or so ), the exe shall fail to load and you will see a message box saying "blah blah balh not found in .. dll )

3) Why are both header files AND import libraries required to use DLL files?
Header files are compile time things. It specifies what parameters the function takes and sees to it that while calling the fucntion, the params passed are matching.

import libraries are needed for resolving. Whatever code is generated, the linker has to know what is go where. For e.g. if you call f(), the compiler is happy as long as you have supplied with a definition for f(). WHen linker takes over , it has to know where f() is to let it put a call instruction to the location of the f() body ( ie. the actual machine code ). This is where libs are needed. The lib will specify what dll it is implemented in and the linker then adds this dll to the import section and also an entry for the function name imported

[QUPTE]4) When would you Import and when would you Export? (This may be part of questions #1 and #2)[/QUOTE]

You have been importing uncosciously all the time if you were writing WIn32 applications. All of these, RegisterClass, CreateWIndow, GetMessage, TranslateMessage, LoadIcon etc are all exports from system dlls that you are importing into your app.

You would export from your dll , when you want the fucntion to be publicly available to other components without them having to merge any of your source code. Also, you would export when your need is to allow the code to be reused and shared. If you have a fucntion , say SpecialDivision implememented in your dll, you could export this and make it available for multiple applications. You just supply the .h file, .lib file and let them use it.

If you do not supply a .lib file, the clients can still use LoadLibrary(),GetprocAddress() to locate the function and use it.

5) What is the difference in the content located in the header files and the import library?
Again, header file has the signature only. No actual implementation.
Import library is used to resolve external symbols. i.e. to let the linker know where the code resides.

6) What content is located in the import library?
the dll in which the code resides, the functions exported by the dll.

8) What content is required to be inside DllMain()?
Usually nothign unless you want to initialize something.. like caching the hInstance to your own variable which might be useful later.

9) Would the project would require a seperate source file for the DllMain function, as well as the other H/CPP file pairs that consist of your library?

Not necessarily, .cpp .h files are all for convenience.. The compiler doesn't care if all code is in one massive 300000000 line file or split into 3000 .cpp and .h files.. All it needs is it has to find definitions and linker has to find the object code.

But having seperate files helsp keep things organized.

To view sections, you can use a tool called dumpbin.exe. It is shipped with visual studio and is a command line too.

[MrDoomMaster]

Very nice information Kirants, thank you.

The whole idea of exporting and importing sounds pretty slow.

So, does the application search for the exported methods and load them into memory, or is a search taking place during runtime for EVERY function call?

[kirants]

It is not slow as you suspect. It is done one time while loading.

And as, I said earlier, it is pretty interesting..

Let us say you call CreateWindow 3 times in your exe. Now the compiler puts in


call XXXPlaceHolderForCreateWindow

call XXXPlaceHolderForCreateWindow

call XXXPlaceHolderForCreateWindow...

kind of code whereever it is called.

What does the linker do.
Linker finds that this is in user32.dll ( as per the .lib you supplied for linking ). But it cannot put the actual call location because it is in another dll and where the dll is loaded at runtime is not known during link time. The runtime location at which a dll is loaded can vary depending on what other dlls are loaded ect. So, what does the linker do.. it goes and patches in all these calls to a single location.. say,


call IATentryForCreateWindow

call IATentryForCreateWindow

call IATentryForCreateWindow


This IATEntryForCreateWindow is a relative location within the exe and this location is in the import section that I talked about earlier.

Note the optimization here. All calls are set to a single location.

Now, when the dll user32.dll is loaded by loader when the exe is executed, one of the things that the loader does it to load use32.dll into the process space. It then locates where CreateWindow address is ( which it gets from the Export section of user32.dll ) and puts this value in IATentryForCreateWindow.

The IATEntryForCreateWindow is called a thunk. The thunk does somethign like this:

jmp to {this location}

Now, all the loader does is, get the CreateWindow function's address in user32.dll and patch it for {this location}.

So, magically, all calls in the exe for CreateWindow , will go to user32.dll with just one patch.

though it is a 2 step process now,

i.e. call to IAT
and then
jmp to actual location,

the loader has to do less work. It doesn't have to go through all calls to CreateWIndow in the exe and patch each one individually.

If you are interested here are a few interesting reads:
An In-Depth Look into the Win32 Portable Executable File Format (http://msdn.microsoft.com/msdnmag/issues/02/02/PE/default.aspx)
An In-Depth Look into the Win32 Portable Executable File Format, Part 2 (http://msdn.microsoft.com/msdnmag/issues/02/03/PE2/default.aspx)

[MrDoomMaster]

Lets say I have two projects. One project is responsible for creating a static library of my set of classes, the second project is in charge of creating a DLL + import library for my set of classes.

Would the constant __declspec( dllimport ) located in the definitions all over my classes cause problems when trying to compile a static library?

For whatever reason I may want to go back and compile a static library, I want it to be easy to do. I don't want to have to make a million code changes to do it. Any ideas?

Secondly, my library contains 18 classes. Each class has its own H/CPP file. This means there's a total of 36 files. What I plan to do is create a 37th file that will contain DllMain() and will #include all 18 header files for all 18 classes above it. This should enable my entire library to be compiled into the DLL. I know what export and import are now, I just don't know where to put them. Obviously at all my class declarations, I would put dllexport, but where do I put dllimport? Do I need to make an extra header file for the imports?

My main confusion here I guess is how to structure the files in my project.

Thanks for the help.

[MrViggy]

Typically, I've done this:
#ifdef MY_EXPORT_MACRO
#define MY_MODIFIER __declspec(dllexport)
#else
#define MY_MODIFIER __declspec(dllimport)
#endif

MY_MODIFIER MyExportedClass
{
// ...
};
Now, in your project to build the DLL, you define the macro MY_EXPORT_MACRO. Then you ship the same header files to your clients, and they will not define MY_EXPORT_MACRO.

You can probably come up with something similar for creating a static library.

Viggy

[kirants]

This is what is done typically:

Say, CClass1 is a class to export from the dll and say it is declared in a file called class1.h

#ifdef EXPORT_FUNCS
DECLSPEC_SPECIFIER __declspec(dllexport)
#else
DECLSPEC_SPECIFIER __declspec(dllimport)
#endif

DECLSPEC_SPECIFIER class CClass1
{
......
};


You would use this header file for compiling your dll which exports the class , as well as to compile the client who uses this dll for the class CClass1.

Now, if you see at the conditional compilation block, you see that , if EXPORT_FUNCS is defined, then your class becomes:

__declspec(dllexport) class CClass1
{
..
};

If not, it becomes

__declspec(dllimport) class CClass1
{
..
};


So, all you need to do is , when you compile the dll, just add EXPORT_FUNCS ( or whatever #define is convenient for you ) to your dll project's preprocessor defines.

The client DLL simply doesn't define this and it automatically uses dllimport.

COming to your using the same files for static/dynamic libs, you can again use preproccessor and say

//use declspecs only if building dynamic link lib
#ifndef BUILD_STATIC
#ifdef EXPORT_FUNCS
DECLSPEC_SPECIFIER __declspec(dllexport)
#else
DECLSPEC_SPECIFIER __declspec(dllimport)
#endif
#endif


So, if you define BUILD_STATIC in your project settings, the DECLSPEC_SPECIFIER is not defines and your class becomes


class CClass1
{
....
}

[MrDoomMaster]

Isn't the code supposed to look like this:

#ifndef BUILD_STATIC
#ifdef EXPORT_FUNCS
#define DECLSPEC_SPECIFIER __declspec(dllexport)
#else
#define DECLSPEC_SPECIFIER __declspec(dllimport)
#endif
#endif

Also, if the DECLSPEC_SPECIFIER isn't even defined when using BUILD_STATIC, wouldn't this cause compiler errors when it encounters this line:

class DECLSPEC_SPECIFIER CClass1;

Since the definition of the constant DECLSPEC_SPECIFIER is skipped, this means this is an undefined constant... I was thinking of something like this:

#ifndef BUILD_STATIC
#ifdef EXPORT_FUNCS
#define DECLSPEC_SPECIFIER __declspec(dllexport)
#else
#define DECLSPEC_SPECIFIER __declspec(dllimport)
#endif
#else
#define DECLSPEC_SPECIFIER
#endif

Above, DECLSPEC_SPECIFIER is defined as *nothing* if BUILD_STATIC is specified. This makes the compiler happy because DECLSPEC_SPECIFIER has been defined, but it causes this:

class DECLSPEC_SPECIFIER CClass1;

to actually look like this:

class CClass1;


What do you say to this idea? It seems legitimate to me.


On a second note:

Where would I place the above conditional defines? Would they appear in every class header file, or in a seperate header file that is included in every single header file in the library? For example, say you have 3 classes, and the headers are named as follows: Class1.h, Class2.h, Class3.h. Now you create a fourth header file which contains the above define statements named Shared.h. Would each of the three class headers include the Shared.h file, so they all share the defined constants?

I'm guessing this is correct, it seems nice and it avoids duplicate definitions.

Let me know. Thanks.

[kirants]

Isn't the code supposed to look like this:
Yep. My bad.


#ifndef BUILD_STATIC
#ifdef EXPORT_FUNCS
#define DECLSPEC_SPECIFIER __declspec(dllexport)
#else
#define DECLSPEC_SPECIFIER __declspec(dllimport)
#endif
#else
#define DECLSPEC_SPECIFIER
#endif
What do you say to this idea? It seems legitimate to me.

Yep. Way to go.

Where would I place the above conditional defines? Would they appear in every class header file, or in a seperate header file that is included in every single header file in the library? For example, say you have 3 classes, and the headers are named as follows: Class1.h, Class2.h, Class3.h. Now you create a fourth header file which contains the above define statements named Shared.h. Would each of the three class headers include the Shared.h file, so they all share the defined constants?

I'm guessing this is correct, it seems nice and it avoids duplicate definitions.

Let me know. Thanks.

Whatever is convenient to you. All that matters is for the compiler to know if there is a __declspec and what it is. You could very well have those #defines put it in a common header file and include it in all the header files..

[MrDoomMaster]

I tested my scenario right before you posted and it worked great.

I really thank you guys for your help, and I wish to close this topic with one final question:

I need to know what to export and what not to export. For example, at each class definition you do:


class __declspec(dllexport) CClass1
{
....
};

But most people put only prototypes inside of the classes and put the actual function definitions in the CPP file. Do you have to export every member function definition in the CPP file? For example:

void __declspec(dllexport) CClass1::MyFunction(void)
{
.....
}

int __declspec(dllexport) CClass1::MyStaticVariable = 0;

I was wondering if there is some sort of reference manual on where to put the dllexport keyword and where to not put it. This involves also static member variables/functions, which are also declared in every source file it is used. Also this goes for namespaces and other things. I will need to know all of this in order to properly convert my library.

Thanks again for all of your help. I will, in the meantime, make a fruitful attempt to find some of this information myself.

[kirants]

Putting it in the header file is good enough..

[MrDoomMaster]

I know I said I was done, but I actually have another question.

As I mentioned before, I have 18 source files which contain my class definitions. I have a header file named Shared.h which I include in all 18 of my class definition source files so that all classes can share the same structures and enumerations I place in this header file.

Shared.h contains the following:

static HRESULT ErrorReturn = 0;

This worked perfectly find when I was creating static libraries, but now that I'm creating a DLL this creates compiler errors.

I tried this:

DXLIB_DLL static HRESULT ErrorReturn = 0;

It gives me this compiler error:

error C2201: 'ErrorReturn' : must have external linkage in order to be exported/imported

So, am I supposed to convert it into "extern DXLIB_DLL HRESULT ErrorReturn;" and then define it in each of my 18 source files?

Also, I have the following structure in Shared.h:


struct DXLIB_DLL AnimeInfo
{
std::vector<Rect> vFrames;
};

Compiler says the following:
warning C4251: 'DXLIB::AnimeInfo::vFrames' : class 'std::vector<_Ty>' needs to have dll-interface to be used by clients of struct 'DXLIB::AnimeInfo'
with
[
_Ty=DXLIB::Rect
]

Thanks for any help

[MrViggy]

I don't know about the static variable, but the second issue, the compiler is saying that "std::vector<Rect>" needs to have export linkage as well, if you're clients are going to have direct access to it.

I don't know what version of Dev Studio you're using, but this link might be helpful:

http://support.microsoft.com/default.aspx?scid=kb;EN-US;168958

Viggy

[kirants]

For the first:

in shared.h

extern static HRESULT ErrorReturn;

So compiler is happy.

If you want to export this symbol from the dll, just like you export functions, do this in shared.h:

DXLIB_DLL static HRESULT ErrorReturn;


in shared.cpp

static HRESULT ErrorReturn = 0;
so linker is happy.

[MrDoomMaster]

For the first:

in shared.h

extern static HRESULT ErrorReturn;

So compiler is happy.

If you want to export this symbol from the dll, just like you export functions, do this in shared.h:

DXLIB_DLL static HRESULT ErrorReturn;


in shared.cpp

static HRESULT ErrorReturn = 0;
so linker is happy.

So I create the declaration in Shared.h and put the definition in Shared.cpp. What about the class named, for example, CDisplay.

In Display.cpp, many of the functions set ErrorReturn to the reason of failure if a function fails. Do I need to put the defintion for ErrorReturn in all 18 source files?

And when I get ready to import this variable, do I have to also put a definition in main.cpp when I'm actually checking this variable for errors?

The extern and static are confusing me... extern can only contain a declaration, and by placing extern in a variable declaration, the linker expects the definition to be in every source file that includes it, correct? I guess maybe I don't understand the full definition of static and extern, and what exactly combining them would do.

[kirants]

extern can only contain a declaration, and by placing extern in a variable declaration, the linker expects the definition to be in every source file that includes it, correct?
Nope. That would be multiple definition. Suffices to put it in one of the cpp files.
You can try it out yourself.

[MrDoomMaster]

It seems impossible enough to create a DLL out of my library since I use CString, std::list, several DirectX functions (which are themselves imported), and several math.h functions.

How do people manage to create DLL files when they use so many other external functions that are not originally in DLL modules?

I can't continue my project until I learn how to overcome the above conflicts.

[kirants]

A dll typicall almost always imports from other libraries..
What specific problem do you have ?

Just as you would have imported this stuff in your static library, you would import in your dll too .

[MrDoomMaster]

Well the thing is, I don't know which standard libraries are in DLL files or not, and if there's any that are NOT in DLL files, how am I supposed to import them? The class definitions have already been made by Microsoft, I can't go through the official files and add __declspec(dllimport).

I just don't know how to import them I guess.

[kirants]

You don't have to manipulate the files. You just need to add the .libs into your linker input list.

[MrDoomMaster]

You don't have to manipulate the files. You just need to add the .libs into your linker input list.

atlstr.h, list.h, and math.h all have LIB files? I did not know this... What are the names of these lib files?

[kirants]

stl would be in msvcrt dlls I guess. So when you specify runtime dll option, this should get automatically included.

[MrDoomMaster]

stl would be in msvcrt dlls I guess. So when you specify runtime dll option, this should get automatically included.

And list.h and math.h? Are they included in msvcrt.dll as well? I get compiler errors about std::list especially.

[kirants]

Hm.. compiler error. In that case it is not a linker issue at all.

You are missing some headers.
What are you including ? Have you included <list> etc ??

[MrDoomMaster]

My apologies, they're not errors but warnings. See below:

warning C4251: 'DXLIB::AnimeInfo::vFrames' : class 'std::vector<_Ty>' needs to have dll-interface to be used by clients of struct 'DXLIB::AnimeInfo'
with
[
_Ty=DXLIB::Rect
]

[MrViggy]

C4251 (http://msdn.microsoft.com/library/default.asp?url=/library/en-us/vccore/html/C4251.asp) basically says that if you have a member variable of an exported class, if your clients want to use that member variable directly, then it's class also needs to be exported, and it isn't.

If your member data is private, you can ignore these warnings (the are annoying, though).

Viggy

[MrDoomMaster]

This is a simple structure, though... it has no children. The data member isn't even private, it's public. See for yourself:

In Shared.h:
struct DXLIB_DLL AnimeInfo
{
private:

public:
std::vector<Rect> vFrames; ///< Animation Frames
int AnimeSpeed; ///< Animation Speed
int AnimeType; ///< Animation Type (Integer Identifier)
bool Animated; ///< Enable/Disable animation

AnimeInfo(void);
AnimeInfo(int AnimeType, bool Animated, int AnimeSpeed);

void Set(int AnimeType, bool Animated, int AnimeSpeed);
};

In Shared.cpp:

//====================================================================================
/**
** \brief Default Initializer
**
** All values are initialized to zero.
**/
DXLIB::AnimeInfo::AnimeInfo(void)
{
Set(0, false, 0);
}

//====================================================================================
/**
** \param AnimeType Identifier which represents the specific animation type
** \param Animated True if animated, false if not
** \param AnimeSpeed Speed of the animation, in milliseconds. This parameter is
** ignored if Animated is false.
**/
DXLIB::AnimeInfo::AnimeInfo(int AnimeType, bool Animated, int AnimeSpeed)
{
Set(AnimeType, Animated, AnimeSpeed);
}

//====================================================================================
/**
** \brief Sets the values for an AnimeInfo object
**
** \param AnimeType Identifier which represents the specific animation type
** \param Animated True if animated, false if not
** \param AnimeSpeed Speed of the animation, in milliseconds. This parameter is
** ignored if Animated is false.
**
** The Animation (Anime) type is an identifier which is used later to set the
** current animation type. If the animation is not animated (contains only 1
** frame) then animation speed is ignored.
**/
void DXLIB::AnimeInfo::Set(int AnimeType, bool Animated, int AnimeSpeed)
{
AnimeInfo::AnimeType = AnimeType;
AnimeInfo::Animated = Animated;
AnimeInfo::AnimeSpeed = AnimeSpeed;
}

[MrViggy]

Yes, and the class "std::vector<Rect>" has not been exported from your DLL. So, if you're clients are going to access the member variable 'vFrames' directly, you could run into problems. That is what the warning is telling you.

Personally, I'd make all the member data private (you'll still get the warning, which is why it's annoying), and have accessor functions that are exported from the class. If you can't do that, then see this link about exporting STL classes from DLL's:

http://support.microsoft.com/default.aspx?scid=kb;EN-US;168958

Viggy

[MrDoomMaster]

Okay so that will help me export the vector class, but what about the directx functions and math.h functions I am using in my DLL? Will they be compiled INTO the DLL, or do I have to export them as well? I get the very same warnings for all of the directx and math.h functions.

[GordonFreeman]

Currently, my library is a static library, but I am going to convert it into a DLL file.

I have indeed gone to codeproject.com and read some tutorials on DLL files, but I still am very confused on a few things:

1) What does Export mean in terms of DLL Libraries?
2) What does Import mean in terms of DLL Libraries?
3) Why are both header files AND import libraries required to use DLL files?
4) When would you Import and when would you Export? (This may be part of questions #1 and #2)
5) What is the difference in the content located in the header files and the import library?
6) What content is located in the import library?
7) Using Visual Studio .NET 2003, must you create the import library and DLL file with two seperate projects? If no, how are both created at once? To me this seems like a project creating two EXE files. Very unusual.
8) What content is required to be inside DllMain()?
9) Would the project would require a seperate source file for the DllMain function, as well as the other H/CPP file pairs that consist of your library?

It may seem as if I didn't read any tutorials at all, but I think I've actually read a very poor tutorial. Maybe someone can post a good tutorial. In the meantime I'll be reading some more.

I thank anyone for answers they may provide.

if you want to have an in-deep look at this topic and have a robust understanding,try reading this PE format description

[Freon]

I'm having a similar problem, and the Microsoft Knowledge Base article didn't help:

STL classes are not designed to be used raw; most applications need to derive classes from them. The client of the derived class must be able to call methods in the STL class. How can a DLL derive a class from an STL class and export it to clients of the DLL?

[thre3dee]

hey, this is a great thread. I've been thinking about consolidating all my static libraries into one big framework DLL and needed many of the same questions answered. thanks!

[MrDoomMaster]

Well I finally got my library into a DLL, but now I have a question on what some of these files are. Visual Studio created these 4 files: DLL, LIB, INK, EXP

I already know what the DLL and LIB files are, but what about the other two? Do I need them? Can I stop them from being created?

Thanks.

[NoHero]

EXP is the nearly the same as LIB although in a different format. Never seen an INK file though, maybe you mean the incremental linker files.

[kirants]

I think it is lnk. They are used for incremental linking. There should be a project option to turn it off. However, it is used for , as the name says, incremental linking.. i.e.. when the linker builds, it keeps spaces empty in the binary so that when some code changes, it doesn't have to patch all the places and so linking is faster. This is to be turned off for release builds because of the problem you see there. It is good to have during develeopment time since it increases your productivity by having the linker do it;s job faster

[MrViggy]

I'm having a similar problem, and the Microsoft Knowledge Base article didn't help:

STL classes are not designed to be used raw; most applications need to derive classes from them. The client of the derived class must be able to call methods in the STL class. How can a DLL derive a class from an STL class and export it to clients of the DLL?
I don't quite understand. STL classes are not meant to be derrived from. Sometimes bad things can happen if you do that.

Do you want to export a class containing an STL object, and have that object accessible by your clients?

Viggy