Notes on `RTLD_GLOBAL` in the code ?

[daohu527]

I found below notes in code. It say RTLD_GLOBAL will cause static global variable initialization problem when reopen the library.

I try load and unload the library, and the static variable is int correct. I don't know under what circumstances the problem will occur ? In fact, I found that some static variables in template classes and inline functions will cause this problem. I did not find these situations in the code.

class_loader/src/class_loader_core.cpp

Lines 235 to 244 in 8864be3

	// Insert into graveyard
	// We remove the metaobject from its factory map, but we don't destroy it...instead it
	// saved to a "graveyard" to the side.
	// This is due to our static global variable initialization problem that causes factories
	// to not be registered when a library is closed and then reopened.
	// This is because it's truly not closed due to the use of global symbol binding i.e.
	// calling dlopen with RTLD_GLOBAL instead of RTLD_LOCAL.
	// We require using the former as the which is required to support RTTI
	insertMetaObjectIntoGraveyard(meta_obj);
	} else {

I found the RTLD_NODELETE dlopen doc, dlopen not use the RTLD_NODELETE option, I also make some test that static variable inited correctly.

RTLD_NODELETE (since glibc 2.2)
Do not unload the library during dlclose(). Consequently, the library's static variables are not reinitialized if the library is reloaded with dlopen() at a later time. This flag is not specified in POSIX.1-2001.

[hidmic]

What are you proposing @daohu527 ? IIUC RTLD_GLOBAL is used to ensure RTTI (and dynamic_cast) works, but it also means it won't get unloaded if a symbol within the executable space depends on a symbol in that loaded shared library. That why it goes to a "graveyard".

[daohu527]

I mean that even if dlopen sets RTLD_GLOBAL, it will be dynamically unloaded, including static variables.

We are divided into 2 situations to discuss

1 case

Some static variable with STB_GNU_UNIQUE will still in memory when call dlclose. you can ref to Destructor of a global static variable in a shared library is not called on dlclose.

There is not this case in below register code. (static variables in template classes and inline functions)

class_loader/include/class_loader/register_macro.hpp

Lines 41 to 52 in c012f77

	struct ProxyExec ## UniqueID \
	{ \
	typedef Derived _derived; \
	typedef Base _base; \
	ProxyExec ## UniqueID() \
	{ \
	if (!std::string(Message).empty()) { \
	CONSOLE_BRIDGE_logInform("%s", Message);} \
	class_loader::impl::registerPlugin<_derived, _base>(#Derived, #Base); \
	} \
	}; \
	static ProxyExec ## UniqueID g_register_plugin_ ## UniqueID; \

2 case

Instructions in the man manual dlopen

not this case too.

RTLD_NODELETE (since glibc 2.2)
Do not unload the shared object during dlclose(). Consequently, the object's static and global variables are not reinitialized if the object is reloaded with dlopen() at a later time.

The following two descriptions may appear, unless the ros module can be referenced by ros module. we will explain in next subsection.

The dynamic linker maintains reference counts for object handles, so a dynamically loaded shared object is not deallocated until dlclose() has been called on it as many times as dlopen() has succeeded on it.

Symbols in this object might be required in another object because this object was opened with the RTLD_GLOBAL flag and one of its symbols satisfied a relocation in another object.

example

If A is an ros module and C is references A.

A       // A is a ros module
C <- A  // C references A

We first load A and then load C, the A module will Implicit load only once (reference counts is 2) and register once, then if we unload the library A, the A will still in memory, because the dlopen reference counts is 1, after that we load A again, and A will not register because A is already in memory.

reason

So the core principle is, can ROS modules be referenced by other ROS modules? But it is unreasonable to say that RTLD_GLOBAL causes this problem.