Interprocess communication is an important missing part in the C++ standard library. This paper proposessome classes to fill two basic areas in interprocess communications: memory mapped files and shared memory.
Standard file mapping and shared memory utilities can offer a wide range of possibilities for manyapplications, to achieve data persistence, data cache and complex data serialization betweenprocesses.
File mapping is the association of a file's contents with a portion of the address space of a process.The system creates a file mapping to associate the file and the address space of the process. Amapped region is the portion of address space that the process uses to access the file's contents.A single file mapping can have several mapped regions, and the user can associate parts of the filewith the address space of the process without mapping the entire file in the address space.Processes read from and write to the file using pointers, just like with dynamic memory.File mapping has the following advantages:
If several processes use the same file mapping to create mapped regions of a file, each process'views contain identical copies of the file on disk. POSIX and Windows mapped file management isvery similar, and highly portable.
POSIX defines a shared memory object as "An object that represents memory that can be mappedconcurrently into the address space of more than one process."
Shared memory is similar to file mapping, and the user can map several regions of a shared memory object,just like with memory mapped files. In some operating systems, like Windows, shared memoryis an special case of file mapping, where the file mapping object accesses memory backed by the systempaging file. However, in Windows, the lifetime of this memory ends when the last process connected to theshared memory object closes connection or the application exits, so there is no data persistence.If an application creates shared memory, fills it withdata and exits, the data is lost. This lifetime is known as process lifetime
In POSIX operating systems the shared memory lifetime is different since for semaphores, shared memory,and message queues it's mandatory that the object and its state (including data, if any) is preserved after theobject is no longer referenced by any process. Persistence of an object does not imply that the state of the objectis preserved after a system crash or reboot, but this can be achieved since shared memory objects can actually beimplemented as mapped files of a permanent file system. The shared memory destruction happenswith an explicit call to unlink(), which is similar to the file destruction mechanism. POSIX shared memory isrequired to have kernel lifetime (the object is explicitly destroyed or it's destroyed when theoperating system reboots) or filesystem persistence (the shared memory object has the same lifetimeas a file).
This lifetime difference is important to achieve portability. Many portable runtimes have tried to achieveperfect portability between Windows and POSIX shared memory but the author of this paper has not seen anysatisfactory effort. Adding a reference count to POSIX shared memory is effective only as long as a processdoes not crash, something that it's very usual. Emulating POSIX behaviour in Windows using native shared memoryis not possible since we could try to dump shared memory to a file to obtain persistence, but a process crashwould avoid persistence. The only viable alternative is to use memory mapped files in Windowssimulating shared memory, but avoiding file-memory synchronization as much as possible.
Many other named synchronization primitives (like named mutexes or semaphores) suffer the same lifetimeportability problem. Automatic shared memory cleanup is useful in many contexts, like shared libraries or DLL-scommunicating with other DLL-s or processes. Even when there is a crash, resources are automatically cleaned upby the operating systems. POSIX persistence is also useful when a launcher program can create and fill sharedmemory that another process can read or modify. Persistence also allows data recovery if a server processcrashes. All the data is still in the shared memory, and the server can recover its state.
This paper proposes POSIX lifetime (kernel or filesystem lifetime) as a more portable solution, but has no strongopinion about this. The C++ committee should take into account the use cases of both approaches to decide which behaviouris better or if both options should be available, forcing the modification of both POSIX and Windows systems.
In both POSIX and Windows systems shared memory, memory mapping and other input-output device mapping mechanismsare similar. The memory mapping operation returns a handle that can be used to create a mappedregion using mmap (POSIX) or MapViewOfFile (Windows) functions. Thisshows that a mapped region is independent from the memory mapping source, and this paper proposesa generic class representing a mapped region that can represent a shared memory, a mapped file ora mapped device region.
The memory_mappable class represents any resource that can be read and written usingthe address space of the process. It's an abstract class from which we can derive several mappableclasses, like file mappings, shared memory objects, device mappings... Like the iostreamframework, the derived classes must initialize memory_mappable in the constructor, callinginit to store the mapping handle and the mapping mode. This information will be storedin the memory_mappable class and it will be used to created mapped_regionobjects:
namespace std {class memory_mappable{private:// Non-copyablememory_mappable(const memory_mappable &);// Non-assignablememory_mappable & operator=(const memory_mappable &);public:typedef /*implementation-defined*/ accessmode_t;typedef /*implementation-defined*/ mapping_handle_t;static const mapping_handle_t invalid_handle;static const accessmode_t read_only;static const accessmode_t read_write;static const accessmode_t invalid_mode;public:memory_mappable();protected:void init(mapping_handle_t handle, accessmode_t mode);public:memory_mappable(memory_mappable &&other);mapping_handle_t get_handle() const;accessmode_t get_mode() const;virtual ~memory_mappable() = 0;};} //namespace std {memory_mappable();
Effects: Default constructs the object.
Postconditions: this->get_handle() == invalid_handle & & this->get_mode() == invalid_mode.
Throws: An exception derived from std::exception on error.
void init(mapping_handle_t handle, accessmode_t mode);
Effects: Stores internally the mapping handle and the mode of the derived class.This function is intended to be called by the derived class once the derived classinitialization has been completed.
Posconditions: this->get_handle() == handle && this->get_mode() == mode.
Throws: An exception derived from std::exception on error.
memory_mappable(memory_mappable &&other);
Effects: Moves resources from other to *this.
Posconditions: other.get_handle() == invalid_handle && other.get_mode() == invalid_mode.
Throws: Nothing.
mapping_handle_t get_handle() const;
Effects: Returns the handle registered by the init() function.
Returns: Returns the handle registered by the init() function. If init() has notbeen called, returns invalid_handle.
Throws: Nothing.
accessmode_t get_mode() const;
Effects: Returns the mode registered by the init() function.
Returns: Returns the handle registered by the init() function. If init() has notbeen called, returns invalid_mode.
Throws: Nothing.
virtual ~memory_mappable() = 0;
Effects: Destroys the object. The class does not free the handle registered in init().This task should be performed by the derived class.
Throws: Nothing.
The mapped_region class represents a portion or region created from a memory_mappableobject. Once a mapped_region object is constructed, a part of thefile, shared memory or device is mapped in the address space of the calling process:
namespace std {class mapped_region{// Non-copyablemapped_region(const mapped_region &);// Non-assignablemapped_region &operator=(const mapped_region &);public:typedef /*implementation-defined*/ offset_t;typedef /*implementation-defined*/ accessmode_t;static const accessmode_t invalid_mode;static const accessmode_t read_only;static const accessmode_t read_write;static const accessmode_t copy_on_write;mapped_region();mapped_region( const memory_mappable & mappable, accessmode_t mode, offset_t mappable_offset, std::size_t size = 0, const void * address = 0);mapped_region(mapped_region &&other);std::size_t get_size() const;void* get_address() const;offset_t get_offset() const;accessmode_t get_mode() const;void flush(std::size_t region_offset = 0, std::size_t numbytes = 0);void swap(mapped_region &other);~mapped_region();};} //namespace std {mapped_region();
Effects: Default constructs a mapped region.
Postconditions: this->get_address() == 0 && this->get_size() == 0&& this->get_offset() == 0 && other.get_mode() == invalid_mode.
Throws: Nothing.
mapped_region(mapped_region &&other);
Effects: Constructs a mapped region taking ownership of the resourcesof another mapped_region.
Postconditions: The object is an default constructed state.
Throws: Nothing.
mapped_region( const memory_mappable & mappable, accessmode_t mode, offset_t mappable_offset = 0, std::size_t size = 0, const void * address = 0);
Effects: Creates a mapped region of the mappable memory mappable object.This mappable object can represent any mappable object that can be mapped usinga generic mapping_handle_t (a file, a shared memory object or a input-output device). The mappableobject can be inmediatelly destroyed after the constructor if no more mapped regions are neededfrom the mappable object.
A mappable object with read_only access should only be used to createread_only mapped_region objects. A mappable object with read_writeaccess can create read_only, read_write or copy_on_writemapped_region objects.
When an object is created with read_only mode, writing to the memory range expressed by thisobject can terminate the process and never will change the mappable object.
When an object is created with read_write mode:
A copy_on_write mapped_region is an special read_write mapped_region:
The constructor will map a region of the mappable object starting inthe offset offset of the mappable object (a file, or shared memory object),and the region's size will be size.
The user can request a mapping address in the process, using the address parameter. Ifaddress is 0, the implementation will choose the mapping address.If size is 0, the mapped_region's size will cover from offset untilthe end of the mappable object.
Postconditions: this->get_address() != 0 && this->get_size() != 0 && this->get_offset() == offset.
Throws: An exception derived from std::exception on any error, for example when themappable object can't be mapped in the address passed in the constructor or whenthe mappable can't be mapped with the required access mode.
std::size_t get_size() const;
Effects: Returns the size of the mapped region. Never throws.
Returns: If default constructed or moved returns 0. Otherwise the size passed in thenon-default constructor.
Throws: Nothing.
void *get_address() const;
Effects: Returns the address where the mappable has been mapped. If the non-default constructor'saddress argument was 0, this value will be chosen by the implementation.
Returns: If the object is default constructed or has been moved returns 0. Otherwise the addresswhere the mappable object was mapped.
Throws: Nothing.
offset_t get_offset() const;
Effects:Returns the offset from the beginning of the mappable where the mapped region starts.
Returns: If default constructed or moved returns 0. Otherwise the offset passed in thenon-default constructor.
Throws: Nothing.
accessmode_t get_mode() const;
Effects: Returns the mode of the mapped region.
Returns: If default constructed or moved returns invalid_mode. Otherwise the mode passedin the non-default constructor.
Throws: Nothing.
void flush(std::size_t region_offset = 0, std::size_t numbytes = 0);
Effects: If we call mapping_address to the result of the expression
static_cast<char*>(this->get_address())the functionflushes the range [mapping_address + region_offset, mapping_address + region_offset + numbytes)to the mappable object passed in the non-default constructor. If the object is default constructed,or moved, the function does nothing.
Throws: An error derived from std::exception on error.
void swap(mapped_region &other);
Effects: Swaps the resources contents of this object with the resources of other.
Throws: Nothing.
~mapped_region();
Effects: Destroys the object. Reading from and writing to the memory address range that theobject was holding is now undefined and might crash the process.
Throws: Nothing.
The mapped_region class is constructed taking a reference to a memory_mappable class.This class contains the mapping handle needed to map a region of the mapping in the process address space.The memory_mappable object used in the constructor can be destroyed before destroying all thecreated mapped_region objects from it since both POSIX and Windows support this possibility. Thissimplifies mapping management, and there is no need to store the memory_mappable object after theuser has constructed all the needed mapped_region objects.
mapped_region is a single class representing a region of a file, shared memory or a devicein the address space of the process, so the user can develop device independent code without the needof virtual functions.
memory_mappable and mapped_region are non-copyable and non-assignablebut both are movable (N1952), so we can store them safely in STL containers, to create groups of mappedregions of files or shared memory objects.
This paper proposes a standard file mapping class derived from memory_mappable.The standard interface would allow just linking a file with the process, so that wecan create several mapped_region objects from the file:
namespace std {class file_mapping : public memory_mappable{public:file_mapping(const char *filename, memory_mappable::accessmode_t mode);~file_mapping();};} //namespace std {file_mapping(const char *filename, memory_mappable::accessmode_t mode);
Effects: Allocates resources to so that the user can map parts of a fileusing mapped_regions. The constructor initializes the base memory_mappableclass calling memory_mappable::init(). with valid handle and mode values.
Postcondition: this->get_handle() != memory_mappable::invalid_handle &&this->get_mode() != memory_mappable::invalid_mode.
Throws: Throws a exception derived from std::exception on error.
~file_mapping();
Effects: Closes the file mapping and frees resources. All mapped regionscreated from this object will be valid after the destruction.
Throws: Nothing.
Using file_mapping and mapped_region, a file can be easily mapped in memory:
//Create a file_mapping objectstd::file_mapping mapping("/home/user/file", std::memory_mappable::read_write);//Create a mapped_region covering the whole filestd::mapped_region region (mapping, std::mapped_region::read_write);//Obtain the size and the address of the mapped regionvoid *address = region.get_address();std::size_t size = region.get_size();//Set the whole filestd::memset(address, 0xFF, size);//Make sure all the data is flushed to diskregion.flush();C++ has file management functions, so that it can create, delete, read and write files. That's whythe file_mapping class has no functions to create, open or modify files. However, thereare no such functions for shared memory objects. This paper proposes two options:
This paper does not make any recommendation, but it's clear that adding new stream classesis more complicated than adding these functions to the mapping class. However, there is no doubtthat a shared memory stream class can be very useful for several applications, like loggingor tracing. This paper proposes an interface for a shared memory object with shared memorycreation and connection interface that can be complementary to the stream classes:
namespace std {namespace detail{typedef /*implementation-defined*/ create_t;typedef /*implementation-defined*/ open_t;typedef /*implementation-defined*/ open_or_create_t;} //namespace detail{extern std::detail::create_t create_only;extern std::detail::create_t open_only;extern std::detail::create_t open_or_create;class shared_memory_object : public memory_mappable{public:shared_memory_object(detail::create_t, const char *name, memory_mappable::accessmode_t mode);shared_memory_object(detail::open_or_create_t, const char *name, memory_mappable::accessmode_t mode);shared_memory_object(detail::open_t, const char *name, memory_mappable::accessmode_t mode);void truncate(std::size_t new_size);static bool remove(const char *name);~shared_memory_object();};} //namespace std {shared_memory_object(detail::create_t, const char *name, memory_mappable::accessmode_t mode);
Effects: Creates a shared memory object with name name, with the accessmodemode. If the shared memory object previously exists, the construction fails. If successful,the constructor initializes the base memory_mappable class calling memory_mappable::init()with valid handle and mode values.
Postcondition: this->get_handle() != memory_mappable::invalid_handle &&this->get_mode() != memory_mappable::invalid_mode.
Throws: An exception derived from std::exception on error.
shared_memory_object(detail::open_or_create_t, const char *name, memory_mappable::accessmode_t mode);
Effects: Tries to create a shared memory object with name name, with theaccessmode mode. If it's previously created, tries to open the shared memory object.If successful, the constructor initializes the base memory_mappable class callingmemory_mappable::init() with valid handle and mode values.
Postcondition: this->get_handle() != memory_mappable::invalid_handle &&this->get_mode() != memory_mappable::invalid_mode.
Throws: An exception derived from std::exception on error.
shared_memory_object(detail::open_t, const char *name, memory_mappable::accessmode_t mode);
Effects: Tries to open a shared memory object with name name, with theaccessmode mode. If successful, the constructor initializes the base memory_mappableclass calling memory_mappable::init() with valid handle and mode values.
Postcondition: this->get_handle() != memory_mappable::invalid_handle &&this->get_mode() != memory_mappable::invalid_mode.
Throws: An exception derived from std::exception on error.
void truncate(std::size_t new_size);
Effects: Sets the new size of the shared memory. The object must have beenopened in read_write mode.
Throws: An exception derived from std::exception on error.
static bool remove(const char *name);
Effects: Erases the shared memory object with name name from the system.This might fail if any process has a mapped_region object created from this object, or theshared memory object does not exist.
Throws: Nothing.
~shared_memory_object();
Effects: Destructor, closes the shared memory object. The shared memory object is not destroyedand can be again opened using another shared_memory_object. All mapped regions are still validafter destructor ends.
Throws: Nothing.
Once the shared memory object has been created, any process can map any part of the shared memoryobject in the process' address space, creating mapped_region objects from the shared memory object:
//Create a shared memory objectstd::shared_memory_object shm(std::create_only, "name", std::memory_mappable::read_write);//Set the size of the shared memoryshm.truncate(1000);//Create a mapped_region using the shared memory objectstd::mapped_region region (shm, std::mapped_region::read_write);//Obtain the size and the address of the mapped regionvoid *address = region.get_address();std::size_t size = region.get_size();//Write the whole memorystd::memset(address, 0xFF, size);
The file mapping class can also offer file mapping possibilities using basic_fstream or basic_filebufobjects as constructor arguments. This way, libraries with access to a basic_filebuf object don't need to knowthe path of the file.
class file_mapping{public://...template file_mapping(const basic_filebuf<E, T> &file, memory_mappable::accessmode_t mode);}; In most systems, the implementation just needs to duplicate the operating system's file handlecontained in the basic_filebuf class and also obtain access to the read-write capabilities of the fileto see if the mapping capabilities requested by the user are allowed by basic_filebuf.
The library can also be simplified reusing std::ios_base bitmask values instead of creatingnew read_only and read_write bitmasks. However, std::ios_baseis missing copy_on_write and there is no way to express create_only,open_or_create or open_only creation modes. Expanding std::ios_base::openmodeto hold these new values is also a possibility.
//Open a filestd::fstream file("filename");//Map the whole filestd::mapped_region region(*file.rdbuf(), std::mapped_region::copy_on_write);If basic_filebuf is defined as a mappable object, we could create the shared memory equivalentof this class. This would unify file and shared memory mapping mechanism:
//Open shared memory objectstd::shmstream shm("shmname");//Map the whole shared memory objectstd::mapped_region region(*shm.rdbuf(), std::mapped_region::copy_on_write);As mentioned, we would still need additional std::ios_base flags to express creation possibilitieslike create only, open or create or open only, independently from the read-write mode.
Filesystem (N1975)proposal, includes a portable path class named basic_path. The proposed file_mappingclass could also have a constructor taking a basic_path argument, to avoid user conversions:
class file_mapping{public://...// Allocates resources to so that the user can map parts of a file// using mapped_regions. Throws if there is an errortemplate<class String, class Traits>file_mapping(const basic_path<String, class Traits> & path, memory_mappable::accessmode_t mode);};The portability of the shared memory object name is an old problem even in POSIX. In Posix, the nameof the shared memory object conforms to the construction rules for a pathname. If the name begins withthe slash, then processes opening a shared memory object with the same value of name refer to the same sharedmemory object. If the initial slash is not specified, or there are more slashes in the name, the effectis implementation defined. So the only portable shared memory has the following pattern:/SharedMemoryName
In Windows platforms the name can contain any character except the backlash character and the usermust add some special prefix to share the object in the global or session namespace.
So the only portable name between both platforms is to use the /SharedMemoryName pattern, butthis name conforms to a UNIX path rule for a file in the root directory. For an easier C++ usage, thispaper proposes a portable name that conforms to a C++ variable name rule or a C++ reserved word:
POSIX implementation just needs to add an initial slash to the name. Windows implementation needsno change.
The proposed shared memory and memory mapped files interface is based in the new interface proposedfor the Boost.Interprocess library (the rework of the Shmem library).This library is being used by several projects needing high performanceinterprocess communications.
This paper is mainly presented to test if there is interest for standard shared memory and memory mappedclasses in the C++ committee. The presented interface is a first approach to the solution and by no means adefinitive proposal.
Shared memory and memory mapped files are basic building blocks for interprocess communication for advancedinterprocess communications, like message queues or named fifos. This paper proposes the standardization of sharedmemory and memory mapped files present in nearly all UNIX and Windows operating systems presenting a commonimplementable subset of features.
Memory mapped files can be also useful for a single process environment in order to obtain data persistence.However, shared memory and memory mapped files for interprocess communications need process synchronizationmechanisms. These should follow the same interface as their thread-synchronization counterparts, so severalutilities like scoped locks could be reused with process-shared synchronization utilities.
聯(lián)系客服
