Re: Memory-Mapped I/O

From: Daniel A. Koepke (dkoepke@circlemud.org)
Date: 03/01/01


On Tue, 27 Feb 2001, Peter Ajamian wrote:

> What do you think of the possibility of mapping all memory in the MUD
> out to a file in this manner?  It holds some truly interesting
> possibilities, the following come to mind:

I mentioned something like this months ago when talking about persistence.
It's certainly not an idea original to me, but it works for what I used it
for.  There are a definite set of problems, however.

The output is heavily dependent on your platform.  Not just between
different machine architectures or operating system or C library, but
between different versions of your C library and operating system.  This
means that you should not rely on any important data to be stored in a
binary memory map (if you're mmap'ing to read the file like one long
string, that's different) if you think you're ever going to have to switch
platforms.  You should not rely upon the correctness of the mapped data
after hardware/software upgrades.

As with longer term persistence, the idea of using this to have data
persist over reboots introduces several problems that I've detailed
before.  You cannot and should not trust the state of an unsync'd, open
memory map at the time of a crash.  You should not sync or close a memory
map after a crash by catching SIGSEGV (you are not given any reasonable
guarantees about the purity of the data and your process at this time).

One way to approach this is to write a separate program that will do the
memory mapping and use some form of IPC to communicate with the server.
The model I was thinking of:

       Data Server --> Secondary Networking Servers --> [ clients... ]
            |
            |
     Central Networking Server --> [ clients... ]

Where the Data Server provides the synchronized state that all networking
servers rely upon.  The data server and central (local) networking server
may reside on separate computers (in my design, they would, since the data
server would be feeding not just the central net server, but many remote
servers).

This design is vastly more scalable (but high-end) than you'll need your
mud to be.  So working with the Data Server and Networking Server as two
separate entities should suffice.

After all is said in done, you're gaining squat over using a more
complete, existing database system such as Postres or MySQL or whatever
passes for an SQL database on your platform of choice.


-dak

--
   +---------------------------------------------------------------+
   | FAQ: http://qsilver.queensu.ca/~fletchra/Circle/list-faq.html |
   | Archives: http://post.queensu.ca/listserv/wwwarch/circle.html |
   +---------------------------------------------------------------+



This archive was generated by hypermail 2b30 : 12/04/01 PST