Re: Incredibly Strange Bug

From: James Turner (turnerjh@XTN.NET)
Date: 09/13/98


"Tony Robbins [Kupek]" <tonyr@NWPACLINK.COM> writes:

>   CREATE(newt, struct txt_block, 1); <-----

[SNIP]

> If you get any ideas, let me know.  I continue to dig, in the meantime...

Generally (at least on Linux systems), if you get a crash in a
malloc/free/realloc, and the params are sane (legit sizes, valid
pointers), then the problem is usually overwriting/underwriting a
malloced region of memory.  Why would that mess up a later malloc or
free on a different block of memory?  Answer below!

GNU malloc (and glibc2) allocate roughly in the following way.  They
mmap /dev/zero anonymously to get raw memory (not positive on this
detail, but it isn't important to tge problem at hand), then they
parcel memory out of these mmaps.  When you allocate, it adds a few
bytes (12 to 16 I believe) for internal handling.  Basically it makes
a linked list roughly shaped like this:

-8 bytes         pointer to previous allocated block
-4 bytes         size of this block
 0 bytes         <the memory you allocated>
+N bytes         pointer to next allocated block

So if you underflow, you can wipe out the pointer to the prev block,
or the size of the cuyrrent block (and therefor make it impossible to
find the remaining blocks in memory).  If you overflow, you make it
impossible to find the rest of the chain.  In all of these situations,
though, the problem is pointers used internally by malloc become
invalid.  The next memory alocation call and wham, probable crash.

(Disclaimer: the above is very simplified.  Don't take the offsets
literally -- they're just for demonstration of the idea involved.  The
actual encoding is much more complicated and basically just not
important to the discussion.  The key idea is that both before an
allocated block and after, there are values that can't safely be
overwritten)

Now, the trick, is finding where the memory is getting munged.  This
can sometimes be very easy, and other times quite difficult.

What I've done with my code is added functions (mud_free, mud_malloc,
mud_calloc, mud_realloc) that use their own magic byte scheme.  This
helps check for allocation problems etc.  Basically I allocate 8 bytes
extra, store the size at -4bytes and again at N bytes, then
periodically check it to be valid.  I almost backended into malloc's
data, but that is tremendously system specific (not to mention
painful, as malloc is somewhat complicated).  Down side: 8 bytes per
alloc lost (and this can be a lot, as my mud averaged 55,000 unfree'd
mallocs at any one time, in about 5,000,000 bytes).

There's a LOT of room for memory management improvements with muds.
I've been considering this, as well as an extremely space (and
somewhat time) efficient string system, but in C++.  Since mud strings
either are very short lived (one function), or persist for a fair
amount of time, two levels of string handling could work well
together.

Oh well, enough rambling :)

--
James Turner                turnerjh@pattern.net         UIN: 1102038
                            http://www.vuse.vanderbilt.edu/~turnerjh/


     +------------------------------------------------------------+
     | Ensure that you have read the CircleMUD Mailing List FAQ:  |
     | http://democracy.queensu.ca/~fletcher/Circle/list-faq.html |
     +------------------------------------------------------------+



This archive was generated by hypermail 2b30 : 12/15/00 PST