• Re: OSX eats unbounded amounts of memory

    From Frederick Cheung@fglc2@srcf.DUH.ucam.org to comp.sys.mac.system on Wednesday, July 23, 2003 18:19:32
    From Newsgroup: comp.sys.mac.system


    Having to reboot every few hours is totally inconsistent with basing the
    OS on Unix. We'll see if this gets better in future releases.

    I have so far not observed this behavior on 10.2.6, which I routinely
    run for days or weeks at a time.


    absolutely, before i rebooted because of the recent security update i
    hadn't rebooted in over 3 weeks of constant problem free use.

    Fred

    --- Synchronet 3.18b-Win32 NewsLink 1.113
  • From Revanant Morituri@revanant@obvious.adelphia.net to comp.sys.mac.system on Wednesday, July 23, 2003 17:21:04
    From Newsgroup: comp.sys.mac.system

    In article <bfmdl7$4ug$1@news.fsu.edu>,
    bellenot@math.fsu.edu (Steve Bellenot) wrote:

    }In article <fishfry-3BB973.08151023072003@netnews.attbi.com>,
    }fishfry <fishfry@your-mailbox.com> wrote:
    I'm trying to understand some virtual memory strangeness I'm seeing.

    After a reboot, I have lots of free memory available (as shown by
    'top'). After a while, the amount of inactive memory increases until }>there's hardly any free memory left, at which point my system starts }>getting lots of pageouts.

    What is confusing to me is that this condition persists even after I
    kill all my applications. Even with no applications running, top
    reports a large figure for inactive memory.

    It seems to me that when a an application process is killed, its
    physical pages should be returned to the free pool, and not left around
    as inactive.

    Or perhaps that's not what's happening, perhaps some hidden process is }>eating all my virtual memory.

    The practical result is that no matter how much physical memory I have
    in my system, after a certain period of time the system gets into a
    state where it's paging out every time I try to do anything. I have to }>reboot every day even with a gig of memory installed.

    Does anyone have an explanation for why this happens?
    }
    }This is normal for a unix system. Generally unix keeps a small
    }free pool of memory and allocates everything else. Suppose you
    }use application X and quit it. Rather than work hard to free all the
    }memory, unix assumes that maybe you will restart application X
    }and it will be able to reuse the old memory rather than loading
    }it once again from disk. Actually it makes sense, what application
    }are you more likely to restart than the one you just quit? (The
    }oh damn I forgot factor.)
    }
    }Old versions of unix even had a way of keeping common applications
    }on the swap partition. It was called the sticky bit.

    Okay, and with the single digit bus speeds of yesteryear, that sorta
    makes sense. The question remains, when does the system decide you
    aren't going to relaunch the program, and let something else use the
    memory? I only use Photoshop every few days, and it hogs a metric
    buttload of space.

    --
    | /\_/\ | The Genome Diversification Project
    |_( ^.^ )_| Because one sentient species on the planet
    | >`-'< | is simply too boring for words.
    +-Pounce!-+------Remove obvious spamtrap to email------
    --- Synchronet 3.18b-Win32 NewsLink 1.113
  • From jimglidewell@jimglidewell@attbi.com (Jim Glidewell) to comp.sys.mac.system on Wednesday, July 23, 2003 18:06:40
    From Newsgroup: comp.sys.mac.system

    In article <fishfry-3BB973.08151023072003@netnews.attbi.com>, fishfry <fishfry@your-mailbox.com> wrote:

    I'm trying to understand some virtual memory strangeness I'm seeing.

    After a reboot, I have lots of free memory available (as shown by
    'top'). After a while, the amount of inactive memory increases until there's hardly any free memory left, at which point my system starts
    getting lots of pageouts.

    What method are you using to determine the number of page-outs? Does top
    show them?

    I see all the behaviors you report *except* pageouts.

    What is confusing to me is that this condition persists even after I
    kill all my applications. Even with no applications running, top
    reports a large figure for inactive memory.

    This is per design.

    It seems to me that when a an application process is killed, its
    physical pages should be returned to the free pool, and not left around
    as inactive.

    Why, besides "purity of essence"? If the pages are needed, they can be
    released instantly (not paged-out) when a new app needs them. (this
    applies to "clean" pages - those that still match their on-disk
    counterpart)

    OTOH, if you re-launch one of the apps you quit, you save a lot of
    disk i/o because you don't need to read those pages in again.

    OSX's philosophy is "the memory is there, make full use of it." Pages
    are retained until something else forces them out. Most of those
    inactive pages are effectively disk cache, which is a better use of
    memory than leaving it empty.

    There are some folks here who claim that the OSX memory management
    strategy is "broken", but it seems sensible to me.

    Or perhaps that's not what's happening, perhaps some hidden process is eating all my virtual memory.

    There are plenty of ways to determine this. "top" is one. Both physical
    and virtual memory size is reported for each process.

    The practical result is that no matter how much physical memory I have
    in my system, after a certain period of time the system gets into a
    state where it's paging out every time I try to do anything. I have to reboot every day even with a gig of memory installed.

    Having memory full is normal, page-outs are not. When this state arises,
    run terminal, run "top", and grab/view the data. Page-outs are reported
    in the header, both in page-outs since boot and (page-outs in the last
    second). Page-ins, BTW, are completely normal and unavoidable.

    Does anyone have an explanation for why this happens?

    If you're actually paging-out, then there probably is an errant process.
    If you memory is full, things are normal. Your "top" output will tell
    you/us a lot.

    --
    Jim Glidewell
    My opinions only
    --- Synchronet 3.18b-Win32 NewsLink 1.113
  • From Tom Harrington@tph@pcisys.no.spam.dammit.net to comp.sys.mac.system on Wednesday, July 23, 2003 12:09:44
    From Newsgroup: comp.sys.mac.system

    In article <05e72c217b66f22390c99c68817956ea@TeraNews>,
    Revanant Morituri <revanant@obvious.adelphia.net> wrote:

    Okay, and with the single digit bus speeds of yesteryear, that sorta
    makes sense. The question remains, when does the system decide you
    aren't going to relaunch the program, and let something else use the
    memory? I only use Photoshop every few days, and it hogs a metric
    buttload of space.

    It lets something else use the memory as soon as the memory is actually
    needed for something else. The "inactive" memory of things kept around
    just in case is available for use by anything else that might need it.

    --
    Tom "Tom" Harrington
    Macaroni, Automated System Maintenance for Mac OS X.
    Version 1.4: Best cleanup yet, gets files other tools miss.
    See http://www.atomicbird.com/
    --- Synchronet 3.18b-Win32 NewsLink 1.113
  • From Gnarlodious@gnarlodiousNULL@VOID.invalid.yahoo.com to comp.sys.mac.system on Wednesday, July 23, 2003 19:39:41
    From Newsgroup: comp.sys.mac.system

    Yeah, I had that exact same prob. The culprit was an early version of
    "System Events" which was unquittable and allocated more and more memory.

    -- Gnarlie

    --- Synchronet 3.18b-Win32 NewsLink 1.113
  • From matt neuburg@matt@tidbits.com to comp.sys.mac.system on Wednesday, July 23, 2003 19:59:42
    From Newsgroup: comp.sys.mac.system

    In <tph-B18FDC.11114723072003@localhost> Tom Harrington wrote:
    In article <fishfry-617321.09383523072003@netnews.attbi.com>,
    fishfry <fishfry@your-mailbox.com> wrote:

    In article <20030723092014459-0700@news.la.sbcglobal.net>,
    matt neuburg <matt@tidbits.com> wrote:

    I don't have an explanation, but you're right, that's the way it
    works. The real problem is that as extra swapfiles are generated
    they too are not given back when you quit applications, so your
    hard disk starts filling up.

    That's not true. I've wondered about this, and I have observed the
    number of swap files both increasing and decreasing as memory
    requirements vary. Decreasing when, as would be expected, I have quit applications.

    We've had this discussion before, so it's pointless to rehash it. I'm
    sure what you're saying is true for you, but what I'm saying is true for
    me. I've never seen the number of swapfiles revert of its own accord to,
    say, the 1 that is present when I restart; and over time my RAM is increasingly eaten by Active (not inactive) memory that is not given
    back even after I've quit everything I started. This has been true for
    the two years I've been monitoring this stuff. (I've had a number of MemoryStick users write to say that the problem seems worse in recent
    versions of the system; I suspect that has something to do with window caching.) m.

    --
    matt neuburg, phd = matt@tidbits.com, http://www.tidbits.com/matt
    REALbasic: The Definitive Guide! 2nd edition! http://www.amazon.com/exec/obidos/ASIN/0596001770/somethingsbymatt
    Subscribe to TidBITS. It's free and smart.
    --- Synchronet 3.18b-Win32 NewsLink 1.113
  • From Wayne C. Morris@wayne.morris@this.is.invalid to comp.sys.mac.system on Wednesday, July 23, 2003 22:36:56
    From Newsgroup: comp.sys.mac.system

    In article <05e72c217b66f22390c99c68817956ea@TeraNews>,
    Revanant Morituri <revanant@obvious.adelphia.net> wrote:

    In article <bfmdl7$4ug$1@news.fsu.edu>,
    bellenot@math.fsu.edu (Steve Bellenot) wrote:

    }This is normal for a unix system. Generally unix keeps a small
    }free pool of memory and allocates everything else. Suppose you
    }use application X and quit it. Rather than work hard to free all the }memory, unix assumes that maybe you will restart application X
    }and it will be able to reuse the old memory rather than loading
    }it once again from disk. Actually it makes sense, what application
    }are you more likely to restart than the one you just quit? (The
    }oh damn I forgot factor.)

    Okay, and with the single digit bus speeds of yesteryear, that sorta
    makes sense. The question remains, when does the system decide you
    aren't going to relaunch the program, and let something else use the
    memory?

    When it runs out of "free" memory.

    Whenever a program needs some memory, OS X will take it from "free"
    memory if possible. If there isn't enough free memory to satisfy the
    request, OS X will take it from the "inactive" pile. And if it runs out
    of both free and inactive memory, you get lots of pageouts and a
    noticeable drop in speed.

    In other words, both "free" and "inactive" memory are available for use,
    but as long as there's a choice, the "free" memory will be used first.
    --- Synchronet 3.18b-Win32 NewsLink 1.113
  • From fishfry@fishfry@your-mailbox.com to comp.sys.mac.system on Wednesday, July 23, 2003 23:41:09
    From Newsgroup: comp.sys.mac.system

    In article <jimglidewell-2307031106400001@green.sdc.cs.boeing.com>,
    jimglidewell@attbi.com (Jim Glidewell) wrote:

    In article <fishfry-3BB973.08151023072003@netnews.attbi.com>, fishfry <fishfry@your-mailbox.com> wrote:

    I'm trying to understand some virtual memory strangeness I'm seeing.

    After a reboot, I have lots of free memory available (as shown by
    'top'). After a while, the amount of inactive memory increases until there's hardly any free memory left, at which point my system starts getting lots of pageouts.

    What method are you using to determine the number of page-outs? Does top
    show them?


    Yes.

    I see all the behaviors you report *except* pageouts.


    What OS version are you using? If 10.2.6 that would be interesting. Apparently some but not all people see this problem


    If you're actually paging-out, then there probably is an errant process.
    If you memory is full, things are normal. Your "top" output will tell
    you/us a lot.

    Once I run out of freespace, everything I do results in hundreds of
    pageouts as shown by top. Reading mail and loading web pages becomes an exercize in watching the spinning beachball. Quitting all my active applications doesn't help. The symptoms are certainly consistent with a serious memory leak.
    --- Synchronet 3.18b-Win32 NewsLink 1.113
  • From Declan MacLeod@declan_macleoid@now-munged-for-your-protection.hotmail.com to comp.sys.mac.system on Thursday, July 24, 2003 00:24:18
    From Newsgroup: comp.sys.mac.system

    In article <fishfry-6FD04B.16414023072003@netnews.attbi.com>,
    fishfry <fishfry@your-mailbox.com> wrote:



    If you're actually paging-out, then there probably is an errant process.
    If you memory is full, things are normal. Your "top" output will tell you/us a lot.

    Once I run out of freespace, everything I do results in hundreds of
    pageouts as shown by top. Reading mail and loading web pages becomes an exercize in watching the spinning beachball. Quitting all my active applications doesn't help. The symptoms are certainly consistent with a serious memory leak.

    Have you by chance read this thread? http://discussions.info.apple.com/WebX?14@63.bntKabLybw1.6@.59972b98

    --
    Hey spambots! Harvest this! .!..
    Unsolicited commercial email (uce) should be sent to: uce@ftc.gov
    --- Synchronet 3.18b-Win32 NewsLink 1.113
  • From Paul McGrane@pmcgrane@mac.com.NOSPAM.invalid to comp.sys.mac.system on Friday, July 25, 2003 02:48:51
    From Newsgroup: comp.sys.mac.system

    In article <20030723092014459-0700@news.la.sbcglobal.net>,
    matt neuburg <matt@tidbits.com> wrote:

    I don't have an explanation, but you're right, that's the way it works.
    The real problem is that as extra swapfiles are generated they too are
    not given back when you quit applications, so your hard disk starts
    filling up. I wrote my free app MemoryStick exactly so that I could see memory usage. I restart when I start getting too many pageouts /
    swapfiles; I do this several times a day (but I have less RAM than you).
    m.

    How many VM swapfiles is enough to make you restart? Why is it
    worthwhile to delete them at restart and have them missing for a few
    hours, only to be rebuilt when necessary? Is your hard drive simply too
    small for your usage pattern? (how big is it?)

    I admit I'm skeptical of the need for all this restarting, but I am
    honestly interested in what you feel the boundary case is. I don't know whether my own use crosses it.

    --
    ...Paul McGrane
    http://pmcg.dnsalias.net/blog/
    --- Synchronet 3.18b-Win32 NewsLink 1.113
  • From laurent.daudelin@laurent.daudelin@ihatespam.verizon.net (Laurent Daudelin) to comp.sys.mac.system on Friday, July 25, 2003 11:00:14
    From Newsgroup: comp.sys.mac.system

    In message <20030723125141384-0700@news.la.sbcglobal.net>, matt neuburg <matt@tidbits.com> wrote:

    In <tph-B18FDC.11114723072003@localhost> Tom Harrington wrote:
    In article <fishfry-617321.09383523072003@netnews.attbi.com>,
    fishfry <fishfry@your-mailbox.com> wrote:

    In article <20030723092014459-0700@news.la.sbcglobal.net>,
    matt neuburg <matt@tidbits.com> wrote:

    I don't have an explanation, but you're right, that's the way it
    works. The real problem is that as extra swapfiles are generated
    they too are not given back when you quit applications, so your
    hard disk starts filling up.

    That's not true. I've wondered about this, and I have observed the
    number of swap files both increasing and decreasing as memory
    requirements vary. Decreasing when, as would be expected, I have quit applications.

    We've had this discussion before, so it's pointless to rehash it. I'm
    sure what you're saying is true for you, but what I'm saying is true for
    me. I've never seen the number of swapfiles revert of its own accord to, say, the 1 that is present when I restart; and over time my RAM is increasingly eaten by Active (not inactive) memory that is not given
    back even after I've quit everything I started. This has been true for
    the two years I've been monitoring this stuff. (I've had a number of MemoryStick users write to say that the problem seems worse in recent versions of the system; I suspect that has something to do with window caching.) m.


    You can usually get rid of some swap files simply by logging out. For me,
    at least, it does work. I guess that simply quitting all running applications. You still have the Finder and the Dock running. Logging out terminates those processes.

    -Laurent.
    --
    =================================================================
    Laurent Daudelin Logiciels Nemesys Software mailto:laurent.daudelin@ihatespam.net <http://nemesys.dyndns.org>
    Replace ihatespam with verizon to reply
    +++++++++++ Sent using an Apple Newton MessagePad 2100 ++++++++++

    --- Synchronet 3.18b-Win32 NewsLink 1.113
  • From claudel@claudel@bolt.sonic.net (claudel) to comp.sys.mac.system on Friday, July 25, 2003 09:59:30
    From Newsgroup: comp.sys.mac.system

    In article <charles.bouldin-83D7EF.06454625072003@text-east.newsfeeds.com>, Charles Bouldin <charles.bouldin@nist.gov> wrote:
    In article <pmcgrane-42F629.02485125072003@news.comcast.giganews.com>,
    Paul McGrane <pmcgrane@mac.com.NOSPAM.invalid> wrote:

    In article <20030723092014459-0700@news.la.sbcglobal.net>,
    matt neuburg <matt@tidbits.com> wrote:

    I don't have an explanation, but you're right, that's the way it works. >> > The real problem is that as extra swapfiles are generated they too are
    not given back when you quit applications, so your hard disk starts
    filling up. I wrote my free app MemoryStick exactly so that I could see >> > memory usage. I restart when I start getting too many pageouts /
    swapfiles; I do this several times a day (but I have less RAM than you). >> > m.

    How many VM swapfiles is enough to make you restart? Why is it
    worthwhile to delete them at restart and have them missing for a few
    hours, only to be rebuilt when necessary? Is your hard drive simply too
    small for your usage pattern? (how big is it?)

    I admit I'm skeptical of the need for all this restarting, but I am
    honestly interested in what you feel the boundary case is. I don't know
    whether my own use crosses it.

    I would be interested in what improved performance you get after the >restarts that makes this worthwhile. If you really have to do this, then >something is terribly wrong, for I let my system(s) run for months at a
    time and don't notice any performance degradation, excessive disk
    activity, slower launches, etc. Is this memory leak associated with some >particular app or group of apps?

    I have Memory Monitor running almost continuously and I also observe
    that memory usage grows over time, but for me (I have 1 gig of RAM), the >usage stabilizes with ~250 megs free and hundreds of megs inactive. This >steady state maintains itself under pretty heavy loads (I typically
    launch 10-20 major apps and just leave them open.) for weeks to months.


    ----== Posted via Newsfeed.Com - Unlimited-Uncensored-Secure Usenet News==---- >http://www.newsfeed.com The #1 Newsgroup Service in the World! >100,000 Newsgroups
    ---= 19 East/West-Coast Specialized Servers - Total Privacy via Encryption =---

    Hi

    In my case, ( 1Ghz PB, 1 Gb RAM, latest Jag ),
    if I do a "fink selfupdate" after running for awhile,
    and recompiling on of the larger components is required,
    I will run out of resources during the build. If I reboot
    and restart the "fink selfupdate" it generally will successfully
    complete at that time. I suspect Safari as at least one of the hogs...

    Another data point.

    Claude

    --- Synchronet 3.18b-Win32 NewsLink 1.113
  • From slavins@slavins@hearsay.demon.co.uk@localhost (Simon Slavin) to comp.sys.mac.system on Friday, July 25, 2003 23:08:37
    From Newsgroup: comp.sys.mac.system

    In article <20030723125141384-0700@news.la.sbcglobal.net>,
    matt neuburg <matt@tidbits.com> wrote:

    I've never seen the number of swapfiles revert of its own accord to,
    say, the 1 that is present when I restart; and over time my RAM is >increasingly eaten by Active (not inactive) memory that is not given
    back even after I've quit everything I started.

    This really is by design and that really is the way it's meant
    to work. OS X uses all RAM all the time. Having 'free'
    memory is a waste of memory.


    --- Synchronet 3.18b-Win32 NewsLink 1.113