• How are changed files marked?

    From AES@siegman@stanford.edu to comp.sys.mac.apps,comp.sys.mac.system on Saturday, April 15, 2006 21:15:18
    From Newsgroup: comp.sys.mac.system

    When SuperDuper! does an incremental backup (using Smart Copy), does it
    have to look at some "Yes, I've been changed since the last backup, or
    no, I haven't" indicator on each and every one of the 270,000 files on
    my HD?

    Or can it just start moving down the nested folder hierarchy from the
    top, looking at some marker associated with each _folder_ which tells
    whether anything inside that folder has been changed or not? -- so that
    if that marker says nothing has been changed inside the folder, it
    doesn't need to look at anything any deeper inside that folder?

    Reason for asking: Incremental backups using the SuperDuper! "Smart
    Copy" feature and similar-sized LaCie PocketDrives seem to go much
    faster on my iBook G4 than on my wife's PowerBook G4, despite my usually having modified many more files between weekly backups. I'm wondering
    if it's because my HD has a deeply nested multi-layer arrangement of
    nested folders, and she keeps all her files in just a few chaotic
    top-level folders.

    Or -- I suppose more likely -- maybe it's that I'm backing up to a
    FireWire PocketDrive and she's still using a USB-only PocketDrive.
    --- Synchronet 3.18b-Win32 NewsLink 1.113
  • From Simon@sdpope@gmail.com to comp.sys.mac.apps,comp.sys.mac.system on Sunday, April 16, 2006 04:35:16
    From Newsgroup: comp.sys.mac.system

    Yeah, firewire is substansially faster than USB. Especially USB 1.1.

    Mind the gap(In transfer speed and use).

    --- Synchronet 3.18b-Win32 NewsLink 1.113
  • From Bob Harris@nospam.News.Bob@remove.Smith-Harris.us to comp.sys.mac.apps,comp.sys.mac.system on Sunday, April 16, 2006 14:01:54
    From Newsgroup: comp.sys.mac.system

    In article <siegman-25E138.21151815042006@news.stanford.edu>,
    AES <siegman@stanford.edu> wrote:

    When SuperDuper! does an incremental backup (using Smart Copy), does it
    have to look at some "Yes, I've been changed since the last backup, or
    no, I haven't" indicator on each and every one of the 270,000 files on
    my HD?

    Or can it just start moving down the nested folder hierarchy from the
    top, looking at some marker associated with each _folder_ which tells whether anything inside that folder has been changed or not? -- so that
    if that marker says nothing has been changed inside the folder, it
    doesn't need to look at anything any deeper inside that folder?

    As far as I know, the modification time on each file is checked
    against the modification time on the existing clone. In addition,
    file names are checked to see if there is a new file or a file has
    been deleted.

    Reason for asking: Incremental backups using the SuperDuper! "Smart
    Copy" feature and similar-sized LaCie PocketDrives seem to go much
    faster on my iBook G4 than on my wife's PowerBook G4, despite my usually having modified many more files between weekly backups. I'm wondering
    if it's because my HD has a deeply nested multi-layer arrangement of
    nested folders, and she keeps all her files in just a few chaotic
    top-level folders.

    Or -- I suppose more likely -- maybe it's that I'm backing up to a
    FireWire PocketDrive and she's still using a USB-only PocketDrive.

    If you and your wife modify a lot of data between backups, and a
    huge volume of information must be copied, the Firewire would have
    the edge. But generally speaking incremental backups are not that
    data intensive.

    More likely is the speeds of the 4 different disks.

    Your iBook disk and your Firewire disk.
    Your Wife's Powerbook disk and her USB disk.

    A lot of time is going to be spend seeking on each disk looking
    for modification dates. The faster the disk, the sooner each seek
    will complete. So if one or more of your disks has a faster
    rotational speed (5400 RPMs or 7200 RPMs vs 4200 RPMs), then you
    will tend to finish sooner. Track to track head movement seek
    times can also be involved, but that info is not generally printed
    on the packaging, where as RPMs generally are.

    Another possibility is how the files are laid out on the disk. If
    for some reason the next file to check the modification date is
    adjacent to the previous file and so on, and so on, then it would
    take less time for you to seek than your wife. But this is less
    likely unless you are running some file re-organization utility on
    a regular basis.

    Bob Harris
    --- Synchronet 3.18b-Win32 NewsLink 1.113
  • From AES@siegman@stanford.edu to comp.sys.mac.apps,comp.sys.mac.system on Sunday, April 16, 2006 10:29:43
    From Newsgroup: comp.sys.mac.system

    In article <nospam.News.Bob-946977.10022716042006@news.verizon.net>,
    Bob Harris <nospam.News.Bob@remove.Smith-Harris.us> wrote:


    More likely is the speeds of the 4 different disks.

    Your iBook disk and your Firewire disk.
    Your Wife's Powerbook disk and her USB disk.

    A lot of time is going to be spend seeking on each disk looking
    for modification dates. The faster the disk, the sooner each seek
    will complete. So if one or more of your disks has a faster
    rotational speed (5400 RPMs or 7200 RPMs vs 4200 RPMs), then you
    will tend to finish sooner. Track to track head movement seek
    times can also be involved, but that info is not generally printed
    on the packaging, where as RPMs generally are.

    Thanks very much for instructive reply. As reward, another query: Any
    idea why Retrospect's incremental backup process seems to take very substasntially longer in the preparation phase than SuperDuper's Smart
    Copy, when backing up the same iBook computer, with the same change
    history, and two similar LaCie FireWire pocket drives as targets?

    (Maybe because Retrospect has to decode encoded data for the files
    already in the backup set?)
    --- Synchronet 3.18b-Win32 NewsLink 1.113
  • From neillmassello@neillmassello@earthlink.net (Neill Massello) to comp.sys.mac.apps,comp.sys.mac.system on Sunday, April 16, 2006 17:41:00
    From Newsgroup: comp.sys.mac.system

    AES <siegman@stanford.edu> wrote:

    When SuperDuper! does an incremental backup (using Smart Copy), does it
    have to look at some "Yes, I've been changed since the last backup, or
    no, I haven't" indicator on each and every one of the 270,000 files on
    my HD?

    Or can it just start moving down the nested folder hierarchy from the
    top, looking at some marker associated with each _folder_ which tells
    whether anything inside that folder has been changed or not? -- so that
    if that marker says nothing has been changed inside the folder, it
    doesn't need to look at anything any deeper inside that folder?

    Normally, the modification date for a folder changes only when an item
    is added to or deleted from it, not when an item inside it is edited or modified. For that reason, folder dates are generally irrelevant to
    backup procedures, and most backup apps look at the mod date for every
    file within the hierarchy they've been instructed to back up.

    --- Synchronet 3.18b-Win32 NewsLink 1.113
  • From Doc O'Leary@droleary.usenet@2q2006.subsume.com to comp.sys.mac.apps,comp.sys.mac.system on Sunday, April 16, 2006 14:57:17
    From Newsgroup: comp.sys.mac.system

    In article <siegman-27BADF.10294316042006@news.stanford.edu>,
    AES <siegman@stanford.edu> wrote:

    Thanks very much for instructive reply. As reward, another query: Any
    idea why Retrospect's incremental backup process seems to take very substasntially longer in the preparation phase than SuperDuper's Smart
    Copy, when backing up the same iBook computer, with the same change
    history, and two similar LaCie FireWire pocket drives as targets?

    I'd wager it's because Retrospect has crufty old code that was the
    result of a limited Carbon port where SuperDuper seems built for OS X. Consider, for example, just these two methods of finding files modified
    in the last week:

    mac64:~ droleary$ time find ~ -mtime -7

    real 0m13.309s
    user 0m0.239s
    sys 0m4.806s

    mac64:~ droleary$ time mdfind -onlyin ~ 'kMDItemFSContentChangeDate > $time.today(-7)'

    real 0m7.480s
    user 0m0.169s
    sys 0m0.387s

    So using Spotlight for just that simple query I can almost halve the
    time, and who knows what other attributes/factors come into play. There
    are all sorts of inefficiencies that have probably compounded over the
    years with Retrospect.

    --
    My personal UDP list: 127.0.0.1, 4ax.com, buzzardnews.com, googlegroups.com,
    heapnode.com, localhost, x-privat.org
    --- Synchronet 3.18b-Win32 NewsLink 1.113
  • From Bob Harris@nospam.News.Bob@remove.Smith-Harris.us to comp.sys.mac.apps,comp.sys.mac.system on Tuesday, April 18, 2006 02:15:31
    From Newsgroup: comp.sys.mac.system

    In article
    <droleary.usenet-6F15C6.14571716042006@sn-ip.vsrv-sjc.supernews.ne
    ,
    Doc O'Leary <droleary.usenet@2q2006.subsume.com> wrote:

    In article <siegman-27BADF.10294316042006@news.stanford.edu>,
    AES <siegman@stanford.edu> wrote:

    Thanks very much for instructive reply. As reward, another query: Any idea why Retrospect's incremental backup process seems to take very substasntially longer in the preparation phase than SuperDuper's Smart Copy, when backing up the same iBook computer, with the same change history, and two similar LaCie FireWire pocket drives as targets?

    I'd wager it's because Retrospect has crufty old code that was the
    result of a limited Carbon port where SuperDuper seems built for OS X. Consider, for example, just these two methods of finding files modified
    in the last week:

    mac64:~ droleary$ time find ~ -mtime -7

    real 0m13.309s
    user 0m0.239s
    sys 0m4.806s

    mac64:~ droleary$ time mdfind -onlyin ~ 'kMDItemFSContentChangeDate > $time.today(-7)'

    real 0m7.480s
    user 0m0.169s
    sys 0m0.387s

    So using Spotlight for just that simple query I can almost halve the
    time, and who knows what other attributes/factors come into play. There
    are all sorts of inefficiencies that have probably compounded over the
    years with Retrospect.

    But spotlight doesn't catalog system files and file types it is
    not interested in.

    Bob Harris
    --- Synchronet 3.18b-Win32 NewsLink 1.113