1. Forum
  2. FidoNet
  3. Z1C

  • MSGID problem

    From Nick Andre@1:229/426 to All on Monday, May 30, 2022 17:36:01
    Am thinking out loud, that the Z1DAILY echo has had daily activity for at least 3 years now. Could it be possible that some systems are not receiving the daily robot-post over duplicate MSGID's?

    It is not possible at the moment for me to write a custom report to check my local message base for duplicates... but it can be done.

    Nick

    --- Renegade vY2Ka2
    * Origin: Joey, do you like movies about gladiators? (1:229/426)
  • From Dale Shipp@1:261/1466 to Nick Andre on Tuesday, May 31, 2022 00:33:00
    On 05-30-22 17:36, Nick Andre <=-
    spoke to All about MSGID problem <=-


    Am thinking out loud, that the Z1DAILY echo has had daily activity
    for at least 3 years now. Could it be possible that some systems are
    not receiving the daily robot-post over duplicate MSGID's?

    If there is one such message per day, that should be highly unlikely.
    The MSGID is (or should be) 32 bits. 3*365 days = 1095, just over 2^10.
    With that many MSGIDs, there would be 1095*1094/2 comparisons, which is
    a little more than 2^19. Hence the odds of a duplicate MSGID would be
    about 2^(-13).

    Of course, if the MSGID being used has more structure than purely
    pseudorandom, that might change the result. I have seen systems using a
    16-bit MSGID and that would produce multiple hits.

    Dale Shipp
    fido_261_1466 (at) verizon (dot) net
    (1:261/1466)


    ... Shipwrecked in Silver Spring, Maryland. 00:23:28, 31 May 2022
    ___ Blue Wave/DOS v2.30

    --- Maximus/NT 3.01
    * Origin: Owl's Anchor (1:261/1466)
  • From mark lewis@1:3634/12 to Nick Andre on Tuesday, May 31, 2022 05:03:38
    Re: MSGID problem
    By: Nick Andre to All on Mon May 30 2022 17:36:01

    Am thinking out loud, that the Z1DAILY echo has had daily activity
    for at least 3 years now. Could it be possible that some systems
    are not receiving the daily robot-post over duplicate MSGID's?

    it could be possible, yes... especially if systems can actually hold years of MSGIDs in their dupe base rather than just weeks or months...

    there's also the complication(?) of systems that use one dupe base for the entire load of messages they carry compared to those that use a separate dupe base for each message area... in the first case, a
    second message with an existing MSGID would be seen as a dupe if it appears in any message base the system carries whereas in the second case the second message would be a dupe only if it appears in the same
    message base as the original...

    then you have systems that purge their MSGID database when they purge old messages while others continue to hold the MSGIDs of messages no longer on the system...

    and finally, there are systems that do not pass on messages they detect as dupes and systems that do pass on all messages even if they see some as dupes...

    couple two or more of these scenarios together and it is possible that some systems will be removing messages and not sending them on because they saw them as dupes...


    FWIW: in the case of sestar, three years plus a few days (1100 days IIRC) of messages and MSGIDs are retained, the MSGIDs are kept in per area databases, and all messages are passed on to all links even if
    sestar sees them as dupes... this allows each link connected to sestar to decide on their own if a message is a dupe or not based on their rules and database retention policies...

    i don't know how many other systems may be configured the same way or how many other systems can actually carry three years of MSGIDs in their databases, though...

    still having first cup of c0ffee so this may not be written as clearly as i hope it is...


    )\/(ark
    --- SBBSecho 3.11-Linux
    * Origin: SouthEast Star Mail HUB - SESTAR (1:3634/12)
  • From Fred Riccio@1:132/174 to Nick Andre on Tuesday, May 31, 2022 17:33:12
    Hello Nick!

    30 May 22 17:36, Nick Andre wrote to All:

    check my
    local message base for duplicates... but it can be done.

    What format is your "local message base" in?

    --- Msged/NT 6.0.1
    * Origin: Somewhere in New Hampshire's White Mountains (1:132/174)
  • From Nick Andre@1:229/426 to Fred Riccio on Tuesday, May 31, 2022 17:48:02
    On 31 May 22 17:33:12, Fred Riccio said the following to Nick Andre:

    check my
    local message base for duplicates... but it can be done.

    What format is your "local message base" in?

    Renegade.

    Nick

    --- Renegade vY2Ka2
    * Origin: Joey, do you like movies about gladiators? (1:229/426)
  • From Fred Riccio@1:132/174 to Nick Andre on Tuesday, May 31, 2022 18:30:54
    Hello Nick!

    31 May 22 17:48, Nick Andre wrote to Fred Riccio:


    Renegade.

    I don't have any experiance with Renegade. Would it be possible to do a rescan of the area so I can get them all in a .pkt?

    --- Msged/NT 6.0.1
    * Origin: Somewhere in New Hampshire's White Mountains (1:132/174)
  • From Nick Andre@1:229/426 to Fred Riccio on Wednesday, June 01, 2022 08:04:26
    On 31 May 22 18:30:54, Fred Riccio said the following to Nick Andre:

    Renegade.

    I don't have any experiance with Renegade. Would it be possible to do a rescan of the area so I can get them all in a .pkt?

    Let me see what I can do over the weekend.

    Nick

    --- Renegade vY2Ka2
    * Origin: Joey, do you like movies about gladiators? (1:229/426)
  • From Fred Riccio@1:132/174 to Nick Andre on Wednesday, June 01, 2022 22:56:30
    Hello Nick!

    01 Jun 22 08:04, Nick Andre wrote to Fred Riccio:

    On 31 May 22 18:30:54, Fred Riccio said the following to Nick Andre:

    do a rescan of the area so I can get them all in a .pkt?

    Let me see what I can do over the weekend.

    If it would be easier for you, just turn the echo on and start sending it directly to me. I wrote code to check for dupes in a squish message base (and delete the dupes) last month for a guy in zone 2. It'll be no problem to run it here daily, weekly, or on whatever schedule you want.

    For what it's worth, I ran it against 2,000 messages in this echo. No dupe message IDs but it did find a few duplicate headers and message bodies.

    --- Msged/NT 6.0.1
    * Origin: Somewhere in New Hampshire's White Mountains (1:132/174)