• MakeNL bug with archived segments

    From Andrew Leary@1:320/219 to All on Fri Aug 11 18:24:08 2017
    Hello everybody!

    I have discovered an issue that can manifest itself when archived segments are submitted to the next *C up the chain. When archiving the new segment prior to submission, MakeNL doesn't check to see if the archive already exists. The result is submission of an archive containing the new segment, as well as an older one.

    There are two solutions that I've come up with:

    1. MakeNL check for an existing archive and remove it prior to archiving the segment for submission.
    2. When uncompressing incoming segment archives, compare file dates on all files unpacked and use the most recent.

    What do you think? I'm leaning towards number 2 first, as that will help in cases where the lower level *C hasn't upgraded his version of MakeNL yet.

    As a work around, the problem can be avoided by disabling the compression of submitted segments by adding:

    THReshold -1 -1

    to your control file.

    Andrew

    --- GoldED+/LNX 1.1.5-b20170303
    * Origin: Phoenix BBS * phoenix.bnbbbs.net (1:320/219)
  • From Ward Dossche@2:292/854 to Andrew Leary on Sat Aug 12 01:32:18 2017

    1. MakeNL check for an existing archive and remove it prior to archiving the
    segment for submission.
    2. When uncompressing incoming segment archives, compare file dates on
    all
    files unpacked and use the most recent.

    Terminate processing with an error and do not produce anything. It's not up to makenl to make such decisions.

    \%/@rd

    --- D'Bridge 3.99
    * Origin: I will always keep a PC running WinXP (2:292/854)
  • From mark lewis@1:3634/12.73 to Andrew Leary on Fri Aug 11 23:19:10 2017

    On 2017 Aug 11 18:24:08, you wrote to All:

    I have discovered an issue that can manifest itself when archived
    segments are submitted to the next *C up the chain. When archiving
    the new segment prior to submission, MakeNL doesn't check to see if
    the archive already exists. The result is submission of an archive containing the new segment, as well as an older one.

    i saw that on my system the other week... i figured it was simply because i had
    it keeping so many old segments... as i needed some more drive space, i just killed them off since they were old and out of date...

    where i saw it was in my "master" directory (which is set via master as well as
    outpath) in the z?? archives... i think janis ran into it some time back, too... i have some here now that have up to three in them... the problem stems from using just the last two numbers of the extension... there can only be 100 z?? files x00 through x99 and then the numbers start repeating again and thus the archive extensions...

    where i'm seeing this is actually at the zone level in the nodelist archives that it creates... that ctl file also has "threshold -1 -1" in it... as this is
    a testing zone, it isn't a big thing here but it is something that probably should be taken care... the region and net segments being created here for this
    testing zone do not compress anything or i'm sure i would be seeing it with them, too... i'm not sure why the zone level is even creating z?? archive files
    for distribution with "threshold -1 -1" unless it is something forced from "make composite" or the existance of "arccopy" and "arcmove" statements... the two test nets don't even have arccopy or arcmove defined... the region and zone
    do but the region is not creating z?? files whereas the zone is... hummm...

    There are two solutions that I've come up with:

    1. MakeNL check for an existing archive and remove it prior to archiving the segment for submission. 2. When uncompressing incoming segment archives, compare file dates on all files unpacked and use the most
    recent.

    What do you think? I'm leaning towards number 2 first, as that will help in cases where the lower level *C hasn't upgraded his version of MakeNL yet.

    i tend to lean toward #2 as well with the addition of deleting the other old files that came in that archive, too... no since in trashing up someone else's system with old garbage files... hopefully when they were extracted, they didn't overwrite any that they might have had retained... but i guess that depends on where you extract the files to before starting to work with them...

    As a work around, the problem can be avoided by disabling the
    compression of submitted segments by adding:

    THReshold -1 -1

    to your control file.

    in this day in time, that might be a good thing to have as a default... fidonet
    really doesn't need to compress things like it once did, does it?

    )\/(ark

    Always Mount a Scratch Monkey
    Do you manage your own servers? If you are not running an IDS/IPS yer doin' it wrong...
    ... Oh No, no, Nurse. I said remove his SPECTACLES!
    ---
    * Origin: (1:3634/12.73)
  • From Michiel van der Vlist@2:280/5555 to Andrew Leary on Sat Aug 12 09:31:32 2017
    Hello Andrew,

    On Friday August 11 2017 18:24, you wrote to All:

    I have discovered an issue that can manifest itself when archived
    segments are submitted to the next *C up the chain. When archiving
    the new segment prior to submission, MakeNL doesn't check to see if
    the archive already exists. The result is submission of an archive containing the new segment, as well as an older one.

    This phenomena is not limited to MakeNl. I have seen it with other nodelist/pointlist processors that use archives as well.

    There are two solutions that I've come up with:

    This is not really a bug in MakeNl as such. The problem is so common that I would classify it as PEBCAK.

    A note in de Docs should solve that. One should simply clean out the working directory at regular intervals. If one suffers from an uncurable hoarding disease and MUST keep all the old junk, do it in a another directory, not in the working directory.

    1. MakeNL check for an existing archive and remove it prior to
    archiving the segment for submission. 2. When uncompressing incoming segment archives, compare file dates on all files unpacked and use the most recent.

    What do you think?

    If you must solve this in MakeNl instead of educate the nodelist clerks, go for
    #1. That solves the problem at the source. #2 is just dealing with the symptoms.

    I'm leaning towards number 2 first, as that will help in cases where
    the lower level *C hasn't upgraded his version of MakeNL yet.

    And make them lazy? Distributing software is cheap and easy these days. There is no excuse for *Cs to not use the latest version.


    Cheers, Michiel

    --- GoldED+/W32-MSVC 1.1.5-b20170303
    * Origin: http://www.vlist.eu (2:280/5555)
  • From Vince Coen@2:250/1 to Andrew Leary on Sat Aug 12 13:43:22 2017
    Hello Andrew!

    Friday August 11 2017 18:24, you wrote to All:

    Hello everybody!

    I have discovered an issue that can manifest itself when archived
    segments are submitted to the next *C up the chain. When archiving
    the new segment prior to submission, MakeNL doesn't check to see if
    the archive already exists. The result is submission of an archive containing the new segment, as well as an older one.

    There are two solutions that I've come up with:

    1. MakeNL check for an existing archive and remove it prior to
    archiving the segment for submission. 2. When uncompressing incoming segment archives, compare file dates on all files unpacked and use the
    most recent.

    What do you think? I'm leaning towards number 2 first, as that will
    help in cases where the lower level *C hasn't upgraded his version of
    MakeNL yet.

    As a work around, the problem can be avoided by disabling the
    compression of submitted segments by adding:

    THReshold -1 -1

    to your control file.

    I just use a script that runs the program that deletes residual files in the netmail and outbound areas prior to running makenl - see below :

    ----
    #!/bin/sh

    . . . removed rem. lines

    export MBSE_ROOT=/home/mbse

    # kill off old *.msg and net250.nnn / region25.nnn node segments

    rm -f $MBSE_ROOT/etc/makenl/outbound/region25.*
    rm -f $MBSE_ROOT/etc/makenl/outbound/region30.*
    rm -f $MBSE_ROOT/etc/makenl/netmail/*.*


    cd $MBSE_ROOT/etc/makenl
    $MBSE_ROOT/bin/makenl region.ctl >> makenl.log 2>>makenl.err $MBSE_ROOT/bin/makenl region-kees.ctl >> makenl.log 2>>makenl.err

    $MBSE_ROOT/bin/makenl region30.ctl >> makenl30.log 2>>makenl30.err


    # Send for region 25 fidonet
    $MBSE_ROOT/bin/mbout a f5003.n280.z2 c ./outbound/region25K.* -q


    $MBSE_ROOT/bin/mbout a f854.n292.z2 c ./outbound/region25.* -q

    # Send for region 30 sportnet
    $MBSE_ROOT/bin/mbout a f0.n24.z24 c ./outbound/region30.* -q

    exit
    ----

    Note that the ./outbound/ section remaps to /home/mbse/etc/makenl/outbound prior to running the script and that my system root for mailer operations is
    at /home/mbse.

    I am running makenl v3.4.5 but show that latest is 3.4.6 in my files area.


    Vince

    --- Mageia Linux v5/Mbse v1.0.7.2/GoldED+/LNX 1.1.501-b20150715
    * Origin: Air Applewood, The Linux Gateway to the UK & Eire (2:250/1)