• JamNNTPd and UTF-8

    From Carlos Navarro@2:341/234.1 to All on Sun Mar 5 08:51:58 2023
    I had noticed that some Unicode characters were not displayed (showing the "rhombus with question mark" instead), precissely those that have a 0x8D byte when encoded in UTF-8. Examples: the cyrillic e (э), 0xD1 0x8D; the "thumbs up" emoji (👍), 0xF0 0x9F 0x91 0x8D.

    I've found that this issue can be fixed by adding "-keepsoftcr" to the corresponding line in jamnntpd.xlat, just like with CP866:

    ================================
    read UTF-8 utf-8 -keepsoftcr
    ================================

    Carlos

    --- Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101 Thunderb
    * Origin: cyb nntp (2:341/234.1)
  • From Tommi Koivula@2:221/360 to Carlos Navarro on Sun Mar 5 14:43:46 2023
    On 5 Mar 2023 9.51, Carlos Navarro wrote:

    I had noticed that some Unicode characters were not displayed (showing the "rhombus with question mark" instead), precissely those that have a 0x8D byte when encoded in UTF-8. Examples: the cyrillic e (), 0xD1 0x8D; the "thumbs up" emoji (), 0xF0 0x9F 0x91 0x8D.




    I've found that this issue can be fixed by adding "-keepsoftcr" to the corresponding line in jamnntpd.xlat, just like with CP866:

    ================================
    read UTF-8 utf-8 -keepsoftcr
    ================================

    Done. :)

    'Tommi

    --- Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101 Thunderbird/102.8.0
    * Origin: nntp://rbb.fidonet.fi - Lake Ylo - Finland (2:221/360.0)
  • From Carlos Navarro@2:341/234.1 to Tommi Koivula on Sun Mar 5 14:12:41 2023
    I had noticed that some Unicode characters were not displayed (showing
    the "rhombus with question mark" instead), precissely those that have a 0x8D byte when encoded in UTF-8. Examples: the cyrillic e (�), 0xD1 0x8D; the "thumbs up" emoji (�), 0xF0 0x9F 0x91 0x8D.




    I've found that this issue can be fixed by adding "-keepsoftcr" to the
    corresponding line in jamnntpd.xlat, just like with CP866:

    ================================
    read UTF-8   utf-8   -keepsoftcr
    ================================

    Done. :)

    Your message arrived here without the 0x8D chars... :-m
    Maybe your tosser removed them?

    Carlos

    --- Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101 Thunderb
    * Origin: cyb nntp (2:341/234.1)
  • From Tommi Koivula@2:221/6 to Carlos Navarro on Sun Mar 5 15:28:12 2023
    On 5 Mar 2023 15.12, Carlos Navarro wrote:

    >> ================================
    >> read UTF-8   utf-8   -keepsoftcr
    >> ================================

    TK> Done. :)

    Your message arrived here without the 0x8D chars... :-m

    Hmm..

    Maybe your tosser removed them?

    Maybe.. No idea..

    'Tommi

    --- FMail-lnx64 2.2.0.0
    * Origin: nntp://news.fidonet.fi (2:221/6.0)
  • From Tommi Koivula@2:221/1 to Carlos Navarro on Sun Mar 5 15:33:41 2023

    Your message arrived here without the 0x8D chars... :-m
    Maybe your tosser removed them?

    Test from HPT. (👍)

    --- Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101 Thunderbird/102.8.0
    * Origin: nntp://rbb.fidonet.fi - Lake Ylo - Finland (2:221/1.0)
  • From Tommi Koivula@2:221/6 to Tommi Koivula on Sun Mar 5 15:37:42 2023

    Test from HPT. ()

    Actually from Fastecho. :)

    --- FMail-lnx64 2.2.0.0
    * Origin: nntp://news.fidonet.fi (2:221/6.0)
  • From Carlos Navarro@2:341/234.12 to Tommi Koivula on Sun Mar 5 15:17:02 2023
    Test from HPT. (👍)

    Looks fine now 😍 (U+1F60D, 0xF0 0x9F 0x98 0x8D)

    --
    Carlos
    --- Hotdoged/2.13.5/Android
    * Origin: cyberiada mobile point (2:341/234.12)
  • From Tommi Koivula@2:221/360 to Carlos Navarro on Sun Mar 5 16:23:07 2023
    On 5 Mar 2023 16.17, Carlos Navarro wrote:

    TK> Test from HPT. (👍)

    Looks fine now 😍 (U+1F60D, 0xF0 0x9F 0x98 0x8D)

    Good!

    I had stripping soft-cr's on in GEcho, now off. Let's see how this goes out.

    (👍)(👍)(👍)(👍)(👍)(👍)(👍)(👍)

    --- Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101 Thunderbird/102.8.0
    * Origin: nntp://rbb.fidonet.fi - Lake Ylo - Finland (2:221/360.0)
  • From Carlos Navarro@2:341/234.12 to Tommi Koivula on Sun Mar 5 16:21:11 2023
    Good! I had stripping soft-cr's on in GEcho, now off. Let's see
    how this goes out. (👍)(👍)(👍)(👍)(👍)(👍)(👍)(👍)

    Perfect! 8 thumbs up's!

    --
    Carlos
    --- Hotdoged/2.13.5/Android
    * Origin: cyberiada mobile point (2:341/234.12)
  • From Oli@2:280/464.47 to Tommi Koivula on Sun Mar 5 18:35:13 2023
    Tommi wrote (2023-03-05):

    On 5 Mar 2023 16.17, Carlos Navarro wrote:

    Test from HPT. (👍)

    Looks fine now 😍 (U+1F60D, 0xF0 0x9F 0x98 0x8D)

    Good!

    I had stripping soft-cr's on in GEcho, now off. Let's see how this goes out.

    (👍)(👍)(👍)(👍)(👍)(👍)(👍)(👍)

    Does it work?
    Did I configure it correctly?

    ---
    * Origin: This site requires JavaScript (2:280/464.47)
  • From Tommi Koivula@2:221/1 to Oli on Sun Mar 5 20:08:24 2023
    Hello, Oli.
    On 05/03/2023 19.35 you wrote:

    Tommi wrote (2023-03-05):
    On 5 Mar 2023 16.17, Carlos Navarro wrote:
    Test from HPT. (👍)
    Looks fine now 😍 (U+1F60D, 0xF0 0x9F 0x98 0x8D)
    Good!
    I had stripping soft-cr's on in GEcho, now off. Let's see how this goes
    out.
    (👍)(👍)(👍)(👍)(👍)(👍)(👍)(👍)
    Does it work?
    Did I configure it correctly?

    Looks ok in hotdoged.

    ---
    * Origin: This site requires JavaScript (2:280/464.47)

    --
    Tommi

    --- HotdogEd/2.13.5 (Android; Google Android; rv:1) Hotdoged/1675406179000 HotdogEd/2.13.5
    * Origin: nntp://rbb.fidonet.fi - Lake Ylo - Finland (2:221/1.0)
  • From Benny Pedersen@2:230/0 to Carlos Navarro on Thu Feb 8 05:08:14 2024
    Hello Carlos!

    05 Mar 2023 08:51, Carlos Navarro wrote to All:

    I had noticed that some Unicode characters were not displayed (showing the "rhombus with question mark" instead), precissely those that have
    a 0x8D byte when encoded in UTF-8. Examples: the cyrillic e (), 0xD1 0x8D; the "thumbs up" emoji (Fl), 0xF0 0x9F 0x91 0x8D.

    I've found that this issue can be fixed by adding "-keepsoftcr" to the corresponding line in jamnntpd.xlat, just like with CP866:

    ================================
    read UTF-8 utf-8 -keepsoftcr
    ================================

    Carlos

    problem with jamnntpd is that it does not support multibyte, so only can change singlebyte in diffrent charsets

    better is to setup thunderbird to NOT send utf-8 at all, so keep what is supported in fidonet, but back from fidonet to thunderbird utf-8 works, have you tryed it ? :=)


    Regards Benny

    ... too late to die young :)

    --- Msged/LNX 6.1.2 (Linux/6.7.4-gentoo-dist (x86_64))
    * Origin: gopher://fido.junc.eu/ (2:230/0)
  • From Tommi Koivula@2:221/0 to Benny Pedersen on Thu Feb 8 07:50:06 2024
    On 08.02.2024 7:08, Benny Pedersen wrote:

    better is to setup thunderbird to NOT send utf-8 at all

    Would you tell us how to do that with these newer TB's?

    'Tommi

    ---
    * Origin: jamnntpd/lnx (2:221/0.0)
  • From Carlos Navarro@2:341/234.99 to Benny Pedersen on Tue Feb 20 17:56:32 2024
    08/02/2024 6:08, Benny Pedersen -> Carlos Navarro:

    I've found that this issue can be fixed by adding "-keepsoftcr" to the
    corresponding line in jamnntpd.xlat, just like with CP866:

    ================================
    read UTF-8 utf-8 -keepsoftcr
    ================================

    problem with jamnntpd is that it does not support multibyte, so only can change singlebyte in diffrent charsets

    I don't understand what you mean. JamNNTPd seems to work fine with UTF-8, except for some issues with quotes.

    Carlos

    --- Mozilla Thunderbird
    * Origin: cyberiada-NNTP (2:341/234.99)
  • From Nicholas Boel@1:154/10 to Carlos Navarro on Thu Feb 22 17:40:04 2024
    On Tue, 20 Feb 2024 23:56:32 +0100, Carlos Navarro -> Benny Pedersen wrote:

    08/02/2024 6:08, Benny Pedersen -> Carlos Navarro:

     CN>>> I've found that this issue can be fixed by adding "-keepsoftcr" to the
     CN>>> corresponding line in jamnntpd.xlat, just like with CP866:

     CN>>> ================================
     CN>>> read UTF-8   utf-8   -keepsoftcr
     CN>>> ================================

     BP>> problem with jamnntpd is that it does not support multibyte, so only can
     BP>> change singlebyte in diffrent charsets

    I don't understand what you mean. JamNNTPd seems to work fine with UTF-8, except for some issues with quotes.

    Bringing the thread from the Golded echo about Jamnntpd over here. Here's a test to see if those characters are before level 2 and 3 quotes on this reply.

    Regards,
    Nick

    ... "Take my advice, I don't use it anyway."
    --- Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:115.0) Gecko/20100101 Thunderb
    * Origin: _thePharcyde distribution system (Wisconsin) (1:154/10)
  • From Nicholas Boel@1:154/10 to Nicholas Boel on Thu Feb 22 17:42:10 2024
    Hello Accession,

    On Thursday February 22 2024 17:40, I wrote to Carlos Navarro:

    On Tue, 20 Feb 2024 23:56:32 +0100, Carlos Navarro -> Benny Pedersen wrote:

    08/02/2024 6:08, Benny Pedersen -> Carlos Navarro:

     CN>>>> I've found that this issue can be fixed by adding
     CN>>>> "-keepsoftcr" to the corresponding line in jamnntpd.xlat,
     CN>>>> just like with CP866:

     CN>>>> ================================
     CN>>>> read UTF-8   utf-8   -keepsoftcr
     CN>>>> ================================

     BP>>> problem with jamnntpd is that it does not support multibyte,
     BP>>> so only can change singlebyte in diffrent charsets

    I don't understand what you mean. JamNNTPd seems to work fine
    with UTF-8, except for some issues with quotes.

    Bringing the thread from the Golded echo about Jamnntpd over here.
    Here's a test to see if those characters are before level 2 and 3
    quotes on this reply.

    Quoted the whole thing in Golded. I don't see anything here. I also hex dumped the message before replying to it, and didn't see anything in there either. *shrug*

    Regards,
    Nick

    ... "Take my advice, I don't use it anyway."
    --- GoldED+/LNX 1.1.5-b20231004
    * Origin: _thePharcyde distribution system (Wisconsin) (1:154/10)
  • From Tommi Koivula@2:221/1.1 to Nicholas Boel on Fri Feb 23 07:05:22 2024
    Hi Nicholas.

    22 Feb 24 17:42:10, you wrote to you:

    I've found that this issue can be fixed by adding
    "-keepsoftcr" to the corresponding line in jamnntpd.xlat,
    just like with CP866:

    ================================
    read UTF-8 utf-8 -keepsoftcr
    ================================

    problem with jamnntpd is that it does not support multibyte,
    so only can change singlebyte in diffrent charsets

    ...

    Bringing the thread from the Golded echo about Jamnntpd over here.
    Here's a test to see if those characters are before level 2 and 3
    quotes on this reply.

    Quoted the whole thing in Golded. I don't see anything here. I also hex dumped the message before replying to it, and didn't see anything in
    there either. *shrug*

    But I do see crap there. And as you may see, it is not only the quotes, also multiple spaces are converted to crap.

    If you don't see the crap, it is probably because of your conversion settings in Golded.

    'Tommi

    ---
    * Origin: Point One (2:221/1.1)
  • From Tommi Koivula@2:221/1 to Nicholas Boel on Fri Feb 23 09:13:32 2024
    On 22.02.2024 23:42, Nicholas Boel wrote:

    Quoted the whole thing in Golded. I don't see anything here. I also hex dumped the message before replying to it, and didn't see anything in
    there either. *shrug*

    https://news.fidonet.fi/Capture.JPG

    ---
    * Origin: smapinntpd/lnx (2:221/1.0)
  • From Tommi Koivula@2:221/1 to Nicholas Boel on Fri Feb 23 07:36:40 2024
    Hello Nicholas Boel!

    23 Feb 24 07:05:22, Tommi Koivula wrote to Nicholas Boel:

    If you don't see the crap, it is probably because of
    your conversion settings in Golded.

    Or mine. I don't know. :)

    'Tommi

    --- ════════╗
    * Origin: ═╩══════════════════════════════ (2:221/1)
  • From Jay Harris@1:229/664 to Tommi Koivula on Fri Feb 23 03:51:26 2024
    On 23 Feb 2024, Tommi Koivula said the following...

    If you don't see the crap, it is probably because of
    your conversion settings in Golded.

    Or mine. I don't know. :)

    When viewing this message in Golded it had as empty tear line & the origin line has a line break. When viewed in Mystic is has a bunch of UTF-8 characters:

    https://postimg.cc/gallery/0khhW7t


    Jay

    ... IPA is what a Canadian says after drinking a bunch

    --- Mystic BBS v1.12 A49 2023/04/30 (Linux/64)
    * Origin: Northern Realms (1:229/664)
  • From Wilfred van Velzen@2:280/464 to Nicholas Boel on Fri Feb 23 10:42:49 2024
    Hi Nicholas,

    On 2024-02-22 17:40:04, you wrote to Carlos Navarro:

     CN>>>> I've found that this issue can be fixed by adding "-keepsoftcr"
     CN>>>> to
     CN>>>> the corresponding line in jamnntpd.xlat, just like with CP866:
    ^^

     CN>>>> ================================
     CN>>>> read UTF-8   utf-8   -keepsoftcr
     CN>>>> ================================
    ^^

     BP>>> problem with jamnntpd is that it does not support multibyte, so
     BP>>> only can change singlebyte in diffrent charsets
    ^^

    I don't understand what you mean. JamNNTPd seems to work fine with
    UTF-8, except for some issues with quotes.

    Bringing the thread from the Golded echo about Jamnntpd over here. Here's a
    test to see if those characters are before level 2 and 3 quotes on this reply.

    They are there on 2 and 3 (now 3 and 4) level quotes.


    Bye, Wilfred.

    --- FMail-lnx64 2.2.1.1
    * Origin: FMail development HQ (2:280/464)
  • From Wilfred van Velzen@2:280/464 to Nicholas Boel on Fri Feb 23 10:46:14 2024
    Hi Nicholas,

    On 2024-02-22 17:42:10, you wrote to you:

     CN>>>>> I've found that this issue can be fixed by adding
     CN>>>>> "-keepsoftcr" to the corresponding line in jamnntpd.xlat,
     CN>>>>> just like with CP866:

     CN>>>>> ================================
     CN>>>>> read UTF-8   utf-8   -keepsoftcr
     CN>>>>> ================================

     BP>>>> problem with jamnntpd is that it does not support multibyte,
     BP>>>> so only can change singlebyte in diffrent charsets

    I don't understand what you mean. JamNNTPd seems to work fine
    with UTF-8, except for some issues with quotes.

    Bringing the thread from the Golded echo about Jamnntpd over here.
    Here's a test to see if those characters are before level 2 and 3
    quotes on this reply.

    Quoted the whole thing in Golded. I don't see anything here. I also hex dumped the message before replying to it, and didn't see anything in there either. *shrug*

    They are in your message! And I see them:

    0160 74 2C 0D 20 C2 A0 43 4E 3E 3E 3E 3E 20 6A 75 73 t,.  CN>>>> jus
    ^^ ^^


    Bye, Wilfred.

    --- FMail-lnx64 2.2.1.1
    * Origin: FMail development HQ (2:280/464)
  • From Tommi Koivula@2:221/1.1 to Jay Harris on Fri Feb 23 10:32:58 2024
    Hi Jay.

    23 Feb 24 03:51:26, you wrote to me:

    On 23 Feb 2024, Tommi Koivula said the following...

    If you don't see the crap, it is probably because of
    your conversion settings in Golded.

    Or mine. I don't know. :)

    When viewing this message in Golded it had as empty tear line & the
    origin line has a line break. When viewed in Mystic is has a bunch of UTF-8 characters:

    https://postimg.cc/gallery/0khhW7t

    The message was written with Gossiped, which should have full utf-8 support. No wonder if some message readers fail to show these utf-8 tear and origin lines:

    https://news.fidonet.fi/tmp/gossiped.JPG

    'Tommi

    ---
    * Origin: Point One (2:221/1.1)
  • From Nicholas Boel@1:154/10 to Jay Harris on Fri Feb 23 16:44:42 2024
    On Fri, 23 Feb 2024 09:51:26 -0500, Jay Harris -> Tommi Koivula wrote:

    When viewing this message in Golded it had as empty tear line & the origin line has a line break.  When viewed in Mystic is has a bunch of UTF-8 characters:

    https://postimg.cc/gallery/0khhW7t

    How did my original reply to Carlos look on your end?

    Regards,
    Nick

    ... "Take my advice, I don't use it anyway."
    --- Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:115.0) Gecko/20100101 Thunderb
    * Origin: _thePharcyde distribution system (Wisconsin) (1:154/10)
  • From Nicholas Boel@1:154/10 to Wilfred van Velzen on Fri Feb 23 16:50:30 2024
    On Fri, 23 Feb 2024 16:46:14 +0100, Wilfred Van Velzen -> Nicholas Boel wrote:

    They are in your message! And I see them:

    0160   74 2C 0D 20  C2 A0 43 4E  3E 3E 3E 3E  20 6A 75 73

    And yet they didn't show in your reply to me, like some of the other ones did. This or the last message you wrote, unless you removed them yourself.

    Regards,
    Nick

    ... "Take my advice, I don't use it anyway."
    --- Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:115.0) Gecko/20100101 Thunderb
    * Origin: _thePharcyde distribution system (Wisconsin) (1:154/10)
  • From Nicholas Boel@1:154/10 to Tommi Koivula on Fri Feb 23 17:35:24 2024
    On Fri, 23 Feb 2024 13:05:22 +0000, Tommi Koivula -> Nicholas Boel wrote:

     NB>>> Bringing the thread from the Golded echo about Jamnntpd over here.
     NB>>> Here's a test to see if those characters are before level 2 and 3
     NB>>> quotes on this reply.

     NB>> Quoted the whole thing in Golded. I don't see anything here. I also hex
     NB>> dumped the message before replying to it, and didn't see anything in
     NB>> there either. *shrug*

    But I do see crap there. And as you may see, it is not only the quotes, also multiple spaces are converted to crap.

    If you don't see the crap, it is probably because of your conversion settings in Golded.

    I can see them in Thunderbird if I view the message source. Otherwise, they don't show. I've googled and found that Thunderbird does seem to have had issues with this in the past. I'm just looking for a way for it to not happen with my current setup.

    If the other two messages I posted from other nntp readers don't have that issue, then it's definitely TB, and I can start checking settings in the config editor.

    Regards,
    Nick

    ... "Take my advice, I don't use it anyway."
    --- Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:115.0) Gecko/20100101 Thunderb
    * Origin: _thePharcyde distribution system (Wisconsin) (1:154/10)
  • From Jay Harris@1:229/664 to Nicholas Boel on Fri Feb 23 19:27:22 2024
    On 23 Feb 2024, Nicholas Boel said the following...

    How did my original reply to Carlos look on your end?

    I think this is the message you're referring to: https://postimg.cc/gallery/kq34nZw


    Jay

    ... I wanna be wise but I'm otherwise

    --- Mystic BBS v1.12 A49 2023/04/30 (Linux/64)
    * Origin: Northern Realms (1:229/664)
  • From Nicholas Boel@1:154/10 to Jay Harris on Fri Feb 23 18:36:20 2024
    On Sat, 24 Feb 2024 01:27:22 -0500, Jay Harris -> Nicholas Boel wrote:

     NB>> How did my original reply to Carlos look on your end?

    I think this is the message you're referring to: https://postimg.cc/gallery/kq34nZw

    Yes, that's the one.. along with most of the others I wrote from Thunderbird. It seems Thunderbird adds non-breaking spaces (or &nbsp in html) anywhere there's two or more concurrent spaces next to each other.

    I've completely disabled html in TB now, and am forcing plain-text. So we will see in the near future if there's still an issue.

    Regards,
    Nick

    ... "Take my advice, I don't use it anyway."
    --- Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:115.0) Gecko/20100101 Thunderb
    * Origin: _thePharcyde distribution system (Wisconsin) (1:154/10)
  • From Nicholas Boel@1:154/10 to Jay Harris on Fri Feb 23 18:53:46 2024
    Hello Jay,

    On Sat, 24 Feb 2024 01:27:22 -0500, you wrote:

    How did my original reply to Carlos look on your end?

    I think this is the message you're referring to: https://postimg.cc/gallery/kq34nZw

    By the way, thanks for the screenshots and jumping into the converesation. It definitely seems like a Thunderbird issue so far, since I haven't seen it anywhere else as of yet.

    Regards,
    Nick

    ... Take my advice, I don't use it anyway.
    --- slrn/pre1.0.4-9 (Linux)
    * Origin: _thePharcyde distribution system (Wisconsin) (1:154/10)
  • From Wilfred van Velzen@2:280/464 to Nicholas Boel on Sat Feb 24 02:10:51 2024
    Hi Nicholas,

    On 2024-02-23 16:50:30, you wrote to me:

    They are in your message! And I see them:

    0160   74 2C 0D 20  C2 A0 43 4E  3E 3E 3E 3E  20 6A 75 73

    And yet they didn't show in your reply to me, like some of the other ones did. This or the last message you wrote, unless you removed them yourself.

    I didn't remove anything. I think it's better if you look at inbound and outbound .pkt files directly, and not rely on your editor to show you what is really in the messages.

    Bye, Wilfred.

    --- FMail-lnx64 2.2.1.1
    * Origin: FMail development HQ (2:280/464)
  • From Wilfred van Velzen@2:280/464 to Nicholas Boel on Sat Feb 24 02:15:52 2024
    Hi Nicholas,

    On 2024-02-23 18:36:20, you wrote to Jay Harris:

     NB>>> How did my original reply to Carlos look on your end?
    ^^
    Still there...

    I think this is the message you're referring to:
    https://postimg.cc/gallery/kq34nZw

    Yes, that's the one.. along with most of the others I wrote from Thunderbird. It seems Thunderbird adds non-breaking spaces (or &nbsp in html) anywhere there's two or more concurrent spaces next to each other.

    I've completely disabled html in TB now, and am forcing plain-text. So we will see in the near future if there's still an issue.


    Bye, Wilfred.

    --- FMail-lnx64 2.2.1.1
    * Origin: FMail development HQ (2:280/464)
  • From Nicholas Boel@1:154/10 to Wilfred van Velzen on Fri Feb 23 19:43:14 2024
    On Sat, 24 Feb 2024 08:15:52 +0100, Wilfred Van Velzen -> Nicholas Boel wrote:

      NB>>>> How did my original reply to Carlos look on your end?
     ^^
    Still there...

    Last attempt for the evening, I suppose. This one I changed the original message body to plain text before replying to it.

     JH>>> I think this is the message you're referring to:
     JH>>> https://postimg.cc/gallery/kq34nZw

     NB>> Yes, that's the one.. along with most of the others I wrote from
     NB>> Thunderbird. It seems Thunderbird adds non-breaking spaces (or &nbsp in
     NB>> html) anywhere there's two or more concurrent spaces next to each other.

    Frustrating, because I can't check it before the message is sent. It's something Thunderbird is doing after the message is saved.

    Regards,
    Nick

    ... "Take my advice, I don't use it anyway."
    --- Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:115.0) Gecko/20100101 Thunderb
    * Origin: _thePharcyde distribution system (Wisconsin) (1:154/10)
  • From Nicholas Boel@1:154/10 to Wilfred van Velzen on Fri Feb 23 19:47:02 2024
    On Sat, 24 Feb 2024 08:15:52 +0100, Wilfred Van Velzen -> Nicholas Boel wrote:

      NB>>>> How did my original reply to Carlos look on your end?
     ^^
    Still there...

     JH>>> I think this is the message you're referring to:
     JH>>> https://postimg.cc/gallery/kq34nZw

     NB>> Yes, that's the one.. along with most of the others I wrote from
     NB>> Thunderbird. It seems Thunderbird adds non-breaking spaces (or &nbsp in
     NB>> html) anywhere there's two or more concurrent spaces next to each other.

    Well, that didn't work. ;)

    Regards,
    Nick

    ... "Take my advice, I don't use it anyway."
    --- Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:115.0) Gecko/20100101 Thunderb
    * Origin: _thePharcyde distribution system (Wisconsin) (1:154/10)
  • From Nicholas Boel@1:154/10 to Wilfred van Velzen on Fri Feb 23 20:55:12 2024
    On Sat, 24 Feb 2024 08:15:52 +0100, Wilfred Van Velzen -> Nicholas Boel wrote:

      NB>>>> How did my original reply to Carlos look on your end?
     ^^
    Still there...

     JH>>> I think this is the message you're referring to:
     JH>>> https://postimg.cc/gallery/kq34nZw

     NB>> Yes, that's the one.. along with most of the others I wrote from
     NB>> Thunderbird. It seems Thunderbird adds non-breaking spaces (or
    Maybe? Maybe not?

    Regards,
    Nick

    ... "Take my advice, I don't use it anyway."
    --- Betterbird (Windows)
    * Origin: _thePharcyde distribution system (Wisconsin) (1:154/10)
  • From Tommi Koivula@2:221/1 to Nicholas Boel on Sat Feb 24 08:16:10 2024
    On 24.02.2024 4:55, Nicholas Boel wrote:

    ... "Take my advice, I don't use it anyway."
    --- Betterbird (Windows)

    There is another thunderbird clone called Epyrus. This one allows
    posting other charsets than utf-8.

    http://www.epyrus.org/

    ---
    * Origin: jamnntpd/lnx (2:221/1.0)
  • From Tommi Koivula@2:221/6 to Nicholas Boel on Sat Feb 24 08:36:26 2024
    There is another thunderbird clone called Epyrus. This one allows
    posting other charsets than utf-8.

    http://www.epyrus.org/

    Seamonkey also allows posting other than utf-8.

    However, the problem is not utf-8, I think it has proved that it is Thunderbird.

    'Tommi

    --- FMail-lnx32 2.2.1.1
    * Origin: news://news.fidonet.fi (2:221/6.0)
  • From Carlos Navarro@2:341/234.99 to Nicholas Boel on Sat Feb 24 08:58:38 2024
    24/02/2024 1:36, Nicholas Boel -> Jay Harris:

    How did my original reply to Carlos look on your end?

    I think this is the message you're referring to:
    https://postimg.cc/gallery/kq34nZw

    Yes, that's the one.. along with most of the others I wrote from Thunderbird. It seems Thunderbird adds non-breaking spaces (or &nbsp in html) anywhere there's two or more concurrent spaces next to each other.


    Exactly.

    I've noticed this in Thunderbird (and Seamonkey):

    1. An extra space in inserted in quoted lines that begin with one or more spaces.

    2. If using format=flowed (def_flowed or user/flowed=on in Smapi/JamNNTPd), when there are two or more consecutive spaces in the quoted text, all but one are replaced by non-breaking spaces (UTF-8: 0xC2 0xA0 / LATIN-1: 0xA0) when posted to the NNTP server.

    As FTN-style quotes are usually prefixed by a space, another one is inserted and then the second one is converted to a NBSP.

    If you manually remove the extra spaces in the quotes before sending the reply, no NBSPs are posted. (I'm doing it in this reply.)

    Carlos

    --- Mozilla Thunderbird
    * Origin: cyberiada-NNTP (2:341/234.99)
  • From Wilfred van Velzen@2:280/464 to Nicholas Boel on Sat Feb 24 10:53:11 2024
    Hi Nicholas,

    On 2024-02-23 19:43:14, you wrote to me:

      NB>>>>> How did my original reply to Carlos look on your end?
    ^^^^

    Still there.

     ^^
    Still there...

    Last attempt for the evening, I suppose. This one I changed the original message body to plain text before replying to it.

    Frustrating, because I can't check it before the message is sent. It's something Thunderbird is doing after the message is saved.

    Check the .pkt file...


    Bye, Wilfred.

    --- FMail-lnx64 2.2.1.1
    * Origin: FMail development HQ (2:280/464)
  • From Wilfred van Velzen@2:280/464 to Nicholas Boel on Sat Feb 24 10:54:43 2024
    Hi Nicholas,

    On 2024-02-23 20:55:12, you wrote to me:

      NB>>>>> How did my original reply to Carlos look on your end?
     ^^
    Still there...

     JH>>>> I think this is the message you're referring to:
     JH>>>> https://postimg.cc/gallery/kq34nZw

     NB>>> Yes, that's the one.. along with most of the others I wrote from
     NB>>> Thunderbird. It seems Thunderbird adds non-breaking spaces (or
    Maybe? Maybe not?

    Still there...


    Bye, Wilfred.

    --- FMail-lnx64 2.2.1.1
    * Origin: FMail development HQ (2:280/464)
  • From Nicholas Boel@1:154/10 to Tommi Koivula on Sat Feb 24 07:45:44 2024
    On Sat, 24 Feb 2024 14:16:10 +0200, Tommi Koivula -> Nicholas Boel wrote:

    There is another thunderbird clone called Epyrus. This one allows
    posting other charsets than utf-8.

    Thank you. I will take a look, but posting in charsets other than utf-8 is not really an interest of mine. Fixing what I already enjoy using, is.

    For now, I will just have to quote one level so there are no complaints. ;)

    Regards,
    Nick

    ... "Take my advice, I don't use it anyway."
    --- Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:115.0) Gecko/20100101 Thunderb
    * Origin: _thePharcyde distribution system (Wisconsin) (1:154/10)
  • From Nicholas Boel@1:154/10 to Tommi Koivula on Sat Feb 24 07:53:30 2024
    On Sat, 24 Feb 2024 14:36:26 +0200, Tommi Koivula -> Nicholas Boel wrote:

    Seamonkey also allows posting other than utf-8.

    However, the problem is not utf-8, I think it has proved that it is Thunderbird.

    Agreed. However, Thunderbird is the easiest on my eyes, my most preferred email client, and even the default font seems to support way more utf-8 characters than any other newsreader I've tried. So I will keep looking into it and see if I can find any way to fix it (maybe even hacking smapinntpd to remove these spaces if present).

    Regards,
    Nick

    ... "Take my advice, I don't use it anyway."
    --- Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:115.0) Gecko/20100101 Thunderb
    * Origin: _thePharcyde distribution system (Wisconsin) (1:154/10)
  • From Nicholas Boel@1:154/10 to Carlos Navarro on Sat Feb 24 07:58:28 2024
    On Sat, 24 Feb 2024 14:58:38 +0100, Carlos Navarro -> Nicholas Boel wrote:

    How did my original reply to Carlos look on your end?

    I think this is the message you're referring to:
    https://postimg.cc/gallery/kq34nZw

     NB>> Yes, that's the one.. along with most of the others I wrote from
     NB>> Thunderbird. It seems Thunderbird adds non-breaking spaces (or &nbsp in
     NB>> html) anywhere there's two or more concurrent spaces next to each other.

    1. An extra space in inserted in quoted lines that begin with one or more spaces.

    2. If using format=flowed (def_flowed or user/flowed=on in Smapi/JamNNTPd),
    when there are two or more consecutive spaces in the quoted text, all but one are replaced by non-breaking spaces (UTF-8: 0xC2 0xA0 / LATIN-1: 0xA0) when posted to the NNTP server.

    As FTN-style quotes are usually prefixed by a space, another one is inserted and then the second one is converted to a NBSP.

    If you manually remove the extra spaces in the quotes before sending the reply, no NBSPs are posted. (I'm doing it in this reply.)

    I've manually removed all but one space in the quotes above to see if this indeed works on my end as well.

    *shakes fist at Thunderbird*

    Regards,
    Nick

    ... "Take my advice, I don't use it anyway."
    --- Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:115.0) Gecko/20100101 Thunderb
    * Origin: _thePharcyde distribution system (Wisconsin) (1:154/10)
  • From Nicholas Boel@1:154/10 to Wilfred van Velzen on Sat Feb 24 08:00:38 2024
    On Sat, 24 Feb 2024 16:53:10 +0100, Wilfred Van Velzen -> Nicholas Boel wrote:

    Check the .pkt file...

    No. I can see them if I view the message sources, and what else would we have to talk/gripe about if this wasn't brought up? Most of us would be hitting "Mark all groups/messages read recursively" or "Catch up all" and not posting anything.

    So you're welcome for drumming up some conversation. :)

    Regards,
    Nick

    ... "Take my advice, I don't use it anyway."
    --- Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:115.0) Gecko/20100101 Thunderb
    * Origin: _thePharcyde distribution system (Wisconsin) (1:154/10)
  • From Wilfred van Velzen@2:280/464 to Nicholas Boel on Sat Feb 24 16:38:07 2024
    Hi Nicholas,

    On 2024-02-24 07:58:28, you wrote to Carlos Navarro:

    How did my original reply to Carlos look on your end?

    I think this is the message you're referring to:
    https://postimg.cc/gallery/kq34nZw

     NB>>> Yes, that's the one.. along with most of the others I wrote from
     NB>>> Thunderbird. It seems Thunderbird adds non-breaking spaces (or
     NB>>> &nbsp in html) anywhere there's two or more concurrent spaces next
     NB>>> to each other.

    They are here on these quotes, but on none of the others...?

    1. An extra space in inserted in quoted lines that begin with one or
    more spaces.

    2. If using format=flowed (def_flowed or user/flowed=on in
    Smapi/JamNNTPd), when there are two or more consecutive spaces in the
    quoted text, all but one are replaced by non-breaking spaces (UTF-8:
    0xC2 0xA0 / LATIN-1: 0xA0) when posted to the NNTP server.

    As FTN-style quotes are usually prefixed by a space, another one is
    inserted and then the second one is converted to a NBSP.

    If you manually remove the extra spaces in the quotes before sending
    the reply, no NBSPs are posted. (I'm doing it in this reply.)

    I've manually removed all but one space in the quotes above to see if this indeed works on my end as well.

    *shakes fist at Thunderbird*


    Bye, Wilfred.

    --- FMail-lnx64 2.2.1.1
    * Origin: FMail development HQ (2:280/464)
  • From Carlos Navarro@2:341/234.99 to Benny Pedersen on Sun Feb 25 12:25:09 2024
    08/02/2024 6:08, Benny Pedersen -> Carlos Navarro:

    problem with jamnntpd is that it does not support multibyte,

    You're right about JamNNTPd not supporting multibyte. It wraps lines at 79 bytes, not at 79 chars. Therefore those that have UTF-8 chars (2 or more bytes) become shorter when wrapped. The more UTF-8 chars a line has, the shorter.

    This is noticeable in readers that don't support flowed text (most NNTP clients).

    ASCII:
    aeiou aeiou aeiou aeiou aeiou aeiou aeiou aeiou aeiou aeiou aeiou aeiou aeiou aeiou

    UTF-8:
    áéíóú áéíóú áéíóú áéíóú áéíóú áéíóú áéíóú áéíóú áéíóú áéíóú áéíóú áéíóú áéíóú áéíóú

    Carlos

    --- Mozilla Thunderbird
    * Origin: cyberiada-NNTP (2:341/234.99)
  • From Carlos Navarro@2:341/234.1 to Nicholas Boel on Sun Feb 25 13:05:30 2024
    24 Feb 2024 07:58, you wrote to me:

    àNB>>> Yes, that's the one.. along with most of the others I wrote
    àNB>>> from Thunderbird. It seems Thunderbird adds non-breaking
    àNB>>> spaces (or &nbsp in html) anywhere there's two or more
    àNB>>> concurrent spaces next to each other.
    ^
    [...]

    I've manually removed all but one space in the quotes above to see if
    this indeed works on my end as well.

    All look fine except the ones I'm quoting above. I guess you did not remove the spaces in that block.

    *shakes fist at Thunderbird*

    :-)

    Have you tried adding '/flowed=off' to your username, or setting 'def_flowed off' in your Jam/SmapiNNTPd config? That way, those spaces will not converted to NBSPs.

    Carlos

    --- GoldED+/W32-MSVC 1.1.5-b20180707
    * Origin: cyberiada (2:341/234.1)
  • From Nicholas Boel@1:154/10 to Wilfred van Velzen on Sun Feb 25 09:10:52 2024
    On Sat, 24 Feb 2024 22:38:06 +0100, Wilfred Van Velzen -> Nicholas Boel wrote:

    How did my original reply to Carlos look on your end?

    I think this is the message you're referring to:
    https://postimg.cc/gallery/kq34nZw

    Yes, that's the one.. along with most of the others I wrote from
    Thunderbird. It seems Thunderbird adds non-breaking spaces (or
    &nbsp in html) anywhere there's two or more concurrent spaces next
    to each other.

    They are here on these quotes, but on none of the others...?

    Yes that was kind of a test. I backspaced all of the 2nd and 3rd level quotes to the newly created quote character except for one paragraph of quoted text to nail down the problem. Now if I understand all of this correctly, there shouldn't be any issues with the above quotes.

    Regards,
    Nick

    ... "Take my advice, I don't use it anyway."
    --- Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:115.0) Gecko/20100101 Thunderb
    * Origin: _thePharcyde distribution system (Wisconsin) (1:154/10)
  • From Nicholas Boel@1:154/10 to Carlos Navarro on Sun Feb 25 09:14:10 2024
    On Sun, 25 Feb 2024 18:25:08 +0100, Carlos Navarro -> Benny Pedersen wrote:

    You're right about JamNNTPd not supporting multibyte. It wraps lines at 79 bytes, not at 79 chars. Therefore those that have UTF-8 chars (2 or more bytes) become shorter when wrapped. The more UTF-8 chars a line has, the shorter.

    This is noticeable in readers that don't support flowed text (most NNTP clients).

    ASCII:
    aeiou aeiou aeiou aeiou aeiou aeiou aeiou aeiou aeiou aeiou aeiou aeiou aeiou aeiou

    UTF-8:
    áéíóú áéíóú áéíóú áéíóú áéíóú áéíóú áéíóú áéíóú áéíóú áéíóú áéíóú áéíóú
    áéíóú áéíóú

    This looked fine in Thunderbird, and if quoted correctly when it gets back to you, should have both of them wrapped at the same point also.

    Regards,
    Nick

    ... "Take my advice, I don't use it anyway."
    --- Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:115.0) Gecko/20100101 Thunderb
    * Origin: _thePharcyde distribution system (Wisconsin) (1:154/10)
  • From Nicholas Boel@1:154/10 to Carlos Navarro on Sun Feb 25 09:17:26 2024
    On Sun, 25 Feb 2024 19:05:30 +0100, Carlos Navarro -> Nicholas Boel wrote:

    I've manually removed all but one space in the quotes above to see if
    this indeed works on my end as well.

    All look fine except the ones I'm quoting above. I guess you did not remove
    the spaces in that block.

    Have you tried adding '/flowed=off' to your username, or setting 'def_flowed off' in your Jam/SmapiNNTPd config? That way, those spaces will
    not converted to NBSPs.

    I have not. I'm under the assumption this is a Thunderbird problem since with the same settings it doesn't occur with any of the other nntp clients I have installed here.

    I can definitely give it a shot, though!

    Regards,
    Nick

    ... "Take my advice, I don't use it anyway."
    --- Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:115.0) Gecko/20100101 Thunderb
    * Origin: _thePharcyde distribution system (Wisconsin) (1:154/10)
  • From Nicholas Boel@1:154/10 to Carlos Navarro on Sun Feb 25 09:45:40 2024
    On Sun, 25 Feb 2024 18:25:08 +0100, Carlos Navarro -> Benny Pedersen wrote:

    problem with jamnntpd is that it does not support multibyte,

    You're right about JamNNTPd not supporting multibyte. It wraps lines at
    79 bytes, not at 79 chars. Therefore those that have UTF-8 chars (2 or more bytes) become shorter when wrapped. The more UTF-8 chars a line
    has, the shorter.

    This is noticeable in readers that don't support flowed text (most NNTP clients).

    ASCII:
    aeiou aeiou aeiou aeiou aeiou aeiou aeiou aeiou aeiou aeiou aeiou aeiou aeiou aeiou

    UTF-8:
    áéíóú áéíóú áéíóú áéíóú áéíóú áéíóú áéíóú áéíóú áéíóú áéíóú áéíóú áéíóú áéíóú áéíóú

    For the record, def_flowed=off changes what looked correct before to this. This can also be a test for the top quoted line to see if those pesky non-breaking spaces are added here or not.

    Regards,
    Nick

    ... "Take my advice, I don't use it anyway."
    --- Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:115.0) Gecko/20100101 Thunderb
    * Origin: _thePharcyde distribution system (Wisconsin) (1:154/10)
  • From Nicholas Boel@1:154/10 to Carlos Navarro on Sun Feb 25 09:55:50 2024
    On Sun, 25 Feb 2024 19:05:30 +0100, Carlos Navarro -> Nicholas Boel wrote:

    Have you tried adding '/flowed=off' to your username, or setting 'def_flowed off' in your Jam/SmapiNNTPd config? That way, those spaces will not converted to NBSPs.

    This seems to work better in this regard.

    However, now obviously we're not using format=flowed text, so is it now hard wrapping the text that I'm currently writing?

    Isn't that supposed to be left alone (ie: no hard wrapping) so the next reader can deal with the message however it seems fit?

    Regards,
    Nick

    ... "Take my advice, I don't use it anyway."
    --- Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:115.0) Gecko/20100101 Thunderb
    * Origin: _thePharcyde distribution system (Wisconsin) (1:154/10)
  • From Tommi Koivula@2:221/10 to Carlos Navarro on Sun Feb 25 19:54:40 2024
    From: Tommi Koivula <tommi@rbb.homeip.net>

    On 25.02.2024 13:30, Carlos Navarro wrote:

    You're right about JamNNTPd not supporting multibyte. It wraps lines
    at 79 bytes, not at 79 chars. Therefore those that have UTF-8 chars
    (2 or more bytes) become shorter when wrapped. The more UTF-8 chars a
    line has, the shorter.

    There are these settings in nntpserv.c:

    #define WRAP_WIDTH 72
    #define LINE_WIDTH 79
    #define MAX_WIDTH 997

    I tried to increase WRAP_WIDTH and LINE_WIDTH to something like 990
    once. It helped with quoting utf-8 lines, but for some reason I went
    back to originals.

    Anyway, the safest setting to avoid utf-8 quoting crap is to use "flowed
    off".

    'Tommi

    ---
    * Origin: LCC SG - Linux 6.1.21-v7+ armv7l (2:221/10)
  • From Wilfred van Velzen@2:280/464 to Nicholas Boel on Sun Feb 25 18:59:27 2024
    Hi Nicholas,

    On 2024-02-25 09:10:52, you wrote to me:

    How did my original reply to Carlos look on your end?

    I think this is the message you're referring to:
    https://postimg.cc/gallery/kq34nZw

    Yes, that's the one.. along with most of the others I wrote from
    Thunderbird. It seems Thunderbird adds non-breaking spaces (or
    &nbsp in html) anywhere there's two or more concurrent spaces next
    to each other.

    They are here on these quotes, but on none of the others...?

    Yes that was kind of a test. I backspaced all of the 2nd and 3rd level quotes to the newly created quote character except for one paragraph of quoted text to nail down the problem. Now if I understand all of this correctly, there shouldn't be any issues with the above quotes.

    Correct...


    Bye, Wilfred.

    --- FMail-lnx64 2.2.1.1
    * Origin: FMail development HQ (2:280/464)
  • From Wilfred van Velzen@2:280/464 to Nicholas Boel on Sun Feb 25 19:01:45 2024
    Hi Nicholas,

    On 2024-02-25 09:55:50, you wrote to Carlos Navarro:

    Have you tried adding '/flowed=off' to your username, or setting
    'def_flowed off' in your Jam/SmapiNNTPd config? That way, those
    spaces will not converted to NBSPs.

    This seems to work better in this regard.

    However, now obviously we're not using format=flowed text, so is it now hard wrapping the text that I'm currently writing?

    Isn't that supposed to be left alone (ie: no hard wrapping) so the next reader can deal with the message however it seems fit?

    There was no hard wrapping in the above qouted 1 lines by you...


    Bye, Wilfred.

    --- FMail-lnx64 2.2.1.1
    * Origin: FMail development HQ (2:280/464)
  • From Nicholas Boel@1:154/10 to Wilfred van Velzen on Sun Feb 25 12:39:48 2024
    On Mon, 26 Feb 2024 01:01:44 +0100, Wilfred Van Velzen -> Nicholas Boel wrote:

    There was no hard wrapping in the above qouted 1 lines by you...

    Confirmed in Golded. However, everything I write and see now in Thunderbird is wrapping at <80 columns, even though in Golded it shows otherwise.

    Oh well, at least the utf-8 "crap" as Tommi calls it is gone.

    Regards,
    Nick

    ... "Take my advice, I don't use it anyway."
    --- Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:115.0) Gecko/20100101 Thunderb
    * Origin: _thePharcyde distribution system (Wisconsin) (1:154/10)
  • From Tommi Koivula@2:221/1 to Nicholas Boel on Sun Feb 25 21:03:50 2024
    On 25.02.2024 20:39, Nicholas Boel wrote:

    There was no hard wrapping in the above qouted 1 lines by you...

    Confirmed in Golded. However, everything I write and see now in
    Thunderbird is wrapping at <80 columns, even though in Golded it shows otherwise.

    Thunderbird has also some wrap settings in its "about:config".

    Oh well, at least the utf-8 "crap" as Tommi calls it is gone.

    :)

    'Tommi

    ---
    * Origin: jamnntpd/lnx (2:221/1.0)
  • From Tommi Koivula@2:221/10 to Nicholas Boel on Sun Feb 25 21:17:10 2024
    From: Tommi Koivula <tommi@rbb.homeip.net>

    On 25.02.2024 20:50, Nicholas Boel wrote:

    Confirmed in Golded. However, everything I write and see now in
    Thunderbird is wrapping at <80 columns, even though in Golded it
    shows otherwise.

    I _think_ that the "flowed" setting does not affect when entering
    messages to msgbase. It's about the nntp client how it sends the text.

    'Tommi

    ---
    * Origin: LCC SG - Linux 6.1.21-v7+ armv7l (2:221/10)
  • From Nicholas Boel@1:154/10 to Tommi Koivula on Sun Feb 25 13:49:48 2024
    On Sun, 25 Feb 2024 21:17:10 GMT, Tommi Koivula -> Nicholas Boel wrote:

    I _think_ that the "flowed" setting does not affect when entering
    messages to msgbase. It's about the nntp client how it sends the text.

    Agreed. And after disabling that option, messages seem to come in and go out with "format=fixed" which seems to change how Thunderbird reacts to how it reads the messages, as well as posting replies to them.

    When it was "format=flowed" I could write the entire window width before it wrapped to the next line. Now, with format=fixed it wraps at whatever it is set to wrap long lines at (probably 75-78).

    Definitely not a game changer, but just weird when I've been used to the other way (long lines) for quite some time with most current writing applications, and even including Golded.

    Regards,
    Nick

    ... "Take my advice, I don't use it anyway."
    --- Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:115.0) Gecko/20100101 Thunderb
    * Origin: _thePharcyde distribution system (Wisconsin) (1:154/10)
  • From Nicholas Boel@1:154/10 to Wilfred van Velzen on Mon Feb 26 16:57:24 2024
    On Mon, 26 Feb 2024 00:59:26 +0100, Wilfred Van Velzen -> Nicholas Boel wrote:

    How did my original reply to Carlos look on your end?

    I think this is the message you're referring to:
    https://postimg.cc/gallery/kq34nZw

    Yes, that's the one.. along with most of the others I wrote from
    Thunderbird. It seems Thunderbird adds non-breaking spaces (or
    &nbsp in html) anywhere there's two or more concurrent spaces next
    to each other.

    They are here on these quotes, but on none of the others...?

    Yes that was kind of a test. I backspaced all of the 2nd and 3rd level
    quotes to the newly created quote character except for one paragraph of
    quoted text to nail down the problem. Now if I understand all of this
    correctly, there shouldn't be any issues with the above quotes.

    Here's testing Smapinntpd with 'def-flowed=on' and Thunderbird with 'mailnews.display.disable_format_flowed_support = true' and 'mailnews.send_plaintext_flowed = false'.

    If this doesn't work, and the only option is to disable it in Smapinntpd,
    I'll probably just avoid Thunderbird, then. Since it seems to work fine in other clients. :(

    Regards,
    Nick

    ... "Take my advice, I don't use it anyway."
    --- Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:115.0) Gecko/20100101 Thunderb
    * Origin: _thePharcyde distribution system (Wisconsin) (1:154/10)
  • From Nicholas Boel@1:154/10 to Tommi Koivula on Mon Feb 26 16:59:40 2024
    On Sun, 25 Feb 2024 21:17:10 GMT, Tommi Koivula -> Nicholas Boel wrote:

    I _think_ that the "flowed" setting does not affect when entering
    messages to msgbase. It's about the nntp client how it sends the text.
    I think I may have gotten it! The two settings regarding 'format-flowed' in Thunderbird need to be changed (see my previous message), and it seems to
    work with 'def-flowed=on' in Smapinntpd config!

    Regards,
    Nick

    ... "Take my advice, I don't use it anyway."
    --- Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:115.0) Gecko/20100101 Thunderb
    * Origin: _thePharcyde distribution system (Wisconsin) (1:154/10)
  • From Wilfred van Velzen@2:280/464 to Nicholas Boel on Tue Feb 27 12:04:19 2024
    Hi Nicholas,

    On 2024-02-26 16:57:24, you wrote to me:

    How did my original reply to Carlos look on your end?

    I think this is the message you're referring to:
    https://postimg.cc/gallery/kq34nZw

    Yes, that's the one.. along with most of the others I wrote from
    Thunderbird. It seems Thunderbird adds non-breaking spaces (or
    &nbsp in html) anywhere there's two or more concurrent spaces
    next to each other.

    They are here on these quotes, but on none of the others...?

    Yes that was kind of a test. I backspaced all of the 2nd and 3rd
    level quotes to the newly created quote character except for one
    paragraph of quoted text to nail down the problem. Now if I
    understand all of this correctly, there shouldn't be any issues with
    the above quotes.

    Here's testing Smapinntpd with 'def-flowed=on' and Thunderbird with 'mailnews.display.disable_format_flowed_support = true' and 'mailnews.send_plaintext_flowed = false'.

    If this doesn't work, and the only option is to disable it in Smapinntpd, I'll probably just avoid Thunderbird, then. Since it seems to work fine in other clients. :(

    Your lines were wrapped as seen above in the quotes. And there were no odd utf-8 characters in front of any of the quotes...

    Bye, Wilfred.

    --- FMail-lnx64 2.2.1.1
    * Origin: FMail development HQ (2:280/464)
  • From Tommi Koivula@2:221/1 to Nicholas Boel on Tue Feb 27 20:17:16 2024

    Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit

    On Sun, 25 Feb 2024 21:17:10 GMT, Tommi Koivula -> Nicholas Boel wrote:

    TK> I _think_ that the "flowed" setting does not affect when entering
    TK> messages to msgbase. It's about the nntp client how it sends the text. I think I may have gotten it! The two settings regarding 'format-flowed' in Thunderbird need to be changed (see my previous message), and it seems to work with 'def-flowed=on' in Smapinntpd config!

    Ok.. Lets test, I just applied those two settings to this TB and logged
    in with tommi/flowed=on.


    Regards,
    Nick

    ... "Take my advice, I don't use it anyway."
    --- Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:115.0) Gecko/20100101 Thunderb
    * Origin: _thePharcyde distribution system (Wisconsin) (1:154/10)

    'Tommi

    ---
    * Origin: jamnntpd/lnx (2:221/1.0)
  • From Nicholas Boel@1:154/10 to Wilfred van Velzen on Tue Feb 27 17:32:58 2024
    On Tue, 27 Feb 2024 18:04:18 +0100, Wilfred Van Velzen -> Nicholas Boel wrote:

    Your lines were wrapped as seen above in the quotes. And there were no odd utf-8 characters in front of any of the quotes...

    Ayyyyy! That means I'm on to something! ;)

    I couldn't bring myself to change the server configuration when most other clients work fine. The change must be done in Thunderbird, in my opinion.

    Regards,
    Nick

    ... "Take my advice, I don't use it anyway."
    --- Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:115.0) Gecko/20100101 Thunderb
    * Origin: _thePharcyde distribution system (Wisconsin) (1:154/10)
  • From Nicholas Boel@1:154/10 to Tommi Koivula on Tue Feb 27 17:37:54 2024
    On Wed, 28 Feb 2024 02:17:16 +0200, Tommi Koivula -> Nicholas Boel wrote:

    I think I may have gotten it! The two settings regarding 'format-flowed' in
    Thunderbird need to be changed (see my previous message), and it seems to
    work with 'def-flowed=on' in Smapinntpd config!

    Ok.. Lets test, I just applied those two settings to this TB and logged
    in with tommi/flowed=on.

    Doesn't look bad at all, and I don't see any utf-8 characters before the quotes.

    However, I wonder why your client is wrapping the end of a couple of the
    lines with no quote mark before it? One of them was the tearline.
    "Thunderb" was wrapped to the next line with no quote character before it.

    Simple solution is to just not quote the tearline. But maybe Jam/Smapinntpd doesn't take the tearline into consideration when quoting..?

    Regards,
    Nick

    ... "Take my advice, I don't use it anyway."
    --- Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:115.0) Gecko/20100101 Thunderb
    * Origin: _thePharcyde distribution system (Wisconsin) (1:154/10)
  • From Wilfred van Velzen@2:280/464 to Nicholas Boel on Wed Feb 28 08:47:17 2024
    Hi Nicholas,

    On 2024-02-27 17:37:54, you wrote to Tommi Koivula:

    However, I wonder why your client is wrapping the end of a couple of
    the lines with no quote mark before it? One of them was the tearline. "Thunderb" was wrapped to the next line with no quote character before
    it.

    I think the wrapping is happening on your end. Because Tommi's message looked fine here: no wrapping of quoted lines...


    Bye, Wilfred.

    --- FMail-lnx64 2.2.1.1
    * Origin: FMail development HQ (2:280/464)
  • From Nicholas Boel@1:154/10 to Wilfred van Velzen on Wed Feb 28 19:01:26 2024
    However, I wonder why your client is wrapping the end of a couple of
    the lines with no quote mark before it? One of them was the tearline.
    "Thunderb" was wrapped to the next line with no quote character before
    it.

    I think the wrapping is happening on your end. Because Tommi's message looked fine here: no wrapping of quoted lines...

    Most likely. It also displays like that in slrn so I'm thinking it has something to do with those line length variables set in jam/smapinntpd's sources. Time for some adjustments! :)

    Regards,
    Nick

    ... Take my advice, I don't use it anyway.
    --- slrn/pre1.0.4-9 (Linux)
    * Origin: _thePharcyde distribution system (Wisconsin) (1:154/10)
  • From Tommi Koivula@2:221/1 to Wilfred van Velzen on Fri Mar 1 08:26:08 2024
    On 28.02.2024 9:47, Wilfred van Velzen wrote:


    On 2024-02-27 17:37:54, you wrote to Tommi Koivula:

    NB> However, I wonder why your client is wrapping the end of a couple of
    NB> the lines with no quote mark before it? One of them was the tearline.
    NB> "Thunderb" was wrapped to the next line with no quote character before
    NB> it.

    I think the wrapping is happening on your end. Because Tommi's
    message looked fine here: no wrapping of quoted lines...

    This is pretty strange. If I write a long line, it shows ok in Golded but not in TB.

    Very good example is HodDogEd, which in my phone wraps to ~70 chars. Some lines are wrapped twice, at ~70 and at ~79. :)

    'Tommi

    ...#define WRAP_WIDTH 972

    ---
    * Origin: jamnntpd/lnx (2:221/1.0)
  • From Tommi Koivula@2:221/1 to Carlos Navarro on Fri Mar 1 08:51:38 2024
    On 25.02.2024 13:25, Carlos Navarro wrote:


    problem with jamnntpd is that it does not support multibyte,

    You're right about JamNNTPd not supporting multibyte. It wraps lines
    at 79 bytes, not at 79 chars. Therefore those that have UTF-8 chars
    (2 or more bytes) become shorter when wrapped. The more UTF-8 chars a
    line has, the shorter.

    Actually by default Jamnntpd wraps to 72, not 79, but that is irrelevant. :) As you said, the problem with utf-8 text is that Jamnntpd counts bytes, not chars.

    I've set "#define WRAP_WIDTH 972" now in this Jamnntpd. I can re-wrap quoted text with Ctrl-R in TB. Seems nice so far.

    'Tommi

    ...0123456789 0123456789 0123456789 0123456789 0123456789 0123456789 0123456789 0123456789 0123456789 0123456789

    'Tommi

    ---
    * Origin: jamnntpd/lnx (2:221/1.0)
  • From Tommi Koivula@2:221/1 to Tommi Koivula on Fri Mar 1 08:57:22 2024
    Hello, Tommi Koivula.
    On 01/03/2024 8.51 you wrote:

    On 25.02.2024 13:25, Carlos Navarro wrote:
    problem with jamnntpd is that it does not support multibyte,
    You're right about JamNNTPd not supporting multibyte. It wraps lines
    at 79 bytes, not at 79 chars. Therefore those that have UTF-8 chars
    (2 or more bytes) become shorter when wrapped. The more UTF-8 chars a
    line has, the shorter.
    Actually by default Jamnntpd wraps to 72, not 79, but that is irrelevant. :) As you said, the problem with utf-8 text is that Jamnntpd counts bytes, not chars.
    I've set "#define WRAP_WIDTH 972" now in this Jamnntpd. I can re-wrap quoted text with Ctrl-R in TB. Seems nice so far.
    'Tommi
    ...0123456789 0123456789 0123456789 0123456789 0123456789 0123456789 0123456789 0123456789 0123456789 0123456789
    'Tommi

    http://pics.rsh.ru/img/Screens5769_saapazgl.jpg


    Looks ok now in Hotdoged.

    ---
    * Origin: jamnntpd/lnx (2:221/1.0)
  • From Nicholas Boel@1:154/10 to Tommi Koivula on Fri Mar 1 19:47:06 2024
    On Fri, 1 Mar 2024 14:57:22 +0200, Tommi Koivula -> Tommi Koivula wrote:

    Hello, Tommi Koivula.
    On 01/03/2024 8.51 you wrote:

    On 25.02.2024 13:25, Carlos Navarro wrote:
    problem with jamnntpd is that it does not support multibyte,
    You're right about JamNNTPd not supporting multibyte. It wraps lines
    at 79 bytes, not at 79 chars. Therefore those that have UTF-8 chars
    (2 or more bytes) become shorter when wrapped. The more UTF-8 chars a
    line has, the shorter.
    Actually by default Jamnntpd wraps to 72, not 79, but that is
    irrelevant. :) As you said, the problem with utf-8 text is that Jamnntpd counts bytes, not chars.
    I've set "#define WRAP_WIDTH 972" now in this Jamnntpd. I can
    re-wrap quoted text with Ctrl-R in TB. Seems nice so far.
    'Tommi
    ...0123456789 0123456789 0123456789 0123456789 0123456789
    0123456789 0123456789 0123456789 0123456789 0123456789
    'Tommi

    http://pics.rsh.ru/img/Screens5769_saapazgl.jpg


    Looks ok now in Hotdoged.


    Definitely doesn't look ok quoted back by hotdoged though. In TB or
    Golded. :)

    Regards,
    Nick

    ... "Take my advice, I don't use it anyway."
    --- Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:115.0) Gecko/20100101 Thunderb
    * Origin: _thePharcyde distribution system (Wisconsin) (1:154/10)
  • From Nicholas Boel@1:154/10 to Tommi Koivula on Fri Mar 1 19:58:16 2024
    On Fri, 1 Mar 2024 14:57:22 +0200, Tommi Koivula -> Tommi Koivula wrote:

    Looks ok now in Hotdoged.

    From what I've gathered, WRAP_WIDTH is your quotes. LINE_WIDTH is what
    you write.

    You probably don't want to mess with stuff you've quoted, as that has to
    be viewable in a 80 column terminal (BBSes, for example).

    LINE_WIDTH may need to be adjusted to 72, instead of 79. So when quoting
    79 characters it won't wrap the last 7 characters on to the next line.

    I'll give it a whirl. :)

    Regards,
    Nick

    ... "Take my advice, I don't use it anyway."
    --- Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:115.0) Gecko/20100101 Thunderb
    * Origin: _thePharcyde distribution system (Wisconsin) (1:154/10)
  • From Tommi Koivula@2:221/360 to Nicholas Boel on Sat Mar 2 07:55:17 2024
    Nicholas Boel wrote:

    http://pics.rsh.ru/img/Screens5769_saapazgl.jpg

    Looks ok now in Hotdoged.

    Definitely doesn't look ok quoted back by hotdoged though. In TB or Golded. :)

    Indeed! :D

    But that's the problem of the author how the written text is formatted. Hotdoded is not the best tool for editing messages. :)

    'Tommi

    ---
    * Origin: nntp://rbb.fidonet.fi - Finland (2:221/360.0)
  • From Tommi Koivula@2:221/6 to Nicholas Boel on Sat Mar 2 08:01:43 2024
    On 02.03.2024 3:58, Nicholas Boel wrote:

    From what I've gathered, WRAP_WIDTH is your quotes. LINE_WIDTH is what
    you write.

    You probably don't want to mess with stuff you've quoted, as that has to
    be viewable in a 80 column terminal (BBSes, for example).

    LINE_WIDTH may need to be adjusted to 72, instead of 79. So when quoting
    79 characters it won't wrap the last 7 characters on to the next line.

    These three settings needs a bit more studying of the code.. ;)

    #define WRAP_WIDTH 72
    #define LINE_WIDTH 79
    #define MAX_WIDTH 997

    'Tommi

    --- FMail-lnx32 2.2.1.1
    * Origin: jamnntpd/lnx (2:221/6.0)
  • From Carlos Navarro@2:341/234.1 to Tommi Koivula on Sat Mar 2 09:51:34 2024
    01 Mar 2024 08:51, you wrote to me:

    Actually by default Jamnntpd wraps to 72, not 79, but that is
    irrelevant. :)

    True. O:-)

    I've found that JamNNTPd wraps at 72 chars in all lines except the last one, that can be up to 79 chars.

    This paragraph:

    1 3 5 7 9 1 3 5 7 9 1 3 5 7 9 1 3 5 7 9 1 3 5 7 9 1 3 5 7 9 1 3 5 7 9 1 3 5 7 9 1 3 5 7 9 1 3 5 7 9 1 3 5 7 9 1 3 5 7 9 1 3 5 7 9 1 3 5 7 9 1 3 5 7 9 1 3 5 7 9 1 3 5 7 9 1 3 5 7 9 1 3 5 7 9 1 3 5 7 9 1 3 5 7 9 1 3 5 7 9 1

    will be displayed like this in a client that doesn't support flowed text:

    1 3 5 7 9 1 3 5 7 9 1 3 5 7 9 1 3 5 7 9 1 3 5 7 9 1 3 5 7 9 1 3 5 7 9 1
    3 5 7 9 1 3 5 7 9 1 3 5 7 9 1 3 5 7 9 1 3 5 7 9 1 3 5 7 9 1 3 5 7 9 1 3
    5 7 9 1 3 5 7 9 1 3 5 7 9 1 3 5 7 9 1 3 5 7 9 1 3 5 7 9 1 3 5 7 9 1 3 5 7 9 1.

    IMO it doesn't make sense (unless I'm missing something).

    To fix this, I suppose that you can either set LINE_WIDTH to 72 or make other changes to the code. Will check later.

    Carlos

    --- GoldED+/W32-MSVC 1.1.5-b20180707
    * Origin: cyberiada (2:341/234.1)
  • From Tommi Koivula@2:221/1 to Carlos Navarro on Sat Mar 2 13:25:38 2024
    On 02.03.2024 10:51, Carlos Navarro wrote:


    TK> Actually by default Jamnntpd wraps to 72, not 79, but that is
    TK> irrelevant. :)

    True. O:-)

    I've found that JamNNTPd wraps at 72 chars in all lines except the last one, that can be up to 79 chars.

    Yes, and it seems that it always wraps the text it shows to the client. Defined by WRAP_WIDTH in nntpserv.c.

    There is only one WRAP_WIDTH in the code:

    /* Find last space before WRAP_WIDTH */

    while(c<=WRAP_WIDTH && text[textpos+c]!=0 && text[textpos+c]!=13)
    {
    if(text[textpos+c]==32) lastspace=c;
    c++;
    }

    So it doesn't care about the FLOWED setting at all when showing the message...

    'Tommi

    ..wrap.. ..wrap.. ..wrap.. ..wrap.. ..wrap.. ..wrap.. ..wrap.. ..wrap.. ..wrap.. ..wrap.. ..wrap.. ..wrap.. ..wrap.. ..wrap.. ..wrap.. ..wrap.. ..wrap.. ..wrap.. ..wrap.. ..wrap.. ..wrap.. ..wrap.. ..wrap.. ..wrap..

    ---
    * Origin: jamnntpd/lnx (2:221/1.0)
  • From Nicholas Boel@1:154/10 to Tommi Koivula on Sat Mar 2 07:16:12 2024
    On Sat, 2 Mar 2024 13:55:16 +0200, Tommi Koivula -> Nicholas Boel wrote:

    But that's the problem of the author how the written text is formatted. Hotdoded is not the best tool for editing messages. :)

    While that may be one case, in Golded the quoted text you sent off went
    the full width of my screen. So I'm not sure any client would know how
    to handle that seeing as though jam/smapinntpd is the one doing the quoting.

    Now I'm testing WRAP_WIDTH = 72 as well as LINE_WIDTH = 72. But I have a feeling my long tearline will wrap early now, which I can't see until I
    post this message.

    Regards,
    Nick

    ... "Take my advice, I don't use it anyway."
    --- Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:115.0) Gecko/20100101 Thunderb
    * Origin: _thePharcyde distribution system (Wisconsin) (1:154/10)
  • From Nicholas Boel@1:154/10 to Nicholas Boel on Sat Mar 2 07:18:46 2024
    Hello Accession,

    On Saturday March 02 2024 07:16, I wrote to Tommi Koivula:

    Now I'm testing WRAP_WIDTH = 72 as well as LINE_WIDTH = 72. But I have
    a feeling my long tearline will wrap early now, which I can't see
    until I post this message.

    Regards,
    Nick

    ... "Take my advice, I don't use it anyway."
    --- Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:115.0) Gecko/20100101 Thunderb
    * Origin: _thePharcyde distribution system (Wisconsin) (1:154/10)

    While it displays OK in Golded when viewed for the first time, once I quote it it's on the next line (which actually with the initials added is probably the proper way to handle it since it's 79 characters *before* being quoted). Though now at least it has the quote initials before it, whereas before it would wrap it to the next line and wouldn't put quote characters before it.

    The only downfall to this is now it will wrap a quoted line 73-79 characters long. But, it seems it will at least quote it properly now.

    Regards,
    Nick

    ... "Take my advice, I don't use it anyway."
    --- GoldED+/LNX 1.1.5-b20231112
    * Origin: _thePharcyde distribution system (Wisconsin) (1:154/10)
  • From Nicholas Boel@1:154/10 to Nicholas Boel on Sat Mar 2 07:31:40 2024
    On Sat, 2 Mar 2024 13:16:12 -0600, Nicholas Boel -> Tommi Koivula wrote:

    Now I'm testing WRAP_WIDTH = 72 as well as LINE_WIDTH = 72. But I have a feeling my long tearline will wrap early now, which I can't see until I post this message.

    Back to the defaults again. I have a feeling it's more noticeable while
    using '-smartquote' also. Since you're adding like 5+ characters to the
    line, and it doesn't seem like those extra characters added are taken
    into account in the WRAP_WIDTH or LINE_WIDTH checks.

    Regards,
    Nick

    ... "Take my advice, I don't use it anyway."
    --- Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:115.0) Gecko/20100101 Thunderb
    * Origin: _thePharcyde distribution system (Wisconsin) (1:154/10)
  • From Carlos Navarro@2:341/234.1 to Tommi Koivula on Sun Mar 31 22:14:54 2024
    02 Mar 2024 13:25, you wrote to me:

    I've found that JamNNTPd wraps at 72 chars in all lines except the
    last one, that can be up to 79 chars.

    Yes, and it seems that it always wraps the text it shows to the
    client. Defined by WRAP_WIDTH in nntpserv.c.

    JamNNTPd always sends wrapped text. If flowed=on, it will append a space to the end of each wrapped line.
    It's up to the client (if it supports format=flowed) to re-join those lines and show it as a paragraph.

    I experimented with setting WRAP_WIDTH and LINE_WIDTH to a high value, like 997. This way JamNNTPd works similarly to other NNTP servers that support paragraphs as very long lines.

    But there's a problem with quoting, as you commented. Some readers (TB, Sylpheed...) will quote a long line as just one quoted long line, without wrapping. And this doesn't work well with all FTN software. Some (GoldED, Xpoint, WinPoint, Aftershock...) can display and re-quote those long quotes just fine, but many other programs can't.

    Carlos

    --- GoldED+/W32-MSVC 1.1.5-b20180707
    * Origin: cyberiada (2:341/234.1)
  • From Tommi Koivula@2:221/10 to Carlos Navarro on Mon Apr 1 08:48:00 2024
    From: Tommi Koivula <tommi@rbb.homeip.net>

    On 31.03.2024 23:20, Carlos Navarro wrote:


    I've found that JamNNTPd wraps at 72 chars in all lines except
    the last one, that can be up to 79 chars.

    Yes, and it seems that it always wraps the text it shows to the
    client. Defined by WRAP_WIDTH in nntpserv.c.

    JamNNTPd always sends wrapped text. If flowed=on, it will append a
    space to the end of each wrapped line. It's up to the client (if it
    supports format=flowed) to re-join those lines and show it as a
    paragraph.

    I did some research between Jamnntpd and a "real" nntp server, like this WendzelNNTPd, which I'm writing now from.

    Jamnntpd with "flowed on": Flowed text in TB does not work ok when
    resizing window.

    WendzelNNTPd : Flowed text work ok when resizing window.

    However, in this server the Content-Type line has no "format=" part at all:

    Content-Type: text/plain; charset="iso-8859-1"

    So, IMHO the "hard wrap" setting WRAP_WIDTH 72 is not working as it
    should with "flowed on". I think Jamnntpd should adjust the WRAP_WIDTH
    setting according to the "flowed" setting and not wrap lines with
    "flowed on".

    'Tommi

    ---
    * Origin: LCC SG - Linux 6.1.21-v7+ armv7l (2:221/10)
  • From Nicholas Boel@1:154/10 to Tommi Koivula on Tue Apr 2 19:48:16 2024
    Hello Tommi,

    On Mon, Apr 01 2024 08:48:00 +0000, you wrote:

    However, in this server the Content-Type line has no "format=" part at all:

    Content-Type: text/plain; charset="iso-8859-1"

    I don't think this matters much.

    So, IMHO the "hard wrap" setting WRAP_WIDTH 72 is not working as it
    should with "flowed on". I think Jamnntpd should adjust the WRAP_WIDTH setting according to the "flowed" setting and not wrap lines with
    "flowed on".

    If I'm understanding you correctly, WRAP_WIDTH should remain 72 only when 'def_flowed=off', and when 'def_flowed=on' it should be disabled completely? If that's the case, I agree.

    I get the two variables mixed up. I thought LINE_WIDTH is what is causing the wrap by setting a maximum number of columns per line. Then I thought WRAP_WIDTH was where it should be looking for a space to wrap at prior to the last word being typed once it hits 79 columns.

    However, I don't see any of those variables being used in the "format=flowed" section, but since there was no check for flowed on/off *until* the format=flowed section, it seems to be doing the wrapping either way.

    Regards,
    Nick

    ... Take my advice, I don't use it anyway.
    --- slrn/pre1.0.4-9 (Linux)
    * Origin: _thePharcyde distribution system (Wisconsin) (1:154/10)
  • From Carlos Navarro@2:341/234.1 to Nicholas Boel on Wed Apr 3 11:02:01 2024
    02 Apr 2024 19:48, you wrote to Tommi Koivula:

    So, IMHO the "hard wrap" setting WRAP_WIDTH 72 is not working as
    it should with "flowed on". I think Jamnntpd should adjust the
    WRAP_WIDTH setting according to the "flowed" setting and not wrap
    lines with "flowed on".

    If I'm understanding you correctly, WRAP_WIDTH should remain 72 only
    when 'def_flowed=off', and when 'def_flowed=on' it should be disabled completely? If that's the case, I agree.

    In format=flowed, long lines are wrapped but they have a trailing space. The client, if it supports this format, joins the lines, showing them as a paragraph.

    From RFC-2646:
    ---------------------------------
    Generating Format=Flowed

    When generating Format=Flowed text, lines SHOULD be shorter than 80
    characters. As suggested values, any paragraph longer than 79
    characters in total length could be wrapped using lines of 72 or
    fewer characters. While the specific line length used is a matter of aesthetics and preference, longer lines are more likely to require
    rewrapping and to encounter difficulties with older mailers. It has
    been suggested that 66 character lines are the most readable.

    (The reason for the restriction to 79 or fewer characters between
    CRLFs on the wire is to ensure that all lines, even when displayed by
    a non-flowed-aware program, will fit in a standard 80-column screen
    without having to be wrapped. The limit is 79, not 80, because while
    80 fit on a line, the last column is often reserved for a line-wrap
    indicator.)

    When creating flowed text, the generating agent wraps, that is,
    inserts 'soft' line breaks as needed. Soft line breaks are added
    between words. Because a soft line break is a SP CRLF sequence, the
    generating agent creates one by inserting a CRLF after the occurance
    of a space.
    ---------------------------------

    Carlos

    --- GoldED+/W32-MSVC 1.1.5-b20180707
    * Origin: cyberiada (2:341/234.1)
  • From Tommi Koivula@2:221/1 to Carlos Navarro on Wed Apr 3 21:16:30 2024
    On 4/3/24 12:02, Carlos Navarro wrote:

    In format=flowed, long lines are wrapped but they have a trailing space.
    The client, if it supports this format, joins the lines, showing them as
    a paragraph.

    Yep.

    But for some reason TB doesn't act like this when using jam/smapinntpd.

    'Tommi

    ...(Flowed on set now)

    ---
    * Origin: jamnntpd/lnx (2:221/1.0)
  • From Nicholas Boel@1:154/10 to Carlos Navarro on Wed Apr 3 17:38:18 2024
    Hello Carlos,

    On Wed, Apr 03 2024 14:02:00 +0000, you wrote:

    If I'm understanding you correctly, WRAP_WIDTH should remain 72 only
    when 'def_flowed=off', and when 'def_flowed=on' it should be disabled
    completely? If that's the case, I agree.

    In format=flowed, long lines are wrapped but they have a trailing space. The client, if it supports this format, joins the lines, showing them as
    a paragraph.

    I understand this. However, what Tommi and I are discussing is that jamnntpd/smapinntpd is not doing this properly. I'm not sure 'WRAP_WIDTH 72' is very constant, as I see wrapping all over the place between 72-79 characters (which sometimes even wraps a word to the next line and doesn't include a quote character before it). So either the 'WRAP_WIDTH 72' and 'LINE_WIDTH 79' settings are messing with each other, or they're not implemented properly.

    So, in my questions to Tommi, I'm more or less asking if anyone has actually figured out what WRAP_WIDTH is actually doing. It is only checked for once in the code, and so is LINE_WIDTH. Neither of them are checked for in the format=flowed section. It almost seems like the format=flowed section was an afterthought, and was added in after what it was already doing. *shrug*

    Regards,
    Nick

    ... Take my advice, I don't use it anyway.
    --- slrn/pre1.0.4-9 (Linux)
    * Origin: _thePharcyde distribution system (Wisconsin) (1:154/10)
  • From Nicholas Boel@1:154/10 to Tommi Koivula on Wed Apr 3 17:38:58 2024
    Hello Tommi,

    On Wed, Apr 03 2024 23:16:30 +0000, you wrote:

    In format=flowed, long lines are wrapped but they have a trailing space.
    The client, if it supports this format, joins the lines, showing them as
    a paragraph.

    Yep.

    But for some reason TB doesn't act like this when using jam/smapinntpd.

    Agreed. Something doesn't seem right.

    Regards,
    Nick

    ... Take my advice, I don't use it anyway.
    --- slrn/pre1.0.4-9 (Linux)
    * Origin: _thePharcyde distribution system (Wisconsin) (1:154/10)
  • From Carlos Navarro@2:341/234.1 to Tommi Koivula on Thu Apr 4 09:57:31 2024
    03 Apr 2024 21:16, you wrote to me:

    In format=flowed, long lines are wrapped but they have a trailing
    space. The client, if it supports this format, joins the lines,
    showing them as a paragraph.

    Yep.

    But for some reason TB doesn't act like this when using
    jam/smapinntpd.

    Works fine for me:

    https://www.cyberiada.org/fido/tmpfiles/flowed-wide-window.jpg

    https://www.cyberiada.org/fido/tmpfiles/flowed-narrow-window.jpg

    Carlos

    --- GoldED+/W32-MSVC 1.1.5-b20180707
    * Origin: cyberiada (2:341/234.1)
  • From Carlos Navarro@2:341/234.1 to Nicholas Boel on Thu Apr 4 11:19:15 2024
    03 Apr 2024 17:38, you wrote to me:

    In format=flowed, long lines are wrapped but they have a trailing
    space. The client, if it supports this format, joins the lines,
    showing them as a paragraph.

    I understand this. However, what Tommi and I are discussing is that jamnntpd/smapinntpd is not doing this properly. I'm not sure
    'WRAP_WIDTH 72' is very constant, as I see wrapping all over the place between 72-79 characters (which sometimes even wraps a word to the
    next line and doesn't include a quote character before it). So either
    the 'WRAP_WIDTH 72' and 'LINE_WIDTH 79' settings are messing with each other, or they're not implemented properly.

    When JamNNTPd sends to the client,
    - lines that are 79 or less are not wrapped
    - lines longer than 79 are wrapped by 72

    Strange as it may seem, that's how format=flowed is defined in the RFC.

    Nevertheless, those are the suggested values (that JamNNTPd uses). Other numbers could be used instead, as long as they are 78 or less.

    BTW, JamNNTPd uses these same values also for format=fixed. Not a big issue IMO, but it could be changed so that it uses a different value in this case.

    So, in my questions to Tommi, I'm more or less asking if anyone has actually figured out what WRAP_WIDTH is actually doing. It is only
    checked for once in the code, and so is LINE_WIDTH. Neither of them
    are checked for in the format=flowed section. It almost seems like the format=flowed section was an afterthought, and was added in after what
    it was already doing. *shrug*

    The code block for format=flowed appends spaces to the end of wrapped lines and does the 'space stuffing' at the beginning (inserts extra space if it begins with space), as defined in the RFC.

    Carlos

    --- GoldED+/W32-MSVC 1.1.5-b20180707
    * Origin: cyberiada (2:341/234.1)
  • From Tommi Koivula@2:221/1 to Carlos Navarro on Thu Apr 4 17:53:18 2024
    On 4/4/24 07:57, Carlos Navarro wrote:
    03 Apr 2024 21:16, you wrote to me:

    >> In format=flowed, long lines are wrapped but they have a trailing
    >> space. The client, if it supports this format, joins the lines,
    >> showing them as a paragraph.

    TK> Yep.

    TK> But for some reason TB doesn't act like this when using
    TK> jam/smapinntpd.

    Works fine for me:

    https://www.cyberiada.org/fido/tmpfiles/flowed-wide-window.jpg

    https://www.cyberiada.org/fido/tmpfiles/flowed-narrow-window.jpg

    Very interesting...

    'Tommi

    ---
    * Origin: smapinntpd/lnx (2:221/1.0)
  • From Tommi Koivula@2:221/1 to Carlos Navarro on Thu Apr 4 18:50:50 2024
    On 4/4/24 14:53, Tommi Koivula wrote:
    On 4/4/24 07:57, Carlos Navarro wrote:
    03 Apr 2024 21:16, you wrote to me:

      >> In format=flowed, long lines are wrapped but they have a trailing
      >> space. The client, if it supports this format, joins the lines,
      >> showing them as a paragraph.

      TK> Yep.

      TK> But for some reason TB doesn't act like this when using
      TK> jam/smapinntpd.

    Works fine for me:

    https://www.cyberiada.org/fido/tmpfiles/flowed-wide-window.jpg

    https://www.cyberiada.org/fido/tmpfiles/flowed-narrow-window.jpg

    Very interesting...

    I stand corrected. After resetting all wrap and flow custom settings in TB, text seems to flow just fine. :)

    'Tommi

    ---
    * Origin: smapinntpd/lnx (2:221/1.0)
  • From Nicholas Boel@1:154/10 to Carlos Navarro on Thu Apr 4 17:29:22 2024
    Hello Carlos,

    On Thu, Apr 04 2024 14:19:14 +0000, you wrote:

    When JamNNTPd sends to the client,
    - lines that are 79 or less are not wrapped
    - lines longer than 79 are wrapped by 72

    Strange as it may seem, that's how format=flowed is defined in the RFC.

    Nevertheless, those are the suggested values (that JamNNTPd uses). Other numbers could be used instead, as long as they are 78 or less.

    Ok. So it's doing what it should be doing, then?

    The code block for format=flowed appends spaces to the end of wrapped lines and does the 'space stuffing' at the beginning (inserts extra
    space if it begins with space), as defined in the RFC.

    Why would you want to stuff spaces at the beginning of a line if it starts with a space? What is the reasoning behind that?

    I think the problem comes in to play with this example:

    Quoting a 78 (78 is just an example, it could probably be anywhere from 74 or 75 or more characters) character line with smartquote will not reformat the line back to 78 characters (or within the 79 line width limit it should be at). It will then append the space, initials, quote character, and another space.. now making it 4 or 5 characters longer. This will then wrap the last word to the next line and it won't have a quote character in front of it.

    I would imagine it would also do this without smartquote enabled, however it wouldn't add as many characters to the line, so you would see the issue less (only when a line hits 77 or 78 characters probably).

    Golded seems to handle this, but Golded also seems to support displaying quoted lines longer than 79 characters, which may very well be because of the issue(s) above.

    Regards,
    Nick

    ... Take my advice, I don't use it anyway.
    --- slrn/pre1.0.4-9 (Linux)
    * Origin: _thePharcyde distribution system (Wisconsin) (1:154/10)
  • From Nicholas Boel@1:154/10 to Tommi Koivula on Thu Apr 4 17:32:16 2024
    Hello Tommi,

    On Thu, Apr 04 2024 20:50:50 +0000, you wrote:

    I stand corrected. After resetting all wrap and flow custom settings in TB, text seems to flow just fine. :)

    Except, now you've added the non-breaking spaces back in.

    Seems like a serious bug in Thunderbird, that nobody seems to want to address.

    Regards,
    Nick

    ... Take my advice, I don't use it anyway.
    --- slrn/pre1.0.4-9 (Linux)
    * Origin: _thePharcyde distribution system (Wisconsin) (1:154/10)
  • From Nicholas Boel@1:154/10 to Tommi Koivula on Thu Apr 4 17:46:00 2024
    Hello Tommi,

    On Thu, 04 Apr 2024 18:50:50 +0300, you wrote to Carlos Navarro:

      TK> Yep.

      TK> But for some reason TB doesn't act like this when using
      TK> jam/smapinntpd.

    Above is where the non-breaking spaces were present. I don't see them here in Golded, or in nano. But they are both setup for reading utf-8. Let's see if Golded passes them on.

    Regards,
    Nick

    ... Take my advice, I don't use it anyway.
    --- GoldED+/LNX 1.1.5-b20240309
    * Origin: _thePharcyde distribution system (Wisconsin) (1:154/10)
  • From Nicholas Boel@1:154/10 to Nicholas Boel on Thu Apr 4 17:52:52 2024
    On Thu, 4 Apr 2024 22:32:16 -0500, you wrote:

    Hello Tommi,

    On Thu, Apr 04 2024 20:50:50 +0000, you wrote:

    I stand corrected. After resetting all wrap and flow custom
    settings in
    TB, text seems to flow just fine. :)

    This is kind of what I mean in my previous post to Carlos. That paragraph should have been reflowed if the quoted line wraps to the next. Unless this is something TB is doing as well, since it was fine before I quoted it. :/

    Regards,
    Nick

    ... Take my advice, I don't use it anyway.
    --- Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:115.0) Gecko/20100101 Thunderb
    * Origin: _thePharcyde distribution system (Wisconsin) (1:154/10)
  • From Carlos Navarro@2:341/234.1 to Nicholas Boel on Sun Apr 7 10:36:11 2024
    04 Apr 2024 17:29, you wrote to me:

    Nevertheless, those are the suggested values (that JamNNTPd
    uses). Other numbers could be used instead, as long as they are
    78 or less.

    Ok. So it's doing what it should be doing, then?

    Yes.

    The code block for format=flowed appends spaces to the end of
    wrapped lines and does the 'space stuffing' at the beginning
    (inserts extra space if it begins with space), as defined in the
    RFC.

    Why would you want to stuff spaces at the beginning of a line if it
    starts with a space? What is the reasoning behind that?

    It seems it's because of this:

    -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
    Space-Stuffing

    In order to allow for unquoted lines which start with ">", and to
    protect against systems which "From-munge" in-transit messages
    (modifying any line which starts with "From " to ">From "),
    Format=Flowed provides for space-stuffing.
    [...]
    -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=

    I think the problem comes in to play with this example:

    Quoting a 78 (78 is just an example, it could probably be anywhere
    from 74 or 75 or more characters) character line with smartquote will
    not reformat the line back to 78 characters (or within the 79 line
    width limit it should be at). It will then append the space, initials, quote character, and another space.. now making it 4 or 5 characters longer. This will then wrap the last word to the next line and it
    won't have a quote character in front of it.

    I would imagine it would also do this without smartquote enabled,
    however it wouldn't add as many characters to the line, so you would
    see the issue less (only when a line hits 77 or 78 characters
    probably).

    That's a different issue....

    I intend to experiment a bit with Thunderbird's mailnews.wraplength setting. Maybe a value higher than the default (72) would work better with Fidonet (but there may be other problems...)

    Golded seems to handle this, but Golded also seems to support
    displaying quoted lines longer than 79 characters, which may very well
    be because of the issue(s) above.

    GoldED is great with handling all types of quotes. I'm afraid we cannot expect all newsreaders to work fine with FTN-style quotes (AFAIK only HotdogEd does).

    Carlos

    --- GoldED+/W32-MSVC 1.1.5-b20180707
    * Origin: cyberiada (2:341/234.1)
  • From Carlos Navarro@2:341/234.99 to Tommi Koivula on Sun Apr 7 20:42:43 2024
    04/04/2024 17:50, Tommi Koivula -> Carlos Navarro:
    On 4/4/24 14:53, Tommi Koivula wrote:
    On 4/4/24 07:57, Carlos Navarro wrote:
    03 Apr 2024 21:16, you wrote to me:

    In format=flowed, long lines are wrapped but they have a trailing
    space. The client, if it supports this format, joins the lines,
    showing them as a paragraph.

    Yep.

    But for some reason TB doesn't act like this when using
    jam/smapinntpd.

    Works fine for me:

    https://www.cyberiada.org/fido/tmpfiles/flowed-wide-window.jpg

    https://www.cyberiada.org/fido/tmpfiles/flowed-narrow-window.jpg

    Very interesting...

    I stand corrected. After resetting all wrap and flow custom settings in TB,
    text seems to flow just fine. :)

    'Tommi

    ---
    * Origin: smapinntpd/lnx (2:221/1.0)

    Testing with "def_nonbsp on" and "def_delssq on" and flowed enabled.

    Your message had NBSPs, they should disappear. There should be no extra spaces in quotes.

    Carlos

    --- Mozilla Thunderbird
    * Origin: cyberiada-NNTP (2:341/234.99)