• Need volonteers to test another patch

    From Vitaliy Aksyonov@1:104/117 to All on Thursday, February 22, 2024 23:10:08
    Hello All.

    I made another patch slowly moving to rework charsets recoding and iconv.

    I'd appreciate if someone may build it from branch 'Aliases' and try to navigate your messages and write new messages.

    Especially if you have xlatcharsetalias entries in your config.

    https://github.com/golded-plus/golded-plus/tree/Aliases

    Vitaliy

    --- GoldED+/LNX 1.1.5-b20231030
    * Origin: Aurora, Colorado (1:104/117)
  • From Wilfred van Velzen@2:280/464 to Vitaliy Aksyonov on Friday, February 23, 2024 12:31:39
    Hi Vitaliy,

    On 2024-02-22 23:10:08, you wrote to All:

    I made another patch slowly moving to rework charsets recoding and
    iconv.

    I'd appreciate if someone may build it from branch 'Aliases' and try to navigate your messages and write new messages.

    I'm using it now. It seems to work just as well as my previous version GED+LNX 1.1.5-b20240209, from 'master', in my current config.

    Especially if you have xlatcharsetalias entries in your config.

    I don't use any of those though...

    https://github.com/golded-plus/golded-plus/tree/Aliases

    --- GoldED+/LNX 1.1.5-b20231030

    You're not using it yourself? ;-)


    Bye, Wilfred.

    --- FMail-lnx64 2.2.1.1
    * Origin: FMail development HQ (2:280/464)
  • From Vitaliy Aksyonov@1:104/117 to Wilfred van Velzen on Friday, February 23, 2024 07:10:06
    Hello Wilfred.

    23 Feb 24 12:31, you wrote to me:

    I made another patch slowly moving to rework charsets recoding
    and iconv.

    I'd appreciate if someone may build it from branch 'Aliases' and
    try to navigate your messages and write new messages.

    I'm using it now. It seems to work just as well as my previous version GED+LNX 1.1.5-b20240209, from 'master', in my current config.

    Sounds good. Thanks.

    Especially if you have xlatcharsetalias entries in your config.

    I don't use any of those though...

    But you have xlatcharset in your config. That was changed too.

    https://github.com/golded-plus/golded-plus/tree/Aliases

    --- GoldED+/LNX 1.1.5-b20231030

    You're not using it yourself? ;-)

    I tested it in my development computer. Just didn't have time to update it on this one. Will do later today.

    Vitaliy

    --- GoldED+/LNX 1.1.5-b20231030
    * Origin: Aurora, Colorado (1:104/117)
  • From Wilfred van Velzen@2:280/464 to Vitaliy Aksyonov on Friday, February 23, 2024 16:31:44
    Hi Vitaliy,

    On 2024-02-23 07:10:06, you wrote to me:

    Especially if you have xlatcharsetalias entries in your config.

    I don't use any of those though...

    But you have xlatcharset in your config. That was changed too.

    I don't have any in my standard configuration. But if I include charsets.cfg from the project. All messages in the stats area with linedrawing characters don't look ok. But that's the same in the previous version of golded that I was using.

    Bye, Wilfred.

    --- FMail-lnx64 2.2.1.1
    * Origin: FMail development HQ (2:280/464)
  • From Vitaliy Aksyonov@1:104/117 to Wilfred van Velzen on Sunday, February 25, 2024 19:06:26
    Hello Wilfred.

    23 Feb 24 16:31, you wrote to me:

    Especially if you have xlatcharsetalias entries in your config.

    I don't use any of those though...

    But you have xlatcharset in your config. That was changed too.

    I don't have any in my standard configuration. But if I include charsets.cfg from the project. All messages in the stats area with linedrawing characters don't look ok. But that's the same in the
    previous version of golded that I was using.

    If you don't have any - pseudo-graphics won't work unless you have console in same encoding as message.

    Because GoldEd will only do charset translations if you have it configured. Otherwise it just prints message as is. And codes for pseudo-graphics is different in cp437 (message charset) and utf-8 (your terminal charset).

    When you have charsets configured - there is another issue. GoldEd may convert from one byte encodings like cp437 to utf-8, but can't convert it back from utf-8 to cp437. I'm working on that. Have no idea, how much time it may take though.

    Vitaliy

    --- GoldED+/LNX 1.1.5-b20240223
    * Origin: Aurora, Colorado (1:104/117)
  • From Wilfred van Velzen@2:280/464 to Vitaliy Aksyonov on Monday, February 26, 2024 13:41:03
    Hi Vitaliy,

    On 2024-02-25 19:06:26, you wrote to me:

    I don't have any in my standard configuration. But if I include
    charsets.cfg from the project. All messages in the stats area with
    linedrawing characters don't look ok. But that's the same in the
    previous version of golded that I was using.

    If you don't have any - pseudo-graphics won't work unless you have console in same encoding as message.

    Because GoldEd will only do charset translations if you have it configured.
    Otherwise it just prints message as is. And codes for pseudo-graphics is different in cp437 (message charset) and utf-8 (your terminal charset).

    It seems to work best for me this way. I never got the utf-8 terminal working correctly...

    When you have charsets configured - there is another issue. GoldEd may convert from one byte encodings like cp437 to utf-8, but can't convert it back from utf-8 to cp437. I'm working on that. Have no idea, how much time it may take though.

    I'll wait for that, before I have another go at it! ;-)

    Bye, Wilfred.

    --- FMail-lnx64 2.2.1.1
    * Origin: FMail development HQ (2:280/464)
  • From Nicholas Boel@1:154/10 to Vitaliy Aksyonov on Monday, February 26, 2024 16:50:40
    On Mon, 26 Feb 2024 01:06:26 -0700, Vitaliy Aksyonov -> Wilfred Van Velzen wrote:

    Hello Wilfred.

    23 Feb 24 16:31, you wrote to me:

     VA>>>>> Especially if you have xlatcharsetalias entries in your config.

     WvV>>>> I don't use any of those though...

     VA>>> But you have xlatcharset in your config. That was changed too.

     WvV>> I don't have any in my standard configuration. But if I include
     WvV>> charsets.cfg from the project. All messages in the stats area with
     WvV>> linedrawing characters don't look ok. But that's the same in the
     WvV>> previous version of golded that I was using.

    If you don't have any - pseudo-graphics won't work unless you have console in same encoding as message.

    This is why I wonder if he has a cp437 terminal setup, and this is why
    these messages look fine to him. Whereas, I'm using a utf-8 terminal so I
    have to translate everything to look ok.

    Regards,
    Nick

    ... "Take my advice, I don't use it anyway."
    --- Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:115.0) Gecko/20100101 Thunderb
    * Origin: _thePharcyde distribution system (Wisconsin) (1:154/10)
  • From Nicholas Boel@1:154/10 to Vitaliy Aksyonov on Monday, February 26, 2024 17:11:48
    On Fri, 23 Feb 2024 05:10:08 -0700, Vitaliy Aksyonov -> All wrote:

    I made another patch slowly moving to rework charsets recoding and iconv.

    I'd appreciate if someone may build it from branch 'Aliases' and try to navigate your messages and write new messages.

    Especially if you have xlatcharsetalias entries in your config.

    https://github.com/golded-plus/golded-plus/tree/Aliases

    When adding this patch does Golded need to be compiled with iconv support (ICONV=1)?

    I would imagine Wilfred wouldn't see anything different if he's not using
    any kind of character translations. But if one uses charsets.cfg this would
    be a good test for you.

    Regards,
    Nick

    ... "Take my advice, I don't use it anyway."
    --- Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:115.0) Gecko/20100101 Thunderb
    * Origin: _thePharcyde distribution system (Wisconsin) (1:154/10)
  • From Vitaliy Aksyonov@1:104/117 to Nicholas Boel on Tuesday, February 27, 2024 06:59:54
    Hello Nicholas.

    26 Feb 24 16:50, you wrote to me:

    ?aVA>>>>>> Especially if you have xlatcharsetalias entries in your
    ?aVA>>>>>> config.

    ?aWvV>>>>> I don't use any of those though...

    ?aVA>>>> But you have xlatcharset in your config. That was changed
    ?aVA>>>> too.

    ?aWvV>>> I don't have any in my standard configuration. But if I
    ?aWvV>>> include charsets.cfg from the project. All messages in the
    ?aWvV>>> stats area with linedrawing characters don't look ok. But
    ?aWvV>>> that's the same in the previous version of golded that I was
    ?aWvV>>> using.

    If you don't have any - pseudo-graphics won't work unless you
    have console in same encoding as message.

    This is why I wonder if he has a cp437 terminal setup, and this is why these messages look fine to him. Whereas, I'm using a utf-8 terminal
    so I have to translate everything to look ok.

    Yes. You need to use translation in this case. The problem is that GoldEd may translate from one byte charset to UTF-8, but not in reverse direction yet. That mean that you'll be able to read messages, but not write them. Actually, you'll be able to write them if no symbols with codes > 127 are used. But CHRS kludge in message will look like UTF-8 2.

    Vitaliy

    --- GoldED+/LNX 1.1.5-b20240223
    * Origin: Aurora, Colorado (1:104/117)
  • From Vitaliy Aksyonov@1:104/117 to Nicholas Boel on Tuesday, February 27, 2024 07:02:32
    Hello Nicholas.

    26 Feb 24 17:11, you wrote to me:

    I made another patch slowly moving to rework charsets recoding
    and iconv.

    I'd appreciate if someone may build it from branch 'Aliases' and
    try to navigate your messages and write new messages.

    Especially if you have xlatcharsetalias entries in your config.

    https://github.com/golded-plus/golded-plus/tree/Aliases

    When adding this patch does Golded need to be compiled with iconv
    support (ICONV=1)?

    No. Iconv support is far from completion yet.

    I would imagine Wilfred wouldn't see anything different if he's not
    using any kind of character translations. But if one uses charsets.cfg this would be a good test for you.

    Yes. He shall not see any difference. And if he does - it mean I have a bug in my code. That's why I want as many people with different settings as possible to try it.

    Vitaliy

    ... He ?o??o? e?u?o? ???? ?e?o?e?!
    --- GoldED+/LNX 1.1.5-b20240223
    * Origin: Aurora, Colorado (1:104/117)
  • From Nicholas Boel@1:154/10 to Vitaliy Aksyonov on Tuesday, February 27, 2024 17:23:26
    On Tue, 27 Feb 2024 12:59:54 -0700, Vitaliy Aksyonov -> Nicholas Boel wrote:

    This is why I wonder if he has a cp437 terminal setup, and this is why
    these messages look fine to him. Whereas, I'm using a utf-8 terminal
    so I have to translate everything to look ok.

    Yes. You need to use translation in this case. The problem is that
    GoldEd may translate from one byte charset to UTF-8, but not in reverse direction yet. That mean that you'll be able to read messages, but not write them. Actually, you'll be able to write them if no symbols with codes > 127 are used. But CHRS kludge in message will look like UTF-8 2.

    Yes, I've known this for quite some time. And my current issue has to do
    with reading messages (the one we've been using as an example) specifically translated from cp437 to utf-8.

    Otherwise, if I absolutely have to reply to a message containing cp437 characters, I just don't quote those - as I know they won't be translated properly.

    Regards,
    Nick

    ... "Take my advice, I don't use it anyway."
    --- Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:115.0) Gecko/20100101 Thunderb
    * Origin: _thePharcyde distribution system (Wisconsin) (1:154/10)
  • From Nicholas Boel@1:154/10 to Vitaliy Aksyonov on Tuesday, February 27, 2024 17:58:26
    Hello Vitaliy,

    On Tuesday February 27 2024 07:02, you wrote to me:

    Yes. He shall not see any difference. And if he does - it mean I have
    a bug in my code. That's why I want as many people with different
    settings as possible to try it.

    Ok, I've installed your branch in a separate location as my other Golded install.

    f5a1c8730f6dd5f8d56154822f68eb3a34f18c5a (b20231112) is what I'm currently using in my main directory, as that is the last version that works for me, just prior to you reverting those commits.

    This version is, as far as I can tell, identical visually to the latest master branch - which doesn't work for me.

    Regards,
    Nick

    ... "Take my advice, I don't use it anyway."
    --- GoldED+/LNX 1.1.5-b20240223
    * Origin: _thePharcyde distribution system (Wisconsin) (1:154/10)
  • From Vitaliy Aksyonov@1:104/117 to Nicholas Boel on Tuesday, February 27, 2024 23:53:06
    Hello Nicholas.

    27 Feb 24 17:23, you wrote to me:

    On Tue, 27 Feb 2024 12:59:54 -0700, Vitaliy Aksyonov -> Nicholas Boel wrote:

    This is why I wonder if he has a cp437 terminal setup, and this
    is why these messages look fine to him. Whereas, I'm using a
    utf-8 terminal so I have to translate everything to look ok.

    Yes. You need to use translation in this case. The problem is
    that GoldEd may translate from one byte charset to UTF-8, but not
    in reverse direction yet. That mean that you'll be able to read
    messages, but not write them. Actually, you'll be able to write
    them if no symbols with codes > 127 are used. But CHRS kludge in
    message will look like UTF-8 2.

    Yes, I've known this for quite some time. And my current issue has to
    do with reading messages (the one we've been using as an example) specifically translated from cp437 to utf-8.

    Otherwise, if I absolutely have to reply to a message containing cp437 characters, I just don't quote those - as I know they won't be
    translated properly.

    I played with your configuration and have a good and bad news for you.

    Good:
    - I reproduced your issue.
    - GoldEd correctly converts pseudo-graphics from cp437 to utf-8.

    Bad:
    - GoldEd does not support unicode. Even if you compile it with ncursesw, it still uses non unicode versions of functions to print text. That's why you see those escape sequences instead of pseudo-graphics symbols.

    I still suggest you to use one-byte locale for GoldEd. And remember, you don't need to switch whole system to that locale, because in Linux locale is a property of a process. So you may have UTF-8 everywhere and cp437 for GoldEd. Most of terminals (including Putty) support different charsets.

    Another option - is to use external editor, but that won't help you with message reader.

    Vitaliy

    --- GoldED+/LNX 1.1.5-b20240223
    * Origin: Aurora, Colorado (1:104/117)
  • From Wilfred van Velzen@2:280/464 to Vitaliy Aksyonov on Wednesday, February 28, 2024 11:01:38
    Hi Vitaliy,

    On 2024-02-23 07:10:06, you wrote to me:

    I made another patch slowly moving to rework charsets recoding
    and iconv.

    I'd appreciate if someone may build it from branch 'Aliases' and
    try to navigate your messages and write new messages.

    I'm using it now. It seems to work just as well as my previous
    version GED+LNX 1.1.5-b20240209, from 'master', in my current config.

    Sounds good. Thanks.

    I switched back to the master branch because this patch was merged. Of course there are no changes in its operation for me.

    I didn't mention this before, but there are some linker warnings:

    ../goldlib/uulib/libuulib.a(uunconc.cpp.o): In function `UUDecode(_uulist*)': uunconc.cpp:(.text+0x34e2): warning: the use of `tempnam' is dangerous, better use `mkstemp'
    CMakeFiles/golded.dir/gedoit.cpp.o: In function `WriteMsgs(GMsg*)': gedoit.cpp:(.text+0xa51): warning: the use of `mktemp' is dangerous, better use `mkstemp'

    They should be easy to address.


    Bye, Wilfred.

    --- FMail-lnx64 2.2.1.1
    * Origin: FMail development HQ (2:280/464)
  • From Vitaliy Aksyonov@1:104/117 to Wilfred van Velzen on Wednesday, February 28, 2024 11:03:48
    Hello Wilfred.

    28 Feb 24 11:01, you wrote to me:

    I didn't mention this before, but there are some linker warnings:

    ../goldlib/uulib/libuulib.a(uunconc.cpp.o): In function `UUDecode(_uulist*)': uunconc.cpp:(.text+0x34e2): warning: the use of `tempnam' is dangerous, better use
    `mkstemp' CMakeFiles/golded.dir/gedoit.cpp.o: In function `WriteMsgs(GMsg*)': gedoit.cpp:(.text+0xa51): warning: the use of `mktemp' is dangerous, better use `mkstemp'

    They should be easy to address.

    I've seen those. I'm not sure that mkstemp is available in all systems GoldEd is possible to build.

    Vitaliy

    ... O?u? ? ?o?e - xy?e ?a?apu?a...
    --- GoldED+/LNX 1.1.5-b20240223
    * Origin: Aurora, Colorado (1:104/117)
  • From Nicholas Boel@1:154/10 to Vitaliy Aksyonov on Wednesday, February 28, 2024 18:33:04
    On Wed, 28 Feb 2024 05:53:06 -0700, Vitaliy Aksyonov -> Nicholas Boel wrote:

    I played with your configuration and have a good and bad news for you.

    Good:
    - I reproduced your issue.
    - GoldEd correctly converts pseudo-graphics from cp437 to utf-8.

    Well, it did. Until you reverted those changes. :(

    Bad:
    - GoldEd does not support unicode. Even if you compile it with ncursesw, it still uses non unicode versions of functions to print text. That's
    why you see those escape sequences instead of pseudo-graphics symbols.

    Something happened recently where it made it quite a bit worse. I was
    reading utf-8 Cyrillic, Greek, Japanese, Chinese, etc just fine in Golded
    until the most recent version.

    What changed with the ncurses init that was reverted?

    And why does that change affect me opposite of a cp437 locale user?

    If all I'm doing is translating from cp437 to utf-8 (or anything to utf-8)
    I should still be able to read it properly, as I have been.. until
    recently. Whatever you were doing with ncurses init helped me. I was able
    to read utf-8 messages perfectly fine. Almost every single "Merry
    Christmas" or "Happy New Year" Michiel posts yearly (except maybe a 2 or 3) were perfectly readable in Golded. The latest version they are not.

    I still suggest you to use one-byte locale for GoldEd. And remember, you don't need to switch whole system to that locale, because in Linux
    locale is a property of a process. So you may have UTF-8 everywhere and cp437 for GoldEd. Most of terminals (including Putty) support different charsets.

    I'll pass on the suggestion :). I'll just keep using the last version that worked for me (minus a couple badly displayed ascii line characters, and
    keep testing newer versions to see if I ever get the display back that I
    lost. :)

    Another option - is to use external editor, but that won't help you with message reader.

    I already do this (I have always used nano with Golded). Viewing and
    writing in an external editor has never been a problem. Until recently,
    only reading became an issue. So the whole time it was working for me you actually broke something instead? :((

    Regards,
    Nick

    ... "Take my advice, I don't use it anyway."
    --- Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:115.0) Gecko/20100101 Thunderb
    * Origin: _thePharcyde distribution system (Wisconsin) (1:154/10)
  • From Nicholas Boel@1:154/10 to Wilfred van Velzen on Wednesday, February 28, 2024 18:36:58
    On Wed, 28 Feb 2024 17:01:38 +0100, Wilfred Van Velzen -> Vitaliy Aksyonov wrote:

    I switched back to the master branch because this patch was merged. Of course there are no changes in its operation for me.

    So everything from the "Aliases" branch has been merged into master now?
    Unless Vitaliy is going to continue doing things there for testing, I may
    as well delete that install (I installed it separately as to not mess up
    what is actually working for me).

    Regards,
    Nick

    ... "Take my advice, I don't use it anyway."
    --- Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:115.0) Gecko/20100101 Thunderb
    * Origin: _thePharcyde distribution system (Wisconsin) (1:154/10)
  • From Wilfred van Velzen@2:280/464.112 to Vitaliy Aksyonov on Thursday, February 29, 2024 08:59:10
    Hi Vitaliy,

    On 28 Feb 24 11:03, Vitaliy Aksyonov wrote to Wilfred van Velzen:
    about: "Re: Need volonteers to test another patch":

    I didn't mention this before, but there are some linker warnings:

    ../goldlib/uulib/libuulib.a(uunconc.cpp.o): In function
    `UUDecode(_uulist*)': uunconc.cpp:(.text+0x34e2): warning: the use of
    `tempnam' is dangerous, better use
    `mkstemp' CMakeFiles/golded.dir/gedoit.cpp.o: In function
    `WriteMsgs(GMsg*)': gedoit.cpp:(.text+0xa51): warning: the use of
    `mktemp' is dangerous, better use `mkstemp'

    They should be easy to address.

    I've seen those. I'm not sure that mkstemp is available in all systems GoldEd is possible to build.

    mkstemp() is: 4.3BSD, POSIX.1-2001

    If not available you could make your own from mktemp() and open() I suppose. I could find atleast 1 example on the net...


    Wilfred.

    --- FMail-W64 2.2.0.0
    * Origin: point@work (2:280/464.112)
  • From Wilfred van Velzen@2:280/464.112 to Nicholas Boel on Thursday, February 29, 2024 08:27:26
    Hi Nicholas,

    On 28 Feb 24 18:36, Nicholas Boel wrote to Wilfred van Velzen:
    about: "Need volonteers to test another patch":

    I switched back to the master branch because this patch was merged.
    Of course there are no changes in its operation for me.

    So everything from the "Aliases" branch has been merged into master now?

    As far as I can tell from the git log: yes

    https://paste.opensuse.org/pastes/8223c3c71052

    Unless Vitaliy is going to continue doing things there for testing, I
    may as well delete that install (I installed it separately as to not
    mess up what is actually working for me).

    Your choice! ;-)


    Wilfred.

    --- FMail-W64 2.2.0.0
    * Origin: point@work (2:280/464.112)
  • From Vitaliy Aksyonov@1:104/117 to Nicholas Boel on Saturday, March 02, 2024 09:13:34
    Hello Nicholas.

    28 Feb 24 18:33, you wrote to me:

    I played with your configuration and have a good and bad news for
    you.

    Good:
    - I reproduced your issue.
    - GoldEd correctly converts pseudo-graphics from cp437 to utf-8.

    Well, it did. Until you reverted those changes. :(

    Most probably it was some combination which made it looking almost correct. I think that screen may be the reason you saw pseudo-graphics more or less correctly. Remember those line wraps? That happens because GoldEd converts those symbols to UTF-8 first. All pseudo-graphics symbols represented as 3 bytes. So that line become 3 times longer in bytes. Then GoldEd tries to split message to lines and it uses bytes! not symbols. That's why it splits the line in the middle of those pseudo-graphics. Even worse, it may tear apart one UTF-8 symbol to two lines and it will be displayed incorrectly.

    GoldEd cannot work correctly with multibyte sequences. And even if it looks "correct", it's just because most English letters has same codes in cp437 and UTF-8.

    If you want to keep using UTF-8, I may only suggest to find version, which "works" for you and stick to it.

    Until full UTF-8 support implemented in GoldEd (if that ever happen), don't expect it to work correctly, sorry.

    Bad:
    - GoldEd does not support unicode. Even if you compile it with
    ncursesw, it still uses non unicode versions of functions to
    print text. That's why you see those escape sequences instead of
    pseudo-graphics symbols.

    Something happened recently where it made it quite a bit worse. I was reading utf-8 Cyrillic, Greek, Japanese, Chinese, etc just fine in
    Golded until the most recent version.

    I understand, that it might "work", but it was pure luck.

    What changed with the ncurses init that was reverted?

    First commit I did - changed order when ncurses initializes to be able to print some text, which doesn't use ncurses functions. That is used, when you run it with "-INSTALL". Before my first change with ncurses, that text was not displayed. I tried to fix that. But it caused some issues to dome sysops and what I did in recent commit - revert it back to what it was before commit 8e9f3518ac9b3b32676e7b7563e92cc44e7b5ba7.

    And why does that change affect me opposite of a cp437 locale user?

    Your setup is not supported and it's hard to say why it even works. Unfortunately I'm not big ncurses expert and hard to say what may go wrong.

    If all I'm doing is translating from cp437 to utf-8 (or anything to
    utf-8) I should still be able to read it properly, as I have been..
    until recently. Whatever you were doing with ncurses init helped me. I
    was able to read utf-8 messages perfectly fine. Almost every single
    "Merry Christmas" or "Happy New Year" Michiel posts yearly (except
    maybe a 2 or 3) were perfectly readable in Golded. The latest version
    they are not.

    Let me try to explain. Some UTF-8 characters have more than one byte. But GoldEd "prints" them to screen one by one. Also it applies some attributes. It uses non-unicode versions of ncurses functions and if that byte looks like special character to ncurses, it escapes it. That's why you see those weird character sequences with ~.

    I still suggest you to use one-byte locale for GoldEd. And
    remember, you don't need to switch whole system to that locale,
    because in Linux locale is a property of a process. So you may
    have UTF-8 everywhere and cp437 for GoldEd. Most of terminals
    (including Putty) support different charsets.

    I'll pass on the suggestion :). I'll just keep using the last version
    that worked for me (minus a couple badly displayed ascii line
    characters, and keep testing newer versions to see if I ever get the display back that I lost. :)

    It's your choice. Just be aware, that if it works - it's just pure luck and don't expect it to last. Until we implement UTF-8 support. It may take years. Or never happen. It's not so easy to do it with backward compatibility wih all older systems like DOS or OS/2.

    Another option - is to use external editor, but that won't help
    you with message reader.

    I already do this (I have always used nano with Golded). Viewing and writing in an external editor has never been a problem. Until
    recently, only reading became an issue. So the whole time it was
    working for me you actually broke something instead? :((

    I'm sorry about that, but it's not much we can do right now.

    Vitaliy

    --- GoldED+/LNX 1.1.5-b20240223
    * Origin: Aurora, Colorado (1:104/117)
  • From Vitaliy Aksyonov@1:104/117 to Nicholas Boel on Saturday, March 02, 2024 09:33:36
    Hello Nicholas.

    28 Feb 24 18:36, you wrote to Wilfred van Velzen:

    I switched back to the master branch because this patch was
    merged. Of course there are no changes in its operation for me.
    So everything from the "Aliases" branch has been merged into master
    now? Unless Vitaliy is going to continue doing things there for
    testing, I may as well delete that install (I installed it separately
    as to not mess up what is actually working for me).

    Aliases branch was not merged yet. It was another set of patches from Nil. I'll work on Aliases branch merge soon.

    And I very appreciate your help with testing! GoldEd has too many settings to be able to test on my computer and even imagine different combinations.

    And it doesn't have any unit tests, that doesn't help. :(

    Vitaliy

    --- GoldED+/LNX 1.1.5-b20240223
    * Origin: Aurora, Colorado (1:104/117)
  • From Vitaliy Aksyonov@1:104/117 to Wilfred van Velzen on Saturday, March 02, 2024 09:35:38
    Hello Wilfred.

    29 Feb 24 08:59, you wrote to me:

    I didn't mention this before, but there are some linker
    warnings:

    ../goldlib/uulib/libuulib.a(uunconc.cpp.o): In function
    `UUDecode(_uulist*)': uunconc.cpp:(.text+0x34e2): warning: the
    use of `tempnam' is dangerous, better use `mkstemp'
    CMakeFiles/golded.dir/gedoit.cpp.o: In function
    `WriteMsgs(GMsg*)': gedoit.cpp:(.text+0xa51): warning: the use
    of `mktemp' is dangerous, better use `mkstemp'

    They should be easy to address.

    I've seen those. I'm not sure that mkstemp is available in all
    systems GoldEd is possible to build.

    mkstemp() is: 4.3BSD, POSIX.1-2001

    If not available you could make your own from mktemp() and open() I suppose. I could find atleast 1 example on the net...

    It's not the biggest issue now. But I'll think about it. Thanks.

    Vitaliy

    --- GoldED+/LNX 1.1.5-b20240223
    * Origin: Aurora, Colorado (1:104/117)
  • From Nicholas Boel@1:154/10 to Vitaliy Aksyonov on Saturday, March 02, 2024 14:44:42
    Hello Vitaliy,

    On Saturday March 02 2024 09:13, you wrote to me:

    Most probably it was some combination which made it looking almost correct. I think that screen may be the reason you saw pseudo-graphics

    I'm no longer using screen or tmux, and this version (along with all previous versions since you made the ncurses change) work the same and show UTF-8 properly, except for what you describe below:

    more or less correctly. Remember those line wraps? That happens
    because GoldEd converts those symbols to UTF-8 first. All
    pseudo-graphics symbols represented as 3 bytes. So that line become 3 times longer in bytes. Then GoldEd tries to split message to lines and
    it uses bytes! not symbols. That's why it splits the line in the
    middle of those pseudo-graphics. Even worse, it may tear apart one
    UTF-8 symbol to two lines and it will be displayed incorrectly.

    Yeah, I've noticed most of this.. and thank you for your explanation. At least now I know why it is happening.

    GoldEd cannot work correctly with multibyte sequences. And even if it looks "correct", it's just because most English letters has same codes
    in cp437 and UTF-8.

    Maybe simple ones, like german umlauts and whatnot. But cp437 doesn't have any Cyrillic, Greek, Japanese, Chinese, etc.

    If you want to keep using UTF-8, I may only suggest to find version,
    which "works" for you and stick to it.

    I already have!

    Until full UTF-8 support implemented in GoldEd (if that ever happen), don't expect it to work correctly, sorry.

    That's ok. I had it somewhat working for awhile, the latest reverts have changed that. I don't have an issue going back to a "lucky" version. ;)

    It's your choice. Just be aware, that if it works - it's just pure
    luck and don't expect it to last. Until we implement UTF-8 support. It
    may take years. Or never happen. It's not so easy to do it with
    backward compatibility wih all older systems like DOS or OS/2.

    I don't mind being lucky sometimes.

    Regards,
    Nick

    ... "Take my advice, I don't use it anyway."
    --- GoldED+/LNX 1.1.5-b20231112
    * Origin: _thePharcyde distribution system (Wisconsin) (1:154/10)
  • From Vitaliy Aksyonov@1:104/117 to Nicholas Boel on Saturday, March 02, 2024 22:42:52
    Hello Nicholas.

    02 Mar 24 14:44, you wrote to me:

    Most probably it was some combination which made it looking
    almost correct. I think that screen may be the reason you saw
    pseudo-graphics

    I'm no longer using screen or tmux, and this version (along with all previous versions since you made the ncurses change) work the same and show UTF-8 properly, except for what you describe below:

    more or less correctly. Remember those line wraps? That happens
    because GoldEd converts those symbols to UTF-8 first. All
    pseudo-graphics symbols represented as 3 bytes. So that line
    become 3 times longer in bytes. Then GoldEd tries to split
    message to lines and it uses bytes! not symbols. That's why it
    splits the line in the middle of those pseudo-graphics. Even
    worse, it may tear apart one UTF-8 symbol to two lines and it
    will be displayed incorrectly.

    Yeah, I've noticed most of this.. and thank you for your explanation.
    At least now I know why it is happening.

    GoldEd cannot work correctly with multibyte sequences. And even
    if it looks "correct", it's just because most English letters has
    same codes in cp437 and UTF-8.

    Maybe simple ones, like german umlauts and whatnot. But cp437 doesn't
    have any Cyrillic, Greek, Japanese, Chinese, etc.

    If you want to keep using UTF-8, I may only suggest to find
    version, which "works" for you and stick to it.

    I already have!

    Until full UTF-8 support implemented in GoldEd (if that ever
    happen), don't expect it to work correctly, sorry.

    That's ok. I had it somewhat working for awhile, the latest reverts
    have changed that. I don't have an issue going back to a "lucky"
    version. ;)

    It's your choice. Just be aware, that if it works - it's just
    pure luck and don't expect it to last. Until we implement UTF-8
    support. It may take years. Or never happen. It's not so easy to
    do it with backward compatibility wih all older systems like DOS
    or OS/2.

    I don't mind being lucky sometimes.

    I played little bit more with the code and different settings and found what was the difference between those two versions.

    First - you use "wide" ncurses - ncursesw. I use ncurses.
    Second. GoldEd incorrectly initializes ncurses in reverted version (like it was before I started to make any changes in GoldEd code). ncurses documentation explicitly says that before call initscr(), locale has to be set with setlocale. Otherwise it will use incorrect settings. Most probably plain C locale. I kinda get picture very similar to yours.

    Did you have an issue with non-English chars when scrolling the message? What I see in UFT-8 terminal now - most of unicode text displayed correctly and pseudo-graphic too. Only pseudo-graphics lines broken similar to yours. And when I scroll text down - it corrupts.

    If you may try to add one line of code to latest master and try it - that would be helpful.

    goldlib/gcui/gkbdbase.cpp
    line 149, right before initscr add this:

    setlocale(LC_ALL, "");

    If that gives you expected result - I'll push this change to master.

    Hope you still want to dig this. :)

    Vitaliy

    --- GoldED+/LNX 1.1.5-b20240223
    * Origin: Aurora, Colorado (1:104/117)
  • From Nicholas Boel@1:154/10 to Vitaliy Aksyonov on Sunday, March 03, 2024 08:46:26
    On Sun, 3 Mar 2024 04:42:52 -0700, Vitaliy Aksyonov -> Nicholas Boel wrote:

    I played little bit more with the code and different settings and found what was the difference between those two versions.

    First - you use "wide" ncurses - ncursesw. I use ncurses.
    Second. GoldEd incorrectly initializes ncurses in reverted version (like it was before I started to make any changes in GoldEd code). ncurses documentation explicitly says that before call initscr(), locale has to
    be set with setlocale. Otherwise it will use incorrect settings. Most probably plain C locale. I kinda get picture very similar to yours.

    For the record, I don't compile golded with WIDE_NCURSE=1. I have tried
    it in the past, but I don't see much, if any difference. However, I have
    also realized that in my distro (Archlinux) I don't have to install two separate packages for that, either. ncurses comes with ncursesw
    included. Is that maybe the difference?

    Did you have an issue with non-English chars when scrolling the message?

    Non-english characters usually display OK, and sometimes when you scroll
    the message, some of them (usually moreso the line and block drawings
    converted from cp437) will gain a red background with green "^"
    characters) or as you say below, corrupts. Using page down instead of
    line scrolling keeps the characters in tact, though.

    What I see in UFT-8 terminal now - most of unicode text displayed correctly and pseudo-graphic too. Only pseudo-graphics lines broken similar to yours. And when I scroll text down - it corrupts.

    Yes, I see this too. However, try using page down instead of the down
    arrow key. Maybe that will point you in a direction as to how both key
    presses are handled and you may be able to fix that as well!

    As for the pseudo-graphics wrapped to the next line, I have a (probably
    dumb) question about this: If the pseudo graphics were originally cp437
    (single byte) and translated to utf-8, once they are translated are they
    now multiple bytes per character?

    If "UTF-8 uses 1 to 4 bytes to encode a single character", I guess what
    I'm wondering is if the character was 1 byte to begin with, why wouldn't
    it stay 1 byte when translated to utf-8? Or is it because those
    _specific_ characters when in utf-8 are already multiple bytes?

    If you may try to add one line of code to latest master and try it -
    that would be helpful.

    goldlib/gcui/gkbdbase.cpp
    line 149, right before initscr add this:

    setlocale(LC_ALL, "");

    If that gives you expected result - I'll push this change to master.

    It worked! Or at least it's back to the way it was, which is a good thing!

    Hope you still want to dig this. :)

    Definitely! I appreciate you willing to help, even though we both know
    it's broken in more ways than one. I do understand that it is not
    currently (and has never been) supported, but if we can make it "kind
    of" work while not affecting others, I'm all for it!

    I would rather have a somewhat-working program than a non-working
    program. :)

    THANK YOU!

    Regards,
    Nick

    ... "Take my advice, I don't use it anyway."
    --- Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:115.0) Gecko/20100101 Thunderb
    * Origin: _thePharcyde distribution system (Wisconsin) (1:154/10)
  • From Michiel van der Vlist@2:280/5555 to Nicholas Boel on Sunday, March 03, 2024 16:45:34
    Hello Nicholas,

    On Sunday March 03 2024 08:46, you wrote to Vitaliy Aksyonov:

    As for the pseudo-graphics wrapped to the next line, I have a
    (probably dumb) question about this: If the pseudo graphics were originally cp437 (single byte) and translated to utf-8, once they are translated are they now multiple bytes per character?

    I prefer dumb quetion, they are easier to answer... ;-)

    Yes, they are translated to multi (usually two for most characters used in Fidonet) byte characters. Only the ASCII characters (0-127) are not translated and so remain one byte.

    If "UTF-8 uses 1 to 4 bytes to encode a single character", I guess
    what I'm wondering is if the character was 1 byte to begin with, why wouldn't it stay 1 byte when translated to utf-8? Or is it because
    those _specific_ characters when in utf-8 are already multiple bytes?

    A non ASCII character can not be translated to one byte for the simple reason that the remaning 128 bytes with the highest bit set are not enough to encode ALL the characters in ALL the single byte characters sets. The whole idea of unicode is to encode ALL the characters of ALL those characters sets, CP437, CP850, CP 866, CP 1250, etc into ONE encoding scheme. One byte is just not enough for all.

    To put it simple: if you want to encode CP437 and CP866, you could put CP437 OR CP866 in the first byte, but you need at least one bit more information which one it is; CP437 or CP866. That is not exactly how UTF-8 works but it should give you an idea of why just one byte can not be enough.


    Cheers, Michiel

    --- GoldED+/W32-MSVC 1.1.5-b20170303
    * Origin: Nieuw Schnøørd (2:280/5555)
  • From Nicholas Boel@1:154/10 to Michiel van der Vlist on Sunday, March 03, 2024 10:33:12
    On Sun, 3 Mar 2024 22:45:34 +0100, Michiel Van Der Vlist -> Nicholas
    Boel wrote:

    MvdV> Yes, they are translated to multi (usually two for most characters used
    MvdV> in Fidonet) byte characters. Only the ASCII characters (0-127) are not
    MvdV> translated and so remain one byte.

    Thanks for the explanation. While reading this, I did check the ASCII
    table and the characters I'm referring to are all above 127. This also
    is kind of reflected while using Golded, if I widen my screen more than
    160 characters, less of those lines are wrapped to the next line.

    However, it doesn't seem like I can widen my window enough to keep them
    all on one line, so I'm guessing when they are translated to utf-8 they
    are more than 2 bytes, since I'm well over double the width of an 80
    character screen - which the original stat message was made for.

    So, at this point it's basically working and displaying properly, but
    then comes in the 'characters' vs 'bytes' thing that Golded isn't
    supporting, so it is wrapping what it thinks is double, triple, or even quadruple the amount of 'characters' that are there.

    MvdV> To put it simple: if you want to encode CP437 and CP866, you could put
    MvdV> CP437 OR CP866 in the first byte, but you need at least one bit more
    MvdV> information which one it is; CP437 or CP866. That is not exactly how
    MvdV> UTF-8 works but it should give you an idea of why just one byte can not
    MvdV> be enough.

    Thank you for the explanation. This definitely helps me to understand
    what is happening.

    Regards,
    Nick

    ... "Take my advice, I don't use it anyway."
    --- Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:115.0) Gecko/20100101 Thunderb
    * Origin: _thePharcyde distribution system (Wisconsin) (1:154/10)
  • From Michiel van der Vlist@2:280/5555 to Nicholas Boel on Sunday, March 03, 2024 22:31:41
    Hello Nicholas,

    On Sunday March 03 2024 10:33, you wrote to me:

    So, at this point it's basically working and displaying properly, but
    then comes in the 'characters' vs 'bytes' thing that Golded isn't supporting, so it is wrapping what it thinks is double, triple, or
    even quadruple the amount of 'characters' that are there.

    When it comes to mapping the number of characters to the number of bytes, when you look at the UTF-8 encoding table, about two screens down here:

    https://en.wikipedia.org/wiki/UTF-8

    You can see that the length of the byte sequence can be determined just by looking at the first byte. Look from bit 7 to the right. The number of '1' bits equals the number of bytes in the character. All the follow up bytes start with '10'. So to get the number of characters ignore the bytes starting with '10' when counting the bytes.

    Breaking a line should only occur /before/ a byte starting with '0' or '11'.

    Knowing all that it should be doable to let Golded display properly.

    Perhaps the best strategy is to have Golded alway use UTF-8 internally. Almost everyone else does these days...

    Thank you for the explanation. This definitely helps me to understand
    what is happening.

    You'r welcome.


    Cheers, Michiel

    --- GoldED+/W32-MSVC 1.1.5-b20170303
    * Origin: Nieuw Schnøørd (2:280/5555)
  • From Vitaliy Aksyonov@1:104/117 to Nicholas Boel on Sunday, March 03, 2024 16:28:22
    Hello Nicholas.

    03 Mar 24 10:33, you wrote to Michiel van der Vlist:

    MvdV>> Yes, they are translated to multi (usually two for most
    MvdV>> characters used in Fidonet) byte characters. Only the ASCII
    MvdV>> characters (0-127) are not translated and so remain one byte.

    Thanks for the explanation. While reading this, I did check the ASCII table and the characters I'm referring to are all above 127. This also
    is kind of reflected while using Golded, if I widen my screen more
    than 160 characters, less of those lines are wrapped to the next line.

    However, it doesn't seem like I can widen my window enough to keep
    them all on one line, so I'm guessing when they are translated to
    utf-8 they are more than 2 bytes, since I'm well over double the width
    of an 80 character screen - which the original stat message was made
    for.

    Pseudo-graphics symbols encoded to three bytes in UTF-8. That's why it's not enough.
    So you either ignore it or make windows at least 3 times wider than that message. It might sound stupid and overkill, but it shall do the trick. :)

    So, at this point it's basically working and displaying properly, but
    then comes in the 'characters' vs 'bytes' thing that Golded isn't supporting, so it is wrapping what it thinks is double, triple, or
    even quadruple the amount of 'characters' that are there.

    MvdV>> To put it simple: if you want to encode CP437 and CP866, you
    MvdV>> could put CP437 OR CP866 in the first byte, but you need at
    MvdV>> least one bit more information which one it is; CP437 or CP866.
    MvdV>> That is not exactly how UTF-8 works but it should give you an
    MvdV>> idea of why just one byte can not be enough.

    Thank you for the explanation. This definitely helps me to understand
    what is happening.

    Unicode is very complex. It's event worse than you may think. For example, some displayed symbols take one char space on screen, but others two.
    Also one letter is not necessarily one Unicode "symbol". Also it may have non-printable symbols and many more.

    Vitaliy

    ... C????o ?a ??epa??ee, ?o ?e ?o??? ?epe? ?e?...
    --- GoldED+/LNX 1.1.5-b20240223
    * Origin: Aurora, Colorado (1:104/117)
  • From Vitaliy Aksyonov@1:104/117 to Michiel van der Vlist on Sunday, March 03, 2024 16:31:50
    Hello Michiel.

    03 Mar 24 22:31, you wrote to Nicholas Boel:

    So, at this point it's basically working and displaying properly,
    but then comes in the 'characters' vs 'bytes' thing that Golded
    isn't supporting, so it is wrapping what it thinks is double,
    triple, or even quadruple the amount of 'characters' that are
    there.

    MvdV> When it comes to mapping the number of characters to the number of
    MvdV> bytes, when you look at the UTF-8 encoding table, about two screens
    MvdV> down here:

    MvdV> https://en.wikipedia.org/wiki/UTF-8

    MvdV> You can see that the length of the byte sequence can be determined
    MvdV> just by looking at the first byte. Look from bit 7 to the right. The
    MvdV> number of '1' bits equals the number of bytes in the character. All
    MvdV> the follow up bytes start with '10'. So to get the number of
    MvdV> characters ignore the bytes starting with '10' when counting the
    MvdV> bytes.

    MvdV> Breaking a line should only occur /before/ a byte starting with '0' or
    MvdV> '11'.

    MvdV> Knowing all that it should be doable to let Golded display properly.

    MvdV> Perhaps the best strategy is to have Golded alway use UTF-8
    MvdV> internally. Almost everyone else does these days...

    That would be perfect. It only takes huge amount of effort. Especially with keeping code backward compatible with systems, which may not have Unicode support. I keep thinking about it and looking for possible ways to implement.

    For now I'd be happy to make iconv work properly. In this case GoldEd user may get rid of most (if not all) of translation tables. The problem is that source code has huge amounts of duplicated code and sometimes functions and variables names say nothing about what do they do. I spend huge amount of time just to understand what's going on.

    For example function, which splits message to lines is almost 1000 lines long! It has variables, used in multiple places, it not only splits the message, but guess charset, do recoding and other fun stuff.

    Vitaliy

    --- GoldED+/LNX 1.1.5-b20240223
    * Origin: Aurora, Colorado (1:104/117)
  • From Nicholas Boel@1:154/10 to Vitaliy Aksyonov on Sunday, March 03, 2024 19:53:50
    On Sun, 3 Mar 2024 22:28:22 -0700, Vitaliy Aksyonov -> Nicholas Boel wrote:

    Pseudo-graphics symbols encoded to three bytes in UTF-8. That's why it's not enough.
    So you either ignore it or make windows at least 3 times wider than that message. It might sound stupid and overkill, but it shall do the trick. :)

    I usually just ignore it. But it would be nice if one day it displayed properly. :)

    By the way, I see you added the Aliases branch to master, were you still planning on setting the locale before initscr?

    I've updated my Golded to the latest, and then added that line again and recompiled. Still seems to be working as expected!

    Regards,
    Nick

    ... "Take my advice, I don't use it anyway."
    --- Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:115.0) Gecko/20100101 Thunderb
    * Origin: _thePharcyde distribution system (Wisconsin) (1:154/10)
  • From Michiel van der Vlist@2:280/5555 to Vitaliy Aksyonov on Monday, March 04, 2024 08:39:58
    Hello Vitaliy,

    On Sunday March 03 2024 16:28, you wrote to Nicholas Boel:

    Unicode is very complex. It's event worse than you may think. For
    example, some displayed symbols take one char space on screen, but
    others two. Also one letter is not necessarily one Unicode "symbol".
    Also it may have non-printable symbols and many more.

    True but we do not need a full 100% implementation for Fidonet. We can have a working implementation and ignore these details for now.


    Cheers, Michiel

    --- GoldED+/W32-MSVC 1.1.5-b20170303
    * Origin: Nieuw Schnøørd (2:280/5555)
  • From Michiel van der Vlist@2:280/5555 to Vitaliy Aksyonov on Monday, March 04, 2024 08:42:23
    Hello Vitaliy,

    On Sunday March 03 2024 16:31, you wrote to me:

    MvdV>> Perhaps the best strategy is to have Golded alway use UTF-8
    MvdV>> internally. Almost everyone else does these days...

    That would be perfect. It only takes huge amount of effort. Especially with keeping code backward compatible with systems, which may not have Unicode support. I keep thinking about it and looking for possible
    ways to implement.

    Backwards compatibility is nice but there always comes a point that it gets in the way of progress and it has to be dropped. Are you thinking about the DOS version? If so I say, forget about it. Freeze the DOS version, the small minority that still uses DOS will have to make do with what they have fo the rest of the life of DOS.

    Another way may be to not use UTF-8 internally but use two byte widechrs everywhere and simple store the raw unicode code point. Conversion to and from code point to UTF-8 is simple. That will limit the use to the first 65535 code points, but that might be enough for the remaining life of Fidonet. OTOH, that is almost the same as what Window XP did. It used UTF-16 internally and Microsoft now regrets that.

    For example function, which splits message to lines is almost 1000
    lines long! It has variables, used in multiple places, it not only
    splits the message, but guess charset, do recoding and other fun
    stuff.

    Wauw!


    Cheers, Michiel

    --- GoldED+/W32-MSVC 1.1.5-b20170303
    * Origin: Nieuw Schnøørd (2:280/5555)
  • From Vitaliy Aksyonov@1:104/117 to Nicholas Boel on Monday, March 04, 2024 07:56:56
    Hello Nicholas.

    03 Mar 24 19:53, you wrote to me:

    Pseudo-graphics symbols encoded to three bytes in UTF-8. That's
    why it's not enough. So you either ignore it or make windows at
    least 3 times wider than that message. It might sound stupid and
    overkill, but it shall do the trick. :)

    I usually just ignore it. But it would be nice if one day it displayed properly. :)

    By the way, I see you added the Aliases branch to master, were you
    still planning on setting the locale before initscr?

    That's the plan.

    I've updated my Golded to the latest, and then added that line again
    and recompiled. Still seems to be working as expected!

    Great. Thanks for help!

    Vitaliy

    --- GoldED+/LNX 1.1.5-b20240223
    * Origin: Aurora, Colorado (1:104/117)
  • From Vitaliy Aksyonov@1:104/117 to Michiel van der Vlist on Monday, March 04, 2024 07:58:48
    Hello Michiel.

    04 Mar 24 08:42, you wrote to me:

    MvdV>>> Perhaps the best strategy is to have Golded alway use UTF-8
    MvdV>>> internally. Almost everyone else does these days...

    That would be perfect. It only takes huge amount of effort.
    Especially with keeping code backward compatible with systems,
    which may not have Unicode support. I keep thinking about it and
    looking for possible ways to implement.

    MvdV> Backwards compatibility is nice but there always comes a point that it
    MvdV> gets in the way of progress and it has to be dropped. Are you thinking
    MvdV> about the DOS version? If so I say, forget about it. Freeze the DOS
    MvdV> version, the small minority that still uses DOS will have to make do
    MvdV> with what they have fo the rest of the life of DOS.

    We've been thinking about that option.

    MvdV> Another way may be to not use UTF-8 internally but use two byte
    MvdV> widechrs everywhere and simple store the raw unicode code point.
    MvdV> Conversion to and from code point to UTF-8 is simple. That will limit
    MvdV> the use to the first 65535 code points, but that might be enough for
    MvdV> the remaining life of Fidonet. OTOH, that is almost the same as what
    MvdV> Window XP did. It used UTF-16 internally and Microsoft now regrets
    MvdV> that.

    Best possible way is to use UTF-8 for all strings inside and only convert text when read/write from/to message base and to screen. And even if drop DOS support - need to take into account OS specifics for Unicode. As long as GoldEd uses fixed size buffers in many places - that's huge refactoring. Better to replace it with std::string almost everywhere.

    For example function, which splits message to lines is almost
    1000 lines long! It has variables, used in multiple places, it
    not only splits the message, but guess charset, do recoding and
    other fun stuff.

    MvdV> Wauw!

    That's one of the reasons, why progress is slow.

    Vitaliy

    --- GoldED+/LNX 1.1.5-b20240223
    * Origin: Aurora, Colorado (1:104/117)
  • From Nicholas Boel@1:154/10 to Vitaliy Aksyonov on Monday, March 04, 2024 18:29:10
    Hello Vitaliy,

    On Monday March 04 2024 07:56, you wrote to me:

    I've updated my Golded to the latest, and then added that line
    again and recompiled. Still seems to be working as expected!

    Great. Thanks for help!

    Since I would just sit there like a deer in headlights if I were to jump into the code myself, I'll leave that up to you and just test where I can. :)

    Regards,
    Nick

    ... "Take my advice, I don't use it anyway."
    --- GoldED+/LNX 1.1.5-b20240302
    * Origin: _thePharcyde distribution system (Wisconsin) (1:154/10)
  • From Nicholas Boel@1:154/10 to Vitaliy Aksyonov on Monday, March 04, 2024 19:18:14
    Hello Vitaliy,

    On Mon, 04 Mar 2024 07:56:56 -0700, you wrote to me:

    Definitely not going to stop traffic or anything, but if you take a look at the above quote header line taken from golded.tpl, it seems when trying to use "@otzoffset" in golded.tpl it adds an extra space before it. I'm specifically using "On @odate @otzoffset," with only one space in between the two tokens, and the date I have set in gedlngus.cfg doesn't have a space at the end of it, either.

    Regards,
    Nick

    ... "Take my advice, I don't use it anyway."
    --- GoldED+/LNX 1.1.5-b20240302
    * Origin: _thePharcyde distribution system (Wisconsin) (1:154/10)
  • From Vitaliy Aksyonov@1:104/117 to Nicholas Boel on Monday, March 04, 2024 21:19:50
    Hello Nicholas.

    04 Mar 24 18:29, you wrote to me:

    I've updated my Golded to the latest, and then added that line
    again and recompiled. Still seems to be working as expected!

    Great. Thanks for help!

    Since I would just sit there like a deer in headlights if I were to
    jump into the code myself, I'll leave that up to you and just test
    where I can. :)

    I'll work on that. Wait couple of days please.

    Vitaliy

    --- GoldED+/LNX 1.1.5-b20240223
    * Origin: Aurora, Colorado (1:104/117)
  • From Vitaliy Aksyonov@1:104/117 to Nicholas Boel on Monday, March 04, 2024 21:20:26
    Hello Nicholas.

    04 Mar 24 19:18, you wrote to me:

    Definitely not going to stop traffic or anything, but if you take a
    look at the above quote header line taken from golded.tpl, it seems
    when trying to use "@otzoffset" in golded.tpl it adds an extra space before it. I'm specifically using "On @odate @otzoffset," with only
    one space in between the two tokens, and the date I have set in gedlngus.cfg doesn't have a space at the end of it, either.

    That shall be supersimple. I'll take a look on that too when have some time.

    Vitaliy

    --- GoldED+/LNX 1.1.5-b20240223
    * Origin: Aurora, Colorado (1:104/117)
  • From Nicholas Boel@1:154/10 to Vitaliy Aksyonov on Tuesday, March 05, 2024 20:27:36
    On Tue, 5 Mar 2024 03:20:26 -0700, Vitaliy Aksyonov -> Nicholas Boel wrote:

    Definitely not going to stop traffic or anything, but if you take a
    look at the above quote header line taken from golded.tpl, it seems
    when trying to use "@otzoffset" in golded.tpl it adds an extra space
    before it. I'm specifically using "On @odate @otzoffset," with only
    one space in between the two tokens, and the date I have set in
    gedlngus.cfg doesn't have a space at the end of it, either.

    That shall be supersimple. I'll take a look on that too when have some time.

    Thank you!

    Regards,
    Nick

    ... "Take my advice, I don't use it anyway."
    --- Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:115.0) Gecko/20100101 Thunderb
    * Origin: _thePharcyde distribution system (Wisconsin) (1:154/10)
  • From Vitaliy Aksyonov@1:104/117 to Vitaliy Aksyonov on Tuesday, March 05, 2024 21:53:14
    Hello Vitaliy.

    04 Mar 24 21:19, I wrote to Nicholas Boel:

    I've updated my Golded to the latest, and then added that line
    again and recompiled. Still seems to be working as expected!

    Great. Thanks for help!

    Since I would just sit there like a deer in headlights if I were
    to jump into the code myself, I'll leave that up to you and just
    test where I can. :)

    I'll work on that. Wait couple of days please.

    Pull request is sent. Wait for commit.

    Unfortunately it will break pseudo-graphics in some other systems. But my change shall be there anyway. Ncurses shall not be initialized before setlocale().

    Vitaliy

    ... 640K ought to be enough for anybody
    --- GoldED+/LNX 1.1.5-b20240223
    * Origin: Aurora, Colorado (1:104/117)
  • From Vitaliy Aksyonov@1:104/117 to Nicholas Boel on Tuesday, March 05, 2024 22:22:24
    Hello Nicholas.

    05 Mar 24 20:27, you wrote to me:

    Definitely not going to stop traffic or anything, but if you
    take a look at the above quote header line taken from
    golded.tpl, it seems when trying to use "@otzoffset" in
    golded.tpl it adds an extra space before it. I'm specifically
    using "On @odate @otzoffset," with only one space in between the
    two tokens, and the date I have set in gedlngus.cfg doesn't have
    a space at the end of it, either.

    That shall be supersimple. I'll take a look on that too when have
    some time.

    Thank you!

    This one sent for review as well.

    PS. Take a look on my tearline. Now I'm using my latest changes. ;)

    Vitaliy

    ... 640K ought to be enough for anybody
    --- GoldED+/LNX 1.1.5-b20240305-beta
    * Origin: Aurora, Colorado (1:104/117)
  • From Michiel van der Vlist@2:280/5555 to Vitaliy Aksyonov on Wednesday, March 06, 2024 13:38:44
    Hello Vitaliy,

    On Monday March 04 2024 07:58, you wrote to me:

    Best possible way is to use UTF-8 for all strings inside and only
    convert text when read/write from/to message base and to screen.

    I agree. That will be the easiest way to make as many Fidonet participants use UTF-8 all the way. With the sceen set to CP65001 writing to and from the screen should need no conversion.

    And even if drop DOS support - need to take into account OS specifics
    for Unicode.

    Such as? Even OS/2 has full UTF-8 support doesn't it?

    As long as GoldEd uses fixed size buffers in many places -
    that's huge refactoring. Better to replace it with std::string almost everywhere.

    Perhaps, but that won't solve the problem that when writing back to the message base strings have to be of fixed lenght for the To:, From:, Subj: and other fields. It may be necessary to truncate in order to fit. Truncating should be done on a UTF-8 sequence boundery. If need be step back until a byte with bit 7 and 6 set.

    For example function, which splits message to lines is almost
    1000 lines long! It has variables, used in multiple places, it
    not only splits the message, but guess charset, do recoding and
    other fun stuff.

    MvdV>> Wauw!

    That's one of the reasons, why progress is slow.

    Keep up the good work!


    Cheers, Michiel

    --- GoldED+/W32-MSVC 1.1.5-b20170303
    * Origin: Nieuw Schnøørd (2:280/5555)
  • From Vitaliy Aksyonov@1:104/117 to Michiel van der Vlist on Wednesday, March 06, 2024 10:32:50
    Hello Michiel.

    06 Mar 24 13:38, you wrote to me:

    Best possible way is to use UTF-8 for all strings inside and only
    convert text when read/write from/to message base and to screen.

    MvdV> I agree. That will be the easiest way to make as many Fidonet
    MvdV> participants use UTF-8 all the way. With the sceen set to CP65001
    MvdV> writing to and from the screen should need no conversion.

    And even if drop DOS support - need to take into account OS
    specifics for Unicode.

    MvdV> Such as? Even OS/2 has full UTF-8 support doesn't it?

    Windows and Linux works with console very differently. Even support those two together is a challenge. Code is not very well encapsulated in GoldEd and OS specifics just everywhere.

    I haven't tried OS/2 at all, so have no idea, how UTF-8 works there. Does it use ncurses too?

    As long as GoldEd uses fixed size buffers in many places -
    that's huge refactoring. Better to replace it with std::string
    almost everywhere.

    MvdV> Perhaps, but that won't solve the problem that when writing back to
    MvdV> the message base strings have to be of fixed lenght for the To:,
    MvdV> From:, Subj: and other fields. It may be necessary to truncate in
    MvdV> order to fit. Truncating should be done on a UTF-8 sequence boundery.
    MvdV> If need be step back until a byte with bit 7 and 6 set.

    For example function, which splits message to lines is almost
    1000 lines long! It has variables, used in multiple places, it
    not only splits the message, but guess charset, do recoding and
    other fun stuff.

    MvdV>>> Wauw!

    That's one of the reasons, why progress is slow.

    MvdV> Keep up the good work!

    Thanks. I try to fix or enhance something which doesn't break existing functionality. Hope some day we'll have UTF-8 support too.

    Vitaliy

    ... 640K ought to be enough for anybody
    --- GoldED+/LNX 1.1.5-b20240305-beta
    * Origin: Aurora, Colorado (1:104/117)
  • From Wilfred van Velzen@2:280/464 to Vitaliy Aksyonov on Wednesday, March 06, 2024 22:36:11
    Hi Vitaliy,

    On 2024-03-02 22:42:52, you wrote to Nicholas Boel:

    If you may try to add one line of code to latest master and try it -
    that would be helpful.

    goldlib/gcui/gkbdbase.cpp
    line 149, right before initscr add this:

    setlocale(LC_ALL, "");

    I pulled in the latest code with this change, and this breaks my configuration again. :-(

    So I reverted that change, and everything is back to normal...


    Bye, Wilfred.

    --- FMail-lnx64 2.2.1.1
    * Origin: FMail development HQ (2:280/464)
  • From Vitaliy Aksyonov@1:104/117 to Wilfred van Velzen on Wednesday, March 06, 2024 14:59:02
    Hello Wilfred.

    06 Mar 24 22:36, you wrote to me:

    If you may try to add one line of code to latest master and try
    it - that would be helpful.

    goldlib/gcui/gkbdbase.cpp
    line 149, right before initscr add this:

    setlocale(LC_ALL, "");

    I pulled in the latest code with this change, and this breaks my configuration again. :-(

    So I reverted that change, and everything is back to normal...

    One of the changes is very similar to that one which was reverted previously.

    Remind me your setup please. Do you use UTF-8 locale? What terminal do you use? TERM variable.

    Do you use local console or remote/ssh?


    Vitaliy

    ... 640K ought to be enough for anybody
    --- GoldED+/LNX 1.1.5-b20240305-beta
    * Origin: Aurora, Colorado (1:104/117)
  • From Michiel van der Vlist@2:280/5555 to Vitaliy Aksyonov on Wednesday, March 06, 2024 23:10:48
    Hello Vitaliy,

    On Wednesday March 06 2024 10:32, you wrote to me:

    I haven't tried OS/2 at all, so have no idea, how UTF-8 works there.
    Does it use ncurses too?

    I don't know. I tried OS/2 some 25-30 years ago and I liked it. But... I could not get it to work with Novell (Personal) Netware and so I dropped it. That problem was fixed later, but for me goodbye is goodbye. Well, most of the time...

    Also OS/2 does not support IPv6 and probably never will, so that is another reason to let it rest in peace.


    Cheers, Michiel

    --- GoldED+/W32-MSVC 1.1.5-b20170303
    * Origin: Nieuw Schnøørd (2:280/5555)
  • From Nicholas Boel@1:154/10 to Vitaliy Aksyonov on Wednesday, March 06, 2024 17:06:58
    On Wed, 6 Mar 2024 03:53:14 -0700, Vitaliy Aksyonov -> Vitaliy Aksyonov
    wrote:

    Pull request is sent. Wait for commit.

    Unfortunately it will break pseudo-graphics in some other systems. But
    my change shall be there anyway. Ncurses shall not be initialized before setlocale().

    Thank you, I'll update when it's pushed!

    Why would it break pseudo-graphics in other systems? Or are you just
    referring to other systems that don't actually use translation tables? :)

    Regards,
    Nick

    ... "Take my advice, I don't use it anyway."
    --- Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:115.0) Gecko/20100101 Thunderb
    * Origin: _thePharcyde distribution system (Wisconsin) (1:154/10)
  • From Nicholas Boel@1:154/10 to Wilfred van Velzen on Wednesday, March 06, 2024 17:15:02
    On Thu, 7 Mar 2024 04:36:10 +0100, Wilfred Van Velzen -> Vitaliy
    Aksyonov wrote:

    I pulled in the latest code with this change, and this breaks my configuration again. :-(

    So I reverted that change, and everything is back to normal...

    Not 100% sure on this, but maybe you were just one of the lucky few that
    never had to use translation tables. However, they should probably be
    used no matter what kind of terminal or setup you have.

    Nothing fancy. Just try including charsets.cfg, mess with 'xlatimport'
    and 'xlatlocalset' a bit and see what happens.

    Regards,
    Nick

    ... "Take my advice, I don't use it anyway."
    --- Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:115.0) Gecko/20100101 Thunderb
    * Origin: _thePharcyde distribution system (Wisconsin) (1:154/10)
  • From Nicholas Boel@1:154/10 to Vitaliy Aksyonov on Wednesday, March 06, 2024 17:23:20
    Hello Vitaliy,

    On Tue, 05 Mar 2024 21:53:14 -0700, you wrote to you:

    Pull request is sent. Wait for commit.

    Unfortunately it will break pseudo-graphics in some other systems. But
    my change shall be there anyway. Ncurses shall not be initialized
    before setlocale().

    The specific things you did for me are working as expected. Thank you!

    Regards,
    Nick

    ... "Take my advice, I don't use it anyway."
    --- GoldED+/LNX 1.1.5-b20240306
    * Origin: _thePharcyde distribution system (Wisconsin) (1:154/10)
  • From fusion@1:120/616 to Vitaliy Aksyonov on Wednesday, March 06, 2024 18:58:04
    Windows and Linux works with console very differently. Even support
    those two together is a challenge. Code is not very well encapsulated in GoldEd and OS specifics just everywhere.

    duno if it'd be useful but nftp (ayukov.com) had a little ui unit for win/lin/os2/etc .. if only to see other examples

    --- Mystic BBS v1.12 A47 2021/12/25 (Windows/32)
    * Origin: cold fusion - cfbbs.net - grand rapids, mi (1:120/616)
  • From Vitaliy Aksyonov@1:104/117 to Nicholas Boel on Thursday, March 07, 2024 07:02:18
    Hello Nicholas.

    06 Mar 24 17:06, you wrote to me:

    Pull request is sent. Wait for commit.

    Unfortunately it will break pseudo-graphics in some other
    systems. But my change shall be there anyway. Ncurses shall not
    be initialized before setlocale().

    Thank you, I'll update when it's pushed!

    Why would it break pseudo-graphics in other systems? Or are you just referring to other systems that don't actually use translation tables?
    :)

    It's not about translation tables, but about displaying the text. Ncurses has special logic to output pseudo-graphics and it most probably won't work correctly in Unicode locale. It was kinda "working" just because ncurses "thought" that it worked in "legacy" mode, not unicode.

    Vitaliy

    ... 640K ought to be enough for anybody
    --- GoldED+/LNX 1.1.5-b20240305-beta
    * Origin: Aurora, Colorado (1:104/117)
  • From Vitaliy Aksyonov@1:104/117 to Nicholas Boel on Thursday, March 07, 2024 07:04:00
    Hello Nicholas.

    06 Mar 24 17:23, you wrote to me:

    Pull request is sent. Wait for commit.

    Unfortunately it will break pseudo-graphics in some other
    systems. But my change shall be there anyway. Ncurses shall not
    be initialized before setlocale().

    The specific things you did for me are working as expected. Thank you!

    Glad it works for you. And anyway my change was necessary, because ncurses was not initialized properly before.

    Vitaliy

    ... 640K ought to be enough for anybody
    --- GoldED+/LNX 1.1.5-b20240305-beta
    * Origin: Aurora, Colorado (1:104/117)
  • From Vitaliy Aksyonov@1:104/117 to fusion on Thursday, March 07, 2024 07:05:08
    Hello fusion.

    06 Mar 24 18:58, you wrote to me:

    Windows and Linux works with console very differently. Even
    support those two together is a challenge. Code is not very well
    encapsulated in GoldEd and OS specifics just everywhere.

    duno if it'd be useful but nftp (ayukov.com) had a little ui unit for win/lin/os2/etc .. if only to see other examples

    Golded's editor has some FTN and own specifics, it's not just plain text editor. And not only editor a problem. GoldEd full of string manipulations. For example when it works with messages templates. Or uses clipboard. That's why while use some other editor with Unicode support won't solve all issues. Whole system need to be updated for Unicode.

    Vitaliy

    ... 640K ought to be enough for anybody
    --- GoldED+/LNX 1.1.5-b20240305-beta
    * Origin: Aurora, Colorado (1:104/117)
  • From Wilfred van Velzen@2:280/464 to Vitaliy Aksyonov on Thursday, March 07, 2024 18:55:17
    Hi Vitaliy,

    On 2024-03-06 14:59:02, you wrote to me:

    If you may try to add one line of code to latest master and try
    it - that would be helpful.

    goldlib/gcui/gkbdbase.cpp
    line 149, right before initscr add this:

    setlocale(LC_ALL, "");

    I pulled in the latest code with this change, and this breaks my
    configuration again. :-(

    So I reverted that change, and everything is back to normal...

    One of the changes is very similar to that one which was reverted previously.

    Remind me your setup please. Do you use UTF-8 locale?

    # sudo -u fido locale
    LANG=POSIX
    LC_CTYPE=en_US.UTF-8
    LC_NUMERIC="POSIX"
    LC_TIME="POSIX"
    LC_COLLATE="POSIX"
    LC_MONETARY="POSIX"
    LC_MESSAGES="POSIX"
    LC_PAPER="POSIX"
    LC_NAME="POSIX"
    LC_ADDRESS="POSIX"
    LC_TELEPHONE="POSIX"
    LC_MEASUREMENT="POSIX"
    LC_IDENTIFICATION="POSIX"
    LC_ALL=

    (I also start golded with 'sudo -u fido')

    But for my golded terminal in Konsole, I have the "Default character encoding" set to IBM850.

    What terminal do you use? TERM variable.

    # sudo -u fido echo $TERM
    xterm

    Do you use local console or remote/ssh?

    Both, but this test I only did in my local console.


    Bye, Wilfred.

    --- FMail-lnx64 2.2.1.1
    * Origin: FMail development HQ (2:280/464)
  • From Nicholas Boel@1:154/10 to Vitaliy Aksyonov on Thursday, March 07, 2024 17:09:40
    On Thu, 7 Mar 2024 13:02:18 -0700, Vitaliy Aksyonov -> Nicholas Boel wrote:

    Why would it break pseudo-graphics in other systems? Or are you just
    referring to other systems that don't actually use translation tables?
    :)

    It's not about translation tables, but about displaying the text.
    Ncurses has special logic to output pseudo-graphics and it most probably won't work correctly in Unicode locale. It was kinda "working" just because ncurses "thought" that it worked in "legacy" mode, not unicode.

    But, I'm the one using a unicode locale, and it is working. It seems to
    only affect people using a cp437/cp850-style single bit locale.

    Regards,
    Nick

    ... "Take my advice, I don't use it anyway."
    --- Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:115.0) Gecko/20100101 Thunderb
    * Origin: _thePharcyde distribution system (Wisconsin) (1:154/10)
  • From Vitaliy Aksyonov@1:104/117 to Nicholas Boel on Thursday, March 07, 2024 17:28:36
    Hello Nicholas.

    07 Mar 24 17:09, you wrote to me:

    Why would it break pseudo-graphics in other systems? Or are you
    just referring to other systems that don't actually use
    translation tables?
    :)

    It's not about translation tables, but about displaying the text.
    Ncurses has special logic to output pseudo-graphics and it most
    probably won't work correctly in Unicode locale. It was kinda
    "working" just because ncurses "thought" that it worked in
    "legacy" mode, not unicode.

    But, I'm the one using a unicode locale, and it is working. It seems
    to only affect people using a cp437/cp850-style single bit locale.

    For you it's "working". For him it shall work flawlessly. Answered in separate message.

    Vitaliy

    ... 640K ought to be enough for anybody
    --- GoldED+/LNX 1.1.5-b20240305-beta
    * Origin: Aurora, Colorado (1:104/117)
  • From Vitaliy Aksyonov@1:104/117 to Wilfred van Velzen on Thursday, March 07, 2024 17:30:02
    Hello Wilfred.

    07 Mar 24 18:55, you wrote to me:

    If you may try to add one line of code to latest master and try
    it - that would be helpful.

    goldlib/gcui/gkbdbase.cpp
    line 149, right before initscr add this:

    setlocale(LC_ALL, "");

    I pulled in the latest code with this change, and this breaks
    my configuration again. :-(

    So I reverted that change, and everything is back to normal...

    One of the changes is very similar to that one which was reverted
    previously.

    Remind me your setup please. Do you use UTF-8 locale?

    # sudo -u fido locale
    LANG=POSIX
    LC_CTYPE=en_US.UTF-8
    LC_NUMERIC="POSIX"
    LC_TIME="POSIX"
    LC_COLLATE="POSIX"
    LC_MONETARY="POSIX"
    LC_MESSAGES="POSIX"
    LC_PAPER="POSIX"
    LC_NAME="POSIX"
    LC_ADDRESS="POSIX"
    LC_TELEPHONE="POSIX"
    LC_MEASUREMENT="POSIX"
    LC_IDENTIFICATION="POSIX"
    LC_ALL=

    (I also start golded with 'sudo -u fido')

    But for my golded terminal in Konsole, I have the "Default character encoding" set to IBM850.

    OK. Your terminal works in IBM850, but you run golded may try to run in UTF-8. That won't work and it worked before just because golded was not initializing correctly!

    Question. What is your XLatLocalSet in GoldEd?

    What terminal do you use? TERM variable.

    # sudo -u fido echo $TERM
    xterm

    Do you use local console or remote/ssh?

    Both, but this test I only did in my local console.

    I don't expect any difference, but it may be some.

    Vitaliy

    ... 640K ought to be enough for anybody
    --- GoldED+/LNX 1.1.5-b20240305-beta
    * Origin: Aurora, Colorado (1:104/117)
  • From Wilfred van Velzen@2:280/464 to Vitaliy Aksyonov on Friday, March 08, 2024 09:02:51
    Hi Vitaliy,

    On 2024-03-07 17:30:02, you wrote to me:

    OK. Your terminal works in IBM850, but you run golded may try to run
    in UTF-8. That won't work and it worked before just because golded was
    not initializing correctly!

    Ok, so how do I fix it? ;)

    Question. What is your XLatLocalSet in GoldEd?

    I don't have any.


    Bye, Wilfred.

    --- FMail-lnx64 2.2.1.1
    * Origin: FMail development HQ (2:280/464)
  • From Wilfred van Velzen@2:280/464 to Vitaliy Aksyonov on Saturday, March 09, 2024 16:57:35
    Hi Vitaliy,

    On 2024-03-08 09:02:51, I wrote to you:

    OK. Your terminal works in IBM850, but you run golded may try to run
    in UTF-8. That won't work and it worked before just because golded
    was not initializing correctly!

    Ok, so how do I fix it? ;)

    Yesterday I did some experimenting with the latest golded (commit: "call setlocale() before initscr() (#86)"), trying different ways to start golded and different golded xlat configurations. I found 2 setups that work for me, so the messages with pseudo graphics in the STATS area look ok:

    My golded.cfg for both:

    include charsets.cfg

    XLATIMPORT CP437
    XLATLOCALSET CP850
    XLATEXPORT CP850


    And starting golded in a local Konsole terminal "TERM=xterm" with the default charset set to utf-8:

    # sudo -u fido LANG=en_EN.CP850 luit -encoding 'CP850' /usr/local/bin/golded -f


    And in the same kind of terminal but with the default charset set to cp850, this one works:

    # sudo -u fido LANG=en_US.CP850 /usr/local/bin/golded -f


    I will still have to test these in a putty terminal later during the coming week.

    And messages which have utf-8 characters, like the ones from Michiel in the UTF area, don't display properly (of course).
    I did try adding:

    XLATCHARSET UTF-8 CP850 utf8_850.chs

    But that didn't help. Of course only a small subset of utf-8 characters can be displayed with cp850...


    Bye, Wilfred.

    --- FMail-lnx64 2.2.1.1
    * Origin: FMail development HQ (2:280/464)
  • From Vitaliy Aksyonov@1:104/117 to Wilfred van Velzen on Saturday, March 09, 2024 12:49:34
    Hello Wilfred.

    09 Mar 24 16:57, you wrote to me:

    OK. Your terminal works in IBM850, but you run golded may try to
    run in UTF-8. That won't work and it worked before just because
    golded was not initializing correctly!

    Ok, so how do I fix it? ;)

    Yesterday I did some experimenting with the latest golded (commit:
    "call setlocale() before initscr() (#86)"), trying different ways to start golded and different golded xlat configurations. I found 2
    setups that work for me, so the messages with pseudo graphics in the STATS area look ok:

    My golded.cfg for both:

    include charsets.cfg

    XLATIMPORT CP437
    XLATLOCALSET CP850
    XLATEXPORT CP850

    Do not use xlatexport cp850. This parameter is used to select charset which will be used when you write messages to message base. Unless you really want cp850. I assume you want to have cp437 there.

    XlatImport on other side will use cp437 if message has no CHRS kludge.

    And starting golded in a local Konsole terminal "TERM=xterm" with the default charset set to utf-8:

    # sudo -u fido LANG=en_EN.CP850 luit -encoding 'CP850' /usr/local/bin/golded -f

    If you use luit - than having xterm in UTF-8 is correct.

    And in the same kind of terminal but with the default charset set to cp850, this one works:

    # sudo -u fido LANG=en_US.CP850 /usr/local/bin/golded -f

    This is correct commandline to if you have your terminal in cp850. Just choose what works better for you. Probably luit and terminal in UTF-8 is most convenient.

    I will still have to test these in a putty terminal later during the coming week.

    Putty will work fine. Only I suggest you to change terminal type in putty from xterm (they use if by default) to putty or putty-256color. It's in Connection->Data section.

    And messages which have utf-8 characters, like the ones from Michiel
    in the UTF area, don't display properly (of course). I did try adding:

    XLATCHARSET UTF-8 CP850 utf8_850.chs

    But that didn't help. Of course only a small subset of utf-8
    characters can be displayed with cp850...

    Golded cannot read from UTF-8. There is no Unicode support. You must use another editor or GoldEd with external editor to work with Unicode.

    It may convert from one-byte charsets to UTF-8, but not vise versa.

    Vitaliy

    ... 640K ought to be enough for anybody
    --- GoldED+/LNX 1.1.5-b20240305-beta
    * Origin: Aurora, Colorado (1:104/117)
  • From Wilfred van Velzen@2:280/464 to Vitaliy Aksyonov on Saturday, March 09, 2024 21:53:51
    Hi Vitaliy,

    On 2024-03-09 12:49:34, you wrote to me:

    Yesterday I did some experimenting with the latest golded (commit:
    "call setlocale() before initscr() (#86)"), trying different ways to
    start golded and different golded xlat configurations. I found 2
    setups that work for me, so the messages with pseudo graphics in the
    STATS area look ok:

    My golded.cfg for both:

    include charsets.cfg

    XLATIMPORT CP437
    XLATLOCALSET CP850
    XLATEXPORT CP850

    Do not use xlatexport cp850. This parameter is used to select charset which
    will be used when you write messages to message base. Unless you really want cp850. I assume you want to have cp437 there.

    My terminal uses cp850, so it's what I write my messages in. If I translate it to cp437, it will possibly loose some characters that are not supported in both. So I think I want it. ;-)

    And starting golded in a local Konsole terminal "TERM=xterm" with
    the default charset set to utf-8:

    # sudo -u fido LANG=en_EN.CP850 luit -encoding 'CP850'
    /usr/local/bin/golded -f

    If you use luit - than having xterm in UTF-8 is correct.

    And in the same kind of terminal but with the default charset set to
    cp850, this one works:

    # sudo -u fido LANG=en_US.CP850 /usr/local/bin/golded -f

    This is correct commandline to if you have your terminal in cp850. Just choose what works better for you. Probably luit and terminal in UTF-8 is most convenient.

    I'll see after I have tested with putty...

    I will still have to test these in a putty terminal later during the
    coming week.

    Putty will work fine. Only I suggest you to change terminal type in putty from xterm (they use if by default)

    I think it's set to 'linux' in putty.

    to putty or putty-256color. It's in Connection->Data section.

    Ok, I'll try that.

    And messages which have utf-8 characters, like the ones from Michiel
    in the UTF area, don't display properly (of course). I did try
    adding:

    XLATCHARSET UTF-8 CP850 utf8_850.chs

    But that didn't help. Of course only a small subset of utf-8
    characters can be displayed with cp850...

    Golded cannot read from UTF-8. There is no Unicode support. You must use another editor or GoldEd with external editor to work with Unicode.

    It may convert from one-byte charsets to UTF-8, but not vise versa.

    Isn't this for incoming messages in utf-8, to show them in your local charset, as far as possible?


    Bye, Wilfred.

    --- FMail-lnx64 2.2.1.1
    * Origin: FMail development HQ (2:280/464)
  • From Nicholas Boel@1:154/10 to Vitaliy Aksyonov on Sunday, March 10, 2024 14:56:14
    On Sat, 9 Mar 2024 18:49:34 -0700, Vitaliy Aksyonov -> Wilfred Van
    Velzen wrote:


    Golded cannot read from UTF-8. There is no Unicode support. You must
    use another editor or GoldEd with external editor to work with
    Unicode.

    It may convert from one-byte charsets to UTF-8, but not vise versa.

    We know it is unsupported, but I'm pretty sure I've said multiple times
    it can display in Golded just fine, with CHRS kludge, UTF-8 Cyrillic
    characters (MANY other characters display fine, also):

    https://pharcyde.org/golded-utf8.png

    It indeed can be done. The problem is everyone's terminals and what else is being used to run golded is different in every case.

    Regards,
    Nick

    ... "Take my advice, I don't use it anyway."
    --- Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:115.0) Gecko/20100101 Thunderb
    * Origin: _thePharcyde distribution system (Wisconsin) (1:154/10)
  • From Wilfred van Velzen@2:280/464 to Vitaliy Aksyonov on Monday, March 11, 2024 09:26:17
    Hi Vitaliy,

    On 2024-03-09 21:53:51, I wrote to you:

    And starting golded in a local Konsole terminal "TERM=xterm" with
    the default charset set to utf-8:

    # sudo -u fido LANG=en_EN.CP850 luit -encoding 'CP850'
    /usr/local/bin/golded -f

    If you use luit - than having xterm in UTF-8 is correct.

    This one also works with putty (which is set to use utf-8 as charset).

    And in the same kind of terminal but with the default charset set
    to cp850, this one works:

    # sudo -u fido LANG=en_US.CP850 /usr/local/bin/golded -f

    This is correct commandline to if you have your terminal in cp850.
    Just choose what works better for you. Probably luit and terminal in
    UTF-8 is most convenient.

    I'll see after I have tested with putty...

    I'll probably switch to this setup since it also works with putty.

    I will still have to test these in a putty terminal later during
    the coming week.

    Putty will work fine. Only I suggest you to change terminal type in
    putty from xterm (they use if by default)

    I think it's set to 'linux' in putty.

    Switching from 'linux' to 'putty' makes things worse:

    https://paste.opensuse.org/pastes/8a290fabf761

    Also some keys, like the F3 key stopped working. So I keep my terminal type as 'linux'.

    to putty or putty-256color. It's in Connection->Data section.

    'putty-256color' makes no difference.


    Bye, Wilfred.

    --- FMail-lnx64 2.2.1.1
    * Origin: FMail development HQ (2:280/464)
  • From Vitaliy Aksyonov@1:104/117 to Nicholas Boel on Tuesday, March 12, 2024 19:32:16
    Hello Nicholas.

    10 Mar 24 14:56, you wrote to me:

    Golded cannot read from UTF-8. There is no Unicode support. You
    must use another editor or GoldEd with external editor to work
    with Unicode.

    It may convert from one-byte charsets to UTF-8, but not vise
    versa.

    We know it is unsupported, but I'm pretty sure I've said multiple
    times it can display in Golded just fine, with CHRS kludge, UTF-8
    Cyrillic characters (MANY other characters display fine, also):

    https://pharcyde.org/golded-utf8.png

    We're trying to find, what could be wrong in GoldEd using ncurses. And it reproduces, so probably we'll find some solution.

    It indeed can be done. The problem is everyone's terminals and what
    else is being used to run golded is different in every case.

    I know. But there are many other problems, not just displaying unicode symbols, but also splitting long paragraphs of non-english letters. They may be 2-4 bytes long in UTF-8 and It will be weird in reader window.

    Vitaliy

    ... 640K ought to be enough for anybody
    --- GoldED+/LNX 1.1.5-b20240305-beta
    * Origin: Aurora, Colorado (1:104/117)
  • From Vitaliy Aksyonov@1:104/117 to Wilfred van Velzen on Tuesday, March 12, 2024 19:34:48
    Hello Wilfred.

    11 Mar 24 09:26, you wrote to me:

    And starting golded in a local Konsole terminal "TERM=xterm"
    with the default charset set to utf-8:

    # sudo -u fido LANG=en_EN.CP850 luit -encoding 'CP850'
    /usr/local/bin/golded -f

    If you use luit - than having xterm in UTF-8 is correct.

    This one also works with putty (which is set to use utf-8 as charset).

    Yep. I've tried luit in my configuration and it works totally fine.

    And in the same kind of terminal but with the default charset
    set to cp850, this one works:

    # sudo -u fido LANG=en_US.CP850 /usr/local/bin/golded -f

    This is correct commandline to if you have your terminal in
    cp850. Just choose what works better for you. Probably luit and
    terminal in UTF-8 is most convenient.

    I'll see after I have tested with putty...

    I'll probably switch to this setup since it also works with putty.

    It's definitely more convenient.

    I will still have to test these in a putty terminal later
    during the coming week.

    Putty will work fine. Only I suggest you to change terminal type
    in putty from xterm (they use if by default)

    I think it's set to 'linux' in putty.

    Switching from 'linux' to 'putty' makes things worse:

    https://paste.opensuse.org/pastes/8a290fabf761

    That may be because terminfo for putty doesn't exist on your system. You may need to install additional packages. I have putty-256color and it works perfect.

    How to check.
    Run infocmp -D
    This will show you which directories are used to find terminfo.
    Then search in those directories, is putty available. If not - you may need install package like ncurses-term. May have different name on your system.

    Let me know it that helped please.

    Also some keys, like the F3 key stopped working. So I keep my
    terminal
    type as 'linux'.

    to putty or putty-256color. It's in Connection->Data section.

    'putty-256color' makes no difference.

    putty-256color just add more colors support. F keys are broken most probably because terminfo is missing.

    Vitaliy

    ... 640K ought to be enough for anybody
    --- GoldED+/LNX 1.1.5-b20240305-beta
    * Origin: Aurora, Colorado (1:104/117)
  • From Wilfred van Velzen@2:280/464 to Vitaliy Aksyonov on Wednesday, March 13, 2024 08:45:13
    Hi Vitaliy,

    On 2024-03-12 19:34:48, you wrote to me:

    This is correct commandline to if you have your terminal in
    cp850. Just choose what works better for you. Probably luit and
    terminal in UTF-8 is most convenient.

    I'll see after I have tested with putty...

    I'll probably switch to this setup since it also works with putty.

    It's definitely more convenient.

    I changed to using luit, for both the putty and konsole terminals for golded.

    I do now have the problem that sometimes (not most of the time), the terminal is in strange state after closing golded. I've fixed that by adding the 'reset' command as the last command in the script that starts golded.

    Putty will work fine. Only I suggest you to change terminal type
    in putty from xterm (they use if by default)

    I think it's set to 'linux' in putty.

    Switching from 'linux' to 'putty' makes things worse:

    https://paste.opensuse.org/pastes/8a290fabf761

    That may be because terminfo for putty doesn't exist on your system. You may need to install additional packages. I have putty-256color and it works
    perfect.

    How to check.
    Run infocmp -D
    This will show you which directories are used to find terminfo.
    Then search in those directories, is putty available. If not - you may need
    install package like ncurses-term. May have different name on your system.

    I had already checked this:

    # ls -l /etc/termcap
    lrwxrwxrwx 1 root root 23 2015-10-29 21:23:18 termcap -> /usr/share/misc/termcap

    # grep putty /etc/termcap
    putty|PuTTY terminal emulator:\
    vt100-putty|Reset PuTTY to pure vt100:\
    putty-256color|PuTTY 0.58 with xterm 256-colors:\
    putty-vt100|VT100+ keyboard layout:\
    putty-sco|putty with SCO function keys:\


    infocmp shows more or less the same:

    # infocmp -D
    /usr/share/terminfo

    # find /usr/share/terminfo -iname 'putty*'
    /usr/share/terminfo/p/putty
    /usr/share/terminfo/p/putty-256color
    /usr/share/terminfo/p/putty-sco
    /usr/share/terminfo/p/putty-vt100

    Let me know it that helped please.

    So terminfo for putty already exists on my system...

    Also some keys, like the F3 key stopped working. So I keep my
    terminal type as 'linux'.

    to putty or putty-256color. It's in Connection->Data section.

    'putty-256color' makes no difference.

    putty-256color just add more colors support. F keys are broken most probably because terminfo is missing.

    That is not the case...


    Bye, Wilfred.

    --- FMail-lnx64 2.2.1.1
    * Origin: FMail development HQ (2:280/464)
  • From Vitaliy Aksyonov@1:104/117 to Wilfred van Velzen on Wednesday, March 13, 2024 14:19:04
    Hello Wilfred.

    13 Mar 24 08:45, you wrote to me:

    This is correct commandline to if you have your terminal in
    cp850. Just choose what works better for you. Probably luit
    and terminal in UTF-8 is most convenient.
    I'll see after I have tested with putty...
    I'll probably switch to this setup since it also works with
    putty.
    It's definitely more convenient.

    I changed to using luit, for both the putty and konsole terminals for golded.

    I do now have the problem that sometimes (not most of the time), the terminal is in strange state after closing golded. I've fixed that by adding the 'reset' command as the last command in the script that
    starts golded.

    It could be something wrong with ncurses deinit. I've never saw that. But I usually doesn't use session for anything else than GoldEd. And usually just log off.

    Putty will work fine. Only I suggest you to change terminal
    type in putty from xterm (they use if by default)

    I think it's set to 'linux' in putty.

    Switching from 'linux' to 'putty' makes things worse:

    https://paste.opensuse.org/pastes/8a290fabf761

    That may be because terminfo for putty doesn't exist on your
    system. You may need to install additional packages. I have
    putty-256color and it works perfect.

    How to check.
    Run infocmp -D
    This will show you which directories are used to find terminfo.
    Then search in those directories, is putty available. If not -
    you may need install package like ncurses-term. May have
    different name on your system.

    I had already checked this:

    # ls -l /etc/termcap
    lrwxrwxrwx 1 root root 23 2015-10-29 21:23:18 termcap -> /usr/share/misc/termcap

    # grep putty /etc/termcap
    putty|PuTTY terminal emulator:\
    vt100-putty|Reset PuTTY to pure vt100:\
    putty-256color|PuTTY 0.58 with xterm 256-colors:\
    putty-vt100|VT100+ keyboard layout:\
    putty-sco|putty with SCO function keys:\


    infocmp shows more or less the same:

    # infocmp -D
    /usr/share/terminfo

    # find /usr/share/terminfo -iname 'putty*'
    /usr/share/terminfo/p/putty
    /usr/share/terminfo/p/putty-256color
    /usr/share/terminfo/p/putty-sco
    /usr/share/terminfo/p/putty-vt100

    Let me know it that helped please.

    So terminfo for putty already exists on my system...

    Also some keys, like the F3 key stopped working. So I keep my
    terminal type as 'linux'.

    to putty or putty-256color. It's in Connection->Data section.

    'putty-256color' makes no difference.

    putty-256color just add more colors support. F keys are broken
    most probably because terminfo is missing.

    That is not the case...

    I spend a lot of time configuring my Putty to work correctly. I'd suggest you first make it working without luit. You may need to play with Putty's settings. And TERM=putty works the best for me. What I mean - it may work and it's just configuration thing. It should be totally fine for you too because you have correct termnifo.


    Vitaliy

    ... 640K ought to be enough for anybody
    --- GoldED+/LNX 1.1.5-b20240305-beta
    * Origin: Aurora, Colorado (1:104/117)
  • From Wilfred van Velzen@2:280/464 to Vitaliy Aksyonov on Wednesday, March 13, 2024 21:58:08
    Hi Vitaliy,

    On 2024-03-13 14:19:04, you wrote to me:

    putty-256color just add more colors support. F keys are broken
    most probably because terminfo is missing.

    That is not the case...

    I spend a lot of time configuring my Putty to work correctly. I'd suggest you first make it working without luit.

    Then I have to set my terminals to cp850 charset. I rather keep them as utf8

    You may need to play with Putty's settings. And TERM=putty works the
    best for me. What I mean - it may work and it's just configuration
    thing. It should be totally fine for you too because you have correct termnifo.

    I can spend a lot of time on it, or keep my current working config... I see no gain, spending the time.

    Bye, Wilfred.

    --- FMail-lnx64 2.2.1.1
    * Origin: FMail development HQ (2:280/464)
  • From Vitaliy Aksyonov@1:104/117 to Wilfred van Velzen on Wednesday, March 13, 2024 15:31:20
    Hello Wilfred.

    13 Mar 24 21:58, you wrote to me:

    putty-256color just add more colors support. F keys are broken
    most probably because terminfo is missing.

    That is not the case...

    I spend a lot of time configuring my Putty to work correctly. I'd
    suggest you first make it working without luit.

    Then I have to set my terminals to cp850 charset. I rather keep them
    as utf8

    Only temporary. F keys shall not be dependent on locale.

    You may need to play with Putty's settings. And TERM=putty works
    the best for me. What I mean - it may work and it's just
    configuration thing. It should be totally fine for you too
    because you have correct termnifo.

    I can spend a lot of time on it, or keep my current working config...
    I see no gain, spending the time.

    Sure. If that works for you. I'm just trying to help. :)

    Vitaliy

    ... 640K ought to be enough for anybody
    --- GoldED+/LNX 1.1.5-b20240305-beta
    * Origin: Aurora, Colorado (1:104/117)
  • From Wilfred van Velzen@2:280/464 to Vitaliy Aksyonov on Thursday, March 14, 2024 08:55:25
    Hi Vitaliy,

    On 2024-03-13 15:31:20, you wrote to me:

    I spend a lot of time configuring my Putty to work correctly. I'd
    suggest you first make it working without luit.

    Then I have to set my terminals to cp850 charset. I rather keep them
    as utf8

    Only temporary. F keys shall not be dependent on locale.

    I suspect it might have to do with different escape sequences, or handling of, for the function keys in the 'linux' and 'putty' settings.

    You may need to play with Putty's settings. And TERM=putty works
    the best for me. What I mean - it may work and it's just
    configuration thing. It should be totally fine for you too
    because you have correct termnifo.

    I can spend a lot of time on it, or keep my current working config...
    I see no gain, spending the time.

    Sure. If that works for you. I'm just trying to help. :)

    And that is highly appreciated!

    Bye, Wilfred.

    --- FMail-lnx64 2.2.1.1
    * Origin: FMail development HQ (2:280/464)
  • From Vitaliy Aksyonov@1:104/117 to Wilfred van Velzen on Thursday, March 14, 2024 07:19:32
    Hello Wilfred.

    14 Mar 24 08:55, you wrote to me:

    I spend a lot of time configuring my Putty to work correctly.
    I'd suggest you first make it working without luit.
    Then I have to set my terminals to cp850 charset. I rather keep
    them as utf8
    Only temporary. F keys shall not be dependent on locale.
    I suspect it might have to do with different escape sequences, or handling of, for the function keys in the 'linux' and 'putty'
    settings.

    You're completely right. That's the reason. And when you change terminal type, ncurses expects different escape sequences.

    My settings are:
    terminal type putty

    Then in Terminal->Keyboard:

    The Backspace key -> Control-?
    The Home and End keys -> Standard
    The Function keys and keypad -> ESC[n~
    Shift/Ctrl/Alt with the arrow keys -> Ctrl toggles app mode
    Initial state of cursor keys -> Normal
    Initial state of numeric keypad -> Normal
    AltGf acts as Compose key -> off
    Control-Alt is different from AltGf -> on

    In Terminal->Features all is off except "Disable application keypad mode"

    You may need to play with Putty's settings. And TERM=putty
    works the best for me. What I mean - it may work and it's just
    configuration thing. It should be totally fine for you too
    because you have correct termnifo.

    I can spend a lot of time on it, or keep my current working
    config... I see no gain, spending the time.
    Sure. If that works for you. I'm just trying to help. :)
    And that is highly appreciated!

    Try my settings. It could work.

    Vitaliy

    ... 640K ought to be enough for anybody
    --- GoldED+/LNX 1.1.5-b20240305-beta
    * Origin: Aurora, Colorado (1:104/117)
  • From Wilfred van Velzen@2:280/464 to Vitaliy Aksyonov on Thursday, March 14, 2024 15:25:54
    Hi Vitaliy,

    On 2024-03-14 07:19:32, you wrote to me:

    I suspect it might have to do with different escape sequences, or
    handling of, for the function keys in the 'linux' and 'putty'
    settings.

    You're completely right. That's the reason. And when you change terminal type, ncurses expects different escape sequences.

    My settings are:
    terminal type putty

    Then in Terminal->Keyboard:

    The Backspace key -> Control-?
    The Home and End keys -> Standard
    The Function keys and keypad -> ESC[n~

    That was the one that was different for me.

    I also had to Disable remote-controled terminal resizing, otherwise my terminal would resize to 80 chars wide after closing golded.

    Shift/Ctrl/Alt with the arrow keys -> Ctrl toggles app mode
    Initial state of cursor keys -> Normal
    Initial state of numeric keypad -> Normal
    AltGf acts as Compose key -> off
    Control-Alt is different from AltGf -> on

    In Terminal->Features all is off except "Disable application keypad mode"

    Try my settings. It could work.

    It fixed the keyboard problems. But not how golded looks. The line drawing characters are still the 'qqqqqq' kind.
    It doesn't matter if I use luit or not.

    What did help, was enabling in Window/Tranlation: Enable VT100 line drawing even in UTF-8 mode
    So now I can use the 'putty-256color' terminal type. :-)


    Bye, Wilfred.

    --- FMail-lnx64 2.2.1.1
    * Origin: FMail development HQ (2:280/464)
  • From Vitaliy Aksyonov@1:104/117 to Wilfred van Velzen on Thursday, March 14, 2024 10:32:22
    Hello Wilfred.

    14 Mar 24 15:25, you wrote to me:

    I suspect it might have to do with different escape sequences,
    or handling of, for the function keys in the 'linux' and
    'putty' settings.

    You're completely right. That's the reason. And when you change
    terminal type, ncurses expects different escape sequences.

    My settings are:
    terminal type putty

    Then in Terminal->Keyboard:

    The Backspace key -> Control-?
    The Home and End keys -> Standard
    The Function keys and keypad -> ESC[n~

    That was the one that was different for me.

    I also had to Disable remote-controled terminal resizing, otherwise my terminal would resize to 80 chars wide after closing golded.

    Good.

    Shift/Ctrl/Alt with the arrow keys -> Ctrl toggles app mode
    Initial state of cursor keys -> Normal
    Initial state of numeric keypad -> Normal
    AltGf acts as Compose key -> off
    Control-Alt is different from AltGf -> on

    In Terminal->Features all is off except "Disable application
    keypad mode"

    Try my settings. It could work.

    It fixed the keyboard problems. But not how golded looks. The line drawing characters are still the 'qqqqqq' kind. It doesn't matter if I use luit or not.

    What did help, was enabling in Window/Tranlation: Enable VT100 line drawing even in UTF-8 mode So now I can use the 'putty-256color'
    terminal type. :-)

    The only difference that I don't run golded in UTF, that's why you had to change this parameter. Glad it works for you!

    Vitaliy

    ... 640K ought to be enough for anybody
    --- GoldED+/LNX 1.1.5-b20240305-beta
    * Origin: Aurora, Colorado (1:104/117)
  • From Wilfred van Velzen@2:280/464 to Vitaliy Aksyonov on Thursday, March 14, 2024 17:37:08
    Hi Vitaliy,

    On 2024-03-14 10:32:22, you wrote to me:

    What did help, was enabling in Window/Tranlation: Enable VT100 line
    drawing even in UTF-8 mode So now I can use the 'putty-256color'
    terminal type. :-)

    The only difference that I don't run golded in UTF, that's why you had to change this parameter. Glad it works for you!

    My terminal is UTF, bug I start golded with 'LANG=en_EN.CP850 luit -encoding 'CP850'', so I don't run golded in UTF either. Or do you mean something else?

    Bye, Wilfred.

    --- FMail-lnx64 2.2.1.1
    * Origin: FMail development HQ (2:280/464)
  • From Nil Alexandrov@2:5015/46 to Wilfred van Velzen on Thursday, March 14, 2024 20:03:22
    Hello, Wilfred!

    Thursday March 14 2024 17:37, from Wilfred van Velzen -> Vitaliy Aksyonov:

    My terminal is UTF, bug I start golded with 'LANG=en_EN.CP850 luit -encoding 'CP850'', so I don't run golded in UTF either. Or do you
    mean something else?

    Make sure your luit has that encoding.
    $ luit -list | grep CP850

    Mine doesn't
    $ luit -list | grep -ic CP850
    0

    Best Regards, Nil
    --- GoldED+/LNX 1.1.5
    * Origin: Linux 2.6.32-042stab145.3 (2:5015/46)
  • From Wilfred van Velzen@2:280/464 to Nil Alexandrov on Thursday, March 14, 2024 18:08:11
    Hi Nil,

    On 2024-03-14 20:03:22, you wrote to me:

    My terminal is UTF, bug I start golded with 'LANG=en_EN.CP850 luit
    -encoding 'CP850'', so I don't run golded in UTF either. Or do you
    mean something else?

    Make sure your luit has that encoding.
    $ luit -list | grep CP850

    # luit -list | grep CP850
    CP850: GL -> G0, GR -> G2, G0: ASCII, G2: CP 850

    And otherwise it probably wouldn't have worked. ;-)

    But thanks for the tip!


    Bye, Wilfred.

    --- FMail-lnx64 2.2.1.1
    * Origin: FMail development HQ (2:280/464)
  • From Vitaliy Aksyonov@1:104/117 to Wilfred van Velzen on Thursday, March 14, 2024 12:29:14
    Hello Wilfred.

    14 Mar 24 17:37, you wrote to me:

    What did help, was enabling in Window/Tranlation: Enable VT100
    line drawing even in UTF-8 mode So now I can use the
    'putty-256color' terminal type. :-)

    The only difference that I don't run golded in UTF, that's why
    you had to change this parameter. Glad it works for you!

    My terminal is UTF, bug I start golded with 'LANG=en_EN.CP850 luit -encoding 'CP850'', so I don't run golded in UTF either. Or do you
    mean something else?

    Yep. That is what I meant. Anyway I glad it works for you now.

    Vitaliy

    ... 640K ought to be enough for anybody
    --- GoldED+/LNX 1.1.5-b20240305-beta
    * Origin: Aurora, Colorado (1:104/117)
  • From Wilfred van Velzen@2:280/464 to Vitaliy Aksyonov on Friday, March 22, 2024 09:38:52
    Hi Vitaliy,

    On 2024-03-14 12:29:14, you wrote to me:

    What did help, was enabling in Window/Tranlation: Enable VT100
    line drawing even in UTF-8 mode So now I can use the
    'putty-256color' terminal type. :-)

    The only difference that I don't run golded in UTF, that's why
    you had to change this parameter. Glad it works for you!

    My terminal is UTF, bug I start golded with 'LANG=en_EN.CP850 luit
    -encoding 'CP850'', so I don't run golded in UTF either. Or do you
    mean something else?

    Yep. That is what I meant. Anyway I glad it works for you now.

    I'm still seeing an issue with some messages in German areas. For instance:

    https://paste.opensuse.org/pastes/0cc6a7ecd695

    Some characters are displayed as '~A', where they should be able to be displayed in their right form, while others are displayed correctly.

    The two High ascii characters in the marked area are:

    \x81 ; latin small letter u with diaeresis
    \xE1 ; greek small letter beta (here used as german ss)

    The last character in the line, an 'e', isn't even displayed, probably because of the expansion of 1 of the characters to 2 characters.

    These characters are the same in CP437 and CP850. So my CP850 terminal should display them correctly.
    It seems to come from Golded itself, or maybe the ncurses library, because it doesn't matter if I use luit for starting Golded or not, or running it in my remote putty terminal, or my local konsole terminal. The \x81 character is always displayed as ~A ... Strange!?


    Bye, Wilfred.

    --- FMail-lnx64 2.3.0.1-B20240319
    * Origin: FMail development HQ (2:280/464)
  • From Wilfred van Velzen@2:280/464 to Vitaliy Aksyonov on Friday, March 22, 2024 10:45:16
    Hi Vitaliy,

    On 2024-03-22 09:38:52, I wrote to you:

    I'm still seeing an issue with some messages in German areas. For instance:

    https://paste.opensuse.org/pastes/0cc6a7ecd695

    Some characters are displayed as '~A', where they should be able to be displayed in their right form, while others are displayed correctly.

    The two High ascii characters in the marked area are:

    \x81 ; latin small letter u with diaeresis
    \xE1 ; greek small letter beta (here used as german ss)

    The last character in the line, an 'e', isn't even displayed, probably because
    of the expansion of 1 of the characters to 2 characters.

    These characters are the same in CP437 and CP850. So my CP850 terminal should
    display them correctly. It seems to come from Golded itself, or maybe the ncurses library, because it doesn't matter if I use luit for starting Golded
    or not, or running it in my remote putty terminal, or my local konsole terminal. The \x81 character is always displayed as ~A ... Strange!?

    Btw: My terminal seems fine with displaying the CP850 high ascii characters (despite the warning):

    https://paste.opensuse.org/pastes/8bcb9d2ecfdf


    Bye, Wilfred.

    --- FMail-lnx64 2.3.0.1-B20240319
    * Origin: FMail development HQ (2:280/464)
  • From Vitaliy Aksyonov@1:104/117 to Wilfred van Velzen on Friday, March 22, 2024 07:04:30
    Hello Wilfred.

    22 Mar 24 10:45, you wrote to me:

    I'm still seeing an issue with some messages in German areas.
    For
    instance:

    https://paste.opensuse.org/pastes/0cc6a7ecd695

    Some characters are displayed as '~A', where they should be able
    to be displayed in their right form, while others are displayed
    correctly.

    The two High ascii characters in the marked area are:

    \x81 ; latin small letter u with diaeresis
    \xE1 ; greek small letter beta (here used as german ss)

    The last character in the line, an 'e', isn't even displayed,
    probably because of the expansion of 1 of the characters to 2
    characters.

    These characters are the same in CP437 and CP850. So my CP850
    terminal should display them correctly. It seems to come from
    Golded itself, or maybe the ncurses library, because it doesn't
    matter if I use luit for starting Golded or not, or running it
    in my remote putty terminal, or my local konsole terminal. The
    \x81 character is always displayed as ~A ... Strange!?

    Btw: My terminal seems fine with displaying the CP850 high ascii characters (despite the warning):

    https://paste.opensuse.org/pastes/8bcb9d2ecfdf

    Looks like your luit doesn't support CP850 or you don't have en_US.CP850. There are some encodings which luit list, but doesn't support actually. For example, mine lists CP866, but doesn't work with it.

    Does it present in `locale -a` output?

    Have you tried to run that script without luit? You don't even need to change locale for it - just encoding in terminal.

    Vitaliy

    ... 640K ought to be enough for anybody
    --- GoldED+/LNX 1.1.5-b20240305-beta
    * Origin: Aurora, Colorado (1:104/117)
  • From Wilfred van Velzen@2:280/464 to Vitaliy Aksyonov on Saturday, March 23, 2024 13:57:19
    Hi Vitaliy,

    On 2024-03-22 07:04:30, you wrote to me:

    Btw: My terminal seems fine with displaying the CP850 high ascii
    characters (despite the warning):

    https://paste.opensuse.org/pastes/8bcb9d2ecfdf

    Looks like your luit doesn't support CP850 or you don't have en_US.CP850. There are some encodings which luit list, but doesn't support actually. For
    example, mine lists CP866, but doesn't work with it.

    Does it present in `locale -a` output?

    I don't know if that says much, because mostly there are just the xx_XX and xx_XX.utf8 versions of the encodings. To give you a sample:

    # locale -a | grep en_
    en_AG
    en_AU
    en_AU.utf8
    en_BE
    en_BE.utf8
    en_BE@euro
    en_BW
    en_BW.utf8
    en_CA
    en_CA.utf8
    en_DK
    en_DK.utf8
    en_GB
    en_GB.iso885915
    en_GB.utf8
    en_HK
    en_HK.utf8
    en_IE
    en_IE.utf8
    en_IE@euro
    en_IN
    en_NG
    en_NZ
    en_NZ.utf8
    en_PH
    en_PH.utf8
    en_SG
    en_SG.utf8
    en_US
    en_US.iso885915
    en_US.utf8
    en_ZA
    en_ZA.utf8
    en_ZM
    en_ZW
    en_ZW.utf8

    Does this mean there is just an utf8 charset and an unspecified one for almost every language-country? That doesn't seem logical!

    Doesn't this show you what encodings luit supports:

    # luit -list
    Known locale encodings:

    C: GL -> G0, GR -> G2, G0: ASCII, G2: ISO 8859-1
    POSIX: GL -> G0, GR -> G2, G0: ASCII, G2: ISO 8859-1
    US-ASCII: GL -> G0, GR -> G2, G0: ASCII, G2: ISO 8859-1
    ...
    CP850: GL -> G0, GR -> G2, G0: ASCII, G2: CP 850
    ...

    Known charsets (not all may be available):

    ISO 646 (1973) (ISO 2022, 94 codes)
    ASCII (ISO 2022, 94 codes)
    ...
    CP 437 (128 codes)
    CP 850 (128 codes)
    CP 852 (128 codes)
    ...

    So luit seems to know about CP850...


    Have you tried to run that script without luit? You don't even need to change locale for it - just encoding in terminal.

    https://paste.opensuse.org/pastes/574e349fadaf

    So without luit it doesn't display anything useful. With luit, although you get the warning, it does display the right characters for CP850 !?


    Bye, Wilfred.

    --- FMail-lnx64 2.3.0.1-B20240319
    * Origin: FMail development HQ (2:280/464)
  • From Vitaliy Aksyonov@1:104/117 to Wilfred van Velzen on Sunday, March 24, 2024 11:12:14
    Hello Wilfred.

    23 Mar 24 13:57, you wrote to me:

    Btw: My terminal seems fine with displaying the CP850 high
    ascii characters (despite the warning):

    https://paste.opensuse.org/pastes/8bcb9d2ecfdf

    Looks like your luit doesn't support CP850 or you don't have
    en_US.CP850. There are some encodings which luit list, but
    doesn't support actually. For example, mine lists CP866, but
    doesn't work with it.

    Does it present in `locale -a` output?

    I don't know if that says much, because mostly there are just the
    xx_XX and xx_XX.utf8 versions of the encodings. To give you a sample:

    # locale -a | grep en_

    [...skipped...]


    Does this mean there is just an utf8 charset and an unspecified one
    for almost every language-country? That doesn't seem logical!

    It just mean that your system doesn't have necessary locale installed. And that perfectly explain why luit shows all chars, but GoldEd doesn't.

    Luit converts them using internal tables to UTF, but GoldEd tries to use en_EN.CP850, which is missing. That's why it doesn't understand that letters with codes > 127 are letters.

    You need to install or generate this locale and GoldEd will show those letters! What Linux distribution do you use? Do you need help with locale generation?

    Doesn't this show you what encodings luit supports:

    # luit -list
    Known locale encodings:

    C: GL -> G0, GR -> G2, G0: ASCII, G2: ISO 8859-1
    POSIX: GL -> G0, GR -> G2, G0: ASCII, G2: ISO 8859-1
    US-ASCII: GL -> G0, GR -> G2, G0: ASCII, G2: ISO 8859-1
    ...
    CP850: GL -> G0, GR -> G2, G0: ASCII, G2: CP 850
    ...

    Known charsets (not all may be available):

    ISO 646 (1973) (ISO 2022, 94 codes)
    ASCII (ISO 2022, 94 codes)
    ...
    CP 437 (128 codes)
    CP 850 (128 codes)
    CP 852 (128 codes)
    ...

    So luit seems to know about CP850...

    Yes. And that's good!

    Have you tried to run that script without luit? You don't even
    need to change locale for it - just encoding in terminal.

    https://paste.opensuse.org/pastes/574e349fadaf

    So without luit it doesn't display anything useful. With luit,
    although you get the warning, it does display the right characters for CP850 !?

    Yep. I meant to run that script without luit in terminal configured for CP850. But you don't need to do that anymore as long as we found root cause already.

    Vitaliy

    ... 640K ought to be enough for anybody
    --- GoldED+/LNX 1.1.5-b20240305-beta
    * Origin: Aurora, Colorado (1:104/117)
  • From Wilfred van Velzen@2:280/464 to Vitaliy Aksyonov on Monday, March 25, 2024 09:01:17
    Hi Vitaliy,

    On 2024-03-24 11:12:14, you wrote to me:

    You need to install or generate this locale and GoldEd will show those letters! What Linux distribution do you use?

    wilnux5:/etc # cat os-release
    NAME="openSUSE Leap"
    VERSION="42.1"
    VERSION_ID="42.1"
    PRETTY_NAME="openSUSE Leap 42.1 (x86_64)"
    ID=opensuse
    ANSI_COLOR="0;32"
    CPE_NAME="cpe:/o:opensuse:opensuse:42.1" BUG_REPORT_URL="https://bugs.opensuse.org"
    HOME_URL="https://opensuse.org/"
    ID_LIKE="suse"

    (Yes, it's old ;-))

    Do you need help with locale generation?

    I think I found out how to do this on my system.

    First I used my systems package manager to install: "glibc-i18ndata - Database Sources for 'locale'"

    Afterwards this command ran without any output:

    # localedef --no-archive -f IBM850 -i en_US en_US.CP850
    #

    And this directory was created with contents:

    /usr/lib/locale/en_US.cp850

    And 'locale -a -v' now shows:
    ...
    locale: en_US directory: /usr/lib/locale/en_US -------------------------------------------------------------------------------
    title | English locale for the USA
    source | Free Software Foundation, Inc.
    address | http://www.gnu.org/software/libc/
    email | bug-glibc-locales@gnu.org
    language | English
    territory | USA
    revision | 1.0
    date | 2000-06-24
    codeset | ISO-8859-1

    locale: en_US.cp850 directory: /usr/lib/locale/en_US.cp850 -------------------------------------------------------------------------------
    title | English locale for the USA
    source | Free Software Foundation, Inc.
    address | http://www.gnu.org/software/libc/
    email | bug-glibc-locales@gnu.org
    language | English
    territory | USA
    revision | 1.0
    date | 2000-06-24
    codeset | IBM850
    ...

    But golded output is borked now, in my current utf-8 configured putty terminal:

    https://paste.opensuse.org/pastes/2a5c7f2fbdc4

    It doesn't matter if I use luit or not, they are displayed the same.
    Also the ~A characters for messages with CHRS: CP437 in the german areas are still there.

    And this is still the same:

    https://paste.opensuse.org/pastes/f3961b7ea085


    Bye, Wilfred.

    --- FMail-lnx64 2.3.0.1-B20240319
    * Origin: FMail development HQ (2:280/464)
  • From Vitaliy Aksyonov@1:104/117 to Wilfred van Velzen on Monday, March 25, 2024 07:14:50
    Hello Wilfred.

    25 Mar 24 09:01, you wrote to me:

    You need to install or generate this locale and GoldEd will show
    those letters! What Linux distribution do you use?

    wilnux5:/etc # cat os-release
    NAME="openSUSE Leap"
    VERSION="42.1"
    VERSION_ID="42.1"
    PRETTY_NAME="openSUSE Leap 42.1 (x86_64)"
    ID=opensuse
    ANSI_COLOR="0;32"
    CPE_NAME="cpe:/o:opensuse:opensuse:42.1" BUG_REPORT_URL="https://bugs.opensuse.org" HOME_URL="https://opensuse.org/"
    ID_LIKE="suse"

    (Yes, it's old ;-))

    Got it. Not too old for FidoNet. :)

    Do you need help with locale generation?

    I think I found out how to do this on my system.

    First I used my systems package manager to install: "glibc-i18ndata - Database Sources for 'locale'"

    Afterwards this command ran without any output:

    # localedef --no-archive -f IBM850 -i en_US en_US.CP850
    #

    And this directory was created with contents:

    /usr/lib/locale/en_US.cp850

    And 'locale -a -v' now shows:
    ...
    locale: en_US directory: /usr/lib/locale/en_US ---------------------------------------------------------------------- ---------
    title | English locale for the USA
    source | Free Software Foundation, Inc.
    address | http://www.gnu.org/software/libc/
    email | bug-glibc-locales@gnu.org
    language | English
    territory | USA
    revision | 1.0
    date | 2000-06-24
    codeset | ISO-8859-1

    locale: en_US.cp850 directory: /usr/lib/locale/en_US.cp850 ---------------------------------------------------------------------- ---------
    title | English locale for the USA
    source | Free Software Foundation, Inc.
    address | http://www.gnu.org/software/libc/
    email | bug-glibc-locales@gnu.org
    language | English
    territory | USA
    revision | 1.0
    date | 2000-06-24
    codeset | IBM850
    ...

    This is what you need. Good.

    But golded output is borked now, in my current utf-8 configured putty terminal:

    https://paste.opensuse.org/pastes/2a5c7f2fbdc4

    This is exactly how I saw it on my computer, when was using pseudo-graphics with wrong or missing locale.

    It doesn't matter if I use luit or not, they are displayed the same.
    Also the ~A characters for messages with CHRS: CP437 in the german
    areas are still there.

    And this is still the same:

    https://paste.opensuse.org/pastes/f3961b7ea085

    This looks like locale is not used. I saw such pictures when Strange.

    Do you use latest GoldEd build? Do you still run it with LANG=en_US.cp850? In some message I saw en_EN.CP850.

    Could you also run:
    LANG=en_US.cp850 locale

    Vitaliy

    ... 640K ought to be enough for anybody
    --- GoldED+/LNX 1.1.5-b20240305-beta
    * Origin: Aurora, Colorado (1:104/117)
  • From Wilfred van Velzen@2:280/464 to Vitaliy Aksyonov on Monday, March 25, 2024 15:13:16
    Hi Vitaliy,

    On 2024-03-25 07:14:50, you wrote to me:

    PRETTY_NAME="openSUSE Leap 42.1 (x86_64)"

    (Yes, it's old ;-))

    Got it. Not too old for FidoNet. :)

    Nope. ;-)

    locale: en_US.cp850 directory: /usr/lib/locale/en_US.cp850

    This is what you need. Good.

    But golded output is borked now, in my current utf-8 configured putty
    terminal:

    https://paste.opensuse.org/pastes/2a5c7f2fbdc4

    This is exactly how I saw it on my computer, when was using pseudo-graphics
    with wrong or missing locale.

    The locale is there, so is it wrong?

    It doesn't matter if I use luit or not, they are displayed the same.
    Also the ~A characters for messages with CHRS: CP437 in the german
    areas are still there.

    And this is still the same:

    https://paste.opensuse.org/pastes/f3961b7ea085

    This looks like locale is not used. I saw such pictures when Strange.

    Do you use latest GoldEd build?

    Almost. I'm using this one:

    commit 4b6c754756d0fa96c0c3210d6ed0b63d49ec8e6a
    Author: Vitaliy Aksyonov <18148062+vitaliy-aksyonov@users.noreply.github.com> Date: Wed Mar 6 13:38:52 2024 -0700

    call setlocale() before initscr() (#86)

    See section Initialization in man 3 ncurses.
    https://www.man7.org/linux/man-pages/man3/ncurses.3x.html
    Locale shall be initialized before ncurses initialization.

    The latest one doesn't seem so change anything regarding screen output.

    Do you still run it with LANG=en_US.cp850?

    Yes.

    In some message I saw en_EN.CP850.

    The localdef command, created the directory with lowercase 'cp850' although I specified it with uppercase 'CP850'. It also shows it with lowercase 'cp' when locale -a is executed. So I switched to specifying it as lowercase in my golded start script. But case probably doesn't matter.

    Could you also run:
    LANG=en_US.cp850 locale

    Here are some tries:

    wilnux5:/home/fido/log # locale
    LANG=POSIX
    LC_CTYPE=en_US.UTF-8
    LC_NUMERIC="POSIX"
    LC_TIME="POSIX"
    LC_COLLATE="POSIX"
    LC_MONETARY="POSIX"
    LC_MESSAGES="POSIX"
    LC_PAPER="POSIX"
    LC_NAME="POSIX"
    LC_ADDRESS="POSIX"
    LC_TELEPHONE="POSIX"
    LC_MEASUREMENT="POSIX"
    LC_IDENTIFICATION="POSIX"
    LC_ALL=

    wilnux5:/home/fido/log # LANG=en_US.cp850 locale
    LANG=en_US.cp850
    LC_CTYPE=en_US.UTF-8
    LC_NUMERIC="en_US.cp850"
    LC_TIME="en_US.cp850"
    LC_COLLATE="en_US.cp850"
    LC_MONETARY="en_US.cp850"
    LC_MESSAGES="en_US.cp850"
    LC_PAPER="en_US.cp850"
    LC_NAME="en_US.cp850"
    LC_ADDRESS="en_US.cp850"
    LC_TELEPHONE="en_US.cp850"
    LC_MEASUREMENT="en_US.cp850"
    LC_IDENTIFICATION="en_US.cp850"
    LC_ALL=

    wilnux5:/home/fido/log # LANG=en_US.CP850 locale
    LANG=en_US.CP850
    LC_CTYPE=en_US.UTF-8
    LC_NUMERIC="en_US.CP850"
    LC_TIME="en_US.CP850"
    LC_COLLATE="en_US.CP850"
    LC_MONETARY="en_US.CP850"
    LC_MESSAGES="en_US.CP850"
    LC_PAPER="en_US.CP850"
    LC_NAME="en_US.CP850"
    LC_ADDRESS="en_US.CP850"
    LC_TELEPHONE="en_US.CP850"
    LC_MEASUREMENT="en_US.CP850"
    LC_IDENTIFICATION="en_US.CP850"
    LC_ALL=

    wilnux5:/home/fido/log # sudo -u fido LANG=en_US.cp850 locale
    LANG=en_US.cp850
    LC_CTYPE=en_US.UTF-8
    LC_NUMERIC="en_US.cp850"
    LC_TIME="en_US.cp850"
    LC_COLLATE="en_US.cp850"
    LC_MONETARY="en_US.cp850"
    LC_MESSAGES="en_US.cp850"
    LC_PAPER="en_US.cp850"
    LC_NAME="en_US.cp850"
    LC_ADDRESS="en_US.cp850"
    LC_TELEPHONE="en_US.cp850"
    LC_MEASUREMENT="en_US.cp850"
    LC_IDENTIFICATION="en_US.cp850"
    LC_ALL=


    Bye, Wilfred.

    --- FMail-lnx64 2.3.0.1-B20240319
    * Origin: FMail development HQ (2:280/464)
  • From Vitaliy Aksyonov@1:104/117 to Wilfred van Velzen on Monday, March 25, 2024 11:09:34
    Hello Wilfred.

    25 Mar 24 15:13, you wrote to me:

    locale: en_US.cp850 directory: /usr/lib/locale/en_US.cp850
    This is what you need. Good.
    But golded output is borked now, in my current utf-8 configured
    putty
    terminal:
    https://paste.opensuse.org/pastes/2a5c7f2fbdc4
    This is exactly how I saw it on my computer, when was using
    pseudo-graphics with wrong or missing locale.
    The locale is there, so is it wrong?

    I have no idea. It looks correct. But output looks like ncurses uses incorrect locale.

    It doesn't matter if I use luit or not, they are displayed the
    same. Also the ~A characters for messages with CHRS: CP437 in
    the german areas are still there.

    Remind me. Do you have XLAT conversion table from cp437 to cp850?

    And this is still the same:

    https://paste.opensuse.org/pastes/f3961b7ea085

    This looks like locale is not used. I saw such pictures when
    Strange.

    Do you use latest GoldEd build?

    Almost. I'm using this one:

    commit 4b6c754756d0fa96c0c3210d6ed0b63d49ec8e6a
    Author: Vitaliy Aksyonov <18148062+vitaliy-aksyonov@users.noreply.github.com>
    Date: Wed Mar 6 13:38:52 2024 -0700

    call setlocale() before initscr() (#86)

    See section Initialization in man 3 ncurses.
    https://www.man7.org/linux/man-pages/man3/ncurses.3x.html
    Locale shall be initialized before ncurses initialization.

    The latest one doesn't seem so change anything regarding screen
    output.

    Looks like incorrect locale to me again.

    Do you still run it with LANG=en_US.cp850?
    Yes.

    In some message I saw en_EN.CP850.

    The localdef command, created the directory with lowercase 'cp850' although I specified it with uppercase 'CP850'. It also shows it with lowercase 'cp' when locale -a is executed. So I switched to specifying
    it as lowercase in my golded start script. But case probably doesn't matter.

    No, I mean that i saw you using en_*EN*.cp850, not en_*US*.cp850. That is important.

    Could you also run:
    LANG=en_US.cp850 locale

    Here are some tries:

    wilnux5:/home/fido/log # locale
    LANG=POSIX
    LC_CTYPE=en_US.UTF-8
    ^^^^^
    This is not correct

    LC_NUMERIC="POSIX"
    LC_TIME="POSIX"
    LC_COLLATE="POSIX"
    LC_MONETARY="POSIX"
    LC_MESSAGES="POSIX"
    LC_PAPER="POSIX"
    LC_NAME="POSIX"
    LC_ADDRESS="POSIX"
    LC_TELEPHONE="POSIX"
    LC_MEASUREMENT="POSIX"
    LC_IDENTIFICATION="POSIX"
    ^^^^^^^
    These guys too.
    LC_ALL=

    [...skipped...]

    Make sure that all LC_-s are en_US.cp850. Especially LC_CTYPE.

    Vitaliy

    ... 640K ought to be enough for anybody
    --- GoldED+/LNX 1.1.5-b20240305-beta
    * Origin: Aurora, Colorado (1:104/117)
  • From Wilfred van Velzen@2:280/464 to Vitaliy Aksyonov on Monday, March 25, 2024 21:18:06
    Hi Vitaliy,

    On 2024-03-25 11:09:34, you wrote to me:

    This is exactly how I saw it on my computer, when was using
    pseudo-graphics with wrong or missing locale.
    The locale is there, so is it wrong?

    I have no idea. It looks correct. But output looks like ncurses uses incorrect
    locale.

    It seems so.

    It doesn't matter if I use luit or not, they are displayed the
    same. Also the ~A characters for messages with CHRS: CP437 in
    the german areas are still there.

    Remind me. Do you have XLAT conversion table from cp437 to cp850?

    Yes.

    In some message I saw en_EN.CP850.

    The localdef command, created the directory with lowercase 'cp850'
    although I specified it with uppercase 'CP850'. It also shows it
    with
    lowercase 'cp' when locale -a is executed. So I switched to specifying
    it as lowercase in my golded start script. But case probably doesn't
    matter.

    No, I mean that i saw you using en_*EN*.cp850, not en_*US*.cp850. That is important.

    O, sorry, I didn't notice the "_EN"...

    That was what I first tried, but that made no sense, since I don't have any en_EN* locales at all (there are the en_GB* locales though).

    wilnux5:/home/fido/log # locale
    LANG=POSIX
    LC_CTYPE=en_US.UTF-8
    ^^^^^
    This is not correct

    Ok.

    Make sure that all LC_-s are en_US.cp850. Especially LC_CTYPE.

    Ok that did the trick! Just setting LC_CTYPE=en_US.cp850 is enough. So now I have the following line for starting golded:

    sudo -u fido LC_CTYPE=en_US.cp850 luit -encoding 'CP850' /usr/local/bin/golded -f

    The linedrawing characters, and the german characters (with @CHRS: CP437) are ok too!

    So this:

    # sudo -u fido LC_CTYPE=en_US.cp850 locale
    LANG=POSIX
    LC_CTYPE=en_US.cp850
    LC_NUMERIC="POSIX"
    LC_TIME="POSIX"
    LC_COLLATE="POSIX"
    LC_MONETARY="POSIX"
    LC_MESSAGES="POSIX"
    LC_PAPER="POSIX"
    LC_NAME="POSIX"
    LC_ADDRESS="POSIX"
    LC_TELEPHONE="POSIX"
    LC_MEASUREMENT="POSIX"
    LC_IDENTIFICATION="POSIX"
    LC_ALL=

    Seems enough to get the right output from golded!?


    Bye, Wilfred.

    --- FMail-lnx64 2.3.0.1-B20240319
    * Origin: FMail development HQ (2:280/464)
  • From Vitaliy Aksyonov@1:104/117 to Wilfred van Velzen on Monday, March 25, 2024 19:03:38
    Hello Wilfred.

    25 Mar 24 21:18, you wrote to me:

    Make sure that all LC_-s are en_US.cp850. Especially LC_CTYPE.

    Ok that did the trick! Just setting LC_CTYPE=en_US.cp850 is enough. So now I have the following line for starting golded:

    Great! Glad to help.

    sudo -u fido LC_CTYPE=en_US.cp850 luit -encoding 'CP850' /usr/local/bin/golded -f

    The linedrawing characters, and the german characters (with @CHRS:
    CP437) are ok too!

    So this:

    # sudo -u fido LC_CTYPE=en_US.cp850 locale
    LANG=POSIX
    LC_CTYPE=en_US.cp850
    LC_NUMERIC="POSIX"
    LC_TIME="POSIX"
    LC_COLLATE="POSIX"
    LC_MONETARY="POSIX"
    LC_MESSAGES="POSIX"
    LC_PAPER="POSIX"
    LC_NAME="POSIX"
    LC_ADDRESS="POSIX"
    LC_TELEPHONE="POSIX"
    LC_MEASUREMENT="POSIX"
    LC_IDENTIFICATION="POSIX"
    LC_ALL=

    Seems enough to get the right output from golded!?

    Other LCs are pretty much not used in GoldEd, shall be fine for your case.

    Vitaliy

    ... 640K ought to be enough for anybody
    --- GoldED+/LNX 1.1.5-b20240305-beta
    * Origin: Aurora, Colorado (1:104/117)
  • From Wilfred van Velzen@2:280/464 to Vitaliy Aksyonov on Tuesday, March 26, 2024 11:20:42
    Hi Vitaliy,

    On 2024-03-25 19:03:38, you wrote to me:

    Ok that did the trick! Just setting LC_CTYPE=en_US.cp850 is enough.
    So now I have the following line for starting golded:

    Great! Glad to help.

    Yes, thank you very much!

    # sudo -u fido LC_CTYPE=en_US.cp850 locale
    LANG=POSIX
    LC_CTYPE=en_US.cp850
    LC_NUMERIC="POSIX"
    LC_TIME="POSIX"
    LC_COLLATE="POSIX"
    LC_MONETARY="POSIX"
    LC_MESSAGES="POSIX"
    LC_PAPER="POSIX"
    LC_NAME="POSIX"
    LC_ADDRESS="POSIX"
    LC_TELEPHONE="POSIX"
    LC_MEASUREMENT="POSIX"
    LC_IDENTIFICATION="POSIX"
    LC_ALL=

    Seems enough to get the right output from golded!?

    Other LCs are pretty much not used in GoldEd, shall be fine for your case.

    Pretty much? Or not at all? ;-)


    On to the next issues... ;-)

    I notice in my putty terminal that alignment of the columns in the area listing is off when there are unread messages in the areas:

    https://paste.opensuse.org/pastes/1b0a6ac1a41e

    This might not be new, and already happening before I started changing/fixing my code page issues. I'll check later if this also happens in my local konsole terminal.

    Also the header with the number of messages on the message reader screen is sometimes missing a space between the first number and 'of', when I go directly to a message from the area messages listing.

    https://paste.opensuse.org/pastes/80901e9650d8

    Could this be related to the above issue?


    Bye, Wilfred.

    --- FMail-lnx64 2.3.0.1-B20240319
    * Origin: FMail development HQ (2:280/464)
  • From Wilfred van Velzen@2:280/464 to Vitaliy Aksyonov on Tuesday, March 26, 2024 18:02:35
    Hi Vitaliy,

    On 2024-03-26 11:20:42, I wrote to you:

    On to the next issues... ;-)

    I notice in my putty terminal that alignment of the columns in the area listing is off when there are unread messages in the areas:

    https://paste.opensuse.org/pastes/1b0a6ac1a41e

    This might not be new, and already happening before I started changing/fixing
    my code page issues. I'll check later if this also happens in my local konsole
    terminal.

    In my local konsole it's fine. There is a '>' character after the 0 on lines for areas where there is new mail.

    Also the header with the number of messages on the message reader
    screen is sometimes missing a space between the first number and
    'of', when I go directly to a message from the area messages listing.

    https://paste.opensuse.org/pastes/80901e9650d8

    Could this be related to the above issue?

    I don't see this happening in my local konsole terminal either.


    This probably means I have to play with my putty settings (again)... :-/ ;-)


    Bye, Wilfred.

    --- FMail-lnx64 2.3.0.1-B20240319
    * Origin: FMail development HQ (2:280/464)
  • From Vitaliy Aksyonov@1:104/117 to Wilfred van Velzen on Tuesday, March 26, 2024 16:17:06
    Hello Wilfred.

    26 Mar 24 18:02, you wrote to me:
    On to the next issues... ;-)

    I notice in my putty terminal that alignment of the columns in
    the area listing is off when there are unread messages in the
    areas:

    https://paste.opensuse.org/pastes/1b0a6ac1a41e

    This might not be new, and already happening before I started
    changing/fixing my code page issues. I'll check later if this
    also happens in my local konsole terminal.

    In my local konsole it's fine. There is a '>' character after the 0 on lines for areas where there is new mail.

    Interesting. I assume you use luit with same configuration/locale in local console and putty?

    Also the header with the number of messages on the message
    reader screen is sometimes missing a space between the first
    number and 'of', when I go directly to a message from the area
    messages listing.

    https://paste.opensuse.org/pastes/80901e9650d8

    Could this be related to the above issue?

    I don't see this happening in my local konsole terminal either.


    This probably means I have to play with my putty settings (again)...
    :-/ ;-)

    You full of surprises! :) Just kidding.

    It may be lost in so many layers you use... I'd suggest to check if putty and local console use same/different TERM.

    echo $TERM

    Vitaliy

    ... 640K ought to be enough for anybody
    --- GoldED+/LNX 1.1.5-b20240305-beta
    * Origin: Aurora, Colorado (1:104/117)
  • From Wilfred van Velzen@2:280/464 to Vitaliy Aksyonov on Wednesday, March 27, 2024 09:58:56
    Hi Vitaliy,

    On 2024-03-26 16:17:06, you wrote to me:

    In my local konsole it's fine. There is a '>' character after the 0
    on lines for areas where there is new mail.

    Interesting. I assume you use luit with same configuration/locale in local console and putty?

    Yes.

    Also the header with the number of messages on the message
    reader screen is sometimes missing a space between the first
    number and 'of', when I go directly to a message from the area
    messages listing.

    https://paste.opensuse.org/pastes/80901e9650d8

    Could this be related to the above issue?

    I don't see this happening in my local konsole terminal either.

    This probably means I have to play with my putty settings (again)...
    :-/ ;-)

    You full of surprises! :) Just kidding.

    Well it helped! I switched to the $TERM=putty-256color settings we discussed before, and now all is well!

    It may be lost in so many layers you use... I'd suggest to check if
    putty and local console use same/different TERM.

    echo $TERM

    Is is 'xterm' in my local konsole, and was 'linux' but now 'putty-256color' in my putty terminal.


    Bye, Wilfred.

    --- FMail-lnx64 2.3.0.1-B20240319
    * Origin: FMail development HQ (2:280/464)
  • From Vitaliy Aksyonov@1:104/117 to Wilfred van Velzen on Wednesday, March 27, 2024 06:57:44
    Hello Wilfred.

    27 Mar 24 09:58, you wrote to me:

    Also the header with the number of messages on the message
    reader screen is sometimes missing a space between the first
    number and 'of', when I go directly to a message from the area
    messages listing.

    https://paste.opensuse.org/pastes/80901e9650d8

    Could this be related to the above issue?

    I don't see this happening in my local konsole terminal either.

    This probably means I have to play with my putty settings
    (again)...
    :-/ ;-)

    You full of surprises! :) Just kidding.

    Well it helped! I switched to the $TERM=putty-256color settings we discussed before, and now all is well!

    Great. Looks like some question need to be saved in FAQ. I only never saw English version of it.

    It may be lost in so many layers you use... I'd suggest to check
    if putty and local console use same/different TERM.

    echo $TERM

    Is is 'xterm' in my local konsole, and was 'linux' but now 'putty-256color' in my putty terminal.

    256color make console very different in other programs too.

    Vitaliy

    ... 640K ought to be enough for anybody
    --- GoldED+/LNX 1.1.5-b20240305-beta
    * Origin: Aurora, Colorado (1:104/117)
  • From Wilfred van Velzen@2:280/464 to Vitaliy Aksyonov on Wednesday, March 27, 2024 14:28:46
    Hi Vitaliy,

    On 2024-03-27 06:57:44, you wrote to me:

    Well it helped! I switched to the $TERM=putty-256color settings we
    discussed before, and now all is well!

    Great. Looks like some question need to be saved in FAQ. I only never saw English version of it.

    I don't think it exists.

    echo $TERM

    Is is 'xterm' in my local konsole, and was 'linux' but now
    'putty-256color' in my putty terminal.

    256color make console very different in other programs too.

    I haven't tested the regular TERM=putty, but compared to TERM=linux, Midnight Commander looks the same. But my own ncurses program (configuration program for FMail tosser), looks ok now too!

    https://paste.opensuse.org/pastes/c77fdc8fd4af

    In the background how it looks with 'putty-256color' and how it is supposed to look. In the forground how it looks if TERM=linux.

    Bye, Wilfred.

    --- FMail-lnx64 2.3.0.1-B20240319
    * Origin: FMail development HQ (2:280/464)
  • From Vitaliy Aksyonov@1:104/117 to Wilfred van Velzen on Wednesday, March 27, 2024 08:07:34
    Hello Wilfred.

    27 Mar 24 14:28, you wrote to me:

    Well it helped! I switched to the $TERM=putty-256color settings
    we discussed before, and now all is well!
    Great. Looks like some question need to be saved in FAQ. I only
    never saw English version of it.
    I don't think it exists.

    I think so. :)

    echo $TERM

    Is is 'xterm' in my local konsole, and was 'linux' but now
    'putty-256color' in my putty terminal.

    256color make console very different in other programs too.

    I haven't tested the regular TERM=putty, but compared to TERM=linux, Midnight Commander looks the same. But my own ncurses program (configuration program for FMail tosser), looks ok now too!

    ncurses uses terminfo and `linux` not the best for Putty. `xterm` may work, but `putty` or `putty-256color` is the best for sure.

    https://paste.opensuse.org/pastes/c77fdc8fd4af

    In the background how it looks with 'putty-256color' and how it is supposed to look. In the forground how it looks if TERM=linux.

    putty-256color just add colors. Not every program will look different. For example `ls` will.

    Vitaliy

    ... 640K ought to be enough for anybody
    --- GoldED+/LNX 1.1.5-b20240305-beta
    * Origin: Aurora, Colorado (1:104/117)
  • From Wilfred van Velzen@2:280/464 to Vitaliy Aksyonov on Wednesday, March 27, 2024 15:34:22
    Hi Vitaliy,

    On 2024-03-27 08:07:34, you wrote to me:

    256color make console very different in other programs too.

    I haven't tested the regular TERM=putty, but compared to TERM=linux,
    Midnight Commander looks the same. But my own ncurses program
    (configuration program for FMail tosser), looks ok now too!

    https://paste.opensuse.org/pastes/c77fdc8fd4af

    In the background how it looks with 'putty-256color' and how it is
    supposed to look. In the forground how it looks if TERM=linux.

    ncurses uses terminfo and `linux` not the best for Putty. `xterm` may work,
    but `putty` or `putty-256color` is the best for sure.

    My program works correctly in 'xterm' too.


    Bye, Wilfred.

    --- FMail-lnx64 2.3.0.1-B20240319
    * Origin: FMail development HQ (2:280/464)