[SOLVED] Something is wrong with my locale.

What I think happened is that I reinstalled glibc to try to get a vim extension to work, and somehow my locale is now corrupt. I didn't notice it right away so I'm not exactly sure what I did, but the German umlaut characters (öäßü) specifically, and some other characters that I'm not even sure what they are, do not render correctly. I now get characters such as ö is place of them. I have tried all the basic locale configuration guides and now am at a loss for what to try next. Any ideas?

https://wiki.archlinux.org/index.php/Locale
https://www.youtube.com/watch?v=ZurF9p6WA4Q

Hey, I appreciate the video, but those are already the things which I have tried.

what do you mean with "do not render" ? are the Umlaute broken in general or only at specific programs ? it would be nice that at minimu to post the output of 'locale' because no one here is jesus, we need informations that we can work with. your explanation of your problem is "ziemlich dünn".

What do you mean with "all"?
Have you tried the Localize Manjaro Guide?

Yes, I've tried the instructions in that guide. If I change /etc/locale.gen/ and then regenerate sudo locale-gen I can see the changes taking place with locale && locale -a. If I type an umlaut in the console, It prints correctly, but if it's generated by my command line translator it prints incorrectly. Though, I am able to echo an umlaut correctly, but if I try to use an umlaut in vim it print incorrectly. Let me know if there's any other info that could be used to narrow this down.

locale:

LANG=de_DE.utf8
LC_CTYPE="de_DE.UTF-8"
LC_NUMERIC="de_DE.UTF-8"
LC_TIME="de_DE.UTF-8"
LC_COLLATE="de_DE.UTF-8"
LC_MONETARY="de_DE.UTF-8"
LC_MESSAGES="de_DE.UTF-8"
LC_PAPER="de_DE.UTF-8"
LC_NAME="de_DE.UTF-8"
LC_ADDRESS="de_DE.UTF-8"
LC_TELEPHONE="de_DE.UTF-8"
LC_MEASUREMENT="de_DE.UTF-8"
LC_IDENTIFICATION="de_DE.UTF-8"
LC_ALL=de_DE.UTF-8

locale -a :

C
de_DE.utf8
en_US.utf8
POSIX

Maybe a font issue?
You have to provide more info about your issue, so we can solve it :man_shrugging:

inxi -SIxxxz
locale
locale -a
localectl

Also paste the wrong text in a code formatted ( in ```)

What about other applications??
Is it only with vim? Maybe check vim settings...

Hey thanks, you were quick on the draw. locale and locale -a are edited in the comment above if you hadn't see those.

Here are the umlauts opened in vim:

äÃß¼

and they are wrong also in nano but different..

����

It is definitely a shell problem as everything in the gui is working as expected.
If I echo umlauts into a file, I can cat them out just fine, but if I try to pipe it to less, it renders as
<E4><F6><FC><DF>

I usually have my locale set to en_US and I have never had a problem with the umlauts. Another character I know is having problems is an arrow used in nerdtree. I tried adding set encoding=utf-8 in .vimrc and that didn't work. I also tried to reinstall vim and ttf-font.

$inxi -SIxxxz

System:    Host: al-pc Kernel: 4.19.88-1-MANJARO x86_64 bits: 64 compiler: gcc v: 9.2.0 Desktop: Xfce 4.14.1 tk: Gtk 3.24.12 
           info: xfce4-panel wm: xfwm4 dm: LightDM 1.30.0 Distro: Manjaro Linux 
Info:      Processes: 248 Uptime: 1h 14m Memory: 15.56 GiB used: 2.47 GiB (15.9%) Init: systemd v: 242 Compilers: gcc: 9.2.0 
           Shell: bash v: 5.0.11 running in: xfce4-terminal inxi: 3.0.37
$localectl 
   System Locale: LANG=de_DE.UTF-8
       VC Keymap: us
      X11 Layout: us
       X11 Model: pc105

How about this? Did you set it on purpose?
Please post:

cat /etc/locale.conf
[ -r "$HOME/.config/locale.conf" ] && cat "$HOME/.config/locale.conf"

Since it's a shell issue, consider checking your terminal apps settings and maybe resetting to default, if possible. Creating a new user and checking this issue would be faster and easier IMHO.

  • Does this also happen in TTY?
  • Does this happen to root user?

Some relevant tag may drag some shell expert to this issue.

Oh yeah, I did set LC_ALL and LC_CTYPE in my .bashrc, and I also have tried to set some vars in .xprofile. Over the last month I've tried several times to fix this. When I first noticed it, I set export LC_ALL=C and some characters were cleaned up (not the umlauts, I think it was some kind of dashes). It hasn't been a huge issue since I only use German as a hobby.

$cat /etc/locale.conf
LANG="de_DE.UTF-8"

I don't have a

$HOME/.config/locale.conf

Yes, the problem persists with the root user, but it does work with TTY! I'll try creating a new user and see what that does.

So I added a new normal user and a system user no luck for either :frowning: . Not sure what to try next. I'm pretty sure it has something to do with glibc or clang.

Start from the beginning...

Check your pacman.log to see what exactly has been done when you used pacman and post info.

To make things clear, post what you have changed in your bashrc or elsewhere, like where you changed LC_ALL. Definitely change/remove that export to nothing/empty, until we need to.
I believe the answer will come out of your next post (pacman.log), ... hopefully...

Okay. I had set

LC_ALL and LC_CTYPE

to each of these

en_US.UTF-8, de_DE.UTF-8, C

I have now removed all exports from .bashrc and removed .xprofile, but LC_ALL is still being set somewhere because I changed the language back to english and regenerated (locale-gen) and now it's set to en_US.UTF-8. What I ultimately want is the language set to English but with the umlauts working (as I have always had before this).

here are the logs from that day and several days after when I was messing around trying to get Vim YouCompleteMe working. I have no idea what all the lib32 stuff is.. my machine is 64.. is there something I don't understand?

[2019-11-15T00:56:37-0500] [ALPM] upgraded gcc-libs (9.2.0-3 -> 9.2.0-4)
[2019-11-15T00:56:37-0500] [ALPM] upgraded libtiff (4.0.10-1 -> 4.1.0-1)
[2019-11-15T00:56:37-0500] [ALPM] upgraded librsvg (2:2.46.3-1 -> 2:2.46.3+6+g4735bd07-1)
[2019-11-15T00:56:39-0500] [ALPM] upgraded perl-libwww (6.39-2 -> 6.41-1)
[2019-11-15T00:56:40-0500] [ALPM] upgraded libedit (20190324_3.1-1 -> 20191025_3.1-1)
[2019-11-15T00:57:08-0500] [ALPM] upgraded bluez-libs (5.51-2 -> 5.52-1)
[2019-11-15T00:57:21-0500] [ALPM] upgraded libssh (0.9.1-1 -> 0.9.2-1)
[2019-11-15T00:57:37-0500] [ALPM] upgraded calibre (4.2.0-2 -> 4.3.0-1)
[2019-11-15T00:57:45-0500] [ALPM] upgraded libwbclient (4.10.8-1 -> 4.10.10-1)
[2019-11-15T00:58:13-0500] [ALPM] upgraded libmm-glib (1.10.8-1 -> 1.12.0-1)
[2019-11-15T00:58:13-0500] [ALPM] upgraded libheif (1.5.1-2 -> 1.6.0-1)
[2019-11-15T00:58:16-0500] [ALPM] upgraded libpwquality (1.4.1-1 -> 1.4.2-1)
[2019-11-15T00:58:16-0500] [ALPM] upgraded libdvdread (6.0.1-1 -> 6.0.2-1)
[2019-11-15T00:58:33-0500] [ALPM] upgraded libdvdnav (6.0.0-1 -> 6.0.1-1)
[2019-11-15T00:58:33-0500] [ALPM] upgraded libshout (1:2.4.3-1 -> 1:2.4.3-2)
[2019-11-15T00:58:40-0500] [ALPM] upgraded libplist (2.0.0+11+gec9ba8b-2 -> 2.1.0-1)
[2019-11-15T00:58:40-0500] [ALPM] upgraded libimobiledevice (1.2.0+87+g92c5462-1 -> 1.2.0+87+g92c5462-4)
[2019-11-15T00:59:14-0500] [ALPM] upgraded lib32-expat (2.2.7-1 -> 2.2.9-1)
[2019-11-15T00:59:17-0500] [ALPM] upgraded lib32-gcc-libs (9.2.0-3 -> 9.2.0-4)
[2019-11-15T00:59:17-0500] [ALPM] upgraded lib32-libcroco (0.6.12+4+g9ad7287-2 -> 0.6.13-1)
[2019-11-15T00:59:17-0500] [ALPM] upgraded lib32-libice (1.0.9-3 -> 1.0.10-1)
[2019-11-15T00:59:17-0500] [ALPM] upgraded lib32-libjpeg-turbo (2.0.2-1 -> 2.0.3-1)
[2019-11-15T00:59:17-0500] [ALPM] upgraded lib32-librsvg (2.46.3-1 -> 2.46.3+6+g4735bd07-1)
[2019-11-15T00:59:19-0500] [ALPM] upgraded lib32-mesa (19.2.2-1 -> 19.2.4-0.1)
[2019-11-15T00:59:19-0500] [ALPM] upgraded lib32-libva (2.4.0-1 -> 2.5.0-1)
[2019-11-15T00:59:19-0500] [ALPM] upgraded lib32-libva-mesa-driver (19.2.2-1 -> 19.2.4-0.1)
[2019-11-15T00:59:19-0500] [ALPM] upgraded lib32-libxi (1.7.9-2 -> 1.7.10-1)
[2019-11-15T00:59:20-0500] [ALPM] upgraded lib32-mesa-vdpau (19.2.2-1 -> 19.2.4-0.1)
[2019-11-15T00:59:20-0500] [ALPM] upgraded lib32-vulkan-intel (19.2.2-1 -> 19.2.4-0.1)
[2019-11-15T00:59:20-0500] [ALPM] upgraded lib32-vulkan-radeon (19.2.2-1 -> 19.2.4-0.1)
[2019-11-15T00:59:20-0500] [ALPM] upgraded libcdio-paranoia (10.2+2.0.0-2 -> 10.2+2.0.1-1)
[2019-11-15T00:59:21-0500] [ALPM] upgraded libmagick6 (6.9.10.70-1 -> 6.9.10.71-1)
[2019-11-15T00:59:21-0500] [ALPM] upgraded libmbim (1.20.0-1 -> 1.20.2-1)
[2019-11-15T00:59:21-0500] [ALPM] upgraded libnm (1.20.4-1 -> 1.20.6-1)
[2019-11-15T00:59:21-0500] [ALPM] upgraded libnma (1.8.22-1.1 -> 1.8.24-1)
[2019-11-15T00:59:21-0500] [ALPM] upgraded libplacebo (1.18.0-2 -> 1.21.0-1)
[2019-11-15T00:59:21-0500] [ALPM] upgraded libva-mesa-driver (19.2.2-1 -> 19.2.4-0.1)
[2019-11-15T01:03:02-0500] [ALPM] running 'glib-compile-schemas.hook'...
[2019-11-16T20:42:45-0500] [ALPM] installed libgtop (2.40.0+2+g31db82ef-1)
[2019-11-16T20:42:47-0500] [ALPM] running 'glib-compile-schemas.hook'...
[2019-11-18T18:25:45-0500] [PACMAN] Running 'pacman -S glibc'
[2019-11-18T18:25:51-0500] [ALPM] reinstalled glibc (2.30-2)
[2019-11-18T18:36:25-0500] [PACMAN] Running 'pacman -S glibc'
[2019-11-18T18:36:31-0500] [ALPM] reinstalled glibc (2.30-2)
[2019-11-18T18:37:13-0500] [PACMAN] Running 'pacman -R glibc'
[2019-11-18T18:37:20-0500] [PACMAN] Running 'pacman -S glibc'
[2019-11-18T18:37:23-0500] [ALPM] reinstalled glibc (2.30-2)
[2019-11-19T13:46:37-0500] [PACMAN] Running 'pacman -R glibc'
[2019-11-19T13:49:40-0500] [PACMAN] Running 'pacman -S glibc'
[2019-11-19T13:49:43-0500] [ALPM] reinstalled glibc (2.30-2)
[2019-11-19T14:05:36-0500] [PACMAN] Running 'pacman -S glibc'
[2019-11-19T14:05:38-0500] [ALPM] reinstalled glibc (2.30-2)
[2019-11-19T14:06:17-0500] [PACMAN] Running 'pacman -R glibc'
[2019-11-21T11:05:11-0500] [ALPM] running 'glib-compile-schemas.hook'...
[2019-11-21T12:26:13-0500] [ALPM] running 'glib-compile-schemas.hook'...
[2019-11-28T07:30:42-0500] [ALPM] upgraded lib32-libva-mesa-driver (19.2.4-0.1 -> 19.2.6-0.1)
[2019-11-28T07:30:46-0500] [ALPM] upgraded lib32-mesa (19.2.4-0.1 -> 19.2.6-0.1)
[2019-11-28T07:30:46-0500] [ALPM] upgraded lib32-mesa-vdpau (19.2.4-0.1 -> 19.2.6-0.1)
[2019-11-28T07:30:47-0500] [ALPM] upgraded lib32-vulkan-intel (19.2.4-0.1 -> 19.2.6-0.1)
[2019-11-28T07:30:47-0500] [ALPM] upgraded lib32-vulkan-radeon (19.2.4-0.1 -> 19.2.6-0.1)
[2019-11-28T07:30:47-0500] [ALPM] upgraded libva-mesa-driver (19.2.4-0.1 -> 19.2.6-0.1)
[2019-11-29T14:16:32-0500] [PACMAN] Running 'pacman -R glibc'
[2019-11-29T14:30:54-0500] [PACMAN] Running 'pacman -S glibc'
[2019-11-29T14:30:56-0500] [ALPM] reinstalled glibc (2.30-2)

So I was correct yesterday when I said I thought it was some kind of dash that was printing incorrectly as well. Both em dash and en dash print these gibberish characters: †or â.

After spending way to much time trying to figure out this issue, I just decided to ignore it, and today when I went to do a translation I found that the umlauts were working correctly! I have no idea what fixed it. I'm wondering if it had something to do with that large update.. Anyway, thanks for your help.

2 Likes

Forum kindly sponsored by