German Umlaut problems (Was: Japanese character file name is not available)

[Thomas Walter]

Hey guys,

I'm bumping the issue that was reported in the following thread, because
I am having similar problems with names containing German umlauts.

https://lists.zx2c4.com/pipermail/password-store/2018-April/003259.html

As soon as I am adding a key for a user that has umlauts, pass starts
printing "sed: RE error: illegal byte sequence" (line 128 seems to be
responsible). I too am using pass on MacOS.

I have found something on Stackoverflow with a similar error:

https://stackoverflow.com/questions/19242275/re-error-illegal-byte-sequence-on-mac-os-x

The suggested workaround is to set LC_ALL=C to just "skip" these letters
(with iconv being a "more correct" solution?).

In my case I noticed the problem with "pass init" in line 128:

current_keys="$(LC_ALL=C $GPG $PASSWORD_STORE_GPG_OPTS -v
--no-secmem-warning --no-permission-warning --decrypt --list-only
--keyid-format long "$passfile" 2>&1 | sed -n 's/^gpg: public key is
\([A-F0-9]\+\)$/\1/p' | LC_ALL=C sort -u)"

There is LC_ALL=C before each the gpg call and the sort command, but not
in front of sed. If I add it, I don't get the errors anymore.

But not knowing the details, I am not sure if that is the correct
solution or if that'll result in problems along the way.

I also noticed that (an unset) $PASSWORD_STORE_GPG_OPTS seems to be used
there instead of $GPG_OPTS. Especially the --quiet that is included in
$GPG_OPTS seems to be missing here, because that way the command would
only return lines with "gpg: public key is ..." and not the Umlaut-Names
in the first place (so not requiring the LC_ALL workaround)?


Either way, with LC_ALL or with --quiet or without any change,
$current_keys is empty after that line. So even a repeated "pass init"
with the same list of keys will always re-encrypt all files.


There seems to be difference with how OSX (BSD) sed handles the plus
sign in regexes.

linux$ echo "foobar" | sed -e "s/o\+//g"
fbar

osx$ $ echo "foobar" | sed -e "s/o\+//g"
foobar

Reason being is that there are basic regular expressions (old) where the
+ sign was a normal character and extended (modern) ones. BSD sed uses
-E to switch to the modern expressions (while on Linux you can just
escape the + for it to work?).

Even though linux sed doesn't seem to list "-E" in the local manual
pages, it works this way:

linux$ echo "foobar" | sed -Ee "s/o+//g"
fbar

osx$ echo "foobar" | sed -Ee "s/o+//g"
fbar

See "-E" as being "undocumented" here:
https://www.gnu.org/software/sed/manual/sed.html

Long story short, see attachment. I am not sure if other sed's need
similar treatment or if other $PASSWORD_STORE_GPG_OPTS need changing
too. And I've only tested it on OSX so far.

Regards from Germany,
Thomas

-- 
()  ascii ribbon campaign - against html e-mail
/\  www.asciiribbon.org   - against proprietary attachments
-------------- next part --------------
--- pass	2018-10-11 10:53:49.000000000 +0200
+++ pass-balu	2018-10-11 10:57:25.000000000 +0200
@@ -125,7 +125,7 @@
 			done
 			gpg_keys="$($GPG $PASSWORD_STORE_GPG_OPTS --list-keys --with-colons "${GPG_RECIPIENTS[@]}" | sed -n 's/^sub:[^:]*:[^:]*:[^:]*:\([^:]*\):[^:]*:[^:]*:[^:]*:[^:]*:[^:]*:[^:]*:[a-zA-Z]*e[a-zA-Z]*:.*/\1/p' | LC_ALL=C sort -u)"
 		fi
-		current_keys="$(LC_ALL=C $GPG $PASSWORD_STORE_GPG_OPTS -v --no-secmem-warning --no-permission-warning --decrypt --list-only --keyid-format long "$passfile" 2>&1 | sed -n 's/^gpg: public key is \([A-F0-9]\+\)$/\1/p' | LC_ALL=C sort -u)"
+		current_keys="$(LC_ALL=C $GPG $GPG_OPTS -v --no-secmem-warning --no-permission-warning --decrypt --list-only --keyid-format long "$passfile" 2>&1 | sed -En 's/^gpg: public key is ([A-F0-9]+)$/\1/p' | LC_ALL=C sort -u)"
 
 		if [[ $gpg_keys != "$current_keys" ]]; then
 			echo "$passfile_display: reencrypting to ${gpg_keys//$'\n'/ }"