Apple devices create all filenames in Unicode Decomposed Normalisation Form (NFD), while every other major OS uses Composed Normalisation Form (NFC). This makes you, as a Mac user, the bad guy, because it is you who is incompatible with the rest of the world.

In a nutshell, the problem is this: Whenever you create files with diacritics they will be copied to other devices with filenames stored as decomposed strings. This is a non‑standard for these OS'es, and you never know what problems that will cause.

This article presents my way of solving the problem by configuring Rclone to create all files in NFC (composed form) instead of NFD (decomposed form) – which is not at all that straightforward is it would seem.

TL;DR: If you just want to solve the problem without actually delving into the problem and its technical details, simply follow the steps below. Otherwise, head over to Technical background.

Prereqs: download Rclone and macFUSE.¹⁾

1. Download the custom-made iconv library – this is actually Apple's own version of the library (which you have on your macOS), but with iconv base updated to the latest version and with support for surrogate pairs (this is actually important, because for example all emojis are surrogate pairs of characters).
2. Tweak this library yourself, because it is not fully compatible with Apple's version. Specifically, open the file ./include/iconv.h.in and comment out these 6 code blocks (I list the lines below already commented):
   1. lines 69–71:

      //#ifndef LIBICONV_PLUG
      //#define iconv_open libiconv_open
      //#endif

   2. lines 79–81:

      //#ifndef LIBICONV_PLUG
      //#define iconv libiconv
      //#endif

   3. lines 85–87:

      //#ifndef LIBICONV_PLUG
      //#define iconv_close libiconv_close
      //#endif

   4. line 129:

      //#define iconv_open_into libiconv_open_into

   5. line 134:

      //#define iconvctl libiconvctl

   6. line 214:

      //#define iconvlist libiconvlist

3. Compile and install the double‑tweaked library using the following commands:

   make -f Makefile.utf8mac autogen
   ./configure --prefix=/usr/local
   make
   make install

   These are stated here by author of the tweaked version and are similar to the building commands of the original GNU libiconv. However, I added the –prefix=/usr/local parameter to the configure command (present in the GNU version, but not in the tweaked version), since I wasn't sure where the library would put itself without it.

4. Now you have two Apple‑style libiconv binaries on your system: the original (and old) Apple one in /usr/bin/iconv and the tweaked (and updated) one in /usr/local/bin/iconv.
   1. First, test that the new binary itself works with the following conversions:

      echo 🙂 | /usr/bin/iconv -f utf-8 -t utf-8-mac
      > �
      echo 🙂 | /usr/local/bin/iconv -f utf-8 -t utf-8-mac
      > 🙂

   2. Second, test that the new dynamic library has a proper symbol table. If everything went right, typing the following command:

      $ nm -gU /usr/local/lib/libiconv.2.dylib

      should output something similar to left column, and not to the right column (the order of lines might wary, the symbol names are important):

      Correct (Apple‑style) symbol table 00000000000e3290 D __libiconv_version 0000000000002ce0 T _iconv 0000000000003430 T _iconv_canonicalize 0000000000002d10 T _iconv_close 00000000000016b0 T _iconv_open 0000000000002d20 T _iconv_open_into 0000000000003160 T _iconvctl 0000000000003270 T _iconvlist 0000000000015eb0 T _libiconv_set_relocation_prefix 0000000000015dd0 T _locale_charset

      Incorrect (GNU‑style) symbol table 00000000000e3290 D __libiconv_version 0000000000002ce0 T _libiconv 0000000000003430 T _iconv_canonicalize 0000000000002d10 T _libiconv_close 00000000000016b0 T _libiconv_open 0000000000002d20 T _libiconv_open_into 0000000000003160 T _libiconvctl 0000000000003270 T _libiconvlist 0000000000015eb0 T _libiconv_set_relocation_prefix 0000000000015dd0 T _locale_charset

5. Now, you need to update the Fuse library to search for the dynamically‑loaded libiconv library on the new place:
   1. First, check that Fuse actually looks for the library under /usr/lib/:

      $ otool -L /usr/local/lib/libfuse.2.dylib
      /usr/local/lib/libfuse.2.dylib:
        /usr/local/lib/libfuse.2.dylib (compatibility version 12.0.0, current version 12.9.0)
        /usr/lib/libiconv.2.dylib (compatibility version 7.0.0, current version 7.0.0)
        … [other libraries]

   2. Now, change the path by using install_name_tool:

      sudo install_name_tool -change /usr/lib/libiconv.2.dylib /usr/local/lib/libiconv.2.dylib /usr/local/lib/libfuse.2.dylib

   3. Finally, check that the change is successful:

      $ otool -L /usr/local/lib/libfuse.2.dylib
      /usr/local/lib/libfuse.2.dylib:
        /usr/local/lib/libfuse.2.dylib (compatibility version 12.0.0, current version 12.9.0)
        /usr/local/lib/libiconv.2.dylib (compatibility version 7.0.0, current version 7.0.0)
        … [other libraries]

Historically, the way of encoding filenames on Mac OS X started to differ from the rest of the world when Apple switched from HFS to HFS+ file system in 1998. HFS+ enforces decomposed form for the filenames²⁾ on disk

Apple adopted NFD (or decomposed form) of the filenames when it switched from HFS to HFS+ file system in 1998. Historically, when Apple switched from HFS to HFS+ file system in 1998

HFS+ enforces decomposed form of the filenames³⁾ on disk

1. files⁴⁾
2. diacritics⁵⁾

through Rclone, they are copied to your clouds with filenames stored as decomposed strings. This creates three different problems:

1. First, if other users on different OS'es are renaming the files you created (or renamed), they need to press Backspace twice when they want to remove a letter with diacritics (e.g. á or ü). Renaming a filename with a lot of diacritics, like XXX, can become pretty lengthy process. And note, this applies also to you when you are accessing these clouds from a web client (i.e. your browser).
2. Second,

https://www.unicode.org/reports/tr15/#Norm_Forms

Due to some technical under-the-hood changes that Apple has made when it switched from HFS+ to APFS file system in its devices back in 2017,

Rclone has a special ‑o switch which will forward its parameters to the underlying macFUSE/FUSE-T system providing the mounting functionality of the remote system.

This way, it is possible to order Fuse to load iconv module and have it automatically converting all filenames to NFC when they are moved to remote cloud. Thus, the straightforward way to solve the problem should be to use the following command when mounting the system:

$ rclone mount […] -o modules=iconv,from_code=UTF-8,to_code=UTF-8-MAC

And this actually works, but with some problems.

The problem is that macOS uses its own “Apple‑tweaked” implementation of iconv, which is (1) very old; (2) non‑standard; and (3) it cannot convert significant parts of Unicode characters – for example, emoji. All three of these problems will be crucial in our attempt to deal with the problem. Moreover, the library itself resides in /usr/bin/iconv, which is under SIP, so you cannot normally do anything with it, and the only way to update it is to actually update the whole macOS.

The first problem: the standard library versions Apple provides are almost always very obsolete.⁶⁾ In the case of iconv, versions supplied with different macOS'es are these:

In a nutshell, despite what the internal Apple versioning says, all macOS'es still use libiconv 1.11 released back in 2006.

$ nm -gU /usr/lib/libiconv.2.dylib
00000000000f2700 D __libiconv_version
0000000000002360 T _iconv
000000000000267a T _iconv_canonicalize
0000000000002382 T _iconv_close
0000000000001049 T _iconv_open
000000000000238f T _iconvctl
0000000000002488 T _iconvlist
0000000000013ff8 T _libiconv_set_relocation_prefix

$ nm -gU /usr/local/lib/libiconv.2.dylib
00000000000e3290 D __libiconv_version
0000000000003430 T _iconv_canonicalize
0000000000002ce0 T _libiconv
0000000000002d10 T _libiconv_close
00000000000016b0 T _libiconv_open
0000000000002d20 T _libiconv_open_into
0000000000015eb0 T _libiconv_set_relocation_prefix
0000000000003160 T _libiconvctl
0000000000003270 T _libiconvlist
0000000000015dd0 T _locale_charset

$ nm -gU /usr/lib/libiconv.2.dylib

00000000000f2700 D __libiconv_version
0000000000002360 T _iconv
000000000000267a T _iconv_canonicalize
0000000000002382 T _iconv_close
0000000000001049 T _iconv_open
000000000000238f T _iconvctl
0000000000002488 T _iconvlist
0000000000013ff8 T _libiconv_set_relocation_prefix

$ nm -gU /usr/local/lib/libiconv.2.dylib
00000000000e3290 D __libiconv_version
0000000000003430 T _iconv_canonicalize
0000000000002ce0 T _libiconv
0000000000002d10 T _libiconv_close
00000000000016b0 T _libiconv_open
0000000000002d20 T _libiconv_open_into
0000000000015eb0 T _libiconv_set_relocation_prefix
0000000000003160 T _libiconvctl
0000000000003270 T _libiconvlist
0000000000015dd0 T _locale_charset

• Facebook
• Twitter
• LinkedIn
• Pinterest
• Reddit
• Email

There is a patched libiconv library on GitHub which adds support for UTF‑8‑MAC encoding. Installing it allows you not only to convert between real UTF‑8 and UTF‑8‑MAC encodings

$ echo "test📖" | /usr/bin/iconv -f utf-8 -t utf-8-mac
test�
 
$ echo "test📖" | /usr/local/bin/iconv -f utf-8 -t utf-8-mac
test📖

• convmv
• BandiNamer
• jaso