Obsah
Rclone: configuring root folders, change polling and cleaning directory cache
Finding root folder for mounting OneDrive is quite complicated, as is refreshing directory cache for folder shortcuts in Google Drive, as they are ignored in change polling.
Setting root folder for mounting Microsoft OneDrive and Google Drive
You can always mount specific subfolder of a cloud by simply using the mount command with a subpath, e.g.:
rclone mount Remote:path/to/subfolder /local/mount
Adding the :path/to/subfolder
to remote cloud will lead to mounting that specific subfolder. But it will also cause the mounted volume name to contain this path, so instead of simply seeing this as Remote
in your computer volume list, you will see Remote:path/to/subfolder
. To avoid this, you can manually set mounted volume name by supplying --volname
parameter, e.g. --volname Remote
.
Even better way to mount only a specific subfolder of a remote cloud is to do this directly in rclone.conf
configuration file by setting root_folder_id
parameter for the remote. This will tell Rclone that it should look for that folder and always use it as root folder for that remote. I am not sure whether every remote supports this, but at least Google Drive and Microsoft OneDrive do.
Finding the ID of a folder we want to use as a root folder
To set root_folder_id
parameter, we need to find the folder ID. In Google Drive, this is straightforward: it is simply the ID you see in browser path when you enter that folder, e.g. 12345abcde67890ghijk12345ABCDE678
in the following path:
https://drive.google.com/drive/u/0/folders/12345abcde67890ghijk12345ABCDE678
The problem, as always, is with Microsoft. In OneDrive, there's absolutely no way of getting the real folder ID, unless you do not want to trace network traffic in browser Developer console. The folder ID is not contained in link of a folder copied for sharing, neither in Sharepoint address of that folder. You cannot find it through Teams app either.
But it is easy with Rclone. You get a list of directories with their IDs by calling this command with a parent directory path:
rclone lsf --dirs-only --format ip --csv Remote:path/to/parent/directory
You will get the list of subdirectories with their IDs, something like this:
b!iyyUEFMz_kWTKjWfSP5ndNq_WW0CwMdIvttUlRwXsgsdVV6DxHKpT6aD8sXU1zSZ#016RGUGOWSHGSXEHLSIZGKWSVETOVWJIZA,interim/ b!iyyUEFMz_kWTKjWfSP5ndNq_WW0CwMdIvttUlRwXsgsdVV6DxHKpT6aD8sXU1zSZ#016RGUGOS6WEFRB72OGBEKYOW76K3KRGOM,raw/
THOSE, finally, are the real folder IDs which need to be entered as values for root_folder_id
parameter!
Directory cache for remotes and their limitations
When using Rclone caching (which you WANT and SHOULD use to speed things up – preferably fully turned on with switch --vfs-cache-mode full
), Rclone will cache directory contents and would not send new network request to remote with each revisit of a folder.1)
There are two mechanisms which will make Rclone refresh its directory cache when the remote cloud changes via another vector (e.g. from a web interface):
- One is
--dir-cache-time
, which sets the maximum time a directory is cached (default 5m0s). When that time is over, Rclone will ask remote cloud again to get the listing of files and subfolders in that directory. - Another and more clever one is
--poll-interval
, which sets the interval Rclone will ask to get a list of changes from the remote (default 1m0s). So there is no need to ever request the whole directory listing the second time, since it is enough for Rclone to get the updates (deltas) from remote.
Most remotes do not support polling, but when they do, you should set --dir-cache-time
as high as possible, since there is no reason for Rclone to ever bluntly drop the whole directory cache.
Google Drive limitations of change-polling in shared drives and shortcuts to folders
Google Drive supports polling for changed files and directories, but only in your own drive. That excludes changes in shared drives AND, what is worse, that even excludes folders that you added as shortcuts to your own drive. Again, Rclone won't get any update notifications about contents of folder shortcuts through polling.2)
When you are working in a business environment where the master folder containing the whole organisation of folders is shared with you, this completely breaks your workflow and makes the whole change polling system of Rclone useless. If that is your case, turning off polling through --poll-interval 0
makes sense, since it won't help you anyway.
The question is, how to cope with that?
Manually cleaning directory cache in Google Drive in folder shortcuts
In the case that you only work within shared directories added to your drive as shortcuts, there is no ideal solution. Folders and files will never update, unless the --dir-cache-time
runs out. So one possible solution is to set it to a low value, e.g. 1 minute. That will solve the problem, but path traversal will become noticeable slower.
The solution which works for me is it keep the --dir-cache-time
high, but at least have the possibility of forcing the specific folder cache to refresh when I know I have updated it.
This requires two steps:
- You must run Rclone
mount
command with remote control turned on. This requires supplying the following parameters:--rc
--rc-addr=IP:PORT
(default “localhost:5572”)- AND either both:
--rc-user=VALUE
--rc-pass=VALUE
- OR:
--rc-no-auth
- Then you can you use Terminal commands to force refreshing or forgetting either the entire Rclone directory cache, or just the cache for a specific folder.
- To remove ("forget") the directory from cache, use:
rclone rc vfs/forget 'dir=path/to/directory'
,- or clean whole cache with
rclone rc vfs/forget
- To re-read ("refresh") directory contents from server, use:
rclone rc vfs/refresh 'dir=path/to/directory'
(optionally addingrecursive=true
if you want all subdirectories to be re‑read)
- If you run mount with
--rc-user
and--rc-pass
values, you also have to supply them here, e.g.:rclone rc vfs/refresh --rc-user=VALUE --rc-pass=VALUE --rc-addr=IP:PORT 'dir=path/to/directory'
Semi-automatically clean the cache using macOS services
Cleaning the cache by running terminal each time is cumbersome and error‑prone. In macOS, the better way is to create Automator workflow, which will then be added to services menu when right clicking on folder:
- Run
Automator.app
- Create new workflow (⌘Cmd+N) of type “Quick action”
- Set the workflow to receive current
folders
(not files) and if you use a file manager (like PathFinder, ForkLift or Crax Commander), also set it to run inany applications
, not just Finder. - Drag
Run shell script
action to empty action list - The actual code would go like this:
# go over each selected folder while read f do origPrefix='/path/to/mount/point/' newPrefix='' cachef="${f/#$origPrefix/$newPrefix}" if [[ "$f" == "$origPrefix"* ]]; then rclone rc vfs/refresh [–-rc-user=VALUE --rc-pass=VALUE --rc-addr=IP:PORT] dir="$cachef" else osascript -e "tell app \"System Events\" to display dialog \"Folder not under rClone folder, skipping:\n $f\"" fi done
My final cache-clearing script using macOS services
My final script ended up a little more tweaked, since I also wanted to take into account the following things:
- I want the script to programatically refresh files & directory listing in the app after clearing the cache, since I am using Path Finder, which does not do it automatically (as Finder does).
- I the selected entry is a regular file, I want to actually clean the cache of its parent folder. If it is a folder, I wanted to refresh that one. Finding parent folder in bash is easy.
- For some reason,
rclone rc vfs/forget
did not really work for me, so I ended up usingrclone rc vfs/refresh
.
# get active app activeApp=$(osascript -e "tell application \"System Events\" return name of first application process whose frontmost is true end tell") # refresh cache for each selected file/folder while read f do prefix='/path/to/remote/mount/point/' if [[ "$f" == "$prefix"* ]]; then cachef="${f/#$prefix/}" if [ -d "$f" ]; then path="$cachef" else path="${cachef%/*}" fi /usr/local/bin/rclone rc vfs/refresh [–-rc-user=VALUE --rc-pass=VALUE --rc-addr=IP:PORT] dir="$parent" #osascript -e "tell app \"System Events\" to display dialog \"Cleared rClone cache for folder:\n $cachef\nwith parent:\n $parent\nfrom active app: $activeApp.\"" else osascript -e "tell app \"System Events\" to display dialog \"Folder not under rClone folder, skipping:\n $f\"" fi done # refresh calling app file listing if [[ "$activeApp" == "Finder" ]]; then # Finder has no Reload method, use its native update method osascript -e "tell application \"Finder\" to tell front window to update every item" else # for all other apps, try sending Cmd+R osascript -e "tell application \"System Events\" keystroke \"r\" using command down end tell" #osascript -e "tell application \"System Events\" to tell process \"Path Finder\" #click menu item \"Reload\" of menu \"View\" of menu bar 1 #end tell" fi
A few comments:
- Without adding absolute path to rclone (
/usr/local/bin/rclone
), the script wasn't able to find it. - To automatically refresh the file listing, the app finds out the active app and discriminates between Finder and other apps:
- If the refresh is called from Finder (= it is the active app), I use Finder-native method for refreshing file listing.
- Otherwise, in other apps, the script sends ⌘+R keystroke to the foremost app. In Path Finder, I have set to
View→Reload
menu item command. So that refreshes the listing of items. - The alternative (commented in code) would be to manually invoke clicking on
View→Reload
menu item, but that would work only in Path Finder. Sending ⌘+R is more versatile – I have set it as the “reload” command in several other apps I use.
- It would be easier to avoid using
osascript
calls and instead of those directly call the AppleScript code:tell application "System Events" keystroke "r" using command down end tell"''
This simply did not work for me. It is possible that it's related to the permissions model in Mojave+. I could not find the way to debug it. Calling the code indirectly through the
osascript
call worked, though, so that's where I stayed. - To find out which AppleScript commands a particular app supports, you can use
Script Editor.app
. Upon launching, go toFile
→Open Dictionary…
(⇧⌘O) and select the app you want to examine. This will show you the dictionary of app's supported AppleScript features.