KDE 5 applications are slow

[ttuegel]

The KDE 5 tests on Hydra regularly time-out due to KDE 5 applications being so slow. This is mainly due to searching so many paths in XDG_DATA_DIRS, as @edolstra pointed out here. I thought it would be more appropriate to open a ticket for this issue because it has come up in different contexts. Let me review the problem itself and possible solutions.

Cause

This problem affects all KDE 5 applications, but the application in particular which triggered this issue was Dolphin, so I will address it specifically. Dolphin is slow because loading icons requires searching every directory in XDG_DATA_DIRS for icons. Dolphin renders many icons, so this issue hits harder than other applications. kdeWrapper adds many paths to this list so that Dolphin and all the libraries it depends on may find their runtime dependencies. The wrapper allows the dependencies to be resolved whether the application is installed in the user or the system profile and it allows the application to run cleanly from the Nix store. It is also necessary to allow installation of applications depending on different versions of KDE Frameworks or even Qt.

Solution (-1): symlink farming

Until recently, kdeWrapper worked by creating a symlink farm for all the runtime dependencies and adding that path to XDG_DATA_DIRS. This is theoretically nice: a filesystem is a tree, so this basically amounts to creating a lookup tree for all the runtime dependencies. The problem is, every symlink takes up one filesystem block; at 4 KB each, that adds up. As @edolstra pointed out in #20433, this can add up to ~50 MB per application and up to ~400 MB for a typical KDE 5 installation. This is obviously unacceptable!

Solution 0: KDE unwrapped

If we remove the wrappers and resort to propagatedUserEnvPkgs, we gain the speed of symlink farming without the overhead. However, it will no longer be possible to run KDE 5 applications from the Nix store. As @abbradar pointed out, this will resurrect the long-standing problem of Qt applications "poisoning" the system against applications using other Qt versions. As a result, it will probably be impossible to install KDE 5 applications using nix-env. It will also break KDE 5 applications running outside of KDE, except by an obscure workaround.

Solution 1: dynamic symlink farming

The problem with symlink farming is the filesystem block occupied by each symlink. Placing the symlinks on a block-less filesystem like tmpfs removes this overhead! This would require creating the symlink farm dynamically. That is non-negligible overhead, but in principle it only needs to be done once per store path per boot, so it may be an acceptable compromise. Unfortunately, it would be a security nightmare: it would essentially be possible for a non-privileged user to modify the XDG_DATA_DIRS of any running application at any time!

Since the only remotely-acceptable solution on this list is "just let KDE be slow," I'm open to any suggestions!

[abbradar]

Just my humble opinion (again, I'm not even using KDE!) but although all those options seem bad, I'd go with -1 instead of status quo -- disk space is much cheaper than our time that we spend waiting applications to load. 400 MB is bad but not as bad, and (as I've already mentioned in another comment) times would be much, much worse for people with spinning drives.

EDIT: disclaimer: I've tested times in a VM with an SDD, so 9P was involved -- this is entirely unscientific. I waited for a minute (also entirely subjective) but maybe it's because of VM, and on host machines times are better.

[ttuegel]

times would be much, much worse for people with spinning drives.

I have spinning drives; it's not noticeable because of the filesystem cache, but on the virtual machine, the extra cache pressure causes a cascading failure.

Ironically, the huge size increase doesn't affect the graphical installer because squashfs is block-less! Now, if only there was a way to get a bunch of squashfs images mounted into the Nix store...

EDIT: sorry, squashfs is block-ful, but it compresses away the overhead.

[abbradar]

@ttuegel Hm, sounds good but relying on cache still seems flaky to me. How long are cold boot times?

On the other hand my worries may be entirely ungrounded indeed -- let's bring in some data! I'll try to measure time of cold-booting Dolphin on my host and in a VM (with SSD). I don't have KDE so it would be a more or less good measure. Test plan: use xdotool to wait for window and close it, measure overall process time.

EDIT: by "good measure" I meant that I don't have anything needed for Dolphin in memory, so we get the worst time possible (which is what we want).

[ttuegel]

@abbradar Before you run any tests, let me check something about the symlink farm overhead. I think I can reduce it significantly because I was doing something wrong before.

[abbradar]

@ttuegel It's too late, I was interested in how to measure this ~_^.

Here's my script (EDIT: updated to do warmup of helper utilities):

#!/bin/sh

for i in `seq 1 10`; do
  # Drop caches
  echo 3 | sudo tee /proc/sys/vm/drop_caches >/dev/null
  # Warmup
  xdotool >/dev/null 2>&1
  grep >/dev/null 2>&1
  sleep >/dev/null 2>&1
  killall >/dev/null 2>&1
  # Actual experiment
  time -f 'ELAPSED: %e' result/bin/dolphin >/dev/null &
  # Wait for window to show up
  while true; do
    win=$(xdotool getactivewindow)
    if xdotool getwindowname $win | grep -q Dolphin; then
      break
    fi
    sleep 0.05
  done
  # Kill the app
  killall -9 dolphin
done

Results for my machine:

ELAPSED: 1.91
ELAPSED: 1.77
ELAPSED: 1.79
ELAPSED: 1.87
ELAPSED: 1.78
ELAPSED: 1.78
ELAPSED: 1.87
ELAPSED: 1.87
ELAPSED: 1.87
ELAPSED: 1.88

This is big but manageable if this is a cold boot. I'll postpone measurements in a VM until your experiment with symlink farm. Notice that this is still a very crude measurement (sleep 0.1 is used to wait).

[ttuegel]

I have a patch for smaller symlink farms now; ~10MB instead of ~50MB. I get these timings before:

ELAPSED: 6.46
ELAPSED: 6.75
ELAPSED: 5.95
ELAPSED: 6.27
ELAPSED: 6.58
ELAPSED: 6.52
ELAPSED: 6.49
ELAPSED: 6.27
ELAPSED: 6.45
ELAPSED: 6.47

and these timings after

ELAPSED: 4.08
ELAPSED: 4.63
ELAPSED: 4.45
ELAPSED: 4.64
ELAPSED: 4.87
ELAPSED: 4.60
ELAPSED: 4.35
ELAPSED: 4.86
ELAPSED: 4.63
ELAPSED: 4.67

That is a significant speed improvement!

@edolstra Is a ~10MB per-application overhead acceptable for a ~30% improvement in startup time?

[FRidh]

kdeWrapper is now used for each application. How about considering an env with KDE5 applications, and symlink everything that is needed for that environment (kde5.withPackages). If you will have only one application in an env, the size increase will remain the same. If you have multiple applications in an env, I would expect a smaller size increase because libraries are symlinked only once.

[ttuegel]

How about considering an env with KDE5 applications, and symlink everything that is needed for that environment (kde5.withPackages). If you will have only one application in an env, the size increase will remain the same. If you have multiple applications in an env, I would expect a smaller size increase because libraries are symlinked only once.

@FRidh This is possible. It is essentially what we do already for the Plasma desktop. Plasma consists of a collection of applications; rather than wrapping them individually, we wrap them into a single desktop. We can make this wrapper more expansive, but that will break application installation by nix-env.

[abbradar]

What about a hybrid approach? Make individually wrapped applications visible and installable in nix-env but also add kde5.withPackages :: (PackageSet -> [Package]) -> Package and promote it to be used in NixOS configurations?

[abbradar]

Maybe do it like this:

• kde5.foobar -- application foobar which is defined as: foobar = wrapPackage (callPackage ./foobar.nix { });;
• wrapPackage adds passthrough attribute unwrapped;
• withPackages takes a list of wrapped packages and re-wraps their unwrapped versions in a one big environment.

This way everybody can use kde5.foobar and those who want more optimized approach can throw it with other packages into withPackages and it will do the right thing.

[ttuegel]

withPackages takes a list of wrapped packages and re-wraps their unwrapped versions in a one big environment.

How should withPackages know which targets to wrap?