Create some test cases that push Nepomuk to the limits of what we are likely to need (I'm thinking 1,000 contacts with presence changes being received for 2 or 3 contacts a second). These tests should be easy to run (as unit tests) and record meaningful quantitative data as to the performance. They should probably be run directly on the NepomukStorage class with the methods being invoked directly by the test case, since it is only Nepomuk performance we are trying to assess.
Presence changes no longer go through nepomuk (plus this has been open for 2 years with no work, it won't happen any time soon) Loading is benchmarked inside libkpeople.