Telemetry in Notesnook: Opt-in vs Opt-out

Being in the FOSS space, you slowly come to realize that people absolutely abhor the idea of telemetry in an open source app. It irks them to know that the software they are using might be sending some data behind their backs. It doesn't matter what data its sending — just that it shouldn't do it.

On the other hand, if a proprietary software is not held up to the same standards. It can get away with tracking its users, usually at the spyware level, without any complaints. The problem therefore is: should there be telemetry in FOSS?

Surveys and Polls

Anyone who has built a software knows how insignificant the results of a survey/poll can be. Only a minority of a minority of users would take the time to fill them out which quickly puts them in the useless category. Even if you are Elon Musk, with 122 million followers, polls are not a great way to get user opinion because the bigger the sample size, the bigger the gap between majority & minority.

Which brings us to...

Opt-out telemetry

We see this practice in a lot of FOSS/proprietary software. There is usually some popup like "Help us improve XYZ by sending completely anonymous usage data." which is already filled out. The users who take the time to read carefully might uncheck that box but most people out there skip ahead and use the app.

That's the problem.

The user hasn't actually consented to being tracked. They might not even know they are being tracked and they certainly don't know how to turn it off. Its like with the Cookie consent popups you see these days thanks to GDPR; they are so annoying that most of us just click allow to get rid of it because any other button will take you to this "Settings" page to check/uncheck a bunch of nonsense data categories, half of which can't even be unchecked.

The bright side of this story is that companies get a lot of data as a consequence and if its not FOSS, only Allah knows what data that actually is. So opt-out is clearly bad but what about opt-in?

Opt-in telemetry

No one actually does this because its too good to be true and too useless to make any sense. Just kidding.

Opt-in telemetry is fully consent-based and I have rarely seen an implementation of it that makes me want to opt-in. A dead checkbox saying, "Help us improve XYZ" doesn't work because as mentioned above, users just move on.

Ethically speaking, there can be no better balance of collecting usage data. Realistically speaking, there's no worse option because, as is the case with surveys, the sample size is just too small to be actionable.

Wrong data is worse than no data.

Suppose only 1% of your users opt-in to telemetry (which is not unrealistic) and after some internal debate about removing some feature X, you decide to just add a telemetry endpoint to see how many users are actually using it. See where this is going?

Out of those 1% users about 0.9% users regularly use feature X. So far so good. You run to your team and tell them about your recent discovery only to be pointed out, "what about the other 99% users?". At this point usually companies either give up or make telemetry opt-out.

What about zero telemetry?

Zero telemetry

This is every privacy enthusiast's pipe dream. Unfortunately, its only being practically used in side projects who rarely anyone cares about. But putting that aside, what's the point of zero telemetry?

Zero telemetry means zero visibility into what's going on in the app. Who is using which feature? What percentage of users are annoyed by this popup? Which is exactly what privacy entails. We see this in VPNs under the "no logs" policy (not sure how true that is).

So what should a company do?

What about Notesnook?

What about it? We recently went "opt-in" from "opt-out" due to privacy reasons. In all honesty, I wanted to go the "zero telemetry" route because the data we collect has very little practical use (aside from encouragement). So why "opt-in" then?

I think this is a safe "middle ground" in the world of telemetry. It won't put us at a disadvantage when we do have important data we want to collect from users & and it also won't allow us to be privacy invasive. The above points regarding "opt-in telemetry" hold true for Notesnook as well because for now we only have that dead checkbox. But that is going to change very soon (more on this in a later blog post).

What we collect?

I think it is very important for users to know what & when data is being collected. Here's a (small) list of events that trigger telemetry:

You might be wondering what else gets sent with all of that. We use Umami analytics for telemetry but instead of their tracker script we have an in-house script to control exactly the amount of data to collect. The main data collection function looks something like this:

async function collect(event: PageView | Event) {
  const {
    screen: { width, height },
    navigator: { language },
    location: { hostname, pathname, search }
  } = window;

  const screen = `${width}x${height}`;
  const currentUrl =
    (event.type === 'pageview' && event.url) || `${pathname}${search}`;

  const body = {
    payload: {
      website: `the-website-id`,
      hostname,
      screen,
      language,
      url: currentUrl,
      ...event
    },
    type: event.type
  };

  try {
    await fetch(baseUrl, {
      method: 'POST',
      headers: {
        'Content-Type': 'application/json'
      },
      body: JSON.stringify(body),
      keepalive: true
    });
  } catch (e) {
    console.error(e);
  }
}

You can read more about Umami at their website.

Conclusion

There's no conclusion, unfortunately. It's all a trade-off between privacy & convenience.