Collect stats from core


#1

Since we’re rapidly developing core, it’d be really helpful to collect some stats from it. That way, we can know that how many people are using mesg and what their specs are. This can help us to how much we should carry about being backwards compatible in this period and can give us a nice overall idea about our users.

We should be only collecting stats that doesn’t reveal anything private from devs perspective. For example we can collect running services count but not their names. Also allowing us to collecting stats should be an optional thing and devs can choose to give perms on core’s installation time.

Stats to collect (please extend this list with your preference)

  • installation count (ilgooz)
  • core’s version (ilgooz)
  • core’s running state to measure active core count and total running hours (ilgooz)
  • deployed, running services count (ilgooz)
  • os stats (ilgooz)

#2

Agreed. Particularly agree that this should be opt-in. When I have seen organizations default to gathering stats without permission it can erode trust.

What kind of stats are you most interested in collecting? This might help us explore the best options here.


#3

I think the stats from @ilgooz are pretty good.
I think that the total running hours will be kind of hard and moreover not really sure that this can be legally nice to do. I would recommend more the number of time that they use each commands from the CLI.

mesg-core start / stop / logs
mesg-core service ls / deploy / start / dev ...

This way we can really see what are the most used features and maybe we can see where the documentation is not clear enough from that, example, users in general use more the commands deploy / start / logs than the dev this probably means that they might not even know about the dev command because it’s not clear enough in the documentation.

We could for every single command executed send the a new metric with:

  • the name of the command
  • the arguments of the command (not sure this is really nice, what about privacy ? Someone who execute a task and give the private key of its ethereum address for example. I think it would be bad except if we can kind of obfuscate that)
  • the host os
  • the execution time
  • the core version

After there is just one point I’m worried about is on the legal part, especially now with all these GDPR and things like this, I’m not sure it’s legally easy to collect these kind of data or maybe just a big warning on the user when they decide to activate the stats but there is probably some legal stuff to do for that.

@jonobacon any suggestions on that ?


#4

I am not an expert on the legal side of things, but I would be very careful about any phone home functionality without the user expressly opting in. I am not sure what the GDPR implications for something such as this are.

In terms of tracking core usage, we can obviously count the number of downloads, but I wonder if there is a way for core to update a manifest of some sort (e.g. like how Ubuntu syncs it’s local package list with the master list in the package archive). This way we can track the number of syncs happening for this manifest on a daily basis. Obviously, you would need to make sure this works in an offline setting too.

Thoughts?


#5

As a start, I’d really like to have the stats described in the first post and I really like the idea of collecting stats from cli commands as @Anthony pointed.

We can also think about the usage scenarios of core, services and application to guess what stats we might also get interested in to collect. :slight_smile:


#6

Another thing that we could do is to have the metrics on each apis like that we can really see what features people are using. We will be able to track their application activity and also service activity.

I’m pretty sure we can find an easy way to integrate that with the groc api with a kind of middleware to inject.


#7

Hey all, was this metrics collection/tracking ever implemented?

If not, then we should kick it off, beginning with 2 or 3 key indicators, and expanding from there.

I will ask Adriaan to make a card for this. To whom should it be assigned?


#8

As discussed, the problem here is not the technical implementation but the legal one.

We need to ask the consent of the user.

@wayne if you could find example and prepare a short text to ask for consent that’s will help us a lot in order to implement this feature :+1:


#9

Is there a point in the “onboarding” process where the user has to agree to Terms and Conditions? @Nicolas do you mean the text should be included there?


#10

OK then, backing up from Core to the general project… Maybe we can start with more basic stats, e.g. daily downloads of the software.

I really can’t think of a more important KPI for MESG anyway.

Commits/week? Issues opened and closed per week? What else do we know that we can show investors, that we don’t need permission to collect? (Don’t need a dozen stats, 3-5 plenty right now.)


#11

You can have the number of downloads in tools like this one http://www.somsubhra.com/github-release-stats/?username=mesg-foundation&repository=core

I’m not sure how precise are theses numbers, I tried one time to download and it was not counted, maybe it’s unique download or maybe it was a cache system but it’s at least this. For now we are at ~350 downloads for all versions.


#12

https://help.github.com/articles/getting-the-download-count-for-your-releases/

I guess this site may use api from github so you can trust it. Or you can download the data directly from github api