Filter events and results by data

proposal

#1

From https://github.com/mesg-foundation/core/issues/195

Events/results filtering should be possible on values.

Let’s make a list of operator to implement:

  • Equal
  • Contain
  • lower, lower or equal, greater, greater or equal

We should make more research on this issue. We need a really efficient filtering system.

The core, once the network is ready, may have to filter thousands and thousands of events per sec


Reply from @Anthony

Also the goal with the network is to reverse this and instead of having many applications that listen and filter some events, have an event than can find all associated application that need this event with the filter applied.

We can easily implement this if we are based only on the event key with a query like service.all.where(listeners.contains(eventKey))

But it gets trickier when we want to search for the applications that matches a specific filter.

Example:
an application that needs to listen an event eventX that contains {foo: String, bar: String} with the filter foo=hello.

When an event {foo: "hello", bar: "world"} is coming we should be able to find the application without having to replay filters for all matching applications.

I’m pretty sure we can find a way to hash these filters if we only use the equal operator and get the matching applications for a specific event but if we add more complex operations gt, lt… i don’t see how so maybe we should implement only the equal operator and do the other filter on the application side only