Simplification of the task's output

proposal

#1

After few months using MESG Core, I realize that create task’s outputs is a bit overcomplicated.
For now we have the following:

One task has one payload of inputs and multiple possible outputs with their own payload.

This is good to have multiple outputs in a function outputs are mostly (if not all the time) something like success or error.

We could simplify this and just have one output from the task with its own payload. Of course the service could still raise an error.

PRO:

  • Simplification of the mesg.yml
tasks:
  xxx:
    inputs: {...}
    outputs:
      foo: { type: String }
  • Simplification of the libraries
// mesg lib catches empty result and do a big try catch on the function
// also now the task don't have the responsibility to call the the submit result the lib can control only one submit result
mesg.listenTask({
  foo: (inputs: object): object => {}
})

CON:

  • A huge breaking change as all existing services will be broken
  • Need update on the API
  • Need update on the libraries

We could have some way to minimize the impact with some conventions. If you are using success/error then we can automatically assign the success as output and catch the error but that will be ugly.


#2

I totally support this simplification even if it breaks every services!

I would like to add also a PRO argument:

The states of execution will also be simplified and more precise.
Currently, there is a error state only for error occurring inside the Core.
If the error output is managed by the Core, then the Core could set the execution’s state to error as well. It will be a very nice feature to allow a better retry system on execution.

As @Anthony said, the libs should catches error throwed or returned by the task function.
But gRPC api must integrate the error.
The current message containing the result of an execution is:

message SubmitResultRequest {
  string executionID = 1;
  string outputKey = 2;
  string outputData = 3;
}

I suggest to transform it to something like:

message SubmitResultRequest {
  string executionID = 1;
  oneof output {
    string outputData = 3;
    string error = 4;
  }
}

#3

I was thinking about this for a while and I agree on this simplification. And since we now support object types in the definition format, it’s still possible for devs create different type of outputs by directly defining them in the output data itself.

One other thing to think about is, we should be able to filter by output types inside the applications. If we remove this feature now, devs will need to do this filtering directly on the data by checking which output presents in the data object.

Also, we don’t have many input data types in events and task inputs too. So having them for only task outputs doesn’t makes too much sense.


#4

I’m not sure to understand. Can you explain more please.

Yes i agree. But this filtering should be done the same way on both task’s output and events!


#5

I’m not sure to understand. Can you explain more please.

Nothing important. I say that we have outputX, outputY etc. in the task results and we don’t have the similar thing for events to accept different inputs like inputY, inputY or for task inputs. So, there is no a good reason to keep this pattern just for task outputs, users can implement their own since we support the object data type now.