Integration of workflows in services (solves service composition)

Goal

Having the possibility to create and connect existing workflows/applications (already discussed here Reusable Workflows & Workflow Marketplace).
Anything that an application is doing, a workflow should be able to do by providing a simple interface (and an advanced mode for the ones who want)

In order to solve all that and even more (like Service composition) we can merge the concept of application and services. This way applications are managed by the engine and the user don’t have to do anything more to start its application.
Also now that the service and the application/workflow are the same they can share the exact same properties and be shared and validate on the network the same way, we don’t need to have two different kinds of data to synchronize.

A service has at least one of the following properties:

  • tasks: to expose tasks for other services to execute
  • events: to expose events for other services to listen
  • workflows: to link events to tasks

Workflows can connect internal and external data. This fixes the Service composition problem.

Here is the list of possible connections for the workflow:

  • Listen to an external event and trigger an external task (pure application workflow)
  • Listen to an external event and trigger an internal task
  • Listen to an internal event and trigger an external task
  • Listen to an internal event and trigger an internal task (possibility to create services based on workflows)

With this in mind, users might want to define tasks and events for their internal use so we can introduce a new attribute visibility on the task and event that can be either external/internal or public/private (not decided yet but public/private makes more sense with “public” by default)

When the engine receives a private event/task, this will not be propagated to anyone except the actual service.

Implementation

Based on the current definition of the Workflow implementation we can use the exact same data structure for the service definition.

{
  tasks: []
  events: []
  workflows: [{
    name: "name of workflow",
    description: "description",
    trigger: {
      service: "serviceHash",
      eventKey: "xxx",
      filter: { x: "y" }
    },
    tasks: [
      { service: "serviceHash", taskKey: "xxx" },
      { service: "serviceHash", taskKey: "xxx" },
    ]
  }]
}

This implementation is limited to only the case “Listen to an external event and trigger an external task” as the serviceHash of the current service will not be defined. There are two ways to introduce internal events/tasks:

  • omit the service (if there is no service then it’s the current service)
  • introduce a keyword for the current service like this or self which is common in a programming language

With all that, we can now create applications with a workflow and also do Service composition as we can listen or execute a task from external services.

Tasks

  • Update service definition with a list of workflows
  • Start the workflow engine based on the workflows from all the services
  • When starting the workflow in the engine, make sure to replace the self/this with the actual service hash
  • (optional/to define) Deploy a workflow based on a definition in the mesg.yml file (other formats could be considered as long as they can “compile” to the format in the service definition)
  • Create a badass documentation to explain all that with nice schemas
  • Also explain the current limitation and the workarounds
    • Create a tree of executions based on results of another branch from another workflow
    • Access data from another branch by having an “aggregator” task in the other branch
  • Celebrate this feature :tada: