Network implementation with CosmosSDK

Goal

Decentralization of the different resources managed by the Engine:

  • Services (to create this marketplace of services in our own network)
  • Instances (to be able to delegate execution to specific nodes)
  • Executions (to process executions on the network)

In order to distribute these resources, we need to make sure that every change of state is replicated on the network and verified by each node of the network before broadcasting it.

For that, we will use CosmosSDK & Tendermint that already implement this distributed state machine with a lot of really cool stuff.

Implementation

Details about CosmosSDK App concept, Architecture, Design

We will use what Cosmos provides:

  • Keeper/Store
  • Messages
  • Handlers
  • Querier

Keeper

The place to store the data. We actually have everything in the database package, this database package will disappear and the resource package (service, instance, and execution) will manage the storing based on these keepers.

Keepers expose getter and setters of the data. We should only read/write the data based on these keepers

Messages

Type of messages that can trigger actions on the data. These are the messages that will transit in the network (or within the local instance). This message will validate the basic data (light validation).

Messages are directly handled by Tendermint and will be propagated automatically (magic)

Handlers

Handlers are the actions that will update the Keeper based on Message received. This has most of the logic and could be delegated to the sdk package.

Handlers are called either directly from the sdk or call based on the routing defined by the application.

Querier

Not sure exactly how this is useful in our case but it allows to read the data, we should probably only read data based on a querier and never try to read directly from the keeper


We can implement all these objects directly in the resource packages:

service
   - type.go
   - keeper.go # implement the keeper
   - msgs.go # implement the messages
   - handler.go # implement the handlers
   - querier.go # implement the queries
   - codec.go # needed codec to save the data
instance
   - type.go
   - keeper.go # implement the keeper
   - msgs.go # implement the messages
   - handler.go # implement the handlers
   - querier.go # implement the queries
   - codec.go # needed codec to save the data
execution
   - type.go
   - keeper.go # implement the keeper
   - msgs.go # implement the messages
   - handler.go # implement the handlers
   - querier.go # implement the queries
   - codec.go # needed codec to save the data

Few notes from development:

  • docs for app.go is rather poor (cosmos-sdk-tutorial) and there is a lot going on there

  • I would rather do not go with the prosposed file structure. The sdk should be kept as seperate package and only save instance/service/execution object in single database (each db as kvstore put in an app struct like here https://github.com/cosmos/sdk-application-tutorial/blob/master//app.go#L64)

  • the rest package can show us how to expose protobuf api for communication. I have to implement BaseReq fro the protobuf. It is used in every request

  • how to we solve the creation of execution, service, instanace owner account?
    do we want some api to create them? How do we handle tokens for them, or there is no token at all?

  • How do we handle the update of execution - suppose the service A creates an execution and it’s the owner of it, then service B wants to put and ouptut for execution, but B is not the owner - I need to figure out this sceneario.

So I continue the creation of very simple example (no cli, no rest api, just protobuf api), simplify app.go to have only one store for execution (and just one method create an execution)

Could we start with no token and no owner of account? A pure trusted environment?

Yes let’s not worry about token yet, for the creation of resources:

  • Service: the node that deploys the service is the owner of it, other nodes cannot edit it
  • Instance: same as service
  • Execution: Shared edition, one node create the execution, the rest of the nodes can add their signature to the list of emitters (and the execution will be executed when enough emitters observed the event and create/update the same execution)

The proposed structure is not in the sdk but in the dedicated resources packages, /service, /instance, /execution but I’m open to discuss that and improve it

let’s not worry about rest or event grpc api now

That would be great, but as I saw, we can start without tokens, but we have to have accounts.

Yes, instances and services are rather easy, this is why I asked mainly about execution.

Have you seen api somewhere for shared edition?

How do you want to test it then? how do you want to create/stop instances, update execution without grpc api? Without api, I won’t be able to test the code.

And last - if accounts must exist, then how do we manage them (also the account must be handled to test cosmos-sdk stories)

This is just in the handler part and its validation, either only the owner can edit et we trigger an error if the sender is not the owner but in the case of the execution there is no owner, just the one that already append their data cannot change them.

We already have these apis, I was thinking you wanted to add new apis. We keep the current apis and adapt them to use the store

So this is the question can you store object (in our case exeuction) in kvstore without owner?

Yes I need to adopt them :slight_smile:

I’m pretty sure you can, owner is just a data like any other data. The ownership is controlled by the validation you are doing in the handlers and messages.