Immutable data on Tendermint

Hey guys, I would like to discuss about implementing immutable data all across the decentralized databases.

After some discussion with @Anthony, we are thinking about implementing a more immutable system on all data that are on the decentralized databases. The data itself should never be updated or deleted but an other data that reference the first can change its behavior (the new data also cannot be updated).

It will also make the calculation of hash simpler by remove exception. Basically, all the data of a resource should be used to calculate the hash. If one the data is updated, then the hash is different and the resource is broken!

For example, the ownership system has been done this way to prevent the edition of service. Instead of having one owner in the service struct, the engine is using the ownership db to get the owner of this service. It will allow to change the owner without updating service. Moreover, it allow to implement a multi-owner system without having to change any data structure (this is a very important argument to not break the decentralized network)!

In my opinion, one exception should be implemented. It’s concerning the “deletion” of resource.
I don’t think having to reference the resource in a “deletion” store is good and practical.
I suggest to add to any resource that can be delete/deprecated/disabled a boolean parameter that will flag the resource as deleted/deprecated/disabled/etc only when it can happen maximum once.
This “deleted” parameter is not used for the hash calculation.

@core what do you think about such a system?

I would like to give a few case as example so we can agree on practical situation:

  1. the service deprecation system

This system will allow__only once__ a service to become deprecated and it will stay like this forever. In this case, is to simply add a parameter deprecated in the service struct that is omitted from hash calculation.

  1. list of nodes that run instances

The current proposal suggests to use a new dedicated store where each entry is referencing the instance hash and the node address. I think if we already agree with the beginning of this proposal, this is ok.
But it doesn’t say how a node can unregister itself. Should it remove its entry from the store? Should it update it by setting a removed bool parameter to true? Should a new store act as removed node?

For this case, I suggest the node unregisters itself by setting an unregistered param to true in this new store of nodes that runs instances.

  1. Execution

Execution is the resource that is and will be updated the most. It has many state: Created, InProgress, Executed, Validated, Failed and should implement a retry system.

Currently, all those data are in one resource that is being updated when needed.

By applying the immutable system, the execution should be splitted in many 3 resources: Execution, ExecutionResult and ExecutionValidation:

  • Execution
    • ParentHash
    • EventHash
    • InstanceHash
    • TaskKey
    • Inputs
    • Executor
    • Validators (for the future task validation system)
    • Tags
    • ProcessHash
    • StepID
      (almost all of current Execution except Outputs, error and Status).
  • ExecutionResult
    • ExecutionHash
    • Outputs
    • Error
  • ExecutionValidation (for the future task validation system)
    • ExecutionResultHash
    • IsValid
    • Validator

It will even be easier to check for write authorization and with the retry system, we will see that one Execution has many ExecutionResult. With the validation system, many ExecutionValidation for one ExecutionResult.

The data itself should never be updated

We can’t update data right now.

or deleted but an other data that reference the first can change its behavior

What is the reason to not delete the data?

It will also make the calculation of hash simpler by remove exception.

What exeption you are taking about and how the hash calucation become simpler?

I suggest to add to any resource that can be delete/deprecated/disabled a boolean parameter that will flag the resource as deleted/deprecated/disabled/etc only when it can happen maximum once.

I not sure about that. That is related with question about deleting the data.

By applying the immutable system, the execution should be splitted in many 3 resources: Execution, ExecutionResult and ExecutionValidation:

We don’t caluclate result in the hash. We don’t know how it should look like in the future so I wound’t rush with it so much.