Brainstorming Service's Alias/SID/Versions and Data Volumes

Alias/SID/Versions

  • 1 - How to solve versioning problem for service? Think about the first reason why we wanted to have service aliases. We wanted to identify services with a unique name. This way we wouldn’t need to change service hash in the workflow everytime there is change on the service. But we didn’t talk about versioning a lot. We said that we could use service’s hash for versioning. But there is a catch. We can only use the latest version of a service if we just use its alias in the workflow. As a second, we can only use a specific version of a service if we use its alias and service hash together in the workflow. We don’t have a way of allowing to use a range of service’s version. Like semver provides. Because of that, any minor change on the service, again will lead us to update service’s hash in the workflow. Or when we use alias alone, any major change on the service can broke the workflow. The question is how to solve versioning problem?

  • 2 - What is the difference between SID and Alias, what problems they solve? How they solve the versioning problem? Can their names be changed after they pushed to network?

Data Volumes

  • 3 - Currently we use aliases to identify a volume. Running same services’ different versions at the same time will end up using the same volumes because aliases are the same for both of them. Do you think that this is proper?

  • 4 - Running a service’s two different versions at the same on a peer is a valid approach (remember Blue/Green Deployments). Do you think that different services or same service with different versions running at the same time should be able to access to the same volume? Multiple services writing data to the same volume is error prone. Do you think that we should disallow attaching same data volume to multiple services that are running at the same time (not in deploy time, on start time)?

  • 5 - How to use same data volume when service’s alias/SID/version changed? This is a valid approach. A new service or an existing service but with a new version should be able to use an old data volume. Do you think that we should provide the opportunity of reusable data volumes? A data volume might not be used by a service anymore, in this case should it be a valid thing to attach existent volumes to a new service or to the same service with a different version? Otherwise devs will need to do data migration for a new volume. This doesn’t make sense in the real world.

  • 6 - Do we have a plan for having decentralised data volumes in future? If so, does it makes sense to identify them with their own unique names which can be defined in the mesg.yml?

  • 7 - What is the relationship between service’s alias/SID/version and data volumes? Do you think that volumes must be strictly related with their service’s alias/SID/version? If so, why?

  • 8 - After you answered the questions above, do you think that somehow automatically managing the relationship between services and volumes are possible? Is it our concern or devs’? Should volumes have their unique names separately defined in the mesg.yml? This way, it’s possible to manage them separately without a need of relationship between service’s alias/SID/version.

First of all, let’s be clear on the vocabulary:
Hash: I would like everyone to use Hash instead ID. Hash is generated from the source code and the mesg.yml so it changed every time a modification is done on the service.
SID (stand for Service ID): a string chosen by the developer and set in the mesg.yml. SID should be as unique as possible and stay the same as long as the service has the same behaviour and use the same data/volume.
Alias: a very user-friendly string for LOCAL USAGE ONLY. I would recommend that the user set the alias when deploying a service.

  • 1 - Versioning is super complicated when it comes to running application. I would suggest to not create a versioning system that has too much automation because it will have limitation.

  • 2 - SID is the unique service id, it the way to identify for sure a service (even when service’s source code change), it should stay the same during the service lifetime. Alias is a feature for local development of application. SID cannot be changed after a push on the network because its part of the mesg.yml. Alias can be change because it’s a local config.

  • 3 - No

  • 4 - No, they should not access the same volume at the same time.

  • 5 - As long as SID stay the same, the new version will use the same volume as the old version. But the 2 versions cannot run at the same time.

  • 6 - No we don’t have plan for decentralized data volume

  • 7 - I think volume should be only and strictly link to SID.

  • 8 - Yes i think it’s possible

1 Like

@Nicolas

  • 1- & 2- In that case SID should only be updated when there is a major change. Otherwise changing SID for minor updates will require updates on Workflows as well. But still, not versioning minor changes removes some flexibility because Workflows may want to really use a specific version of a service and may don’t want to have the minor changes. Because changes may introduce side effects or even a new bug. To avoid these we need to provide the ability of using a fixed version of service otherwise every change on service but with the same SID can open a security issue for Workflow users because it’s controlled by service’s author. We can use service’s HASH here but it’s hard to keep track which version range it points to in the UX side.

  • 2- cont’d: Having both Alias and SID a bit too much. Isn’t it possible to use SID for local development too? If not why?

  • 7- I don’t agree because I think, It’s a valid thing to completely change SID and want to keep the same data volume. Think about Gmail and Inbox as MESG services. The’ve different names but one is created to replace other. It feels better to have the ability of using volumes in totally different services. But of course, not at the same time.

  • 8- Regarding to question 7, it is not possible to automatically manage volumes for services.

Note 1:
@Nicolas btw, it’d be nice if you extend your previous message and give some Alias and SID examples by keeping versioning in mind. So we can see how they look like.

Note 2:
I think having both SID and semver could be a solution for versioning services:
com.mesg.http-server:1.2.1.

This way it can be used by workflows with this syntax:
com.mesg.http-server:*,
com.mesg.http-server:^1.2.1.

Note 3:
By keeping “managing volumes automatically cannot be done by strictly linking them to services” in mind, we should also give unique names to volumes in the mesg.yml. This solves the data volume issue, gives the responsibility to devs and provides the most flexibility. This way, each data volume can be used by any other services but not at the same time.

This is a very big security issue. Services should never be able to access other services volume. Only when it’s update of the same service, then the volume is accessible.


name: "Bitcoin Service"
sid: "com.mesg.service.bitcoin.v1"
repository: "https://github.com/mesg-foundation/service-bitcoin"
$ mesg-core deploy https://github.com/mesg-foundation/service-bitcoin --alias bitcoin
hash: hash1
SID com.mesg.service.bitcoin.v1 -> hash1
alias bitcoin -> hash1
volumes com.mesg.service.bitcoin.v1 -> service with hash1

Then the user can either use the hash, SID or alias to use the service.


In case of small update (same volume):

name: "Bitcoin Service"
sid: "com.mesg.service.bitcoin.v1" // same same
repository: "https://github.com/mesg-foundation/service-bitcoin"
$ mesg-core deploy https://github.com/mesg-foundation/service-bitcoin --alias bitcoin
A service already use the SID. Do you want to remove the previous service with hash "hash1"? [y/N] y
hash: hash2 // different hash
SID com.mesg.service.bitcoin.v1 -> hash2
alias bitcoin -> hash2
volumes com.mesg.service.bitcoin.v1 -> service with hash2

In case of BIG update (different volume):

name: "Bitcoin Service"
sid: "com.mesg.service.bitcoin.v2" // different
repository: "https://github.com/mesg-foundation/service-bitcoin"
$ mesg-core deploy https://github.com/mesg-foundation/service-bitcoin --alias bitcoin
A service already use the alias "bitcoin". Do you want to steal the alias for this service? [y/N] y
hash: hash3 // different hash
SID com.mesg.service.bitcoin.v1 -> hash2
SID com.mesg.service.bitcoin.v2 -> hash3
alias bitcoin -> hash3
volumes com.mesg.service.bitcoin.v1 -> service with hash2
volumes com.mesg.service.bitcoin.v2 -> service with hash3

the alias in this system is not really useful. SID can be enough.

@Nicolas can you give a bit more explanation about how please? Since devs will have their private keys, they can use it to access their services/volumes and attach their own volumes to their own services. If a dev shouldn’t have access to a volume in the network, isn’t it the responsibility of private keys? Because we use private keys for authorization.

Do you mean to calculate the volume key with SID or DID + a new volume ID?

I’m not sure but should be something like that:
Company/Organization/User ID + a new uniquely chosen volume ID in the yml (What is a DID?).

Actually this is about how we integrate private keys in the network to identify service/volumes. It’s surely doable. I don’t know if want to introduce organizations etc. for services and volumes but authorization for volumes should be possible without them too. For example we can add some kind of cryptologic value as a prefix to SID or volume names that only can be generated by the devs who has the private keys.

I don’t have enough research about the topic for now but this should be doable.

Yes that’s doable. But let’s focus on the current problem :wink:

DID: Developer ID

So, in this context I think Note 3 does not lead to a security problem where private keys are correctly used.

Exactly as you said, if the workflow/app want to use a specific version, then it has to use the Hash.
The thing is with a Semver version based system, a lot of developer doesn’t respect (or know) the logic. And again, the best is to specify a x.y.z version, so in our case, it’s the same as Hash.

1 Like

In this context, I’m ok with only having SID without semver. This is surely something that we can easily improve in future if we feel that we need to. Adding semver later on doesn’t lead to an architectural change on SID or anything, so it’s ok.

One small advantage of having semver next to SID is, this will make SID to be constant because version will be defined within another prop. With a separate version prop, it’s easier to identify services because SID will stay as the same and another version of the service will not look like a totally different service.

But still we can provide the same feeling by encouraging devs using a SID naming notation like this: com.mesg.service.http-server.v1 (version at the end(as @Nicolas suggested before)). And we can even add a rule for this.

Btw, in Workflows, services will look like:
com.mesg.service.http-server.v1 > always the latest version of v1
com.mesg.service.http-server.v1:HASH > specific version of v1

This proposal has been released in Core v0.5.