Create an API for Data Backups

Right now there is no way to download instance’s data without hacking around with Docker’s APIs. There is a need for a new API to download(export) and upload(import) data of stateful services like databases.

There are many motivation behind this but this is specially needed to prevent data loss by scheduling regular backups.

My suggestion is to add new funcs to Instance API to download and upload data to/from specific directory inside data volume.

Services like service-mongodb should support creation of multiple databases because a service might be used by multiple users in the network. All users should have access to their own databases by providing a secret that genered during the creation of database. This behaviour should be supported by “good quality” services.

This is why it’s important to have the ability to download a sub directly inside a data volume. Because a user only should be able to download its own database data from the filesystem.

Sample API

Instance.DownloadBackup(instanceHash, directoryName)
Instance.UploadBackup(instanceHash, directoryName)

directoryName actually the “secret” of user.
for example:

  • if directoryName is: 88jugi4963mk56pzh2dn6nh6nijg8gr1ltzdc64i
  • DownloadBackup() will download from a path like this: /var/data/88jugi4963mk56pzh2dn6nh6nijg8gr1ltzdc64i

An another option would be directly uploading backups to IPFS with encryption or enabling layered data synchronisations to a cloud storage service.

This is a good idea, I’m not sure about the export/import api, maybe having more information in the instance object like the list of volumes used, the different configurations, etc… and then a tool outside of the engine can do the export/import. This can even be done by the CLI itself or a dedicated tool that only manages import/export and I like the idea to have synchronization on IPFS with encryption but IPFS doesn’t really guarantee the persistence of data and it might be scary for people to put that on a public network (even if there is encryption).

IPS scares me too. IPS is also scary for me for “private” services to be uploaded there. Maybe MESG should give an option to users about if they prefer to use IPFS or bring their own cloud storage. But anyway, it’s an option.

I believe that putting too much logic into CLI will do no good since it limits users with javascript. It’s not required to put every API directly inside the Engine but there should be a way to interact with all features that MESG offers, in every programming language via an API.