Test of MESG Services

It will be really useful to have an integrate way to execute test on MESG services.

I was thinking about a command mesg-core service test that run tests against the service.

I think we have two approaches on testing:

Integration test

Advantage:

  • language agnostic. we provide a way test services in any language because of the design of MESG. It could be based on a test.yml file that describe the tests as a list of tasks with inputs and expected outputs.
  • test communication with core

Disadvantage:

  • no way to force a service to emit an event without introducing some kind of hack
  • could introduce a MESG_IS_TESTING env variable
    • tell the service to not really execute the task (only valid in some special case, eg: Ethereum Transaction)
    • force to emit event
    • the main problem is the service logic/behavior could completely change during test
  • slow
  • the coverage of test could be very limited because of the technology the service connect
    • could not be sure all type of events will be emitted
    • could possibly not execute all task because of security (private keys) and duplication of data, etc…

Unit test

Advantage:

  • fast to execute
  • test written by the developer
  • can inject mock in service when testing

Disadvantage:

  • language specific: so we cannot offer a way to unit test in every language
    • but we could provide test helpers in go-service and mesg-js libraries (see end of post for mesgtest package)
  • no test communication with core

MESGTEST

@ilgooz already create a package mesgtest in the go-service library

Code for testing of the log task:

_, execution, err := server.Execute("log", data)
assert.Nil(t, err)
assert.Equal(t, "success", execution.Key()) 

source

@core I would like to have your feedback on:

  • Do we really want to offer an integrated way to do unit test if the service is using our library (go-service and mesg-js) ?
  • If so, how should we proceed in order to have the minimal logic to implement in order to accept unit test?
    • Bypass gRPC or create a gRPC server?
  • How could we execute language specific unit test from the command service test?
    • @Anthony was suggesting to accept a custom command in the mesg.yml to use for test instead of the default in the Dockerfile of the service( eg in mesg.yml: test-command: "go test ./...")
1 Like

Ideas About Unit Testing

In Nodejs it’s very easy & natural to modify methods of imported packages by using stubs. Please check sinon. So, it’s not required to provide a testing package for Nodejs. But it’s nice to have because it’s a bit odd to force devs to stub our gRPC calls.

Since Go is statically typed, it is not possible to modify methods of imported packages. So the same thing with the Nodejs case is not possible here.

There are several options while unit testing a package that uses go-service or go-application.

Options

not an option

Sample case:

Providing a package level interface for mocking methods of go-application is not an option, here is why:

func (a *Application) Close() error
func (a *Application) WhenEvent(serviceID string, conditions ...EventCondition) *EventEmitter
func (a *Application) WhenResult(serviceID string, conditions ...ResultCondition) *ResultEmitter

The methods above are the public api of go-application package and it’s not possible to have a ApplicationInterface for them like below:

type ApplicationInterface interface {
   Close() error
   WhenEvent(serviceID string, conditions ...EventCondition) *EventEmitter
   WhenResult(serviceID string, conditions ...ResultCondition) *ResultEmitter
}

The reason why it’s not possible is because EventEmitter and ResultEmitter types has un-exported fields and they cannot be properly initialized from outside of the package context.

One way to avoid this limitation is to make every public package api to return or accept an interface but I don’t like this approach because it’s not natural to have interfaces all the time just for testing.

network level option

Packages that deals with IO (like go-application) gets a custom net.Conn in the initialization time. And it can be used to fake an actual network connection in testing. mesgtest does the same thing and provides some extra functionalities for faking calls. Most network programs tested by this approach.

interface level option

Packages that deals with IO (like go-application) gets a custom interface to deal with network. This is very similar to network level option but as a difference, network methods are being mocked instead of faking in the network layer. mesgtest package can adopt both network and interface level options under the hood.

Which Option to Chose?

Regardless of which option we chose, I think it’s better to provide a package like mesgtest to devs. Otherwise we’ll require devs to create their own testing utilities (just another implementation of mesgtest) and this is not something that they should deal with.

Because they’ll be required to create mocks for our gRPC calls which they even shouldn’t be aware of. Also if we don’t provide a testing package, community can end-up creating a bunch of different ones which is not good to me because not everybody going to provide the best solution and this will create a bad noise.

I would definitely not do anything about the testing without a lot of feedbacks for that. For my part the only thing that we can do is let the user create their own tests and decide if they do unit/integration test.

For unit test

They use the framework they want and they is no link with MESG and we give the possibility in the mesg.yml to add your test command that can be executed by the CLI/Core.

For integration tests

The only thing that we could do is to create a test service that will emit some events that developers can connect to and some tasks like assertBool, assertContains
With that developers can connect test events to trigger their task using MESG application (and later on workflow) and also assert their results using the MESG test service and the assert* tasks.

The unit test part is quite easy and brings a lot of value and the integration is something additional that developers decide to use or not but the good point is that it’s something totally independent from the Core so we have a big flexibility.

1 Like

I really like @Anthony suggest about the Integration test: just create special task.

Let’s not do anything about tests.
If a developer of a service want to do unit tests on its service, (s)he choose how to do their unit test based on the framework they prefer.
If they want to do integration tests, they can also use the test framework they like and they will just need to communicate with the engine API.

You can find an example of integration tests here https://github.com/Roms1383/mesg-pusher/

This way anyone can use the test frameworks they like with the CI/CD they want and without having to learn about another test framework