3 Pluggable Message Routing

Mike Blackstock edited this page 2018-05-24 18:17:09 -07:00

Table of Contents

• Description
• Use cases
• Pluggable routing module
• Runtime changes
• RouterModule.open(_runtime) => Promise
• RouterModule.send(Node source, Object msg) => Promise
• RouterModule.close(_runtime) => Promise
• RouterModule.send(Node source, Object msg): Promise<[[String]]>
• RouterModule.send(nodeId, [[String]] wires, Object msg): Promise<[[String]]>
• User interface
• Router configuration storage.
• Module runtime configuration
• Discussion

This is a work in progress.

Description

Trello: https://trello.com/c/J7UDbQVP/66-pluggable-message-routing

The mechanism by which messages are passed from one node to the next should be a pluggable component of the runtime. This would enable, for example, a flow that spans multiple runtime instances. Other use cases:

• flow debugger
• adding custom low-level logging of node send/receive events including the full message data

Use cases

From above:

• flow debugging
• adding custom low-level logging of node send/receive events including the full message data

Expanding on multiple runtime instances:

• instances running on separate cores routed manually or based on a policy or algorithm.
• runtimes on separate machines in a cluster
• runtimes in a fog deployment, e.g. on devices, gateways, cloud routed dynamically depending on the mobility of a device, associated connectivity, location, etc..

There are at least two approaches. The first requires the user to manually specify, and possibly configure the router to use on a given wire, the next is about providing a runtime component that handles the delivery of some or all messages between nodes. Based on discussions, the intent of this epic is the latter.

Note: The method for managing and distributing instances running in different cores or machines is outside the scope of this document. This document discusses how to 'tap' messages sent between connected nodes in flows.

Pluggable routing module

A routing module should be able to (potentially) tap any or all messages between nodes to make decisions on how to route messages at runtime and provide a (optional) way for the user to configure installed routing module(s).

Runtime changes

An interface to a module that can intercept messages sent by any node is required. An implementation can then decide what to do with the message, e.g. log, forward to an external system or protocol.

Considerations

• There are potentially two points to tap: the send or receive node methods.
• There are a number of (potential) optimizations for local routing that should be maintained.

From scanning the code, it looks like the send side is the best option since this allows a routing implementation to optimize how messages are sent between nodes on different or the same systems as is done in the local node-node case.

The default implementation would be the same as it is now, i.e. call node.receive() on downstream nodes (with current message cloning and other optimizations).

A minimal interface for the pluggable routing module could look something like this:

RouterModule.open(_runtime) => Promise

Initialise the router, connecting to external systems as needed to eventually make routing decisions and deliver messages.

RouterModule.send(Node source, Object msg) => Promise

Asynchronously send message to downstream nodes in the port/wire lists of the source node.

Note that there is currently no support for determining whether a message has been delivered. This is planned for future API. To anticipate this, the router module could return a Promise to complete.

RouterModule.close(_runtime) => Promise

Clean up connection to external system used for routing on Node-RED close.

This simple interface implies that a router implementation takes responsibility for sending all messages. There is no way for a router to indicate to the runtime that some or all wires need to be delivered by the default local implementation.

To address this we need a way for the router to indicate which wires it has taken complete responsibility for delivery, and which wires should be handled by the default local implementation.

This could be done on a per-wire basis. The router send calls could return a subset of the source port/wire lists containing the wires that have not been handled by the router plug in. The default local routing could then handle delivery along the ports/wires not handled by any plug in.

A method signature with this capability could look something like this:

RouterModule.send(Node source, Object msg): Promise<[[String]]>

or

RouterModule.send(nodeId, String wires, Object msg): Promise<[[String]]>

A noop implementation would simply do nothing on initialization and return the source wire list on send().

It should be possible to chain routers if desired, e.g. a router to log messages that returns the full port/wire list, followed by a router that handles delivery of some messages to different cores, followed by the default local router implementation.

Open Issues/Questions:

• performance and optimizations may be more difficult or limited. Code currently optimizes for no wires, single port & message, and anticipates pre-fetching downstream nodes. Not sure what is possible.
• should we first query the router for the revised wire list, then send the message? This might allow the local implementation to optimize, and allow local messages to be sent before routed messages.
• how will this work with extension to APIs needed to provide message delivery guarantees (ref)? Does the router provide a guarantee that it has sent the message to all wires it has handled? Is there a promise or callback?

User interface

A router implementation could supply a user interface to configure itself using a node implementation in the same package. This node could extend the node-red user interface by adding (for example) an additional 'Router' side panel as the Dashboard and Debug nodes do. If needed, this panel could be used to communicate with an endpoint added by the node for configuration.

Router configuration storage.

A router can use any way it wants to store its configuration, but the router module will have access to the runtime in case it wants to store and retrieve configuration in a flow.

Module runtime configuration

To configure node-red to use a router, the router module would be set in the settings, in a similar way to a storage module.

Discussion

Discussion from slack and elsewhere - these may be edited!

**dceejay** - another approach could be (rather than start to give wires properties) - for nodes to declare a “location” where location could be physical, or abstract, and the deploy process would then take care of interpreting what it means to go from location A (localhost) to B(mqtt://foo.bar.com:1883) to C(http://moo.com:8080) . If two connected nodes are in the same location then the existing node to node comms would be the default etc.

mike - Yes. I was thinking a router module (idea #2) could use node properties on each side of the wire to decide when and how to route. I will write that down. Good use of system wide node properties also on the board

dceejay - Yes indeed - in the general case it is “just” a system wide property for each node - that code be instantiated config node style (ie not in palette per se) - and itself have config nodes to config sub-properties.

mike - Hmm. I see. The pluggable router code could be instantiated on some or all sending nodes. If it exists, node send() could delegate to that.

knolleary - The intention of the item is pluggable message routing that is - at its core - invisible to the user so its very much more your #2 rather than #1

dceejay - (I think the original thought came more from the ability to be able to replace the internal messaging (for debug / trace etc) - and grew out to other transports - which then of course intersects with the distributed Node-RED flows thoughts - so in my mind there is quite a bit of overlap)

so both debates need to happen… … at some point.

• Design Notes
• Tech Interchange Meetings