frodovdr/node-red
1
0
Fork 0
mirror of https://github.com/node-red/node-red.git synced 2023-10-10 13:36:53 +02:00
Code Issues Releases Wiki Activity

Updates and cleanup

Mike Blackstock 2018-05-24 18:17:09 -07:00
parent 7d458c47e4
commit cacadc4947
1 changed files with 30 additions and 44 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

74
Pluggable-Message-Routing.md

@ -1,3 +1,7 @@
_**This is a work in progress.**_
## Description
Trello: https://trello.com/c/J7UDbQVP/66-pluggable-message-routing
> The mechanism by which messages are passed from one node to the next should be a pluggable component of the runtime.
@ -6,45 +10,24 @@ Trello: https://trello.com/c/J7UDbQVP/66-pluggable-message-routing
> * flow debugger
> * adding custom low-level logging of node send/receive events including the full message data
## Use cases
From above:
* flow debugging
* adding custom low-level logging of node send/receive events including the full message data
Expanding on multiple runtime instances:
* instances running on separate cores routed manually or based on a policy or algorithm.
* runtimes on separate machines in a cluster
* runtimes in a _fog_ deployment, e.g. on devices, gateways, cloud routed dynamically depending on the mobility of a device, associated connectivity, location, etc..
Thinking about at least two approaches. The first requires the user to manually specify, and possibly configure the router to use on a given wire, the next is about providing a runtime component that handles the delivery of some or all messages between nodes, closer to the intent of this epic I think.
There are at least two approaches. The first requires the user to manually specify, and possibly configure the router to use on a given wire, the next is about providing a runtime component that handles the delivery of some or all messages between nodes. Based on discussions, the intent of this epic is the latter.
> Note: The method for managing and distributing instances running in different cores or machines is outside the scope of this document. This document discusses how to 'tap' messages sent between wire-connected nodes in flows.
> Note: The method for managing and distributing instances running in different cores or machines is outside the scope of this document. This document discusses how to 'tap' messages sent between connected nodes in flows.
## 1. Manual routing using _wire_ nodes
## Pluggable _routing_ module
Note: This is really the same as creating a new node, and dropping it between wires.
In this idea, _Routers_ are a special type of node that are associated with wires. They always have one input and one output.
They are not completely hidden in the UI, but do not appear in the palette, and do not appear as nodes on the canvas. Instead they have a UI on top of a wire such as the name and status. The routers available could also appear in the info when you click on any wire. (Currently wire info is not used.)
The default routing is 'none', meaning send directly to connected node.
To change a wire routing:
1. User double clicks on a wire.
1. Wire configuration dialog appears. There we can choose what type of routing to use depending on router nodes installed, e.g. MQTT broker, IPC, TCP/UDP in a pop up.
1. User chooses 'MQTT router' for example.
1. The configuration for the MQTT wire routing node appears.
1. User configures router, configures associated configuration node if needed, leveraging existing node configurations.
1. User hits OK.
1. Wire node is added between source and destination node on the wire by the runtime. Wire is somehow annotated on the canvas indicating there is a router node associated with the wire, e.g name hovering over the wire as shown.
1. User hits deploy.
1. Flow works as usual. Router node implementation handles communications with outside systems or protocols.
[[https://raw.githubusercontent.com/wiki/SenseTecnic/node-red/images/wire-nodes.png]]
Wire nodes could be shown as a dot on a wire (for example). When the user hovers, the node name and status could be shown.
## 2. Routing using a pluggable _routing_ module
The first idea doesn't handle the case where the type of routing to choose between specific nodes in a flow is not configured by the user beforehand. Another approach is needed to handle the use cases where we want to tap all messages between nodes or need to make decisions on how to route messages between nodes at runtime.
To do this type of message routing, we need a way to tap any message between nodes, and an (optional) way for the user to configure the current routing system.
A routing module should be able to (potentially) tap any or all messages between nodes to make decisions on how to route messages at runtime and provide a (optional) way for the user to configure installed routing module(s).
### Runtime changes
@ -55,37 +38,40 @@ Considerations
* There are potentially two points to tap: the send or receive node methods.
* There are a number of (potential) optimizations for local routing that should be maintained.
From scanning the code, it looks like the send side is the best option since this allows a routing implementation to optimize how messages are sent between nodes on different or the same systems as is done in the local node-node case.
From scanning the code, it looks like the _send_ side is the best option since this allows a routing implementation to optimize how messages are sent between nodes on different or the same systems as is done in the local node-node case.
The default implementation would be the same as it is now, i.e. call node.receive() on downstream nodes (with current message cloning and other optimizations).
The default implementation would be the same as it is now, i.e. call `node.receive()` on downstream nodes (with current message cloning and other optimizations).
The minimal interface for the pluggable routing module could look something like this:
A minimal interface for the pluggable routing module could look something like this:
**function init(_runtime)**
#### `RouterModule.open(_runtime) => Promise`
Initialise the router, connecting to external systems as needed to eventually make routing decisions and deliver messages.
Initialise the router, connecting to external systems as needed to make routing decisions, deliver messages.
#### `RouterModule.send(Node source, Object msg) => Promise`
Asynchronously send message to downstream nodes in the port/wire lists of the source node.
**function send(Node source, Object msg)**
> Note that there is currently no support for determining whether a message has been delivered. This is planned for future API. To anticipate this, the router module could return a Promise to complete.
Send message to downstream nodes in the port/wire lists of the source node.
#### `RouterModule.close(_runtime) => Promise`
Clean up connection to external system used for routing on Node-RED close.
This simple interface would mean that a router implementation takes responsibility for sending _all_ messages. There is no way for a router to tap only specific messages or wires, leaving some to be delivered by the default local routing implementation, or to log or copy messages sent between wires before they are delivered locally.
This simple interface implies that a router implementation takes responsibility for sending _all_ messages. There is no way for a router to indicate to the runtime that some or all wires need to be delivered by the default local implementation.
To address this we need a way for the router to indicate which wires it has taken responsibility for, and which wires can should be handled by the default local implementation.
To address this we need a way for the router to indicate which wires it has taken complete responsibility for delivery, and which wires should be handled by the default local implementation.
This could be done in an all-or-nothing manner, returning true or false if the router handles message delivery, or on a per-wire basis. The router could return a subset of the source port/wire lists containing the wires that have *not* been handled by the router plug in. The default local routing could then handle delivery along the ports/wires not handled by the plug in.
This could be done on a per-wire basis. The router send calls could return a subset of the source port/wire lists containing the wires that have *not* been handled by the router plug in. The default local routing could then handle delivery along the ports/wires not handled by any plug in.
A method signature with this capability could look something like this:
**function send(Node source, Object msg): `[[String]]`**
#### `RouterModule.send(Node source, Object msg): Promise<[[String]]>`
or
**function send(nodeId, `[[String]]` wires, Object msg): `[[String]]`**
#### `RouterModule.send(nodeId, `[[String]]` wires, Object msg): Promise<[[String]]>`
A noop implementation would simply do nothing on initialization and return the source wire list on `send()`.
There could be a single router installed at any one time. Perhaps a composite router could be developed to chain routers if desired, e.g. a router to log messages that returns the full port/wire list, followed by a router that handles delivery of some messages to different cores, followed by the default local router implementation.
It should be possible to chain routers if desired, e.g. a router to log messages that returns the full port/wire list, followed by a router that handles delivery of some messages to different cores, followed by the default local router implementation.
**Open Issues/Questions:**
* performance and optimizations may be more difficult or limited. Code currently optimizes for no wires, single port & message, and anticipates pre-fetching downstream nodes. Not sure what is possible.