frodovdr
/
node-red
mirror of https://github.com/node-red/node-red.git
1
0
Fork 0
Code Issues Releases Wiki Activity

Updated Architecture: Cluster Multi Process (markdown)

TJKoury 2016-10-20 18:07:42 -04:00
parent fd62de8cbd
commit 918ab7742f
1 changed files with 9 additions and 3 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

12
Architecture:-Cluster---Multi-Process.md

@ -10,12 +10,18 @@ Communication from the Worker processes to the Master is handled via IPC.
|clusterRED.js| In charge of general process management. Returns a 'clusterRED' object. Has methods to initialize master and Worker processes, spawn and kill new processes, provide status on child processes. |
|red.js| Controls instantiation of admin server, api , core admin UI, and static server.|
|red/runtime/events.js| Modification to the runtime event bus. Sends messages from Worker to Master, which are then broadcast to all other Workers. Messages from master are (marked)(https://github.com/TJKoury/node-red/blob/cluster/clusterRED.js#L134) so as not to cause a rebroadcast cascade.|
|red/api/comms.js| Enables websocket comms bus if on Master process.|
|red/api/index.js| Enables api if on Master process.|
|red/red.js| Initializes the runtime and launches the adminApp, nodeApp, and assigns the server property to the adminApp server if the process is Master.|
|red/runtime/index.js|Enables logging for the Master process, sends messages that go across the runtime comm bus to the Master process for distribution.|
|red/runtime/nodes/Node.js| Intercepts messages from nodes that are running on the Master and redirects to a Worker based on a round-robin (https://github.com/TJKoury/node-red/blob/cluster/clusterRED.js#L88)|
|red/runtime/nodes/flows/index.js| Restart Workers after flow modification. Brute force method ensures no lingering issues with long-running processes, bound ports, etc., but does open up the possibility of broken pipes when writing to files / sending data / etc.|
|red/runtime/storage/localfilesystem.js|Modified to only allow Master to log.|
|settings.js|Modified to expose the cluster module in function nodes, also provides the default setting for number of Workers.|
##Issues
- [ ] **Modified nodes**. As nodes are capable of binding ports, listening to file-system events, spawning other processes, etc., simply clustering by forking can lead to serious issues. There appear to be two options: modifying the nodes to play nice, or build in strict limitations on what nodes can do.
- [ ] **Config node capabilities**. Currently, config nodes (and certain nodes like TCP) are allowed to bind ports, meaning that if flows are loaded equally in the Master and child processes, the Master process will listen on that port, and then block the port from being [bound specifically to distribute incoming to Workers](https://github.com/nodejs/node/blob/master/lib/cluster.js#L116). Some nodes might be run only in the Master (inject) and [serve events to Workers based on a round-robin approach](https://github.com/TJKoury/node-red/blob/cluster/clusterRED.js#L88). Limiting config nodes to run only on Worker processes, then moving any code that could cause issues into a config node, could be a standardization solution to this issue.
- [ ] **Config node capabilities**. Currently, config nodes (and certain nodes like TCP) are allowed to bind ports, meaning that if flows are loaded equally in the Master and child processes, the Master process will listen on that port, and then block the port from being [bound specifically to distribute incoming to Workers](https://github.com/nodejs/node/blob/master/lib/cluster.js#L116). Some nodes might be run only in the Master (inject) and [serve events to Workers based on a round-robin](https://github.com/TJKoury/node-red/blob/cluster/clusterRED.js#L88). Limiting config nodes to run only on Worker processes, then moving any code that could cause issues into a config node, could be a standardization solution to this issue.
- [ ] **Static server**. Run single-processed on Master, or instantiate new static path on each clustered http-server instance? Does this option reside in the http-server config node menu, along with a static path?