0 Design: Flow Linter

Kunihiko Toumura edited this page 2019-01-18 18:04:19 +09:00

Unescape Escape

Table of Contents

• Purpose
• Use cases
• Requirements / ideas
• User interface
• Command-line interface (batch-style)
• Integrated in Editor (in Sidebar, batch-style)
• Architecture
• Flow manipulation API
• pluggable rules
• Configuration
• Report format
• Other Consideration
• Implementation plan
• First step
• Next steps
• Related works

This file contains invisible Unicode characters

This file contains invisible Unicode characters that are indistinguishable to humans but may be processed differently by a computer. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

Purpose

Node-RED makes it easy to program by wiring nodes together. But sometimes, because of its less restrictive nature, programmer may write flows that are hard to understand. For example:

• Function node that has no name or description. The reader need to read internal JavaScript code to understand what the node do,
• Excessive amount of nodes in a single flow.

And, some subtle differences in programming style cause readability low.

• Size of flow on Flow Editor
• Grid size
• JavaScript coding style in function nodes.

The flow linter provides a framework that automatically checks whether a flow is compliant with rules and conventions.

Use cases

• To use the consistent coding rules in a team, the tool checks a flow before submit to a repository.
• To prevent bugs, the tool checks a flow when the programmer requests it, or continuously checks as a background task of Flow Editor.
  • Editor displays suggestions to user: 'This HTTP-in node doesn't have corresponding HTTP-out node.', 'These nodes in this flow must preserve message properties set by HTTP-in', etc.
• To prevent move/edit/delete node accidentally, some node position or property change can be restricted by rules.

Requirements / ideas

• There are various rules/conventions, and each organization/community/etc. has different policies on it. Because of this, the rules has to be pluggable and customizable.
• Linter is used in both batch-style (e.g. command-line interface) and on-the-fly-style (e.g. integrated in Editor). The core service of linter can be used in both usages.
• Rule configuration can be exported as JSON format and can be included in flow.json so that developers can distribute flow templates with their own restrictions.
• Use compatible format with other linter/tester tools for result output.

User interface

Command-line interface (batch-style)

Integrated in Editor (in Sidebar, batch-style)

Architecture

Flow manipulation API

The flow manipulation API provides a highlevel interface to handle flow.json file. Programmers need not to know about the format of flow.json and structure of flow object.

pluggable rules

Developer can create their own rules by writing plug-in module.

Following code shows example of plug-in, that checks existance of name of function node.

function check (afs, conf, cxt) {
    const funcs = [...afs]               // extract all nodes 
        .filter(e => e.type==='function')  // filter out unrelated nodes
        .map(e => {
            return {id:e.id, name:e.name}; // extract their node id and name
        });
    const verified = funcs
        .filter(e=> e.name === undefined || e.name === "")  // check existance of name
        .map(e=> {
            return {rule:"no-func-name", id:e.id, result:conf}; // generate result
        });

    return verified;
}

module.exports = {
    check: check
};

• Flow-check code in plug-in is used on both CLI (runs on node.js) and Editor (runs on browser). To use same code in both node.js and browser, there are tools for generate codes for both:
  • Browserify: get all dependent npm modules and put in one script file.
  • Babel: convert modern JavaScript (ES2015+) code to (traditional) JavaScript code that can be executed on various browsers.
• Language Server Protocol defines the protocol used between an editor and a language server that provides language features like lint. It may be alternative option to implement lint function as a language server embedded in a Node-RED server.
  • Language Server Protocol itself is aimed for line-oriented text programming languages. It is not suitable for visual programming language like Node-RED. If we adopt the LSP, We might incorporate only their 'Client-Server' architecture, and not incorporate their protocol or data model.
  • If we adopt this architecture, the linter need not to generate code for server and browser. But we have to estimate an overhead to send flow object from browser to server.

Configuration

Linter reads configuration files in following order:

1. Default configuration: nrlint.js in Node-RED install directory
2. Project-specific configuration: $HOME/.node-red/project/projectname/nrlint.js
3. Per-user configuration: $HOME/.node-red/nrlint.js
4. Command-line argument: nrlint --config nrlint.js / node-red --lintconfig nrlint.js

If there is a conflict between them, latter definition overrides former one.

• File format:

module.exports = {
    "plugins": [
        "func-style"
    ],
    "rules": {
        "func-style/no-func-name": "warn",
        "func-style/eslint": { semi: 2 }
    }
}

Report format

• should be similar format with other linting tools (e.g. ESlint, JShint, ...)

Other Consideration

• When validation is invoked in Editor, validation codes should be executed in editor. API call to server-side is expencive, and it is need to add extra adminAPI in the core.

Implementation plan

First step

• Implement CLI version
  • Considering future extention, e.g. use from Flow Editor, on-the-fly checking, etc.

Next steps

• Implement Editor-integrated version (in Sidebar, batch-style)
• Rule configuration UI
• Make rules be exported as JSON format and can be included in flow.json so that developers can distribute flow templates with their own restrictions.
• On-the-fly checking in Editor

Related works

• Design: Flow Manipulation API

• Design Notes
• Tech Interchange Meetings