frodovdr
/
node-red
mirror of https://github.com/node-red/node-red.git
1
0
Fork 0
Code Issues Releases Wiki Activity
Page: Design: Version Control
Pages
API Reference Architecture Overview Architecture: Cluster Multi Process Contribution Process Dashboard layout tool Definitions Deprecated: Message properties overriding set node properties Design Notes Design: dynamic palette Design: API Token management Design: CLI tool to produce a node skeleton Design: Concurrent editing Design: Custom Node admin route handling Design: Dynamic MQTT node Design: Dynamic node's icon Design: Editable port labels Design: Editor Events Design: Editor Themes Design: Editor UI Customisation Design: Encryption of credentials Design: Enhancements to Split Join nodes Design: Flow Linter Design: Flow Manipulation API Design: Flow file format v2 Design: Function Library node Design: Library Sidebar Design: Logging Framework Design: Node Generator Design: Node Messaging API Design: Node blacklist whitelist function Design: Node module lifecycle Design: Node settings Design: Palette Management UI Design: Persistable Context Design: Platform Specific Nodes Design: Projects Design: Runnable Project Design: Runtime Editor Split Design: Runtime Events Design: Runtime extension points Design: Subflow Enhancements Design: Subflow Instance properties Design: Subflow Node modules Design: Usage telemetry Design: Using environment variables Design: Version Control Design: adminAuth User Management Design: external file for function and templates nodes Design: i18n de Design: i18n Design: msg.parts description of usage Design: multiuser Design: subflows vNext Design: subflows Design:Shared Global Context Editor Mouse Actions Editor Runtime API Flow Format Flow testing Home Input node feature definitions Live runtime information Node Design Guidelines Node documentation guide Node msg Conventions Output node feature definitions Pluggable Message Routing Query node feature definitions ReadMe Raspberry Pi Advanced Runtime Editor Comms protocol Technical Interchange 2018 04 13 Technical Interchange Meetings Testing UI testing Watch node feature definitions Web Nodes outline adminAuth Credential Authentication dynamic palette join in auto mode npm installable nodes
0 Design: Version Control
Nick O'Leary edited this page 2018-03-16 14:37:58 +00:00
Table of Contents

This entire design is out of date - and has been superseded by Design: Projects. Do not believe anything it says - @knolleary

Target: 0.15 + 1

When we talk about version control of node-red instances, there are two things that it covers:

  1. the nodes added to the environment via npm
  2. the deployed flows (+credentials)

It is desirable to have a single artefact (or logical set of artefacts) that can be maintained in version control and used to restore a system to a previous known state.

The priority is version control of flows/credentials - and is the main focus of this work. The Added Nodes scenarios are for a future piece of work.

Deployed Flows

Use cases include:

  • a user creates a new version of the flows by simply clicking deploy
  • a user can optionally add a comment to the version
  • a user can see a history of versions - showing date/time + comment (aka git log)
  • a user can load a previous version into the editor, which, if then deployed becomes a new version

API

Whilst clearly influenced by git as a likely common backend, the version control API needs to be careful to abstract the operations it provides sufficiently so that alternative VC systems can be used underneath it. It does not require the full breadth of capability that git provides (branching/merging etc).

The basic operations are:

  • get a version - defaults to the most recent version, but a version id can also be specified
  • save a new version of flows/credentials with an optional description. This returns a version id.
  • get the version history - returns a list of id/user/timestamp/description

The API should also allow a 'user' identified to passed to each of these calls.

The existing API has separate methods for saving flows and credentials. The new API must version them as a single entity, although underlying implementations may choose to store them separately.

Storage API

Storage.init(runtime)

  • runtime : a RED runtime object

Returns a promise that resolves when the storage subsystem has been initialised and is ready for subsequent API requests.

Passing in the runtime object allows the storage mechanism to access the settings, log components properly as well as comms if it needs to send notifications to the user.

Storage.getFlows([id])

  • id : optional : identifies a specific version to return

Returns a promise that resolves to the combined flow/credential configuration

Storage.setFlows(flowConfig,[options])

  • flowConfig : the combined flow/credential configuration
  • options : optional : an options object that can contain the following properties:
    • user : optional : an identifier for the user setting the flows
    • summary : optional : a short summary to associate with the change

Returns a promise that resolves to the id of the version saved.

Storage.getFlowHistory()

Returns a promise that resolves to an array of history entries. Each entry consists of:

  • id : the version id
  • date : the timestamp of the version (format tbd)
  • summary : the short summary of the change (will have sensible default, or blank, if none specified)
  • user : an identifier for the user who saved the version

Notes

  • The format of the combined flow/credential configuration is a vital detail that needs some more thinking. The starting point will be the already documented new flow format. With an additional credential property at the top level that contains the node credentials.

  • An implementation is not required to maintain any history - it could return a single entry from the getFlowHistory call. This reduces the burden on implementations

Runtime API

All of the following API calls should include the Node-RED-API-Version HTTP Header to be set to v2. If that header is not provided, it will assume to be a request to the v1 level of the API (ie, what we had before we added this functionality).

GET /flows/:rev

Returns a flow configuration. If :rev is not provided, it returns the active flow configuration. If :rev is provided, it can be used to return a previous version of the flow configuration.

Response:

{
    rev: "flow-revision",
    flows: [ {}, {}, {} ]
}

POST /flows

Updates the current flow configuration. The body can contain an additional summary property to associate with the flow version. The runtime will generate a rev value and return it in the response. The version will also be associated with the user authenticated to make the request.

Body:

{
    summary: "optional summary",
    flows: [ {}, {}, {} ]
}

Response:

{
    rev: "flow-revision"
}

GET /flows/history

TBD

UI Changes

There are two parts additions to the UI:

  1. checkbox on Deploy menu to prompt for deploy description
  • if checked, a modal dialog appears each time Deploy is clicked to allow the user to provide a description for the deploy
  • if not checked, clicking deploy does not prompt for a message
  1. an option on Deploy menu to show Deploy History
  • the option opens a new sidebar panel that lists the history of deploys.
  • each list entry provides the ability to load that version into the editor. If the workspace is currently dirty, doing so will first prompt to confirm changes will be lost.

Quick mock-up:

Implementation details

Local File System (default)

By default, the localfilesystem implementation should be entirely file based.

  • the base flow/credential files remain unchanged to how they work today

  • a new directory structure is created alongside the flow files in the :

     my-flows.json
 my-flows_cred.json
 .my-flows_history/
     log.json
     0.my-flows.json
     0.my-flows_cred.json
     1.my-flows.json
     1.my-flows_cred.json

    The .<file>_history directory contains the history of flow files; each prefixed with its version id.

    The log.json file contains the version history.

Git Repository

An optional storage module will be provided that uses a Git repository for its storage.

There are two uses to consider, that have a bearing on what configuration options need to be exposed.

  1. a user doesn't care how the repository is laid out, they just want it in a git repo that allows it to be maintained externally (eg GitHub).

    • User provides flow file name, eg my-flows.json - as they can today
    • Node-RED uses a repository under <userDir>/my-flows - creating it if needed
    • Node-RED stores my-flows.json and my-flows_cred.json under that directory

  2. a user wants to maintain a repo that contains more then just the flow files.

    • User provides git url for an existing repo, and a flow file name as a relative path within the repo.
    • If it is a local repo, Node-RED uses that repo directly. A user could configure a post-commit hook to handle pushing updates remotely.
    • Otherwise, Node-RED clones that repo under <userDir>/.flows and uses that, ensuring to do git push back to the origin
    • The user should be able to provide authentication settings for the git repo via settings - or by well-defined environment variables.

Added Nodes

Target: future

Currently, a user simply npm installs nodes in their user directory to add them to the runtime - or uses the admin api to do so. There is no requirement to maintain a package.json file. But it is that file that needs to be version controlled in order to have a restorable system.

This raises some questions:

  • should we try to create/maintain a package.json file under the user directory - in much the same way that npm install --save does.
  • should we do this via the Storage API so alternative (non-filesystem) based solutions can make use of it