mlsentiment: allow custom tokenizers (#1026)

* allow custom tokenizers

allows use of tokens object to specifiy custom tokenizers

updated mlsentiment to v2.0.0 (with temporary fix for comparative score applied)

updated documentation

* update unit tests
This commit is contained in:
Chuan Khoo
2023-09-01 06:25:00 +10:00
committed by GitHub
parent 8a29f9e379
commit b67452d569
5 changed files with 138 additions and 90 deletions

View File

@@ -4,14 +4,16 @@
<h3>Outputs</h3>
<dl class="message-properties">
<dt>sentiment <span class="property-type">object</span></dt>
<dd>contains the resulting AFINN-111 sentiment.</dd>
<dd>contains the resulting AFINN-165 sentiment.</dd>
<dt>sentiment.score <span class="property-type">number</span></dt>
<dd>the sentiment score.</dd>
</dl>
<h3>Inputs</h3>
<dl class="message-properties">
<dt>words <span class="property-type">object</span></dt>
<dd>an object of words and scores to override or add words can be supplied - <code>{ word:score,... }</code>.</dd>
<dd>besides sending a raw string in msg.payload, an object of words and scores to override or add words can be supplied - <code>{ word:score,... }</code>.</dd>
<dt>tokens <span class="property-type">object</span></dt>
<dd>a `tokens` object allows custom tokenizers which may be required by some languages - <code>{ ['世界',...] }</code>.</dd>
</dl>
<dl class="message-properties">
<dt>lang <span class="property-type">string</span></dt>

View File

@@ -4,17 +4,19 @@
<h3>出力</h3>
<dl class="message-properties">
<dt>sentiment <span class="property-type">オブジェクト</span></dt>
<dd>AFINN-111による感情分析の結果</dd>
<dd>AFINN-165による感情分析の結果</dd>
<dt>sentiment.score <span class="property-type">数値</span></dt>
<dd>感情分析スコア</dd>
</dl>
<h3>入力</h3>
<dl class="message-properties">
<dt>overrides <span class="property-type">オブジェクト</span></dt>
<dt>words <span class="property-type">オブジェクト</span></dt>
<dd>単語スコアの上書きをするためのオブジェクト - <code>{ word:score,... }</code></dd>
<dt>tokens <span class="property-type">object</span></dt>
<dd>一部の言語で必要となるカスタム トークナイザーが可能になります - <code>{ ['世界',...] }</code>.</dd>
</dl>
<h3>詳細</h3>
<p>ゼロ以上のスコアはポジティブゼロ以下はネガティブを意味します</p>
<p>スコアの範囲は通常-5から+5ですがより大きかったり小さかったりすることもあります</p>
<p>詳細は<a href="https://github.com/marcellobarile/multilang-sentiment/blob/develop/README.md" target="_blank">the Multilang-Sentiment docs here</a></p>
</script>
</script>