Add unified sentiment analysis based on UIE. #3694

1649759610 · 2022-11-07T11:04:04Z

PR types

New features

PR changes

Others

Description

Add unified sentiment analysis based on UIE.

…o develop

wawltor · 2022-11-08T02:42:25Z

applications/sentiment_analysis/README.md


-本功能在预测时需要传入测试集文件路径，可将测试集文件命名为`test.txt`， 然后放入 `./data` 目录下。需要注意的是，测试集文件每行均为一个待预测的语句，如下所示。
+#### 2.2.1 属性/观点分析


这里说的属性是指情感维度吗？看看要不要整体统一一下

这里的情感维度是属性，一般叫属性说法的会比较多一点

wawltor · 2022-11-08T02:44:54Z

applications/sentiment_analysis/README.md


-如果你希望自己尝试进行属性级情感分类模型训练，可使用4.1节中提供的 `cls_data` Demo 数据，或自己业务的标注数据重新训练模型，本项目已将属性级情感分类模型的相关训练和测试代码放入 `classification` 目录下，请到该目录下执行模型训练即可，更多的实现细节和使用方式，请参考[这里](classification/README.md)。
+#### 2.2.3 属性+情感分析


属性 + 情感极性分析？

已改为：属性 + 情感极性分析

wawltor · 2022-11-08T02:46:22Z

applications/sentiment_analysis/README.md


-## 6. 引用
+- 👉 [通用情感分析抽取](./unified_sentiment_extraction/README)


这里建议更多的进行一些引导，因为上面的方案介绍更像是介绍统一信息抽取的方案

后续整体ready后，会继续完善这块

wawltor · 2022-11-08T02:54:01Z

applications/sentiment_analysis/unified_sentiment_extraction/README.md

+<a name="2.1"></a>
+
+### 2.1 运行环境
+- python >= 3.6


python的版本建议升级3.7+ 后续Paddle的框架和套件不再支持python 3.6

已改为：python >= 3.7

wawltor · 2022-11-08T02:55:19Z

applications/sentiment_analysis/unified_sentiment_extraction/README.md

+
+**安装PaddlePaddle**：
+
+环境中paddlepaddle-gpu或paddlepaddle版本应大于或等于2.3, 请参见[飞桨快速安装](https://www.paddlepaddle.org.cn/install/quick?docurl=/documentation/docs/zh/install/pip/linux-pip.html)根据自己需求选择合适的PaddlePaddle下载命令。


这里也可以建议增加conda的安装方式

已修改为：
环境中paddlepaddle-gpu或paddlepaddle版本应大于或等于2.3, 具体可以参见飞桨快速安装根据自己需求选择合适的PaddlePaddle下载命令。如下命令可以安装linux系统，CUDA版本为10.2环境下的paddlepaddle，具体版本号为支持GPU的2.3.2版本。

conda install paddlepaddle-gpu==2.3.2 cudatoolkit=10.2 --channel https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/Paddle/

wawltor · 2022-11-08T03:13:02Z

applications/sentiment_analysis/unified_sentiment_extraction/README.md

+
+# visualization for aspect
+save_path = os.path.join(args.save_dir, "aspect_wc.png")
+vs.plot_aspect_with_frequency(sr.aspect_frequency, save_path, image_type="wordcloud")


这里的sr 定义的地方没有放出code

已添加：

define SentimentResult to process the result of sentiment result.

sr = SentimentResult(args.file_path, sentiment_name=args.sentiment_name)

wawltor · 2022-11-08T03:15:24Z

applications/sentiment_analysis/unified_sentiment_extraction/README.md

+```
+
+<div align="center">
+    <img src="https://user-images.githubusercontent.com/35913314/200213998-e646c422-7ab5-48ae-9e28-d6068cdf7b8f.png"/>


后续可以看看和pipelines的联动

已支持pipeline

wawltor · 2022-11-08T03:16:42Z

applications/sentiment_analysis/unified_sentiment_extraction/model.py

@@ -0,0 +1,41 @@
+# Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.


霖捷本周会把UIE放到 transformers/ernie/modeling.py里面，后续可以直接调用

同时可能要看看支持UIE-M模型，这个模型结构稍有区别

wawltor · 2022-11-08T03:18:36Z

applications/sentiment_analysis/unified_sentiment_extraction/predict/batch_predict.py

+from utils import load_txt, write_json_file
+
+
+def parse_args():


这里的yapf disable一下

这些代码后续会删掉，统一用taskflow

wawltor · 2022-11-08T03:22:13Z

applications/sentiment_analysis/unified_sentiment_extraction/utils.py

@@ -0,0 +1,821 @@
+# Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.


这个有个疑问，后续也是提供uie-base的微调能力？还是我们提供定制化的base medium nano mini相关版本

这里计划是提供uie-base的微调能力

codecov · 2022-12-16T13:53:54Z

Codecov Report

Merging #3694 (87782d3) into develop (d6ea3b0) will decrease coverage by 0.12%.
The diff coverage is 10.43%.

@@             Coverage Diff             @@
##           develop    #3694      +/-   ##
===========================================
- Coverage    33.11%   32.98%   -0.13%     
===========================================
  Files          400      400              
  Lines        56139    56396     +257     
===========================================
+ Hits         18588    18604      +16     
- Misses       37551    37792     +241

Impacted Files	Coverage Δ
paddlenlp/transformers/ernie/configuration.py	`100.00% <ø> (ø)`
paddlenlp/transformers/ernie/tokenizer.py	`41.93% <ø> (ø)`
paddlenlp/taskflow/sentiment_analysis.py	`12.23% <10.10%> (-9.20%)`	⬇️
paddlenlp/taskflow/taskflow.py	`75.29% <100.00%> (ø)`

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

wawltor

LGTM

1649759610 and others added 29 commits June 24, 2022 13:43

initial commit

37d27f9

refine readme

d65208e

refine codestyle

ac4d644

refine readme

3f433b9

refine readme

d3f6ada

fix model saving bug

54ed34b

Merge branch 'develop' into develop

63b0a76

initial commit

4669194

Merge branch 'PaddlePaddle:develop' into develop

f6f93e1

Merge branch 'develop' of https://github.com/1649759610/PaddleNLP int…

fb3ade1

…o develop

initial commit

a83a902

initial commit

7bd988a

Merge branch 'PaddlePaddle:develop' into develop

700810a

Merge branch 'PaddlePaddle:develop' into develop

68e025a

use common metric instead of eval_metrics.py and remove unuseful code

02a997b

Merge branch 'develop' of https://github.com/1649759610/PaddleNLP int…

1500e5f

…o develop

Merge branch 'develop' into develop

faaf5f5

Merge branch 'PaddlePaddle:develop' into develop

6a512a0

Merge branch 'PaddlePaddle:develop' into develop

a99fc68

Merge branch 'PaddlePaddle:develop' into develop

4b5fa30

mv stage project to ASO_analysis

b837b67

add unified sentiment analysis

f415740

Merge branch 'PaddlePaddle:develop' into develop

41b020d

Merge branch 'develop' of https://github.com/1649759610/PaddleNLP int…

252860d

…o develop

refine readme

aed2d64

refine readme

8899a12

refnie readme

425a273

add unified sentiment analysis

acd9add

refine readme

4796016

wawltor reviewed Nov 8, 2022

View reviewed changes

1649759610 added 15 commits December 16, 2022 12:24

add deploy with SimpleServer

ee5938b

refine codestyle

46c417c

refine readme

96bd92f

add uie-senta-x to support_schema_list

fbf6567

fix hard coding for mdoel anme

ac7c5ed

refine codestyle

d21452b

refine codestyl

4128753

refine codestyle

4d072d7

refine codestyle

661e944

refine codestyle

77c090f

refine codestyle

128d154

refine codestyle

8c59f76

refine codestyle

8651ae2

refine codestyle

e14ff4a

fix senta response

aae9da1

1649759610 closed this Dec 16, 2022

1649759610 force-pushed the develop branch from e14ff4a to 2063b13 Compare December 16, 2022 13:25

add uie_senta_x

bb441ca

1649759610 reopened this Dec 16, 2022

1649759610 added 5 commits December 19, 2022 11:33

refine codestyle

d99c204

remove lambda expressions

5650404

add link of senta pipeline

1614c6c

refine codestyle

5f89ceb

remove local path

9a9be2c

wawltor approved these changes Dec 19, 2022

View reviewed changes

Merge branch 'develop' into develop

87782d3

1649759610 merged commit f96e787 into PaddlePaddle:develop Dec 20, 2022

1649759610 mentioned this pull request Dec 22, 2022

PaddleNLP 2.4.6 Release Note Candidate #4206

Closed

1649759610 mentioned this pull request Jan 12, 2023

PaddleNLP 2.5.0 Release Note Candidate #4439

Closed


		本功能在预测时需要传入测试集文件路径，可将测试集文件命名为`test.txt`，然后放入 `./data` 目录下。需要注意的是，测试集文件每行均为一个待预测的语句，如下所示。
		#### 2.2.1 属性/观点分析


		如果你希望自己尝试进行属性级情感分类模型训练，可使用4.1节中提供的 `cls_data` Demo 数据，或自己业务的标注数据重新训练模型，本项目已将属性级情感分类模型的相关训练和测试代码放入 `classification` 目录下，请到该目录下执行模型训练即可，更多的实现细节和使用方式，请参考[这里](classification/README.md)。
		#### 2.2.3 属性+情感分析


		## 6. 引用
		- 👉 [通用情感分析抽取](./unified_sentiment_extraction/README)


		安装PaddlePaddle：

		环境中paddlepaddle-gpu或paddlepaddle版本应大于或等于2.3, 请参见[飞桨快速安装](https://www.paddlepaddle.org.cn/install/quick?docurl=/documentation/docs/zh/install/pip/linux-pip.html)根据自己需求选择合适的PaddlePaddle下载命令。

		@@ -0,0 +1,41 @@
		# Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.

		from utils import load_txt, write_json_file


		def parse_args():

		@@ -0,0 +1,821 @@
		# Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.

Add unified sentiment analysis based on UIE. #3694

Add unified sentiment analysis based on UIE. #3694

Uh oh!

Conversation

1649759610 commented Nov 7, 2022

PR types

PR changes

Description

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

define SentimentResult to process the result of sentiment result.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

codecov bot commented Dec 16, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

wawltor left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

codecov bot commented Dec 16, 2022 •

edited

Loading