情感分析总结

1. last3embedding

参考：https://blog.csdn.net/weixin_45839693/article/details/109378849

模型比较简单，就是将最后3层的cls取出来拼接在一起，接dense即可。

def forward(self, inputs):
    x = self.bert_layer(inputs, output_hidden_states=True)
    hiden_states = x.hidden_states
    pooled_output = x.pooled_output
    tensor = torch.cat([hidden_states[-1][:,0],hidden_states[-2][:,0],hidden_states[-3][:,0],pooled_output, dim=1)
    drop_out_l = self.drop_out(tensor)
    Dense_l = self.liner_layer(drop_out_l)
    outputs = self.soft_max(Dense_l)

改进：

• 接lstm

• 接textcnn