RT journal article
T1 Fuzzy logic-driven confidence aggregation for multimodal sentiment classification
A1 Balderas Díaz, Sara
A1 Guerrero Contreras, Gabriel José
A1 Bueno Crespo, Andrés
A1 Martínez España, Raquel
A2 Ingeniería Informática
K1 Multimodal sentiment analysis
K1 Fuzzy logic
K1 Confidence aggregation
K1 Deep learning
AB Computational intelligence focuses on intelligent computer systems that mimic human nature and linguistic reasoning. Sentiment analysis is an area of considerable relevance within computational intelligence. Multimodal sentiment analysis is an extension of textual sentiment analysis, where the sentiments of people’s opinions are analysed by including multimedia content in addition to textual content. This mode of sentiment analysis faces multiple problems, as the sentiments of text and multimedia content may be contradictory. In addition, another added factor is the imbalance of the data that these problems suffer from in certain topics, which causes a problem when generating intelligent models. In this paper, we design a novel approach for multimodal sentiment analysis, proposing a new way of labelling tweets, not always prioritising polarized classes but using annotator confidence. Moreover, during this design, an information integration and fusion methodology is proposed for the construction of a metamodel that includes fuzzy logic to perform information weighting according to the confidence of the annotator. This proposal has been applied a public unbalanced dataset of tweets with text and images, with a large unbalance towards the negative class label. Applying the proposed fuzzy methodology, we reached a macro-F1 score of 0.493 for the negative class, 0.681 for the neutral class, and 0.832 for the positive class. The model obtains satisfactory performance since the individual image and text sentiment analysis results are worse, especially the negative class, which in initial image classification achieves an F1 score of 0.08.
PB Springer Nature
SN 1573-7721
YR 2026
FD 2026-03-19
LK http://hdl.handle.net/10498/39210
UL http://hdl.handle.net/10498/39210
LA eng
DS Repositorio Institucional de la Universidad de Cádiz
RD 21-sep-2026