Python PyTorch HuggingFace License: MIT

A structured approach to opinion mining — extracting who expresses what sentiment toward whom, with a polarity label.

📋 Table of Contents

🎯 Problem Statement

Formal Definition

Illustrative Example

Sentence: "Even though the price is decent for Paris, I would not recommend this hotel."
Opinion Tuple 1:
 Holder → "I"
 Expression → "would not recommend"
 Target → "this hotel"
 Polarity → NEGATIVE
Opinion Tuple 2:
 Holder → (implicit)
 Expression → "decent"
 Target → "the price"
 Polarity → POSITIVE

🏗️ Architecture

 ┌──────────────────────────────────────────┐
 │ Input Text │
 └───────────────────┬──────────────────────┘
 │
 ┌───────────────────▼──────────────────────┐
 │ [CLS] w1 w2 ... wn [SEP] │
 │ Token Embedding Layer │
 └───────────────────┬──────────────────────┘
 │
 ┌───────────────────▼──────────────────────┐
 │ Norwegian BERT Encoder │
 │ (NbAiLab / nb-bert-base) │
 │ Contextual Hidden State Representations │
 └──────────┬────────────────────┬──────────┘
 │ │
 ┌───────────────▼──────┐ ┌─────────▼────────────┐
 │ CRF for BIO Tagging│ │ Polarity Head │
 │ │ │ ([CLS] token) │
 │ B-Source I-Source │ │ │
 │ B-Target I-Target │ │ ┌──────────────┐ │
 │ B-Polar I-Polar │ │ │ Positive │ │
 │ O │ │ │ Negative │ │
 └──────────┬───────────┘ │ │ Neutral │ │
 │ │ └──────────────┘ │
 │ └─────────┬────────────┘
 │ │
 ┌──────────▼─────────────────────────▼──────────┐
 │ Structured Opinion Generation │
 │ │
 │ Merge CRF spans + polarity predictions │
 │ Apply offset mapping & span filtering │
 │ │
 │ Output: (Source, Target, Expression, Polarity)│
 └─────────────────────────────────────────────────┘

Model Components at a Glance

🔬 Methodology

1. Initial Processing & Tokenization

# Encoder produces hidden states for every token position
hidden_states = roberta_encoder(input_ids, attention_mask)
# Shape: [batch_size, seq_len, hidden_dim]

2. Tag Classification Layer (BIO Tagging)

BIO Tag Space

O → Outside any opinion component
B-Source → Beginning of a Holder span
I-Source → Inside a Holder span
B-Target → Beginning of a Target span
I-Target → Inside a Target span
B-Polar → Beginning of a Polar Expression span
I-Polar → Inside a Polar Expression span

Projection Formula

3. Conditional Random Field (CRF)

CRF Training Objective

4. Polarity Classification

Sentiment Labels:
 ✅ Positive → Favorable / approving sentiment
 ❌ Negative → Critical / disapproving sentiment
 ➖ Neutral → Factual / non-opinionated expression

5. Structured Opinion Generation

🔁 Baseline System

Pipeline Overview

Step 1: Span Extraction
 ├── BiLSTM → Extract Holders
 ├── BiLSTM → Extract Targets
 └── BiLSTM → Extract Polar Expressions
Step 2: Relation Prediction
 └── BiLSTM + Max Pooling
 ├── Full text representation
 ├── Holder / Target representation
 └── Expression representation
 └── Concatenate → Linear → Sigmoid
 → Binary: has_relation? (threshold = 0.5)
Step 3: Assemble tuples → (holder, target, expression, polarity)

✨ Features

📦 Datasets

Subtask 1 — Monolingual

Subtask 2 — Cross-Lingual

Data Format

{
 "sent_id": "opener/en/hotel/english00164-6",
 "text": "Even though the price is decent for Paris, I would not recommend this hotel.",
 "opinions": [
 {
 "Source": [["I"], ["44:45"]],
 "Target": [["this hotel"], ["66:76"]],
 "Polar_expression": [["would not recommend"], ["46:65"]],
 "Polarity": "negative",
 "Intensity": "average"
 }
 ]
}

📊 Results

Monolingual Performance

Cross-Lingual Performance

🔍 Key Observations

🚀 Installation

Prerequisites

1. Clone the Repository

git clone https://github.com/your-username/Structured-Sentiment-Analysis-IIITD-NLP-PROJECT.git
cd Structured-Sentiment-Analysis-IIITD-NLP-PROJECT

2. Install Core Dependencies

pip install torch torchvision transformers
pip install nltk scikit-learn tqdm gensim

3. Install Baseline Dependencies

pip install -r baseline/sequence_labeling/requirements.txt

4. Install Data Processing Dependencies

pip install -r data/requirements.txt

5. Prepare External Datasets

cd data/mpqa && bash process_mpqa.sh

cd data/darmstadt_unis && bash process_darmstadt.sh

🛠️ Usage

Evaluation

python evaluate.py <input_dir> <output_dir>

Baseline — Training

# Train all BiLSTM baseline models across datasets
cd baseline/sequence_labeling
bash get_baselines.sh

Baseline — Inference

# Run inference on a specific dataset and split
python baseline/sequence_labeling/inference.py \
 --DATADIR opener_en \
 --FILE dev.json

baseline/sequence_labeling/saved_models/relation_prediction/<DATADIR>/prediction.json

Predictions Format

{
 "sent_id": "unique-sentence-id",
 "text": "Raw input sentence here.",
 "opinions": [
 {
 "Source": [["holder text"], ["start:end"]],
 "Target": [["target text"], ["start:end"]],
 "Polar_expression": [["expression text"], ["start:end"]],
 "Polarity": "positive"
 }
 ]
}

📁 Project Structure

Structured-Sentiment-Analysis-IIITD-NLP-PROJECT/
│
├── 📄 evaluate.py ← Official evaluation script (SF1 / SP / SR)
│
├── 📁 baseline/
│ └── sequence_labeling/
│ ├── extraction_module.py ← BiLSTM span extractor (Holder / Target / Expr)
│ ├── relation_prediction_module.py ← BiLSTM relation classifier
│ ├── inference.py ← End-to-end inference pipeline
│ ├── convert_to_bio.py ← Convert JSON data to BIO format
│ ├── convert_to_rels.py ← Convert predictions to relation pairs
│ ├── utils.py ← Data loading & vocabulary utilities
│ ├── WordVecs.py ← Pretrained word embedding loader
│ ├── get_baselines.sh ← Script to train all baseline models
│ └── requirements.txt
│
├── 📁 data/
│ ├── norec/ ← Norwegian multi-domain reviews
│ │ ├── train.json
│ │ ├── dev.json
│ │ └── test.json
│ ├── opener_en/ ← English hotel reviews
│ ├── opener_es/ ← Spanish hotel reviews
│ ├── multibooked_ca/ ← Catalan hotel reviews
│ ├── multibooked_eu/ ← Basque hotel reviews
│ ├── mpqa/ ← MPQA news corpus
│ ├── darmstadt_unis/ ← English university reviews
│ └── README.md ← Data format documentation
│
└── 📁 predictions/
 └── norec/
 └── predictions.json ← Sample model predictions

🔭 Further Work

🕸️ Dependency Graph Parsing

🌐 Multi-task & Cross-lingual Learning

🔗 Advanced Graph Parsers

📐 Span-level Polarity

🤖 Serialized Large Language Models

📚 References

Barnes, J. et al. (2021). SemEval-2022 Task 10: Structured Sentiment Analysis.
 Proceedings of the 16th Workshop on Semantic Evaluation.
Liu, Y. et al. (2019). RoBERTa: A Robustly Optimized BERT Pretraining Approach.
 arXiv:1907.11692.
Kurtz, et al. (2020). Improving Low-Resource NMT through Relevance Based Linguistic Features.
 ACL 2020.
Samuel, D. & Straka, M. (2020). ÚFAL at MRP 2020: Permutation-Invariant Semantic Parsing.
 CoNLL 2020 Shared Task.

Made with ❤️ at IIIT Delhi | NLP Course Project

For questions or contributions, please open an issue or pull request.