APSI Logo

Frequently Asked Questions (FAQ)

Answers to common questions about APSI: what it measures, how it works, how to interpret outputs, and key limitations and ethical guidance.


1. What is APSI?

APSI (Automated Political Stance Identification) is a theory-driven text analysis tool that estimates political positions expressed in text using Natural Language Inference (NLI).


2. What does APSI measure?

APSI analyzes text across three dimensions: Economic Left–Right ideology, Support for Liberal Democratic values, and Populist vs. Pluralist rhetoric.


3. How does APSI differ from traditional classifiers?

APSI does not train a supervised classifier on labeled stance datasets. Instead, it uses a hypothesis-based scoring framework with a pretrained political NLI model, enabling zero-shot and interpretable results.


4. What model does APSI use?

APSI uses the Political DEBATE model (Burnham et al.), which is specialized for political discourse analysis.


5. What is the Political DEBATE model built on?

It is built on a multilingual transformer base (mDeBERTa-v3 NLI) and then fine-tuned on political NLI data.


6. What is Natural Language Inference (NLI)?

NLI is a method where a model estimates whether a text supports (entails) a hypothesis statement, rather than relying on keywords alone.


7. What does “zero-shot” mean in APSI?

Zero-shot means APSI can estimate political stances without training on a labeled stance dataset for your specific task.


8. How does APSI decide whether a text is politically relevant?

APSI runs a topic relevance pre-check. If the text does not strongly match political topic hypotheses (threshold 0.6), it returns no score.


9. What happens if a text is not politically relevant?

The tool returns score = NA and an explanation that it cannot infer a political stance from the text.


10. How are hypotheses created?

Hypotheses are derived from political theory and expert survey codebooks, then written as natural-language statements suitable for NLI scoring.


11. How is the final score calculated?

APSI averages entailment probabilities for each side of a dimension, takes the difference, and scales it to a 0–10 score (clipped to stay within bounds).


12. How do I interpret Populism–Pluralism scores?

Lower values indicate more pluralist rhetoric; higher values indicate more populist rhetoric. The tool maps score ranges to labels like “Pluralist”, “Moderate”, “Populist”, etc.


13. How do I interpret Economic Left–Right scores?

Lower scores indicate more economically left-leaning language; higher scores indicate more economically right-leaning language.


14. How do I interpret Liberal Democracy scores?

Lower scores indicate weaker support for liberal democratic values; higher scores indicate stronger support for liberal democratic values.


15. What is the confidence score?

Confidence reflects internal consistency of the model’s hypothesis probabilities (e.g., lower variance means higher consistency).


16. What is contradiction detection?

Contradiction is flagged when both sides of a dimension receive comparatively strong support, suggesting mixed or conflicting signals in the text.


17. What does APSI output?

APSI outputs a score, interpretation label, confidence level, contradiction status, and indicators of which hypotheses influenced the score.


18. What kinds of texts can I analyze?

You can analyze speeches, articles, social media posts, policy statements, interviews, and transcripts—any text where political language might appear.


19. Does APSI use any contextual information?

No. APSI analyzes only the text provided and does not know the author, audience, or circumstances surrounding the text.


20. Does APSI measure beliefs or rhetoric?

APSI measures rhetorical framing in language, not verified beliefs, intent, or factual truth.


21. How was APSI validated?

APSI was evaluated by comparing its scores to political science expert ratings on a validation set of political texts (20 texts per dimension).


22. How many expert responses were collected?

A total of 147 expert responses were collected across the three dimensions.


23. What data was the base model trained on?

The base multilingual NLI model uses millions of hypothesis–premise training pairs (2.73M) spanning 26 languages.


24. What is the PolNLI dataset?

PolNLI is a political natural language inference dataset (200k hypothesis–premise pairs) used to fine-tune the Political DEBATE model.


25. What tasks does the fine-tuned model support?

The political fine-tuning supports multiple political NLP tasks such as stance detection, topic classification, and related text classification tasks.


26. What are the main design principles of APSI?

APSI is theory-driven, transparent, interpretable, reproducible, and scalable—designed to enable auditing of how scores are produced.


27. Can APSI make mistakes?

Yes. Outputs are approximate and can be inaccurate or incomplete due to complex political language, model limitations, or bias.


28. Should APSI be used to make decisions about individuals?

No. APSI should not be used for legal, employment, or automated decision-making affecting people. Human interpretation is required.


29. Who is APSI designed for?

APSI is intended for research, journalism, think tanks, civil society organizations, and critical analysis of political discourse.


30. What are the main limitations of APSI?

APSI depends on hypothesis design, can reflect training-data bias, measures rhetoric rather than intent, and lacks real-world context.

Analyzing text…