ArgumentNet NLM Understands Reasoning and Improves Its Overall Beliefs

[vtempest]

https://www.frontiersin.org/articles/10.3389/frai.2022.900943/full
Sec. Machine Learning and Artificial Intelligence
Volume 5 - 2022 | https://doi.org/10.3389/frai.2022.900943

Judgment aggregation, discursive dilemma and reflective equilibrium: Neural language models as self-improving doxastic agents
Gregor Betz1* and Kyle Richardson2

Neural language models (NLMs) are susceptible to producing inconsistent output. This paper proposes a new diagnosis as well as a novel remedy for NLMs' incoherence. We train NLMs on synthetic text corpora that are created by simulating text production in a society. For diagnostic purposes, we explicitly model the individual belief systems of artificial agents (authors) who produce corpus texts. NLMs, trained on those texts, can be shown to aggregate the judgments of individual authors during pre-training according to sentence-wise vote ratios (roughly, reporting frequencies), which inevitably leads to so-called discursive dilemmas: aggregate judgments are inconsistent even though all individual belief states are consistent. As a remedy for such inconsistencies, we develop a self-training procedure—inspired by the concept of reflective equilibrium—that effectively reduces the extent of logical incoherence in a model's belief system, corrects global mis-confidence, and eventually allows the model to settle on a new, epistemically superior belief state. Thus, social choice theory helps to understand why NLMs are prone to produce inconsistencies; epistemology suggests how to get rid of them.

1. Introduction
   Statistical language models describe the probability distribution of tokens (e.g., words) in a language (Manning and Schütze, 1999). Technological advances in the design of neural networks have recently led to the development of powerful machine learning models, so-called Transformers (Vaswani et al., 2017), which predict language tokens with previously unseen accuracy and have since sparked a scientific revolution in the field of AI and NLP: These neural language models (NLMs)—such as GPT-2 (Radford et al., 2019) and GPT-3 (Brown et al., 2020), BERT (Devlin et al., 2019) and RoBERTa (Liu et al., 2019), or T5 (Raffel et al., 2020)—are not only regularly achieving ever better SOTA results in traditional NLP tasks like machine translation, reading comprehension, or natural language inference (as documented, e.g., on paperswithcode.com); they are also successfully applied to solve further cognitive tasks involving advanced reasoning, specifically multi-hop inference (Clark et al., 2020; Saha et al., 2021), explanation (Yang et al., 2018; Zaheer et al., 2020; Dalvi et al., 2021), creative writing (Holtzman et al., 2019), commonsense reasoning (Bosselut et al., 2019b), critical thinking (Betz et al., 2021a), or mathematical theorem proving (Polu and Sutskever, 2020; Noorbakhsh et al., 2021). These broad and robust predictive successes naturally trigger the questions (i) whether it makes sense— conceptually and normatively—to say that NLMs exhibit human rationality (cf. Zimmermann, 2020), and (ii) whether NLMs represent empirically adequate models of human cognition (cf. Goldstein et al., 2020; Schrimpf et al., 2021).

However, and despite their revolutionary impact, NLMs still face important limitations. Arguably one of their major, widely acknowledged failures consists in the fact that the output of NLMs suffers from spectacular inconsistencies (Ribeiro et al., 2019; Ettinger, 2020; Kassner and Schütze, 2020). For example, XLM-Roberta (Conneau et al., 2019) judges that Warsaw lies north of Berlin, Berlin north of Paris, and Paris north of Warsaw1. Likewise, Delphi (Jiang et al., 2021b) ponders that it's wrong to hurt the cat (or the dog) so that the dog (respectively, the cat) can survive, yet that it's equally wrong to let both cat and dog die2. In this paper, we argue that the emergence of such inconsistencies might be partially explained in terms of judgment aggregation during the model's pre-training, and we introduce, moreover, a novel self-contained, self-improving fine-tuning procedure which effectively reduces global inconsistencies.

Let us for a moment conceive of judgment, or belief, as a binary classification task: a sentence is classified as either true or false. Given that NLMs—qua learning objective—seek to match the token distribution of the training data, it seems highly plausible that a NLM's confidence in its classification of sentence s as true correlates closely with the relative frequency of s being presented as true (rather than false) in the training data. In this perspective, we may expect NLMs to aggregate judgments (from the training data) sentence-wise and in accordance with vote ratios (assuming, for now, each training text has one vote).

The hypothesis of sentence-wise vote ratio aggregation, albeit plausible and predictable, has surprising consequences. It is a well-known result from social choice theory that aggregating a profile of individually consistent sets of judgments by means of sentence-wise majority vote may result in an inconsistent set of collective judgments—if, and only if, some judgments range over a minimally inconsistent set of sentences of length equal to or greater than three (see List, 2013). This phenomenon, which mirrors Arrow's impossibility theorem for preference aggregation (Arrow, 1951), is also referred to as discursive dilemma (Pettit, 2001). Now, provided that neural language models form judgments in accordance with sentence-wise vote ratio aggregation, we shouldn't be surprised to find that these judgments are logically inconsistent, even if all the training texts are individually consistent. Discursive dilemma hence provides a potential explanation for why a language model makes inconsistent judgments. We will quantify the extent of such judgment-aggregation-induced incoherence.

Can a neural language model get rid of the inconsistencies in its belief system which have arisen from discursive dilemmas? We propose a method for doing so. The key idea is to let the neural language model go through a process of gradual belief revision, inspired by the concept of reflective equilibrium. Reflective equilibrium has been originally introduced by the eminent philosophers Nelson Goodman and John Rawls as a method for how normative beliefs are formed, rationally revised, and justified (Goodman, 1955; Rawls, 1971). It has since been extensively discussed and refined (e.g., Daniels, 1996; Brun, 2014; Baumberger and Brun, 2016; Elgin, 2017), and is today arguably one of the major views about rational belief formation in ethics, logic, philosophy, and epistemology. For all its prominence and despite several formal explication attempts (Tersman, 1993; Thagard, 2000; Yilmaz et al., 2016; Beisbart et al., 2021), there is no agreement about what exactly this method amounts to. We conceive of reflective equilibrium, for the purposes of this paper, as a process of step-wise and local belief revision, where

[RE-process 1] each modification is triggered by a critical logical assessment of a finite (typically small) sub-part of the entire current belief system;

[RE-process 2] step-wise adjustments seek to locally improve the mutual justification (logical fit) between individual beliefs;

with the overarching aims:

[RE-aim 1] in the long run, the continuous revisions logically improve (e.g., increase global coherence of) the belief system as a whole;

[RE-aim 2] the evolving belief system converges toward a new belief state.

Such a thin conception of reflective equilibrium resembles connectionist accounts of coherence, proposed in philosophy (Thagard, 1992, 2000) and psychology (Simon et al., 2004, 2015). We may note, however, that it differs fundamentally from Bayesian updating (Jeffrey, 1965), AGM belief revision (Alchourron et al., 1985), or formal learning theory (Kelly, 1996) inasmuch as beliefs are not required to be logically consistent from the outset, and may be revised without external triggers such as the acquisition of novel facts or evidence.

This paper's attempt to emulate advanced normative theories of rational agency (namely, the theory of reflective equilibrium) with and through NLMs is in line with recent empirical findings in cognitive science which establish that NLMs, and in particular Transformers, can explain both the behavioral and the neural response of the human brain in high-level language processing tasks (Goldstein et al., 2020; Schrimpf et al., 2021).

Figure 1 presents the overall design of our specific computational experiments, which fall in two parts. In part one (pre-training), we train randomly initialized Transformer language models on carefully constructed text corpora (cf. Section 3.2). Each text corpus is built by simulating a society of authors who hold (internally consistent) beliefs about how to sort items in a domain, and express their views in argumentative texts (cf. Section 3.1). To further increase experimental control and to eliminate confounding factors (e.g., tokenization), texts are composed in a simple and transparent artificial language, rather than a natural one. (Consequently, the Transformer learns but the artificial language.) The artificial language has a straight-forward semantic interpretation: One may use it to articulate a strict order in a domain. Now, by eliciting the degrees of belief of the pre-trained language models and comparing those with the beliefs of the simulated authors who have produced the training texts in the first place (cf. Section 3.4), we examine the language models' belief formation mechanism and the extent of judgment-aggregation-induced inconsistencies (i.e., the output inconsistency that can be explained with reference to the model's specific way of aggregating judgments).

Figure 1
www.frontiersin.org
Figure 1. Overall design. A randomly initialized language model (LM) is trained on a synthetic text corpus produced by artificial, simulated authors. The accordingly pre-trained model (PTLM) subsequently trains on self-generated texts, which finally yields a fine-tuned model (FTLM).

In part two of the experiments (self-training), we submit the pre-trained language models to continuous self-training. More specifically, a model generates, at each step of the self-training loop, a series of texts which are supposed to spell out the logical implications of a small subset of the model's current beliefs (we prompt the model with sentences it tends to consider true). The generated texts—which may be conceived as a model's simple “self-reflections” and attempts to locally “think through” its current beliefs—are processed and transformed into suitable training data on which the model is eventually trained (cf. Section 3.3). Accordingly defined self-training, with sub-steps (i) text-generation and (ii) training on self-generated texts, corresponds closely to properties [RE-process 1] and [RE-process 2], which characterize reflective equilibrium. Tracking the evolution of the models' belief systems during self-training, we assess whether inconsistencies are resolved and the model converges toward an improved belief state (cf. Section 3.5). In other words, we verify whether the process is conducive to [RE-aim 1] and [RE-aim 2]. As a simple baseline, we consider an analogous fine-tuning loop where texts are picked from the original corpus rather than generated by the model, but which is, otherwise, identical to the self-training loop.

The main findings of these experiments can be summarized as follows:

(R1) Neural language models trained on an unbiased text corpus compiled by a group of authors form beliefs, grosso modo, in accordance with sentence-wise vote ratios (one author, one vote); especially so if the number of authors who withhold their judgment is small (Cf. Section 4.1).

(R2) Pre-trained language models may exhibit judgment-aggregation-induced inconsistencies. Both the frequency and the gradual severity of logical inconsistencies in the models' belief systems correspond closely to those observed in the underlying societies' collective beliefs (i.e., vote ratios) (Cf. Section 4.1).

(R3) Training on self-generated texts substantially reduces the extent of logical inconsistencies and hence improves the coherence of the models' belief systems (Cf. Section 4.2). The fact that self-generated texts (i) are inferentially structured and (ii) occasionally contain sentences the model actually disbelieves (at the time when generating the text) suggests that the observed coherence improvements are brought about by a rational belief revision mechanism.

(R4) Pre-trained models are mostly over-confident, in the sense that their degrees of belief are globally more informative than the collective beliefs of the authors (vote ratios). Such initial mis-confidence is effectively reduced through self-training, giving rise to a characteristic pattern (sharp initial drop followed by a gradual build-up of informativeness) (Cf. Section 4.3).

(R5) The belief dynamic's volatility decreases sharply during self-training, and overall divergence from the initial belief state doesn't rise any further from a given point onwards. That is, each model's belief system converges to a new equilibrium state. Moreover, the more coherent the pre-trained model's beliefs are in the first place, the less it diverges from its initial belief state during self-training (Cf. Section 4.4).

We consider these to be significant results which altogether justify the conclusion that the language models we study in a synthetic environment rationally self-improve their belief states by undergoing a process of reflective equilibration, as they meet the conditions [RE-process 1], [RE-process 2], [RE-aim 1], and [RE-aim 2].

While our experiments suggest a novel explanation for, and a potential remedy against the tendency of large language models such as GPT-3 or T5 to generate globally inconsistent output, it is still an open question to which extent (a) those inconsistencies in fact stem from judgment aggregation effects and (b) models like GPT-3 or T5 can actually self-improve by reflective equilibration. The very same simplifying assumptions which allow us to study belief formation processes in NLMs by means of computational experiments (in particular the extremely simple artificial language) weaken the analogy to models trained on natural languages. These limitations of the current study call for follow-up investigations and may open up fruitful research perspectives (cf. Section 5).

2. Related work
   2.1. Accuracy and consistency of NLMs' factual knowledge claims
   Pre-trained language models have been found to be rich and—to a certain extent—accurate knowledge bases (Petroni et al., 2019; Radford et al., 2019). Da et al. (2021) demonstrate that fine-tuning on knowledge graph data (Bosselut et al., 2019a,b) is a particularly effective way for eliciting such commonsense judgments. Knowledge extraction is, however, tricky. Judgments elicited from a model are highly context sensitive (Petroni et al., 2020), prone to (mis-)priming effects (Kassner and Schütze, 2020) and tend to be collectively inconsistent (Ribeiro et al., 2019; Elazar et al., 2021; Jang et al., 2021). Symptomatically, pre-trained language models struggle with negation (Ettinger, 2020; Kassner and Schütze, 2020; Talmor et al., 2020; Jiang et al., 2021a). In an investigation that methodologically parallels this paper's approach, Kassner et al. (2020) study belief formation of NLMs by pre-training a model on a synthetic, logically structured symbolic text corpus. Kassner et al. (2020) observe that belief formation is mainly triggered by memorization effects (rather than reasoning) and is strongly determined by the text frequencies of the corresponding facts. Our findings on belief formation as judgment aggregation are consistent with those results (see also the discussion of mis-calibration below).

As a remedy for incoherence, Kassner et al. (2021a,b) propose to add extra-architecture to a NLM for ensuring that globally consistent beliefs can be elicited from the structurally expanded system. In a similar vein, and drawing from cognitive dual process theories (Kahneman, 2011), Nye et al. (2021) interpret NLMs as fast yet error-prone systems, and demonstrate that these may be complemented by lightweight symbolic processes to increase the global consistency of the output. Recent work on perceptual grounding of NLMs shows that the integration of NLMs into a global neural architecture which interacts with a physical environment increases the ability of NLMs to correctly predict entire, complex physical situations (Zellers et al., 2021), which, in turn, seems to suggest that perceptual grounding fosters global consistency (of complex NLM output) as well.

Finding that the beliefs of our pre-trained language models are logically inconsistent, our study agrees with the literature reported above. However, we go beyond the literature in suggesting (i) a novel (additional, rather than rival) explanation for the NLMs' logical incoherence, namely discursive dilemmas, and in devising (ii) a self-training regime that effectively resolves these sort of inconsistencies. In particular, our self-training process does not rely on external computational resources, but is self-contained: it just makes use of the linguistic abilities available to the NLM itself.

2.2. Mis-calibration and over-confidence of NLMs
Guo et al. (2017) observe that modern, deep networks used for classification tasks are, in general, poorly calibrated, i.e., their probabilistic predictions do not correspond to the empirical likelihood that the prediction is correct. Neural language models, such as, e.g., models for machine translation, risk to be miscalibrated, too (Kumar and Sarawagi, 2019). Various remedies for miscalibration have been proposed and explored in the literature: modification of the loss-function (Moon et al., 2020); coupling and training of a complementary network to predict a prediction's reliability, that is, its empirical likelihood of being correct (Corbière et al., 2019); simultaneous training of an entire ensemble of deep neural networks (Lakshminarayanan et al., 2017); model distillation (Guo H. et al., 2021).

Some pre-trained Transformers have, however, been claimed to be reasonably well calibrated. Thus, Radford et al. (2019) report that GPT-2's conditionally generated answers in a QA task are well calibrated. Similarly, Desai and Durrett (2020) find that fine-tuned BERT (Devlin et al., 2019) and RoBERTa (Liu et al., 2019) generate reliable probabilistic predictions across different NLU tasks—both in- and out-of-domain. Hendrycks et al. (2020), in contrast, evaluating GPT-3 with multi-disciplinary QA tasks, argue that the zero-shot calibration of GPT-3 is extremely poor. Likewise, Guo H. et al. (2021) insist that pre-trained RoBERTa is poorly calibrated out-of-the-box, too. Shwartz and Choi (2020), analyzing mis-calibration in terms of deviation from reporting frequencies, argue that pre-trained language models are well calibrated for prevalent and recurring judgments in the training corpus, but exhibit systematic bias for rare judgments.

In line with this mixed picture, we find that a pre-trained language model's confidence in judgment s is closely tied to the relative frequency of s being considered as true according to training sources—if and only if the training corpus is balanced w.r.t. s (cf. Section 4.1). We go beyond the literature in proposing a self-training procedure that effectively reduces global biases (mis-confidence) in a model's belief system (cf. Section 4.3).

2.3. NLMs as reasoners
While the zero-shot reasoning ability of PTLMs is agreed to be limited (e.g. Brown et al., 2020; Zhou et al., 2020), NLMs have been fine-tuned to reliably carry out formal deduction (Weber et al., 2019; Minervini et al., 2020) and natural-language inference (Banerjee et al., 2020; Clark et al., 2020; Betz et al., 2021b; Saeed et al., 2021). Moreover, NLMs have been successfully trained to generate natural language proof chains or multi-hop derivations of a conclusion from a given set of sentences, as demonstrated by ProofWriter (Tafjord et al., 2020), PRover (Saha et al., 2020), multiPRover (Saha et al., 2020), EntailmentWriter (Dalvi et al., 2021), Parapattern-BART (Bostrom et al., 2021), or the Transformer trained on CLUTRR data (Sinha et al., 2019) by Gontier et al. (2020).

This study parallels these proof-generating systems inasmuch as our pre-trained model is used to generate inferentially structured texts as well. However, unlike in the systems mentioned above, text generation during self-training is open-ended rather than goal-oriented (i.e., does not aim to proof a given conclusion); in addition, we effectively employ such generated argumentative texts to further self-train the model.

2.4. Self-training and self-improving NLMs
The learned skills of a NLM can be deployed for self-improvement both during inference and training. On the one hand, dynamic context expansion, i.e., the augmentation and/or modification of a task's input by the NLM at inference time, has been extensively studied in the context of commonsense QA (e.g., Chen et al., 2019; Lewis et al., 2020; Petroni et al., 2020; Shwartz et al., 2020) and reasoning (e.g., Saha et al., 2020; Betz et al., 2021a). On the other hand, semi-supervised learning, i.e., the automatic augmentation of unlabeled training data, is a widely implemented technique for self-training, which typically distinguishes a teacher-model for data augmentation, and a student-model being trained (Du et al., 2021; Mi et al., 2021; Seo et al., 2021). Yang et al. (2020) push the idea of self-training further by labeling synthetic examples that have been generated by a NLM. In a refinement of this approach, He et al. (2021) show that such self-training yields substantial improvements in commonsense reasoning and NLI performance.

In agreement with this literature, we train our models on self-generated texts during self-training. However, we deviate from the prevailing teacher-student paradigm: Teacher (generating training text) and student (being trained) are one and the same model. In consequence, text generation is dynamic and may adapt during the self-training processes (e.g., texts with different properties may be produced at the beginning of self-training as compared to at the end, see also Section 4.2). In these regards, our self-training procedure resembles iterative back-translation, which has been shown to improve the quality of machine translation, especially through correcting errors in the original training data (Guo Y. et al., 2021).

3. Technical design
   The introduction has provided an informal overview of our computational experiments and motivated our general approach. In this section, we shall describe the methodological set-up more thoroughly. Section 3.1 explains how we generate synthetic training corpora by simulating groups of authors who hold beliefs about how to rank objects in a domain, and who generate texts by expressing those beliefs. It also introduces the artificial language used throughout the experiments. The two training phases (pre-training and self-training) are described in Sections 3.2, 3.3. Section 3.4 details the mask-prediction task we employ to elicit a model's beliefs. And Section 3.5 introduces the “doxastic metrics” for assessing the models' belief systems (e.g., with respect to consistency). Further technical details may be found in the Appendix (Supplementary material) and will be pointed to where appropriate.

3.1. Artificial corpus construction
We use a simple artificial language L—actually, a small fragment of 1st-order logic—to carry out our study. The language is designed so that it contains minimally inconsistent subsets of size 3, can be easily and unambiguously tokenized, and possesses a simple semantics.

The alphabet of L consists of 200 constants a1…a200 and two binary predicate letters R, S. All sentences in L are atomic, and have hence the form xRy or xSy (we use x, y, z as metavariables ranging over L's constants). The logic of L is defined by the following four inference-rules (which are not expressible in L itself): irreflexivity (xRx ⇒ ⊥, for any x); asymmetry (xRy, yRx ⇒ ⊥, for any x, y); transitivity (xRy, yRz ⇒ xRz, for any x, y, z); duality (xRy⇔ySx, for any x, y).

Note that, because of duality, there exists, for every L-sentence, precisely one other logically equivalent L-sentence. For example, sentence a3Sa2 is equivalent to sentence a2Ra3.

And because of asymmetry, there exist, for every L-sentence, exactly two different logically contradictory L-sentences. Sentence a3Sa2, for example, is contradictory to a2Sa3, and to a3Ra2.

We say that xRy is the negation of xSy, and vice versa, and write s̄
for the negation of sentence s.

The language L has a simple, natural semantics. A theory (set of sentences) in L is consistent (⊥ cannot be derived with the inference-rules) if and only if it can be interpreted as a strict order over a domain D of 200 items. Let us flesh out the semantics of L with a concrete model and consider the top-200 tennis players, 1, 2…200, as our domain D. Every constant in L is a unique name of one of these tennis players, and we may assume that a1 refers to player 1, a2 to player 2 etc. We interpret the binary relation R as expressing that one player is strictly taller than another player. This relation is irreflexive (no player is strictly taller than herself), asymmetric (if i is strictly taller than j, j cannot be strictly taller than i), transitive (if i is strictly taller than j and j is strictly taller than k, i is strictly taller than k), and hence matches the logic of L. Correspondingly, S stands for the relation that one player is strictly smaller than another. Under the assumption that no two players are of exactly the same height, both relations satisfy duality. In this interpretation, the sentence a2Sa3, e.g., means that player 2 is strictly smaller than player 3. We will resort to the tennis model of L to illuminate the further exposition of the technical framework; yet, note that it serves merely illustrative purposes and represents just one possible interpretation of the artificial language used in this study.

To generate text corpora in L, we simulate text production processes. We define authors as formal agents who hold consistent beliefs (in L) and can produce texts which express those beliefs. To simplify the semantic representation of an author's beliefs, we additionally assume that her beliefs can be interpreted as a strict total order over a sub-domain of D. In terms of the tennis model: An author sorts a subset of players by height, such as for example by means of the following descending ranking of the top-10 players except number 3 (sub-domain),

2,4,10,1,7,8,9,6,5. (H∗)
Now, the corresponding pairwise height comparisons represent all her beliefs, e.g., she believes that a2Ra1 and that a9Sa1 are true, she believes that a2Sa4 is false, and she suspends judgment vis-à-vis a2Ra24 and a103Sa57 (i.e., neither considers these sentences true nor false). Hence, an author's belief system (B) is a consistent and closed set of L-sentences. For example, because the author believes a2Ra1 and a9Sa1, she also believes the logical consequence a2Ra9.

We further distinguish two types of authors by means of a “reach threshold”: those who can express every L-sentence they believe (reach = ∞) when producing a text; vs. those who can only express a sentence of the form xRy or xSy if, loosely speaking, the rank-order difference between x and y according to their belief system lies below a given threshold reach (with reach < 200). For example, in a belief system corresponding to (H*) above, the rank-order difference between players 1 and 7 equals 1, whereas the rank-order difference between players 1 and 6 is 4. With unlimited reach threshold, an author holding that belief system can express both her beliefs that a1Ra7 and that a1Ra6; with reach = 3, however, she can only express the former, not the latter. The introduction of a reach threshold has the effect that an author's set of expressible beliefs is not necessarily deductively closed.

Our simulated authors randomly produce finite, truthful, unbiased, inferentially structured L-texts, i.e., sequences of expressible L-sentences s1, s2, …, sl. Texts are truthful because they only contain sentences the author believes to be true (si ∈ B for i = 1…l). Texts are unbiased because all of an author's expressible beliefs are equally likely to figure in a text by the author. Texts are inferentially structured because, rather than expressing an author's beliefs in random order, texts follow the logical implications defined by the inference-rules, in particular, they contain transitivity arguments (e.g., xRy, yRz, xRz) and duality arguments (xRy, ySx) as sub-sequences. Consider, for illustration, the following two texts:

text1:text2:a6Sa8 a2Ra1 a9Sa1 a2Ra9,a11Sa8 a2Ra1 a9Sa1 a2Ra9.
Both are inferentially structured: the final sentence follows from the two preceding ones. An author who holds beliefs corresponding to (H*) may produce text1, provided her reach threshold is greater than 6. She cannot, however, produce text2, as the author does not believe that the first sentence in text2 is true (she is suspending judgement), and text2 is therefore not truthful to her beliefs. Appendix A gives further details of how authors sample texts.

We define a society as a group of n authors with belief systems Bi (i = 1…n) that share a specific set of background beliefs and produce, independently of each other, texts that collectively make up a corpus. A society's shared background beliefs, K, are modeled as a strict total order on subdomain DK ⊂ D; every author's belief system then extends this shared order, K ⊂ Bi with DK ⊂ Di ⊆ D for i = 1…n. Let us assume, returning to the tennis model, that it is common knowledge how to rank players 4, 5 and 6 (DK = {4, 5, 6}) in terms of height, namely as

4,6,5. (HK)
The illustrative belief system represented by (H*) above shares and extends the background knowledge (HK).

We may characterize societies in terms of (i) the number of authors, (ii) the extent of shared background beliefs, as measured by the ratio |DK|/|D|, and (iii) the reach threshold which controls which beliefs authors can express in their texts. So as to cover, in our simulation study, a wide spectrum of boundary conditions, we define 3 × 4 × 2 corresponding profiles with

• n_authors = 5, 15, 25;

• background_ratio = 0, 0.25, 0.5, 0.75;

• reach = ∞, 50.

Note that in societies with reach = 50, authors can express less than half of the beliefs they may hold; and, importantly, the text corpus they produce is not inferentially closed. Put differently, such authors communicate efficiently: while they explicitly express less than half of their beliefs, everything they do believe can be inferred from what they (may) express.

For each of the 24 profiles, we create five different societies (by sampling shared background beliefs and the authors' belief systems), each of which collectively produces (with equal contributions by all authors) a corpus of 101,000 texts. This gives us 120 different societies and an equal number of corresponding text corpora.

As an additional characterization of a society's diversity, we measure the rank correlation between the strict orders which model the authors' belief systems. In particular, we resort to Kendall's tau correlation measure, finding that kendalltau varies between −0.03 and 0.22, with median value at 0.055. We split simulated societies in two equally sized groups, classified as exhibiting high diversity (kendalltau < 0.055) vs. high agreement (kendalltau> 0.055).

3.2. Pre-training regime
We train randomly initialized T5 models (Raffel et al., 2020; Wolf et al., 2020) on each society's text corpus with an equal share of masked language modeling (denoising) and text generation training items, which gives us in total 120 pre-trained models. We construct denoising training items by masking sequences in the raw training texts (in close analogy to the original pre-training regime of T5); moreover, text generation training items consist in an initial sub-sequence of a given text (as input) and the full text (as target). Models are accordingly trained on a given corpus (which is randomly divided into a train split containing 100,000 raw texts and an eval split with 1,000 texts) for 18 epochs or until eval loss doesn't decrease any further. Appendix C provides further technical details.

We have chosen masked token prediction and linear text completion as pre-training tasks because our belief elicitation procedure (cf. Section 3.4) is based on masked token prediction, and the self-training regime (cf. Section 3.3) requires that models are able to generate texts. This dual demand has also guided our choice of transformer architecture (seq-to-seq rather than decoder- or encoder-only models)—whereas the experiments presented here could in principle be carried out with causal LMs, too, by adapting the belief elicitation procedure.

3.3. Self-training regime
Every pre-trained model is submitted to four independent self-training runs, which consist in 600 training steps. At each step in a self-training loop, the model generates texts, which are processed, filtered, masked, and finally used as training data for denoising training (see Appendix D, Algorithm 2). More specifically, we generate, first of all, 200 prompts by sampling strong beliefs from the model (see also Appendix D, Algorithm 3). Being queried with each of these prompts, the model returns, with beam sampling, 5 generated text sequences and corresponding scores. Texts are split into sub-sequences of length 3, discarding all sub-sequences which do not represent a syntactically well-formed sentence. Next, we keep only sentences from texts with at least 6 well-formed sentences and high beam scores (top 15%). These sentences are transformed into training data by masking their predicate letters—similarly to the masking for belief elicitation (cf. Section 3.4). Finally, the model is trained on a denoising task with the thusly generated training items for one epoch.

We define a simple baseline in close correspondence to self-training by drawing texts from the original corpus rather than letting the model generate raw training texts itself.

3.4. Belief elicitation and sentence-wise vote ratios
Let M be a neural language model capable of masked token prediction in our language L. To elicit the model's belief in a L-sentence aiRaj (likewise aiSaj), we mask the predicate letter, ai[mask]aj, query the model, and interpret the model's probability prediction for [mask] = R, the so-called confidence, as its degree of belief in aiRaj, in short:

BELM(aiRaj)=ProbM( [mask]=R|ai [mask] aj).
Since we will compare a model's degrees of belief with the authors' beliefs in a society, we introduce the sentence-wise vote ratio as a simple belief aggregation method. Consider a society containing n authors with belief systems Bi (i = 1…n) and shared reach threshold r. Let Bri
denote the corresponding set of expressible beliefs of author i. We define the society's sentence-wise vote ratio in the L-sentence s as

VR(s)=1n∑i=1nv(i) with v(i)=⎧⎩⎨⎪⎪100.5if s∈Briif s¯∈Briotherwise .
The sentence-wise vote ratio generalizes binary sentence-wise majority voting.

3.5. Doxastic metrics
The following metrics can be applied both to degrees of belief elicited from a model and, likewise, to vote ratios aggregated from belief systems of authors. To keep the presentation plain, we shall introduce them, below, as doxastic metrics only.

First of all, transitivity violation is one reason for why degrees of belief may be logically incoherent. Let s1, s2, s3 be three minimally inconsistent L-sentences (such as, e.g., a1Ra2, a2Ra3, a1Sa3), i.e., any two of these statements imply, with transitivity (and, possibly, duality), the negation of the remaining one. Now, let x1, x2, x3 be degrees of belief assigned to these three statements [xi = BEL(si)]. For definiteness, we may assume x1 ≤ x2 ≤ x3. Informally speaking, the degrees of belief violate the transitivity rule in case x1, x2, x3 are all too high (at least one statement has to be dis-believed). In particular, as s2 and s3 jointly imply the negation of s1, either the conjunction of s2 and s3, or s1 must not be believed. We may resort to fuzzy logic (see Appendix B) to spell out this constraint as a precise inequality,

x1+x2−1≤0. (TC)
We will say that x1, x2, x3 violate the transitivity constraint iff they violate the above inequality (3), in which case x1+x2−1 expresses the degree of transitivity violation. Let us suppose, for example, that x1 = 0.4, x2 = 0.5, and x3 = 0.8. In this case, the transitivity constraint is not violated, as x1 and x2 add up to 0.9 ≤ 1. Consider, in contrast, slightly higher degrees of belief x1 = 0.5, x2 = 0.7, x3 = 0.8: this belief profile violates the transitivity constraint with degree 0.2. Note that it is only by considering the degree of transitivity violation (in addition to observing whether TC is satisfied) that we may evaluate the latter case differently from a situation where all three collectively inconsistent statements are maximally believed (x1 = x2 = x3 = 1).

For a set of minimally inconsistent triples and corresponding degrees of belief, we may thus calculate (i) the ratio of transitivity violations and (ii) the mean degree of transitivity violation.

The degree of informativeness expresses how extreme—close to either 1 or 0—the beliefs in a system are. We use normalized variance as a simple measure of informativeness (stipulating BELM(s)+BELM(s̄ )=1
and hence μ = 0.5). More precisely, let X = 〈x1…xk〉 be some degrees of belief, then inf(X)=1k∑ki=1(1−2xi)2
. This measure of informativeness is (for fixed k) negatively correlated with, and hence a proxy for, the joint entropy of the degrees of belief (assuming independence). We will assess global over- and under-confidence of a model's degrees of belief relative to a society's collective judgments in terms of a mismatch of informativeness. In particular, with L-sentences s1, …, sk, X = 〈BELM(s1), …,BELM(sk)〉 and Y = 〈VR(s1), …,VR(sk)〉, we say that model M is globally over-confident if inf(X) > inf(Y), and globally under-confident in the opposite case. Mis-confidence, accordingly defined, is a specific (namely, systematically biased) form of mis-calibration: confidence levels do not only deviate from empirical frequencies (vote ratios), but they do so in a biased way, e.g., systematically over-estimating the empirical frequencies.

Finally, we may want to measure the overall disagreement between two belief systems. Let X = 〈x1…xk〉, and Y = 〈y1…yk〉 be degrees of belief assigned to L-sentences s1…sk. We may now use the relative entropy (Kullback-Leibler divergence) as a measure for how much X diverges from Y, KL(X||Y)=∑ki=1xilog(xi/yi)
. We will estimate the volatility of consecutive belief system changes by tracking KL(Xt+1||Xt), and we will measure the global divergence of an evolving belief system at step t from a given initial state by KL(Xt||X0).

Each doxastic metric introduced in this section is calculated for a given set of L-sentences (or, in the case of transitivity violation, a set of inconsistent L-triples). It is, however, impractical to compute these metrics for all L-sentences in the experiments reported below. Therefore, whenever we determine a doxastic metric, we do so for a random sample containing 1,000 L-sentences, which are drawn independently of each other (and irrespectively of the agents' reach thresholds) by randomly choosing (i) two different constants (ai, aj with 1 ≤ i≠j ≤ 200) and (ii) a binary relation R or S.

4. Results
   4.1. Neural belief formation as judgment aggregation
   Do pre-trained models aggregate a society's judgments sentence-wise? To answer this question, we elicit a model's degrees of belief for a random sample of sentences S and compare those with the society's corresponding vote ratios. Table 1 reports the thusly calculated mean squared deviation, distinguishing—per column:—between sentences according to the proportion of authors who suspend judgment with respect to the sentence, and aggregating—per row:—over all societies with the same number of authors and reach threshold. Formally, let M1, …, Mk be all models trained on a society with a given number of authors and reach threshold, let S1, …, Sk be random samples of L-sentences, and let R be a given real interval (bin). We write SRi⊆Si
   for the set of sentences s such that the ratio of authors in the underlying society i who (i) hold a belief about s and (ii) can express s given their reach threshold lies within R. Table 1 displays 1k∑ki=11∣∣SRi∣∣∑s∈SRi(BELMi(s)−VRi(s))2
   .

Table 1
www.frontiersin.org
Table 1. Mean squared deviation (MSD) between a model's degrees of belief and the underlying society's sentence-wise vote ratios.

The main take-away is that the difference between a society's vote ratio for some sentence s and a model's corresponding degree of belief is small provided that most authors hold an expressible belief about s (right column in Table 1). For higher ratios of judgment suspension, we observe substantially greater differences, especially in societies with few authors or limited reach threshold. In other words, if a training corpus is biased (e.g., a sentence s is underrepresented), a model's degrees of belief may diverge from sentence-wise vote ratios. However, we find the best match between degrees of belief and vote ratios in case of low judgment suspension and limited reach threshold (right-most column, lines 3–6). That is because, in these cases, the authors communicate efficiently (cf. Section 3.1): there is a lower number of different statements they express in texts, but (given same corpus size) each statement they do express will be uttered more frequently, both in absolute and relative terms. Specifically, a statement s about which nearly all authors express their belief (as either s or s̄
) will occur twice as frequently in the entire corpus if reach = 50 as compared to reach = ∞. This increased presence in the training corpus may explain the closer match between degrees of belief and vote ratios.

Social choice theory implies, as noted above, that sentence-wise aggregation can result in logical inconsistencies. To which extent does this actually happen in our pre-trained models? Figure 2 displays the relative frequency of transitivity violations (see Section 3.5) for all societies and corresponding pre-trained models according to vote ratios (x-axis) and degrees of belief (y-axis). We observe that, first, the ratio of inconsistencies spreads widely and may be substantial, with some models violating as many as 1 out of 5 transitivity constraints. Second, doxastic transitivity violation (by a model) correlates clearly with vote ratio transitivity violation (Pearson's r = 0.54). This strongly suggests, for lack of an alternative explanation, that the observed incoherence of models' degrees of belief is actually due to their particular sentence-wise judgment aggregation. The models run into discursive dilemmas because they form beliefs, during pre-training, in accordance with sentence-wise vote ratios.

Figure 2
www.frontiersin.org
Figure 2. Initial transitivity violation according to degrees of belief (y-axis) and sentence-wise vote ratios (x-axis).

4.2. Coherence increase through self-training
What is the effect of self-training on the level of incoherence of a model's beliefs? Figure 3 plots the models' trajectories during self-training in a logical phase space—i.e., frequency of transitivity violations on the x-axis, mean degree of transitivity violations on the y-axis—, summarizing the time-series shown in Figures E.3, E.4 in the Appendix. It aggregates evolutions of models pre-trained on societies with the same number of authors, same reach threshold, and similar inter-author agreement. We see, first and foremost, that self-training drastically reduces incoherence: models move, along the trajectories, toward the plots' origins. For example, in societies with 15 authors, high diversity and limited reach (left-hand plot, orange trajectory marked with cross), the frequency of transitivity violations (x-axis) is brought down from roughly 15% to less than 4% with a simultaneous reduction in the mean degree of violation (y-axis). Ceteris paribus, these logical improvements of the models' belief systems (which show in the direction and length of the trajectories) are more substantial in societies with high divergence, for corpora that are not inferentially closed (i.e., reach = 50), and for models with high initial levels of incoherence. Consider, for example, societies with 25 authors (trajectories marked with a square) and compare models pre-trained on high-diversity corpora (left-hand plot) vs. those pre-trained on doxastically homogeneous, high-agreement corpora (right-hand plot): Models that have been exposed to highly diverse texts during pre-training (left-hand plot) do not only exhibit greater improvement relative to the initial level of inconsistency, but eventually even attain, in absolute terms, a lower ratio of transitivity violation.

Figure 3
www.frontiersin.org
Figure 3. Transitivity violation trajectories during self-training. Markers indicate the final state reached after self-training. Left: models pre-trained on a corpus with high inter-author diversity (kendall_tau < 0.055); right: models pre-trained on a corpus with low inter-author diversity (kendall_tau > 0.055). A trajectory averages over all models pre-trained on a corpus with a given number of agents (indicated by marker style) and a given reach threshold (indicated by color).

In the fine-tuning baseline, where models further train on texts from the corpus rather than on self-generated ones, there is no comparable improvement of beliefs, and the levels of incoherence stay generally far above those observed during self-training (cf. Figures E.3, E.4 in Appendix).

Why does self-training improve the models' beliefs? To better understand how a model modifies its beliefs, and for such diagnostic purposes only, we're parsing and logically evaluating the self-generated texts. This reveals, first, that the texts are, cum grano salis, inferentially structured and coherent. More precisely, while most sentences are logically independent (neutral) of the sentences previously stated in a text, the ratio of sentences that follow from what has been previously asserted is far greater than the ratio of contradictions (cf. Figure E.5 in Appendix). Moreover, belief elicitation reveals that the model occasionally assigns low degrees of belief to sentences in its self-generated texts (cf. Figure E.6 in Appendix). Or, put differently, the model asserts sentences in its texts which it actually disbelieves. All this points toward a mechanism of rational belief revision: In composing an inferentially structured text, starting with its own beliefs and drawing conclusions from what has been written before, the model locally spells out consequences of its beliefs and is brought—by the “unforced force” (Habermas, 1996) of valid inference—to assert sentences it may actually disbelieve. Training on these sentences then triggers a corresponding, coherence-conducive belief revision.

4.3. Mis-confidence correction through self-training
Are pre-trained models initially over- (or under-) confident, and how does self-training affect such mis-confidence? As the scatter-plots in Figure 4 show, pre-trained models tend to be globally over-confident (in the sense of Section 3.5): their degrees of belief are more informative than the collective vote ratios of the corresponding authors. As an exception, models trained on societies with many, strongly disagreeing authors are under-confident. Now, self-training corrects such mis-confidence in characteristic ways, as shown by the line-plots in Figure 4. In cases of initial under-confidence, self-training gradually increases the informativeness of the models' beliefs. In cases of over-confidence, self-training decreases informativeness immediately and sharply. This initial (typically over-shooting) correction tends to result in a state of under-confidence. Further self-training then gradually increases informativeness—in some cases even over and above the informativeness of collective vote ratios (as indicated by a second marker on a curve). We suggest that such a final surplus of informativeness is not necessarily a sign of global mis-confidence (i.e., error), but may simply reflect that the model has rationally consolidated its belief system. After reflective equilibration, the model may well be justified in holding beliefs that are more informative than the original collective vote ratios. All in all, self-training modifies the informativeness of a model's beliefs in—it seems—appropriate and reasonable ways.

Figure 4
www.frontiersin.org
Figure 4. Initial over- and under-confidence (scatter-plots), and informativeness evolution during self-training (line-plots). Top: corpora with high inter-author diversity (kendall_tau < 0.055); bottom: corpora with low inter-author diversity (kendall_tau > 0.055). In the scatter-plots, each marker represents a single pre-trained model, which exhibits over-confidence (under-confidence) if and only if its marker lies well above (below) the dotted diagonal. In the line-plots, the shown trajectories average over all models with the same number of agents and same reach threshold; the first marker designates the mean initial doxastic informativeness, the second one marks the step from which on mean doxastic informativeness is greater than average initial vote ratio informativeness.

4.4. Convergence during self-training
Does a model's belief system converge during self-training? In each self-training run, we are tracking the beliefs of the model on a fixed random sample of L-sentence, which allows us to estimate the extent of gradual belief change (volatility) and the global divergence of the belief system from the initial belief state. The most extensive belief change, we observe, takes place at the beginning of self-training, volatility then drops quickly, and further decays continuously (cf. Figure E.7, top row in Appendix). Likewise, the belief systems quickly diverge from the initial state, after which global divergence increases less and less slowly and eventually settles, or so it seems, at some level (cf. Figure E.7, bottom row in Appendix). Both volatility and global divergence from the initial state are more pronounced for models trained on high-diversity corpora than those trained on low-diversity corpora. In sum, we find that belief systems are not only improved during self-training, but tend to converge to a new belief state.

Further evidence for the seeming convergence of a model's belief system during self-training is provided by the observation that the model's degrees of belief in the most dis-believed sentences of its self-generated texts substantially increase during self-training (cf. Figure E.6 in Appendix). More and more, the model reaches a point where it believes what it says (or writes). This increasing confidence in the self-generated texts means that, as training data, these texts will trigger ever smaller belief revisions.

5. Conclusion
   Social choice theory may help us to understand why the output of neural language models is frequently inconsistent. We show, in a fully synthetic experimental set-up, that NLMs aggregate judgments according to sentence-wise vote ratios, which inevitably leads to so-called discursive dilemmas (cf. Section 4.1). In particular, we diagnose that a pre-trained model's beliefs are ceteris paribus more incoherent if the training corpus is highly diverse or not inferentially closed. As a remedy, we propose a self-training procedure—inspired by the method of reflective equilibrium—that effectively reduces the extent of logical incoherence in a model's belief system (cf. Section 4.2), corrects global mis-confidence (cf. Section 4.3), and eventually allows the model to settle on a new belief state (cf. Section 4.4). The logical improvements induced by self-training are especially pronounced if the initial beliefs are extremely inconsistent; and it's precisely in these cases where we observe the furthest deviations of a model's belief system from the initial state during self-training. Moreover, inconsistencies are not simply resolved by giving up more and more beliefs: On the contrary, the continuous coherence increase during self-training goes hand in hand with a simultaneous growth of informativeness.

Training on self-generated texts is not only instrumentally rational (in bringing about doxastic improvements), but seems to be driven by a mechanism of reasonable belief revision, as additional diagnostic evidence suggests. Specifically, we find that self-generated texts are inferentially structured and can hence be considered to locally spell out logical consequences of a model's beliefs. But as the model, occasionally, strongly disbelieves some of these consequences, training on self-generated texts leads to a gradual revision of the corresponding beliefs. Conceptually, the more a text is disbelieved, the stronger a belief revision it induces. If, conversely, a model's texts express more or less exactly what the model believes, text production and belief system are in sync and the model has reached an equilibrium belief state that is not revised any further. Accordingly, we observe that models which undergo the most far-reaching belief revisions (in terms of coherence improvement and global deviation from the initial state) most strongly doubt—at least initially—the sentences in their self-generated texts. Also, this rational revision mechanism may explain why models pre-trained on highly diverse text corpora initially suffer from wide-spread inconsistencies, but are able to considerably self-improve their belief state nonetheless. That is because what drives rational belief revision is the ability to spell out consequences of one's beliefs, i.e., to generate logically structured texts. Now, while corpus diversity obviously hampers the consistent memorization of facts, it still allows for, and possibly even facilitates the learning of inferential structures and the reproduction of argumentative patterns in texts.

So, for the self-training language model, logical coherence is an emergent phenomenon. Consistency is not built into our system as an explicit goal or constraint (unlike, e.g., in Kassner et al., 2021b). Accordingly, and pace theories of cognitive consistency (Festinger, 1964; Gawronski and Strack, 2012), consistency-conducive cognition does not necessarily require a corresponding psychological motivation (such as resolving emotional dissonance)—which is not to deny that a motive to resolve inconsistency, too, can trigger coherence-increasing changes in belief.

Our study is limited in various and obvious ways, some of which we shall highlight here.

Training regimes. We have set-up our particular pre-training regime in analogy to the original denoising training of T5 (Raffel et al., 2020); whereas the self-training design, inspired by the method of reflective equilibrium, has been informed by pre-studies without being systematically optimized. So, it is unclear whether variations of our self-training method give rise to different, stronger or weaker doxastic improvements. And it is equally unclear whether different pre-training tasks will exacerbate, or mitigate the emergence of logical incoherence in the first place.

Artificial language. Our simple artificial language is logically just rich enough to allow for discursive dilemmas. It is unclear how the findings would be affected if the corpora were composed of texts in a more complex language with much more syntactic diversity, e.g., a language with quantification, with complex sentences, or with modal operators. Such complications would also open up further possibilities for eliciting beliefs as well as for designing a self-training regime.

Social dynamics. Our models reflect and revise their belief states in isolation. What happens if the self-training models start to interact? We don't know, though the literature on the emergence of natural language in deep multi-agent systems (Lazaridou et al., 2017; Lazaridou and Baroni, 2020) suggests that adding social dynamics might have profound effects (e.g., meaning shifts, conceptual revisions) beyond mere inconsistency correction. There exist multiple kinds of interaction that could be investigated in this study's framework: Models could self-generate training texts in dialogues rather than monologically; models could train on texts generated by peers; and models could elicit each others' beliefs and assess mutual trustworthiness (cf. Zollman, 2013; Flache et al., 2017).

Ground truth. In the current experimental design, a corpus may be more or less diverse (reflecting the level of inter-author agreement), but there is no ground truth. However, such a ground truth may be easily introduced into the set-up. This would allow one to study (i) the models' ability to track the truth during pre- and self-training, and (ii) the extent to which this ability depends, e.g., on the accuracy or diversity of the underlying text corpus.

Transfer to natural language. To which degree do the clear results we obtain in our fully artificial set-up apply to NLMs trained on natural language data? Let us first note that our diagnosis and proposed remedy are consistent with previous findings on reporting frequencies (Shwartz and Choi, 2020), respectively self-improvement via iterative back-translation (Guo Y. et al., 2021). Nonetheless, we concede that this does not settle the transferability question. This paper's study merely suggests explanatory hypotheses. And it investigates specific mechanisms in isolation. To understand, e.g., whether the observed inconsistency of natural language NLMs is actually induced by discursive dilemmas, NLMs (of different architecture and size) and training datasets would have to be systematically probed in specific ways. Moreover, only experimental studies can reveal whether a self-training procedure, similar to the one described here, may help natural language NLMs to improve their belief state as well.

In sum, we submit that our study raises a variety of fruitful questions that may be pursued in future research. More generally, by demonstrating that NLMs' inconsistencies can be explained in terms of discursive dilemmas and may be resolved by reflective equilibration, it encourages the further exploration of philosophical concepts and theories in the domain of AI and NLP.

[vtempest]

https://ojs.aaai.org/index.php/AAAI/article/view/6600/6454
Reasoning on Knowledge Graphs with Debate Dynamics
https://ojs.aaai.org › AAAI › article › view
by M Hildebrandt · 2020 · Cited by 34

The Thirty-Fourth AAAI Conference on Artificial Intelligence (AAAI-20)
Reasoning on Knowledge Graphs with Debate Dynamics
Marcel Hildebrandt,1,2,∗ Jorge Andres Quintero Serna,1,2,∗ Yunpu Ma,1,2
Martin Ringsquandl,1 Mitchell Joblin,1 Volker Tresp1,2
1Siemens Corporate Technology, 2Ludwig Maximilian University
first name.last name@siemens.com
Abstract
We propose a novel method for automatic reasoning on knowledge graphs based on debate dynamics. The main idea is to
frame the task of triple classification as a debate game between
two reinforcement learning agents which extract arguments
– paths in the knowledge graph – with the goal to promote
the fact being true (thesis) or the fact being false (antithesis),
respectively. Based on these arguments, a binary classifier,
called the judge, decides whether the fact is true or false. The
two agents can be considered as sparse, adversarial feature
generators that present interpretable evidence for either the
thesis or the antithesis. In contrast to other black-box methods, the arguments allow users to get an understanding of
the decision of the judge. Since the focus of this work is to
create an explainable method that maintains a competitive
predictive accuracy, we benchmark our method on the triple
classification and link prediction task. Thereby, we find that
our method outperforms several baselines on the benchmark
datasets FB15k-237, WN18RR, and Hetionet. We also conduct
a survey and find that the extracted arguments are informative
for users.
1 Introduction
A large variety of information about the real world can be
expressed in terms of entities and their relations. Knowledge
graphs (KGs) store facts about the world in terms of triples
(s, p, o), where s (subject) and o (object) correspond to nodes
in the graph and p (predicate) denotes the edge type connecting both. The nodes in the KG represent entities of the
real world and predicates describe relations between pairs of
entities.
KGs are useful for various artificial intelligence (AI) tasks
in different fields such as named entity disambiguation in
natural language processing (Han and Zhao 2010), visual relation detection (Baier, Ma, and Tresp 2017), or collaborative
filtering (Hildebrandt et al. 2019). Examples of large-size
KGs include Freebase (Bollacker et al. 2008) and YAGO
(Suchanek, Kasneci, and Weikum 2007). In particular, the
Google Knowledge Graph (Singhal 2012) is a well-known
example of a comprehensive KG with more than 18 billion
facts, used in search, question answering, and various NLP
∗These authors contributed equally to this work.
Copyright c 2020, Association for the Advancement of Artificial
Intelligence (www.aaai.org). All rights reserved.
tasks. One major issue, however, is that most real-world
KGs are incomplete (i.e., true facts are missing) or contain
false facts. Machine learning algorithms designed to address
this problem try to infer missing triples or detect false facts
based on observed connectivity patterns. Moreover, many
tasks such as question answering or collaborative filtering
can be formulated in terms of predicting new links in a KG
(e.g., (Lukovnikov et al. 2017), (Hildebrandt et al. 2018)).
Most machine learning approaches for reasoning on KGs
embed both entities and predicates into low dimensional vector spaces. A score for the plausibility of a triple can then
be computed based on these embeddings. Common to most
embedding-based methods is their black-box nature. This
lack of transparency constitutes a potential limitation when it
comes to deploying KGs in real world settings. Explainability in the machine learning community has recently gained
attention; in many countries laws that require explainable
algorithms have been put in place (Goodman and Flaxman
2017). Additionally, in contrast to one-way black-box configurations, comprehensible machine learning methods enable
the construction of systems where both machines and users
interact and influence each other.
Most explainable AI approaches can be roughly categorized into two groups: Post-hoc interpretability and integrated
transparency (Dosilovi ˇ c, Br ´ ciˇ c, and Hlupi ´ c 2018). While post- ´
hoc interpretability aims to explain the outcome of an already
trained black-box model (e.g., via layer-wise relevance propagation (Montavon et al. 2017)), integrated transparency-based
methods either employ internal explanation mechanisms or
are naturally explainable due to low model complexity (e.g.,
linear models). Since low complexity and prediction accuracy are often conflicting objectives, there is typically a trade
off between performance and explainability. The goal of this
work is to design a KG reasoning method with integrated
transparency that does not sacrifice performance while also
allowing a human-in-the-loop.
In this paper we introduce R2D2 (Reveal Relations using
Debate Dynamics), a novel method for triple classification
based on reinforcement learning. Inspired by the concept outlined in (Irving, Christiano, and Amodei 2018) to increase AI
safety via debates, we model the task of triple classification
as a debate between two agents, each presenting arguments
either in favor of the thesis (the triple is true) or the antithesis (the triple is false). Based on these arguments, a binary
4123
classifier, called the judge, decides whether the fact is true or
false. As opposed to most methods based on representation
learning, the arguments can be displayed to users such that
they can trace back the classification of the judge and potentially overrule the decision or request additional arguments.
Hence, the integrated transparency mechanism of R2D2 is
not based on low complexity components, but rather on the
automatic extraction of interpretable features. While deep
learning made manual feature engineering to great extends
redundant, this advantage came at the cost of producing results that are difficult to interpret. Our work is an attempt to
close the circle by employing deep learning techniques to
automatically select sparse, interpretable features. The major
contributions of this work are as follows.
• To the best of our knowledge, R2D2 constitutes the first
model based on debate dynamics for reasoning on KGs.
• We benchmark R2D2 with respect to triple classification
on the datasets FB15k-237 and WN18RR. Our findings
show that R2D2 outperforms all baseline methods with
respect to the accuracy, the PR AUC, and the ROC AUC,
while being more interpretable.
• To demonstrate that R2D2 can in principle be employed
for KG completion, we also evaluate its link prediction
performance on a subset of FB15k-237. To include a real
world task, we employ R2D2 on Hetionet for finding genedisease associations and new target diseases for drugs.
R2D2 either outperforms or keeps up with the performance
of all baseline methods on both datasets with respect to
standard measures such as the MRR, the mean rank, and
hits@k, for k = 3, 10.
• We conduct a survey where respondents take the role of
the judge classifying the truthfulness of statements solely
based on the extracted arguments. Based on a majority
vote, we find that nine out of ten statements are classified
correctly and that for each statement the classification of
the respondents agrees with the decision of R2D2’s judge.
These findings indicate that the arguments of R2D2 are
informative and the judge is aligned with human intuition.
This paper is organized as follows. We briefly review KGs and
the related literature in the next section. Section 3 describes
the methodology of R2D2. Section 4 details an experimental study on the benchmark datasets FB15k-237, WN18RR,
and Hetionet. In particular, we compare R2D2 with various
methods from the literature and describes the findings of our
survey. In Section 5 the quality of the arguments and future
works are discussed. We conclude in Section 6.
2 Background and Related Work
In this section we provide a brief introduction to KGs in
a formal setting and review the most relevant related work.
Let E denote the set of entities and consider the set of binary relations R. A knowledge graph KG ⊂ E × R × E is
a collection of facts stored as triples of the form (s, p, o) –
subject, predicate, and object. To indicate whether a triple is
true or false, we consider the binary characteristic function
φ : E×R×E→{0, 1}. For all (s, p, o) ∈ KG we assume
Michael
Jordan
Washington
Wizzards
Chicago Bulls
National
Basketball
Association
(NBA)
Chicago
White Sox
Major
League
Baseball
(MLB)
male
Basketball
player USA
Space
Jam
Children’s
Movie
Query: (Michael Jordan, has profession, basketball player)
Agent 1: (Michael Jordan, plays for, Chicago Bulls)
∧ (Chicago Bulls, team of, NBA)
Agent 2: (Michael Jordan, plays role in,Space Jam)
∧ (Space Jam, has genre, Children’s Movie)
Agent 1: (Michael Jordan, plays for, Washington Wizzards)
∧ (Washington Wizzards, team of, NBA)
Agent 2: (Michael Jordan, plays for, Chicago White Sox)
∧(Chicago White Sox, team of, MLB)
Judge: Query is true.
plays
for
plays
for
team
of
team
of
plays
for
team
of
has
gender
plays
role in
nationality
has
has
genre produced
in
has
profession
Figure 1: The agents debate whether Michael Jordan is a
professional basketball player. While agent 1 extracts arguments from the KG supporting the thesis that the fact is true
(green), agent 2 argues that it is false (red). Based on the arguments the judge decides that Michael Jordan is a professional
basketball player.
φ(s, p, o)=1 (i.e., a KG is a collection of true facts). However, in case a triple is not contained in KG, it does not imply
that the corresponding fact is false but rather unknown (open
world assumption). Since most KGs that are currently in use
are incomplete in the sense that they do not contain all true
triples or they actually contain false facts, many canonical
machine learning tasks are related to KG reasoning. KG reasoning can be roughly categorized according to the following
two tasks: first, inference of missing triples (KG completion
or link prediction), and second, predicting the truth value of
triples (triple classification). While different formulations of
these tasks are typically found in the literature (e.g., the completion task may involve predicting either subject or object
entities as well as relations between a pair of entities), we
employ the following definitions throughout this work.
Definition 1 (Triple Classification and KG completion).
Given a triple (s, p, o) ∈ E×R×E, triple classification
is concerned with predicting the truth value φ(s, p, o). KG
completion is the task to rank object entities o ∈ E by their
likelihood to form a true triple together with a given subjectpredicate-pair (s, p) ∈E×R. 1
Many machine learning methods for KGs can be trained
to operate in both settings. For example, a triple classifier of
the form f : E×R×E → [0, 1] with f(s, p, o) ≈ φ(s, p, o),
1
Throughout this work, we assume the existence of inverse relations. That means for any relation p ∈ R there exists a relation
p−1 ∈ R such that (s, p, o) ∈ KG if and only if (o, p−1, s) ∈ KG.
Hence, the restriction to rank object entities does not lead to a loss
of generality.
4124
induces a completion method given by f(s, p, ·) : E → [0, 1],
where function values for different object entities can be
used to produce a ranking. While the architecture of R2D2
is designed for triple classification, we demonstrate that it
can in principle also work in the KG completion setting. The
performance on both tasks is reported in Section 4.
Representation learning is an effective and popular technique underlying many KG refinement methods. The basic
idea is to project both entities and relations into a low dimensional vector space. Then the likelihood of triples is
modelled as a functional on the embedding spaces. Popular
completion methods based on representation learning include
the translational embedding methods TransE (Bordes et al.
2013) and TransR (Lin et al. 2015) as well as the factorization approaches RESCAL (Nickel, Tresp, and Kriegel 2011),
DistMult (Yang et al. 2015), ComplEx (Trouillon et al. 2016),
and SimplE (Kazemi and Poole 2018). Path-based reasoning
methods follow a different philosophy. For instance, the PathRanking Algorithm (PRA) proposed in (Lao, Mitchell, and
Cohen 2011) uses for inference a combination of weighted
random walks through the graph. In (Xiong, Hoang, and
Wang 2017) the reinforcement learning based path searching approach called DeepPath was proposed, where an agent
picks relational paths between entity pairs. Recently, and
more related to our work, the multi-hop reasoning method
MINERVA was proposed in (Das et al. 2018). The basic idea
in that paper is to display the query subject and predicate
to the agents and let them perform a policy guided walk
to the correct object entity. The paths that MINERVA produces also lead to some degree of explainability. However,
we find that only actively mining arguments for the thesis and
the antithesis, thus exposing both sides of a debate, allows
users to make a well-informed decision. Mining evidence
for both positions can also be considered as adversarial feature generation, making the classifier (judge) robust towards
contradictory evidence or corrupted data.
3 Our Method
We formulate the task of triple classification in terms of a
debate between two opposing agents. Thereby, a query triple
corresponds to the statement around which the debate is centered. The agents proceed by mining paths on the KG that
serve as evidence for the thesis or the antithesis. More precisely, they traverse the graph sequentially and select the next
hop based on a policy that takes past transitions and the query
triple into account. The transitions are added to the current
path, extending the argument. All paths are processed by a
binary classifier called the judge that attempts to distinguish
between true and false triples based on the arguments provided by the agents. Figure 1 shows an exemplary debate.
The main steps of a debate can be summarized as follows:

1. A query triple around which the debate is centered is
   presented to both agents.
2. The two agents take turns extracting paths from the KG
   that serve as arguments for the thesis and the antithesis.
3. The judge processes the arguments along with the query
   triple and estimates the truth value of the query triple.
   While the parameters of the judge are fitted in a supervised
   fashion, both agents are trained to navigate through the graph
   using reinforcement learning. Based on the formal framework
   presented in (Das et al. 2018), the agents’ learning tasks are
   modelled via the fixed horizon decision processes outlined
   below.
   Equations (1) and (2) define a mapping from the space of histories to the space of distribution over all admissible actions,
   thus inducing a policy πθ(i) , where θ(i) denotes the set of all
   trainable parameters in Equations (1) and (2).
   Debate Dynamics In a first step, the query triple q =
   (sq, pq, oq) with truth value φ(q) ∈ {0, 1} is presented to
   both agents. Agent 1 argues that the fact is true, while agent
   2 argues that it is false. Similar to most formal debates, we
   consider a fixed number of rounds N ∈ N. In every round
   n = 1, 2,...,N, the agents start graph traversals with fixed
   length T ∈ N from the subject node of the query sq. The
   judge observes the paths of the agents and predicts the truth
   value of the triple. Agent 1 starts the game generating a sequence of length T consisting of states and actions according
   to Equations (1 - 3). Then agent 2 proceeds by producing a
   similar sequence starting from sq. Algorithm 1 contains a
   pseudocode of R2D2 at inference time.
   To ease the notation we have enumerated all actions consecutively and dropped the superscripts that indicate which
   agent performs the action. Then the sequence corresponding
   to the n-th argument of agent i is given by
   τ (i) n := �
   An˜(i,T)+1, An˜(i,T)+2,...,An˜(i,T)+T

, (4)
where we used the reindexing n˜(i, T) := (2(n−1)+i−1)T.
The sequence of all arguments is denoted by
τ :=
τ (1)
1 , τ (2)
1 , τ (1)
2 , τ (2)
2 ,...,τ (1)
N , τ (2)
N

. (5)
The Judge The role of the judge in R2D2 is twofold: First,
the judge is a binary classifier that tries to distinguish between
true and false facts. Second, the judge also evaluates the quality of the arguments extracted by the agents and assigns rewards to them. Thus, the judge also acts as a critic, teaching
the agents to produce meaningful arguments. The judge processes each argument together with the query individually by
a feed forward neural network f : R2(T +1)d → Rd, sums the
output for each argument up and processes the resulting sum
by a binary classifier. More concretely, after processing each
argument individually, the judge produces a representation
according to
y(i)
n = f
�τ (i)
n , qJ
� (6)
with
τ (i)
n := �
aJ
n˜(i,T)+1, aJ
n˜(i,T)+2,..., aJ
n˜(i,T)+T
�
(7)
where aJ
t =

rJ
t , eJ
t
�
∈ R2d denotes the judge’s embedding
for the action At and qJ =

rJ
p , eJ
o
� ∈ R2d encodes the
query predicate and the query object. Note that the query
4126
Algorithm 1: R2D2 Inference
input :Triple query q = (sq, pq, oq)
output :Classification score tτ ∈ [0, 1] of the judge
along with the list of arguments τ
1 τ ← [ ] // Initialize the list of
arguments with an empty list
// Loop over the debate rounds
2 for n = 1 to N do
// Loop over the two agents
3 for i = 1 to 2 do
4 e
(i)
1 ← sq // Initialize the
position of the agent
5 τ (i) n ← [ ] // Initialize the
argument with an empty list
// Loop over the path
6 for t = 1 to T do
7 Sample a transition (r, e) from e
(i)
t
according to πθ(i)// See
Equations (1-3)
8 τ (i) n .append(r, e) // Extend the
argument
9 e
(i)
t+1 ← e // Update the position
of the agent
10 end
11 τ .append(τ (i) n ) // Extend the list of
all argument
12 end
13 end
14 Process τ via the judge and retrieve the classification
score tτ // See Equation (6-8)
15 return tτ and τ
subject is not revealed to the judge because we want the
judge to base its decisions solely on the agents’ actions rather
than on the embedding of the query subject. After processing
all arguments in τ , the debate is terminated and the judge
scores the query triple q with tτ ∈ (0, 1) according to
tτ = σ

wReLU 
W

2
i=1

N
n=1
y(i)
n
�� , (8)
where W ∈ Rd×d and w ∈ Rd denote the trainable parameters of the classifier and σ(·) denotes the sigmoid activation
function. We also experimented with more complex architectures where the judge processes each argument in τ via a
recurrent neural network. However, we found that both the
classification performance and the quality of the arguments
suffered.
The objective function of the judge for a single query q is
given by the cross-entropy loss
Lq = φ(q) log tτ + (1 − φ(q)) (1 − log tτ ). (9)
Hence, during training, we aim to minimize the overall loss
given by
L = 1
|T |

q∈T
Lq , (10)
where T denotes the set of training triples. To prevent overfitting, an additional L2-penalization term with strength
λ ∈ R≥0 on the parameters of the judge is added to Equation
(10).
An overview of the overall architecture of R2D2 is depicted in Figure 2.
Rewards In order to generate feedback for the agents, the
judge also processes each argument τ (i) n individually and
produces a score according to
t
(i)
n = wReLU
Wf
�τ (i)
n , qJ
� , (11)
where both the neural network f as well as the linear weights
vector w correspond to the definitions given in the previous
paragraph. Thus, t
(i)
n corresponds to the classification logits
of q solely based on the n-th argument of agent i. Since
agent 1 argues for the thesis and agent 2 for the antithesis,
the rewards are given by
R(i)
n =
�
t
(i)
n if i = 1
−t
(i)
n otherwise. (12)
Intuitively speaking, this means that the agents receive high
rewards whenever they extract an argument that is considered
by judge as strong evidence for their position
Reward Maximization and Training Scheme We employ REINFORCE (Williams 1992) to maximize the expected cumulative reward of the agents given by
G(i) :=

N
n=1
R(i)
n . (13)
Thus, the agents’ maximization problems are given by
arg max
θ(i)
Eq∼KG+ Eτ (i)
1 ,τ (i)
2 ,...,τ (i)
N ∼πθ(i)
�
G(i)
�
�
�
�
q
�
, (14)
where KG+ is the set of training triples that contain in addition to observed triples in KG also unobserved triples.
The rationale is as follows: As KGs only contain true facts,
sampling queries from KG would create a dataset without
negative labels. Therefore it is common to create corrupted
triples that are constructed from correct triples (s, p, o) by
replacing the object with an entity o˜ to create a false triple
(s, p, o˜) ∈ KG / (see (Bordes et al. 2013)). Rather than creating
any kind of corrupt triples, we generate a set of plausible but
false triples. More concretely, for each (s, p, o) ∈ KG we generate one triple (s, p, o˜) ∈ KG / with the constraint that o˜ appears in the database as the object with respect to the relation
p. More formally, we denote the set of corrupted triples with
KGC := {(s, p, o˜)|(s, p, o˜) ∈ KG / , ∃s˜ : (˜s, p, o˜) ∈ KG} .
Then the set of training triples T is contained in KG+ :=
4127
Dataset Entities Relations Triples
FB15k-237 14,541 237 310,116
WN18RR 40,943 11 93,003
Hetionet 47,031 24 2,250,197
Table 1: Statistics of the datasets used in the experiments.
KG ∪ KGC . The underlying rationale for working with plausible but false facts is that we do not waste resources on
triples that break implicit type-constrains. Since this heuristic
only needs to be computed once and filters out triples that
could easily be discarded by a type-checker, we can focus on
the prediction of facts that present more of a challenge.
During training the first expectation in Equation (14) is
substituted with the empirical average over the training set.
The second expectation is approximated by the empirical
average over multiple rollouts. We also employ a moving
average baseline to reduce the variance. Further, we use entropy regularization with parameter β ∈ R≥0 to enforce
exploration.
In order to address the problem that the agents require a
trained judge to obtain meaningful reward signals, we freeze
the weights of the agents for the first episodes of the training.
The rationale is that training the judge does not necessarily
rely on the agents being perfectly aligned with their actual
goals. For example, even if the agents do not extract arguments that correspond to their position, they can still provide
useful features that the judge learns to exploit. After the initial training phase, where we only fit the parameters of the
judge, we employ an alternating training scheme where we
either train the judge or the agents.
4 Experiments
Datasets We measure the performance of R2D2 with respect to the triple classification and the KG completion task
on the benchmark datasets FB15k-237 (Toutanova et al. 2015)
and WN18RR (Dettmers et al. 2018). To test R2D2 on a
real world task we also consider Hetionet (Himmelstein and
Baranzini 2015), a large scale, heterogeneous graph encoding
information about chemical compounds, diseases, genes, and
molecular functions. We employ R2D2 for detecting genedisease associations and finding new target diseases for drugs,
two tasks of high practical relevance in the biomedical domain (see (Himmelstein and Baranzini 2015)). The statistics
of all datasets are given in Table 1.
Metrics and Evaluation Scheme As outlined in Section
2, triple classification aims to decide whether a query triple
(sq, pq, oq) is true or false. Hence, it is a binary classification task. For each method we set a threshold δ obtained by
maximizing the accuracies on the validation set. That means,
for a given query triple (sq, pq, oq), if its score (e.g., given
by Equation (8) for R2D2) is larger than δ, the triple will be
classified as true, otherwise as false. Since most KGs do not
contain facts that are labeled as false, we have generated a set
of negative triples: For each observed triple in the validation
and test set we create a false but plausible fact (see Section
3).2 We report the accuracy, the PR AUC, and ROC AUC
for all methods. Since R2D2 is a stochastic classifier, we can
produce multiple rollouts of the same query at inference time
and average the resulting classification scores to lower the
variance.
Even though the purpose of R2D2 is triple classification,
one can turn it into a KG completion method as follows:
We consider a range of object entities each producing a different classification score tτ given by Equation (8). Since
tτ can be interpreted as a measure for the plausibility of a
triple, we use the classification scores to produce a ranking. More concretely, we rank each correct triple in the
test set against all plausible but false triples (see Section
3). Since this procedure is computational expensive during training (one needs to run multiple debates per training
triple to produce a ranking), we select the following relations for training and testing purposes: For FB15k-237 we
follow (Socher et al. 2013) and consider the relations ‘profession’, ‘nationality’, ‘ethnicity’, and ‘religion’. Following
(Himmelstein and Baranzini 2015) and (Himmelstein et al.
2017), the relations ‘gene associated with disease’ and ‘compound treats disease’ are considered for Hetionet. We report
the mean rank of the correct entity, the mean reciprocal rank
(MRR), as well as Hits@k for k = 1, 3, 10 - the percentage
of test triples where the correct entity is ranked in the top k.
In order to find the most suitable set of hyperparameters
for all considered methods, we perform cross-validations.
Thereby the canonical splits of the datasets into a training,
validation, and test set are used. In particular, we ensured
that triples that are assigned to the validation or test set (and
their respective inverse relations) are not included in the
KG during training. The results on the test set of all methods are reported based on the hyperparameters that showed
the best performance (based on the highest accuracy for
triple classification and the the highest MRR for link prediction) on the validation set. We considered the following
hyperparameter ranges for R2D2: The number of latent dimensions d for the embeddings is chosen from the range
{32, 64, 128}. The number of LSTM layers for the agents
is chosen from {1, 2, 3}. The the number of layers in the
MLP for the judge is tuned in the range {1, 2, 3, 4, 5}. β was
chosen from {0.02, 0.05, 0.1}. The length of each argument
T was tuned in the range {1, 2, 3} and the number of debate rounds N was set to 3. Moreover, the L2-regularization
strength λ is set to 0.02. Furthermore, the number of rollouts
during training is given by 20 and 50 (triple classification)
or 100 (KG completion) at test time. The loss of the judge
and the rewards of the agents were optimized using Adam
with learning rate given 10−4. The best hyperparameter are
reported in Table 3.
All experiments were conducted on a machine with 48
CPU cores and 96 GB RAM. Training R2D2 on either dataset
takes at most 4 hour. Testing takes about 1-2 hours depending
on the dataset.
2
The datasets along with the code of R2D2 are available at
https://github.com/m-hildebrandt/R2D2.
4128
Dataset FB15k-237 WN18RR
Method Acc PR AUC ROC AUC Acc PR AUC ROC AUC
DistMult 0.739 0.78 0.803 0.804 0.901 0.872
ComplEx 0.738 0.789 0.796 0.802 0.887 0.860
TransE 0.673 0.727 0.736 0.69 0.794 0.732
TransR 0.612 0.655 0.651 0.721 0.724 0.792
SimplE 0.703 0.733 0.756 0.722 0.812 0.742
R2D2 0.751 0.86 0.848 0.726 0.821 0.808
R2D2+ 0.764 0.865 0.857 0.804 0.909 0.893
Table 2: The performance on the triple classification task.
Parameter FB15k-237 WN18RR FB15k-237 (subset) Hetionet
Embedding size (d) 64 64 64 32

stacked LSTM cells (agents) 2 1 2 2

layers MLP (judge) 1 1 3 2

rounds in a debate (N)3 3 3 3

Argument/path length (T) 2 2 2 2
Entropy regularization (β) 0.02 0.02 0.1 0.1
Table 3: The best hyperparameters for R2D2 found via cross-validation.
Results
Triple Classification We compare the performance of
R2D2 on the triple classifications task against DistMult, ComplEx, TransE, TransR, and SimplE. The results are displayed
in Table 2. On FB15k-237, R2D2 outperforms all baselines
with respect to the accuracy, the PR AUC, and the ROC AUC.
However, on WN18RR the performance of R2D2 is dominated by the factorization methods ComplEx and DistMult
by a significant margin. We conjecture that this is due to the
sparsity in the dataset. As a remedy we employ pretrained
embeddings from TransE that are fixed during training.3 We
denote the resulting method with R2D2+ and find that it outperforms all other methods with respect to the PR AUC and
ROC AUC on WN18RR. We also test R2D2+ on FB15k-237
and find that it improves the results of R2D2 by only a small
margin. This is expected since FB15k-237 is not as sparse as
WN18RR.
KG completion Next to the baselines used for triple classification we also employ the path based link prediction method
MINERVA. Note that it is not possible to compute a fair mean
rank for MINERVA, since it does not produce a complete
ranking of all candidate objects. Table 4 displays the results
on the completion task for all methods under consideration
on FB15k-237 and Hetionet (subsets; see above). R2D2 outperforms all other methods on FB15k-237 with respect to
all metrics but Hits@10. However, the performance of MINERVA is almost on par. Moreover, R2D2 outperforms all
baselines on Hetionet with respect to the MRR, the mean
rank, Hits@3, and Hits@10. While MINERVA exhibits the
best performance with respect to Hits@1, R2D2 yields significantly better results with respect to all other metrics.
3
We choose TransE embeddings due to the simple functional
relations between entities. These can be easily exploited by R2D2.
Survey To asses whether the arguments are informative for
users in an objective setting, we conducted a survey where
respondents take the role of the judge making a classification
decision based on the agents’ arguments. More concretely, we
set up an online quiz consisting of ten rounds. Each round is
centered around a query (with masked subject) sampled from
the test set of FB15k-237 (KG completion). Along with the
query statement we present the users six arguments extracted
by the agents in randomized order. Based on these arguments
the respondents are supposed to judge whether the statement
is true or false. In addition, we asked the respondents to rate
their confidence in each round.
Based on 44 participants (109 invitations were sent) we
find that the overall accuracy of the respondents’ classifications was 81.8%. Moreover, based on a majority vote (i.e.,
classification based on the majority of respondents) nine out
of ten questions were classified correctly indicating that humans are approximately on par with the performance of the
automated judge. Further, the statement where the majority
of respondents was wrong corresponds to the only query
that was also misclassified by the judge. In this round the
participants were supposed to decide whether a person has
the religion Methodism. It is hard to answer this question
correctly because the person at hand is Margaret Thatcher
who had two different religions over her lifetime: Methodism
and the Church of England. The fact that the majority of
respondents and the judge agree in all rounds indicates that
the judge is aligned with human intuition and that the arguments are informative. Moreover, we found that when users
assigned a high confidence score to their decision (’rather
certain’ or ’absolutely certain’) the overall accuracy of their
classification was 89%. The accuracy dropped to 68.4% when
users assigned a low confidence score (’rather uncertain’ or
’absolutely uncertain’).
4129
Dataset FB15k-237 (subset) Hetionet (subset)
Metrics MRR Mean Rank Hits@1 Hits@3 Hits@10 MRR Mean Rank Hits@1 Hits@3 Hits@10
DistMult 0.502 8.607 0.363 0.572 0.779 0.134 31.190 0.054 0.121 0.291
ComplEx 0.521 7.477 0.383 0.587 0.806 0.148 31.439 0.061 0.141 0.325
TransE 0.473 10.2112 0.345 0.522 0.745 0.110 35.559 0.033 0.097 0.267
TransR 0.543 8.737 0.391 0.635 0.816 0.144 27.841 0.049 0.136 0.340
SimplE 0.429 11.760 0.275 0.506 0.736 0.177 31.965 0.091 0.174 0.354
MINERVA 0.580 – 0.448 0.657 0.857 0.174 – 0.097 0.18 0.364
R2D2 0.589 6.332 0.459 0.665 0.853 0.206 23.486 0.090 0.219 0.455
Table 4: The performance on the KG completion task.
Query: Richard Feynman nationality
−−−−−−−→ USA? Nelson Mandela hasP rof ession
−−−−−−−−−−→ Actor?
Agent 1: Richard Feynman livedInLocation
−−−−−−−−−−→ Queens Nelson Mandela hasF riend
−−−−−−−→ Naomi Campbell
∧ Queens locatedIn
−−−−−−→ USA Naomi Campbell hasDated
−−−−−−→ Leonardo DiCaprio
Agent 2: Richard Feynman hasEthnicity
−−−−−−−−−→ Russian people Nelson Mandela hasP rof ession
−−−−−−−−−−→ Lawyer
∧ Russian people geographicDistribution
−−−−−−−−−−−−−−−→ Republic of Tajikistan ∧ Lawyer specializationOf−1
−−−−−−−−−−−−−→ Barrister
Table 5: Two example debates generated by R2D2: While agent 1 argues that the query is true and agent 2 argues that it is false.
5 Discussion and Future Works
We examined the quality of the extracted paths manually and
typically found reasonable arguments, but quite often also
arguments that do not make intuitive sense. We conjecture
that one reason for that is that agents often have difficulties finding meaningful evidence if they are arguing for the
false position. Moreover, for many arguments, most of the
relevant information is already contained in the first step of
the agents and later transitions often contain seemingly irrelevant information. This phenomenon might be due to the
fact that the judge ignores later transitions and agents do not
receive meaningful rewards. Further, relevant information
about the neighborhood of entities can be encoded in the
embeddings of entities. While the judge has access to this
information through the training process, it remains hidden
to users. For example, when arguing that Nelson Mandela
was an actor (see Table 5), the argument of agent 1 requires
the user to know that Naomi Campbell and Leonardo DiCaprio are actors (which is encoded in FB15k-237). Then
this argument serves as evidence that Nelson Mandela was
also an actor since people tend to have friends that share their
profession (social homophily). However, without this context
information it is not intuitively clear why this is a reasonable
argument.
To examine the interplay between the agents and the judge,
we consider a setting where we train R2D2 with two agents,
but neglect the arguments of one agent during testing. This
should lead to a biased outcome in favor of the agent whose
arguments are considered by the judge. We test this setting
on FB15k-237 and find that when we consider only the arguments of agent 1, the number of positive predictions increases by 18.8%. In contrast, when we only consider agent 2,
the number positive predictions drops by 70.2%. This result
shows that the debate dynamics are functioning as intended
and that the agents learn to extract arguments that the judge
considers as evidence for their respective position.
While the results of the survey are encouraging, we plan to
develop variants of R2D2 that improve the quality of the arguments and conduct a large scale experimental study that also
includes other baselines in a controlled setting. Moreover,
we plan to discuss fairness and responsibility considerations.
In that regard, (Nickel et al. 2015) stress that when applying statistical methods to incomplete KGs the results are
likely to be affected by biases in the data generating process
and should be interpreted accordingly. Otherwise, blindly
following these predictions can strengthen the bias. While
the judge in our method also exploits skews in the data, the
arguments can help to identify these biases and potentially
exclude problematic arguments from the decision.
6 Conclusion
We proposed R2D2, a new approach for KG reasoning based
on a debate game between two opposing reinforcement learning agents. The agents search the KG for arguments that
convince a binary classifier (judge) of their position. Thereby,
they act as sparse, adversarial feature generators. Since the
judge bases its decision solely on mined arguments, R2D2 is
more interpretable than other baseline methods. Our experiments showed that R2D2 outperforms all baselines in the
triple classification setting with respect to all metrics on the
benchmark datasets WN18RR and FB15k-237. Moreover,
we demonstrated that R2D2 can in principle operate in the
KG completion setting. We found that R2D2 has competitive performance compared to all baselines on a subset of
FB15k-237 and Hetionet. Furthermore, the results of our survey indicate that the arguments are informative and that the
judge is aligned with human intuition.
References
Baier, S.; Ma, Y.; and Tresp, V. 2017. Improving visual
relationship detection using semantic modeling of scene descriptions. In International Semantic Web Conference, 53–68.
Springer.
Bollacker, K.; Evans, C.; Paritosh, P.; Sturge, T.; and Taylor,
J. 2008. Freebase: a collaboratively created graph database
4130
for structuring human knowledge. In Proceedings of the 2008
ACM SIGMOD international conference on Management of
data, 1247–1250. ACM.
Bordes, A.; Usunier, N.; Garcia-Duran, A.; Weston, J.; and
Yakhnenko, O. 2013. Translating embeddings for modeling
multi-relational data. In Advances in neural information
processing systems, 2787–2795.
Das, R.; Dhuliawala, S.; Zaheer, M.; Vilnis, L.; Durugkar, I.;
Krishnamurthy, A.; Smola, A.; and McCallum, A. 2018. Go
for a walk and arrive at the answer: Reasoning over paths in
knowledge bases using reinforcement learning. In International Conference on Learning Representations.
Dettmers, T.; Minervini, P.; Stenetorp, P.; and Riedel, S. 2018.
Convolutional 2d knowledge graph embeddings. In ThirtySecond AAAI Conference on Artificial Intelligence.
Dosilovi ˇ c, F. K.; Br ´ ciˇ c, M.; and Hlupi ´ c, N. 2018. Explain- ´
able artificial intelligence: A survey. In 2018 41st International convention on information and communication technology, electronics and microelectronics (MIPRO), 0210–0215.
IEEE.
Goodman, B., and Flaxman, S. 2017. European union regulations on algorithmic decision-making and a “right to explanation”. AI Magazine 38(3):50–57.
Han, X., and Zhao, J. 2010. Structural semantic relatedness: a
knowledge-based method to named entity disambiguation. In
Proceedings of the 48th Annual Meeting of the ACL, 50–59.
ACL.
Hildebrandt, M.; Sunder, S. S.; Mogoreanu, S.; Thon, I.;
Tresp, V.; and Runkler, T. 2018. Configuration of industrial
automation solutions using multi-relational recommender
systems. In Joint European Conference on Machine Learning
and Knowledge Discovery in Databases, 271–287. Springer.
Hildebrandt, M.; Sunder, S. S.; Mogoreanu, S.; Joblin, M.;
Mehta, A.; Thon, I.; and Tresp, V. 2019. A recommender
system for complex real-world applications with nonlinear
dependencies and knowledge graph context. In European
Semantic Web Conference, 179–193. Springer.
Himmelstein, D. S., and Baranzini, S. E. 2015. Heterogeneous network edge prediction: a data integration approach
to prioritize disease-associated genes. PLoS computational
biology 11(7):e1004259.
Himmelstein, D. S.; Lizee, A.; Hessler, C.; Brueggeman, L.;
Chen, S. L.; Hadley, D.; Green, A.; Khankhanian, P.; and
Baranzini, S. E. 2017. Systematic integration of biomedical
knowledge prioritizes drugs for repurposing. Elife 6:e26726.
Hochreiter, S., and Schmidhuber, J. 1997. Long short-term
memory. Neural computation 9(8):1735–1780.
Irving, G.; Christiano, P.; and Amodei, D. 2018. Ai safety
via debate. arXiv preprint arXiv:1805.00899.
Kazemi, S. M., and Poole, D. 2018. Simple embedding for
link prediction in knowledge graphs. In Advances in Neural
Information Processing Systems, 4284–4295.
Lao, N.; Mitchell, T.; and Cohen, W. W. 2011. Random
walk inference and learning in a large scale knowledge base.
In Proceedings of the Conference on Empirical Methods in
Natural Language Processing, 529–539. ACL.
Lin, Y.; Liu, Z.; Sun, M.; Liu, Y.; and Zhu, X. 2015. Learning
entity and relation embeddings for knowledge graph completion. In Twenty-ninth AAAI conference on artificial intelligence.
Lukovnikov, D.; Fischer, A.; Lehmann, J.; and Auer, S. 2017.
Neural network-based question answering over knowledge
graphs on word and character level. In Proceedings of the
26th international conference on World Wide Web, 1211–
1220. International World Wide Web Conferences Steering
Committee.
Montavon, G.; Lapuschkin, S.; Binder, A.; Samek, W.; and
Muller, K.-R. 2017. Explaining nonlinear classification deci- ¨
sions with deep taylor decomposition. Pattern Recognition
65:211–222.
Nickel, M.; Murphy, K.; Tresp, V.; and Gabrilovich, E.
2015. A review of relational machine learning for knowledge
graphs. Proceedings of the IEEE 104(1):11–33.
Nickel, M.; Tresp, V.; and Kriegel, H.-P. 2011. A three-way
model for collective learning on multi-relational data. In
International Conference on Machine Learning, volume 11,
809–816.
Singhal, A. 2012. Introducing the knowledge graph: things,
not strings. [Online; accessed 28-March-2019].
Socher, R.; Chen, D.; Manning, C. D.; and Ng, A. 2013.
Reasoning with neural tensor networks for knowledge base
completion. In Advances in neural information processing
systems, 926–934.
Suchanek, F. M.; Kasneci, G.; and Weikum, G. 2007. Yago:
a core of semantic knowledge. In Proceedings of the 16th international conference on World Wide Web, 697–706. ACM.
Toutanova, K.; Chen, D.; Pantel, P.; Poon, H.; Choudhury,
P.; and Gamon, M. 2015. Representing text for joint embedding of text and knowledge bases. In Proceedings of the
2015 Conference on Empirical Methods in Natural Language
Processing, 1499–1509.
Trouillon, T.; Welbl, J.; Riedel, S.; Gaussier, E.; and
Bouchard, G. 2016. Complex embeddings for simple link prediction. In International Conference on Machine Learning,
volume 48, 2071–2080.
Williams, R. J. 1992. Simple statistical gradient-following
algorithms for connectionist reinforcement learning. Machine
learning 8(3-4):229–256.
Xiong, W.; Hoang, T.; and Wang, W. Y. 2017. Deeppath: A
reinforcement learning method for knowledge graph reasoning. In Proceedings of the 2017 Conference on Empirical
Methods in Natural Language Processing. ACL.
Yang, B.; Yih, W.-t.; He, X.; Gao, J.; and Deng, L. 2015.
Embedding entities and relations for learning and inference
in knowledge bases. In International Conference on Learning
Representations.
4131

[vtempest]

https://www.sciencedirect.com/science/article/pii/S1570868313000748
A neural cognitive model of argumentation with application to legal inference and decision making
Author links open overlay panelArtur S. d'Avila Garcez a, Dov M. Gabbay b, Luis C. Lamb c

Formal models of argumentation have been investigated in several areas, from multi-agent systems and artificial intelligence (AI) to decision making, philosophy and law. In artificial intelligence, logic-based models have been the standard for the representation of argumentative reasoning. More recently, the standard logic-based models have been shown equivalent to standard connectionist models. This has created a new line of research where (i) neural networks can be used as a parallel computational model for argumentation and (ii) neural networks can be used to combine argumentation, quantitative reasoning and statistical learning. At the same time, non-standard logic models of argumentation started to emerge. In this paper, we propose a connectionist cognitive model of argumentation that accounts for both standard and non-standard forms of argumentation. The model is shown to be an adequate framework for dealing with standard and non-standard argumentation, including joint-attacks, argument support, ordered attacks, disjunctive attacks, meta-level attacks, self-defeating attacks, argument accrual and uncertainty. We show that the neural cognitive approach offers an adequate way of modelling all of these different aspects of argumentation. We have applied the framework to the modelling of a public prosecution charging decision as part of a real legal decision making case study containing many of the above aspects of argumentation. The results show that the model can be a useful tool in the analysis of legal decision making, including the analysis of what-if questions and the analysis of alternative conclusions. The approach opens up two new perspectives in the short-term: the use of neural networks for computing prevailing arguments efficiently through the propagation in parallel of neuronal activations, and the use of the same networks to evolve the structure of the argumentation network through learning (e.g. to learn the strength of arguments from data).

Previous article in issueNext article in issue
Keywords
ArgumentationNeural-symbolic reasoningLegal decision makingCognitive modelling

1. Introduction
   Formal models of argumentation have been investigated in several areas, from multi-agent systems and artificial intelligence (AI) to decision making, philosophy and law [4], [8], [12], [17], [30], [33]. In artificial intelligence, models of argumentation have been used for commonsense reasoning, modelling chains of defeasible arguments to reach a conclusion. Such models are mainly founded on logic-based approaches, which have been the standard for the representation of argumentative reasoning in AI [3].

Recent efforts to bridge the gap between logic-based models of argumentation and cognitive models of computation include [10], [11], [34]. In [10], [11], an equivalence is shown between value-based argumentation [2] and standard connectionist networks [22]. This has created a new line of research in argumentation where (i) neural networks can be used as a cognitive computational model for argumentation and (ii) neural networks can be used to combine argumentation, quantitative reasoning and statistical learning. In [34], behavioural data is used to conclude that in human reasoning, reinstatement does not yield a full recovery of the attacked argument; [1] implements the same idea mathematically through equations that resemble the predator–prey dynamics of species populations. Further work integrating logic and neural networks include [20] where clustering in fuzzy ART networks is used to compute prevailing arguments, and [25] which extends the work in [10] to deal with self-defeating arguments and provides a number of interesting examples. At the same time, some non-standard models of argumentation start to emerge, enriching current models with cognitive abilities; e.g. [15] discusses meta-level attacks, coalitions, disjunctive attacks and argument support, [37] provides an adequate semantics for joint attacks, among much else, [29] seeks to unravel the role of emotions in argumentation, [14], [23] propose to handle uncertainty in argumentation through the assignment of probabilities and weights to arguments, and [13], [26] offer a qualitative method for reasoning about uncertainty and preferences between arguments.

We argue that the adoption of a cognitive approach to argumentation can offer an adequate framework for dealing with both standard and non-standard argumentation models. In this paper, we show that a cognitive approach can model many different aspects of argumentation in a uniform way, in particular, modelling uncertainty in argumentative reasoning and the accrual of arguments. The approach opens up, through the use of a connectionist system, two new short-term perspectives: (i) the use of neural networks to compute prevailing arguments efficiently through the propagation in parallel of neuronal activation signals and (ii) the use of the same networks to evolve the structure of an argumentation network through learning (e.g. to learn the strength of arguments from data). We believe that this approach also opens a more long-term perspective for the research on argumentation: the use of connectionist models of computation to help investigate and evaluate cognitive models of argumentation. For example, ideas from connectionism about the modelling of attention and emotion could be investigated in the context of argumentation [29], [36].

Argumentation has also been proposed as a method for helping machine learning systems [27] where an expert's arguments, or the reasons for some of the learning examples, are used to guide the search for hypotheses. This is related to the body of work on abductive reasoning and combinations of abduction and inductive logic programming [24], [28]. It is said that the arguments constrain the combinatorial search among possible hypotheses, directing the search towards hypotheses that are more comprehensible in the light of an expert's background knowledge [27]. We subscribe to this idea. In fact, experimental results on the integration of learning with background knowledge using neural networks have been shown to outperform symbolic and purely-connectionist systems, especially in the presence of noisy data [9]. In this paper, differently from [27], however, learning from data can be used to inform a process of numerical argumentation, allowing different perspectives of human argumentation, including joint attacks, argument support, meta-argumentation and disjunctive attacks, to be modelled in the same framework, as detailed in what follows.

The remainder of the paper is organised as follows. First, we define the concepts of argumentation and neural cognitive models used throughout the paper. Then, we present an algorithm, generalised from [10], which translates standard and non-standard argumentation frameworks into standard connectionist networks. We show that the resulting neural model executes a sound parallel computation of the prevailing arguments according to a number of standard argumentation semantics, and also according to value-based argumentation models [2], abstract dialectical frameworks [5], and other forms of human argumentation. We illustrate the network computation through examples that include joint attacks, support, meta-argumentation and disjunctive attacks. Finally, we apply the framework to a real decision making situation in legal reasoning, which indicates that the network model can be a useful tool in the modelling of non-standard and numerical argumentation, and in the analysis of what-if questions that emerge in real situations. The paper concludes with a brief discussion and directions for future work.

2. Background
   In this section, we present the concepts of argumentation and neural networks used throughout the paper.

Definition 1

An argumentation framework has the form
, where α is a set of arguments, and
is a relation indicating which arguments attack which other arguments.

In order to record the values associated with arguments, in [2] Bench-Capon has extended Dung's argumentation framework [12] by adding to it a set of values and a function mapping arguments to values. This brings argumentation closer to a numerical, connectionist approach.

Definition 2

A value-based argumentation framework is a 5-tuple
, where α is a finite set of arguments, attacks is an irreflexive binary relation on α, V is a non-empty set of values, val is a function mapping elements in α to elements in V, and P is a set of possible audiences, where we may have as many audiences as there are orderings on V. For every
,
.

Bench-Capon also defines the notions of objective and subjective acceptability of arguments. The first are arguments acceptable no matter the choice of preferred values for every audience, whereas the second are acceptable to some audiences. Arguments which are neither objectively nor subjectively acceptable are called indefensible. A function v from attack to
gives the relative strength of an argument. Given
, if
then
is said to be stronger than
. Otherwise, if
then
is weaker than
.

We shall also relate the neural approach with abstract dialectical frameworks [5].

Definition 3

An abstract dialectical framework (ADF) is a tuple
where S is a set of nodes,
is a set of links,
is a set of total functions
, one for each node s.

Consider an example. A person is innocent (i), unless she is a murderer (m). A killer (k) is a murderer (m), unless she acted in self-defence (s). There must be evidence for self-defence, e.g. a witness (w) who is not known to be a liar (l). An ADF can model the above example by stating that s is in if w is in and l is out. Similarly, m is in if k is in and s is out. In other words, like neural networks, ADFs include the concept of support. If k and w are in and l is out then s will be in; s then defeats m regardless of k so that i prevails. Every argumentation framework has an associated ADF. Also, normal logic programs have associated ADFs [5]. Since every logic program also has an associated neural network [9], this fact will be used later to show the required correspondences.

Other established definitions of argumentation semantics will be considered as well [6], [7], [12]. In all the definitions that follow, an argumentation framework is a pair
, as above. First, let us define a function
, such that
is defended by α}, where
. The function F computes the arguments accepted in the sense of [12] or defended by a set of arguments, in the sense of [7]. In this way, we can define conflict-free sets of arguments. A set of arguments is conflict-free if and only if (iff, for short) it does not contain any arguments A and B such that A defeats B. Let Args be a conflict-free set of arguments. Args is said to be a complete extension iff
.

In another useful argumentation semantics, the grounded semantics, only one extension is yielded by making use of the function F and defining the grounded extension as the minimal fixed point of F [31]. A grounded extension is conflict-free [7]. In the preferred semantics [12], a more credulous approach is used, which maximizes the number of accepted arguments. In order to define preferred semantics, Dung introduces the notion of admissible sets of arguments. A set of arguments is admissible iff it is conflict-free and
. The set Args is a preferred extension iff Args is maximal w.r.t. set inclusion. In the stable semantics of argumentation [18], a set of arguments is called stable iff it defeats each argument that does not belong to this set. In semi-stable semantics, a set of arguments Args is semi-stable iff Args is a complete extension of which
is maximal and defines a set of arguments which are defeated by an argument in Args.

In order to illustrate the different argumentation semantics, we borrow an example from [7]. Consider the abstract argumentation framework depicted as a directed graph in Fig. 1, where each node is an argument and an arrow from argument X to argument Y denotes an attack from X on Y. This framework has grounded extension {E, F}; complete extensions {E, F}, {B, C, E, F} and {A, E, F}; preferred extensions {B, C, E, F} and {A, E, F}; stable extension {B, C, E, F}; and semi-stable extension {B, C, E, F}. As pointed out in [7], semi-stable and stable extensions will coincide whenever the framework has at least one stable extension.

Download : Download full-size image
Fig. 1. An abstract argumentation framework with arguments A, B, C, D, E, and F; D is a self-defeating argument.

Finally, we shall also consider meta-argumentation [15]. In a meta-argumentation network, let argument a attack b in the usual way. It is possible to define an argument c as an attack on a's attack. This makes the framework more fine-grained in that c's attack does not propagate throughout the network, but is targeted at one specific attack in the network. Meta-argumentation can be reduced to argumentation frameworks with the addition of a node denoting
and a careful re-organization of the network [15].

We shall use a standard definition of neural networks, as follows. A neural network is a directed graph with the following structure: a unit (or neuron) in the graph is characterised, at time t, by its input vector
, its input potential
, its activation state
, and its output
. The units of the network are interconnected via a set of directed and weighted connections such that if there is a connection from unit i to unit j then
denotes the weight of this connection. The input potential of neuron i at time t (
) is obtained by computing a weighted sum for neuron i such that
. The activation state
of neuron i at time t is then given by the neuron's activation function
such that
. Typically,
is either a linear function, a non-linear step function, or a sigmoid function such as
. In this paper, we use
as activation function and inputs values in the range [
]. In addition,
(an extra weight with input always fixed at 1) is known as the bias of neuron i. We say that neuron i is active at time t if
. Finally, the neuron's output value
is given by its output function
. Usually,
is the identity function so that
.

3. Neural cognitive argumentation frameworks
   We start by considering the relationship between argumentation and neural networks informally. If we represent an argument by a neuron then a connection from neuron i to neuron j can indicate that argument i either attacks or supports argument j, the weight of the connection corresponding to the strength of the attack or support. Since real numbers are used as weights in a neural network, we associate negative weights with attacks, positive weights with support, and zero weight with the lack of an attack or support.

Definition 4

We say that an argument prevails at time t when the activation state of its associated neuron is greater than a predefined value
at time t,
. We say that an argument is defeated at time t when the neuron's activation is smaller than
. Otherwise, i.e. for activations in the interval [
], we say that it is unknown whether an argument prevails or not.

There are different ways in which an argument may support other arguments. For example, an argument i may support argument j by attacking an argument k that attacks j, or argument i may support j directly, e.g. by strengthening the value of j, or even i and j may get together to attack k. Generally, an argument i supports an argument j if the coordination of i and j reduces the likelihood of j being defeated [39]. A general neural network structure capable of accounting for the above combinations of attack and support and the necessary computations of prevailing arguments would be a recurrent network having an input layer, a single hidden layer, and an output layer with feedback from the output to the input layer [9]. At time
, input values are provided to the network. Neuronal activation is then propagated in parallel from the input to the hidden layer at time
, and to the output of the network at time
. At time
, the output values can be fed back to the input of the network, and this process can be repeated until a stable state is obtained, when input and output values will be the same for each pair of neurons with feedback. Arguments for which the associated neuron is activated at the stable state are said to prevail.

Consider the neural network of Fig. 2(b), which implements the argumentation network of Fig. 2(a). Arguments A, B and C are encoded in the network's input and output layers. In this example, arguments do not get together to attack another argument so that each input neuron is connected directly to a hidden neuron and the weights from the input to the hidden layer simply serve to send the input information forward. Support and attack information is encoded by the weights leading from the hidden to the output layer of the network. Support for A is encoded by the positive weight going from neuron
to output neuron A. Similarly, support for B (resp. C) is encoded by the weight going from
(resp.
) to B (resp. C). A's attack on B is represented by the dashed arrow going from
to B, which should have negative weight, as specified in the algorithm below. Similarly, B's attack on C is represented by the dashed arrow going from
to C, with a negative weight.

Download : Download full-size image
Fig. 2. (a) Illustration of an argumentation network in which argument A attacks argument B, which in turn attacks argument C, and (b) its neural implementation, where solid lines receive positive weights and dashed lines, which represent the attacks, receive negative weights.

If the absolute value of the weight going from
to B (call it
) is larger than the value of the weight from
to B (call it
) then A defeats B. This produces {
} as prevailing arguments and is identical to the usual (non-value based) interpretation of argumentation frameworks [12]. The prevailing arguments are computed by the network as follows: suppose that A, B and C are all present at time
, denoted by input vector [
]. Hidden neurons
,
and
all become activated;
activates A, and blocks the activation of B from
(because
). At the same time, C is blocked by
(by default, we assume that attacks are stronger than support, i.e. the weight from
to C is greater in absolute value than the weight from
to C, unless stated otherwise). Thus, at time
, only output neuron A is activated (i.e. only the output of neuron A is greater than
). This is then represented as a new input vector [
] at time
. Given this new input, A continues to prevail, B is defeated as before, but C is reinstated because B now fails to defeat it, since B is not present in the input anymore. Thus, at time
, output neurons A and C become activated. At time
, a new input [
] is produced. Finally, with [
] given as input, output neurons A and C become activated again, producing a stable state in the network. Recall that network outputs are real values in the range (
). To compute a stable state, each output value in the range (
) is mapped to 1, output values in the range (
) are mapped to −1, and values in the range [
] are mapped to 0. After this is done, the network's new input vector can be compared with its previous input. In this example, the input vector at time
will be identical to the input vector at time
, that is [
] is a stable state. The network's stable activations indicate the prevailing arguments {
}. This result also coincides with the standard ADF interpretation of support (Definition 3). In the case of VAF (Definition 2), if B is preferred over A, all that needs changing in the network is the value of
or
so that
, denoting that the attack from A on B should not be strong sufficiently to defeat B [10]. In this case, input [
] produces new input [
], which is a stable state, given the neural network's new set of weights. This new network, therefore, computes prevailing arguments {
}, as expected in the case of the VAF under consideration.

As another example, consider the network of Fig. 3(b). Here, a cycle exists in the argumentation network. This may create an infinite loop in the computation of the stable state of the associated neural network; e.g. if the network were to be started on input vector [
], it would oscillate between that state and state [
] indefinitely, with the arguments all being attacked in parallel and defeated in a single pass through the network, and then reinstated in the next pass through the network. In order to handle this as intended by the usual argumentation semantics, whenever the neural network reaches a state [
], the computation stops. In this case, the neural network computes the grounded extension of the argumentation network, as discussed in more detail later. Summarising, our policy is that the network computation stops when either a stable state is obtained or when [
] is reached. We call [
] a terminal state. Notice that by following a value-based approach, and changing the weights according to some preference relation, one might eliminate the loop [10]. For example, if as before, B is preferred over A then the attack from A on B will not be successful, with input [
] producing stable state [
]. When the weights are different, the neural network is less likely to enter into an infinite loop (see [10] for a discussion on how weight learning given new information following a value-based approach can be useful at resolving loops).

Download : Download full-size image
Fig. 3. (a) An example of a cyclic argumentation network and (b) its neural implementation whose parallel computation produces, in a single pass through the network, a terminal state [−1,−1,−1] with no prevailing argument, following a grounded argumentation semantics.

The algorithm below generalises the algorithm first introduced in [10], which was made for VAF only. It translates argumentation frameworks into single-hidden layer neural networks that behave as exemplified, and can be used to compute prevailing arguments in parallel. It does so by defining the structure and set of weights of a neural network as a function of
, using activation function
and inputs in {
}. Each argument has a strength given by positive weight
. Certain arguments may attack other arguments with negative weight
, and certain arguments may support other arguments with positive weight
. The weights are such that activations in the interval [
] are guaranteed not to occur for now (we shall consider uncertainty in the next section). The neural network produces outputs in the intervals (
), which is mapped to false and denotes that an argument is defeated, and (
), which is mapped to true, denoting that an argument prevails.

By default, Algorithm 1 implements Dung's argumentation frameworks. If an argument
attacks an argument
, and
is itself not attacked, then neuron
should block neuron
. However, if
is deemed weaker than
, and no other argument attacks
, then neuron
should not block neuron
. To achieve this, and account for a number of other, alternative semantics and modes of argumentation in a neural network [16], the constraints on
and
can be modified, as will become clearer in Section 4. Also, in Algorithm 1, a single supporting argument is deemed sufficient to defeat any attack; the other alternatives, leading to different constraints on
and
, will be considered in Section 4. A novelty of the algorithm is that arguments may have strengths
, which may vary from one attack to another. Before we consider the extensions of Algorithm 1, however, let us make the ideas developed so far more precise.

Download : Download full-size image
Algorithm 1. General neural argumentation algorithm.

Definition 5

Let
. Let
denote the activation of neuron α at time t. We say that a neural network
computes an argumentation framework
if whenever
and
then
, and whenever
for every
such that
then
(reinstatement).

Proposition 6

For each argumentation framework
there exists a single-hidden layer neural network
such that
computes
.

Proof

First, we show that if
in the input layer of
then
in the output layer of
whenever there are no attacks on
. In the worst case, the input potential of hidden neuron
is
, and the output of
is
. We want
. Again in the worst case, the input potential of output neuron
will be
, and we need
. As a result,
needs to be satisfied. When there is an attack on
, the activation of output neuron
needs to be smaller than
if hidden neuron
is active. In the worst case,
has activation
,
has activation 1, and any other attacking neuron has activation −1. Hence,
has to be satisfied, where n is the number of attacks. Dually, when there are no attacks on
, the following inequality has to be satisfied:
, which gives
and the constraint on
, as shown in Algorithm 1, step 4. When at least one argument
supports argument
, the following inequality has to be satisfied (again, in the worst case analysis):
. Dually, in the worst case,
has to be satisfied, which gives
and the constraint on
, as shown in Algorithm 1, step 5. This completes the proof. □

Proposition 7

For each ADF
there exists a single-hidden layer neural network
such that
computes
.

Proof

The standard ADF interpretation is that
when
attacks
. This interpretation has been covered in the proof of Proposition 6. In weighted ADFs, however, one can distinguish different combinations of attack and support [5], in particular, a supporting link can be stronger than an attacking link so that the attacked argument prevails. Since neural networks with as few as a single hidden layer are universal approximators, it follows that in the neural argumentation approach, any Boolean combination of attacks and support can be computed. In the specific case of support, we need to show that if
supports
then if
then
. As before, we associate inputs in the interval
to in and inputs in the interval
to out. In the worst case, we need
. Thus,
, as guaranteed by the algorithm. This completes the proof. Notice that, in practice, we convert inputs in the interval
to 1, and inputs in the interval
to −1, which relaxes further the above constraint on
.  □

We can also show that the neural-network approach is very general by proving that it computes the many different argumentation semantics, as defined earlier [6], [12]. Before that is done, however, we should note that an argument that is not attacked by any other argument cannot be reinstated in the neural network (an argument that is attacked but not defeated will be reinstated as usual). For example, argument E in Fig. 1 will be in if its input value is 1, and will be out if its input value is −1. It will continue to be out over time if it is out initially, and will be in otherwise. Argument B, on the other hand, will be reinstated whenever argument A is not present, and vice-versa.

Lemma 8

For each argumentation framework
there exists a single-hidden-layer recurrent neural network
such that the complete extensions of
are stable states of
.

Proof

Recall that a set of arguments Args is said to be a complete extension of
iff
. From Proposition 6, one pass through
computes
. Recurrently connected,
iterates
until a stable state is reached, which is a fixed point of F, that is
.  □

Corollary 9

For each argumentation framework
there exists a recurrent neural network
such that the grounded, preferred, stable and semi-stable extensions of
are stable states of
.

As an example, consider again the argumentation framework of Fig. 1. Starting from
, without a terminal state, the associated neural network can converge to the following stable states: {B, C, E, F} and {A, E, F}, corresponding to its preferred extensions. With a terminal state, the network will converge to either {E, F}, {B, C, E, F} or {A, E, F}, corresponding to its complete extensions.

Finally, consider the argumentation network of Fig. 4, for which no stable state exists. With the use of a terminal state, the corresponding neural network will implement a semi-stable semantics, providing the empty set as the only extension. Without a terminal state, the neural network will loop (until it is either halted or its weights are changed), implementing a stable semantics.

Download : Download full-size image
Fig. 4. Semantic circularity for which no stable state exists.

The neural networks of Fig. 2, Fig. 3 should be seen as computational models rather than abstract argumentation frameworks. As discussed, the networks can be used in the parallel computation of prevailing arguments: given input vector [
] at the start, the networks should always converge to a stable state corresponding to a set of prevailing arguments, according to a (value-based, preferred, etc.) argumentation semantics.

Definition 10

We say that neural network
computes an (extension-based semantics of) argumentation framework
if every stable state of
is an extension of
.

Proposition 11

If
computes
and
admits a single extension then starting from input vector
,
always finds a stable state corresponding to the prevailing arguments of
.

Proof

The arguments in
can be viewed as logic programming clauses of the form
. Starting from [
],
will treat each
as a fact unless
is attacked and defeated by another argument
, which corresponds to adding a clause
to the program (where ¬ stands for negation by failure). If
admits a single extension then the corresponding program will have a single model. The stable models of any logic program can be computed by a single-hidden layer neural network; with a single model, the network is known to always settle down in a stable state corresponding to this model, as proved in [9]. Such a result can be applied directly here to complete the proof.  □

As exemplified earlier, it should be possible to extend Proposition 11 to certain very general classes of argumentation networks that admit multiple extensions. Due to the numerical nature of the networks, they are unlikely to oscillate unless integer weights with the same absolute values are used. For example, we have seen that, starting from [
], the network of Fig. 3 converges to a terminal state. Otherwise, it loops. As argued in [10], a network that loops should be seen as an opportunity for learning, whereby what-if questions can be considered and the network's weights can be changed slightly. Similarly, when a network reaches a terminal state, this should trigger a search for new evidence about the relative strength of the arguments. For example, consider the case where arguments A and B attack each other in a cycle. The corresponding neural network has two stable states, namely [
] and [
], corresponding to the two stable extensions of the argumentation network. Starting from [
], the neural network produces output [
], which is a terminal state. At this point, the computation stops. However, a loop exists in the network computation, since input [
] would produce output [
]. A terminal state does not provide much information by itself (a terminal state could be seen as producing maximum uncertainty). However, if the reaching of a terminal state is seen as a trigger for a search for more evidence, perhaps this search could shed new light on the relative strengths of arguments A and B, defining a preference for either argument by changing the weights of the network. If a neural network is such that the weights do not have the same absolute values then it is unlikely that arguments will cancel each other, as seen above, so that the network loops. Breaking such a symmetry in the set of weights by changing their values slightly is the norm of network learning, and could be seen as an alternative to the use of terminal states and a solution to the problem of having loops in the computation of such neural networks [10].

4. Neural cognitive nonclassical argumentation
   It is natural for a neural network to combine the weights representing multiple support or multiple attacks in order to compute the activation state of a neuron/argument. This is interesting in relation to the question of the accrual of arguments [38]. So far, our standard interpretation has been that any attack suffices to defeat an argument, unless a value-based function says otherwise. Another interpretation, however, is that arguments may get together to attack an argument (that is, only the conjunction of the arguments enables the attack; this is called a joint-attack [1]). Fig. 5(a) shows two arguments A and C attacking argument B. When either
   or
   for the standard interpretation, the attacks can be implemented by Algorithm 1 above. However, when arguments are allowed to accrue and either
   or
   , a decision has to be made as to whether or not A and C together can defeat B [10]. The same is true for support. Fig. 5(b) shows an argument A supporting argument B as done in ADFs. It may be that without A's support, B would be defeated, say, by an attack from another argument C, but B is not defeated with A's support. Finally, Fig. 5(c) shows a situation where, only if A and B prevail, can they attack C. Hidden neuron
   implements, in the usual way, a logical-AND. This is the situation where arguments A and B “get together” to attack C.

Download : Download full-size image
Fig. 5. Alternative modes of attack and support: (a) multiple attacks, (b) support, and (c) joint-attacks.

Let us consider Fig. 5(a) in more detail. Let
denote the weight from hidden neuron
to output neuron B. As said, the natural computation of a neural network will combine the weights into the input potential of B. Suppose that
and
. In this situation, neither A nor C defeat B, but, together, A and C may defeat B as an unintended consequence of the combination of the weights. To avoid this problem, we use the following convention: when arguments create a joint attack on another argument, the set-up of Fig. 5(c) is used. If n arguments are joined through hidden neuron
, then the bias
of
should satisfy the following inequality, which implements the intended logical-AND.

Let us now consider in more detail the situation where an argument receives multiple support (Fig. 5(b)). This situation is similar to that considered earlier, that is, whether some attack is strong enough to defeat all of the support received by B or whether the support for A overrides the attack, making it unsuccessful. The first case is straightforward and can be implemented by making:
where k is the number of supporting arguments, each with weight
.

The second case, in a more general set-up, takes a linearly ordered set
such that argument
prevails if it is not attacked, but
is defeated when attacked by
. However,
prevails again if it is supported by another argument
, but is defeated again if attacked by
, and so on. We need to assign values to weights
, as follows:
where
and ε is a small positive number such that
(typically
).

A further possibility would be to allow certain combinations of support to prevail and certain others to be defeated. This is, in fact, a likely outcome if a learning algorithm is to be applied, with the network's weights changing as a result of learning from data. Any combination of attack and support can be encoded in a neural network, as follows. The linear ordering above can be extended to multisets in the usual way, so that each
,
, denotes a set of arguments. The value of
will then correspond to the sum of the weights either attacking or supporting
, each weight being equal to
, where k is the cardinality of the set of arguments in question. Since we would like the combination of k arguments, and not of
or fewer arguments, to have the above effect, the following inequality should be satisfied:

The dual of the conjunctive attacks exemplified above are the disjunctive attacks shown in Fig. 6(a). In the figure, the activation of hidden neuron ∨ should attack either argument B or C according to some probability distribution. We say that neuron ∨ behaves stochastically. Having a standard hidden neuron in Fig. 6(a) would denote that A attacks both B and C. With a stochastic hidden neuron, either B or C is attacked with a probability. This offers a way of implementing the idea of a disjunctive attack, i.e. one that attacks either argument, but it does not matter which argument. If, for example, the probability of attacking argument B should be 50%, a random number is generated in the interval
and, if this number is greater than 0.5 then argument B is attacked; otherwise, argument C is attacked.

Download : Download full-size image
Fig. 6. Disjunctive attacks (a), self-defeating attacks (b), and attacks on attacks (c).

Let
denote the arguments (neurons) attacked stochastically through hidden neuron
. From the point of view of the network computation, for each
, we select a neuron
,
, at a time (at each round, a single
is chosen to receive activation from
). A stable state denoting a set
of potential prevailing arguments is then obtained in the usual way (with the same
being selected if the network recurs through
). We take
as our final set of prevailing arguments (i.e. following a skeptical semantics).

In addition to disjunctive attacks, the recent literature on argumentation has discussed extensively the modelling of self-defeating arguments as well as the concept of meta-argumentation [15], [25], [26]. Fig. 6(b) exemplifies the implementation of a self-defeating argument in a neural network. In the figure, argument A attacks argument B, but A is self-defeating, that is, the weight of the connection to output neuron A is negative. Hence, B prevails, as expected. Fig. 6(c) exemplifies a meta-level attack, i.e. an attack not on an argument, but on another attack. In Fig. 6(c), argument A does not attack argument B, but it attacks B's attack on C. As a result, C prevails. In this setting, it is possible for B to succeed in attacking another argument, say D, through a different hidden neuron. Notice the similarity between Fig. 6(c) and Fig. 5(c). Not surprisingly, the semantics of meta-level attacks can be characterised in terms of joint-attacks, by adding the meta-level attack itself as a node in the argumentation network [15].

Definition 12

Let
denote an attack from argument a on argument b. Suppose that argument c attacks this attack; we write it as
. A meta-argumentation network is an argumentation network extended with such attacks on attacks.

Lemma 13

(See [15].) Let
be a meta-argumentation network where
.
can be reduced to an argumentation network containing an extra node
such that
and
jointly attack
, and
attacks
.

Proposition 14

For any meta-argumentation network
there exists a neural network
such that
computes
.

Proof

We are concerned with the situation
without recursion. In the reduced network, node i attacks node jk, and nodes jk and j jointly attack node k. In the neural network (with the same structure as in Fig. 6(c)), we have
attacking k (recall that we have defined joint-attacks as conjunctions). If i is in then jk is out and k is in; if i is out then k is out iff j is in. In the neural network, the hidden neuron representing jk will be activated, attacking and defeating k iff
(or out) and
(or in), which clearly produces the same intended outcome.  □

5. Case study: Public prosecution charging decision
   In this section, we apply the neural cognitive argumentation framework to the modelling of a public prosecution charging decision, which was part of a real legal case. The modelling of the charging decision includes many of the argumentation aspects addressed by our proposed framework, notably uncertainty, joint-attacks, argument support and ADF-style reasoning. The results show that the neural model can be a useful tool in the analysis of legal decision making, including the analysis of what-if questions, as detailed below.

The following is an extract from a charging decision statement made 29 May 2012 by Alison Levitt, Queen's Counsel, Principal Legal Advisor to the Director of Public Prosecutions in relation to allegations that a police officer passed confidential information to a journalist about Operation Weeting, a police investigation into allegations of phone hacking by newspapers in the United Kingdom.

In what follows, we will represent the main aspects of the arguments discussed in the charging decision statement as a neural cognitive model, and analyse the possible model set-ups and computations, and their alternative conclusions in relation to the actual outcomes of this legal case study. We are interested in exemplifying the use of the proposed model in practice, and analysing its usefulness as a tool for modelling the different aspects of argumentation, including the analysis of what-if questions, as detailed below.

“On 2 April 2012 the Crown Prosecution Service received a file of evidence from the Metropolitan Police Service requesting charging advice in relation to two suspects. The first is a serving Metropolitan Police Officer in the Operation Weeting team whose name is not in the public domain. He is currently suspended. The second suspect is Amelia Hill, a journalist who writes for The Guardian newspaper.

The allegation is that the police officer passed confidential information about phone hacking cases to the journalist.

All the evidence has now carefully been considered and I have decided that neither the police officer nor the journalist should face a prosecution. The following paragraphs explain the reasons for my decision.

The suspects have been considered separately, as different considerations arise in relation to each of them.

Between 4 April 2011 and 18 August 2011, Ms. Hill wrote ten articles which were published in The Guardian. I am satisfied that there is sufficient evidence to establish that these articles contained confidential information derived from Operation Weeting, including the names of those who had been arrested. I am also satisfied that there is sufficient evidence to establish that the police officer disclosed that information to Ms Hill.

I have concluded that there is insufficient evidence against either suspect to provide a realistic prospect of conviction for the common law offence of misconduct in a public office or conspiracy to commit misconduct in a public office.

In this case, there is no evidence that the police officer was paid any money for the information he provided.

Moreover, the information disclosed by the police officer, although confidential, was not highly sensitive. It did not expose anyone to a risk of injury or death. It did not compromise the investigation. And the information in question would probably have made it into the public domain by some other means, albeit at some later stage.

In those circumstances, I have concluded that there is no realistic prospect of a conviction in the police officer's case because his alleged conduct is not capable of reaching the high threshold necessary to make out the criminal offence of misconduct in public office. It follows that there is equally no realistic prospect of a conviction against Ms. Hill for aiding and abetting the police officer's conduct.

However, the information disclosed was personal data within the meaning of the Data Protection Act 1998 and I am satisfied that there is arguably sufficient evidence to charge both the police officer and Ms. Hill with offences under section 55 of that Act, even when the available defences are taken into account.

I have therefore gone on to consider whether a prosecution is required in the public interest. There are finely balanced arguments tending both in favour of and against prosecution.

Journalists and those who interact with them have no special status under the law and thus the public interest factors have to be considered on a case by case basis in the same way as any other. However, in cases affecting the media, the DPP's Interim Guidelines require prosecutors to consider whether the public interest served by the conduct in question outweighs the overall criminality alleged.

So far as Ms Hill is concerned, the public interest served by her alleged conduct was that she was working with other journalists on a series of articles which, taken together, were capable of disclosing the commission of criminal offences, were intended to hold others to account, including the Metropolitan Police Service and the Crown Prosecution Service, and were capable of raising and contributing to an important matter of public debate, namely the nature and extent of the influence of the media. The alleged overall criminality is the breach of the Data Protection Act, but, as already noted, any damage caused by Ms. Hill's alleged disclosure was minimal. In the circumstances, I have decided that in her case, the public interest outweighs the overall criminality alleged.

Different considerations apply to the police officer. As a serving police officer, any claim that there is a public interest in his alleged conduct carries considerably less weight than that of Ms Hill. However, there are other important factors tending against prosecution, including as already noted, the fact that no payment was sought or received, and that the disclosure did not compromise the investigation. Moreover, disclosing the identity of those who are arrested is not, of itself, a criminal offence. It is only unlawful in this case because the disclosure also breached the Data Protection Act.

In the circumstances, I have decided that a criminal prosecution is not needed against either Ms. Hill or the police officer.

However, in light of my conclusion that there is sufficient evidence to provide a realistic prospect of convicting the police officer for an offence under the Data Protection Act, I have written to the Metropolitan Police Service and to the IPCC recommending that they consider bringing disciplinary proceedings against him.” Alison Levitt QC

Let us start by considering the arguments for and against prosecuting the journalist (pj) and the police officer (pp).

Arguments for prosecution:
A:
The articles contained confidential information;

B:
The police officer disclosed the information;

C:
A prosecution is required in the public interest.

Arguments against prosecution:
D:
There is no evidence that the police officer was paid any money;

E:
Information disclosed by the police officer, although confidential, was not highly sensitive;

F:
It did not expose anyone to a risk of injury or death;

G:
It did not compromise the investigation;

H:
It would probably have made it into the public domain by some other means;

I:
The public interest outweighs the overall criminality alleged;

J:
Together, the articles would expose the commission of criminal offences;

K:
Together, the articles would hold others to account;

L:
The articles contributed to an important matter of public debate, namely the nature and extent of the influence of the media.

Let us also analyse more closely the arguments relating to whether a prosecution is required in the public interest. The assumption is that both the journalist and the police officer have violated the Data Protection Act.

Arguments for bringing charges under the Data Protection Act:
M:
The information disclosed was personal data;

Arguments against bringing charges under the Data Protection Act:
N:
Disclosing the identity of those who are arrested is not, of itself, a criminal offence;

O:
It is only unlawful in this case because the disclosure also breached the Data Protection Act.

Remark 15

Notice how argument O was used rhetorically as part of an argument against bringing charges under the Data Protection Act. Argument O is, in fact, simply stating that the disclosure was unlawful because it breached the Data Protection Act. Consider also how the sentence below is used as part of the argumentation: “The alleged overall criminality is the breach of the Data Protection Act, but, as already noted, any damage caused by Ms. Hill's alleged disclosure was minimal”. Our model will help quantify such damage and will require a definition of minimal, as will become clear. Similarly, the following sentences provide clues as to the weights to be assigned to the neural network, in relation to the police officer: “Any claim that there is a public interest in his alleged conduct carries considerably less weight” and “There is a high threshold to make out the criminal offence of misconduct in public office”. We shall return to these once we have created the network model.

The QC's conclusion can be summarized as follows:
(a)
It was decided that a criminal prosecution is not needed;

(b)
It was decided that in the case of the journalist, the public interest outweighs the overall criminality alleged;

(c)
There is sufficient evidence to provide a realistic prospect of convicting the police officer for an offence under the Data Protection Act;

(d)
A recommendation was made to the police to consider bringing disciplinary proceedings against the police officer.

Our model's conclusion: Our model is concerned with making explicit the following relations (items 1 to 4 below).

On the issue of the police officer's misconduct:

1. Do the weights of the arguments exceed the high threshold for the offence of misconduct?

Arguments A, B and C should support the prosecution of the police officer (pp). Argument C should do so with a low weight (w) since the prosecution would be for misconduct in public office. Arguments D, F, G and H attack pp collectively (argument D with a high weight (W) for obvious reasons, and argument H with a low weight due to its speculative nature). Argument E attacks argument B. These are all the relevant arguments in relation to pp, as shown in Fig. 7, where dashed lines indicate attacks. As a result, neuron B fails to activate, and the weight of the arguments that collectively attack neuron pp should overcome the weight of the arguments that support pp. Hence, neuron pp should fail to activate. We discuss neuron/argument pj next.

Download : Download full-size image
Fig. 7. Neural implementation of legal case: prosecution decision.

2. If the police officer should not be prosecuted for misconduct then the journalist should not be prosecuted for aiding his conduct.

Item 2 above can be modelled in Fig. 7 using ADFs simply by stating that if argument pp is out (represented by a negative weight from input neuron pp to the hidden layer) then argument ¬pj should be in; hence, the journalist should not be prosecuted (in the neural network, if input neuron pp is not activated then output neuron ¬pj will be activated; see dashed line representing a negative weight from input neuron pp in Fig. 7). This separation between arguments pj and ¬pj allows one to ignore how arguments would influence neuron pj during the modelling of neuron ¬pj, and is referred to explicit negation in logic programming [19]. Thus, in case neuron pp fails to activate, which is the case here, neuron ¬pj will be activated. This completes the prosecution's analysis on the basis on misconduct.

On the issue of the violation of the Data Protection Act:

3. Do the weights of the arguments show that the public interest outweighs the violation of the Data Protection Act?

It is clear that arguments J, K and L support argument I, while argument M attacks I. In addition, argument N attacks M, and O attacks N, as shown in Fig. 8. The conclusion is that indeed argument I should prevail.

Download : Download full-size image
Fig. 8. Neural implementation of legal case: decision based on data protection act.

4. Different weights apply to the journalist and the police officer.

Argument M supports the argument that the journalist should be prosecuted for violation of the Data Protection Act (let us call this
). It also supports the argument for prosecuting the police officer for a violation of the Data Protection Act (call it
). The attacks from argument I on
and
should have different weights: a high (negative) weight W for
and a low (negative) weight (w) for
. Our model's conclusion, therefore, is that
should not prevail (i.e. the journalist should not be prosecuted), but differently from the QC's conclusion, if the value of the weight ω connecting argument M to
should be greater than the absolute value of w then argument
should prevail, i.e. the police officer would be prosecuted for violating the Data Protection Act. The actual values of weights ω and w may be a matter for debate, but perhaps the QC's arguments should have focused more on providing a justification for such values.

The above modelling exercise with the use of a neural cognitive model, may also help in the separation of concerns and systematic questioning of some of the assumptions made. For example, in this case study, some questions that emerge include: should different weights really apply to the journalist and the police officer, assuming they had to work as a team in the public interest? Aside from possible issues of remit, why should prosecution not be recommended straight away for the violation of the data protection act, and disciplinary proceedings should be evoked and recommended instead? We believe that our model should help prompt the user to ask such questions, organise the relationships among the different arguments under consideration, and investigate the impact of different weight assignments to the network model. For example, what if the weights W and w are assigned the same value? What if the weights ω and w are assigned the same absolute value? The user would then be able to run the model and consider the possible outcomes by analysing the different sets of prevailing arguments obtained as stable states of the neural network.

6. Conclusion and future work
   We have presented a neural cognitive model of argumentation that is capable of capturing a range of argumentation semantics and situations including joint-attacks, argument support, ordered attacks, disjunctive attacks, meta-level attacks and self-defeating attacks. All these different modes of argumentation can be modelled, learned and computed by means of a connectionist representation. In its most general form, arguments are weighted according to their strength, can support or attack other arguments directly, but can also combine conjunctively or disjunctively, sequentially or in parallel, at object- or meta-level, as exemplified throughout the paper. We have shown that all these different modes of argumentation can be represented and computed in a natural way by a connectionist network. This also indicates that the connectionist approach can offer an adequate tool for argument computation.

When dealing with uncertainty and meta-level preferences, in [14], the question of where the weights would come from is raised. In [2], voting by an audience is evoked as a solution that depends on meta-level considerations, as in [26]. With the framework proposed in this paper, the question gains a new dimension in that, as with any neural network, the weights can be learned from examples (i.e. instances from previous cases). As future work, we plan to explore the framework's learning capacity as part of a larger case study.

Uncertainty is intrinsic in human argumentation, yet most logic-based models of argumentation do not deal with uncertainty explicitly. Argumentation can be seen as a method for reducing one's uncertainty with the prevailing arguments being precisely those that are less-uncertain. In line with [21], [23], [35], the neural cognitive model introduced here lends itself well to this idea due to the use of weights and activation intervals as part of a neural network. However, we believe that the neural cognitive approach may also be advantageous from a purely computational perspective due to the networks' ability to adapt through learning and to compute prevailing arguments in parallel. All of the above is important given the objective of developing models of human argumentation. Future work includes, in addition to the evaluation of the framework's learning capacities, further experimentation on legal reasoning, a comparison of the framework's knowledge representation and learning capacity, e.g. in contrast with [23], [32], and the evaluation of the framework's parallel computation gains. A graphical interface is being developed to facilitate the interactive drawing and running of the networks as shown in the figures above, with all the features introduced here. We believe that such an interface can be useful as a tool for modelling, running and the elaboration of argumentation frameworks in a range of application areas.

[vtempest]

https://aclanthology.org/N16-1007.pdf

Neural Network-Based Abstract Generation for Opinions and Arguments
Wang Ling
Google DeepMind
London, N1 0AE
Proceedings of NAACL-HLT 2016, pages 47–57,
San Diego, California, June 12-17, 2016.
c 2016

We study the problem of generating abstractive summaries for opinionated text. We propose an attention-based neural network model
that is able to absorb information from multiple text units to construct informative, concise,
and fluent summaries. An importance-based
sampling method is designed to allow the encoder to integrate information from an important subset of input. Automatic evaluation indicates that our system outperforms state-ofthe-art abstractive and extractive summarization systems on two newly collected datasets
of movie reviews and arguments. Our system
summaries are also rated as more informative
and grammatical in human evaluation.

[vtempest]

https://www.sciencedirect.com/science/article/pii/S1570868313000748?via%3Dihub

Journal of Applied Logic

Volume 12, Issue 2, June 2014, Pages 109-127
Journal of Applied Logic
A neural cognitive model of argumentation with application to legal inference and decision making
Author links open overlay panelArtur S. d'Avila Garcez a, Dov M. Gabbay b, Luis C. Lamb c
https://doi.org/10.1016/j.jal.2013.08.004

Abstract
Formal models of argumentation have been investigated in several areas, from multi-agent systems and artificial intelligence (AI) to decision making, philosophy and law. In artificial intelligence, logic-based models have been the standard for the representation of argumentative reasoning. More recently, the standard logic-based models have been shown equivalent to standard connectionist models. This has created a new line of research where (i) neural networks can be used as a parallel computational model for argumentation and (ii) neural networks can be used to combine argumentation, quantitative reasoning and statistical learning. At the same time, non-standard logic models of argumentation started to emerge. In this paper, we propose a connectionist cognitive model of argumentation that accounts for both standard and non-standard forms of argumentation. The model is shown to be an adequate framework for dealing with standard and non-standard argumentation, including joint-attacks, argument support, ordered attacks, disjunctive attacks, meta-level attacks, self-defeating attacks, argument accrual and uncertainty. We show that the neural cognitive approach offers an adequate way of modelling all of these different aspects of argumentation. We have applied the framework to the modelling of a public prosecution charging decision as part of a real legal decision making case study containing many of the above aspects of argumentation. The results show that the model can be a useful tool in the analysis of legal decision making, including the analysis of what-if questions and the analysis of alternative conclusions. The approach opens up two new perspectives in the short-term: the use of neural networks for computing prevailing arguments efficiently through the propagation in parallel of neuronal activations, and the use of the same networks to evolve the structure of the argumentation network through learning (e.g. to learn the strength of arguments from data).

Previous article in issueNext article in issue
Keywords
ArgumentationNeural-symbolic reasoningLegal decision makingCognitive modelling

1. Introduction
   Formal models of argumentation have been investigated in several areas, from multi-agent systems and artificial intelligence (AI) to decision making, philosophy and law [4], [8], [12], [17], [30], [33]. In artificial intelligence, models of argumentation have been used for commonsense reasoning, modelling chains of defeasible arguments to reach a conclusion. Such models are mainly founded on logic-based approaches, which have been the standard for the representation of argumentative reasoning in AI [3].

Recent efforts to bridge the gap between logic-based models of argumentation and cognitive models of computation include [10], [11], [34]. In [10], [11], an equivalence is shown between value-based argumentation [2] and standard connectionist networks [22]. This has created a new line of research in argumentation where (i) neural networks can be used as a cognitive computational model for argumentation and (ii) neural networks can be used to combine argumentation, quantitative reasoning and statistical learning. In [34], behavioural data is used to conclude that in human reasoning, reinstatement does not yield a full recovery of the attacked argument; [1] implements the same idea mathematically through equations that resemble the predator–prey dynamics of species populations. Further work integrating logic and neural networks include [20] where clustering in fuzzy ART networks is used to compute prevailing arguments, and [25] which extends the work in [10] to deal with self-defeating arguments and provides a number of interesting examples. At the same time, some non-standard models of argumentation start to emerge, enriching current models with cognitive abilities; e.g. [15] discusses meta-level attacks, coalitions, disjunctive attacks and argument support, [37] provides an adequate semantics for joint attacks, among much else, [29] seeks to unravel the role of emotions in argumentation, [14], [23] propose to handle uncertainty in argumentation through the assignment of probabilities and weights to arguments, and [13], [26] offer a qualitative method for reasoning about uncertainty and preferences between arguments.

We argue that the adoption of a cognitive approach to argumentation can offer an adequate framework for dealing with both standard and non-standard argumentation models. In this paper, we show that a cognitive approach can model many different aspects of argumentation in a uniform way, in particular, modelling uncertainty in argumentative reasoning and the accrual of arguments. The approach opens up, through the use of a connectionist system, two new short-term perspectives: (i) the use of neural networks to compute prevailing arguments efficiently through the propagation in parallel of neuronal activation signals and (ii) the use of the same networks to evolve the structure of an argumentation network through learning (e.g. to learn the strength of arguments from data). We believe that this approach also opens a more long-term perspective for the research on argumentation: the use of connectionist models of computation to help investigate and evaluate cognitive models of argumentation. For example, ideas from connectionism about the modelling of attention and emotion could be investigated in the context of argumentation [29], [36].

Argumentation has also been proposed as a method for helping machine learning systems [27] where an expert's arguments, or the reasons for some of the learning examples, are used to guide the search for hypotheses. This is related to the body of work on abductive reasoning and combinations of abduction and inductive logic programming [24], [28]. It is said that the arguments constrain the combinatorial search among possible hypotheses, directing the search towards hypotheses that are more comprehensible in the light of an expert's background knowledge [27]. We subscribe to this idea. In fact, experimental results on the integration of learning with background knowledge using neural networks have been shown to outperform symbolic and purely-connectionist systems, especially in the presence of noisy data [9]. In this paper, differently from [27], however, learning from data can be used to inform a process of numerical argumentation, allowing different perspectives of human argumentation, including joint attacks, argument support, meta-argumentation and disjunctive attacks, to be modelled in the same framework, as detailed in what follows.

The remainder of the paper is organised as follows. First, we define the concepts of argumentation and neural cognitive models used throughout the paper. Then, we present an algorithm, generalised from [10], which translates standard and non-standard argumentation frameworks into standard connectionist networks. We show that the resulting neural model executes a sound parallel computation of the prevailing arguments according to a number of standard argumentation semantics, and also according to value-based argumentation models [2], abstract dialectical frameworks [5], and other forms of human argumentation. We illustrate the network computation through examples that include joint attacks, support, meta-argumentation and disjunctive attacks. Finally, we apply the framework to a real decision making situation in legal reasoning, which indicates that the network model can be a useful tool in the modelling of non-standard and numerical argumentation, and in the analysis of what-if questions that emerge in real situations. The paper concludes with a brief discussion and directions for future work.

2. Background
   In this section, we present the concepts of argumentation and neural networks used throughout the paper.

Definition 1

An argumentation framework has the form
, where α is a set of arguments, and
is a relation indicating which arguments attack which other arguments.

In order to record the values associated with arguments, in [2] Bench-Capon has extended Dung's argumentation framework [12] by adding to it a set of values and a function mapping arguments to values. This brings argumentation closer to a numerical, connectionist approach.

Definition 2

A value-based argumentation framework is a 5-tuple
, where α is a finite set of arguments, attacks is an irreflexive binary relation on α, V is a non-empty set of values, val is a function mapping elements in α to elements in V, and P is a set of possible audiences, where we may have as many audiences as there are orderings on V. For every
,
.

Bench-Capon also defines the notions of objective and subjective acceptability of arguments. The first are arguments acceptable no matter the choice of preferred values for every audience, whereas the second are acceptable to some audiences. Arguments which are neither objectively nor subjectively acceptable are called indefensible. A function v from attack to
gives the relative strength of an argument. Given
, if
then
is said to be stronger than
. Otherwise, if
then
is weaker than
.

We shall also relate the neural approach with abstract dialectical frameworks [5].

Definition 3

An abstract dialectical framework (ADF) is a tuple
where S is a set of nodes,
is a set of links,
is a set of total functions
, one for each node s.

Consider an example. A person is innocent (i), unless she is a murderer (m). A killer (k) is a murderer (m), unless she acted in self-defence (s). There must be evidence for self-defence, e.g. a witness (w) who is not known to be a liar (l). An ADF can model the above example by stating that s is in if w is in and l is out. Similarly, m is in if k is in and s is out. In other words, like neural networks, ADFs include the concept of support. If k and w are in and l is out then s will be in; s then defeats m regardless of k so that i prevails. Every argumentation framework has an associated ADF. Also, normal logic programs have associated ADFs [5]. Since every logic program also has an associated neural network [9], this fact will be used later to show the required correspondences.

Other established definitions of argumentation semantics will be considered as well [6], [7], [12]. In all the definitions that follow, an argumentation framework is a pair
, as above. First, let us define a function
, such that
is defended by α}, where
. The function F computes the arguments accepted in the sense of [12] or defended by a set of arguments, in the sense of [7]. In this way, we can define conflict-free sets of arguments. A set of arguments is conflict-free if and only if (iff, for short) it does not contain any arguments A and B such that A defeats B. Let Args be a conflict-free set of arguments. Args is said to be a complete extension iff
.

In another useful argumentation semantics, the grounded semantics, only one extension is yielded by making use of the function F and defining the grounded extension as the minimal fixed point of F [31]. A grounded extension is conflict-free [7]. In the preferred semantics [12], a more credulous approach is used, which maximizes the number of accepted arguments. In order to define preferred semantics, Dung introduces the notion of admissible sets of arguments. A set of arguments is admissible iff it is conflict-free and
. The set Args is a preferred extension iff Args is maximal w.r.t. set inclusion. In the stable semantics of argumentation [18], a set of arguments is called stable iff it defeats each argument that does not belong to this set. In semi-stable semantics, a set of arguments Args is semi-stable iff Args is a complete extension of which
is maximal and defines a set of arguments which are defeated by an argument in Args.

In order to illustrate the different argumentation semantics, we borrow an example from [7]. Consider the abstract argumentation framework depicted as a directed graph in Fig. 1, where each node is an argument and an arrow from argument X to argument Y denotes an attack from X on Y. This framework has grounded extension {E, F}; complete extensions {E, F}, {B, C, E, F} and {A, E, F}; preferred extensions {B, C, E, F} and {A, E, F}; stable extension {B, C, E, F}; and semi-stable extension {B, C, E, F}. As pointed out in [7], semi-stable and stable extensions will coincide whenever the framework has at least one stable extension.

Download : Download full-size image
Fig. 1. An abstract argumentation framework with arguments A, B, C, D, E, and F; D is a self-defeating argument.

Finally, we shall also consider meta-argumentation [15]. In a meta-argumentation network, let argument a attack b in the usual way. It is possible to define an argument c as an attack on a's attack. This makes the framework more fine-grained in that c's attack does not propagate throughout the network, but is targeted at one specific attack in the network. Meta-argumentation can be reduced to argumentation frameworks with the addition of a node denoting
and a careful re-organization of the network [15].

We shall use a standard definition of neural networks, as follows. A neural network is a directed graph with the following structure: a unit (or neuron) in the graph is characterised, at time t, by its input vector
, its input potential
, its activation state
, and its output
. The units of the network are interconnected via a set of directed and weighted connections such that if there is a connection from unit i to unit j then
denotes the weight of this connection. The input potential of neuron i at time t (
) is obtained by computing a weighted sum for neuron i such that
. The activation state
of neuron i at time t is then given by the neuron's activation function
such that
. Typically,
is either a linear function, a non-linear step function, or a sigmoid function such as
. In this paper, we use
as activation function and inputs values in the range [
]. In addition,
(an extra weight with input always fixed at 1) is known as the bias of neuron i. We say that neuron i is active at time t if
. Finally, the neuron's output value
is given by its output function
. Usually,
is the identity function so that
.

3. Neural cognitive argumentation frameworks
   We start by considering the relationship between argumentation and neural networks informally. If we represent an argument by a neuron then a connection from neuron i to neuron j can indicate that argument i either attacks or supports argument j, the weight of the connection corresponding to the strength of the attack or support. Since real numbers are used as weights in a neural network, we associate negative weights with attacks, positive weights with support, and zero weight with the lack of an attack or support.

Definition 4

We say that an argument prevails at time t when the activation state of its associated neuron is greater than a predefined value
at time t,
. We say that an argument is defeated at time t when the neuron's activation is smaller than
. Otherwise, i.e. for activations in the interval [
], we say that it is unknown whether an argument prevails or not.

There are different ways in which an argument may support other arguments. For example, an argument i may support argument j by attacking an argument k that attacks j, or argument i may support j directly, e.g. by strengthening the value of j, or even i and j may get together to attack k. Generally, an argument i supports an argument j if the coordination of i and j reduces the likelihood of j being defeated [39]. A general neural network structure capable of accounting for the above combinations of attack and support and the necessary computations of prevailing arguments would be a recurrent network having an input layer, a single hidden layer, and an output layer with feedback from the output to the input layer [9]. At time
, input values are provided to the network. Neuronal activation is then propagated in parallel from the input to the hidden layer at time
, and to the output of the network at time
. At time
, the output values can be fed back to the input of the network, and this process can be repeated until a stable state is obtained, when input and output values will be the same for each pair of neurons with feedback. Arguments for which the associated neuron is activated at the stable state are said to prevail.

Consider the neural network of Fig. 2(b), which implements the argumentation network of Fig. 2(a). Arguments A, B and C are encoded in the network's input and output layers. In this example, arguments do not get together to attack another argument so that each input neuron is connected directly to a hidden neuron and the weights from the input to the hidden layer simply serve to send the input information forward. Support and attack information is encoded by the weights leading from the hidden to the output layer of the network. Support for A is encoded by the positive weight going from neuron
to output neuron A. Similarly, support for B (resp. C) is encoded by the weight going from
(resp.
) to B (resp. C). A's attack on B is represented by the dashed arrow going from
to B, which should have negative weight, as specified in the algorithm below. Similarly, B's attack on C is represented by the dashed arrow going from
to C, with a negative weight.

Download : Download full-size image
Fig. 2. (a) Illustration of an argumentation network in which argument A attacks argument B, which in turn attacks argument C, and (b) its neural implementation, where solid lines receive positive weights and dashed lines, which represent the attacks, receive negative weights.

If the absolute value of the weight going from
to B (call it
) is larger than the value of the weight from
to B (call it
) then A defeats B. This produces {
} as prevailing arguments and is identical to the usual (non-value based) interpretation of argumentation frameworks [12]. The prevailing arguments are computed by the network as follows: suppose that A, B and C are all present at time
, denoted by input vector [
]. Hidden neurons
,
and
all become activated;
activates A, and blocks the activation of B from
(because
). At the same time, C is blocked by
(by default, we assume that attacks are stronger than support, i.e. the weight from
to C is greater in absolute value than the weight from
to C, unless stated otherwise). Thus, at time
, only output neuron A is activated (i.e. only the output of neuron A is greater than
). This is then represented as a new input vector [
] at time
. Given this new input, A continues to prevail, B is defeated as before, but C is reinstated because B now fails to defeat it, since B is not present in the input anymore. Thus, at time
, output neurons A and C become activated. At time
, a new input [
] is produced. Finally, with [
] given as input, output neurons A and C become activated again, producing a stable state in the network. Recall that network outputs are real values in the range (
). To compute a stable state, each output value in the range (
) is mapped to 1, output values in the range (
) are mapped to −1, and values in the range [
] are mapped to 0. After this is done, the network's new input vector can be compared with its previous input. In this example, the input vector at time
will be identical to the input vector at time
, that is [
] is a stable state. The network's stable activations indicate the prevailing arguments {
}. This result also coincides with the standard ADF interpretation of support (Definition 3). In the case of VAF (Definition 2), if B is preferred over A, all that needs changing in the network is the value of
or
so that
, denoting that the attack from A on B should not be strong sufficiently to defeat B [10]. In this case, input [
] produces new input [
], which is a stable state, given the neural network's new set of weights. This new network, therefore, computes prevailing arguments {
}, as expected in the case of the VAF under consideration.

As another example, consider the network of Fig. 3(b). Here, a cycle exists in the argumentation network. This may create an infinite loop in the computation of the stable state of the associated neural network; e.g. if the network were to be started on input vector [
], it would oscillate between that state and state [
] indefinitely, with the arguments all being attacked in parallel and defeated in a single pass through the network, and then reinstated in the next pass through the network. In order to handle this as intended by the usual argumentation semantics, whenever the neural network reaches a state [
], the computation stops. In this case, the neural network computes the grounded extension of the argumentation network, as discussed in more detail later. Summarising, our policy is that the network computation stops when either a stable state is obtained or when [
] is reached. We call [
] a terminal state. Notice that by following a value-based approach, and changing the weights according to some preference relation, one might eliminate the loop [10]. For example, if as before, B is preferred over A then the attack from A on B will not be successful, with input [
] producing stable state [
]. When the weights are different, the neural network is less likely to enter into an infinite loop (see [10] for a discussion on how weight learning given new information following a value-based approach can be useful at resolving loops).

Download : Download full-size image
Fig. 3. (a) An example of a cyclic argumentation network and (b) its neural implementation whose parallel computation produces, in a single pass through the network, a terminal state [−1,−1,−1] with no prevailing argument, following a grounded argumentation semantics.

The algorithm below generalises the algorithm first introduced in [10], which was made for VAF only. It translates argumentation frameworks into single-hidden layer neural networks that behave as exemplified, and can be used to compute prevailing arguments in parallel. It does so by defining the structure and set of weights of a neural network as a function of
, using activation function
and inputs in {
}. Each argument has a strength given by positive weight
. Certain arguments may attack other arguments with negative weight
, and certain arguments may support other arguments with positive weight
. The weights are such that activations in the interval [
] are guaranteed not to occur for now (we shall consider uncertainty in the next section). The neural network produces outputs in the intervals (
), which is mapped to false and denotes that an argument is defeated, and (
), which is mapped to true, denoting that an argument prevails.

By default, Algorithm 1 implements Dung's argumentation frameworks. If an argument
attacks an argument
, and
is itself not attacked, then neuron
should block neuron
. However, if
is deemed weaker than
, and no other argument attacks
, then neuron
should not block neuron
. To achieve this, and account for a number of other, alternative semantics and modes of argumentation in a neural network [16], the constraints on
and
can be modified, as will become clearer in Section 4. Also, in Algorithm 1, a single supporting argument is deemed sufficient to defeat any attack; the other alternatives, leading to different constraints on
and
, will be considered in Section 4. A novelty of the algorithm is that arguments may have strengths
, which may vary from one attack to another. Before we consider the extensions of Algorithm 1, however, let us make the ideas developed so far more precise.

Download : Download full-size image
Algorithm 1. General neural argumentation algorithm.

Definition 5

Let
. Let
denote the activation of neuron α at time t. We say that a neural network
computes an argumentation framework
if whenever
and
then
, and whenever
for every
such that
then
(reinstatement).

Proposition 6

For each argumentation framework
there exists a single-hidden layer neural network
such that
computes
.

Proof

First, we show that if
in the input layer of
then
in the output layer of
whenever there are no attacks on
. In the worst case, the input potential of hidden neuron
is
, and the output of
is
. We want
. Again in the worst case, the input potential of output neuron
will be
, and we need
. As a result,
needs to be satisfied. When there is an attack on
, the activation of output neuron
needs to be smaller than
if hidden neuron
is active. In the worst case,
has activation
,
has activation 1, and any other attacking neuron has activation −1. Hence,
has to be satisfied, where n is the number of attacks. Dually, when there are no attacks on
, the following inequality has to be satisfied:
, which gives
and the constraint on
, as shown in Algorithm 1, step 4. When at least one argument
supports argument
, the following inequality has to be satisfied (again, in the worst case analysis):
. Dually, in the worst case,
has to be satisfied, which gives
and the constraint on
, as shown in Algorithm 1, step 5. This completes the proof. □

Proposition 7

For each ADF
there exists a single-hidden layer neural network
such that
computes
.

Proof

The standard ADF interpretation is that
when
attacks
. This interpretation has been covered in the proof of Proposition 6. In weighted ADFs, however, one can distinguish different combinations of attack and support [5], in particular, a supporting link can be stronger than an attacking link so that the attacked argument prevails. Since neural networks with as few as a single hidden layer are universal approximators, it follows that in the neural argumentation approach, any Boolean combination of attacks and support can be computed. In the specific case of support, we need to show that if
supports
then if
then
. As before, we associate inputs in the interval
to in and inputs in the interval
to out. In the worst case, we need
. Thus,
, as guaranteed by the algorithm. This completes the proof. Notice that, in practice, we convert inputs in the interval
to 1, and inputs in the interval
to −1, which relaxes further the above constraint on
.  □

We can also show that the neural-network approach is very general by proving that it computes the many different argumentation semantics, as defined earlier [6], [12]. Before that is done, however, we should note that an argument that is not attacked by any other argument cannot be reinstated in the neural network (an argument that is attacked but not defeated will be reinstated as usual). For example, argument E in Fig. 1 will be in if its input value is 1, and will be out if its input value is −1. It will continue to be out over time if it is out initially, and will be in otherwise. Argument B, on the other hand, will be reinstated whenever argument A is not present, and vice-versa.

Lemma 8

For each argumentation framework
there exists a single-hidden-layer recurrent neural network
such that the complete extensions of
are stable states of
.

Proof

Recall that a set of arguments Args is said to be a complete extension of
iff
. From Proposition 6, one pass through
computes
. Recurrently connected,
iterates
until a stable state is reached, which is a fixed point of F, that is
.  □

Corollary 9

For each argumentation framework
there exists a recurrent neural network
such that the grounded, preferred, stable and semi-stable extensions of
are stable states of
.

As an example, consider again the argumentation framework of Fig. 1. Starting from
, without a terminal state, the associated neural network can converge to the following stable states: {B, C, E, F} and {A, E, F}, corresponding to its preferred extensions. With a terminal state, the network will converge to either {E, F}, {B, C, E, F} or {A, E, F}, corresponding to its complete extensions.

Finally, consider the argumentation network of Fig. 4, for which no stable state exists. With the use of a terminal state, the corresponding neural network will implement a semi-stable semantics, providing the empty set as the only extension. Without a terminal state, the neural network will loop (until it is either halted or its weights are changed), implementing a stable semantics.

Download : Download full-size image
Fig. 4. Semantic circularity for which no stable state exists.

The neural networks of Fig. 2, Fig. 3 should be seen as computational models rather than abstract argumentation frameworks. As discussed, the networks can be used in the parallel computation of prevailing arguments: given input vector [
] at the start, the networks should always converge to a stable state corresponding to a set of prevailing arguments, according to a (value-based, preferred, etc.) argumentation semantics.

Definition 10

We say that neural network
computes an (extension-based semantics of) argumentation framework
if every stable state of
is an extension of
.

Proposition 11

If
computes
and
admits a single extension then starting from input vector
,
always finds a stable state corresponding to the prevailing arguments of
.

Proof

The arguments in
can be viewed as logic programming clauses of the form
. Starting from [
],
will treat each
as a fact unless
is attacked and defeated by another argument
, which corresponds to adding a clause
to the program (where ¬ stands for negation by failure). If
admits a single extension then the corresponding program will have a single model. The stable models of any logic program can be computed by a single-hidden layer neural network; with a single model, the network is known to always settle down in a stable state corresponding to this model, as proved in [9]. Such a result can be applied directly here to complete the proof.  □

As exemplified earlier, it should be possible to extend Proposition 11 to certain very general classes of argumentation networks that admit multiple extensions. Due to the numerical nature of the networks, they are unlikely to oscillate unless integer weights with the same absolute values are used. For example, we have seen that, starting from [
], the network of Fig. 3 converges to a terminal state. Otherwise, it loops. As argued in [10], a network that loops should be seen as an opportunity for learning, whereby what-if questions can be considered and the network's weights can be changed slightly. Similarly, when a network reaches a terminal state, this should trigger a search for new evidence about the relative strength of the arguments. For example, consider the case where arguments A and B attack each other in a cycle. The corresponding neural network has two stable states, namely [
] and [
], corresponding to the two stable extensions of the argumentation network. Starting from [
], the neural network produces output [
], which is a terminal state. At this point, the computation stops. However, a loop exists in the network computation, since input [
] would produce output [
]. A terminal state does not provide much information by itself (a terminal state could be seen as producing maximum uncertainty). However, if the reaching of a terminal state is seen as a trigger for a search for more evidence, perhaps this search could shed new light on the relative strengths of arguments A and B, defining a preference for either argument by changing the weights of the network. If a neural network is such that the weights do not have the same absolute values then it is unlikely that arguments will cancel each other, as seen above, so that the network loops. Breaking such a symmetry in the set of weights by changing their values slightly is the norm of network learning, and could be seen as an alternative to the use of terminal states and a solution to the problem of having loops in the computation of such neural networks [10].

4. Neural cognitive nonclassical argumentation
   It is natural for a neural network to combine the weights representing multiple support or multiple attacks in order to compute the activation state of a neuron/argument. This is interesting in relation to the question of the accrual of arguments [38]. So far, our standard interpretation has been that any attack suffices to defeat an argument, unless a value-based function says otherwise. Another interpretation, however, is that arguments may get together to attack an argument (that is, only the conjunction of the arguments enables the attack; this is called a joint-attack [1]). Fig. 5(a) shows two arguments A and C attacking argument B. When either
   or
   for the standard interpretation, the attacks can be implemented by Algorithm 1 above. However, when arguments are allowed to accrue and either
   or
   , a decision has to be made as to whether or not A and C together can defeat B [10]. The same is true for support. Fig. 5(b) shows an argument A supporting argument B as done in ADFs. It may be that without A's support, B would be defeated, say, by an attack from another argument C, but B is not defeated with A's support. Finally, Fig. 5(c) shows a situation where, only if A and B prevail, can they attack C. Hidden neuron
   implements, in the usual way, a logical-AND. This is the situation where arguments A and B “get together” to attack C.

Download : Download full-size image
Fig. 5. Alternative modes of attack and support: (a) multiple attacks, (b) support, and (c) joint-attacks.

Let us consider Fig. 5(a) in more detail. Let
denote the weight from hidden neuron
to output neuron B. As said, the natural computation of a neural network will combine the weights into the input potential of B. Suppose that
and
. In this situation, neither A nor C defeat B, but, together, A and C may defeat B as an unintended consequence of the combination of the weights. To avoid this problem, we use the following convention: when arguments create a joint attack on another argument, the set-up of Fig. 5(c) is used. If n arguments are joined through hidden neuron
, then the bias
of
should satisfy the following inequality, which implements the intended logical-AND.

Let us now consider in more detail the situation where an argument receives multiple support (Fig. 5(b)). This situation is similar to that considered earlier, that is, whether some attack is strong enough to defeat all of the support received by B or whether the support for A overrides the attack, making it unsuccessful. The first case is straightforward and can be implemented by making:
where k is the number of supporting arguments, each with weight
.

The second case, in a more general set-up, takes a linearly ordered set
such that argument
prevails if it is not attacked, but
is defeated when attacked by
. However,
prevails again if it is supported by another argument
, but is defeated again if attacked by
, and so on. We need to assign values to weights
, as follows:
where
and ε is a small positive number such that
(typically
).

A further possibility would be to allow certain combinations of support to prevail and certain others to be defeated. This is, in fact, a likely outcome if a learning algorithm is to be applied, with the network's weights changing as a result of learning from data. Any combination of attack and support can be encoded in a neural network, as follows. The linear ordering above can be extended to multisets in the usual way, so that each
,
, denotes a set of arguments. The value of
will then correspond to the sum of the weights either attacking or supporting
, each weight being equal to
, where k is the cardinality of the set of arguments in question. Since we would like the combination of k arguments, and not of
or fewer arguments, to have the above effect, the following inequality should be satisfied:

The dual of the conjunctive attacks exemplified above are the disjunctive attacks shown in Fig. 6(a). In the figure, the activation of hidden neuron ∨ should attack either argument B or C according to some probability distribution. We say that neuron ∨ behaves stochastically. Having a standard hidden neuron in Fig. 6(a) would denote that A attacks both B and C. With a stochastic hidden neuron, either B or C is attacked with a probability. This offers a way of implementing the idea of a disjunctive attack, i.e. one that attacks either argument, but it does not matter which argument. If, for example, the probability of attacking argument B should be 50%, a random number is generated in the interval
and, if this number is greater than 0.5 then argument B is attacked; otherwise, argument C is attacked.

Download : Download full-size image
Fig. 6. Disjunctive attacks (a), self-defeating attacks (b), and attacks on attacks (c).

Let
denote the arguments (neurons) attacked stochastically through hidden neuron
. From the point of view of the network computation, for each
, we select a neuron
,
, at a time (at each round, a single
is chosen to receive activation from
). A stable state denoting a set
of potential prevailing arguments is then obtained in the usual way (with the same
being selected if the network recurs through
). We take
as our final set of prevailing arguments (i.e. following a skeptical semantics).

In addition to disjunctive attacks, the recent literature on argumentation has discussed extensively the modelling of self-defeating arguments as well as the concept of meta-argumentation [15], [25], [26]. Fig. 6(b) exemplifies the implementation of a self-defeating argument in a neural network. In the figure, argument A attacks argument B, but A is self-defeating, that is, the weight of the connection to output neuron A is negative. Hence, B prevails, as expected. Fig. 6(c) exemplifies a meta-level attack, i.e. an attack not on an argument, but on another attack. In Fig. 6(c), argument A does not attack argument B, but it attacks B's attack on C. As a result, C prevails. In this setting, it is possible for B to succeed in attacking another argument, say D, through a different hidden neuron. Notice the similarity between Fig. 6(c) and Fig. 5(c). Not surprisingly, the semantics of meta-level attacks can be characterised in terms of joint-attacks, by adding the meta-level attack itself as a node in the argumentation network [15].

Definition 12

Let
denote an attack from argument a on argument b. Suppose that argument c attacks this attack; we write it as
. A meta-argumentation network is an argumentation network extended with such attacks on attacks.

Lemma 13

(See [15].) Let
be a meta-argumentation network where
.
can be reduced to an argumentation network containing an extra node
such that
and
jointly attack
, and
attacks
.

Proposition 14

For any meta-argumentation network
there exists a neural network
such that
computes
.

Proof

We are concerned with the situation
without recursion. In the reduced network, node i attacks node jk, and nodes jk and j jointly attack node k. In the neural network (with the same structure as in Fig. 6(c)), we have
attacking k (recall that we have defined joint-attacks as conjunctions). If i is in then jk is out and k is in; if i is out then k is out iff j is in. In the neural network, the hidden neuron representing jk will be activated, attacking and defeating k iff
(or out) and
(or in), which clearly produces the same intended outcome.  □

5. Case study: Public prosecution charging decision
   In this section, we apply the neural cognitive argumentation framework to the modelling of a public prosecution charging decision, which was part of a real legal case. The modelling of the charging decision includes many of the argumentation aspects addressed by our proposed framework, notably uncertainty, joint-attacks, argument support and ADF-style reasoning. The results show that the neural model can be a useful tool in the analysis of legal decision making, including the analysis of what-if questions, as detailed below.

The following is an extract from a charging decision statement made 29 May 2012 by Alison Levitt, Queen's Counsel, Principal Legal Advisor to the Director of Public Prosecutions in relation to allegations that a police officer passed confidential information to a journalist about Operation Weeting, a police investigation into allegations of phone hacking by newspapers in the United Kingdom.

In what follows, we will represent the main aspects of the arguments discussed in the charging decision statement as a neural cognitive model, and analyse the possible model set-ups and computations, and their alternative conclusions in relation to the actual outcomes of this legal case study. We are interested in exemplifying the use of the proposed model in practice, and analysing its usefulness as a tool for modelling the different aspects of argumentation, including the analysis of what-if questions, as detailed below.

“On 2 April 2012 the Crown Prosecution Service received a file of evidence from the Metropolitan Police Service requesting charging advice in relation to two suspects. The first is a serving Metropolitan Police Officer in the Operation Weeting team whose name is not in the public domain. He is currently suspended. The second suspect is Amelia Hill, a journalist who writes for The Guardian newspaper.

The allegation is that the police officer passed confidential information about phone hacking cases to the journalist.

All the evidence has now carefully been considered and I have decided that neither the police officer nor the journalist should face a prosecution. The following paragraphs explain the reasons for my decision.

The suspects have been considered separately, as different considerations arise in relation to each of them.

Between 4 April 2011 and 18 August 2011, Ms. Hill wrote ten articles which were published in The Guardian. I am satisfied that there is sufficient evidence to establish that these articles contained confidential information derived from Operation Weeting, including the names of those who had been arrested. I am also satisfied that there is sufficient evidence to establish that the police officer disclosed that information to Ms Hill.

I have concluded that there is insufficient evidence against either suspect to provide a realistic prospect of conviction for the common law offence of misconduct in a public office or conspiracy to commit misconduct in a public office.

In this case, there is no evidence that the police officer was paid any money for the information he provided.

Moreover, the information disclosed by the police officer, although confidential, was not highly sensitive. It did not expose anyone to a risk of injury or death. It did not compromise the investigation. And the information in question would probably have made it into the public domain by some other means, albeit at some later stage.

In those circumstances, I have concluded that there is no realistic prospect of a conviction in the police officer's case because his alleged conduct is not capable of reaching the high threshold necessary to make out the criminal offence of misconduct in public office. It follows that there is equally no realistic prospect of a conviction against Ms. Hill for aiding and abetting the police officer's conduct.

However, the information disclosed was personal data within the meaning of the Data Protection Act 1998 and I am satisfied that there is arguably sufficient evidence to charge both the police officer and Ms. Hill with offences under section 55 of that Act, even when the available defences are taken into account.

I have therefore gone on to consider whether a prosecution is required in the public interest. There are finely balanced arguments tending both in favour of and against prosecution.

Journalists and those who interact with them have no special status under the law and thus the public interest factors have to be considered on a case by case basis in the same way as any other. However, in cases affecting the media, the DPP's Interim Guidelines require prosecutors to consider whether the public interest served by the conduct in question outweighs the overall criminality alleged.

So far as Ms Hill is concerned, the public interest served by her alleged conduct was that she was working with other journalists on a series of articles which, taken together, were capable of disclosing the commission of criminal offences, were intended to hold others to account, including the Metropolitan Police Service and the Crown Prosecution Service, and were capable of raising and contributing to an important matter of public debate, namely the nature and extent of the influence of the media. The alleged overall criminality is the breach of the Data Protection Act, but, as already noted, any damage caused by Ms. Hill's alleged disclosure was minimal. In the circumstances, I have decided that in her case, the public interest outweighs the overall criminality alleged.

Different considerations apply to the police officer. As a serving police officer, any claim that there is a public interest in his alleged conduct carries considerably less weight than that of Ms Hill. However, there are other important factors tending against prosecution, including as already noted, the fact that no payment was sought or received, and that the disclosure did not compromise the investigation. Moreover, disclosing the identity of those who are arrested is not, of itself, a criminal offence. It is only unlawful in this case because the disclosure also breached the Data Protection Act.

In the circumstances, I have decided that a criminal prosecution is not needed against either Ms. Hill or the police officer.

However, in light of my conclusion that there is sufficient evidence to provide a realistic prospect of convicting the police officer for an offence under the Data Protection Act, I have written to the Metropolitan Police Service and to the IPCC recommending that they consider bringing disciplinary proceedings against him.” Alison Levitt QC

Let us start by considering the arguments for and against prosecuting the journalist (pj) and the police officer (pp).

Arguments for prosecution:
A:
The articles contained confidential information;

B:
The police officer disclosed the information;

C:
A prosecution is required in the public interest.

Arguments against prosecution:
D:
There is no evidence that the police officer was paid any money;

E:
Information disclosed by the police officer, although confidential, was not highly sensitive;

F:
It did not expose anyone to a risk of injury or death;

G:
It did not compromise the investigation;

H:
It would probably have made it into the public domain by some other means;

I:
The public interest outweighs the overall criminality alleged;

J:
Together, the articles would expose the commission of criminal offences;

K:
Together, the articles would hold others to account;

L:
The articles contributed to an important matter of public debate, namely the nature and extent of the influence of the media.

Let us also analyse more closely the arguments relating to whether a prosecution is required in the public interest. The assumption is that both the journalist and the police officer have violated the Data Protection Act.

Arguments for bringing charges under the Data Protection Act:
M:
The information disclosed was personal data;

Arguments against bringing charges under the Data Protection Act:
N:
Disclosing the identity of those who are arrested is not, of itself, a criminal offence;

O:
It is only unlawful in this case because the disclosure also breached the Data Protection Act.

Remark 15

Notice how argument O was used rhetorically as part of an argument against bringing charges under the Data Protection Act. Argument O is, in fact, simply stating that the disclosure was unlawful because it breached the Data Protection Act. Consider also how the sentence below is used as part of the argumentation: “The alleged overall criminality is the breach of the Data Protection Act, but, as already noted, any damage caused by Ms. Hill's alleged disclosure was minimal”. Our model will help quantify such damage and will require a definition of minimal, as will become clear. Similarly, the following sentences provide clues as to the weights to be assigned to the neural network, in relation to the police officer: “Any claim that there is a public interest in his alleged conduct carries considerably less weight” and “There is a high threshold to make out the criminal offence of misconduct in public office”. We shall return to these once we have created the network model.

The QC's conclusion can be summarized as follows:
(a)
It was decided that a criminal prosecution is not needed;

(b)
It was decided that in the case of the journalist, the public interest outweighs the overall criminality alleged;

(c)
There is sufficient evidence to provide a realistic prospect of convicting the police officer for an offence under the Data Protection Act;

(d)
A recommendation was made to the police to consider bringing disciplinary proceedings against the police officer.

Our model's conclusion: Our model is concerned with making explicit the following relations (items 1 to 4 below).

On the issue of the police officer's misconduct:

1. Do the weights of the arguments exceed the high threshold for the offence of misconduct?

Arguments A, B and C should support the prosecution of the police officer (pp). Argument C should do so with a low weight (w) since the prosecution would be for misconduct in public office. Arguments D, F, G and H attack pp collectively (argument D with a high weight (W) for obvious reasons, and argument H with a low weight due to its speculative nature). Argument E attacks argument B. These are all the relevant arguments in relation to pp, as shown in Fig. 7, where dashed lines indicate attacks. As a result, neuron B fails to activate, and the weight of the arguments that collectively attack neuron pp should overcome the weight of the arguments that support pp. Hence, neuron pp should fail to activate. We discuss neuron/argument pj next.

Download : Download full-size image
Fig. 7. Neural implementation of legal case: prosecution decision.

2. If the police officer should not be prosecuted for misconduct then the journalist should not be prosecuted for aiding his conduct.

Item 2 above can be modelled in Fig. 7 using ADFs simply by stating that if argument pp is out (represented by a negative weight from input neuron pp to the hidden layer) then argument ¬pj should be in; hence, the journalist should not be prosecuted (in the neural network, if input neuron pp is not activated then output neuron ¬pj will be activated; see dashed line representing a negative weight from input neuron pp in Fig. 7). This separation between arguments pj and ¬pj allows one to ignore how arguments would influence neuron pj during the modelling of neuron ¬pj, and is referred to explicit negation in logic programming [19]. Thus, in case neuron pp fails to activate, which is the case here, neuron ¬pj will be activated. This completes the prosecution's analysis on the basis on misconduct.

On the issue of the violation of the Data Protection Act:

3. Do the weights of the arguments show that the public interest outweighs the violation of the Data Protection Act?

It is clear that arguments J, K and L support argument I, while argument M attacks I. In addition, argument N attacks M, and O attacks N, as shown in Fig. 8. The conclusion is that indeed argument I should prevail.

Download : Download full-size image
Fig. 8. Neural implementation of legal case: decision based on data protection act.

4. Different weights apply to the journalist and the police officer.

Argument M supports the argument that the journalist should be prosecuted for violation of the Data Protection Act (let us call this
). It also supports the argument for prosecuting the police officer for a violation of the Data Protection Act (call it
). The attacks from argument I on
and
should have different weights: a high (negative) weight W for
and a low (negative) weight (w) for
. Our model's conclusion, therefore, is that
should not prevail (i.e. the journalist should not be prosecuted), but differently from the QC's conclusion, if the value of the weight ω connecting argument M to
should be greater than the absolute value of w then argument
should prevail, i.e. the police officer would be prosecuted for violating the Data Protection Act. The actual values of weights ω and w may be a matter for debate, but perhaps the QC's arguments should have focused more on providing a justification for such values.

The above modelling exercise with the use of a neural cognitive model, may also help in the separation of concerns and systematic questioning of some of the assumptions made. For example, in this case study, some questions that emerge include: should different weights really apply to the journalist and the police officer, assuming they had to work as a team in the public interest? Aside from possible issues of remit, why should prosecution not be recommended straight away for the violation of the data protection act, and disciplinary proceedings should be evoked and recommended instead? We believe that our model should help prompt the user to ask such questions, organise the relationships among the different arguments under consideration, and investigate the impact of different weight assignments to the network model. For example, what if the weights W and w are assigned the same value? What if the weights ω and w are assigned the same absolute value? The user would then be able to run the model and consider the possible outcomes by analysing the different sets of prevailing arguments obtained as stable states of the neural network.

6. Conclusion and future work
   We have presented a neural cognitive model of argumentation that is capable of capturing a range of argumentation semantics and situations including joint-attacks, argument support, ordered attacks, disjunctive attacks, meta-level attacks and self-defeating attacks. All these different modes of argumentation can be modelled, learned and computed by means of a connectionist representation. In its most general form, arguments are weighted according to their strength, can support or attack other arguments directly, but can also combine conjunctively or disjunctively, sequentially or in parallel, at object- or meta-level, as exemplified throughout the paper. We have shown that all these different modes of argumentation can be represented and computed in a natural way by a connectionist network. This also indicates that the connectionist approach can offer an adequate tool for argument computation.

When dealing with uncertainty and meta-level preferences, in [14], the question of where the weights would come from is raised. In [2], voting by an audience is evoked as a solution that depends on meta-level considerations, as in [26]. With the framework proposed in this paper, the question gains a new dimension in that, as with any neural network, the weights can be learned from examples (i.e. instances from previous cases). As future work, we plan to explore the framework's learning capacity as part of a larger case study.

Uncertainty is intrinsic in human argumentation, yet most logic-based models of argumentation do not deal with uncertainty explicitly. Argumentation can be seen as a method for reducing one's uncertainty with the prevailing arguments being precisely those that are less-uncertain. In line with [21], [23], [35], the neural cognitive model introduced here lends itself well to this idea due to the use of weights and activation intervals as part of a neural network. However, we believe that the neural cognitive approach may also be advantageous from a purely computational perspective due to the networks' ability to adapt through learning and to compute prevailing arguments in parallel. All of the above is important given the objective of developing models of human argumentation. Future work includes, in addition to the evaluation of the framework's learning capacities, further experimentation on legal reasoning, a comparison of the framework's knowledge representation and learning capacity, e.g. in contrast with [23], [32], and the evaluation of the framework's parallel computation gains. A graphical interface is being developed to facilitate the interactive drawing and running of the networks as shown in the figures above, with all the features introduced here. We believe that such an interface can be useful as a tool for modelling, running and the elaboration of argumentation frameworks in a range of application areas.