Expert answer:Technical Summary
Solved by verified expert:I need clear practical technical summary of this paper : 3 page double spaced (12 font),1 paragraoh for section12 paragraph for section 21 paragraph for each sections (3,4,5,6)
paper2.pdf
paper2.pdf
Unformatted Attachment Preview
TABLE OF CONTENTS
1.0 INTRODUCTION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1
1.1 An Overview of Our Thesis Work . . . . . . . . . . . . . . . . . . . . . . . .
4
1.1.1 Context-aware Argument Mining Models . . . . . . . . . . . . . . . .
5
1.1.2 Intrinsic Evaluation: Cross-validation . . . . . . . . . . . . . . . . . .
6
1.1.3 Extrinsic Evaluation: Automated Essay Scoring . . . . . . . . . . . .
7
1.2 Thesis Statements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
8
1.3 Proposal Outline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
9
2.0 BACKGROUND . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
10
2.1 Argumentation Theories . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
10
2.2 Argument Mining in Di↵erent Domains . . . . . . . . . . . . . . . . . . . .
13
2.3 Argument Mining Tasks and Features . . . . . . . . . . . . . . . . . . . . .
15
2.3.1 Argument Component Identification . . . . . . . . . . . . . . . . . . .
15
2.3.2 Argumentative Relation Classification . . . . . . . . . . . . . . . . . .
17
2.3.3 Argumentation Structure Identification . . . . . . . . . . . . . . . . .
18
3.0 EXTRACTING ARGUMENT AND DOMAIN WORDS FOR IDENTIFYING ARGUMENT COMPONENTS IN TEXTS – COMPLETED
WORK . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
20
3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
20
3.2 Persuasive Essay Corpus . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
21
3.3 Argument and Domain Word Extraction . . . . . . . . . . . . . . . . . . . .
23
3.4 Prediction Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
25
3.4.1 Stab & Gurevych 2014 . . . . . . . . . . . . . . . . . . . . . . . . . .
25
iv
3.4.2 Nguyen & Litman 2015 . . . . . . . . . . . . . . . . . . . . . . . . . .
26
3.5 Experimental Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
27
3.5.1 Proposed vs. Baseline Models . . . . . . . . . . . . . . . . . . . . . .
27
3.5.2 Alternative Argument Word List . . . . . . . . . . . . . . . . . . . . .
29
3.6 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
30
4.0 IMPROVING ARGUMENT MINING IN STUDENT ESSAYS USING ARGUMENT INDICATORS AND ESSAY TOPICS – COMPLETED WORK . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
31
4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
31
4.2 Academic Essay Corpus . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
32
4.3 Prediction Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
34
4.3.1 Stab14 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
34
4.3.2 Nguyen15v2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
35
4.3.3 wLDA+4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
35
4.3.4 wLDA+4 ablated models . . . . . . . . . . . . . . . . . . . . . . . . .
37
4.4 Experimental Result . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
38
4.4.1 10-fold Cross Validation . . . . . . . . . . . . . . . . . . . . . . . . . .
38
4.4.2 Cross-topic Validation . . . . . . . . . . . . . . . . . . . . . . . . . . .
40
4.4.3 Performance on Held-out Test Sets . . . . . . . . . . . . . . . . . . . .
42
4.5 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
43
5.0 EXTRACTING CONTEXTUAL INFORMATION FOR IMPROVING
ARGUMENTATIVE RELATION CLASSIFICATION – PROPOSED
WORK . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
45
5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
45
5.2 Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
47
5.3 Two Problem Formulations and Baseline Models . . . . . . . . . . . . . . .
49
5.3.1 Relation with Argument Topic . . . . . . . . . . . . . . . . . . . . . .
49
5.3.2 Pair of Argument Components . . . . . . . . . . . . . . . . . . . . . .
50
5.3.3 Baseline Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
50
5.3.4 Evaluations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
51
v
5.4 Software Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
51
5.5 Pilot Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
52
5.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
53
6.0 IDENTIFYING ARGUMENT COMPONENT AND ARGUMENTATIVE RELATION FOR AUTOMATED ARGUMENTATIVE ESSAY
SCORING – PROPOSED WORK . . . . . . . . . . . . . . . . . . . . . . .
55
6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
55
6.2 Argument Strength Corpus . . . . . . . . . . . . . . . . . . . . . . . . . . .
56
6.3 Argument Mining Features for Automated Argument Strength Scoring . . .
56
6.3.1 First experiment: impact of performance of argument component identification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
57
6.3.2 Second experiment: impact of performance of argumentative relation
identification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
57
6.3.3 Third experiment: only argument mining features . . . . . . . . . . .
58
6.4 Argument Mining Features for Predicting Peer Ratings of Academic Essays
58
6.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
60
7.0 SUMMARY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
61
8.0 TIMELINE OF PROPOSED WORK . . . . . . . . . . . . . . . . . . . . .
63
APPENDIX A. LISTS OF ARGUMENT WORDS . . . . . . . . . . . . . . .
64
APPENDIX B. PEER RATING RUBRICS FOR ACADEMIC ESSAYS .
66
BIBLIOGRAPHY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
67
vi
1.0
INTRODUCTION
Argumentation can be defined as a social, intellectual, verbal activity serving to justify or
refute an opinion, consisting of statements directed towards obtaining the approbation of
an audience. Originally proposed within the realms of Logic, Philosophy, and Law, computational argumentation has become an increasingly central core study within Artificial
Intelligence (AI) which aims at representing components of arguments, and the interactions
between components, evaluating arguments and distinguishing legitimate from invalid arguments [Bench-Capon and Dunne, 2007].
With the rapid growth of textual data and tremendous advances in text mining, argument
(argumentation) mining in text1 has apparently been an emerging research field that is
to draw a bridge between formal argumentation theories and everyday life argumentative
reasoning. Aiming at automatically identifying argument components (e.g., premises, claims,
conclusions) in natural language text, and the argumentative relations (e.g., support, attack)
between components, argument mining is found to promise novel opportunities for opinion
mining, automated essay evaluation as well as o↵ers great improvement for current legal
information systems or policy modeling platforms. Argument mining has been studied in a
variety of text genres like legal documents [Moens et al., 2007, Mochales and Moens, 2008,
Palau and Moens, 2009], scientific papers [Teufel and Moens, 2002,Teufel et al., 2009,Liakata
et al., 2012], news articles [Palau and Moens, 2009,Goudas et al., 2014,Sardianos et al., 2015],
user-generated online comments [Cabrio and Villata, 2012, Boltužić and Šnajder, 2014], and
student essays [Burstein et al., 2003, Stab and Gurevych, 2014b, Rahimi et al., 2014, Ong
et al., 2014]. Problem formulations of argument mining have ranged from the separation of
argumentative from non-argumentative text, the classification of argument components and
1
Argument mining for short.
1
Essay 75:
(0) Do
arts and music improve the quality of life?
(1) My
view is that the [government should give priorities to invest more money on the
basic social welfares such as education and housing instead of subsidizing arts relative
programs]M ajorClaim .
(2) [Art
is not the key determination of quality of life, but education is]Claim . (3) [In
order to make people better o↵, it is more urgent for governments to commit money to
some fundamental help such as setting more scholarships in education section for all
citizens]P remise . (4) This is simply because [knowledge and wisdom is the guarantee of
the enhancement of the quality of people’s lives for a well-rounded social system]P remise .
(5) Admittedly,
[art, to some extent, serve a valuable function about enriching one’s
daily lives]Claim , for example, [it could bring release one’s heavy burden of study pressure and refresh human bodies through a hard day from work ]P remise . (6) However, [it
is unrealistic to pursuit of this high standard of life in many developing countries, in
which the basic housing supply has still been a huge problem with plenty of lower income family have squeezed in a small tight room]P remise . (7) By comparison to these
issues, [the pursuit of art seems unimportant at all ]P remise .
(8) To
conclude, [art could play an active role in improving the quality of people’s
lives]P remise , but I think that [governments should attach heavier weight to other social
issues such as education and housing needs]Claim because [those are the most essential
ways enable to make people a decent life]P remise .
Figure 1: A sample student essay taken from the corpus in [Stab and Gurevych, 2014a]. The
essay has sentences numbered and argument components enclosed in tags for easy look-up.
argumentative relations, to the identification of argumentation structures/schemes.
To illustrate di↵erent tasks in argument mining, let us consider a sample student essay
in Figure 1. The first sentence in the example is the writing prompt. The MajorClaim
which states the author’s stance towards the writing topic is placed at the first of the essay’s
body, i.e., sentence 1. The student author used di↵erent Claims (controversial statements)
to validate/support and attack the major claim, e.g., claims in sentences {2, 5, 8}. Validity
of the claims are underpinned/rebutted by Premises (reasons provided by the author), e.g.,
premises in sentences {5, 6, 7}. As the first task in argument mining, Argument Component Identification aims at recognizing argumentative portions in the text (Argumentative
Discourse Units – ADUs [Peldszus and Stede, 2013]), e.g., a subordinate clause in sentence
1, or the whole sentence 2, and classifying those ADUs accordingly to their argumentative
2
MajorClaim(1)
Support
Attack
Claim(2)
Claim(5)
Support
Premise(5)
Attack
Premise(7)
Support
Premise(6)
Figure 2: Graphical representation of a part of argumentation structure in the example essay.
Argumentative relations are illustrated based on annotation by [Stab and Gurevych, 2014a].
roles, e.g., MajorClaim, Claim, and Premise. The two sub-tasks are often combined into
a multi-way classification problem by introducing the None class. Thus, possible class labels for a candidate ADU are {MajorClaim, Claim, Premise, None}. However, determining
boundaries of candidate ADUs to prepare input for argument mining models is a nontrivial preprocessing task. In order to simplify the main argument mining task, sentences are
usually taken as primary units [Moens et al., 2007], or the gold-standard boundaries are
assumed available [Stab and Gurevych, 2014b].
The second task, Argumentative Relation Classification [Stab and Gurevych, 2014b],
considers possible pairs of argument components in a definite scope, e.g., paragraph,2 or
pairs of argument component and argument topic. For each pair, determines if a component
supports or attacks the other component. As we have in the example essay, the Claim in
2
The definite scope is necessary to make the distribution less skewed. In fact, the number of pairs that
hold an argumentative relation is far smaller than the total number of possible pairs.
3
sentence 2 supports the MajorClaim in sentence 1: Support(Claim (2) , MajorClaim (1) ). We
also have Attack (Claim (5) , MajorClaim (1) ), Support(Premise (5) , Claim (5) ). Given the direct
relations as in examples, one can infer Attack (Premise (5) , MajorClaim (1) ) and so on.
While in argumentative relation classification one does not di↵erentiate direct and inferred relations, Argumentation Structure Identification [Mochales and Moens, 2011] aims
at constructing the graphical representation of argumentation in which edges are direct attachments between argument components. Attachment is an abstraction of support/attack
relations, and is illustrated as arrowhead connectors in Figure 2. Attachment between argument components does not necessarily correspond to the components’ relative positions
in the text. For example, Premise(6) is placed between Claim(5) and Premise(7) in the essay,
but Premise(7) is the direct premise of Claim(5) as shown in the figure.
1.1
AN OVERVIEW OF OUR THESIS WORK
In education, teaching argumentation and argumentative writing to student are in particular need of attention [Newell et al., 2011, Barstow et al., 2015]. Automated essay scoring
(AES) systems have been proven e↵ective to reduce teachers’ workload and facilitate writing
practices, especially in large-scale [Shermis and Burstein, 2013]. AES research has recently
showed interest in automated assessment of di↵erent aspects of written arguments, e.g., evidence [Rahimi et al., 2014], thesis and argument strength [Persing and Ng, 2013,Persing and
Ng, 2015]. However, the application of argument mining in automatically scoring argumentative essays has been studied limitedly [Ong et al., 2014, Song et al., 2014]. Motivated by
the promising application of argument mining as well as the desire of automated support for
argumentative writings in school, our research aims at building models that automatically
mines arguments in natural language text, and applying argument mining outcome to automatically scoring argumentative essays. In particular, we propose context-aware argument
mining models to improve state-of-the-art argument component identification and argumentative relation classification. In order to make the proposed approaches more applicable to
the educational context, our research conducts both intrinsic and extrinsic evaluation when
4
comparing our proposed models to the prior work. Regarding intrinsic evaluation, we perform both random folding cross validation and cross-topic validation to assess the robustness
of models. For extrinsic evaluation, our research investigates the uses of argument mining
for automated essay scoring. Overall, our research on argument mining can be divided into
three components with respect to their functional aspects.
1.1.1
Context-aware Argument Mining Models
The main focus of our research is building models for argument component identification and
argumentative relation classification. As illustrated in [Stab and Gurevych, 2014a], context3
is crucial for identifying argument components and argumentation structures. However,
context dependence has not been addressed adequately in prior work [Stab et al., 2014].
Most of argument mining studies built prediction models that process each textual input4
isolatedly from the surrounding text. To enrich the feature space of such models, history
features such as argumentative roles of one or more preceding components, and features
extracted separately from preceding and/or following text spans have been usually used
[Teufel and Moens, 2002, Hirohata et al., 2008, Palau and Moens, 2009, Guo et al., 2010,
Stab and Gurevych, 2014b]. However, the idea of using surrounding text as a context-rich
representation of the prediction input for feature extraction was studied limitedly in few
research [Biran and Rambow, 2011].
In many writing genres, e.g., debates, student essays, scientific articles, the availability of writing topics provides valuable information to help identify argumentative text as
well as classify their argumentative roles [Teufel and Moens, 2002, Levy et al., 2014]. Especially, [Levy et al., 2014] defined the term Context Dependent Claim to emphasize the
role of discussion topic in distinguishing claims relevant to the topic from the irrelevant
statements. The idea of using topic and discourse information to help resolve ambiguities
are commonly used in word sense disambiguation and sentiment analysis [Navigli, 2009, Liu,
3
The thesis di↵erentiates between global context and local context. While global context refers to the
main topic/thesis of the document, the local context is instantiated by the actual text segment covering the
textual unit of interest, e.g., preceding and following sentences.
4
E.g., candidate ADU in argument component identification, or pair of argument components in argumentative relation classification.
5
2012]. Based on these observations, we hypothesize that argument component identification
and argumentative relation classification can be improved with respect to prediction performance by considering contextual information at both local and global levels when developing
prediction features.
Definition 1. Context segment of a textual unit is a text segment formed by neighboring
sentences and the unit itself. The neighboring sentences are called context sentences, and
must be in the same paragraph with the textual unit.
Instead of building prediction models that process each textual input isolatedly, our
context-aware approach considers the input within its context segment 5 to enable advanced
contextual features for argumentative relation classification. In particular, our approach
aims at extracting discourse relations within the context segment to better characterize the
rhetorical function of the unit in the entire text. Besides, the context segments instead of
their units will be fed to textual entailment and semantic similarity scoring functions to
extract semantic relation features. We expect that a score set by possible pairs extracted
from two segments better represents the semantic relations of the two input units than
their single score. As defining the context and identifying boundaries of context segment
are not a focus of our research, we propose to use di↵erent heuristics, e.g., window-size,
topic segmentation, to approximate the context segment given a textual unit, and evaluate
contribution of such techniques to the final argument mining performance.
Definition 2. Argument words are words that signal the argumentative content, and commonly used across di↵erent argument topics, e.g., ‘believe’, ‘opinion’. In contrast, domain
words are specific terminologies commonly used within the topic, e.g., ‘art’, ‘education’. Domain words are a subset of content words that form the argumentative content.
As of a use of global context, we propose an approach that uses writing topics to guide a
semi-supervised process for separating argument words from domain words.6 The extracted
5
Term “context sentences” was used in [Qazvinian and Radev, 2010] to refer sentences surrounding a
citation, that contain information about the cited source but do not explicitly cite it. In this thesis, we place
no other constrains to context sentences than requiring them to be adjacent to the textual unit.
6
Our definition of argument and domain words shares similarities with the idea of shell language and
content in [Madnani et al., 2012] in that we aim to model the lexical signals of argumentative content.
However while Madnani et al. emphasized the boundaries between argument shell and content, we do not
require such a physical separation between the two aspects of an argument component.
6
voca …
Purchase answer to see full
attachment
How it works
- Paste your instructions in the instructions box. You can also attach an instructions file
- Select the writer category, deadline, education level and review the instructions
- Make a payment for the order to be assignment to a writer
- Download the paper after the writer uploads it
Will the writer plagiarize my essay?
You will get a plagiarism-free paper and you can get an originality report upon request.
Is this service safe?
All the personal information is confidential and we have 100% safe payment methods. We also guarantee good grades