Reviews and Comments on Paper 305
Paper information
| Paper #305: Georgiana Puscasu, Patricio Martinez-Barco and Estela Saquete Boro. On the Identification of Temporal Clauses |
| Abstract: This paper describes a machine learning approach to the identification of temporal clauses by disambiguating the subordinating conjunctions used to introduce them. Temporal clauses are regularly marked by subordinators, many of which are ambiguous, being able to introduce clauses of different semantic roles. The paper also describes our work on generating an annotated corpus of sentences embedding clauses introduced by ambiguous subordinators that might have temporal value. Each such clause is annotated as temporal or non-temporal by testing whether or not it answers the questions "when", "how often" or "how long" with respect to the action of its superordinate clause. Using this corpus, we then train and evaluate personalised classifiers for each ambiguous subordinator, in order to set apart temporal from non-temporal usages. Several classifiers have been evaluated, and the best performing ones achieve an average accuracy of 89.23% across the set of ambiguous connectives. (file) |
Summary of received reviews and comments
Reviews superseded by other reviews are shown in the grey color in the table.
| date | PC member | reviewer | confidence | score | |
| Review 1 | 4 | 3 | |||
| Review 2 | 3 | 2 | |||
| Review 3 | 1 | 2 | |||
Reviews and Comments
Review 1
| PC member: | |
| Overall rating: | 3 (strong accept) |
| Confidence: | 4 |
| Relevance: Is this paper relevant for this conference? | 3 (strong accept) |
| Soundness: Is this paper technically sound and complete? | 3 (strong accept) |
| Are the claims sufficiently supported by experimental/theoretical results? | 3 (strong accept) |
| Significance: Are the results/ideas interesting for other AI researchers? | 3 (strong accept) |
| Originality: Are the results or ideas novel and previously unknown? | 2 (accept (I will argue for this paper)) |
| Readability: Is the paper well-organized and easy to understand? | 3 (strong accept) |
| Language: Is the paper written in correct English and style? | 3 (strong accept) |
| Format: Is the paper correctly and consistently formatted? | 3 (strong accept) |
| Review: | CONTRIBUTION OF THE PAPER: The paper presents an interesting approach to a very useful task, identification of temporal clauses. POSITIVE ASPECTS: The proposed system is well motivated, introduced and described. The proposed methodology, the feature selection and description are good. The results are really good. An empirical evaluation is presented Without doubt it is the best paper from the set I have reviewed. NEGATIVE ASPECTS: None CHANGES TO IMPROVE THE PAPER: FURTHER COMMENTS: ITEMS BELOW ARE JUSTIFICATION OF THE SCORES IF NEGATIVE: (1) IS THIS PAPER RELEVANT FOR THIS CONFERENCE? (2) IS THIS PAPER TECHNICALLY SOUND AND COMPLETE? (3) ARE THE CLAIMS SUFFICIENTLY SUPPORTED BY EXPERIMENTAL OR THEORETICAL RESULTS? (4) ARE THE RESULTS/IDEAS INTERESTING FOR OTHER AI RESEARCHERS? (5) ARE THE RESULTS OR IDEAS NOVEL AND PREVIOUSLY UNKNOWN? (6) IS THE PAPER WELL-ORGANIZED AND EASY TO UNDERSTAND? (7) IS THE PAPER WRITTEN IN CORRECT ENGLISH AND STYLE? (8) IS THE PAPER CORRECTLY AND CONSISTENTLY FORMATTED? |
| PC only: | Without doubt it is the best paper from the set I have reviewed. |
| Time: | Jun 27, 13:18 |
Review 2
| PC member: | |
| Overall rating: | 2 (accept: I will argue for this paper) |
| Confidence: | 3 |
| Relevance: Is this paper relevant for this conference? | 2 (accept (I will argue for this paper)) |
| Soundness: Is this paper technically sound and complete? | 2 (accept (I will argue for this paper)) |
| Are the claims sufficiently supported by experimental/theoretical results? | 2 (accept (I will argue for this paper)) |
| Significance: Are the results/ideas interesting for other AI researchers? | 2 (accept (I will argue for this paper)) |
| Originality: Are the results or ideas novel and previously unknown? | 1 (weak accept (vote accept but don't mind rejecting)) |
| Readability: Is the paper well-organized and easy to understand? | 2 (accept (I will argue for this paper)) |
| Language: Is the paper written in correct English and style? | 2 (accept (I will argue for this paper)) |
| Format: Is the paper correctly and consistently formatted? | 2 (accept (I will argue for this paper)) |
| Review: | CONTRIBUTION OF THE PAPER: Tests several combination of features for a Machine Learning algorithm to classify temporal vs. non temporal clauses. POSITIVE ASPECTS: Well written paper. Structure flows. I enjoyed reading it, although it has minor flaws. NEGATIVE ASPECTS: Not very original (although useful); currently the main trend is to apply ML to everything related to NLP problemas. It does not explain clearly why the selected machine learning algorithm is chosen. It does not explain clearly why cooccurrence features' span was chosen to be only the main clause. It would be good to evaluate using information already available manually from Susanne corpus, so that errors are not derived from things like "syntactic parser fails in identifying verbs" CHANGES TO IMPROVE THE PAPER: I would suggest not to use references as the subject of the sentence, as in "[12] consider...", it would be better to say "Lascarides and Oberlander [12] consider", but it is not mandatory. There are several cases of this. Expanding authors can improve readability of your paper. Particularly, referring to Reuters Corpus only as [18] does not look good. Please check first paragraph of Section 2. It has a Spanish-like grammar. In page 3, a reference is missing: [22,?] Page 9 says characterising instead of characterizing, and claus instead of clause. In page [10] references to the examples are confusing ([22]) seems to refer to bibliography. I may suggest to use only (22) and change squared brackets to round brackets. FURTHER COMMENTS: I understand that collocations were used only as n-grams. It would be interesting to add functional-dependency collocations and test how it could be done. In other works (H. Calvo, A. Gelbukh, Acquiring Selectional Preferences from Untagged Text for Prepositional Phrase Attachment Disambiguation, LNCS 3136, Springer 2004, 207–216) doing this has shown that improvement can be significant. ITEMS BELOW ARE JUSTIFICATION OF THE SCORES IF NEGATIVE: (1) IS THIS PAPER RELEVANT FOR THIS CONFERENCE? (2) IS THIS PAPER TECHNICALLY SOUND AND COMPLETE? (3) ARE THE CLAIMS SUFFICIENTLY SUPPORTED BY EXPERIMENTAL OR THEORETICAL RESULTS? (4) ARE THE RESULTS/IDEAS INTERESTING FOR OTHER AI RESEARCHERS? (5) ARE THE RESULTS OR IDEAS NOVEL AND PREVIOUSLY UNKNOWN? (6) IS THE PAPER WELL-ORGANIZED AND EASY TO UNDERSTAND? (7) IS THE PAPER WRITTEN IN CORRECT ENGLISH AND STYLE? (8) IS THE PAPER CORRECTLY AND CONSISTENTLY FORMATTED? |
| PC only: | |
| Time: | Jul 13, 04:29 |
Review 3
| PC member: | |
| Overall rating: | 2 (accept: I will argue for this paper) |
| Confidence: | 1 |
| Relevance: Is this paper relevant for this conference? | 3 (strong accept) |
| Soundness: Is this paper technically sound and complete? | 2 (accept (I will argue for this paper)) |
| Are the claims sufficiently supported by experimental/theoretical results? | 0 (neutral (please avoid this option)) |
| Significance: Are the results/ideas interesting for other AI researchers? | 1 (weak accept (vote accept but don't mind rejecting)) |
| Originality: Are the results or ideas novel and previously unknown? | 2 (accept (I will argue for this paper)) |
| Readability: Is the paper well-organized and easy to understand? | -1 (weak reject (vote reject but don't mind accepting)) |
| Language: Is the paper written in correct English and style? | 3 (strong accept) |
| Format: Is the paper correctly and consistently formatted? | 3 (strong accept) |
| Review: | CONTRIBUTION OF THE PAPER: Paper introduces a methodology based on machine learning, which aims to pinpoint temporal clauses and works by disambiguating subordinated conjunctions that introduce them. POSITIVE ASPECTS: Thorough experimental analysis NEGATIVE ASPECTS: Paper is difficult to read and in some parts rather clumsy. CHANGES TO IMPROVE THE PAPER: Structure the paper in a way that it is easy to spot paper contribution and to replicate your results. FURTHER COMMENTS: ITEMS BELOW ARE JUSTIFICATION OF THE SCORES IF NEGATIVE: (1) IS THIS PAPER RELEVANT FOR THIS CONFERENCE? (2) IS THIS PAPER TECHNICALLY SOUND AND COMPLETE? (3) ARE THE CLAIMS SUFFICIENTLY SUPPORTED BY EXPERIMENTAL OR THEORETICAL RESULTS? (4) ARE THE RESULTS/IDEAS INTERESTING FOR OTHER AI RESEARCHERS? (5) ARE THE RESULTS OR IDEAS NOVEL AND PREVIOUSLY UNKNOWN? (6) IS THE PAPER WELL-ORGANIZED AND EASY TO UNDERSTAND? (7) IS THE PAPER WRITTEN IN CORRECT ENGLISH AND STYLE? (8) IS THE PAPER CORRECTLY AND CONSISTENTLY FORMATTED? |
| PC only: | Paper needs more structure to improve readability. |
| Time: | Jul 20, 20:59 |