Reviews and Comments on Paper 64

Paper information

Paper #64: Eduardo Rodriguez-Tello, Jin-Kao Hao and Jose Torres-Jimenez. A Refined Evaluation Function for the MinLA Problem

Abstract: This paper introduces a refined evaluation function, called $\Phi$, for the Minimum Linear Arrangement problem (MinLA). Compared with the classical evaluation function ($LA$), $\Phi$ integrates additional information contained in an arrangement to distinguish arrangements with the same $LA$ value. The main characteristics of $\Phi$ are analyzed and its practical usefulness is assessed within both a Steepest Descent (SD) algorithm and a Memetic Algorithm (MA). Experiments show that the use of $\Phi$ allows to boost the performance of SD and MA, leading to the improvement on some previous best known solutions.
(file)

Summary of received reviews and comments

Reviews superseded by other reviews are shown in the grey color in the table.

	confidence	score
Review 1	3	3
Review 2	3	2
Review 3	2	2

Reviews and Comments

Review 1

PC member:
Reviewer:
Overall rating:	3 (strong accept)
Confidence:	3
Relevance: Is this paper relevant for this conference?	3 (strong accept)
Soundness: Is this paper technically sound and complete?	3 (strong accept)
Are the claims sufficiently supported by experimental/theoretical results?	3 (strong accept)
Significance: Are the results/ideas interesting for other AI researchers?	3 (strong accept)
Originality: Are the results or ideas novel and previously unknown?	2 (accept (I will argue for this paper))
Readability: Is the paper well-organized and easy to understand?	3 (strong accept)
Language: Is the paper written in correct English and style?	3 (strong accept)
Format: Is the paper correctly and consistently formatted?	3 (strong accept)
Review:	CONTRIBUTION OF THE PAPER: The paper presents a novel evaluation function for the Minimum Linear Arrangement problem that incorporates semantic information related to the arrangement. This evaluation function is incorporated in a Steepest Descent Algorithm, a simple Memetic Algorithm and a state of the art Memetic Algorithm. Results show that the performance of the algorithms is greatly improved when using the new evaluation function, leading, in the last case, to new best known solutions for benchmark instances in the literature. POSITIVE ASPECTS: This is a very interesting, well written and organized paper. Decisions are well motivated. Experiments are extensive and well performed. Results are good. NEGATIVE ASPECTS: none CHANGES TO IMPROVE THE PAPER: I would suggest the authors to enlarge the fonts and the markers size in figure 2. It would also be interesting to show relative improvements in addition to (or instead of) absolute ones. Is a pair of parenthesis or commas missed in the second sentence of the second paragraph of subsection 2.2? FURTHER COMMENTS: ITEMS BELOW ARE JUSTIFICATION OF THE SCORES IF NEGATIVE: (1) IS THIS PAPER RELEVANT FOR THIS CONFERENCE? (2) IS THIS PAPER TECHNICALLY SOUND AND COMPLETE? (3) ARE THE CLAIMS SUFFICIENTLY SUPPORTED BY EXPERIMENTAL OR THEORETICAL RESULTS? (4) ARE THE RESULTS/IDEAS INTERESTING FOR OTHER AI RESEARCHERS? (5) ARE THE RESULTS OR IDEAS NOVEL AND PREVIOUSLY UNKNOWN? (6) IS THE PAPER WELL-ORGANIZED AND EASY TO UNDERSTAND? (7) IS THE PAPER WRITTEN IN CORRECT ENGLISH AND STYLE? (8) IS THE PAPER CORRECTLY AND CONSISTENTLY FORMATTED?
PC only:
Time:	Jul 13, 11:41

Review 2

PC member:
Reviewer:
Overall rating:	2 (accept: I will argue for this paper)
Confidence:	3
Relevance: Is this paper relevant for this conference?	2 (accept (I will argue for this paper))
Soundness: Is this paper technically sound and complete?	2 (accept (I will argue for this paper))
Are the claims sufficiently supported by experimental/theoretical results?	2 (accept (I will argue for this paper))
Significance: Are the results/ideas interesting for other AI researchers?	2 (accept (I will argue for this paper))
Originality: Are the results or ideas novel and previously unknown?	0 (neutral (please avoid this option))
Readability: Is the paper well-organized and easy to understand?	2 (accept (I will argue for this paper))
Language: Is the paper written in correct English and style?	2 (accept (I will argue for this paper))
Format: Is the paper correctly and consistently formatted?	2 (accept (I will argue for this paper))
Review:	CONTRIBUTION OF THE PAPER: The paper defines a new evaluation function for guiding local search procedures for the Minimum Linear Arrangement Problem (minLA). The paper clearly shows that the new evaluation function contributes significantly to improve local search algorithms for the minLA and with a more fine-tuned memetic algorithm very good computational results are obtained, showing that the evaluation function contributes towards obtaining a new state-of-the-art algorithm. POSITIVE ASPECTS: Nice paper; clear contribution. NEGATIVE ASPECTS: -- CHANGES TO IMPROVE THE PAPER: - I'd suggest to use statistical tests to examine whether the differences observed in the results are statistically significant (I guess they are, but it should become standard to check the statistical significance of results, especially if the differences are not extremely large). - give a short reason, why in the MA you use a first-improvement descent instead of the previously best-improvement one (I guess it is simply faster). - If space is available, maybe add a very concise description of where the MA of Rodriguez-Tello et al. differs from the one you used previously FURTHER COMMENTS: --- ITEMS BELOW ARE JUSTIFICATION OF THE SCORES IF NEGATIVE: (1) IS THIS PAPER RELEVANT FOR THIS CONFERENCE? (2) IS THIS PAPER TECHNICALLY SOUND AND COMPLETE? (3) ARE THE CLAIMS SUFFICIENTLY SUPPORTED BY EXPERIMENTAL OR THEORETICAL RESULTS? (4) ARE THE RESULTS/IDEAS INTERESTING FOR OTHER AI RESEARCHERS? (5) ARE THE RESULTS OR IDEAS NOVEL AND PREVIOUSLY UNKNOWN? (6) IS THE PAPER WELL-ORGANIZED AND EASY TO UNDERSTAND? (7) IS THE PAPER WRITTEN IN CORRECT ENGLISH AND STYLE? (8) IS THE PAPER CORRECTLY AND CONSISTENTLY FORMATTED?
PC only:
Time:	Jul 15, 20:54

Review 3

PC member:
Reviewer:
Overall rating:	2 (accept: I will argue for this paper)
Confidence:	2
Relevance: Is this paper relevant for this conference?	2 (accept (I will argue for this paper))
Soundness: Is this paper technically sound and complete?	2 (accept (I will argue for this paper))
Are the claims sufficiently supported by experimental/theoretical results?	1 (weak accept (vote accept but don't mind rejecting))
Significance: Are the results/ideas interesting for other AI researchers?	1 (weak accept (vote accept but don't mind rejecting))
Originality: Are the results or ideas novel and previously unknown?	2 (accept (I will argue for this paper))
Readability: Is the paper well-organized and easy to understand?	1 (weak accept (vote accept but don't mind rejecting))
Language: Is the paper written in correct English and style?	2 (accept (I will argue for this paper))
Format: Is the paper correctly and consistently formatted?	2 (accept (I will argue for this paper))
Review:	CONTRIBUTION OF THE PAPER: Authors introduce a refinement in the evaluation function for the Minimum Linear Arrangement problem which enables to continue the minimization procedure where the classical evaluation function can not report improvement and algorithms like Steepest Descent (SD) stops. POSITIVE ASPECTS: The experiments report improvement in all cases using SD and the Memetic Algorithm (MA). NEGATIVE ASPECTS: Some Negative results when comparing with heuristic algorithms AMG and MAMP are minimized CHANGES TO IMPROVE THE PAPER: The section 2.1 where the refinement is presented has some terminology problems, since d_k is called the "absolute differences with value k between adjacent vertices of the graph" but, for the given use should be called : frequency of k as an absolute diference In the same section where it reads "... it does not make distinctions among the absolute differences (d_k)" should be "... it does not make distinctions among the absolute differences (k)" instead Also instead of "...each counter of absolute differences d_k should have a different ..." should be "...frequency d_k should have a different ..." you see? sometimes d_k is described as a counter and others as the absolute difference. It is not consistant. In section 2.2 there is an apendix of the described terminology problem, instead of "... is easier to reduce the absolute differences d_2=4 in ..." should be : "... is easier to reduce the frequency d_2=4 in .." I guess in section 3.1 instead of "...can be obtained by flipping the labels of any pairs of different vertices (u,v) ..." should be "...can be obtained by swapping the labels of any pairs of different vertices (u,v) ..." or clarify that since you flip a coin but how do you flip pairs of vertices? Table 2 should include the product of column I (total iterations) and column T (seconds per iteration) The "Average solution quality" of figure 2 is undefined FURTHER COMMENTS: ITEMS BELOW ARE JUSTIFICATION OF THE SCORES IF NEGATIVE: (1) IS THIS PAPER RELEVANT FOR THIS CONFERENCE? (2) IS THIS PAPER TECHNICALLY SOUND AND COMPLETE? (3) ARE THE CLAIMS SUFFICIENTLY SUPPORTED BY EXPERIMENTAL OR THEORETICAL RESULTS? Instead of picking 10 "representative cases" out of the 21 proposed in the cited articles, all of them should be included in the test set. (4) ARE THE RESULTS/IDEAS INTERESTING FOR OTHER AI RESEARCHERS? For those involved in the minimum Arrangement problem (5) ARE THE RESULTS OR IDEAS NOVEL AND PREVIOUSLY UNKNOWN? (6) IS THE PAPER WELL-ORGANIZED AND EASY TO UNDERSTAND? Sections 3 and 4 should be merged, the readers should not mind if a test was decided after or before another experiment (7) IS THE PAPER WRITTEN IN CORRECT ENGLISH AND STYLE? Except for some terminolgy problem describe earlier un this review (8) IS THE PAPER CORRECTLY AND CONSISTENTLY FORMATTED?
PC only:
Time:	Jul 17, 18:45