GermEval Task 2017 - Shared Task on Aspect-based Sentiment in Social Media Customer Feedback
Website: https://sites.google.com/view/germeval2017-absa/
We invite everyone from academia and industry to submit to the shared task on German Sentiment Analysis!
—- Introduction —-
In the connected, modern world, customer feedback is a valuable source for insights on the quality of products or services. This feedback allows other customers to benefit from the experiences of others and enables businesses to react on requests, complaints or recommendations. However, the more people use a product or service, the more feedback is generated, which results in the major challenge of analyzing huge amounts of feedback in an efficient, but still meaningful way.
We conduct a shared task on automatically analyzing customer reviews about “Deutsche Bahn” - the German public train operator with about two billion passengers each year. The task is associated with the GSCL 2017 conference in Berlin, and will take place there as a half-day workshop on September 12, 2017.
—- Data —-
The data for the task has been annotated as part of a joint project between TU Darmstadt and Deutsche Bahn, see https://sites.google.com/view/germeval2017-absa/data for details. All together it consists of 22,000 messages from various social media and web sources. The data is annotated with relevance, document sentiment, aspect-based sentiment and opinion target expressions and is provided in both xml and tsv.
—- Task description —-
To exploit the richness of the data, we subdivided the task into four subtasks. Participants can freely choose in what and how many sub-tasks they participate.
Subtask A) Relevance Classification Determine whether a social media post contains feedback about the “Deutsche Bahn” or if the post is off-topic/contains no evaluation. Example: “Ehrlich die männer in Der Bahn haben keine manieren?” In the given post, the task is to identify that the post is ”relevant”.
Subtask B) Document-level Polarity Identify, whether the customer evaluates the “Deutsche Bahn” or travel as positive, negative or neutral. Example: ”Ingo Lenßen Guten morgen Ingo…bei mir kein regen aber bahn fehr wieder nicht…” In the given post, the task is to identify the post polarity : negative
Subtask C) Aspect-level Polarity Identify all aspects which are positively and negatively evaluated within the review. In order to increase comparability, the aspects are previously divided into predefined categories. Consequently, the aim of the subtasks is to identify all contained categories and their associated polarity. Example: “Alle so “Yeah, Streik beendet”” Bahn so “Okay, dafür werden dann natürlich die Tickets teurer” Alle so “Können wir wieder Streik haben?”” In the given post, the task is to identify the aspects and their polarity: <Ticketkauf#Haupt:negativ> <Allgemein#Haupt:positive>
Subtask D) Opinion Target Extraction
Identify the linguistic expression in the posts which are used to express
the aspect-based sentiment (subtask C). The opinion target expression is
defined by its starting and ending offsets.
Example: @m_wabersich IC 2151? Der fährt nicht. Ich habe Ihnen die
Alternative bereits genannt. /je
In the given post the task ist to identify the target expression
The organizers will provide a baseline system and a scoring method that all participants can use to lower the obstacles for participation. Data, baseline system as well as the description of the tasks are distributed to the participants via the website https://sites.google.com/view/germeval2017-absa/.
Participating team/participants may submit several runs.
Submissions consist of one TSV or XML file per subtask providing predictions for the test data and a paper of 4 pages (excluding references) describing the chosen approach and analyzing the performance. Each participating team/participant should submit only one paper regardless of how many subtasks they participate in; per subtask, participants can add 1 page to the paper, up to a maximum of 7 pages. Papers should follow the GSCL 2017 style files. We expect authors to present summaries of their systems at the GSCL workshop and to participate in the discussions.
—- Important Dates —-
—- Organizing committee —-
To contact the organizing committee, please post to the GermEval 2017 ABSA mailing list at https://groups.google.com/forum/#!forum/germeval2017-absa, or for private communication, send an e-mail to Michael Wojatzki.