UPPSALA UNIVERSITY : Department of Linguistics and Philology : STP : MT
Uppsala universitet
Hoppa över länkar

Course Home Page

Aim
Assignments
Evaluation
Links


Uppsala University
Department of Linguistics and Philology
Language Technology Program
Machine Translation
Lab 1

Machine Translation in Practice

Aim

In this lab session you will gain hands-on experience of using real machine translations services. You will see how translators work with machine translation on a daily basis to produce polished translations. You will compare the quailty of different systems and consider their other pros and cons, translating both in-domain and out-of-domain texts, and using both manual and automatic evaluation methods.


Before the session begins

Before the lab session starts, one person in each pair should visit the BTS platform and sign up for an account by clicking on the "Register" link at the top of the page. You can leave "Country" and "Business phone" blank; just fill in first name, last name, email, password and the verification code before clicking "Create new account". A Convertus administrator will then approve your request ready for the session.


Assignments

BTS

Begin by logging in to the BTS platform. You should automatically land on the "My requests" tab. Here you can upload a new document to be translated by clicking on the "Add request" button. In this folder you will find a selection of course plans from Luleå University. Pick one at random to translate from Swedish to English. Take a quick look at the content of the document and upload it to BTS. Make sure you select the correct source and target languages, and that the domain is set to "Medicine/Pharmacology". This ensures that the MT system is tailored towards translating texts in this domain. Tick the box "Receive e-mail notification when translation is ready", and make sure the translation type is set to "Automated translation with human revision". You can leave the due date and comments fields blank.

Your document will now be segmented and tokenised before being translated with Convertus' rule-based MT system. It should then appear in the list of documents in "My requests", and when it is ready for editing its status will read "Human revision" (you will need to refresh the page to check if the status has changed). Find your document and click on "view". Here you will be able to see the segments and their translations. By clicking on "Claim" you will be able to click through the segments and edit the translations.

(NB: If the translation is taking a long time (more than say 5-10 minutes), you can move on to the "Evaluation" section of the lab session, using the pre-translated files, and come back to this section when your course plan is ready. Alternatively, you may if you prefer log out of your BTS account and log back in with the following credentials:
Email: aaron.smith@lingfil.uu.se
Password: IcYd1MB?
Under "My requests" you will find pre-translated versions of all course plans; find one that hasn't already been claimed and continue the lab session.)

Spend about 20 minutes going through and editing the English sentences. Try to improve the quality of the translation. You don't have to edit every sentence; just do as many as you can in 20 minutes and then answer the following questions:

If you want after editing your course plan to look at the reference translation, you will find it in this folder.

Evaluation

In this section we will carry out some experiments with text from two very different domains: course plans and movie subtitles. Create a new directory for this lab and copy the following four files into this new work directory:

Source files:
pirates50.sv.txt
kursplan50.sv.txt

Reference files:
pirates50.en.txt
kursplan50.en.txt

Translate the sentences in the Swedish source files using both Convertus's BTS platform and Google Translate. When you click on "Add request" in BTS, you can click on the "Translate text" tab and copy the text directly into the text box. You can choose which domain you think is most appropriate! Save the translation results in separate text files in your work directory.

Note that the Convertus engine may take a few minutes to run. If it is taking a long time you may use the following pre-translated versions instead:
kursplan50.BTS.en.txt
pirates50.BTS.en.txt

Automatic evaluation

Manual Evaluation

Look at the translation into English and evaluate the top 20 text lines of each file (for both BTS and Google) using the following subjective assessment scale:

Compute the average score of your manual evaluations for each translation file.

Lab report

Report your results for all the above assignments. Furthermore, discuss the following questions:

Send your report in either English or Swedish via e-mail to Aaron Smith (aaronsmith540@gmail.com).
Deadline for handing in the report: 13 April, 2016

Background information