Exploring the use of online machine translation for independent language learning

Research in Learning Technology 2020. © 2020 A.N. Sfhea. Research in Learning Technology is the journal of the Association for Learning Technology (ALT), a UK-based professional and scholarly society and membership organisation. ALT is registered charity number 1063519. http://www.alt.ac.uk/. This is an Open Access article distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), allowing third parties to copy and redistribute the material in any medium or format and to remix, transform, and build upon the material for any purpose, even commercially, provided the original work is properly cited and states its license.


Independent language learning, online machine translation and translation in foreign language teaching
ILL as it is practised in foreign language teaching is not only a process but also a philosophy of education, in which a student acquires knowledge by his/her own efforts and develops the ability for inquiry and critical evaluation (Candy 1991quoting Forster 1972). An important part is giving the students freedom of choice in determining their learning objectives and freedom of process to carry them out using the wide plethora of online language resources and apps. This approach is meant to stimulate autonomy and problem-solving skills.
In order to put this approach into practice, we need to describe the elements that constitute the OMT as ILL ecosystem (see Figure 1). First of all, we need to believe in translation as a language learning skill (Beeby Lonsdale 1996;Carreres, Muñoz-Calvo, and Noriega-Sánchez 2017;Cook 2010;Laviosa 2014;Newmarks 1991;Pym, Malmkjaer, and Plana 2013), a communicative activity (Károly 2014) and as a pedagogical tool that can help to enhance and further improve reading, writing, speaking and listening skills (Leonardi 2010), as well as increasing language students' crosscultural and cross-linguistic awareness (Zanettin 2001).
We also need to understand that OMT is widely accessible as an online ILL resource. Currently, Google Translate supports more than 100 languages and is used by more than 500 million users translating more than 100 billion words every day. It can translate 37 languages via photo, 32 via voice in 'conversation mode' and 27 via real-time video in 'augmented reality mode'. OMT as a learning tool is also integrated in an ever-increasing digital world, where technology and digital literacy are indispensable and can help boost employability opportunities. The use of machine translation as CALL (Computer-Assisted Language Learning) was already pointed out by Niño (2009) and, more recently by Tsai (2019). In this context, OMT as CALL takes the form of a multimodal integrative environment that can encourage meaningful collaborative language practice interactions with various ILL resources and peer students. This environment encourages learner centeredness 3 (page number not for citation purpose) which, as pointed out by Warschauer and Kern (2000), implies allowing students more control over planning for what and how they learn. This is in line with the ILL pedagogical framework that we are presenting here.
Last but not least, we live in a globalised world which needs multilingual and cross-cultural communication by means of translation so, as language educators, we have an important role to play ensuring language students of any kind are aware of this.
Having described, the elements to bear in mind when implementing OMT as ILL, we are now focusing on the pedagogic framework to put all these elements into practice in the language learning class.
Following Benson's (2011, p. 125) model for the promotion of autonomous learning in the class (see Figure 2), we developed our own pedagogical framework for the integration of OMT as a form of an autonomous ILL task (see Figure 3).
From Benson's (2011) model, we learnt that there are many levels on which learner autonomy can operate, namely a resource-based approach, where emphasis is put on independent use of learning resources, a curriculum-based approach with an emphasis on curriculum control, a classroom-based approach with an emphasis on classroom control decisions, a teacher-based approach, which concentrates on teacher roles, a learner-based approach fostering the development of autonomous learning skills and a technology-based approach promoting the independent use of learning technologies. In this study, we will focus on a technology-, learner-and resource-based approach, as these forms of autonomous development lend themselves very well to the autonomous use of OMT for ILL purposes. Bearing Benson's (2011) model in mind, it makes sense to support a learner-centred methodology, where the learner develops autonomy by exploring, experimenting and interacting with learning technologies and where the language teachers act as mere facilitators of learning encouraging language exploration and promoting digital literacy.

Learning design: what did the students do
Since ILL can take many shapes depending on the context, interests and language needs of the students, various learning tasks were designed, integrating the use of OMT and four main language skills directly linked to translation as a language learning skill, namely reading and listening comprehensions into English (most of the participants were either natives or had an advanced English proficiency), written, audio and voice translations from English into the target language.

(page number not for citation purpose)
The main learning objective was to encourage students to evaluate the usefulness (or lack) of OMT for ILL in order to see whether it actually benefits language learning (i.e. improves comprehension, production and mediation skills) or whether it is counterproductive. In line with our student-centred methodology and inspired by Reinders's (2010) framework for self-directed learning (based on Knowles 1975; see Appendix 1), students were given a free rein in the choice of resources and translation themes around their own language learning needs and interests, in order to encourage experimentation, autonomous language use and motivation, and at the end we did an in class peer discussion with a view to reflect and analyse the whole learning experience.
The learning tasks were initially designed to be done at home via an online survey (see Appendix 2), addressing non-specialist language students from all levels studying a wide variety of languages at the University Language Centre, The University of Manchester. However, due to the low initial return (only a few students actually completed the online survey by May 2018), the author decided to conduct the learning tasks in a face-to-face session inviting her advanced C1 Spanish students. In the end, 37 students completed the survey (see Table 1 for a breakdown of languages and levels). The objective of the session was communicated to students in advance. In previous classes, I had already introduced the term OMT to students and how I am interested in investigating its use by language students. Students were made aware of how this task could be a part of our revision at the end of the year, and, with this in mind, students were encouraged to make use of structures seen in class. This face-toface session lasted for 2 hours, and students were encouraged to attend on a voluntary basis. Students brought their laptops and completed the various tasks and questions sharing some comments/recommendations about language items and online resources with their fellow students in an oral reflection at the end of the session. Their responses are commented in Sections 4-7.
From Table 1, we learn that only two lower-level students and five post-intermediate students completed the online survey, and the remaining 30 advanced students of Spanish completed it in the class. This can be due to various reasons; first, the comprehensive nature of the survey and a time in the year (May) when students are busy revising; second, this low return can also indicate that students at lower levels, even though they may have used OMT at some point of their language learning journey, do need more guidance from the language tutor to actually complete a comprehensive list of ILL tasks of this kind. This is not new to language teachers since the nature of ILL demands a good mixture of guidance and free rein to make it work.
OMT is no exception and its integration in the class is more meaningful if some faceto-face in class sessions are arranged to introduce the technology, its capabilities and deficiencies as an ILL resource. Unlike García and Pena (2011), this research does not include conclusive data on lower level students; however, the author doubts whether they would have benefited from this learning task as much as their more advanced level fellow students. Translation into the target language as a language learning task is mostly encouraged from level B2 onwards when students have enough knowledge and/or command of the target language to judge the correctness of their own productions into the L2 and be challenged with complex grammar structures or meanings, phraseology and idiomatic expressions into the target language of study. Nonetheless, students at a beginners' or intermediate level are still learning the lexico-semantic and morpho-syntactic structures of their language of study and are, therefore, not ready to assess the correctness of the OMT output. OMT when used for ILL production purposes may confuse them or, even worse, create some sort of dependence, which can be counterproductive for their language learning process. Having said this, with proper guidance, OMT could help boost beginners' reading comprehension in the target language by helping anticipate meaning on a source text of their choice and helping with pronunciation (as discussed in Sections 5.2 and 7).

Independent language learning and online machine translation knowledge
In the online survey, students were enquired about their ILL skills, their interaction with educational technology and their OMT knowledge. First, participants were asked to quantify their ILL awareness. For this purpose, we gave them the following prompt: I consider myself a good independent learner, i.e. I can identify my language learning needs, set goals, plan, select my own resources, monitor and reflect on my progress. This was meant to determine whether the students were aware of the important elements of ILL such as needs, analysis, process management and reflection.
According to the results shown in Table 2, the majority of the students consulted consider themselves good ILL learners. This is because, at the University Language Centre in Manchester, they have completed assessed ILL in the form of a dossier/portfolio of language tasks, where students practiced their language skills further around their own needs or interests, and reflected on their strengths and weaknesses, on the strategies and learning resources used and on what they learnt out of it. Students were also asked whether they enjoy interacting with education technology apps and/or websites. As shown in Table 3, most of the students do. This positive attitude towards interacting with online educational technologies may help explain why a majority of the respondents decided to come to this session on a voluntary basis, complete the various tasks and experiment on OMT for their ILL purposes.
The majority of the students consulted (see Table 4) had used OMT before, mostly for comprehension purposes, and some of them also for drafting, for detecting languages and for pronunciation purposes. Just a minority used it for spoken communication in a language they do not have command of, for travelling purposes, and for picture translation with non-Latin script languages they did not understand.

Language learning tasks
The learning tasks comprised four subtasks, namely reading and listening from the target language of study into English, and written, audio and voice translation from English into the target language. In the following sections, we explain what the students did, for example the type of texts they fed into the OMT system, the level of accuracy of OMT, the level of correctness of the students' OMT output post-editing and their impressions.

Task 1: Reading comprehension in the target language and translation into English via online machine translation
The first task consisted of reading a text in the target language of study and translating it into English via OMT, correcting any inaccuracies. This was meant to help testing their reading comprehension skills from the target language into English via OMT.  Table 4. Participants' use of online machine translation to support language learning.

Response rate
Had used online machine translation before 36 97% Did not use online machine translation before 1 3% Since students had total freedom in choosing texts for all the tasks, we could see that they had an interest in a wide variety of topics from current affairs to their main subjects of study, e.g. science, medicine, literature, sports, cinema, music, history, etc. This is representative of the nature of Institution Wide Language Programme students, that is language non-specialists coming from a wide variety of disciplines.
Since the majority of our samples are from advanced students of Spanish, reading comprehension is not a big issue for them, as shown in Figure 4, where orange and red fonts (depending on the severity) showed OMT's errors and the student's post-edited text (My translation) mistakes, respectively. Correctly translated utterances appear in green font. Understandingly, in their reflective comments, many of them commented that they can do better than the machine and they do not need the OMT output to help them comprehend. Still, as shown in Figure 5, students may have the odd comprehension error (seguida de: followed from), but, mostly,  students edited the OMT output successfully improving the text with some stylistic changes.
At beginners' level, however, students can struggle to understand certain words and phrases. To illustrate this, there is an example in Figure 5 from an Erasmus beginners' Portuguese student who did not correct the English text, probably because s/he thought it was correct.
Students were great at playing and experimenting with the language. For example in Figure 6, there is an extract from a nursery rhyme containing the word 'zopilotes', which is common in Central American Spanish, and was well translated into English via OMT. The advanced student agreed with the OMT output's correctness.
As shown in table 5, the majority of the students found that OMT is useful for reading comprehension purposes because it helped them to understand words, it is quick and mostly accurate for comprehension purposes and it helps with tense agreements in lower levels. However, students from higher levels were also aware of the lack of accuracy, and they all stated that they do not need the OMT output for comprehension purposes. They were also very much aware of genre limitations.
As shown in table 6, the majority (83.80%) of the students reported to have understood more with the aid of OMT although they are aware that OMT's accuracy very much depends on the language pair and direction.

Task 2: Audio translation into English via online machine translation
For the second task, the students translated an audio or video recording extract from the target language into English via OMT. In doing so, they were checking both the OMT accuracy and their own correctness at understanding the target language in its spoken form. Unfortunately, unlike with the written translation, it is not yet possible in Google Language Tools to keep a record of spoken translations with apps/websites such as Google Translate; however, many of these apps produce both a voice translation and a written transcript and we can keep a record of the latter. Something to bear in mind is that, in an automated voice-to-voice translation, it is one thing what the OMT picks up (automatic speech recognition) but quite another what it actually translates (text-to-speech translation). Systems such as Google Translate have usually been broken into three separate components: automatic speech recognition to transcribe the source speech as text, machine translation to translate the transcribed text into the target language and text-to-speech synthesis to generate speech in the target language from the translated text. The main drawback for these systems is speech recognition. As pointed out by Matic Horvat, a researcher in natural language processing quoted in Leprince-Ringuet (2018):  'Systems adapt to the training dataset they have been fed and the quality of speech recognition degrades when you introduce it to things it has not heard before. If your training dataset is conversational speech, it will not do so well at recognising speech in a busy environment, for example' so 'with echoes, noise or even heavy accents the algorithm will make mistakes'.
Students were aware of some of the OMT's limitations mentioned previously. They experimented both with audio recordings and with their own voices, and in this regard, a student from Northern Ireland made a sarcastic comment about the OMT 'not picking up her accent'. Overall, during the face-to-face session, it seemed students did not find audio OMT very useful. Amongst the reasons they provided were the following: because it cannot make out exact sounds, just picked up occasional words, not sentences or specific terms, skipped words, did not make sense, it contains too many errors up to the point that some said they were better off understanding the video themselves. The only exception is illustrated in Figure 7. It shows an example from an advanced student of Spanish who chose a Spanish scientific audio from her main subject of study (Physics) with very simple vocabulary and produced a rather well-translated voice/audio output via OMT in the target language with almost no need for revision. The native English student stylistically edited the target language text in English.
As shown in Table 7, not all the comments from the students were negative, though. Some students (especially from lower levels) were happy that voice OMT exists because it helps with (spoken) word understanding; they loved the transcripts and the idea of using it as an aural comprehension checker for small extracts.
Surprisingly, as shown in Table 8, 53.1% reported to have understood more with the aid of OMT, as opposed to 46.9% who understood less.

Task 3: Written translation from English into the target language via online machine translation
In this task, students had to translate a written text of their choice from English into the target language via OMT. This involved spotting and correcting any inaccuracies in the OMT output.  Students once again displayed a wide range of text types and genres, including their own texts like in the example shown in Figure 8. In this example, an advanced Spanish student of Geography wrote a source text about his studies and was happy with the OMT's text level of correctness. The student was not aware that there was a spelling error in the original, and that there was also a minor capitalisation error and a prepositional error but, overall, this rather simplified input language produced a good language quality output in the target language with very little revision needed.
In Figure 9, there is another example of an advanced Spanish student of Physics who translated an extract from her lab report and was able to spot where the OMT errors were (right thinking: passive translated into impersonal 'se' structure), but she got the verb forms wrong in this case.  This is another example of a beginners' student of Portuguese (see Figure 10). This time Google Translate; when I copied and pasted the students' source text (trying to replicate what the student did), Google Translate underlined it. For some reason, it underlined the source text but not the translation, which could have been a step forward in helping language educators identifying OMT output in language learning.
The student on this occasion did not correct the OMT so, we assume, s/he might have thought it was correct, which is mostly right, with the exception of a couple of lexical inaccuracies and a preposition error.
The intermediate level student example (see Figure 11) goes to show that students explored various OMT apps and sites, not just Google Translate, and evaluated their accuracy and contrasting various OMT output texts (SpanishDict, Reverso, Microsoft Translator, Skype Translator). For the English-Spanish pair, the translation quality happened to be rather similar across many of these OMT systems/apps in terms of language quality, as shown in this example. Figure 10. Example of an independent written production task via OMT. Figure 11. Example of written production outputs from different OMT systems as trialled by a student. Table 9, the vast majority found the written OMT output mostly useful (44.1%) or useful (26.5%). They valued the various translation options provided by the different OMT systems/apps, and they also experimented with pre-editing the source text to see how this affects the language quality of the target text. In general, the OMT output seemed to them a quick and accurate translation, especially useful for vocabulary, grammar and structure, and (in their own words) 'they enjoyed playing with the language and using OMT as a language checker'.

As shown in
On the other hand, some students (advanced and more confident in their abilities) thought OMT translations were too literal, awkward, ungrammatical, contained many mistakes and, in sum, they thought they could do better without the OMT output.
As shown in Table 10, it is interesting to point out how the majority of students thought that the written OMT output had a 62.9% correctness percentage but the majority of them (advanced) also thought they were slightly better than the written OMT output with 70.4% correctness in their post-edited text. The fact that there is such a small difference between the percentage of OMT output correctness perceived by (mostly) advanced students, and the perceived correctness percentage of the students' post-edited text suggests that students were aware of the rather high degree of accuracy of the OMT output. Despite feeling they could do (slightly) better, the majority of the students admitted that OMT helped them to improve their writing in the target language (see Table 11). These data suggest that, in this task and with this group of advanced learners, the use of OMT for ILL was not counterproductive to the learning of written production skills but rather the opposite.

Task 4: Online voice translation from English into the target language
The last task was voice translation from English into the target language via OMT. In the example shown in Figure 12, the advanced student of Spanish distinguished between what she said, what the OMT (Google Translate in this case, GT) picked up, the transcript, the voice translation and her own correction. In the example, we can see that Google Translate still very much relies on its written transcript for translation. The student in this case was successful in post-editing the OMT output with the exception of a couple of lexical mistakes that were left uncorrected.
As mentioned before, students were imaginative in their use of OMT and some of them even experimented with lyrics. The example in Figure 13 comes from beginners' students of French and Spanish, also learning English at university. As we know, lyrics are very difficult to translate due to its slang use of the language, sometimes full of omissions, cleft sentences, idioms, inappropriate language, etc. This beginners' student did not correct the English, probably because s/he did not know how to, but still s/he showed an interest in this particular use of OMT.
Overall, the voice OMT output into the target language was not perceived as very useful by the students (see Table 12). Some of the reasons provided were as follows: Table 10. Correctness percentage: written online machine translation versus students' postediting into the target language.

Correctness percentage of the written online machine translation output into the target language
70%-100% correct 62.9 60%-70% correct 28.6 40%-50% correct 8.6 Correctness percentage of the students' post-edited text into the target language 70%-100% correct 70.4 60%-70% correct 25.9 40%-50% correct 3.7 Table 11. Did the aid of written online machine translation improved your writing or did not help?

Respondents Percentage
Improved my writing 25 67 Did not help 7 18 Other 2 5 because the translation was mostly wrong and was not heard for a start, it contained lexical and grammatical errors at sentence level and it was not good with proper names and colloquialisms. As shown in Table 13, the students reported 60% correctness of their post-edited voice OMT output into the target language, as opposed to a rather unexpected 53.8% level of correctness for the voice OMT output. In other words, the students reported to have done slightly better than voice OMT, and this is somehow predictable given the low quality of current voice OMT output.
So for the future, after having completed these tasks and explored the connection between OMT and ILL, students concluded that they will use OMT as an aid for reading comprehension and written production mostly and not so much for listening and oral production purposes.
From the students' responses, we learned that OMT is useful for vocabulary development (it is possible to keep a record of short written translations as vocabulary flashcards), to help draft a text, to help with grammar structure and also for pronunciation. However, OMT is still controversial for ILL because the students feel they  still need correction (at all levels), it does not help with phraseology, colloquialisms and idiomatic expressions (important at advanced level), it may create dependence or confuse them (especially at lower levels), and it is sometimes perceived an unnatural way of expression/communication. It was quick to translate and provided full sentences [×2] The biggest shorƞall of OMT is misunderstanding colloquial language [×1] Table 13. Voice online machine translation correctness versus students' post-edited voice translation into the target language correctness.

Oral reflection
At the end of the face-to-face session with the advanced students of Spanish, we did an oral reflection exercise in class, following the four learning tasks on OMT for ILL as an opportunity to put in common their answers to the last part of the online survey (see Appendix 2). Students were prompted with five questions for discussion: (1) What is your opinion on using OMT for ILL? (2) What is your overall opinion on OMT language quality? (3) What uses for OMT can you think of ?, how can be used? (4) What would you not recommend OMT for? (5) Do you think in the future, with more accurate OMT, there will not be a need for learning languages? Why?
The students' answers were recorded and the following paragraphs summarise the students' responses to these questions.
Students agreed that OMT works better for comprehension purposes but is not yet as effective for production purposes, especially in less widely talked languages such as Hindi and Gujarati. They were aware of OMT's limitations at the sentence or text level; however, they think it works pretty well as a quick reference for words in context and with verb conjugations. They were also very much aware of OMT's limitations, e.g. culture-related items, idioms and colloquialisms. The students agreed that OMT output still needs human input to bring it to an acceptable level of accuracy. This prompted discussions about the value of OMT editing/evaluation skills for language-related career's employability. On a more positive note, a researcher stated 'in my field I need to do research in various languages so OMT could help me explore articles in languages I do not speak'. Students also enjoyed voice OMT and they could see its usefulness for instant multilingual communication.
Overall the students thought OMT is really easy to use and a quick communication aid, for example when travelling to a new country or for an emergency. On a more cautious note, a student noted that these tools are very helpful as long as they are relatively accurate. Some students said that its use can be questionable for medical or legal interpretation purposes because of the ambiguity and consequent misunderstandings that can arise. From this task, some of them learnt that nuances of meaning such as cultural references, colloquialisms, sarcasm or humour cannot be translated via OMT only and need to be post-edited.
Discussing the future of language learning, I was glad to hear the students debating in favour of learning languages formally. They agreed that the language learning process cannot be compared with OMT. Students thought there will always be a need for languages and for humans to check the OMT output for correctness. A Social Anthropology student added that learning a language also constitutes learning a new way of thinking and that OMT somehow comes across as unnatural in comparison.
Some students reflected on the use of translation in language learning and the linguistic gain of OMT post-editing. On this topic, one student commented: you still need to rephrase it to make it your personal translation. I think translating is really a matter of understanding the core meaning and being able to express it in another language, so it is worth doing the effort of trying to communicate meanings in different languages and it shows one's ability of comprehension, not only one's language skills. Regarding translation evaluation, it is clear that many students have been exposed to this technology since they started learning languages in secondary school. There were quite a few comments along the lines of '[online machine translation] it has got much better since I last used it at school'. Students explicitly compared various OMT websites and/or apps and provided comments on their variable abilities. Comments vary from 'SpanishDict is very helpful as its translations are quite accurate' to 'Google Translate can trip you up since it often ignores idiomatic phrases and literally translates word for word'. Another student commented: 'Google Translate is OK for a language you do not speak, if you need to understand something. However, YouTube translation is hopeless'.

Pedagogical implications
What follows summarises the students' answers to various questions from the last part of the survey about issues that can be of interest to language educators, for example the integration of OMT in the language class, the use of OMT as a form of ILL, OMT and academic dishonesty, translation evaluation, the most and least useful OMT features and some alternatives to avoid too much dependence on it.
From the teaching perspective, not all students were sure they would like to be taught about the advantages and disadvantages of OMT for language learning. They were undivided, 45.5% would not like this and 42.4% thought it would be useful if their teacher taught them about the potential and limitations of OMT for ILL. The words of one of the students are: Many people are not aware of why online machine translations are not that accurate and rely on them too much. It would be useful to know how to use online machine translations properly to aid your learning instead of replacing it.
The other 12% preferred to find tools that work for them; they already know (being advanced students), or think limitations need to be underlined at beginners' level.
When asked if they would approve if their institution would add OMT to the list of banned/plagiarism-inducing online resources, 90.9% of the students did not think that OMT is a plagiarism-inducing online language resource. To support this view, they argued that it is a student's right to choose their own resources, and that the university should provide everything that is possible and allow the students to make their informed choice. Quite a few students pointed out that many students use OMT, so it would be pointless to not to make full use of it. At the same time, they are aware that its use would be hard to regulate. To illustrate this, a student said: If a student chooses to just use online machine translation to translate their work, then they will suffer the consequences of any mistakes. They can also use a native speaker of the language to help with their work, which would essentially be as much "cheating/plagiarism" as using online machine translation regardless.
In relation to plagiarism detection, students were very much aware of the limitations of enforcing OMT plagiarism policies. In the words of one of the students: 21 (page number not for citation purpose) online machine translation can still be a useful tool to aid learning if you teach people how to use it in a constructive way. If a text has been translated with an online machine translation and not amended it is obvious but otherwise I would say it is almost impossible to spot the difference between someone using an online machine translation and then applying their own knowledge to improve it, and someone who is just using their own knowledge at a high level.
For the students, OMT for ILL constitutes a quick, easily accessible lexical aid that can help them with vocabulary development (75.7%), to draft a text (73%), to help with unfamiliar grammatical structures (35.1%), to get communicated in a language they do not speak (29.7%) and with pronunciation (13.5%). Students were also aware of the downside of the use of OMT for ILL and how it can be controversial because it still needs correction (78.4%), it can create dependence (54.1%), the OMT may confuse them (40.5%) and it is rather unnatural (24.3%).
Comparing the use of OMT as a language resource with other available online resources/apps such as Linguee, WordReference, online monolingual and multilingual dictionaries (e.g. Pleco, Spanish Dict) and verb conjugators, the students showed a preference for the latter. When asked about the reasons why they preferred these, they said online dictionaries provide contextual meanings and examples, and concordancers such as Linguee are reliable because you are 100% certain that they are good models, they help with real-life translations of more colloquial or idiomatic phrases. They also liked apps such as WordReference because they provide good examples in context, several translation options/more knowledgeable translations (usually by native speakers) and forum discussions on more complex words and phrases.
Finally, the participants of this study were asked about their opinion on the most useful features of OMT for ILL, and on the OMT limitations that will be more difficult to overcome.
According to the students, the most useful features of OMT are the dictionary and collaborative dictionary tool (81.3%), the conjugation aid (50%), grammar and spell checkers (50%), the fact that OMT provides examples in context (50%), being able to create personalised vocabulary lists (28.1%), the fact that there are currently many language combinations available (25%), the fact that voice OMT serves as pronunciation aid (21.9%) and its offline access (18.8%). Being able to edit and customise its dictionaries (15.6%), and accessing your translation history (12.5%), its speech integrated recognition (6.3%), being able to launch it with your voice (3.1%) and the possibility that some OMT apps offer to connect with a human live translator for a charged better quality translation end product (3.1%) were also acknowledged by the students.
Students were also asked about current OMT limitations that are difficult to overcome. They mentioned the following: not being able to translate puns, double meaning, homophones, etc. (76.7%), not being able to translate colloquialisms, idioms and fixed expressions (70%), not being able to detect sarcasm and irony (66.7%), not distinguishing between formal and informal register (60%), not recognising cultural items (46.7%), lack of text structure, that is cohesion, coherence and co-reference (36.7%), not recognising proper names (33.3%), grammar equivalence (30%), terminology and phraseology management (23.3%) and language varieties (23.3%).

Conclusion and final remarks
From all these data, we can conclude that the use of OMT technology does not seem to be a hindrance for ILL; it actually seems to help with (mostly written) comprehension, vocabulary in context and as a quick language checker for small (written or oral) utterances.
In contrast to text-to-text OMT, voice OMT is still underdeveloped. Google's successful use of neural networks in text-to-text translation cannot yet be compared with its speech-to-speech counterpart. There is still yet to be seen an improvement in speech recognition technology and context-based sentence translation by means of machine learning systems. At the moment, we can experiment with technologies such as Google Pixel Buds, Google-assistant-enabled headphones, which can successfully help with real-time translation of basic questions such as 'where is the bathroom?', however, with more complex sentences, background noise or heavy accents, the system often gets lost in translation.
In the next few years, we will witness a rapid expansion of spoken multilingual corpora together with voice and audio OMT apps. This area is no doubt of interest to language educators, and its study to support pronunciation and multilingual oral communication across languages is an exciting research pathway of OMT technology.
The (mostly advanced) students who took part in this study benefited from becoming aware of their own language learning capacities and of OMT's strengths (quick and good lexical reference) and limitations (accuracy, genre, register and audio translation), which are fundamental when planning to use OMT as an ILL resource. They also benefited from interacting with the various OMT systems and apps, with other online language reference tools and with their fellow students practicing receptive, productive and mediation skills (translation, translation quality assessment, pre-editing and post-editing). This not only expanded their digital literacy but also reinforced previous learning, enabled monitoring of individual student comprehension and production skills, and provided with an opportunity to discuss various intercultural, subject-related and linguistic intrinsic questions with their peer students. Last but not least, it also enhanced metalinguistic reflection. In this respect, it makes sense to integrate OMT in the language class. Only then, we can explain what this technology is capable of doing and provide some safe use recommendations for its ILL use outside of the language class.
Given today's globalised and multicultural society, translation and knowing how to use OMT as an ILL resource can be very useful to language students. Reflecting on this point, it is a great relief to learn that non-specialist language students like the ones that participated in this study, understand that OMT is not a quick linguistic shortcut to multilingualism. However, it constitutes an invaluable online language reference tool, which has the potential to aid instant multilingual communication.
As pointed out in this paper, an effective use of OMT for ILL requires a careful learning design, including attention to learners' needs, intended learning objectives, learning environment, tools and resources, and underpinning learning and teaching approach. The benefits of OMT for ILL, including digital literacy, conscious language revision and practice, translation practice and translation quality assessment amongst others, are sought-after skills for employers and constitute valuable assets towards internationalisation and globalisation.
With all this in mind, if OMT technology is here to stay and progress with time, we should not be afraid of its use for ILL, but at the same time, it would be advisable to seek opportunities to integrate it in the language class in a meaningful and realistic way, rather than ignore it or ban it altogether.

(page number not for citation purpose)
3. If you have used online machine translation before, what was your overall impression in terms of language quality? □ Very good □ Good □ Very bad 4. For what particular purposes have you used online machine translation? □ To help me understand foreign language written text □ To help me understand foreign language oral speech □ For travelling communication □ To help me with pronunciation □ To help me drafting a text in a foreign language □ For picture translation □ For detecting languages □ Other: _________________________________________

Online Machine Translation (online machine translation) for reading comprehension
For this section, please have your online machine translation (Google Translate, Reverso Context, PONS Translator, etc.) website or app ready for use. 1. Read a little extract (paragraph) of a foreign language text of your interest with the aid of an online machine translation app. You can copy your foreign language text + translation into English here. _________________________________________ 2. Can you make a brief summary in English of what you understood? _________________________________________ 3. Was the online machine translation output useful?
Not useful at all 1 2 3 4 5 6 Very useful 4. Why? * _________________________________________ 5. Did you understand more or less with the aid of the online machine translation output? □ More □ Less 6. Why?* _________________________________________