Questions tagged [machine-translation]
For questions related to machine translation (MT), which is the task of translating text using a computer, machine or software.
51
questions
1
vote
0
answers
15
views
Modifying Cross Entropy Loss to work with multiple correct target sequences?
Let's say I'm training a transformer model to perform a seq to seq task, but there are multiple correct answers. For example, the following outputs would all be considered correct:
source: A B C -> ...
0
votes
0
answers
4
views
Individual Data point size within a dataset for finetuning for translation
I need to fine tuning a LLM for a translation task for specific domain. Could someone please advise if length of each data point within the data set has any impact on eventual performance of the LLMs. ...
1
vote
1
answer
287
views
Why can decoder-only transformers be so good at machine translation?
In my understanding encoder-decoder transformers for translation are trained with sentence or text pairs. How can it be explained in simple (high-level) terms that decoder-only transformers (e.g. GPT) ...
0
votes
0
answers
29
views
Translating relatively little known language
I want to build a model that does translation from some little known foreign language to English. No clue how to go about and wanted to get some advice.
Here’s what I have:
An old pdf (looks like ...
- 101
1
vote
0
answers
52
views
Why would increasing layers in PyTorch Transformer significantly increase loss?
I have a simple torch.nn.Transformer module for machine translation on the Multi30k dataset. It performs pretty well (32.2 Bleu score) but I looked at scaling up ...
- 111
1
vote
0
answers
41
views
The model's accuracy becomes suddenly so unreasonably good at beginning of the training process. I need an explaination
I am practicing machine translation using seq2seq model (more specifically with GRU/LSTM units). The following is my first model:
This model first archived about 0.03 accuracy score and gradually ...
- 11
3
votes
1
answer
214
views
Machine Translation Transformers: Why Mask in Decoder?
I am trying to understand the purpose of masking in the decoder in the "Attention is All you Need" paper. Why wouldn't we want to rely on bidirectional context when translating? What goes ...
0
votes
1
answer
91
views
Fine-Tuning T5 with specific penalty
Currently I am finetuning transformers T5 model for translation task. As part of the dataset, I am given sentences in Japanese, their translation to English, and ...
- 121
1
vote
0
answers
35
views
Using root and modifiers in translation task
I am doing a project of translation task by finetuning T5 model. I am given sentences in Chinese, their translation to English, and for every English sentence I am ...
- 131
1
vote
2
answers
94
views
sentence alignment between same language texts
Take this two texts as an example:
"I will start the recording and share the presentation. I hope it's
all clear from what we saw last time on classical cryptography and if
you remember we got ...
- 11
0
votes
1
answer
366
views
Fine Tuning Transformer Model for Machine Translation
I am working on the Transformer example demonstrated on TensorFlow's website. https://www.tensorflow.org/text/tutorials/transformer
In this example, Machine Translation model is trained to translate ...
- 101
3
votes
1
answer
940
views
How do Transformer decoders handle arbitrary length input?
I am working through a Tensorflow Neural Machine Translation tutorial (https://www.tensorflow.org/text/tutorials/transformer) and am confused about how the decoder handles inputs when making ...
0
votes
1
answer
511
views
BERT2: How to use GPT2LMHeadModel to start a sentence, not complete it
I am using GPT2LMHeadModel to change the way GPT2 choose the next word in a sentence. At this point, I have to give the initial part of the sentence and GTP2 starts to predict the better next word.
I ...
- 101
2
votes
2
answers
2k
views
What are the differences between BLEU and METEOR?
I am trying to understand the concept of evaluating the machine translation evaluation scores.
I understand how what BLEU score is trying to achieve. It looks into different n-grams like BLEU-1,BLEU-2,...
- 293
0
votes
2
answers
126
views
Why is the WMT16 dataset favoured for evaluating machine translation models?
The Workshop on Statistical Machine Translation has released translation challenges each year from 2004 on, which feature a dataset of sentence pairs in a variety of languages.
Even though the ...
- 101
1
vote
1
answer
51
views
Is image machine translation done in two steps?
Suppose I have images of hand-written Japanese text. If I want to translate those images, would my ML algorithm be a 2-step model (for example, a CNN to convert the image into Japanese characters/...
- 41
0
votes
1
answer
32
views
What does it mean to apply decomposition at inference-time in a machine translation system?
I'm reading this paper for sub-character decomposition for logographic languages and the authors mention decomposition at inference-time. They're using Transformer architecture.
More specifically, the ...
- 31
3
votes
0
answers
2k
views
What is input (and shape) to K/V/Q of self-attention of EACH Decoder block of Language-translation model Transformer's tokens during Inference?
Transformer model of the original Attention paper has a decoder unit that works differently during Inference than Tranining.
I'm trying to understand the shapes used during decoder (both self-...
- 181
0
votes
1
answer
93
views
Given the immaturity of NLP tools for non-English languages, should I first translate the non-English language to English before text pre-processing?
For non-English languages (in my case Portuguese), what is the best approach? Should I use the not-so-complete tools in my language, or should I translate the text to English, and after using the ...
1
vote
1
answer
325
views
Does it make sense to use BLEU or ROUGE for any machine translation task?
Many machine translation metrics such as BLEU or ROUGE are used to evaluate sequence to sequence models where, usually, the sequences are pieces of natural language.
Is it possible to use these ...
- 73
1
vote
1
answer
61
views
How is Google Translate able to translate texts of arbitrarily large length?
Sequence-to-sequence models with attention are known to be limited by a maximum sequence length. So how can we handle sequences of arbitrarily large size? Do we just set a very large maximum sequence ...
- 73
0
votes
1
answer
76
views
Semantic-based evaluation of translations instead of BLEU
I have a text generation model and I want to evaluate its output by comparing it to a set of gold human-annotated references.
I went through machine-translation metrics and I found that BLEU is used ...
- 123
1
vote
0
answers
20
views
Can unsupervised models learn something from cat vocalizations?
I love cats, and over the years have noticed that they have recurrent patterns of vocalizations. For example, upon seeing a bird, a cat may start chittering, but the same cat would never chitter at ...
- 111
1
vote
0
answers
65
views
Training seq2seq translation model with one source and multiple target
So basically I'm training a sequence to sequence model that translates English sentences to Arabic sentences. I'm using the data provided by Anki @ manythings. I realized that some of the sentences in ...
- 11
1
vote
1
answer
653
views
What is the purpose of "alignment" in the self-attention mechanism of transformers?
I've been reading about transformers & have been having some difficulty understanding the concept of alignment.
Based on this article
Alignment means matching segments of original text with their ...
0
votes
0
answers
27
views
Finding or creating a dataset for Neural Text Simplification
I'm currently starting a research project focused on NLP.
One of the steps involved in this project will be the development of a text simplification system, probably using a neural encoder-decoder ...
- 41
2
votes
1
answer
438
views
How is Google Translate able to convert texts of different lengths?
According to my experience with Tensorflow and many other frameworks, neural networks have to have a fixed shape for any output, but how does Google translate convert texts of different lengths?
- 49
1
vote
1
answer
734
views
Can we use NLP to understand/parse/compile programming code?
I wonder if we can use Natural Language Processing (NLP) to process programming code:
Given a piece of code, can we
Translate it to human language to understand what it does? The input could be a ...
- 11
1
vote
1
answer
778
views
What are the main differences between a language model and a machine translation model?
What are the main differences between a language model and a machine translation model?
- 1,613
1
vote
0
answers
21
views
How to autocorelate multiple variants of same text into one?
I want to improve quality of translations for open-source projects in Ukrainian language. We have multiple translations from different authors. We can also translate messages using machine ...
5
votes
1
answer
153
views
Is there any resource that describes in detail a naive example-based machine translation algorithm?
I'm looking to develop a machine translation tool for a constructed language. I think that the example-based approach is the most suitable because the said language is very regular and I can have a ...
user34314
3
votes
1
answer
141
views
How to do machine translation with no labeled data?
Is it be possible to train a neural network, with no parallel bilingual data, for machine translation?
- 897
1
vote
0
answers
38
views
What's the best method to predict/generate signal from one sensor (source) to signal from another another (target)?
I was wondering what is the best method out there to find relationship between two 1D signals so that I can predict/generate one (source) from the other (target). For example, let's say that in ...
3
votes
0
answers
25
views
How to back propagate for implementation of Sequence-to-Sequence with Multi Decoders
I am proposing a modified version of Sequence-to-Sequence model with dual decoders. The problem that I am trying to solve is Neural Machine Translation into two languages at once. This is the ...
- 143
4
votes
0
answers
63
views
Can sequence-to-sequence models be used to convert source code from one programming language to another?
Sequence-to-sequence models have achieved good performance in natural language translation. Could these models also be applied to convert source code written in one programming language to source code ...
- 41
2
votes
1
answer
221
views
What happens when the output length in the brevity penalty is zero?
The brevity penalty is defined as
$$bp = e^{(1- r/c)},$$
where $r$ is the reference length and $c$ is the output length.
But what happens if the output length gets zero? Is there any standard way ...
2
votes
0
answers
46
views
Best approach for online Machine Translation with few hundred of samples?
I want to implement a model that improves itself with the passage of time.
My main task is to build a machine translator (from English to Urdu).. The problem I am facing is that I have very little ...
- 121
1
vote
0
answers
102
views
Why do both sine and cosine have been used in positional encoding in the transformer model?
The Transformer model proposed in "Attention Is All You Need" uses sinusoid functions to do the positional encoding.
Why have both sine and cosine been used? And why do we need to separate the odd ...
- 11
2
votes
0
answers
45
views
Small Machine Translation Model
I would like to perform Machine Translation in tensorflow.js, in the browser. The issue is that state-of-the-art models have many gigabytes (fairseq ensemble), and translation is slow.
Do you know ...
- 121
4
votes
1
answer
535
views
What are the differences between Bytenet and Wavenet?
I recently read Bytenet and Wavenet and I was curious why the first model is not as popular as the second. From my understanding, Bytenet can be seen as a seq2seq model where the encoder and the ...
- 1,108
4
votes
1
answer
53
views
Is this technique image processing or computer vision?
If I use my mobile camera on a signboard or announcement board on a road or in a street (like the one attached in photo) where the message is written in Russian and my mobile shows me that message in ...
- 171
30
votes
9
answers
7k
views
What is the actual quality of machine translations?
As an AI layman, till today I am confused by the promised and achieved improvements of automated translation.
My impression is: there is still a very, very far way to go. Or are there other ...
4
votes
1
answer
173
views
How do I identify a monologue or dialogue in a conversation?
How do I identify monologues and dialogues in a conversation (or transcript) using natural language processing? How do I distinguish between the two?
1
vote
0
answers
375
views
How are the attention weights normalised in the transformer?
In the Transformer (adopted in BERT), we normalize the attention weights (dot product of keys and queries) using a softmax in the Scaled Dot-Product mechanism. It is unclear to me whether this ...
- 131
1
vote
2
answers
80
views
Convolutional Sequence to Sequence Learning kernel parameters
I am reading the paper Convolutional Sequence to Sequence Learning by Facebook AI researchers and having trouble to understand how the dimensions of convolutional filters work here. Please take a look ...
- 365
2
votes
0
answers
92
views
Translate product names with AI
I frequently need to translate product names for hundreds of similar products -- and I have a list of past product names. Is it possible to train AI to review past translations and translate? It doesn'...
- 121
1
vote
2
answers
453
views
Do I need an encoder-decoder architecture to predict the next item of a sequence?
I am trying to understand how RNNs are used for sequence modelling.
On a tutorial here, it mentions that if you want to translate say a sentence from English to French you can use an encoder-decoder ...
- 173
4
votes
0
answers
201
views
What is the intuition behind the calculation of the similarity between encoder and decoder states?
Suppose that we are doing machine translation. We have a conditional language model with attention where we are are trying to predict a sequence $y_1, y_2, \dots, y_J$ from $x_1, x_2, \dots x_I$: $$P(...
- 41
1
vote
1
answer
87
views
Neural network with logical hidden layer - how to train it? Is it policy gradient problem? Chaining NNs?
I am doing neural machine translation task from language S to language T via interlingua L. So - there is the structure:
...
- 833
3
votes
2
answers
253
views
Why can't we use Google Translate for every translation task?
Once a book is published in a language, why can't the publishers use Google Translate AI or some similar software to immediately render the book in other languages? Likewise for Wikipedia: I'm not ...
- 413