Questions tagged [machine-translation]

For questions related to machine translation (MT), which is the task of translating text using a computer, machine or software.

Filter by
Sorted by
Tagged with
1 vote
0 answers
15 views

Modifying Cross Entropy Loss to work with multiple correct target sequences?

Let's say I'm training a transformer model to perform a seq to seq task, but there are multiple correct answers. For example, the following outputs would all be considered correct: source: A B C -> ...
Brayden Alexander Rudisill's user avatar
0 votes
0 answers
4 views

Individual Data point size within a dataset for finetuning for translation

I need to fine tuning a LLM for a translation task for specific domain. Could someone please advise if length of each data point within the data set has any impact on eventual performance of the LLMs. ...
user11740597's user avatar
1 vote
1 answer
287 views

Why can decoder-only transformers be so good at machine translation?

In my understanding encoder-decoder transformers for translation are trained with sentence or text pairs. How can it be explained in simple (high-level) terms that decoder-only transformers (e.g. GPT) ...
Hans-Peter Stricker's user avatar
0 votes
0 answers
29 views

Translating relatively little known language

I want to build a model that does translation from some little known foreign language to English. No clue how to go about and wanted to get some advice. Here’s what I have: An old pdf (looks like ...
Jama's user avatar
  • 101
1 vote
0 answers
52 views

Why would increasing layers in PyTorch Transformer significantly increase loss?

I have a simple torch.nn.Transformer module for machine translation on the Multi30k dataset. It performs pretty well (32.2 Bleu score) but I looked at scaling up ...
Matt Harrison's user avatar
1 vote
0 answers
41 views

The model's accuracy becomes suddenly so unreasonably good at beginning of the training process. I need an explaination

I am practicing machine translation using seq2seq model (more specifically with GRU/LSTM units). The following is my first model: This model first archived about 0.03 accuracy score and gradually ...
Đạt Trần's user avatar
3 votes
1 answer
214 views

Machine Translation Transformers: Why Mask in Decoder?

I am trying to understand the purpose of masking in the decoder in the "Attention is All you Need" paper. Why wouldn't we want to rely on bidirectional context when translating? What goes ...
gumbelgrumbelgumbel's user avatar
0 votes
1 answer
91 views

Fine-Tuning T5 with specific penalty

Currently I am finetuning transformers T5 model for translation task. As part of the dataset, I am given sentences in Japanese, their translation to English, and ...
Dani's user avatar
  • 121
1 vote
0 answers
35 views

Using root and modifiers in translation task

I am doing a project of translation task by finetuning T5 model. I am given sentences in Chinese, their translation to English, and for every English sentence I am ...
Daniel's user avatar
  • 131
1 vote
2 answers
94 views

sentence alignment between same language texts

Take this two texts as an example: "I will start the recording and share the presentation. I hope it's all clear from what we saw last time on classical cryptography and if you remember we got ...
Axde's user avatar
  • 11
0 votes
1 answer
366 views

Fine Tuning Transformer Model for Machine Translation

I am working on the Transformer example demonstrated on TensorFlow's website. https://www.tensorflow.org/text/tutorials/transformer In this example, Machine Translation model is trained to translate ...
boyaronur's user avatar
  • 101
3 votes
1 answer
940 views

How do Transformer decoders handle arbitrary length input?

I am working through a Tensorflow Neural Machine Translation tutorial (https://www.tensorflow.org/text/tutorials/transformer) and am confused about how the decoder handles inputs when making ...
Dylan Larrabee's user avatar
0 votes
1 answer
511 views

BERT2: How to use GPT2LMHeadModel to start a sentence, not complete it

I am using GPT2LMHeadModel to change the way GPT2 choose the next word in a sentence. At this point, I have to give the initial part of the sentence and GTP2 starts to predict the better next word. I ...
Mucida's user avatar
  • 101
2 votes
2 answers
2k views

What are the differences between BLEU and METEOR?

I am trying to understand the concept of evaluating the machine translation evaluation scores. I understand how what BLEU score is trying to achieve. It looks into different n-grams like BLEU-1,BLEU-2,...
Exploring's user avatar
  • 293
0 votes
2 answers
126 views

Why is the WMT16 dataset favoured for evaluating machine translation models?

The Workshop on Statistical Machine Translation has released translation challenges each year from 2004 on, which feature a dataset of sentence pairs in a variety of languages. Even though the ...
Zwiebak's user avatar
  • 101
1 vote
1 answer
51 views

Is image machine translation done in two steps?

Suppose I have images of hand-written Japanese text. If I want to translate those images, would my ML algorithm be a 2-step model (for example, a CNN to convert the image into Japanese characters/...
cookiecutter's user avatar
0 votes
1 answer
32 views

What does it mean to apply decomposition at inference-time in a machine translation system?

I'm reading this paper for sub-character decomposition for logographic languages and the authors mention decomposition at inference-time. They're using Transformer architecture. More specifically, the ...
ticiarai's user avatar
3 votes
0 answers
2k views

What is input (and shape) to K/V/Q of self-attention of EACH Decoder block of Language-translation model Transformer's tokens during Inference?

Transformer model of the original Attention paper has a decoder unit that works differently during Inference than Tranining. I'm trying to understand the shapes used during decoder (both self-...
Joe Black's user avatar
  • 181
0 votes
1 answer
93 views

Given the immaturity of NLP tools for non-English languages, should I first translate the non-English language to English before text pre-processing?

For non-English languages (in my case Portuguese), what is the best approach? Should I use the not-so-complete tools in my language, or should I translate the text to English, and after using the ...
Vinícius Araújo's user avatar
1 vote
1 answer
325 views

Does it make sense to use BLEU or ROUGE for any machine translation task?

Many machine translation metrics such as BLEU or ROUGE are used to evaluate sequence to sequence models where, usually, the sequences are pieces of natural language. Is it possible to use these ...
Blencer's user avatar
  • 73
1 vote
1 answer
61 views

How is Google Translate able to translate texts of arbitrarily large length?

Sequence-to-sequence models with attention are known to be limited by a maximum sequence length. So how can we handle sequences of arbitrarily large size? Do we just set a very large maximum sequence ...
Blencer's user avatar
  • 73
0 votes
1 answer
76 views

Semantic-based evaluation of translations instead of BLEU

I have a text generation model and I want to evaluate its output by comparing it to a set of gold human-annotated references. I went through machine-translation metrics and I found that BLEU is used ...
Minions's user avatar
  • 123
1 vote
0 answers
20 views

Can unsupervised models learn something from cat vocalizations?

I love cats, and over the years have noticed that they have recurrent patterns of vocalizations. For example, upon seeing a bird, a cat may start chittering, but the same cat would never chitter at ...
Alex Stone's user avatar
1 vote
0 answers
65 views

Training seq2seq translation model with one source and multiple target

So basically I'm training a sequence to sequence model that translates English sentences to Arabic sentences. I'm using the data provided by Anki @ manythings. I realized that some of the sentences in ...
OmarAI's user avatar
  • 11
1 vote
1 answer
653 views

What is the purpose of "alignment" in the self-attention mechanism of transformers?

I've been reading about transformers & have been having some difficulty understanding the concept of alignment. Based on this article Alignment means matching segments of original text with their ...
codinggirl123's user avatar
0 votes
0 answers
27 views

Finding or creating a dataset for Neural Text Simplification

I'm currently starting a research project focused on NLP. One of the steps involved in this project will be the development of a text simplification system, probably using a neural encoder-decoder ...
joão correia's user avatar
2 votes
1 answer
438 views

How is Google Translate able to convert texts of different lengths?

According to my experience with Tensorflow and many other frameworks, neural networks have to have a fixed shape for any output, but how does Google translate convert texts of different lengths?
Gavin's user avatar
  • 49
1 vote
1 answer
734 views

Can we use NLP to understand/parse/compile programming code?

I wonder if we can use Natural Language Processing (NLP) to process programming code: Given a piece of code, can we Translate it to human language to understand what it does? The input could be a ...
LMD's user avatar
  • 11
1 vote
1 answer
778 views

What are the main differences between a language model and a machine translation model?

What are the main differences between a language model and a machine translation model?
DRV's user avatar
  • 1,613
1 vote
0 answers
21 views

How to autocorelate multiple variants of same text into one?

I want to improve quality of translations for open-source projects in Ukrainian language. We have multiple translations from different authors. We can also translate messages using machine ...
Volodymyr M. Lisivka's user avatar
5 votes
1 answer
153 views

Is there any resource that describes in detail a naive example-based machine translation algorithm?

I'm looking to develop a machine translation tool for a constructed language. I think that the example-based approach is the most suitable because the said language is very regular and I can have a ...
user avatar
3 votes
1 answer
141 views

How to do machine translation with no labeled data?

Is it be possible to train a neural network, with no parallel bilingual data, for machine translation?
Lerner Zhang's user avatar
1 vote
0 answers
38 views

What's the best method to predict/generate signal from one sensor (source) to signal from another another (target)?

I was wondering what is the best method out there to find relationship between two 1D signals so that I can predict/generate one (source) from the other (target). For example, let's say that in ...
apples_of_doom's user avatar
3 votes
0 answers
25 views

How to back propagate for implementation of Sequence-to-Sequence with Multi Decoders

I am proposing a modified version of Sequence-to-Sequence model with dual decoders. The problem that I am trying to solve is Neural Machine Translation into two languages at once. This is the ...
dee cue's user avatar
  • 143
4 votes
0 answers
63 views

Can sequence-to-sequence models be used to convert source code from one programming language to another?

Sequence-to-sequence models have achieved good performance in natural language translation. Could these models also be applied to convert source code written in one programming language to source code ...
pfds2222's user avatar
2 votes
1 answer
221 views

What happens when the output length in the brevity penalty is zero?

The brevity penalty is defined as $$bp = e^{(1- r/c)},$$ where $r$ is the reference length and $c$ is the output length. But what happens if the output length gets zero? Is there any standard way ...
ScientiaEtVeritas's user avatar
2 votes
0 answers
46 views

Best approach for online Machine Translation with few hundred of samples?

I want to implement a model that improves itself with the passage of time. My main task is to build a machine translator (from English to Urdu).. The problem I am facing is that I have very little ...
Abdul Rehman's user avatar
1 vote
0 answers
102 views

Why do both sine and cosine have been used in positional encoding in the transformer model?

The Transformer model proposed in "Attention Is All You Need" uses sinusoid functions to do the positional encoding. Why have both sine and cosine been used? And why do we need to separate the odd ...
Shiyu's user avatar
  • 11
2 votes
0 answers
45 views

Small Machine Translation Model

I would like to perform Machine Translation in tensorflow.js, in the browser. The issue is that state-of-the-art models have many gigabytes (fairseq ensemble), and translation is slow. Do you know ...
MrMartin's user avatar
  • 121
4 votes
1 answer
535 views

What are the differences between Bytenet and Wavenet?

I recently read Bytenet and Wavenet and I was curious why the first model is not as popular as the second. From my understanding, Bytenet can be seen as a seq2seq model where the encoder and the ...
razvanc92's user avatar
  • 1,108
4 votes
1 answer
53 views

Is this technique image processing or computer vision?

If I use my mobile camera on a signboard or announcement board on a road or in a street (like the one attached in photo) where the message is written in Russian and my mobile shows me that message in ...
DSP_CS's user avatar
  • 171
30 votes
9 answers
7k views

What is the actual quality of machine translations?

As an AI layman, till today I am confused by the promised and achieved improvements of automated translation. My impression is: there is still a very, very far way to go. Or are there other ...
Hans-Peter Stricker's user avatar
4 votes
1 answer
173 views

How do I identify a monologue or dialogue in a conversation?

How do I identify monologues and dialogues in a conversation (or transcript) using natural language processing? How do I distinguish between the two?
Surya Shrivastava's user avatar
1 vote
0 answers
375 views

How are the attention weights normalised in the transformer?

In the Transformer (adopted in BERT), we normalize the attention weights (dot product of keys and queries) using a softmax in the Scaled Dot-Product mechanism. It is unclear to me whether this ...
Robin's user avatar
  • 131
1 vote
2 answers
80 views

Convolutional Sequence to Sequence Learning kernel parameters

I am reading the paper Convolutional Sequence to Sequence Learning by Facebook AI researchers and having trouble to understand how the dimensions of convolutional filters work here. Please take a look ...
ytolochko's user avatar
  • 365
2 votes
0 answers
92 views

Translate product names with AI

I frequently need to translate product names for hundreds of similar products -- and I have a list of past product names. Is it possible to train AI to review past translations and translate? It doesn'...
ytk's user avatar
  • 121
1 vote
2 answers
453 views

Do I need an encoder-decoder architecture to predict the next item of a sequence?

I am trying to understand how RNNs are used for sequence modelling. On a tutorial here, it mentions that if you want to translate say a sentence from English to French you can use an encoder-decoder ...
piccolo's user avatar
  • 173
4 votes
0 answers
201 views

What is the intuition behind the calculation of the similarity between encoder and decoder states?

Suppose that we are doing machine translation. We have a conditional language model with attention where we are are trying to predict a sequence $y_1, y_2, \dots, y_J$ from $x_1, x_2, \dots x_I$: $$P(...
AIguy122's user avatar
1 vote
1 answer
87 views

Neural network with logical hidden layer - how to train it? Is it policy gradient problem? Chaining NNs?

I am doing neural machine translation task from language S to language T via interlingua L. So - there is the structure: ...
TomR's user avatar
  • 833
3 votes
2 answers
253 views

Why can't we use Google Translate for every translation task?

Once a book is published in a language, why can't the publishers use Google Translate AI or some similar software to immediately render the book in other languages? Likewise for Wikipedia: I'm not ...
Siddhartha's user avatar