What are the "BLEU/COMET scores" in translation?

 

What are the "BLEU/COMET scores" in translation?

BLEU and COMET are automatic metrics used to score the quality of machine-translated text by comparing it to human reference translations.

BLEU (Bilingual Evaluation Understudy)

  • BLEU measures how many words and short phrases (n‑grams) in the machine translation also appear in the reference translation, giving higher weight to longer matching sequences.
  • It outputs a score between 0 and 1 (often shown as 0–100); higher scores mean the MT output is more like the reference.
  • BLEU is fast and widely used, but it captures surface overlap and is less sensitive to meaning when the wording differs but is still correct.

COMET

  • COMET is a newer, neural-network–based metric that uses pretrained language models to judge similarity in meaning between MT output and reference (and often also considers the source sentence).
  • Instead of just counting overlapping words, it embeds sentences in a semantic space and predicts a quality score that correlates more closely with human judgments, especially for strong modern systems where BLEU differences are small.
  • COMET scores are typically real numbers (e.g., around −1 to 1); higher values indicate better translations in terms of adequacy and fluency as estimated by the model.

In practice, researchers report both: BLEU for comparability with older work and COMET (or similar neural metrics) for a more meaning-aware view of translation quality.

 

Comments

Post a Comment

Popular posts from this blog

CÓMO CONVERTIRSE EN UN MEJOR TRADUCTOR

  CÓMO CONVERTIRSE EN UN MEJOR TRADUCTOR Primero, déjenme ser claro: no soy experto en este tema, solo quiero compartir ideas y conceptos en los que puedas no estar de acuerdo, o que pueda usted contribuir a mejorar o refutar. Para dominar el mercado actual, el traductor no solo debe "conocer idiomas", sino también transformarse en un arquitecto de flujos de trabajo de IA . La tecnología ya no es un enemigo, sino el "brazo operativo" que debe guiarse con precisión. Aquí tienes las habilidades técnicas que necesitas para mantenerte competitivo Ingeniería avanzada de prompts para LLMs: Saber cómo escribir instrucciones complejas para modelos como GPT-4 o Claude para obtener traducciones que respeten un tono de voz específico, límite de caracteres o estilo editorial. Por ejemplo, saber cómo usar el  few-shot prompting  (proporcionando ejemplos de estilo) para personalizar el resultado. Gestión de Memoria de Glosario y Traducción (Herramientas CAT AI-Enhanced)...
Read more

COME DIVENTARE UN TRADUTTORE MIGLIORE

  Prima di tutto, lasciatemi essere chiaro: non sono un esperto in materia, voglio solo condividere idee e concetti su cui potreste non essere d'accordo, o che potreste contribuire a migliorare o smentire. Per dominare il mercato attuale, il traduttore non deve solo "sapere le lingue", ma deve trasformarsi in un  architetto di flussi di lavoro AI . La tecnologia non è più un nemico, ma il "braccio operativo" da guidare con precisione. Ecco le competenze tecniche indispensabili per restare competitivi: Advanced Prompt Engineering per LLM:  Saper scrivere istruzioni complesse per modelli come GPT-4 o Claude per ottenere traduzioni che rispettino un  tono di voce specifico , un limite di caratteri o uno stile editoriale. Ad esempio, saper usare il  few-shot prompting  (fornire esempi di stile) per personalizzare l'output. Gestione di Glossari e Memorie di Traduzione (CAT Tools AI-Enhanced):  Padroneggiare software come Phrase o Tr...
Read more

BECOMING A BETTER TRANSLATOR

  To dominate the current market, the translator must not only "know languages", but must transform himself into a AI workflow architect . Technology is no longer an enemy, but the "operational arm" to be guided with precision. Here are the technical skills you need to stay competitive Advanced Prompt Engineering for LLMs: Knowing how to write complex instructions for models such as GPT-4 or Claude to obtain translations that respect a specific tone of voice, character limit, or editorial style. For example, knowing how to use few-shot prompting (providing style examples) to customize the output. Glossary and Translation Memory Management (CAT Tools AI-Enhanced): Master software such as Phrase or Trados that now integrate AI. The competence lies in knowing how to feed the algorithm with clean and specialized data to avoid terminological "hallucinations". AI Post-Editing (MTPE) and Quality Assurance: Develop a critical eye to identify typical mac...
Read more