T 1903/20 (Rare-word processing/GOOGLE) 30-10-2023
Download and more information:
Neural machine translation systems with rare word processing
I. The applicant appealed against the decision of the examining division refusing European patent application No. 15787870.3, which was published as international publication WO 2016/065327.
II. In its statement of grounds of appeal, the appellant requested that the decision under appeal be set aside and that a patent be granted on the basis of "the main request considered by the examining division", i.e. on the basis of the sole set of claims considered in the decision under appeal.
III. In a communication accompanying the summons to oral proceedings, the board expressed the preliminary opinion that the subject-matter of claim 1 of the sole request lacked an inventive step.
IV. In response to the summons to oral proceedings, the appellant informed the board that it would neither attend the oral proceedings nor make further written submissions.
V. The board cancelled the oral proceedings.
VI. Claim 1 of the sole request reads as follows:
"A computer-implemented translation system for translating natural language text from a source sentence in a source language to a target sentence in a target language, the translation system comprising one or more computers and one or more storage devices storing translation instructions and translation data, wherein:
the translation data includes:
a word dictionary that maps words in the source language to translations of the words into the target language;
a neural network translation model trained to track the origin in source sentences of unknown words in target sentences and to emit for each out-of-vocabulary (OOV) word in the target sentence a respective unknown token, the model being operable to emit (i) pointer tokens, pointer tokens being unknown tokens that identify a respective source word in the source sentence corresponding to the unknown token, and (ii) null unknown tokens, null unknown tokens being tokens that do not identify any source word in the source sentence;
the translation instructions are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising:
for every pointer token in a target sentence emitted by the neural network translation model from a source sentence, replacing the pointer token according to the corresponding source word in the source sentence, wherein replacing a first pointer token in the target sentence comprises using the word dictionary to perform a word translation from the corresponding source word for the first pointer token in the source sentence and replacing the first pointer token with the result of the translation."
VII. The appellant's arguments, where relevant to this decision, are discussed in detail below.
1. The application
1.1 The application relates to neural-network-based machine translation (NMT). An NMT system includes a neural network that maps a source sentence in one natural language to a target sentence in a different natural language (see page 1, first paragraph, of the published application).
1.2 According to the background section of the application, a major limitation of current NMT systems is their reliance on a fixed and modest-size vocabulary, which results in poor translation performance on sentences with many rare words.
2. The invention as defined by claim 1
2.1 Claim 1 is directed to a "computer-implemented translation system for translating natural language text from a source sentence in a source language to a target sentence in a target language".
2.2 The system includes a "neural network translation model".
In the light of claim 1 as a whole, the board understands that the translation model is used to translate a source sentence to a target sentence (as confirmed by page 6, lines 14 and 15, of the description). The model has been trained to insert "pointer tokens" in the target sentence, where each pointer token identifies a word in the source sentence that was not recognised by the translation model.
2.3 The system further includes "translation instructions" and a word dictionary.
The translation instructions replace each pointer token in a target sentence emitted by the translation model with a word in the target language by using the word dictionary to map the unrecognised word in the source sentence to a word in the target language.
2.4 Claim 1 also mentions "null unknown tokens", which can be present in an emitted target sentence and which are "tokens that do not identify any source word in the source sentence".
3. Inventive step
3.1 As confirmed on page 6, lines 4 to 7, of the description of the application, the claimed translation system can be implemented as a computer program running on a computer. Hence, the subject-matter of claim 1 differs from a notorious general-purpose computer in features defining a computer program providing the functionality described in point 2. above. Such features contribute to inventive step only to the extent that they interact with the technical subject-matter of the claim to provide a technical effect.
3.2 In the present case, the board is unable to identify a technical effect achieved by the distinguishing features. In particular, the translation of text from a source language to a target language is a matter of linguistics and not a technical effect. This is so even if the computer program includes algorithmic aspects which are not directly based on linguistic concepts.
3.3 Citing a passage from decision T 1177/97, Reasons 3, the appellant argued that a computerised translation process was a technical application and conferred technical character to non-technical aspects. The board does not agree.
3.3.1 The passage cited by the appellant states that "[i]mplementing a function on a computer system always involves, at least implicitly, technical considerations and means in substance that the functionality of a technical system is increased" and adds that "[t]he implementation of the information and methods related to linguistics as a computerized translation process similarly requires technical considerations and thus provides a technical aspect to per se non-technical things such as dictionaries, word matching or to translating compound expressions into a corresponding meaning".
3.3.2 In opinion G 3/08, OJ EPO 2011, 10, Reasons 13.5, the Enlarged Board pointed out that "although it may be said that all computer programming involves technical considerations since it is concerned with defining a method which can be carried out by a machine, that in itself is not enough to demonstrate that the program which results from the programming has technical character; the programmer must have had technical considerations beyond 'merely' finding a computer algorithm to carry out some procedure".
Hence, merely finding a computer algorithm to implement an automated translation process does not render the resulting computer program technical (see also decisions T 598/14, Reasons 2.3, and T 2825/19, Reasons 5.3 and 5.4). The features of claim 1 indeed define the program merely in terms of a (high-level) algorithm.
3.4 In view of the above, the board concludes that the subject-matter of claim 1 lacks an inventive step over a notorious general purpose computer (Article 56 EPC).
4. Since the sole request on file cannot be allowed, the appeal is to be dismissed.
For these reasons it is decided that:
The appeal is dismissed.