T 0606/21 28-02-2023
Download and more information:
EVALUATION FRAMEWORK FOR PREDICTED TRAJECTORIES IN AUTONOMOUS DRIVING VEHICLE TRAFFIC PREDICTION
I. The applicant's appeal is directed against the decision of the examining division to refuse European patent application No. 17847781.6. The examining division refused the application because, for the subject-matter of claim 1 of the then main request and the then second auxiliary request, the invention was not disclosed in the application in a manner sufficiently clear and complete for it to be carried out by a person skilled in the art, claim 1 of the then main request, the then first auxiliary request and the then second auxiliary request comprised added subject-matter and claim 1 of the then main request and the then first auxiliary request was not clear.
II. With the statement setting out the grounds of appeal, the appellant requested that the appealed decision be set aside and that a patent be granted on the basis of the claims of the sole request filed with the statement of grounds of appeal.
As an auxiliary measure oral proceedings were requested.
III. In a communication annexed to the summons to oral proceedings, the board expressed its provisional opinion, that the application did not disclose the claimed invention in a manner sufficiently clear and complete for it to be carried out by a person skilled in the art, contrary to Article 83 EPC.
IV. By letter dated 29 December 2022, the appellant put forward arguments for sufficiency of disclosure and requested that the appealed decision be set aside and that a patent be granted "based on the enclosed amended set of claims" (see page 5, penultimate paragraph). As an auxiliary request the appellant requested that the case be remitted to the examining division for further prosecution, in particular if the board would see potential other issues beyond Article 83 EPC.
V. By letter dated 21 February 2023, the appellant withdrew its request that oral proceedings be held.
VI. Oral proceedings took place on 28 February 2023.
The chairman noted that no one was present for the duly summoned appellant. With regard to the appellant's requests on file, the chairman observed that no amended set of claims had been submitted by the letter of 29 December 2022 and that, moreover, the appellant had stated on page 1, second paragraph of this letter that "[t]he application documents are maintained unamended". He further noted that the board understood from the appellant's written submissions on file that the appellant had requested in writing that the decision under appeal be set aside and that a patent be granted on the basis of the claims of the request filed with the statement of grounds of appeal (main request) and, as an auxiliary request, that the case be remitted to the examining division for further prosecution, "in particular if the Board of Appeal would see potential other issues beyond Art. 83 EPC, the only issued [sic] discussed in the preliminary opinion of the board".
At the end of the oral proceedings the chairman announced the board's decision.
VII. Claim 1 as filed with the grounds of appeal reads as follows:
"A computer-implemented method for evaluating predictions of trajectories by autonomous driving vehicles, characterized in that the method comprises:
generating a Deep Neural Network, DNN, model, comprising:
receiving (701) a first predicted trajectory of an object that was predicted based on first perception data of the object;
extracting (702) predicted features from the first predicted trajectory;
receiving (703) an actual trajectory that the object actually travelled;
extracting (704) actual features of the actual trajectory for at least some trajectory points selected from a plurality of trajectory points of the actual trajectory, wherein both the extracted predicted features and the extracted actual features comprise a set of physical attributes and a set of trajectory related attributes, the physical attributes including a relative heading direction and a speed of the trajectory point; and
training and generating (705) the DNN model based on the predicted features and the actual features to generate a first similarity score representing a similarity between the first predicted trajectory and the actual trajectory, wherein the first similarity score represents a similarity, modeled by the DNN model, between the known predicted trajectory and the known actual trajectory, the first similarity score closer to a first predetermined value indicating the predicted trajectory is similar to the corresponding known actual trajectory, and the first similarity score closer to a second predetermined value indicating the predicted trajectory being dissimilar to the corresponding known actual trajectory;
comparing the first similarity score with an expected similarity score; and
in response to the generated first similarity score being not within the expected similarity score, iteratively training the DNN model until the generated first similarity score is within the expected similarity score;
receiving (801) a second predicted trajectory of the object that was generated using a prediction method based on second perception data perceiving the object within a driving environment surrounding an Autonomous Driving Vehicle, ADV;
for at least some trajectory points selected from a plurality of trajectory points of the second predicted trajectory, extracting (802) a plurality of features from the selected trajectory points, wherein the extracted features of each trajectory point comprise a set of physical attributes and a set of trajectory related attributes, the physical attributes including a relative heading direction and a speed of the trajectory point;
applying (803) the trained DNN model to the extracted features from the selected trajectory points of the second predicted trajectory to generate a second similarity score, wherein the second similarity score indicates whether the second predicted trajectory is more likely close to an actual trajectory, the object likely moves in the near future; and
determining (804) an accuracy of the prediction method based on the second similarity score."
1. The appeal is admissible.
2. Subject-matter of claim 1 - sufficiency of disclosure (Article 83 EPC)
2.1 The examining division considered that the application did not clearly disclose how the DNN model could provide a reliable output at the time of testing because it was fed with an incomplete input. The input at the time of testing only included the predicted trajectory, not the actual trajectory (see contested decision, section 13).
2.2 In the appellant's view the application clearly disclosed the invention. It was evident that the similarity score output at the testing/evaluation stage represented the similarity between the unknown predicted trajectory and one or more known actual trajectories in the training set of trajectories. The appellant referred to the description, paragraphs [0053] and [0065] and concluded that the term "known trajectories" referred to prior trajectories to be utilized as a training set of trajectories for the purpose of training a DNN model. Thus it was evident that the similarity score output at the testing/evaluation stage represented the similarity between the unknown predicted trajectory and one or more known actual trajectories in the training set of trajectories. Because the known actual trajectories had been input into the DNN model at the training stage, the trained DNN model could output the similarity score by merely inputting the predicted trajectory, e.g., unknown (future) predicted trajectory, without inputting the known actual trajectories again. Thus, the similarity score indicated whether the predicted trajectory was more likely to be close to the actual trajectory that the corresponding object was likely to move in the near future, i.e., the similarity score indicated whether the predicted trajectory was more likely to be close to a possible actual trajectory in the future (see grounds of appeal, section 2).
The applicant further submitted that claim 1 clearly defined the generation and training of the DNN model, and that during the training, the actual trajectory had been input to the DNN, so that it could be readily understood that during the testing or evaluation stage, when the trained DNN model was used to generate the second similarity score, the previously input actual trajectory would be used. Paragraph [0065] of the description clearly disclosed that the generated similarity score represented the similarity between the predicted trajectory and one or more prior actual trajectories. In paragraph [0053] of the description and Figure 4, it was disclosed that the collected actual trajectories were stored as "known actual trajectories" which were used as a training set. During the evaluation stage, the system 400, having stored the one or more prior actual trajectories, used them to evaluate the predicted trajectories to generate the second similarity score. It was therefore evident to the skilled person that the one or more prior actual trajectories referred to the known actual trajectories input during the training stage. Any method could be used to select the one or more actual trajectories from the variety of trajectories input during the training phase which could also include selecting all prior actual trajectories. Further, it would be reasonable for the person skilled in the art to select the prior actual trajectory with the highest similarity score for use in determining of the accuracy of the prediction method (see letter of 29 December 2022, section 1).
2.3 The board does not share the appellant's view. The method of claim 1 comprises two main stages. The first stage relates to the generation of a deep neural network, DNN, and the training of this DNN by inputting pairs of trajectories in order to analyse the similarity of these trajectories and to improve the accuracy of this analysis. The second stage refers to a testing or evaluation stage, in which a second predicted trajectory of the object is input. The board considers that there is a lack of disclosure regarding the second stage, the testing and evaluation stage. This phase defines that a second predicted trajectory of the object is received, that features of points of the second predicted trajectory are extracted and that the trained DNN model is applied to the extracted features of the second predicted trajectory. According to claim 1, the trained DNN model then generates a second similarity score, the second similarity score indicating whether the second predicted trajectory is more likely to be close to an (unspecified) actual trajectory along which the object is likely to move in the near future. However, claim 1 does not define where this unspecified actual trajectory comes from, how it is generated or how it is input. Even if the trained DNN model can output the similarity score representing a similarity between the unknown predicted trajectory and one or more known actual trajectories from the training stage by merely inputting the predicted trajectory without inputting the known actual trajectories from the training stage again, as the appellant suggests, it is not clear how it is possible to generate the defined second similarity score. Claim 1 does not define that the trained DNN model generates a second similarity score indicating whether the second predicted trajectory is more likely to be close to one or more prior actual trajectories or to known actual trajectories. On the contrary. According to claim 1, the second similarity score indicates whether the second predicted trajectory is more likely to be close to an actual trajectory, the object likely moves in the near future. The board does not consider an actual trajectory that the object likely moves in the near future to be equivalent to a prior actual trajectory stored previously in the training stage. The description does also not explain, neither in paragraphs [0053] and [0065], nor elsewhere, how the "actual trajectory, the object likely moves in the near future" is received at the evaluation stage, or that it corresponds to a known actual trajectory, as the appellant suggests.
2.4 The board therefore concludes that the invention defined in claim 1 is not disclosed in a manner sufficiently clear for it to be carried out by the skilled person, contrary to Article 83 EPC.
3. Since the application does not meet the requirements of Article 83 EPC, the appeal has to be dismissed.
For these reasons it is decided that:
The appeal is dismissed.