T 2263/18 (Extraction method/FUJITSU) 11-10-2021
Download and more information:
Extraction method and device
Inventive step - main request and first auxiliary request (no)
Amendment after summons - exceptional circumstances
Amendment after summons - second auxiliary request (no)
I. The applicant (appellant) appealed against the decision of the examining division refusing European patent application No. 15156933.2, published as EP 2 921 973. The application claims a priority date of 17 March 2014.
II. The documents cited in the contested decision included:
D2: US 6,160,950, published on 12 December 2000.
III. The examining division refused the application for lack of inventive step in the subject-matter of the independent claims of a main request and of each of first to third auxiliary requests over the prior art disclosed in document D2. Furthermore, the examining division decided that the independent claims of the third auxiliary request were unclear.
IV. In its statement of grounds of appeal, the appellant requested that the decision be set aside and that a patent be granted on the basis of a new main request or a new first auxiliary request, both submitted with the grounds of appeal. The main request is identical to the third auxiliary request considered in the contested decision.
V. In a communication under Article 15(1) RPBA 2020 following the summons to oral proceedings, the board expressed, among other things, its provisional opinion that the subject-matter of claim 1 of the main request and of claim 1 of the first auxiliary request lacked inventive step in view of document D2. Moreover, the board stated that claim 1 of the main request was unclear.
VI. By a letter of 20 August 2021, the appellant maintained its requests, submitted a second auxiliary request and arguments, and informed the board that it would not be attending the oral proceedings.
VII. The board cancelled the oral proceedings.
VIII. The appellant requested that the decision under appeal be set aside and that a patent be granted on the basis of the claims of the main request or, in the alternative, on the basis of the claims of the first or second auxiliary request.
IX. Claim 1 of the main request reads as follows:
"An extraction program (50) that causes a computer (40) to execute processing comprising:
extracting (54) a portion of video data that corresponds to a continuation time as a scene of interest in a sports game, and determining the type of the scene of interest,
wherein the continuation time is a period of time during which a sound volume level of audio data included in footage including audio data and the video data, captured in a sports game, is detected to have a volume greater than a predetermined level indicative of a scene of interest, and
said extracting comprises comparing the continuation time with each of plural thresholds which have different lengths according to a respective type of scene of interest,
the processing further comprising:
detecting, as a start point, a point in time at which the change amount in the sound volume has exceeded a predetermined change amount threshold; and
determining, as the continuation time, a period from the start point in which the sound volume is continually at a level equal to or higher than a specific level based on the sound volume corresponding to the start point."
X. Claim 1 of the first auxiliary request reads as follows:
"An extraction program (50) that causes a computer (40) to execute processing comprising:
extracting (54) a portion of video data that corresponds to a continuation time as a scene of interest in a sports game, and determining the type of the scene of interest,
wherein the continuation time is a period of time during which a sound volume level of audio data included in footage including audio data and the video data, captured in a sports game, is detected to have a volume equal to or higher than a specific level indicative of a scene of interest, and
said determining comprises comparing the continuation time with each of plural thresholds which have different lengths corresponding to a respective type of scene of interest,
the processing further comprising:
detecting, as a start point, a point in time at which a change amount in the sound volume has exceeded a predetermined change amount threshold, wherein the change amount is a difference or ratio between an average value of the sound volume at respective sampling points in a first specific interval at the start point, and the average value of the sound volume at respective sampling points included in a second specific interval immediately before the first specific interval; and
determining, as the continuation time, a period from the start point in which the sound volume is continually at a level equal to or higher than the specific level, wherein the specific level is based on the sound volume corresponding to the start point."
XI. Claim 1 of the second auxiliary request differs from claim 1 of the first auxiliary request in that the following text has been added at the end of the claim:
"; generating (56) metadata including information of the determined type of the scene of interest, and the start point and an end point of corresponding continuation time, and appending the generated metadata to the video data".
1. The appellant's statement that it would not be attending the oral proceedings is to be understood, in the absence of any indication to the contrary, as a withdrawal of its request for oral proceedings (see T 3/90, OJ EPO 1992, 737, reasons 1, and the further decisions cited in Case Law of the Boards of Appeal, 9th edition, 2019, III.C.4.3.2). The decision is therefore taken without oral proceedings being held.
The invention
2. The application relates to the extraction of scenes from a captured video of a sports game. In known video distribution services it is sometimes desired to identify scenes of interest from a video, for example scenes in which a point is scored. It was known to recognise a scene of interest by detecting a sharp rise in sound volume in audio data of the video (description as published, paragraphs [0002] to [0004]).
The invention proposes to improve the known video extraction techniques for extracting a scene which is defined by a start point and a continuation time (duration of the scene) in a sports video.
Main request
3. Inventive step
3.1 The examining division objected to claim 1 under Article 56 EPC using document D2 as a starting point (see contested decision, point 14.4). It argued that the features distinguishing the subject-matter of claim 1 over D2 were as follows:
A The type of scene of interest is determined by comparing the continuation time against a variety of thresholds, each associated with a respective type of scene of interest.
B The parameter used to set the start point and the continuation time of a scene of interest is the change amount [in the sound volume].
According to the examining division, these distinguishing features had no synergistic effect and solved different problems. Feature A did not provide a technical contribution.
While the use of the change amount according to distinguishing feature B was different from the use of a sound volume threshold as disclosed in document D2, the determination of the scene of interest was based on a cognitive interpretation of what a scene was and thus on a non-technical requirement. Starting from document D2 and this non-technical requirement, the skilled person was able to implement the claimed solution without exercising inventive skill.
3.2 When submitting its complete case (statement of grounds of appeal, pages 5 to 10), the appellant did not contest that the distinguishing features were features A and B, but argued that each of these features was already inventive by itself.
The appellant argued that feature A solved the problem of identifying scene types by comparing the continuation time with each of a set of thresholds corresponding to different scene lengths. These scene lengths defined respective types of scenes of interest. Thus the claimed type determination was based on numerical comparisons, not on any "cognitive interpretation". The objective technical problem solved by feature A was how to identify scene types, and the solution was not obvious.
Feature B avoided the incorrect extraction of video portions due to incorrectly calibrated audio levels or if the sound level in an arena recorded in sports footage was continuously high. A relative rise in the sound level could be reliably detected by using the "change amount" of the sound level in detecting the start point. Thus using the "change amount" would contribute to accurate detection of the "continuation time" (statement of grounds of appeal, page 8). The extraction method disclosed in document D2 did not address the problems solved.
Finally, features A and B in combination provided the synergistic effect of improving the accuracy of identified scene types, which was a technical effect.
Consequently, the subject-matter of claim 1 was not obvious.
3.3 The board does not recognise the alleged synergistic technical effect. The board agrees that features A and B interact, but the identified scene types correspond to different types of scenes of a sports game (see for example description, paragraph [0021]: goal scenes, disappointment scenes, attack defense switch scenes, foul scenes) and thus relate to the non-technical cognitive content of the video. Hence the combined purpose of features A and B of identifying scene types and their starting points is non-technical.
Moreover, the fact that features A and B interact does not imply that they produce a synergistic effect. Rather, the overall effect is the result of aggregating these features: feature B concerns how the starting point and continuation time are obtained and feature A determines the scene type based on the continuation time. That the scene type is more accurate when the detection of the starting point and the continuation time is more accurate is the expected result of the aggregation of features A and B.
3.4 As the identification of types of scenes of a sports game is not a technical purpose and as the board does not see that feature A involves any "further technical considerations" (see opinion G 0003/08, OJ EPO 2011, 10, Reasons 13.5 and 13.5.1), the board agrees with the examining division that feature A does not contribute to solving a technical problem. Hence it does not enter into the assessment of inventive step (see decision T 154/04, OJ EPO 2008, 46, point 5 (F) of the reasons: "Non-technical features, to the extent that they do not interact with the technical subject matter of the claim for solving a technical problem, i.e. non-technical features 'as such', do not provide a technical contribution to the prior art and are thus ignored in assessing novelty and inventive step").
3.5 The definition of the scenes by means of characteristics of the audio signal volume is based on a heuristic rule reflecting non-technical considerations about the behaviour of spectators at sports games which is usually observed, namely that the sound level increases at time periods of particular interest for the public watching a sports game such as when a goal is scored in soccer. Such time periods then correspond to various types of scenes of interest in the video of the sports game. The fact that the implementation of such a heuristic rule in the system may involve the use of technical means for determining the change amount in the sound volume does not render the underlying heuristic itself technical (see for example decision T 1670/07 of 11 July 2013, reasons 9).
According to the established case law of the boards of appeal, when assessing inventive step in accordance with the problem/solution approach, an aim to be achieved in a non-technical field may legitimately appear in the formulation of the problem as part of the framework of the technical problem to be solved as a constraint that has to be met (see decisions T 641/00, OJ EPO 2003, 352; T 154/04, OJ EPO 2008, 46). Consequently, the heuristic rule may be added as a non-technical constraint to the objective technical problem to be solved.
The board agrees with the examining division that the implementation of the heuristic rule in the program known from document D2 with the aim of determining the type of scenes of interest was straightforward and could not be the basis for acknowledging inventive step. The skilled person was familiar with "change amounts" of sound levels and able to come up with the necessary concepts for implementation.
3.6 In view of the above, the board concludes that the subject-matter of claim 1 according to the main request lacks inventive step over document D2 (Article 56 EPC).
First auxiliary request
4. Claim 1 of the first auxiliary request differs from claim 1 of the main request in that it amends the detecting feature as follows (additions to claim 1 of the main request are shown in italics and deletions in strikethrough):
detecting, as a start point, a point in time at which a [deleted: the] change amount in the sound volume has exceeded a predetermined change amount threshold, wherein the change amount is a difference or ratio between an average value of the sound volume at respective sampling points in a first specific interval at the start point, and the average value of the sound volume at respective sampling points included in a second specific interval immediately before the first specific interval.
In addition, in the feature starting with "wherein the continuation time", the expression "a volume greater than a predetermined level" has been amended to "a volume equal to or higher than a specific level". In the feature of claim 1 of the main request starting with "said extracting comprises", the words "extracting" and "according" have been amended to "determining" and "corresponding", respectively. Finally, the expression "a specific level based on" in the last feature of claim 1 of the main request has been amended to "the specific level, wherein the specific level is based on".
5. Admission
5.1 The board shares the appellant's view that the set of claims according to the first auxiliary request is a legitimate attempt to overcome clarity objections in response to the contested decision that prima facie does not raise new issues. Consequently, the board admits this auxiliary request into the proceedings (Article 12(4) RPBA 2007).
6. Inventive step
6.1 As the first auxiliary request essentially clarifies the main request, the appellant did not submit specific arguments in favour of inventive step beyond those already submitted for the main request.
6.2 The board considers that its above objection under Article 56 EPC in relation to the main request also applies mutatis mutandis to claim 1 according to the first auxiliary request. In particular, as for the added details for detecting the start point, it was obvious to the skilled person to consider implementing a change amount for the sound level based on the average value of the sound volume at sampling points in an interval before and after the start point. The sound level may frequently change in a sports game recording, so a meaningful comparison of a change amount makes most sense by comparing sound level averages over two time intervals before and after a specific point in time. Hence these additional details do not go beyond routine work for the skilled person and are obvious.
Second auxiliary request
7. Claim 1 of the second auxiliary request adds the following feature C to claim 1 of the first auxiliary request:
- generating (56) metadata including information of the determined type of the scene of interest, and the start point and an end point of corresponding continuation time, and appending the generated metadata to the video data.
7.1 According to the appellant, the basis for feature C is the originally-filed claim 9 and the description, page 5, last paragraph to page 6, third full paragraph. As for inventive step, the appellant argued that feature C together with features A and B achieved the synergistic technical effect of facilitating easy access to a scene with a predetermined condition. In other words, appending the generated metadata to the video data made the scenes searchable and retrievable.
8. Admission
8.1 Under Article 13(2) RPBA 2020 any amendment to a party's appeal case made after notification of a summons to oral proceedings shall, in principle, not be taken into account unless there are exceptional circumstances, which have been justified with cogent reasons by the party concerned.
8.2 The second auxiliary request was filed after notification of the summons to oral proceedings. It adds to independent claim 1 feature C, which had not been present in any of the independent claims filed with the statement of grounds of appeal.
8.3 In its reply, the appellant argued that this amendment overcame concerns expressed in section 8 of the board's communication and did not give rise to any new objection or issue. Moreover, the amendment was based on dependent claim 7 of the first auxiliary request, so admitting the second auxiliary request was not detrimental to procedural economy.
8.4 In the present case, point 8 of the board's communication discussed inventive step for the main request. In point 8.4 of that communication, to which the appellant particularly referred, the board provided its preliminary opinion that it tended to agree with the examining division's assessment. Consequently, the board is not convinced that the appellant filed the second auxiliary request in reaction to an objection raised for the first time in the board's communication. In view of the above, the board sees no exceptional circumstances allowing the second auxiliary request to be admitted.
Moreover, the amendments made raise a fresh issue in that they introduce features from the description for the first time and the appellant argues that these added features bring about a new synergistic effect relating to the searchability of the video. Thus the amendment raises a fresh issue that would shift the discussion of inventive step considerably.
In view of the above, the board does not admit the second auxiliary request into the proceedings under Article 13(2) RPBA 2020.
9. Conclusion
Since none of the requests admitted into the appeal proceedings is allowable, the appeal is to be dismissed.
For these reasons it is decided that:
The appeal is dismissed.