T 0968/17 19-10-2021
Download and more information:
METHODS AND APPARATUS FOR MOTION SKIP MODE WITH MULTIPLE INTER-VIEW REFERENCE PICTURES
All requests, added subject-matter (yes)
All requests, novelty (yes)
All requests, inventive step (no)
I. The appeal is against the decision to refuse European patent application No. 08 794 873.3 published as international application WO 2009/020542 A1.
II. The decision under appeal cited the following documents:
D1:|Han-Suh Koo et al, "CE11: MVC Motion Skip Mode", Joint Video Team (JVT) of ISO/IEC MPEG & ITU-T VCEG (ISO/IEC JTC1/SC29/WG11 and ITU-T SG16 Q.6), JVT-V069, 22nd Meeting, Marrakesh, Morocco, 13-19 January 2007, XP030006877 |
D2:|US 2007/0109409 A1 |
D3:|WO 2008/108566 A1|
III. The decision was based on the grounds that the subject-matter of the claims of the sole request then on file extended beyond the content of the application as filed (Article 123(2) EPC) and lacked novelty (Article 54(1) EPC) over the disclosure of document D1 (Article 54(2) EPC) or document D3 (Article 54(3) EPC).
IV. The applicant ("appellant") filed notice of appeal. With the statement of grounds of appeal, the appellant filed amended claims according to a main request and three auxiliary requests. It provided a basis in the application as filed for the claims as well as arguments why the requests met the requirements of clarity and novelty.
The appellant also submitted that it trusted that the objections raised in the decision to refuse had been overcome and "[o]nly auxiliary" requested oral proceedings before any decision to refuse this application was taken (see statement of grounds of appeal, page 2, last two items).
V. On 26 November 2020, the board issued a summons to oral proceedings pursuant to Rule 115(1) EPC. The oral proceedings were scheduled to take place on 8 October 2021.
VI. By letter dated 3 December 2020, the appellant informed "the Examining Division" that it would not attend the oral proceedings scheduled for 8 October 2021. The appellant further requested a decision according to the state of the file.
VII. In a communication pursuant to Article 15(1) RPBA 2020, the board made the following comments.
(a) "The board understands the appellant's requests as being that the decision under appeal be set aside and a patent be granted on the basis of the claims of the main request, or, in the alternative, one of the auxiliary requests, all requests filed with the statement of grounds of appeal" (see point 2.5 of the board's communication).
(b) "If the appellant does not indicate otherwise before 8 September 2021, the board will assume that its statements in its letter dated 3 December 2020 ... are still valid. In that case, the board will cancel the oral proceedings and issue a decision in writing" (see section 3 of the board's communication; emphasis in the original text).
The board also introduced the following documents ex officio into the appeal proceedings on the basis of Article 114(1) EPC:
D4:|Anthony Vetro et al., "Joint Multiview Video Model (JMVM) 1.0", Joint Video Team (JVT) of ISO/IEC MPEG & ITU-T VCEG (ISO/IEC JTC1/SC29/WG11 and ITU-T SG16 Q.6), JVT-T208, 20th Meeting, Klagenfurt, Austria, 15-21 July 2006, XP030006640 |
D5:|Anthony Vetro et al., "Joint Multiview Video Model (JMVM) 5.0", Joint Video Team (JVT) of ISO/IEC MPEG & ITU-T VCEG (ISO/IEC JTC1/SC29/WG11 and ITU-T SG16 Q.6), JVT-X207, 24th Meeting, Geneva, CH, 29 June - 5 July 2007, XP030007206 |
D6:|Keng-Pang Lim et al., "Text Description of Joint Model Reference Encoding Methods and Decoding Concealment Methods", Joint Video Team (JVT) of ISO/IEC MPEG and ITU-T VCEG, JVT-O079, Busan, Korea, April 2005, XP030006022 |
The board gave the following preliminary opinion.
- None of the requests fulfilled the requirements of Article 123(2) EPC (see point 4.5 of the board's communication).
- The subject-matter of claim 1 of the main request was new over the disclosure of document D1 (see point 6.5 of the board's communication).
- The subject-matter of claim 1 of the main request lacked inventive step in view of the combined disclosure of documents D1 and D4 and the common general knowledge of the person skilled in the art (see point 7.6 of the board's communication).
- The subject-matter of claim 1 of the first auxiliary request lacked inventive step in view of the combined disclosure of documents D1, D4, D5 and D6 and the common general knowledge of the person skilled in the art (see point 8.9 of the board's communication).
- The subject-matter of claim 1 of the second and third auxiliary requests lacked inventive step for the same reasons as given for the main request and the first auxiliary request, respectively (see point 9.3 of the board's communication).
VIII. The appellant did not submit any comment in reply to the board's communication.
IX. On 24 September 2021, the appellant was notified that the oral proceedings appointed for 8 October 2021 had been cancelled.
X. In view of points VII.(a) and VIII. above, it appears from the file that the appellant's final requests are that the decision under appeal be set aside and that a patent be granted on the basis of the claims of the main request filed with the statement of grounds of appeal or, alternatively, on the basis of the claims of one of the auxiliary requests filed with the statement of grounds of appeal.
XI. Claim 1 of the main request reads as follows:
"An apparatus, comprising an encoder (100) for encoding an image block relating to multi-view video content, said apparatus comprising means for:
i) selecting, for the image block, an inter-view reference picture list from a set of two inter-view reference picture lists, an inter-view reference picture from said selected inter-view reference picture list, and a disparity vector from among a set of disparity vectors corresponding to said selected inter-view reference picture, the inter-view reference picture and the disparity vector being selected based on neighboring macroblocks with respect to the image block;
ii) identifying a corresponding inter-view block based on said selected inter-view reference picture list, said selected inter-view reference picture, and said selected disparity vector; and
iii) extracting motion information for the image block from the corresponding inter-view block;
wherein said selecting comprises, when more than one neighboring macroblock use inter-view prediction, selecting, from inter-view reference pictures associated with said neighboring macroblocks, an inter-view picture with the smallest reference index and its associated disparity vector."
XII. Claim 1 of the first auxiliary request reads as follows:
"An apparatus, comprising an encoder (100) for encoding an image block relating to multi-view video content, said apparatus comprising means for:
810) finding the best coding mode (MODEbest) among traditional coding modes for encoding the image block, and passing the control to means 815;
815) determining whether or not the image block is located in a P or B slice, if so, then passing the control to means 820, otherwise passing the control to means 865;
820) determining if there is a neighboring macroblock that uses inter-view prediction, if so, passing the control to means 870, otherwise passing the control to
means 865;
870) finding a neighboring macroblock associated with an inter-view picture belonging to a first inter-view pictures list (LIST0), and passing the control to means 825;
825) using a disparity vector (DV) and an inter-view reference picture associated with said found neighboring macroblock to find a corresponding macroblock to the image block, and passing the control to means 830;
830) determining whether or not the corresponding macroblock is an intra macroblock, if so, passing the control to means 875, otherwise passing the control to
means 845;
875) finding a neighboring macroblock associated with an inter-view picture belonging to a second inter-view pictures list (LIST1) and passing the control to
means 835;
835) using a disparity vector (DV) and an inter-view reference picture associated with said found neighboring macroblock to find a corresponding macroblock to the image block and passing the control to means 840;
840) determining whether or not the corresponding macroblock is an intra macroblock, if so, passing the control to means 865, otherwise passing the control to means 845;
845) extracting motion information for motion skip mode (MODEms), and passing the control the (sic) means 850;
850) getting the rate-distorsion (RD) cost (COSTms) with the motion skip mode and passing the control to means 855;
855) determining whether or not the rate-distorsion cost (COSTms) with the motion skip mode is smaller than the rate-distorsion cost with the best coding mode(MODEbest), if so, passing the control to the means 860, otherwise passing the control to the means 865;
860) setting the best coding mode to the motion skip mode, and passing the control to the means 865;
865) encoding the image block with the best coding mode;
wherein when more than one neighboring macroblock use inter-view prediction (820), finding a neighboring macroblock (870, 875) comprises selecting, from inter-view reference pictures associated with the determined neighboring macroblocks (820), an inter-view picture with the smallest reference index and, the found neighboring macroblock is associated with said selected inter-view reference picture, and
wherein the inter-view reference picture and the disparity vector being selected based on neighboring macroblocks with respect to the image block."
XIII. Claim 1 of the second auxiliary request (respectively, claim 1 of the third auxiliary request) corresponds to claim 1 of the main request (respectively, claim 1 of the first auxiliary request), except for the addition of the following text before the full stop:
", and
wherein if more than one neighboring macroblock share the same inter-view reference picture, then the median disparity vector from the corresponding macroblocks will serve as the disparity vector of the image block".
XIV. The appellant's arguments, as far as relevant for this decision, can be summarised as follows.
(a) The amendments according to claim 1 of the main request had a basis in the following passages of the application as filed: page 15, line 17 to page 16, line 19; page 14, line 24 to page 16, line 16; and page 15, lines 8 to 10 (see section 1 on pages 3 and 4 of the statement of grounds of appeal).
(b) The subject-matter of claim 1 of the main request was new over the disclosure of document D1. D1 did not disclose the selection of an inter-view reference picture list from a set of inter-view reference picture lists or the selection of an inter-view reference picture from the selected inter-view reference picture list (see statement of grounds of appeal, page 6, paragraph starting with "Though D1 mentions inferring reference indices"). D1 was "directed to a current view and ONLY one neighboring view", i.e. "the immediately neighboring view" (see statement of grounds of appeal, page 6, last full paragraph and page 7, paragraph starting with "Page 1, section 1, D1 discloses"; emphasis in the original text). Moreover, D1 failed to disclose the last feature of claim 1, i.e. "wherein said selecting comprises, when more than one neighboring macroblock use inter-view prediction, selecting, from inter-view reference pictures associated with said neighboring macroblocks, an inter-view picture with the smallest reference index and its associated disparity vector" (see statement of grounds of appeal, page 7, penultimate item).
(c) The claims of the first auxiliary request were based on the "initial claims", and the added steps were supported by the passage from page 14, line 24 to page 17, line 20 of the specification. The remarks of section 1 of the statement of grounds of appeal also applied (see section 4 on page 8 of the statement of grounds of appeal).
(d) The additional feature in claim 1 of the second auxiliary request had a basis on page 15, in lines 10 to 12 of the application as filed. The remarks of section 1 of the statement of grounds of appeal also applied (see section 7 on page 9 of the statement of grounds of appeal).
(e) The amendments according to the third auxiliary request did not add subject-matter which extended beyond the content of the application as filed. The remarks in sections 4 and 7 of the statement of grounds of appeal applied (see section 10 on page 10 of the statement of grounds of appeal).
1. The appeal is admissible.
2. All requests, added subject-matter (Article 123(2) EPC)
2.1 A European patent application must not be amended in such a way that it contains subject-matter which extends beyond the content of the application as filed (Article 123(2) EPC).
This means that an amendment can only be made within the limits of what the person skilled in the art would derive directly and unambiguously, using common general knowledge, and seen objectively and relative to the date of filing, from the whole disclosure of the description, claims and drawings of the application as filed (see Case Law of the Boards of Appeal of the European Patent Office, 9th edition 2019 ("Case Law"), II.E.1.1).
2.2 It is established case law that an "intermediate generalisation", i.e. a generalisation of a particular embodiment disclosed in the original application which is still more specific than the original definition of the invention, is justified only in the absence of any clearly recognisable functional or structural relationship among the features of the particular embodiment or if the extracted features are not inextricably linked with the remaining features of the particular embodiment (see Case Law, II.E.1.9).
2.3 Claim 1 of all requests refers to "neighboring macroblock(s)" without specifying whether these are spatially or temporally neighbouring macroblocks.
2.3.1 All the passages of the application as filed provided by the appellant as a basis for the subject-matter of claim 1 of its requests (see point XIV. above items(a), (c), (d) and (e)) are part of the "second exemplary embodiment" described from page 14, line 24 to page 17, line 20 of the description as filed. Indeed, this embodiment is the only disclosed embodiment in which an inter-view picture and a disparity vector are selected based on neighbouring macroblocks. In this embodiment, the neighbouring macroblocks from which disparity information is derived are the three spatially adjacent macroblocks shown in Figure 7 (see "A", "B" and "C").
2.3.2 Claim 1 of all requests represents an intermediate generalisation of this embodiment since it does not specify whether the neighbouring macroblocks are spatially or temporally neighbouring macroblocks.
2.3.3 In the board's view, the person skilled in the art would not have derived from this embodiment that a "neighboring macroblock" may be a temporally neighbouring macroblock. The content of the multi-view video may differ noticeably from one time instant to the next. Therefore, the board is not convinced that it is implicit that the current macroblock may derive its temporal motion vector from a temporally neighbouring macroblock.
2.3.4 It follows from the above that claim 1 of all requests represents an unallowable intermediate generalisation of the disclosure of the application as filed.
2.4 Additionally, claim 1 of the main request and the second auxiliary request - which relates to an apparatus comprising an encoder - specifies neither the function of the two inter-view reference picture lists nor what it means for an inter-view reference picture list to be selected from these two lists.
2.4.1 In the encoding method of the "second exemplary embodiment" described in connection to Figure 8 (see the passage from page 15, line 17 to page 16, line 19 of the description as filed referred to by the appellant in the statement of grounds of appeal; see point XIV.(a) above)), the two reference picture lists "LIST0" and "LIST1" are considered in turn. LIST1 is only considered if the reference index and the disparity vector obtained after considering LIST0 refers to an intra coded macroblock (see Figure 8, reference signs 870, 825, 830 and 875). Thus, in this embodiment, LIST1 has the function of a fallback list if LIST0 does not provide a usable disparity vector.
2.4.2 In the board's view, the inter-relationship between the two lists is essential to the solution provided by this embodiment. The person skilled in the art is not taught how two reference picture lists may be used otherwise when considering information of neighbouring macroblocks.
2.4.3 Thus, claim 1 of the main request and the second auxiliary request also contravenes Article 123(2) EPC because it does not comprise the features identified under point 2.4.1 above.
2.5 In view of the above, the board concludes that none of the requests fulfils the requirements of Article 123(2) EPC.
3. Main request, novelty (Article 54(1),(2) EPC)
3.1 An invention is to be considered new if it does not form part of the state of the art (Article 54(1) EPC).
3.2 Document D1 discloses an apparatus comprising an encoder for encoding an image block relating to multi-view video content (see title, "MVC" (also known as Multi-view Video Coding))
The disclosed apparatus comprises a means for selecting, for the image block, an inter-view reference picture list from a set of reference picture lists (see D1, section 1, "Current MVC model adds a picture of the neighboring view of a current picture to reference picture lists ... the motion information ... is inferred from the corresponding macroblock in the neighboring view"), an inter-view reference picture from the selected inter-view reference picture list (see D1, section 1, "the motion information ... is inferred from the corresponding macroblock in the neighboring view") and a disparity vector from a set of disparity vectors corresponding to the selected inter-view reference picture (see D1, section 2.1, "When there are more than two neighboring partitions referring to the picture of neigboring view ... each component of the disparity vector of current macroblock is given by the median operation of corresponding components of the disparity vector of neighboring partitions"). The disparity vector is selected based on neighbouring macroblocks with respect to the image block (see D1, Figure 1). The inter-view reference picture depends on whether the selected disparity vector points to a macroblock having motion information (see D1, paragraph bridging pages 2 and 3), i.e. it is also "selected based on neighboring macroblocks with respect to the image block" as specified by claim 1 (see lines 8 to 10).
The "Current MVC model" mentioned in section 1 of document D1 is the model disclosed in document D4 (see D1, section 6, reference [1]). Document D4 discloses the use of two reference picture lists (in accordance with the H.264/MPEG-4 AVC standard) (see subclauses G.7.3.1, G.7.4.1 and G.8.2 on pages 21 to 23).
The disclosed apparatus also comprises means for identifying a corresponding inter-view block based on the selected inter-view reference picture list, the selected inter-view reference picture and the selected disparity vector, as well as a means for extracting motion information for the image block from the corresponding inter-view block (see section 1, "the motion information ... is inferred from the corresponding macroblock in the neighboring view").
3.3 The board disagrees with the appellant that document D1 does not disclose the selection of an inter-view reference picture list from a set of inter-view reference picture lists or the selection of an inter-view reference picture from the selected inter-view reference picture list (see point XIV.(b) above). To access the neighbouring view stored in the decoded picture buffer, the apparatus of document D1 must access the index to the picture that was inserted into one of the reference picture lists (see document D4, subclause G.8.2, explaining how the reference picture lists are constructed). This implies "selecting" (one way or another) one of the two reference picture lists and an inter-view reference picture (i.e. the neighbouring view) from the selected inter-view reference picture list.
3.4 The board also disagrees that "D1 is directed to a current view and ONLY one neighboring view", i.e. "the immediately neighboring view" (see point XIV.(b) above; emphasis in the original text). The paragraph bridging pages 2 and 3 specifies that other views are exploited if the corresponding macroblock in the neighbouring view "is under intra prediction mode".
3.5 The above comments notwithstanding, the board agrees with the appellant that the apparatus defined in claim 1 differs from the apparatus disclosed in document D1 in that "said selecting comprises, when more than one neighboring macroblock use inter-view prediction, selecting, from inter-view reference pictures associated with said neighboring macroblocks, an inter-view picture with the smallest reference index and its associated disparity vector" (see claim 1, lines 16 to 19) and, therefore, that the subject-matter of claim 1 of the main request is new over the disclosure of document D1 (see point XIV.(b) above).
4. Main request, inventive step (Article 56 EPC)
4.1 An invention is to be considered to involve an inventive step if, having regard to the state of the art, it is not obvious to a person skilled in the art (Article 56 EPC).
4.2 Document D1 may be considered the closest state of the art in the context of the problem and solution approach (Case Law, I.D.2).
4.3 As pointed out under point 3.2, document D1 is to be read taking into account the "Current MVC model" disclosed in document D4 (see D1, sections 1 and 6). Subclause G.7.3.1 of document D4 teaches signalling - for each view and each of the two reference picture lists - the indices of the views that can be used for inter-view prediction. According to subclause G.8.2., these indices are appended to the reference picture lists when constructing the reference picture lists.
4.4 It would have been obvious for the person skilled in the art to order the views from closest to furthest in the arrays specified in subclause G.7.3.1 and, consequently, to associate the immediately neighbouring view with the smallest reference index in the reference picture lists in the context of subclause G.8.2.
4.5 A person skilled in the art faced with the problem of implementing the apparatus disclosed in document D1 would therefore have arrived at an apparatus falling within the scope of claim 1.
4.6 The appellant has not submitted any arguments why the distinguishing feature identified under point 3.5 above involved an inventive step.
4.7 In view of this, the board concludes that the subject-matter of claim 1 of the main request lacks inventive step in view of the combined disclosure of documents D1 and D4 and the common general knowledge of the person skilled in the art.
5. First auxiliary request, inventive step (Article 56 EPC)
5.1 If the features or sets of features of a claim are a mere aggregation of these features or sets of features which are not functionally interdependent, i.e. do not mutually influence each other to achieve a technical success over and above the sum of their respective individual effects, it is to be established whether each set of features is separately obvious in light of the state of the art (see Case Law, I.D.9.2.2).
5.2 It follows from the analysis made under point 3.2 above that document D1 discloses steps 820, 840, 845, 865 and the very last feature ("wherein the inter-view reference picture and the disparity vector being selected based on neighboring macroblocks with respect to the image block") of claim 1 of the first auxiliary request. Document D1 also implicitly discloses "determining whether or not the image block is located in a P or B slice" (step 815) since, by definition, motion estimation (and, thus, the motion skip mode) can only be carried out in a P or a B slice.
5.3 In view of this, the board finds that the apparatus of claim 1 of the first auxiliary request differs from the apparatus disclosed in document D1 in that it comprises the following features.
(a) The motion skip mode is set as the best coding mode if it leads to a smaller rate-distortion cost than the other modes called "traditional coding modes" (see claim 1, steps 810, 850, 855, 860)("Feature (a)").
(b) Within the motion skip mode, a motion vector is first searched for in an inter-view reference picture of a first inter-view reference picture list. If the "corresponding macroblock is an intra macroblock", a motion vector is searched for in an inter-view reference picture of a second inter-view reference picture list (see claim 1, steps 870, 825, 830, 875, 835)("Feature (b)").
(c) When more than one neighbouring macroblocks use inter-view prediction, finding a neighbouring macroblock comprises selecting, from inter-view reference pictures associated with the determined neighbouring macroblocks, an inter-view picture with the smallest reference index (see claim 1, penultimate feature) ("Feature (c)").
5.4 Feature (a) defines an algorithm for selecting a best coding mode among several. Feature (b) increases the number of corresponding macroblocks considered when searching for a motion vector. Feature (c) defines a heuristic for selecting the inter-view picture from which the motion vector is to be derived if available.
5.5 Features (a), (b) and (c) do not mutually influence each other to achieve a technical success over and above the sum of their respective individual effects. Hence, it is to be established whether each of them is separately obvious in light of the state of the art (see point 5.1 above).
5.6 Selecting the H.264 coding mode leading to the smallest rate-distortion cost is disclosed in section 2.1 of document D6 (see point 2.1.1.1). The "SKIP" and "DIRECT" modes are mentioned in point 2.1.2.2.4 (see equations (2-31) and (2-32)). This disclosure renders feature (a) obvious.
5.7 Reference is also made to document D5, the latest Joint Multiview Video Model before the priority date of the application in hand. Section 1.3 discloses a motion skip mode deriving motion information from previously coded inter-view pictures. The last paragraph specifies that "[t]his mode derives motion information from L0 reference first, and if it is not available, L1 reference is exploited instead according to the specified view dependency". This renders feature (b) obvious.
5.8 Feature (c) corresponds to the distinguishing feature identified under point 3.5 above considered obvious in section 4.
5.9 In view of the above, the board concludes that the subject-matter of claim 1 of the first auxiliary request lacks inventive step in view of the combined disclosure of documents D1, D4, D5 and D6 and the common general knowledge of the person skilled in the art.
6. Second and third auxiliary requests, inventive step (Article 56 EPC)
6.1 In comparison to claim 1 of the main request (respectively, the first auxiliary request), claim 1 of the second auxiliary request (respectively, the third auxiliary request) further specifies that "if more than one neighboring macroblock share the same inter-view reference picture, then the median disparity vector from the corresponding macroblocks will serve as the disparity vector of the image block".
6.2 This feature is not an additional distinguishing feature since it is disclosed in section 2.1 of document D1.
6.3 Therefore, the subject-matter of claim 1 of the second and third auxiliary requests lacks inventive step for the same reasons as given under sections 4. and 5., respectively.
7. Conclusion
7.1 Since none of the appellant's requests is allowable, the appeal must be dismissed.
For these reasons it is decided that:
The appeal is dismissed.