Informationen
Diese Entscheidung ist nur auf Französisch verfügbar.
T 0353/16 11-02-2021
Download und weitere Informationen:
VIDEO ENCODING DEVICE AND METHOD, VIDEO ENCODING PROGRAM, AND RECORDING MEDIUM CONTAINING THE PROGRAM
Summary of Facts and Submissions
I. The appeal is against the decision to refuse European patent application No. 08 791 042.8, published as international application WO 2009/011279 A1.
II. The documents cited in the decision under appeal included the following:
D1:|JP 2007 158430 A |
D3:|Gary Sullivan, "Seven Steps Toward a More Robust Codec Design", Joint Video Team (JVT) of ISO/IEC MPEG & ITU-T VCEG (ISO/IEC JCT1/SC29/WG11 and ITU-T SG16 Q.6), 3rd Meeting, Fairfax, Virginia, 6-10 May 2002, JVT-C117, XP030005228|
The decision also referred to the translation of document D1 into English, annexed to the communication of the examining division dated 10 April 2015, in turn annexed to the summons to oral proceedings. This translation will be referred to below as document T1.
III. The decision was based on the ground that the subject-matter of claim 1 of the main request and of the auxiliary request then on file lacked inventive step (Article 56 EPC) over the combined disclosure of documents D3 and D1.
IV. The applicant ("appellant") filed notice of appeal. With the statement of grounds of appeal, it requested that the decision under appeal be set aside and that a European patent be granted on the basis of the claims of the main request or of the auxiliary request forming the basis of the decision under appeal. It also submitted reasons why the subject-matter of claim 1 of both requests involved an inventive step.
V. On 13 May 2020, a summons to oral proceedings was issued. In a communication under Article 15(1) of the Rules of Procedure of the Boards of Appeal 2020 (RPBA 2020, OJ EPO 2019, A63) dated 23 July 2020, the board introduced the following document ex officio into the appeal proceedings on the basis of Article 114(1) EPC:
D4:|ITU-T Recommendation H.264, "Advanced video coding for generic audiovisual services", March 2005 |
A copy of Annex A of document D4 was annexed to the board's communication.
The board gave its preliminary opinion that (i) claim 1 of both requests and claim 2 of the main request lacked clarity (Article 84 EPC), (ii) the subject-matter of claim 1 of the main request lacked inventive step (Article 56 EPC) over the disclosure of document D1 combined with the common general knowledge of the person skilled in the art, and (iii) the subject-matter of claim 1 of both requests lacked inventive step (Article 56 EPC) in view of the conventional video encoding apparatus referred to in paragraphs [0048] and [0049] of the application combined with the disclosure of document D1 and the common general knowledge of the person skilled in the art.
Furthermore, the board invited the appellant to provide a copy of a document disclosing the conventional video encoding apparatus referred to in paragraphs [0048] and [0049] of the application.
VI. By letter dated 15 December 2020, the appellant requested that the oral proceedings scheduled for 11 February 2021 be held by videoconference.
VII. By letter dated 17 December 2020, the appellant re-filed the claims of the main request and of the auxiliary request (which it renamed "first auxiliary request"). The appellant also filed claims according to a new second auxiliary request and a new third auxiliary request.
The appellant provided arguments as to why the clarity objections raised by the board in its preliminary opinion were not justified and why the subject-matter of the independent claims of all the requests on file involved an inventive step.
The appellant also filed a copy of the following document, as well as a translation into English:
D5:|Sakae Okubo et al, "H.264/AVC Textbook", First Edition, Impress Corporation, 11 August 2004, page 44 |
VIII. By a communication dated 19 January 2021, the registrar of the board informed the appellant that the oral proceedings scheduled for 11 February 2021 would be held by videoconference.
IX. On 11 February 2021, the oral proceedings before the board were held by videoconference.
The appellant's final requests were that the decision under appeal be set aside and that a European patent be granted on the basis of the claims of the main request or, alternatively, on the basis of the claims of one of the first to third auxiliary requests, all requests filed by letter dated 17 December 2020.
At the end of the oral proceedings, the chairwoman announced the board's decision.
X. Claim 1 of the main request reads as follows:
"A video encoding apparatus for applying orthogonal transformation to a prediction error signal between a video signal of an encoding target area and a predicted signal for the video signal, and quantizing an obtained orthogonal transformation coefficient by using a preset quantization step size so as to encode the coefficient, the apparatus comprising:
a computation device that computes a prediction error power which is a power of the prediction error signal;
a determination device that receives the prediction error power computed by the computation device, the preset quantization step size, and an upper limit of an amount of code generated for the encoding target area, and determines whether or not an amount of code generated when performing quantization using the preset quantization step size exceeds the upper limit; and
a change device that changes an encoding process based on a result of the determination by the determination device,
wherein the determination device computes a permissive power for the prediction error power based on the upper limit and the preset quantization step size, and compares the permissive power with the prediction error power computed so as to determine whether or not the amount of code generated when performing the quantization using the preset quantization step size exceeds the upper limit; and
when it is determined that the amount of code generated when performing the quantization using the preset quantization step size exceeds the upper limit, the change device changes the quantization step size of the quantization for the encoding target area from the preset quantization step size to a quantization step size by which the amount of code generated for the encoding target area does not exceed the upper limit."
XI. Claim 1 of the first auxiliary request reads as follows (features added to or deleted from claim 1 of the main request are underlined or crossed out, respectively):
"A video encoding apparatus for applying orthogonal transformation to a prediction error signal between a video signal of an encoding target area corresponding to a macroblock and a predicted signal for the video signal, and quantizing an obtained orthogonal transformation coefficient by using a preset quantization step size set from an amount of generated code and a bit rate so as to encode the coefficient, the apparatus comprising:
a computation device that computes a prediction error power which is a power of the prediction error signal;
a determination device that receives the prediction error power computed by the computation device, the preset quantization step size, and an upper limit of an amount of code generated for the encoding target area, and determines whether or not an amount of code generated when performing quantization using the preset quantization step size exceeds the upper limit; and
a change device that changes an encoding process based on a result of the determination by the determination device,
wherein the determination device computes a permissive power for the prediction error power based on the upper limit and the preset quantization step size, and compares the permissive power with the prediction error power computed so as to determine whether or not the amount of code generated when performing the quantization using the preset quantization step size exceeds the upper limit; [deleted: and]
when it is determined that the amount of code generated when performing the quantization using the preset quantization step size exceeds the upper limit, the change device changes the quantization step size of the quantization for the encoding target area from the preset quantization step size to a quantization step size by which the amount of code generated for the encoding target area does not exceed the upper limit[deleted: .];
and wherein the determination device computes the permissive power for the prediction error power by setting variables of a function, which are the upper limit and the quantization step size, to the values of the upper limit and the quantization step size, where the value of the function is the permissive power, or wherein the determination device computes the permissive power for the prediction error power by referring to a table in which a relationship between data values of the upper limit, the quantization step size, and the permissive power is defined."
XII. Claim 1 of the second auxiliary request reads as follows (features added to claim 1 of the main request are underlined):
"A video encoding apparatus for applying orthogonal transformation to a prediction error signal between a video signal of an encoding target area and a predicted signal for the video signal, and quantizing an obtained orthogonal transformation coefficient by using a preset quantization step size so as to encode the coefficient, the apparatus comprising:
a computation device that computes a prediction error power which is a power of the prediction error signal;
a determination device that receives the prediction error power computed by the computation device, the preset quantization step size, and an upper limit of an amount of code generated for the encoding target area, and determines whether or not an amount of code generated when performing quantization using the preset quantization step size exceeds the upper limit; and
a change device that changes an encoding process based on a result of the determination by the determination device,
wherein the determination device computes a permissive power for the prediction error power based on the upper limit and the preset quantization step size, and compares the permissive power with the prediction error power computed so as to determine whether or not the amount of code generated when performing the quantization using the preset quantization step size exceeds the upper limit; and
when it is determined that the amount of code generated when performing the quantization using the preset quantization step size exceeds the upper limit, the change device changes the quantization step size of the quantization for the encoding target area from the preset quantization step size to a quantization step size by which the amount of code generated for the encoding target area does not exceed the upper limit[deleted: .], the orthogonal transformation coefficient for the encoding target area is quantized using the changed quantization step size, and the quantized orthogonal transformation coefficient for the encoding target area is encoded."
XIII. Claim 1 of the third auxiliary request reads as follows (features added to or deleted from claim 1 of the first auxiliary request are underlined or crossed out, respectively):
"A video encoding apparatus for applying orthogonal transformation to a prediction error signal between a video signal of an encoding target area corresponding to a macroblock and a predicted signal for the video signal, and quantizing an obtained orthogonal transformation coefficient by using a preset quantization step size set from an amount of generated code and a bit rate so as to encode the coefficient, the apparatus comprising:
a computation device that computes a prediction error power which is a power of the prediction error signal;
a determination device that receives the prediction error power computed by the computation device, the preset quantization step size, and an upper limit of an amount of code generated for the encoding target area, and determines whether or not an amount of code generated when performing quantization using the preset quantization step size exceeds the upper limit; and
a change device that changes an encoding process based on a result of the determination by the determination device,
wherein the determination device computes a permissive power for the prediction error power based on the upper limit and the preset quantization step size, and compares the permissive power with the prediction error power computed so as to determine whether or not the amount of code generated when performing the quantization using the preset quantization step size exceeds the upper limit;
when it is determined that the amount of code generated when performing the quantization using the preset quantization step size exceeds the upper limit, the change device changes the quantization step size of the quantization for the encoding target area from the preset quantization step size to a quantization step size by which the amount of code generated for the encoding target area does not exceed the upper limit, the orthogonal transformation coefficient for the encoding target area is quantized using the changed quantization step size, and the quantized orthogonal transformation coefficient for the encoding target area is encoded;
and [deleted: wherein the determination device computes the permissive power for the prediction error power by setting variables of a function, which are the upper limit and the quantization step size, to the values of the upper limit and the quantization step size, where the value of the function is the permissive power, or] wherein the determination device computes the permissive power for the prediction error power by referring to a table in which a relationship between data values of the upper limit, the quantization step size, and the permissive power is defined."
XIV. The arguments submitted by the appellant, as far as relevant to the present decision, may be summarised as follows:
The appellant contested that the objective technical problem could be formulated as being how to reduce the likelihood of a macroblock being re-encoded in PCM mode, i.e. the likelihood of the code amount generated for a macroblock exceeding the upper limit set by the H.264 video coding standard. According to the appellant, the invention not only reduced the likelihood of the code amount generated for a macroblock exceeding the upper limit, it guaranteed that the code amount would not exceed this limit. In any case, the person skilled in the art starting from document D1 would not have tried to solve the objective technical problem formulated by the board. As shown by document D3, section 1, second paragraph ("two macroblocks per row are allowed to exceed the limit"), the upper code amount limit set by the H.264 video coding standard did not have to be met for all macroblocks. Thus the person skilled in the art would have had no motivation to modify the apparatus disclosed in document D1 to reduce the likelihood of this upper limit being exceeded.
Reasons for the Decision
1. The appeal is admissible.
2. The invention
2.1 In an H.264-compliant video encoder, a macroblock is compressed by undergoing steps of prediction, transformation, quantization and entropy coding. Quantization is carried out using a quantization step size usually selected under a certain bit rate constraint.
2.2 It may happen that these steps will generate a greater code amount for the macroblock than if they had not been carried out. In such a case, i.e. when the generated code amount exceeds an upper limit set by the H.264 standard, it is known to re-encode the macroblock without compression, i.e. in a mode called pulse code modulation (PCM) mode (see paragraphs [0002], [0003], [0008] and [0009] of the application).
2.3 This re-encoding introduces a processing delay and requires additional memory (see paragraphs [0011] and [0012] of the application).
2.4 The aim of the application is to find an alternative solution that does not require re-encoding (see paragraph [0001]).
2.5 The invention as described rests upon the realisation that the code amount generated for a macroblock with a certain quantization step size is correlated to the power of the prediction error (see paragraph [0028] of the application). It involves (i) converting the upper code amount limit set by the H.264 video coding standard into an upper limit of prediction error power (called "permissive power") for the quantization step size that is to be applied to the macroblock (called "preset quantization step size"), (ii) comparing the actual prediction error power of a current macroblock with the permissive power to determine (or, rather, predict) whether the preset quantization step size will lead to a code amount that exceeds the upper limit, and (iii) where required, changing the quantization step size to a value that will (or rather is more likely to) generate a code amount that does not exceed the upper code amount limit.
3. Main request, inventive step (Article 56 EPC)
3.1 An invention is to be considered as involving an inventive step if, having regard to the state of the art, it is not obvious to a person skilled in the art (Article 56 EPC).
3.2 Closest prior art
3.2.1 It is undisputed that document D1 may be considered the closest prior art in the established "problem and solution approach" (Case Law of the Boards of Appeal of the European Patent Office ("Case Law"), 9th edition 2019, I.D.2).
3.2.2 Document D1 discloses a video encoding apparatus for applying orthogonal transformation to a prediction error signal between a video signal of an encoding target area and a predicted signal for the video signal, and quantizing an obtained orthogonal transformation coefficient by using a preset quantization step size set so as to encode the coefficient (see document T1, Figure 1 and paragraph [0010]).
The apparatus comprises a code amount estimation part (see document T1, paragraph [0010], "code amount estimation part 105") that computes a power of the prediction error signal for the current macroblock (see document T1, paragraph [0022], "differential square sum (SSD)") and converts it into a code amount that is likely to be generated for this macroblock when quantized with the preset quantization step size (see document T1, paragraph [0022] and Figure 8b).
It also comprises a change device (see document T1, paragraph [0010], "rate control determination part 106") that sets the quantization step for the next macroblock so as to reduce the difference between the code amount generated thus far for the picture and a target code amount (see document T1, paragraph [0015]).
3.3 Document D1 does not disclose the following features of claim 1:
- "a determination device that receives the prediction error power computed by the computation device, the preset quantization step size, and an upper limit of an amount of code generated for the encoding target area, and determines whether or not an amount of code generated when performing quantization using the preset quantization step size exceeds the upper limit"
- "wherein the determination device computes a permissive power for the prediction error power based on the upper limit and the preset quantization step size, and compares the permissive power with the prediction error power computed so as to determine whether or not the amount of code generated when performing the quantization using the preset quantization step size exceeds the upper limit"
- "when it is determined that the amount of code generated when performing the quantization using the preset quantization step size exceeds the upper limit, the change device changes the quantization step size of the quantization for the encoding target area from the preset quantization step size to a quantization step size by which the amount of code generated for the encoding target area does not exceed the upper limit"
3.4 Objective technical problem
3.4.1 An objective definition of the problem to be solved by the invention should normally start from the problem described in the application (Case Law, I.D.4.3.2). If, having regard to the prior art, and irrespective of what may be asserted in the description, it does not appear credible that the invention as claimed would actually be capable of solving the problem, then an objection under Article 56 EPC may be raised, possibly requiring a reformulation of the problem (Case Law, I.D.2). Only the effect actually achieved vis-à-vis the closest prior art should be taken into account (Case Law, I.D.4.1).
3.4.2 Paragraph [0021] of the application states that "an object of the present invention is to provide a novel image encoding technique which does not require re-encoding or encoding corresponding to two or more encoding modes, and implements an encoding whose amount of generated code does not exceed an upper limit without awaiting a measured result of the amount of generated code". The appellant submitted that the features identified under point 3.3 above guaranteed that the code amount generated for a macroblock would not exceed this limit.
3.4.3 The board does not accept this view. Claim 1 does not specify how the permissive power is computed for the prediction error power based on the upper limit and the preset quantization step size. It is self-evident to the board that not all computational models make it possible to determine with one hundred percent certainty "whether or not the amount of code generated when performing quantization using the preset quantization step size exceeds the upper limit" (main request, claim 1, lines 19 to 21) or, in other words, guarantees that the code amount generated for a macroblock does not exceed the upper code amount limit set by the H.264 video coding standard. Given the variety and complexity of real images, the board has doubts that such a model even exists. Thus it is appropriate to reformulate the objective technical problem.
3.4.4 The board acknowledges that the features identified under point 3.3 above increase the likelihood that the code amount generated for the macroblock will not exceed the upper limit set by the H.264 standard, i.e. that re-encoding in PCM mode will not be required. In view of this, it reformulates the objective technical problem as being how to reduce the likelihood of a macroblock being re-encoded in PCM mode, i.e. the likelihood of the code amount generated for a macroblock exceeding the upper limit.
3.5 Could-would approach
3.5.1 The issue of the formulation of the objective technical problem must be distinguished from the issue of whether the person skilled in the art would have modified the teaching in the closest prior art document in the light of other teachings in the prior art so as to arrive at the claimed invention (Case Law, I.D.5).
3.5.2 Unlike the appellant, the board finds that this question should be answered in the affirmative in the present case, for the following reasons:
3.5.3 Since the video encoding apparatus disclosed in document D1 is based on the H.264 standard (see document T1, paragraphs [0002], [0021] and [0028]), it is also subject to the upper code amount limit mentioned under point 2.2 above. Document D1 also refers to the PCM mode mentioned under point 2.2 above (see document T1, paragraph [0032]). The person skilled in the art would have been aware of the additional processing time and memory needed to re-encode a macroblock in PCM mode (see paragraphs [0011] and [0012] of the application) and thus would have tried to solve the objective technical problem starting from document D1.
3.5.4 The appellant's argument that the upper code amount limit set by the H.264 video coding standard does not apply to all macroblocks (see point XIV. above) has not convinced the board. The reasons are as follows:
(a) Section A.3.2 of Annex A of ITU-T Recommendation H.264 (document D4) specifies that "Bitstreams conforming to the High, High 10, High 4:2:2, or High 4:4:4 profiles at a specified level shall obey the following constraints:
j) Number of bits of macroblock_layer( ) data for any macroblock is not greater than 128 + RawMbBits" (emphasis added by the board).
Thus the H.264 standard sets an upper code amount limit for all macroblocks, for at least certain video profiles.
Paragraph [0008] of the application itself specifies that "in order to reliably encode each macroblock of any input image with a number of bits less than an upper limit, H.264 employs a pulse code modulation (PCM) mode in which the pixel value is directly transmitted without compression (i.e., without quantization)" (emphasis added by the board). From this it can also be deduced that the constraint defined in H.264 applies to all macroblocks.
Since document D3 is only a proposal made in the context of the development of the H.264 standard - not the H.264 standard itself - the board has not been convinced of its relevance in this matter.
(b) In any case, claim 1 of the main request does not specify that the upper code amount limit applies to all macroblocks of the video. Claim 1 merely refers to "an encoding target area" (line 2) (emphasis added by the board). Thus even if the upper code amount limit set by the H.264 video coding standard did not apply to all macroblocks, the person skilled in the art would still have tried to solve the objective technical problem formulated under point 3.4.4 above for the reason given under point 3.5.3 above.
3.5.5 The correlation between prediction error power and generated code amount - upon which the claimed solution rests (see point 2.5 above) - is disclosed in document D1. Paragraph [0022] and Figure 8(b) of document T1 disclose QP-dependent conversion tables that map the sum of absolute differences (SAD) of the prediction error of a macroblock to a code amount. Paragraph [0022] indicates that the sum of squared differences (SSD), i.e. the power, may be used instead of the SAD.
3.5.6 The board considers that the mere disclosure of this correlation renders the distinguishing features identified under point 3.3 obvious. Document D1 teaches using the prediction error power of a macroblock to predict the code amount likely to be generated when a preset quantization step size is used. The person skilled in the art would have applied this teaching when faced with the objective technical problem, and would have used the prediction error power of a macroblock to predict whether the code amount generated with the preset quantization step size is likely to exceed the upper limit set by the H.264 standard. Moreover, to reduce the likelihood of generating a code amount exceeding the upper limit, the person skilled in the art would have thought of changing the quantization step size if required.
3.5.7 Converting the upper code amount limit to a "permissive power" to be compared to the actual prediction error power of the macroblock is an obvious way of making use of the conversion tables disclosed in document D1.
3.6 In view of the above, the board comes to the conclusion that the subject-matter of claim 1 of the main request lacks inventive step over the disclosure of document D1 combined with the common general knowledge of the person skilled in the art.
4. First auxiliary request, inventive step (Article 56 EPC)
4.1 In comparison with claim 1 of the main request, claim 1 of the first auxiliary request further specifies that:
(a) the encoding target area corresponds to a macroblock
(b) the preset quantization step size is set from an amount of generated code and a bit rate
(c) the "determination device computes the permissive power for the prediction error power by setting variables of a function, which are the upper limit and the quantization step size, to the values of the upper limit and the quantization step size, where the value of the function is the permissive power, or wherein the determination device computes the permissive power for the prediction error power by referring to a table in which a relationship between data values of the upper limit, the quantization step size, and the permissive power is defined".
4.2 As mentioned under point 3.2.2 above, document D1 discloses a rate control determination part that sets the quantization step for the next macroblock so as to reduce the difference between the code amount generated thus far for the picture and a target code amount (paragraph [0015]). Thus features 4.1(a) and 4.1(b) are known from document D1.
4.3 Moreover, as indicated under point 3.5.5 above, converting the upper code amount limit to a "permissive power" to be compared to the actual prediction error power of the macroblock would have been an obvious way of making use of the QP-dependent conversion tables disclosed in document D1. This renders obvious the second alternative defined by feature 4.1(c) ("wherein the determination device computes the permissive power for the prediction error power by referring to a table in which a relationship between data values of the upper limit, the quantization step size, and the permissive power is defined").
4.4 For the sake of completeness, the board also considers the first alternative defined by feature 4.1(c) ("determination device computes the permissive power for the prediction error power by setting variables of a function, which are the upper limit and the quantization step size, to the values of the upper limit and the quantization step size, where the value of the function is the permissive power") to be obvious in view of the disclosure of document D1. Indeed, paragraph [0030] specifies that the correlation between code amount and prediction error may be modelled as a function instead of a conversion table. Converting the upper code amount limit to a "permissive power" to be compared to the actual prediction error power of the macroblock would have been an obvious way of making use of these QP-dependent conversion functions.
4.5 In view of the above, the board comes to the conclusion that the subject-matter of claim 1 of the first auxiliary request also lacks inventive step over the disclosure of document D1 combined with the common general knowledge of the person skilled in the art.
5. Second auxiliary request, inventive step (Article 56 EPC)
5.1 In comparison with claim 1 of the main request, claim 1 of the second auxiliary request further specifies that "the orthogonal transformation coefficient for the encoding target area is quantized using the changed quantization step size, and the quantized orthogonal transformation coefficient for the encoding target area is encoded".
5.2 Claim 1 of the main request already specified that "the change device changes the quantization step size of the quantization for the encoding target area". Thus the fact that "the orthogonal transformation coefficient for the encoding target area is quantized using the changed quantization step size" was already clear from the wording of claim 1 of the main request and had already been taken into account in the analysis performed under section 3.
5.3 Moreover, document D1 discloses encoding the quantized orthogonal transformation coefficient using CABAC (see page 11, lines 9 to 12). Thus the fact that "the quantized orthogonal transformation coefficient for the encoding target area is encoded" does not represent an additional distinguishing feature either.
5.4 In view of the above, the board comes to the conclusion that the subject-matter of claim 1 of the second auxiliary request also lacks inventive step over the disclosure of document D1 combined with the common general knowledge of the person skilled in the art.
6. Third auxiliary request, inventive step (Article 56 EPC)
6.1 In comparison with claim 1 of the first auxiliary request, claim 1 of the third auxiliary request further specifies that "the orthogonal transformation coefficient for the encoding target area is quantized using the changed quantization step size, and the quantized orthogonal transformation coefficient for the encoding target area is encoded". Moreover, the alternative consisting in computing the permissive power by setting variables of a function has been deleted.
6.2 The added feature has already been discussed in the previous section and was considered not to contribute to inventive step.
6.3 The deletion of the alternative consisting in computing the permissive power by setting variables of a function cannot contribute to inventive step either since the remaining alternative was already considered obvious by the board (see point 3.5.7 above).
6.4 In view of the above, the board comes to the conclusion that the subject-matter of claim 1 of the third auxiliary request also lacks inventive step over the disclosure of document D1 combined with the common general knowledge of the person skilled in the art.
7. Since none of the requests is allowable, the appeal is to be dismissed.
Order
For these reasons it is decided that:
The appeal is dismissed.