T 2256/17 08-11-2022
Download and more information:
IMPROVED RTP PAYLOAD FORMAT DESIGNS
Amendment after summons - exceptional circumstances (yes)
Inventive step - (yes)
I. The appeal is against the examining division's decision to refuse European patent application No. 14 724 585.6, published as international patent application WO 2014/160970 A1. The application claims the priority of US 61/806,705, filed on 29 March 2013, and of US 14/228,164, filed on 27 March 2014.
II. The prior-art documents cited in the decision under appeal included the following:
D2 Y.-K. Wang et al., "RTP Payload Format for H.264 Video", Internet Engineering Task Force (IETF), Request for Comments: 6184, May 2011, pages 1 to 101, XP015075997
D4 T. Schierl et al., "Wireless broadcasting using the scalable extension of H.264/AVC", 2005 IEEE International Conference on Multimedia and Expo, 6 July 2005, pages 884 to 887, XP002484405
III. The decision under appeal was based on the ground that the subject-matter of none of the claims of the sole request then on file involved an inventive step within the meaning of Article 56 EPC in view of prior-art documents D2 and D4.
IV. With the statement of grounds of appeal, the appellant maintained the set of claims according to the sole request on which the impugned decision was based and provided arguments to support its opinion that these claims met the requirements of Article 56 EPC.
V. The board issued a summons to oral proceedings and a communication under Article 15(1) of the Rules of Procedure of the Boards of Appeal in the 2020 version (RPBA 2020, see OJ EPO 2019, A63). In this communication, the board gave the following preliminary opinion.
(a) The objection of lack of inventive step raised by the examining division was not justified.
(b) The priority of US 61/806,705, which was filed on 29 March 2013, was not validly claimed for the subject-matter of independent claims 1, 7 and 8.
(c) Since the priority of US 61/806,705 was not valid, the following document, cited in the international preliminary report on patentability, was prior art under Article 54(2) EPC:
D1: T. Schierl et. al., "RTP Payload Format for High Efficiency Video Coding draft-schierl-payload-rtp-h265-02.txt", Network Working Group, Internet Draft, 11 June 2013, pages 1 to 69, XP015090627
(d) The subject-matter of claims 1, 7 and 8 of the main request did not involve an inventive step in view of the disclosure of document D1 (Article 56 EPC).
VI. By letter dated 15 September 2022, the appellant filed amended claims according to a first and a second auxiliary request. It also re-filed the claims of the sole request underlying the decision under appeal as its main request. The appellant provided a basis for the claimed subject-matter and argued why the first and second auxiliary requests should be admitted into the appeal proceedings. Furthermore, the appellant provided arguments to support its opinion that the priority was validly claimed for the subject-matter of the independent claims of the main request, first auxiliary request and second auxiliary request. The appellant took the view that document D1 did not belong to the prior art and that the claimed subject-matter was not obvious in view of the available prior art.
VII. On 8 November 2022, oral proceedings were held before the board.
During the oral proceedings, the appellant filed claims according to an amended first auxiliary request and a corresponding description.
The appellant then made the amended first auxiliary request its sole request and withdrew all of the other requests on file.
The appellant's final requests were that the decision under appeal be set aside and that a European patent be granted on the basis of the claims of the sole request and the description, both filed during the oral proceedings on 8 November 2022, and the drawings of the application as filed.
At the end of the oral proceedings, the chair announced the board's decision.
VIII. The independent claims 1, 3, 4 and 7 of the sole request read as follows:
"1. A method of processing video data, the method comprising:
receiving a first aggregation packet according to a real-time transfer protocol (RTP), wherein the first aggregation packet comprises a payload header followed by payload data comprising a first aggregation unit, that is the first aggregation unit of the first aggregation packet, including a first network abstraction layer (NAL) unit, the payload data further comprising a second aggregation unit, that follows the first aggregation unit in the first aggregation packet, including a second NAL unit, wherein if a transmission mode is equal to multi-session transmission, MST, or if the maximum number of NAL units that precede a NAL unit in a de-packetization buffer in reception order and follow the NAL unit in decoding order is greater than 0, the first aggregation unit further includes a first syntax element and the second aggregation unit further includes a second syntax element;
wherein the first syntax element consists of the 16 least significant bits of the decoding order number for the first NAL unit;
wherein the second syntax element plus 1 specifies a difference between the decoding order number for the first NAL unit and the decoding order number for the second NAL unit; and
determining a decoding order for the second NAL unit included in the second aggregation unit, wherein the decoding order for the second NAL unit included in the second aggregation unit is derived as equal to the first syntax element plus the second syntax element plus 1 modulo 65536.
3. A method of processing video data, the method comprising:
packetizing two or more network abstraction layer (NAL) units into a first aggregation packet according to an RTP protocol, wherein the first aggregation packet comprises a payload header followed by payload data comprising a first aggregation unit, that is the first aggregation unit of the aggregation packet, including a first NAL unit, the payload data further comprising a second aggregation unit, that follows the aggregation unit in the aggregation packet, including a second NAL unit, wherein if a transmission mode is equal to multi-session transmission, MST, or if the maximum number of NAL units that precede a NAL unit in a depacketization buffer in reception order and follow the NAL unit in decoding order is greater than 0, the first aggregation unit further including a first syntax element and the second aggregation unit further including a second syntax element;
setting a value for the first syntax element of the first aggregation unit, the first syntax element consisting of the 16 least significant bits of the decoding order number for the first NAL unit; and
setting a value for the second syntax element of the second aggregation unit, and wherein the second syntax element plus 1 specifies a difference between the decoding order number for the first NAL unit included in the first aggregation unit and a decoding order number for the second NAL unit, wherein the value for the second syntax element is derived as equal to a decoding order for the second aggregation unit minus the first syntax element minus 1 modulo 65536.
4. A device for processing video data, the apparatus comprising:
a memory;
a receiver configured to receive real-time transport protocol (RTP) packets;
one or more processors configured to:
receive a first aggregation packet according to a real-time transfer protocol (RTP), wherein the first aggregation packet comprises a payload header followed by payload data comprising a first aggregation unit, that is the first aggregation unit of the first aggregation packet, including a first network abstraction layer (NAL) unit, the payload data further comprising a second aggregation unit, that follows the first aggregation unit in the first aggregation packet, including a second NAL unit, wherein if a transmission mode is equal to multi-session transmission, MST, or if the maximum number of NAL units that precede a NAL unit in a de-packetization buffer in reception order and follow the NAL unit in decoding order is greater than 0" the first aggregation unit further includes a first syntax element and the second aggregation unit further includes a second syntax element;
wherein the first syntax element consists of the 16 least significant bits of the decoding order number for the first NAL unit;
wherein the second syntax element plus 1 specifies a difference between the decoding order number for the first NAL unit and the decoding order number for the second NAL unit; and
determine a decoding order for the second NAL unit included in the second aggregation unit, wherein the decoding order for the second NAL unit included in the second aggregation unit is derived as equal to the first syntax element plus the second syntax element plus 1 modulo 65536.
7. A computer-readable storage medium storing instructions that when executed by one or more processors cause the one or more processors to perform the method according to any one of claims 1 to 3."
Claim 2 is dependent on claim 1. Claims 5 and 6 are dependent on claim 4.
IX. In the decision under appeal, the examining division held that document D2 was the closest prior art with regard to the subject-matter of claim 1 of the sole request then on file.
The subject-matter of claim 1 of the sole request then on file differed from the disclosure of document D2 in that the first parameter (i.e. the first syntax element in claim 1 of the current sole request) consisted of the 16 least significant bits of the decoding order for the first NAL unit.
The problem to be solved by the present invention was how to alternatively signal the decoding order number for the first NAL unit of an aggregation packet.
The feature of including the decoding order number in the first NAL unit instead of the header of the aggregation packet was considered an obvious development that the person skilled in the art would have considered when deploying the system according to document D2 in other networking environments, such as in document D4, page 3, left column, lines 2 to 7.
The examining division therefore held that the subject-matter of claim 1 did not involve an inventive step within the meaning of Article 56 EPC.
1. The appeal is admissible.
2. Sole request - admittance (Article 13(2) RPBA 2020)
2.1 The sole request was filed during the oral proceedings and therefore constitutes an amendment within the meaning of Article 13(2) RPBA 2020.
2.2 The board found that the objection of lack of inventive step raised by the examining division was not justified. However, the board held that the claimed priority was not valid and raised an objection of lack of inventive step starting from document D1 for the first time in its communication under Article 15(1) RPBA 2020 (see point V. above). In response to this communication, the appellant filed further requests. During the oral proceedings, the board raised objections of lack of clarity against these further requests for the first time. In response, the appellant filed another set of claims with the aim of overcoming these new objections. These are exceptional circumstances within the meaning of Article 13(2) RPBA 2020. Exercising its discretion under this provision, the board has thus decided to admit the sole request into the appeal proceedings.
3. Sole request - added subject-matter (Article 123(2) EPC)
3.1 Claim 1 combines the features of claims 1 to 5 as originally filed.
3.2 Claim 1 has been further amended to specify that "if a transmission mode is equal to multi-session transmission, 'MST', or if the maximum number of NAL units that precede a NAL unit in a de-packetization buffer in reception order and follow the NAL unit in decoding order is greater than 0".
This amendment finds a basis in paragraphs [0073] and [0076] of the application as filed, which read "If tx-mode is equal to 'MST' or sprop-depack-buf-nalus is greater than 0, the DONL field MUST be present in an aggregation unit that is the first aggregation unit in an AP" and "If tx-mode is equal to 'MST' or sprop-depack-buf-nalus is greater than 0, the DOND field MUST be present in an aggregation unit that is not the first aggregation unit in an AP".
The sprop-depack-buf-nalus parameter mentioned therein is defined in paragraph [0101] of the application as filed as "the maximum number of NAL units that precede a NAL unit in the de-packetization buffer in reception order and follow the NAL unit in decoding order".
3.3 In addition, claim 1 has been amended to specify that the first aggregation unit further includes a first syntax element and the second aggregation unit further includes a second syntax element.
This amendment finds a basis in paragraph [0071] of the application as filed, which reads "The first aggregation unit in an AP may consist of an optional 16-bit DONL field", and paragraph [0074], which reads "An aggregation unit that is not the first aggregation unit in an AP may consist of an optional 8-bit DOND field".
3.4 Claim 1 has been further amended to specify that the first syntax element consists of the 16 least significant bits of the decoding order number for the first NAL unit.
This amendment finds a basis in paragraph [0072] of the application as filed, which reads "The DONL field, when present, specifies the value of the 16 least significant bits of the decoding order number of the aggregated NAL unit".
3.5 Claim 1 has been further amended to specify that the second syntax element plus 1 specifies a difference between the decoding order number for the first NAL unit and the decoding order number for the second NAL unit.
This amendment finds a basis in paragraph [0075] of the application as filed, which reads "When present, the DOND field plus 1 may specify the difference between the decoding order number values of the current aggregated NAL unit and the preceding aggregated NAL unit in the same AP".
3.6 Claim 1 has been further amended to specify that the decoding order for the second NAL unit included in the second aggregation unit is derived as equal to the first syntax element plus the second syntax element plus 1 modulo 65536.
This amendment finds a basis in paragraph [0073] of the application as filed, which reads "the variable DON for the aggregated NAL unit is derived as equal to the value of the DONL field", and paragraph [0076], which reads "the variable DON for the aggregated NAL unit is derived as equal to the DON of the preceding aggregated NAL unit in the same AP plus the value of the DOND field plus 1 modulo 65536".
3.7 Claim 3 combines the features of original claims 28 to 30. Furthermore, amendments corresponding to those set out under points 3.2 to 3.6 apply.
3.8 Claim 4 combines the features of original claims 10 to 14. Furthermore, amendments corresponding to those set out under points 3.2 to 3.6 apply.
3.9 Dependent claim 2 is based on original dependent claim 6 and the original application paragraphs [0073] and [0076].
3.10 Dependent claim 5 is based on original dependent claim 15 and the original application paragraphs [0073] and [0076].
3.11 Dependent claim 6 corresponds to original claim 19.
3.12 Claim 7 is based on original claim 20.
3.13 In view of the above, the claims of the sole request meet the requirements of Article 123(2) EPC.
4. Sole request - validity of the claimed priority
4.1 The application claims priority from the following US applications:
P1: US 61/806,705 (filing date 29 March 2013)
P2: US 14/228,164 (filing date 27 March 2014).
4.2 Document P1 discloses the features of claim 1 as follows:
A method of processing video data, the method comprising:
receiving a first aggregation packet according to a real-time transfer protocol (RTP) (see paragraph [0019], which reads "A receiver can identify the type of an RTP packet payload ... The three different payload structures are as follows ... Aggregation packet (AP)"), wherein the first aggregation packet comprises a payload header followed by payload data comprising a first aggregation unit, that is the first aggregation unit of the first aggregation packet, including a first network abstraction layer (NAL) unit, the payload data further comprising a second aggregation unit, that follows the first aggregation unit in the first aggregation packet, including a second NAL unit (see paragraph [0029], which reads "Each NAL unit to be carried in an AP is encapsulated in an aggregation unit. ... An AP consists of a payload header (denoted as PayloadHdr) followed by one or more aggregation units"), wherein if a transmission mode is equal to multi-session transmission, MST, or if the maximum number of NAL units that precede a NAL unit in a de-packetization buffer in reception order and follow the NAL unit in decoding order is greater than 0, the first aggregation unit further includes a first syntax element and the second aggregation unit further includes a second syntax element (see paragraph [0034], which reads "If tx-mode is equal to 'MST' or sprop-depack-buf-nalus is greater than 0, the DONL field MUST be present in aggregation unit that is the first aggregation unit in an AP", and paragraph [0037], which reads "If tx-mode is equal to 'MST' or sprop-depack-buf-nalus is greater than 0, the DOND field MUST be present in an aggregation unit that is not the first aggregation unit in an AP", and paragraph [0062], which reads "sprop-depack-buf-nalus: This parameter specifies the maximum number of NAL units that precede a NAL unit in the de-packetization buffer in reception order and follow the NAL unit in decoding
order");
wherein the first syntax element consists of the 16 least significant bits of the decoding order number for the first NAL unit (see paragraph [0032], which reads "The first aggregation unit in an AP consists of an optional 16-bit DONL field", and paragraph [0033], which reads "The DONL field, when present, specifies the value of the 16 least significant bits of the decoding order number of the aggregated NAL unit");
wherein the second syntax element plus 1 specifies a difference between the decoding order number for the first NAL unit and the decoding order number for the second NAL unit (see paragraph [0036], which reads "When present, the DOND field plus 1 specifies the difference between the decoding order number values of the current aggregated NAL unit and the preceding aggregated NAL unit in the same AP"); and
determining a decoding order for the second NAL unit included in the second aggregation unit, wherein the decoding order for the second NAL unit included in the second aggregation unit is derived as equal to the first syntax element plus the second syntax element plus 1 modulo 65536 (see paragraph [0037], which reads "the variable DON for the aggregated NAL unit is derived as equal to the DON of the preceding aggregated NAL unit in the same AP plus the value of the DOND field plus 1 modulo 65536").
Hence, for claim 1 the priority claimed from document P1 is valid.
4.3 The same applies to independent method claim 3, which comprises features corresponding to those of claim 1 but in the form of packetising NAL units into an aggregation packet. Claim 1 of document P1 discloses a method of processing video operating in packetisation mode.
4.4 The same applies to independent device claim 4, which comprises the same features as claim 1 but in the form of one or more processors configured to carry out the method steps of claim 1. Claim 13 of document P1 discloses a video processing device configured to perform previously specified method steps and comprising one or more processors.
4.5 The same applies to the computer-readable storage medium of claim 7 (see paragraph [0112] of document P1).
4.6 In view of the above, the priority claimed from document P1 (filing date 29 March 2013) is valid for independent claims 1, 3, 4 and 7.
As a consequence, for the assessment of inventive step of these independent claims, document D1, which was published on 11 June 2013, is not prior art under Article 54 EPC.
5. Sole request - inventive step (Article 56 EPC)
5.1 Both the examining division and the appellant regarded document D2 as the closest prior art for the assessment of inventive step of the subject-matter of claim 1. The board concurs with this finding.
5.2 Document D2 specifies RTP payload formats for H.264 video. Under point 5.7.2 on pages 25 to 29, document D2 discloses:
- receiving an RTP packet including an aggregation packet containing two aggregation units (see description of Figure 12)
- the aggregation packet payload consists of a 16 bit decoding order number base (DONB) and one or more aggregation units each containing a network abstraction layer (NAL) unit (see point 5.7.2, first paragraph, and Figure 9)
- the DONB is the decoding order number of the first NAL unit in decoding order among the NAL units in the aggregation packet (see point 5.7.2, first and second paragraphs)
- each aggregation unit includes a NAL unit and a syntax element identifying the decoding order number difference (DOND) for the NAL unit (see page 26, first paragraph)
- the decoding order number (DON) for a NAL unit contained in an aggregation unit is equal to (DONB + DOND) modulo 65536 (see page 26, second paragraph)
5.3 It follows that document D2 does not disclose the following distinguishing features of claim 1:
(a) the first syntax element consists of the decoding order number for the first NAL unit in the aggregation packet
(b) the first syntax element is included in the first aggregation unit
(c) the second syntax element plus 1 specifies a difference between decoding numbers
(d) transmission mode is equal to multi-session transmission, MST, or the maximum number of NAL units that precede a NAL unit in a de-packetization buffer in reception order and follow the NAL unit in decoding order is greater than 0
5.4 Hence, the board finds that there are more distinguishing features than those identified by the examining division for claim 1 of the sole request then on file (see point IX. above).
5.5 The technical effect of distinguishing features (a) and (b) is to specify the decoding order numbers for the NAL units within an aggregation packet in an alternative manner which reduces the number of bits required.
Distinguishing feature (c) is a mere shift of the value range of the second syntax element which has no identified technical effect.
Distinguishing feature (d) concerns the transmission of aggregation packets via multiple RTP flows. This aspect is unrelated to the payload format within a single RTP packet as defined by the distinguishing features (a), (b) and (c). During the oral proceedings, the appellant could not identify a technical effect resulting from distinguishing feature (d).
5.6 The objective technical problem may thus be formulated as how to specify the decoding order numbers for the NAL units within an aggregation packet in an alternative manner.
5.7 In this regard, the person skilled in the art would consider document D4 because it discloses a modified aggregation unit within an aggregation packet (see D4, Figure 4: "modified Multi Time Aggregation Unit"). However, document D4 teaches to replace the decoding order number difference (DOND) within an aggregation unit with an absolute value of a decoding order number (see D4, page 3, left column, first paragraph). Applying this teaching to the RTP payload specified in document D2 would lead to a format in which the decoding order numbers would be given as absolute values for all aggregation units. This would not lead to the claimed subject-matter according to which the second syntax element specifies a difference between decoding order numbers.
Therefore, the board does not share the examining division's view that distinguishing features (a) and (b) were rendered obvious by the combined disclosures of documents D2 and D4 (see point IX. above).
5.8 However, the question arises as to whether the person skilled in the art would have regarded the subject-matter of claim 1 as an obvious alternative to the solution proposed in D2.
5.8.1 One straightforward modification that the skilled person would have considered was to shift a syntax element from a higher layer to a lower layer in order to increase adaptivity.
However, in the present case, shifting the decoding order number base from the general payload data to the first aggregation unit would not have led to the claimed subject-matter. Indeed, the decoding order number base would then still be the decoding order number of the first NAL unit in the decoding order within the aggregation packet, but not the decoding number of the first NAL unit in the aggregation packet as in claim 1.
5.8.2 As an alternative modification, the person skilled in the art might have considered determining the decoding order number base of document D2 in a different manner, for example as the decoding order number of the first NAL unit, or as a mean value of all decoding order numbers in the aggregation packet.
Yet even this would not have led to the feature(s) of claim 1, according to which the first syntax element is included in the first aggregation unit.
5.8.3 In order to arrive at both distinguishing features (a) and (b), the person skilled in the art would have needed to change both the meaning of the first syntax element (from the lowest decoding order number of all aggregation units to the decoding order number of the first aggregation unit) and its position within the RTP payload format (from the general payload to the first aggregation unit).
The board does not find this to be obvious on the basis of common general knowledge, as it would significantly change the structure of an aggregation packet from the uniform structure in document D2, where all aggregation units have the same length and syntax elements, to a structure where the first aggregation unit would have a length and a syntax element differing from those of the subsequent aggregation units.
5.9 Therefore, the board does not consider the subject-matter of claim 1 to be obvious in view of the disclosure of document D2 when combined with either the disclosure of document D4 or the common general knowledge of the person skilled in the art.
5.10 The same applies to the subject-matter of independent claims 3, 4 and 7, which comprises features corresponding to those of claim 1.
5.11 Moreover, the board cannot see any other document or combination of documents on file from which the person skilled in the art would have arrived at the subject-matter of the independent claims in an obvious manner.
5.12 Claims 2, 5 and 6 are dependent claims. Therefore, the subject-matter thereof is not obvious either.
5.13 In view of the above, the claimed subject-matter involves an inventive step within the meaning of Article 56 EPC.
6. The features of the independent claims reflect the structure of an RTP payload. Using a two-part form would distort this structure and is thus not appropriate (Rule 43(1)(b) EPC).
7. The description has been amended in line with the claims of the sole request and complies with the EPC.
8. Conclusion
In view of the above, the present case is to be remitted to the examining division with the order to grant a patent on the basis of the appellant's sole request.
For these reasons it is decided that:
1. The decision under appeal is set aside.
2. The case is remitted to the department of first instance with the order to grant a patent in the following version:
Description
Pages: 1 to 35 filed during the oral proceedings on 8 November 2022
Claims
Nos.: 1 to 7 according to the sole request filed as amended first auxiliary request during the oral proceedings on 8 November 2022
Drawings
Sheets: 1/11 to 11/11 as originally filed