T 0836/09 08-05-2013
Download and more information:
Apparatus and method for combining images in a terminal device
I. The appeal is against the decision of the examining division to refuse European patent application No. 05 025 792.2 under Article 97(2) of the European Patent Convention (EPC).
II. The application was refused on the grounds that the subject-matter of the independent claims of the main request filed with letter of 28 March 2008 did not involve an inventive step (Article 56 EPC). The independent claims of the then auxiliary request were found to infringe Article 123(2) EPC.
III. Claim 1 of the main request reads as follows:
"A mobile communication terminal comprising:
- a broadcast receiver (120) configured to receive digital multimedia broadcast signals;
- an image capture module (144) configured to obtain images from the digital multimedia broadcast signals;
- a camera device (130) configured to capture images of objects;
- a memory (170) for storing the images from the image capturing module (144) and from the camera device (130);
- a display (150) for displaying the images;
- a processor (110, 140) configured to cooperate with the broadcast receiver (120), the image capture module (144), the camera device (130), the memory (170), and the display (150) to perform the steps of:
- performing image processing on a selected one of a broadcast image and a camera preview image,
- combining the image-processed image with the other of the broadcast image and camera preview image, and
- displaying the combined image on the display (150)."
IV. The applicant appealed against this decision and filed a statement of grounds of appeal which was received outside the time limit prescribed by Article 108, third sentence, EPC. The appellant then filed a request for re-establishment of rights. In an interlocutory decision dated 17 February 2010, the board in the present composition re-established the appellant in its rights in relation to the filing of a statement of grounds of appeal within the missed time limit.
V. In the statement of grounds of appeal the appellant submitted an explanation of the technical meaning of the expression "camera preview image" in claim 1 and arguments as to why, on the basis of this meaning, the terminal of claim 1 involved an inventive step. The auxiliary request was not pursued.
VI. The board issued a communication pursuant to Article 15(1) of the Rules of Procedure of the Boards of Appeal (RPBA), annexed to a summons to oral proceedings. The board indicated its doubts that the subject-matter of claim 1 involved an inventive step even if the board adopted the appellant's interpretation of the "camera preview image" as meaning the image displayed before pressing the shutter button. The board indicated that a functionality of combining a received image with a camera preview image was known in the context of mobile communication devices. In this respect the board referred to the following document mentioned in the decision under appeal:
D3: EP 1 396 997 A2.
VII. With a letter of reply dated 5 April 2013 the appellant filed claims 1 to 14 of a new auxiliary request.
VIII. Claim 1 of the new auxiliary request has the same wording as claim 1 of the main request with the following feature added at the end:
"- wherein the processor (110, 140) is further configured to extract an object image along a contour line of a designated object from a stored broadcast image and overlap the extracted object image on the camera preview image."
IX. Oral proceedings before the board were held on 8 May 2013. The appellant's final requests were that the decision under appeal be set aside and that a patent be granted on the basis of claims 1 to 16 filed with letter of 28 March 2008 (main request) or on the basis of claims 1 to 14 filed with letter of 5 April 2013 (auxiliary request). At the end of the oral proceedings, the chairman announced the board's decision.
X. The reasons given in the decision under appeal for refusing the main request may be summarised as follows:
The closest prior art was considered to be document
D1: WO 2004/051658 A1.
D1 disclosed a mobile communication terminal having all the features specified in claim 1 of the main request except that according to this claim 1
(i) the multimedia signals were multimedia broadcast signals and the receiver was a broadcast receiver;
(ii) the image taken from the camera was a camera preview image.
With respect to feature (i), D1 disclosed that a still or moving image involved in an image editing operation was downloaded through a network, e.g. via the internet. It was common general knowledge that the internet was also used for broadcast distribution of multimedia data.
With respect to feature (ii), any image captured by a camera was once a "camera preview image" prior to capture, if the meaning of a "camera preview image" was that of an image obtained from the camera and stored in some memory or buffer. It was well-known that digital cameras had the option of including a preview function. Claim 1 did not further define the expression "camera preview image". Thus feature (ii) could not contribute to an inventive step.
XI. The appellant's arguments may be summarised as follows:
The essential idea underlying the invention was the combination of broadcast images with camera preview images in a mobile communication terminal. D1 did not consider broadcast images at all. A broadcast image, typically a streamed television image, was not the same as a (unicast) image sent to an individual recipient by e-mail or an image downloaded by an individual recipient via the internet. Thus the decision under appeal was incorrect in its finding that it would have been obvious to receive broadcast images via the internet using the mobile communication terminal of D1.
Moreover, a preview image was volatile by its very nature even if it was temporarily buffered in a display buffer. In contrast thereto, a captured image was permanently stored. Thus a captured image was not a preview image, even if temporarily, before capture, a preview image had existed. Hence the decision under appeal was incorrect in its finding that a "camera preview image" might be considered to be an image obtained from the camera and stored in some memory or buffer.
The reasons given in the decision under appeal were based on an ex-post-facto analysis. A person skilled in the art would not have considered using preview images as an input data source for the generation of combined images.
The functionality of combining a preview image with a broadcast image improved the convenience of the user of the mobile communication terminal as there was no need to repeatedly capture images if fine-tuning of the combined image was desired. Instead it was sufficient to re-position the terminal. Thus the invention enabled a faster, easier and more flexible generation of combined images from a broadcast image received by the terminal on the one hand and a camera image from a camera of the terminal on the other hand.
The auxiliary request specified a particular way of combining a broadcast image with a preview image which allowed to insert an object extracted from a broadcast image into an image taken by the camera of the mobile communication terminal.
1. The appeal is admissible.
Main request
2. Claim 1: inventive step (Article 56 EPC 1973)
2.1 The closest prior art and the distinguishing features of the claimed terminal
It is undisputed that document D1 may be considered as the closest prior art, and that the mobile communication terminal of claim 1 differs from that known from D1 in features (i) and (ii) as set out in the decision under appeal (see point X above). The terminal of claim 1 comprises a broadcast receiver configured to obtain images from the digital multimedia broadcast signals, and the mobile communication terminal is configured to display a combined image which includes the camera preview image.
2.2 The problem solved
The board agrees with the appellant that the above distinguishing features allow a user to more conveniently use the mobile communication terminal for combining different images. Thus the objective technical problem may be seen as increasing the versatility of the mobile communication terminal known from D1.
2.3 The board agrees with the decision under appeal that, starting from D1, an obvious solution to this problem was to to combine a broadcast image and a camera preview image. The appellant's argument that the functionalities of specifically combining a broadcast image with a camera preview image and displaying this combined image specifically led to a terminal which was not suggested by the available prior art did not convince the board that the claimed terminal involves an inventive step. The reasons are as follows:
2.3.1 There is no disclosure in the present application that the original source of the broadcast image is of any relevance for the subsequent combination of images. For instance, the present application discloses in paragraph [0059] that a broadcast image may be stored in the terminal once it has been received. In this case a preview image would be combined with the stored image. The application does not disclose in which respect such a stored image would be different from another image stored in the terminal but originating from a different source, or in which respect the way of combining the images would be affected by the original source of the stored image.
2.3.2 Mobile communication devices having the functionality of combining a preview image with another stored image are known from the state of the art, for instance from document D3.
In particular, D3 concerns an image display system for use in a mobile device such as a camera-equipped cellular phone (see paragraphs [0001] to [0004] and [0006]). The image display system of D3 stores a frame image (which is one of the two images to be combined) in the display memory 50 (see paragraph [0024]). Moreover, an image is captured and stored in a buffer memory 22 in accordance with a "synchronous signal" and processed such that a specific area thereof is directly transferred with a direct memory access method to the display memory 50, but not to the work memory (see paragraphs [0026] to [0034]). The result is the display of an in-frame image which combines the frame image with the area image (see paragraphs [0034] and [0035]). The display is successively updated and this results in a more smooth moving picture display (see paragraph [0037]). Thus, in the context of D3 the "captured image" (as it is called in D3) is only stored in the (volatile) buffer and display memories and thus constitutes the preview image, i.e. the live image displayed before pressing the shutter button.
Thus the appellant's argument that the documents considered in the decision under appeal did not mention or hint at the use of preview images for generating combined images is not convincing.
2.3.3 The effects resulting from the specific combination of a broadcast image with a camera preview image were foreseeable for a person skilled in the art. Although these effects are not explicitly addressed in any of the available prior art documents, a person skilled in the art would have been aware of these advantages (and possible disadvantages) when considering this combination of images.
2.4 In view of the above, the board finds that a person skilled in the art of mobile communication devices, starting from document D1, would have considered increasing the versatility of the mobile communication device of D1 by implementing a functionality of combining a received broadcast image with a camera preview image and displaying the combined image on the terminal's display.
2.5 Thus a person skilled in the art would have arrived at the terminal of claim 1 in an obvious manner.
Auxiliary request
3. Claim 1: inventive step (Article 56 EPC 1973)
3.1 It is undisputed that the feature introduced in claim 1 of the auxiliary request specifies a particular way of combining a broadcast image with a preview image in that it allows overlapping of an object image extracted from a stored broadcast image on the camera preview image.
3.2 Hence this feature also leads to an (albeit more specific) increase in the versatility of the mobile communication terminal.
3.3 Since object extraction per se undisputedly is an image processing technique known at the priority date of the present application and the present application does not disclose a specific way of extracting objects from an image which might define a difference with respect to prior-art object extraction techniques, this further limitation (of the terminal of claim 1 of the main request) does not change the assessment in section 2 above. Moreover, the appellant did not provide any arguments as to technical effects caused by the further limitation in claim 1 of the auxiliary request. Also, in the embodiments of the present application object extraction is merely one of the three variants of image combinations (picture-in-picture, alpha-blending, and object extraction from one image and insertion of the object in the other (background) image) which are presented as alternatives which are known per se and which each have their known advantages and disadvantages.
3.4 In view of the above, the board finds that a person skilled in the art of mobile communication devices, starting from document D1, would have considered increasing the versatility of the mobile communication device of D1 by implementing a functionality of combining a received broadcast image with a camera preview image and displaying the combined image on the terminal's display, with the additional limitation that the image combination involves extracting an object image along a contour line of a designated object from a stored broadcast image and overlapping the extracted object image on the camera preview image.
3.5 Thus a person skilled in the art would have arrived at the terminal of claim 1 of the auxiliary request in an obvious manner.
4. Hence the board judges that the mobile communication terminal of claim 1 of both the main and the auxiliary request does not involve an inventive step as defined in Article 56 EPC 1973.
5. Therefore, the appeal is to be dismissed.
For these reasons it is decided that:
The appeal is dismissed.