T 0497/18 16-03-2023
Download and more information:
FOVEATED IMAGING SYSTEM AND METHOD
Amendments - allowable (no)
Inventive step - (no)
I. The appeal is against the examining division's decision to refuse European patent application No. 14 739 996.8.
II. The documents cited in the decision under appeal included the following.
D1|WO 2008/138543 A1 |
D2|G. Bub et al., "Pixel multiplexing for high-speed multi-resolution fluorescence imaging", arXiv.org, 5 October 2009, retrieved from http://arxiv.org/abs/0910.0789, XP055147152|
III. The application was refused on the following grounds.
(a) The subject-matter of claims 1 to 4 of the then sole request lacked inventive step over the disclosure of D1 combined with the common general knowledge of the person skilled in the art (Article 56 EPC).
(b) The subject-matter of claims 2 and 4 was not new over document D1 (Article 54 EPC).
(c) Claims 3 and 4 were not clear (Article 84 EPC).
The examining division also raised an objection of lack of clarity under Article 84 EPC to the expression "orthogonal frames".
IV. The applicant ("appellant") filed notice of appeal. With the statement of grounds of appeal, the appellant filed claims according to a main request and an auxiliary request. The appellant provided arguments to support its opinion that the amendments to the claims had a basis in the application as filed, that the claims of both requests were clear and that the subject-matter of these claims was new and involved an inventive step.
V. The appellant requested that the decision under appeal be set aside and that a patent be granted on the basis of the claims of the main request or, alternatively, of the claims of the auxiliary request. The appellant also requested that oral proceedings be held if the board was minded to refuse the application.
VI. A summons to oral proceedings was issued. In a communication under Article 15(1) RPBA 2020, the board gave, inter alia, the following preliminary opinion.
(a) The amendment deleting the term "orthogonal" from the subject-matter of claim 1 of both requests did not meet the requirement of Article 123(2) EPC.
(b) The subject-matter of claim 1 of both requests lacked inventive step over the disclosure of document D1 combined with the common general knowledge of the skilled person (Article 56 EPC).
VII. By letter dated 21 February 2023, the appellant withdrew its request for oral proceedings and requested a decision on the state of the file.
VIII. The oral proceedings scheduled for 28 March 2023 were cancelled.
IX. Claim 1 of the main request reads as follows:
"A method of sensing a time-varying image having at least one time-varying region of interest therewithin, the method comprising:
spatially modulating the time-varying image with an array of modulating pixels to form a modulated image, the array of modulating pixels having a modulator resolution; and
sensing the modulated image with an array of sensor pixels, the array of sensor pixels having a sensor resolution, the modulator resolution being finer than the sensor resolution;
wherein each sensor pixel corresponds to a plurality of modulating pixels, said sensor pixel and said plurality of modulating pixels subtending the same region in the time-varying image;
wherein outside the region of interest, the modulated image is synthesized at the sensor resolution and at a sensor video frame rate;
wherein inside the region of interest, the modulated image is synthesized at the modulator resolution and at an effective synthesized video frame rate that is less than the sensor video frame rate;
wherein each modulating pixel is switchable between an on state and an off state;
wherein the on state comprises directing at least a portion of time-varying image light, incident on said modulating pixel, to the modulated image; and
wherein the off state comprises blocking the time-varying image light, incident on said modulating pixel;
wherein inside the region of interest, said plurality of modulating pixels are modulated in a sequence of frames whereby each frame in the sequence has a different combination of modulating pixels in the on state; and
wherein in one frame of the sequence, all the modulating pixels in the plurality are in the on state."
X. Claim 1 of the auxiliary request differs from claim 1 of the main request in that it inserts, before the full stop, the following text:
", and said one frame in which all the modulating pixels in the plurality are in the on state is used to synthesize the modulated image inside the region of interest"
XI. The appellant's arguments relevant to the present decision can be summarised as follows.
(a) The deleted term "orthogonal" was not present in claim 1 as originally filed. The amended wording "sequence" was supported in the application as originally filed, for example, on page 5, line 26 and page 13, line 10 (see the statement of grounds of appeal, page 2, first and second paragraphs).
(b) The examining division's interpretation that the high spatial resolution images of the licence plates of vehicles constituted video images was not correct (see the statement of grounds of appeal, page 3, third to eighth full paragraphs).
(c) If the person skilled in the art wished to maximise light flux, they would have exposed shutter subsets for the full integration time Ti apart from a short transient period according to the embodiment of document D1 described on page 12, lines 19 to 21 (see the statement of grounds of appeal, page 5, fifth full paragraph).
1. The appeal is admissible.
2. Decision in written proceedings (Article 12(8) RPBA 2020)
2.1 The board informed the appellant in its communication pursuant to Article 15(1) RPBA 2020 that neither of the requests seemed to be allowable. By letter dated 21 February 2023, the appellant withdrew its request for oral proceedings and requested a decision according to the state of the file.
2.2 Hence, the board decided to cancel the oral proceedings and to issue a decision in writing in accordance with Article 12(8) RPBA 2020.
2.3 The principle of the right to be heard pursuant to Article 113(1) EPC is observed because the appellant's submissions are fully taken into account and because the present decision is based solely on grounds and evidence notified to the appellant and considered by the board in its communication under Article 15(1) RPBA 2020.
2.4 The appellant did not comment on or contest the board's preliminary opinion set out in its communication and did not provide any further arguments or submissions. Under these circumstances, the board sees no reason to deviate from its preliminary opinion, which is reflected in the following findings.
Annex(es): | |
3. Main request and auxiliary request - amendments (Article 123(2) EPC)
3.1 According to the consistent interpretation of Article 123(2) EPC in the case law, an amendment can only be made within the limits of what the person skilled in the art would derive directly and unambiguously, using common general knowledge, and seen objectively and relative to the date of filing, from the whole disclosure of the description, claims and drawings of the application as filed (see Case Law of the Boards of Appeal of the European Patent Office, 10th edition, 2022, "Case Law", II.E.1.1).
3.2 Claim 1 of both requests has been amended to delete the term "orthogonal" from the expression "orthogonal sequence" of frames.
3.3 The appellant's argument that the deleted term was not present in claim 1 as originally filed and that it had a basis in the description as originally filed (see point XI.(a) above) did not persuade the board for the following reasons.
3.3.1 Claim 1 as originally filed did not include a step of modulating pixels in a generic sequence of frames, and the dependent claims as originally filed used the expression "orthogonal sequence of frames". Thus, the claims as originally filed do not provide a basis for modulating pixels to form a generic sequence of frames.
3.3.2 Furthermore, the passages on page 5, line 26 and page 13, line 10 of the description as filed do not provide a basis for the amendment. In each of these passages, the term "sequence" is part of the expression "sequence of orthogonal frames", which is used interchangeably with an "orthogonal sequence" in the application. More specifically, in the embodiment of Figure 8, which according to the appellant corresponded to the subject-matter of claim 1, the claimed feature of combining modulating pixels in the on state (i.e. more than one) is disclosed only for a sequence of orthogonal frames (see page 13, lines 13 to 15 of the description).
Therefore, none of these passages indicates that the person skilled in the art would regard sequences of frames, other than orthogonal sequences of frames, as directly and unambiguously disclosed in the application for the embodiment of Figure 8.
3.4 In view of the above, the amendment deleting the term "orthogonal" from the subject-matter of the claims of the main request and the auxiliary request does not meet the requirement of Article 123(2) EPC.
4. Main request and auxiliary request - inventive step (Article 56 EPC)
4.1 The examining division identified document D1 as an appropriate springboard for the assessment of inventive step (see the decision under appeal, point 2.1 of the "Grounds for the Decision"). The appellant did not dispute this, and the board agrees with the examining division.
4.2 Document D1 discloses an imaging sensor comprising a shutter array in which multiple shutter elements, each smaller than an individual sensor pixel, control the exposure of each pixel sensor (see page 2, line 28 to page 3, line 4; page 11, lines 7 to 9). The spatial resolution of the sensor may be increased by opening different sets of shutter elements with each frame (see page 11, lines 11 and 12). In the board's view, it is implicit that to increase the spatial resolution, images from multiple frames obtained by opening different sets of shutter elements would need to be combined.
Based on user requirements, an image may be divided into regions imaged at different resolutions. This allows the selection of an optimal trade-off between frame rate and resolution for each region (see page 10, line 23 to page 11, line 13). For instance, top and bottom halves of an image may be captured at different resolutions and thus also at different frame rates (see page 10, lines 25 to 28).
In another example for vehicle speed monitoring, the sensor could be used to obtain specific regions of interest at different resolutions from the remaining regions of the image. That is, high spatial resolution images of licence plates may be obtained while continuously monitoring vehicle motion at low spatial resolution (see page 20, lines 6 to 15).
4.3 According to the appellant (see the statement of grounds of appeal, paragraph bridging pages 3 and 4 and first full paragraph on page 4), document D1 did not disclose the following features of claim 1 of the main request:
(a) wherein outside the region of interest, the modulated image is synthesized at the sensor resolution and at a sensor video frame rate;
wherein inside the region of interest, the modulated image is synthesized at the modulator resolution and at an effective synthesized video frame rate that is less than the sensor video frame rate
(b) in one frame of the (orthogonal) sequence, all the modulating pixels in the plurality are in the on state
4.4 Regarding feature (a), the appellant contested the interpretation of the examining division that the high spatial resolution images of the licence plates of vehicles constituted video images (see point XI.(b) above).
4.4.1 However, apart from the licence plate example, document D1 discloses an example in which image data is obtained for different regions at different resolutions and frame rates (see point 4.2 above, second paragraph). Since there is a trade-off between resolution and frame rate, a first region with a high resolution (for instance, a region of interest) would be captured at a lower frame rate than a second region with a lower resolution.
4.4.2 Document D1 does not exemplify obtaining image data for a first region at sensor resolution and for a second region at modulator resolution.
4.4.3 However, in the board's view, the selection of frame rate and resolution for the different regions depends on the capabilities of the image sensor and the requirements of the envisaged application. For example, in surveillance applications, it may be desirable to obtain image data in a region of interest at the highest available resolution dictated by the modulator pixels (shutter elements of D1), while it may be sufficient to capture the remaining image data in standard definition ("SD", 720x480 pixels) using the sensor resolution of an SD sensor. Thus, the person skilled in the art would have arrived at the claimed resolutions for the image regions in an obvious manner by choosing resolutions and frame rates in accordance with the constraints associated with the envisaged application.
4.5 Feature (b) has a technical effect separate from that of selecting appropriate resolutions and frame rates for different regions of the image. The formulation of a partial objective technical problem is thus appropriate (see Case Law, I.D.9.3.2).
4.5.1 In the decision under appeal, the examining division identified an improvement of the signal to noise ratio of light detectors (light flux) as the technical effect associated with feature (b) and the corresponding problem as maximising a light flux (see the decision under appeal, section 3 of the "Grounds for the Decision").
4.5.2 The board is not convinced by the appellant's argument that the person skilled in the art would have maximised the light flux by exposing shutter subsets for the full integration time Ti apart from a short transient period according to an embodiment of D1 (see point XI.(c) above).
The person skilled in the art trying to maximise the light flux would not have been limited to the teachings of document D1. Instead, they would have considered mask pattern sequences that provide as many non-redundant linear equations as shutter elements. This is a necessary condition for increasing spatial resolution (see the decision under appeal, section 3, second paragraph of the "Grounds for the Decision" or D1, page 12, lines 14 to 16).
4.5.3 The board agrees with the examining division that the person skilled in the art would have recognised that mask patterns including as many open shutter elements as possible maximise the signal to noise ratio of light detectors (for example, individual pixel sensors) and thus the quality of a captured image (see the decision under appeal, section 3, third paragraph of the "Grounds for the Decision"). Mask pattern sequences would be constructed using different individual mask patterns (see, for example, D1, page 11, lines 11 and 12) to obtain the non-redundant linear equations. Thus, the person skilled in the art would have designed each of the different individual mask patterns to maximise the light flux. Selecting one of the mask patterns with all shutter elements open, and the remaining patterns in the sequence including as many open shutter elements as possible, is an obvious choice to achieve the aim of maximising the light flux.
4.6 In view of the above, the subject-matter of claim 1 of the main request lacks inventive step over the disclosure of document D1 combined with the common general knowledge of the skilled person (Article 56 EPC).
4.7 The subject-matter of claim 1 of the auxiliary request adds the feature "and said one frame in which all the modulating pixels in the plurality are in the on state is used to synthesize the modulated image inside the region of interest".
The board interpreted claim 1 of the main request as including this feature. Thus, the subject-matter of claim 1 of the auxiliary request does not involve an inventive step for the same reasons as claim 1 of the main request.
5. Conclusion
5.1 Since none of the appellant's requests is allowable, the appeal is to be dismissed.
For these reasons it is decided that:
The appeal is dismissed.