T 2629/18 21-07-2021
Download and more information:
VEHICULAR CAMERA AND METHOD FOR PERIODIC CALIBRATION OF VEHICULAR CAMERA
Connaught Electronics Ltd.
TRW Systems Limited
Amendments - extension beyond the content of the application as filed (no)
Inventive step - main request (yes)
I. The appeals of the opponent 1 (appellant I) and the patent proprietor (appellant II) are directed against the decision of the opposition division to maintain European patent No. 2 523 831 in amended form on the basis of auxiliary request 6.
II. The appellant I relied on the following evidence filed during the opposition procedure:
- E1: US 2006/038895 A1;
- E3: US 2009/179773 A1;
- D6: JP 2005 077107 A.
The appellant I filed the following further evidence with its statement of grounds of appeal:
- E7: EP 1 550 982 A2.
III. In its decision the opposition division held that the ground of Art. 100(c) EPC prejudiced the maintenance of the patent as granted (main request) and that the auxiliary requests 1-5 filed on 28 February 2017 were likewise unallowable. The subject-matter of claim 1 according to the auxiliary request 6 as filed on 28 February 2017 complied with the requirements of Art. 123(2) EPC and was considered inventive in view of each of the combinations E1 and E3, D10 and E3, E1 and D6.
IV. Oral proceedings before the Board took place on 21 July 2021.
The appellant I (opponent 1) requested that the decision under appeal be set aside and the patent be revoked in its entirety.
The appellant II (patent proprietor) requested that the decision under appeal be set aside and the patent be maintained in amended form on the basis of the main request, or in the alternative, on the basis of one of the auxiliary requests 1-8, all requests filed with letter dated 21 May 2021.
As a procedural request the appellant II requested remittal of the case to the department of first instance in case E7 be admitted into the appeal proceedings.
The opponent 2 (party as of right) neither participated in the oral proceedings nor did it file any submissions or requests during the appeal proceedings.
V. Claim 1 according to the main request broken into a feature analysis adopted by the parties reads as follows (the deletions made in claim 1 as granted are indicated in strike-through and additions underlined):
M1.1 A vision system for a vehicle, said vision system
comprising:
Ml.2 a camera 11 having a lens 20 and an imager 16;
wherein the lens 20 is positioned to capture a
view exterior a vehicle,
M1.3 and wherein, with the camera 11 installed at the
vehicle, the view of the camera 11 includes a
portion of the vehicle;
M1.4* wherein the imager 16 has an imager
microcontroller and an image sensor that is
configured to receive images from the lens 20,
the imager microcontroller having a statistics
engine generating statistical data from the
images;
Ml.5* wherein the camera comprises a camera
microcontroller 18 operable to control the
application of an overlay 26 to the images 25;
M1.6 an in-vehicle display 24 operable to display
images captured by the imager 16; and
M1.7* the camera microcontroller 18 is operable to
receive statistical data from the imager
microcontroller of the imager 16
M1.8 and the camera microcontroller 18 is operable to
detect a reference point 34 in the images using
the statistical data
M1.9 and, responsive to a determination of an offset
of the detected reference point 34 relative to an
expected location of the reference point 34, the
camera microcontroller 18 is operable to
determine an offset amount with which to shift
the overlay on the images 25; and
characterized by:
Ml.10 wherein said camera microcontroller 18 is
operable to detect reference points during
reference point detection cycles
Ml.11* and wherein said reference point detection
cycles are conducted on different driving days
and at different driving times;
[deleted: triggered at least one of (i) while the vehicle is driving forward, (ii) while the in-vehicle display is not displaying camera images, (iii) while the vehicle is driving at least 40 km/hour, (iv) while the vehicle's steering angle is within a selected amount of degrees from zero, (v) while the outside temperature is within a selected range, (vi) while the vehicle headlights are off, (vii) while a heading direction of the vehicle is within a selected range, (viii) after a windshield wiper of the vehicle has been off for a selected period of time, (ix) while the time of day is within a selected range, (x) after an amount of time that the vehicle has been driven without stopping exceeds a selected amount of time, (xi) while a determination of an offset amount has not taken place already on the current day and (xii) after a selected period of time has elapsed since the previous determination of an offset amount;]
Ml.12 and responsive to the determined offset, said
camera microcontroller 18 shifts the overlay 26
applied to the displayed images.
VI. The appellant I (opponent 1) essentially argued as follows:
Objections under Article 123(2) EPC
The amendments in claim 1 of the main request extended beyond the content of the application as filed, namely:
- As regards the location for the camera according to feature M1.3, paragraph [0085] originally described only two possibilities for different locations and set thus a limitation to two specific locations for the camera, namely "side mounted or rear mounted". This paragraph also referred to images of the "bumper".
- Feature M1.7 as granted lacked a reference to a statistics engine. Amended features M1.4* and M1.7* gave the skilled person the impression that the statistics engine (stemming from paragraph [0041]) was generating any kind of statistical data, i.e. not only providing (as later described) certain types of statistical information. Moreover, paragraph [0042] recited that the camera microcontroller was "periodically using the statistical data provided by the imager microcontroller 16b". Paragraph [0051] described that the imager provided statistical information "regarding a search window from the image", so feature M1.7* was now claiming a broader embodiment which was not initially disclosed.
- The sole indication of feature M1.11* in the original description (paragraph [0046]) contained the explicit limitation "with no more than one successful reference point detection cycle per day", which was not included in claim 1. Moreover, feature M1.11* deviated considerably from granted feature M1.11 which specified many conditions.
When taking features from the description and at the same time some features were left out , the scope of protection was broadened in a way not originally disclosed.
Late-filed arguments on features M1.3 and M1.4* were occasioned by the patent proprietor having re-organised its requests at a late stage of the appeal proceedings.
Inventive step
Document E1 represented the closest prior art and disclosed features M1.1 to M1.10 and M1.12 of claim 1 as already described in the notice of opposition.
The subject-matter of claim 1 differed from E1 in that the reference point detection cycles were conducted on different days and at different driving times (M1.11*). It was clear that the calibration as described in E1 occurred at least once at some specific point in time.
The objective technical problem to be solved was adapting the calibration as disclosed in E1 to different driving situations.
Starting from E1 it was obvious for the person skilled in the art to combine the calibration as disclosed in one of documents E3, D6 or E7 (relating to calibration of a vehicle camera) to the one from E1 without the use of an inventive skill.
- In E3 automatic calibration of a vehicle camera was achieved by calibrating "each time the ignition system of the motor vehicle 1 is activated" (see paragraph [0130]). It was clear for a person skilled in the art that a motor vehicle was usually started at different days and at different times. Therefore, the indication when the calibration should be performed as defined in E3 was within the scope of feature M1.11*.
- According to D6 (paragraphs [0071] to [0074], see machine translation) the calibration routine was "repeatedly started at predetermined time intervals". The wording of feature M1.11* was vague, without any specific indication when to conduct the reference point detection cycle, and meant nothing else but conducting the cycles at predetermined time intervals.
- The automatic calibration as described in new evidence E7 (found in a further search in view of the surprising position of the opposition division) was performed depending on the use of the camera system or the loading state of the vehicle (see paragraphs [0022], [0023]: alternative calibration during normal use of the sensor, i.e. of camera or the car), which implied to conduct the calibration routine on different driving days and at different driving times, so feature M1.11* was known. E7 was highly relevant and should be admitted into the proceedings.
The same conclusion was reached when following the more general problem formulated by the Board in its preliminary opinion and taking into account the term "statistics engine" included in claim 1 of the new main request, which was not clearly defined. This term only defined and described generally the classical way how a microcontroller worked, as known to the skilled person. When considering how this term was used in the original disclosure (relating to a "search window"), it only described the way the pixels were discriminated and treated, as already known from E1 (paragraph [0153]).
E7 showed more than E3 and D6 in this respect. The pattern recognition referred to in paragraph [0021] of E7 ("Mustererkennung") was well-known at that time and corresponded to what was vaguely defined in claim 1. This term implied using information in a statistical way to provide the pattern, which was the most common way in the year 2007 when dealing with the issue on how to divide an image into pixels and then derive data of the image.
VII. The appellant II (patent proprietor) essentially argued as follows:
Objections under Article 123(2) EPC
The objections against features M1.3 and M1.4* were raised late without giving reasons to justify the late filing, which resulted in an amendment to the case of opponent 1 which had not been considered in opposition proceedings and in the grounds of appeal. Moreover, the opponent 1 did not raise an objection for the alleged failure to include the "statistics engine" in feature M1.7 of claim 1 as upheld in opposition proceedings, and opponent 2 raised a different objection.
If the new objection and argument was to be admitted into the proceedings, the case should be remitted to have the objection considered by two instances.
Feature M1.3 of claim 1 of the main request specified that the view of the camera included a portion of the vehicle, which did not present the skilled reader with new technical information. Features M1.4* and M1.7* were amended in direct response to the objection that a statistics engine was lacking in claim 1. The allegedly missing feature "no more than one successful reference cycle per day" in amended feature M1.11* was already implied in any normal reading of "on different driving days", as found by the opposition division (see paragraph 25 of the contested decision). Since no reason had been advanced why this finding ought to be considered as flawed, the opponent's objection should be rejected as insufficiently reasoned and the reasoning of the opposition division remained unchallenged.
Inventive step
Apart from features M1.5, M1.8 to M1.10 and M1.12 (and also amended features M1.4* and M1.7* specifying "statistical data"), which provided a further distinction over E1 (and on which the opposition division declined to express a view in its decision), feature M1.11* was lacking from E1 and also from E3. Hence, the combination of E1 and E3 did not render the claims obvious at least because M1.11* was disclosed neither in E1 nor in E3, as held in the contested decision. This applied mutatis mutandis to the combination E1 and D6. D6 did not contain any teaching of a predetermined time interval chosen to ensure calibration on different driving days and at different driving times. The entire argument on D6 relied on the phrase "repeatedly started at predetermined time intervals" which arose only in the context of an Espacenet automated translation, but which was no longer present in a current and likely more accurate translation of D6 into English (filed as E9, now reading on page 18: "The routine shown in FIG. 8 is repeatedly started every predetermined time"). Fig. 8 related to the post-installation factory calibration and to the worker having activated the adjustment/calibration mode on the vehicle (see page 11). Repeatedly starting the procedure meant only that the routine was running in the background. Therefore, D6 was only teaching a once-off calibration conducted in the factory on installation of the camera (see also page 10). It was not disclosed or suggested that the calibration was carried out repeatedly.
Moreover, E3 failed to disclose features M1.9 and M1.10, and D6 did not show features M1.8 to M1.10.
In view of the case law of the Boards of Appeal (see V.A.4.11.3 a)), a decision surprising the opponent was not a justification for the late filing of E7, so new evidence E7 should not be admitted into the appeal proceedings. No justification has been provided as to why it was not submitted or could not have been submitted at first instance. Moreover, it was requested not to admit E7 in view of the lack of prima facie relevance as a novelty-destroying document. Alternatively, remittal to first instance appeared to be appropriate.
E7 was not highly relevant, since several features of claim 1 were lacking from E7 with regard to claim 1 as upheld in opposition proceedings, namely:
- feature M1.3: no mention in E7 of capturing a portion of the vehicle in the camera image, but only of the roadway behind the vehicle
- feature M1.5: no mention of a control unit in E7
- feature M1.8: roadway test field mentioned in E7 was a two-dimensional marking, no reference point
- feature M1.9: no expected location of a reference point, i.e. no determination of an offset in E7
- feature M1.11*: assuming the interpretation of feature M1.10 of the opponent that a continually running calibration process was one in which reference point detection cycles (run on the same driving day) were inherent, feature M1.11* was lacking from E7
- feature M1.12: no disclosure of an overlay being moved by an offset amount (as determined by the movement of a reference point to an unexpected position) in E7.
Moreover, E7 was not relevant for claim 1 of the main request, which required a statistics engine and a camera comprising an imager with imager microcontroller and a camera microcontroller.
The skilled person recognised that E7 contemplated the use of a two-dimensional marking that was specifically established at an automobile production area, an automobile repair workshop or in a public street space (parking or lane markings). Thus it was inconceivable that a calibration event would span different days, much less different days and different times. The actual disclosure in E7 was a manual adjustment of the position of an image provided by the camera. Therefore, E7 was constructed differently and worked differently.
1. Amendments
1.1 The subject-matter of claim 1 of the main request does not extend beyond the application as originally filed (Articles 100(c) and 123(2) EPC).
1.2 The objection against feature M1.3 of claim 1 of the main request, which was already present in granted claim 1, is an objection under Article 100(c) EPC.
The original disclosure in paragraph [0085] might refer to a side mounted or rear mounted camera. However, in the same paragraph it is explicitly said that "the algorithm itself may be different for different models of vehicle and different cameras, and for different locations for the camera". This makes clear that the location of installation of the camera at the vehicle is not limited to e.g. the particular embodiment of Fig. 1 showing a rear mounted camera (as also confirmed by paragraph [0004]: "In a particular embodiment, the camera ... is positioned to receive images from behind a vehicle including a portion of the bumper of the vehicle"). Therefore, the Board finds that the application as filed provides a general disclosure of the camera being installed at the vehicle as specified in feature M1.3, without requiring any specific mounting location.
Paragraph [0085] of the application as filed also refers to images including a portion of the bumper of the vehicle, i.e. the view of the camera. However, as set out above, a specific exemplary mounting location for the camera and thus a specific view or image to be taken is not required. Moreover, the algorithm as described in the application as filed relies on a reference point detection based on images which include a portion of the vehicle, and this has been specified in feature M1.3.
Therefore, the Board concludes that feature M1.3 according to claim 1 of the main request does not amount to an unallowable intermediate generalisation.
1.3 Features M1.4* and M1.7* have been amended in response to the objection (see grounds of appeal of appellant I) that a "statistics engine" was lacking in claim 1, as originally disclosed in paragraph [0041].
A basis for the added term "the imager microcontroller having a statistics engine generating statistical data from the images" in feature M1.4* can be found in paragraph [0041] of the application as filed, reciting "the imager microcontroller 16b has a statistics engine which is capable of providing certain types of statistical information regarding the images".
The Board cannot share the view of the appellant I that claim 1 now means that any kind of statistical data was generated, since amended feature M1.4* explicitly refers to "statistical data from the images", as originally disclosed. Moreover, the wording of claim 1 already includes the aspect of periodically using the statistical data (as recited in paragraph [0042] of the application as filed), which allegedly was missing. Claim 1 requires detection of reference points in the images using statistical data (feature M1.8*) during reference point detection cycles (feature M.10) conducted on different driving days and at different driving times (feature M1.11*).
The allegation of appellant I that a further limitation in feature M1.7* to "statistical information regarding a search window" (as disclosed in paragraph [0051] of the application as filed) had to be included cannot be followed. First of all, the appellant I has failed to identify any functional and structural relationship with the combination of features specified in claim 1.
Moreover, paragraph [0051] (reciting "As noted above, the imager does not provide the image 25 itself to the microcontroller 18, but instead provides statistical information regarding a search window") refers back to paragraph [0041] (stating "the camera microcontroller 18 does not have access to the images 25 itself"). In paragraph [0041] a search window is only optionally defined ("the statistics engine ... can divide a selected search window ... into a plurality of vertically stacked bars"). Thus, the Board cannot see that "a search window" is inextricably linked to the statistical data of amended feature M1.7*.
1.4 As regards feature M1.11*, the Board fully concurs with the reasoning in the contested decision (section 25: no need to include 'with no more than one successful reference point detection cycle per day' ... because this is already implied in any normal reading of 'on different driving days') that the requirements of Art. 123(2) EPC are met.
The appellant I further argues, without substantiating which requirement of the EPC was not complied with, that (compared to granted feature M1.11) feature M1.11* did not define a larger number of conditions triggering the reference point detection cycles. In this respect, the Board also fully agrees with the reasoning in the contested decision (see sections 20 to 25) that the formal requirements of the EPC are fulfilled.
1.5 In view of the foregoing, the question whether or not late-filed arguments with respect to features M1.3 and M1.4* should be admitted into the appeal proceedings, or whether the opponent's objection regarding feature M1.11* should be rejected as insufficiently reasoned, can be left unanswered.
Moreover, since the Board took a position in favour of the appellant II regarding features M1.3 and M1.4*, its request to remit the case if the new objection was to be admitted into the proceedings can be disregarded.
2. Inventive step
2.1 The subject-matter of claim 1 according to the main request involves an inventive step (Article 56 EPC).
2.2 Undisputedly, E1 represents the closest prior art and fails to disclose at least feature M1.11*.
2.2.1 As regards the combination of E1 and E3, the Board fully concurs with the finding of the opposition division (see contested decision, section 33), which refuted the argument of appellant I that "each time the ignition of the motor is activated" as disclosed in E3 equated to "on different driving days and at different driving times" as claimed in feature M1.11*. Moreover, the Board cannot see that the calibration as performed in E3 was within the scope of feature M1.11*, as argued by the appellant I. Therefore, regardless of what other features are disclosed by E1 (such as a "statistics engine" now included in claim 1), the subject-matter of claim 1 is inventive over the combination E1 and E3.
2.2.2 Similarly, the Board agrees with the finding of the opposition division that there is no motivation for the skilled person to apply feature M1.11* to E1 based on the general disclosure of D6 (see contested decision, section 39). The general statement in D6, according to which the calibration routine was allegedly "repeatedly started at predetermined time intervals", does not prompt the skilled person to arrive at the particular (i.e. more specific) case claimed in feature M1.11*. This applies all the more in the light of the doubts raised by appellant II on what is disclosed in D6, based on a current translation (E9) of document D6, which seems to be teaching a once-off calibration conducted in the factory on installation of the camera and which has not been contested by the appellant I.
2.2.3 Also taking into account the disclosure of document E7, which was only filed with the grounds of appeal of the appellant I, the presence of inventive step has to be acknowledged already in view of feature M1.11* which is neither known nor suggested from E7, irrespective of whether E7 shows the use of statistical data.
According to the appellant I, feature M1.11* was disclosed in paragraphs [0022] and [0023] of E7. However, the Board concurs with appellant II that the passages referred to by appellant I only contemplate the use of two-dimensional markings at an automobile production area or repair shop, or in a public street place (such as parking or lane markings). The Board cannot see that the disclosure in E7 would suggest a calibration event spanning different days and different times, as required by feature M1.11*.
2.3 In view of the above, the question of whether document E7 had been filed in view of a surprising position put forward by the opposition division, or it should have been submitted at first instance, can remain unanswered.
Moreover, since the teaching of E7 cannot challenge the inventiveness of the subject-matter of claim 1 of the main request, remittal to the first instance for this reason needs not be considered.
For these reasons it is decided that:
1. The decision under appeal is set aside.
2. The case is remitted to the first instance with the order to maintain the European patent in amended form according to the main request (claims 1 to 15) filed with letter dated 21 May 2021 and a description to be adapted.