T 1924/22 (Motion capture/TELECOM ITALIA) 06-08-2025

1. The admissibility of the appeal

In view of the facts set out at points I, III and IV above, the appeal fulfils the admissibility requirements under the EPC and is consequently admissible.

2. Summary of the invention

2.1 The application concerns capturing the motion of at least a part of the human body, such as the hand, for instance for monitoring the progress of a neurodegenerative disease, such as Parkinson's disease; see page 7, lines 2 to 7.

2.2 As shown in figure 1, the invention uses a wearable device (102), for example a glove (see figures 2a/2b), comprising a white colour calibration marker (202) and one or more coloured markers (203-207). The movement of the markers is captured by two cameras: a video camera (103) which captures a sequence of colour frames, and a range camera (104) which records a sequence of depth frames of the same scene. The outputs of the two cameras are fed to a processing unit (105) which reconstructs the three-dimensional (3D) positions of the markers and thus the body part and sends this information to a display (160); see page 14, line 23, to page 16, line 11.

2.3 The application acknowledges that optical motion capture systems, such as the "Kinect" sensor, for tracking a coloured glove in gaming applications were known in the prior art; see page 2, line 23, to page 4, line 27. These systems suffered from the drawbacks that their accuracy was limited and their performance depended on the ambient lighting; see page 5, lines 14 to 28.

2.4 The invention seeks to capture the motion of a hand in real time with a sufficient accuracy that kinematic quantities, such as the range, velocity, acceleration and frequency, of hand movements can be derived.

2.5 The depth information from the depth frames is used to isolate a validity area within the colour frames, the coloured markers being searched for based on their colours exclusively within said validity area; see page 7, lines 23 to 27. According to the paragraph bridging pages 11 and 12, the 3D positions of the markers are determined by processing a colour frame and a corresponding depth frame.

2.6 The white colour calibration marker in the centre of the palm (202) of the glove is used in a colour calibration step to compensate for the ambient illumination; see page 17, lines 20 to 22, and page 18, lines 8 to 10. There is also at least one second marker (203-207) for each fingertip whose movement is to be captured; see page 17, lines 23 to 27. The markers on the fingertips (203-207) have different colours from each other and the palm marker (202).

2.7 Figure 3 illustrates three functional steps (blocks) carried out by the processing unit: acquiring the colour and range frames to calculate the 3D trajectory of the markers (301); calculating kinematic quantities relating to the user's movements (302) and providing a quantitative evaluation of the user's motor skills (303); see page 19, lines 9 to 27.

2.8 Figure 4 concerns the "first block" (301), comprising frame acquisition (402), brightness calibration 403), segmentation (404) to derive a "validity area" considered to be part of the hand (see figure 5 and page 24, line 4, to page 27, line 31), colour calibration (405; see figure 6 and page 28, line 9, to page 32, line 27), colour correction (406) and detection of marker positions by locating "color blobs" (408); see figure 7, page 19, line 31, to page 24, line 1, and page 33, line 22, to page 37, line 27.

2.9 The details of the second and third "blocks" are not material to a decision in this case.

3. The board's understanding of the expression "Color calibration marker"

As explained in the oral proceedings, the board does not understand the expression in the independent system and method claims of auxiliary requests 2 and 3 "said at least two markers [...] comprising a color calibration marker [...] whose marker color is white" as implying a colour calibration step. The expression merely requires that one of the at least two markers has a marker colour that is white.

4. Document D1

4.1 The examining division, in its summons to oral proceedings dated 11 August 2021, raised novelty and inventive step objections starting from D1; see points 3 to 6.

4.2 The board notes that D1 relates to using a Microsoft "Kinect" sensor (see abstract, lines 14 to 15) as a depth camera to capture the positions of the fingertips of a glove worn by the user, each fingertip having a coloured marker; see page 4, paragraph bridging left and right columns. The present application acknowledges that the "Kinect" device comprises a range camera and an RGB (red green blue) camera; see page 3, line 28, to page 4, line 1. The system of D1 projects a light pattern onto a background pre-distorted according to the topology of the background, so that it appears undistorted to the user; see abstract. Figure 3 shows an interaction module flowchart in which, having acquired a depth map and RGB image of a scene, the pixels having depth values above a threshold are discarded; see second step and page 2, figure 2. The board understands this to achieve the "background subtraction", illustrated in figure 4, leaving pixels from the "foreground". Returning to the flowchart of figure 3, in the third step pixels lying outside the HSV (meaning "Hue Saturation Value") range of interest are discarded, leaving foreground pixels in the HSV range of interest. Figure 9a/b shows an example of a "Paint" application in which the user wears a glove having one red fingertip serving as a pointer. The system tracks the trajectory of the pointer to project an image of a drawing; see page 5, left column, last paragraph.

4.3 The board takes the view that the "background subtraction" disclosed in D1 (see figure 2 and page 2, right column, lines 5 to 10) to leave pixels not "above object_threshold values" (see figure 3, fourth step), which can be termed the "foreground", falls under processing at least one depth frame of said sequence of depth frames (see figure 2) for identifying a portion of at least one colour frame (the "foreground") of said sequence of color frames comprising pixels (not "only pixels", as claimed) representing said wearable device. The "foreground" in D1 is understood to define an area where pixels representing the wearable device can be found.

4.4 In the terms of claim 1 of the second auxiliary request, D1 discloses a motion capture system comprising: a wearable device (see abstract, lines 16 to 17; "colored fingertip gloves") suitable for being fitted on at least a part of the body of a user (hand), said wearable device comprising a marker (figure 9a/b fingertip of glove) having a respective marker color; a video camera (implicit in Kinect sensor) suitable for acquiring a sequence of color (RGB) frames of a scene comprising said wearable device, said marker being visible in said sequence of color frames; a range camera (implicit in Kinect sensor) suitable for acquiring a sequence of depth frames of said scene; and a processing unit (page 4, left column, 5th line from bottom; "i7 computer") configured to process said sequence of color frames received from said video camera and said sequence of depth frames received from said range camera for reconstructing three-dimensional positions of said at least one marker, said processing comprising processing at least one depth frame of said sequence of depth frames for identifying a portion of at least one colour frame of said sequence of color frames comprising pixels representing said wearable device and searching said marker in said at least one color frame based on said respective marker color (see figure 3, fifth step); and to capture a motion of said part of the body of said user on the basis of a sequence of said three-dimensional positions of said marker.

5. Inventive step, Article 56 EPC, auxiliary request 2

5.1 In view of the above analysis, the subject-matter of claim 1 of auxiliary request 2 differs from the disclosure of D1 in the following features:

a. said wearable device comprises at least two markers having respective marker colours different from each other, the marker colour of one of said markers being white,

b. said processing comprising processing at least one depth frame of said sequence of depth frames for identifying a portion of at least one colour frame of said sequence of color frames comprising only pixels representing said wearable device, said search being confined to said portion.

5.2 Features "a" and "b" solve separate, unrelated problems; each could be implemented on its own. Hence the board considers the contributions of differences "a" and "b" to inventive step separately.

5.3 Difference "a" solves the problem of tracking more than one body part, for example more than one finger. As stated above, this seems to be hinted at in D1 and would anyway have been implemented by the skilled person as a matter of usual design, for instance to recognise different hand gestures, such as a "pinch" gesture.

5.4 The appellant has argued that D1 does not disclose or suggest the glove having a white calibration marker. As set out above, the board construes the reference to a "colour calibration marker [...] whose marker color is white" as merely implying a white marker. The board regards white as an obvious design choice having no unexpected advantage. As claim 1 is not understood to imply a colour calibration step, the fact that white could sometimes aid colour calibration does not make the choice of white inventive.

5.5 Hence difference "a" would have been obvious to the skilled person starting from D1.

5.6 Limiting the search for the markers to "only pixels representing [the] wearable device" speeds up the search of markers itself but at the price of the time necessary to segment out the wearable device. The total time taken is not necessarily reduced. The board understands the limitation to improve the detection of colour markers because the marker colour occurring in places other than on the wearable device (e.g. the screen) will not be mistakenly detected as a colour marker. In the board's understanding, D1 addresses the same problem when it limits the search for the marker colour to the detected foreground; see, for instance, figure 3. The board thus takes difference "b" to solve the problem of improving the accuracy with which colour markers are detected. The board considers it to be obvious to solve that problem by further limiting the search for the pixels in the colour images relating to the markers and, in particular, by limiting that search to precisely the object of interest, i.e. the wearable device. In this respect, the board also notes that D1 stresses the importance of an accurate fingertip detection (see page 4, left column, item 1) and that claim 1 does not go beyond stating the goal of limiting the search to pixels relating to only the wearable device and contains no features as to how this is to be achieved.

5.7 The appellant argued in the oral proceedings that D1 did not disclose tracking the wearable device, in particular segmenting out individual fingers in the image. Moreover the approach in D1 relied upon red not being used on the screen, as, for instance, was the case in figures 9a/b, showing lines in grey, cyan and magenta on the screen.

5.8 The board is not persuaded by these arguments, since the system known from D1 must track the red marker in order to "paint" lines on the projected image; see figures 9a/b. By tracking the red finger marker, the system of D1 implicitly tracks the wearable device, i.e. the glove. The skilled person would have also been aware that the marker and part of the projected image could be the same colour, making it all the more necessary to distinguish between the wearable device and the projected image. Indeed, in the case shown in figures 9a/b of D1, the board regards it as unlikely that the skilled person would have accepted not being able to use the colour red in the projected image. Furthermore, claim 1 does not exclude part of the colour frames being the same colour as one of the at least two marker colours, for instance red.

5.9 Hence the board finds that difference "b" would also have been obvious starting from D1.

5.10 It follows that the subject-matter of claim 1 does not involve an inventive step, Article 56 EPC, in view of D1 alone.

6. Inventive step, Article 56 EPC, main request and auxiliary request 1

6.1 Interpreting claim 1 of these requests in the light of the description, in other words that "respective" implies "different" and that a white calibration marker is always present, claim 1 of these requests sets out the same subject-matter as that of auxiliary request 2.

6.2 On this interpretation, the board finds that claim 1 of the main request and auxiliary request 1 lacks inventive step, Article 56 EPC, for analogous reasons to those set out above for auxiliary request 2.

7. The admittance of new auxiliary requests 1 to 3 into the proceedings

7.1 The independent claims of new auxiliary requests 1 and 2 contain amendments to those of the main request filed in response to objections by the board under Articles 84 and 123(2) EPC which were raised for the first time in the board's (first) preliminary opinion, namely the clarifications that the respective marker colours were "different from each other" and "said at least two markers comprising a color calibration marker whose marker color is white". The board accepts that its new objections constituted exceptional circumstances, justifying admittance of the new requests, especially as auxiliary request 2 addresses both of the board's objections, Article 13(2) RPBA 2020.

7.2 The independent claims of new auxiliary request 3 are based on those of auxiliary request 4 filed with the grounds of appeal, amended by insertion of the two mentioned features of new auxiliary request 2.

7.3 The board stated in its communication of 28 January 2025 (point 7.5) that then auxiliary request 4 contained many new features relating to identifying the first marker in a colour frame, apparently based on original claim 6. These features - as a limitation of original claim 1 - were found to be obvious over common general knowledge in the WO-ISA, which was endorsed in the first European examination report. Otherwise, these features were not discussed during examination. It did not seem procedurally economical to introduce these features at this late stage of the proceedings. Consequently the board was not inclined to admit auxiliary request 4 into the appeal proceedings.

7.4 With a response, dated 6 March 2025, the appellant submitted new auxiliary requests 1 to 3, withdrawing auxiliary requests 1 to 4 currently on file.

7.5 The mentioned new features of auxiliary request 4 filed with the statement of grounds of appeal were maintained in new auxiliary request 3 and consequently do not fall under the remit of Article 13(2) RPBA. As they were only filed on appeal, the question arises whether theyshould have been submitted before the examining division (Article 12(6) RPBA).

7.6 In the WO-ISA, dependent claim 6 was found to lack an inventive step with only very cursory arguments. In its response to the first European examination report endorsing the WO-ISA, the applicant argued that the examining division's inventive step objection to claim 1 was incorrect. In its subsequent examination reports, the examining division only addressed deficiencies under Articles 84 and 123(2) EPC. In its annex to the summons to oral proceedings, the examining division also raised a novelty objection to claim 1 (of the still sole request) vis-à-vis D1. Dependent claim 6 was stated to lack inventive step, but with no specific reasoning; see points 5 to 5.5 of the annex to the summons. In its response to the summons, the applicant argued why the examining division's novelty objection was incorrect and only addressed the objections under Articles 84 and 123(2) EPC by way of amendment. Given the state and the focus of the examination proceedings, the board accepts that, up until that point, the applicant did not have occasion to amend independent claim 1 using the features of claim 6. The board also considers that the applicant's indication that it would not be present at the oral proceedings before the examining division does not change that conclusion. Therefore, the board concludes that Article 12(6) RPBA, first sentence, does not prevent admittance of auxiliary request 4 as filed with the grounds of appeal.

7.7 Nonetheless, admittance of auxiliary request 4 was at the board's discretion according to Article 12(4) RPBA. The board notes that inventive step had only been cursorily discussed during examination and therefore considers that that auxiliary request 4 constituted a reasonable fallback position in any future discussion on inventive step that might have been had once the objections in the decision under appeal had been overcome. The same applies mutatis mutandis to present auxiliary request 3.

7.8 Hence, in view of its considerations under Articles 12(4) and (6) RPBA regarding auxiliary request 4 as filed with the grounds of appeal and under Article 13(2) RPBA regarding present auxiliary request 2, the board admitted auxiliary request 3 into the appeal proceedings.

8. Remittal, Articles 111(1) EPC and 11 RPBA

8.1 The board finds that the claims according to auxiliary request 3 overcome the reasons for refusal and the objections under Articles 84 and 123(2) EPC raised by the board in points 8.6 and 9 of the provisional opinion of 28 January 2025, and otherwise comply with Article 123(2) EPC regarding added subject-matter and Article 84 EPC regarding clarity. The amendments also overcome the above inventive step objection which depends on the fact that claim 1 of the higher ranking requests lacks any details on the claimed segmentation procedure. Claim 1 of auxiliary request 3 now sets out such details.

8.2 In view of the fact that inventive step has not been decided upon in examination and said features (from original claim 6) have not been examined or discussed in any detail, the board finds that there are special reasons for remitting the case to the examining division for further prosecution, Article 11 RPBA.