T 1010/20 (Saving digital content by person-based clustering/SAMSUNG) 27-10-2022
Download and more information:
System and method of saving digital content classified by person-based clustering
Inventive step - (no)
Claims - clarity (no)
I. The appeal lies from the decision of the examining division to refuse European patent application No. 08778382.5, which was published as international application WO 2009/014323.
II. The following document was cited in the decision under appeal:
D1: US 2004/0156535 A1, published on 12 August 2004.
III. The examining division decided that the subject-matter of claims 1 to 9 of the main request lacked novelty over document D1, and that the subject-matter of independent claims 1 and 9 of the first to fourth auxiliary requests was not inventive.
As obiter dicta the examining division expressed its opinion that claim 1 of the main request and independent claims 1 and 9 of the fourth auxiliary request added subject-matter beyond the content of the application as filed and that claim 1 of the first, second and third auxiliary requests and independent claims 1 and 9 of the fourth auxiliary request were unclear.
IV. With the grounds of appeal the appellant requested that the decision under appeal be set aside and that a patent be granted on the basis of the first auxiliary request considered in the decision under appeal, which thus became the sole request.
V. In a communication accompanying a summons to oral proceedings, the board expressed its preliminary opinion that claim 1 of the sole request did not fulfil the requirements of Articles 84 and 56 EPC.
VI. With a letter of reply the appellant informed the board that it would not appear at the oral proceedings and requested a decision on the present status of the file. The appellant did not provide any arguments in reply to the board's preliminary opinion.
VII. Oral proceedings were cancelled.
VIII. The appellant's final request was that the contested decision be set aside and that a patent be granted on the basis of the sole request.
IX. Claim 1 reads as follows:
"A system (100) for saving digital content classified by clustering on the basis of face recognition, comprising:
a database (180) to save a plurality of digital content classified by clustering on the basis of face recognition;
a data structure generation unit (120) to generate a data structure using the plurality of digital content, wherein the data structure is a linked list data structure that comprises person nodes (315) linked together and face nodes (325), wherein the face nodes are linked to the person nodes to save face information of the digital content that is determined to be the same face;
a face recognition unit (130) located on one or more processors to extract a face descriptor of new digital content to be saved in the database;
a cluster classification unit (150) located on one or more processors to classify the new digital content and the plurality of digital content by clustering on the basis of face recognition using the extracted face descriptor; and
a data structure update unit (170) located on one or more processors to update the data structure according to the classification of the cluster classification unit, wherein updating comprises applying to the data structure the cluster generation, deletion or combination performed by the cluster classification unit."
Application
1. The application concerns a solution for managing digital content such as photos in a database in view of the increasing demand to browse or retrieve content in diverse forms (see the international publication, page 1, paragraphs [2] and [3]).
The invention supports automatic classification of digital content by person-based clustering and saving the classified content in a database, including updating the database by addition or deletion of digital content (paragraph [6]).
The system according to the invention includes a data structure generation unit, a face recognition unit, a cluster classification unit, a data structure update unit, a database, and a content browsing unit (paragraph [32], Figure 1). The database stores digital content that includes images, such as photos or moving images, according to a schema as illustrated in Figure 2 (paragraphs [34] and [35]). The face recognition unit recognises a person in an image and extracts a face extractor to classify the content by person-based clustering (paragraph [38]).
The data structure generation unit generates a data structure using the schema of the database classified by person-based clustering. The data structure consists of a "two-dimensional (2D) linked list" with one linked list of "person nodes", each person node being linked to a corresponding linked list of "face nodes", as illustrated in Figure 3. Person nodes are generated by person-based clustering of digital content and linked in the list of person nodes. Face nodes classified as corresponding to a person are generated and linked to the respective person node, forming a linked list for each person node. In the example of Figure 11, the new "Face3-3" photo to be saved in the database is judged to correspond to "Person3" by the face recognition unit and the cluster classification unit and therefore the "Face3-3" node is added to the "Person3" node's list. (paragraph [37] and [57], Figures 3 and 11).
Sole request
2. Clarity - claim 1
2.1 As obiter dicta the examining division expressed its opinion that the feature "person nodes linked together" was unclear.
2.2 The appellant argued that the feature clearly explained how the nodes were interconnected and that it was evident from Figure 3 of the application what connection was referred to by that wording in the claim.
2.3 The board is however of the opinion that claim 1 does not describe the data structure in a clear manner supported by the description. For example, the claim does not specify that the face nodes corresponding to a same person are linked together in a linked list. The claim covers data structures for which there is no support in the description, such as a data structure in which each face node is connected individually to a corresponding person node instead of being connected in a linked list. The appellant's arguments are not convincing since a claim has to be clear from its wording alone.
2.4 Therefore, claim 1 does not satisfy the requirements of Article 84 EPC.
3. Inventive step - claim 1
3.1 In the decision under appeal, the examining division decided that the only feature distinguishing the claimed subject-matter from document D1 was "a linked list data structure that comprises person nodes linked together".
3.2 In its grounds of appeal, the appellant expressed its opinion that the examining division's interpretation of the prior-art document D1 was unjustly broad. In its novelty analysis, the examining division had incorrectly combined different embodiments of document D1 which were clearly disclosed as alternatives.
3.3 Document D1 discloses a system for managing digital images of people in a public venue, such as photographs taken by a theme park for sale to patrons visiting the theme park or photographs taken at weddings or parties, so that the images corresponding to a given person can be retrieved (paragraphs [0002] to [0004] and [0167]).
As an overview, document D1 describes the system as supporting four main steps (paragraph [0056], Figure 1) which are further detailed throughout the document as follows: subject remote identification (paragraphs [0070] to [0103]), image capture (paragraphs [0104] to [0109]), identified image storage (paragraphs [0110] to [0149]), and image distribution (paragraphs [0150] to [0191]). The board is thus of the opinion that a number of embodiments of D1 are to be understood as variations of such a system with the four main steps.
The system of D1 automatically establishes the identity of the subjects within a captured image with "remote identification" and stores the image. Individual patrons may then retrieve the images without requiring human intervention or conscious interaction (such as remembering a number). This facilitates the distribution of images to the proper patrons in a large public venue (paragraphs [0056] to [0058]). In further embodiments, document D1 also discloses the distribution per internet of images to patrons after smaller events such as a wedding or a party (paragraphs [0167] to [0169]) or the retrieval of images from an on-line photo sharing system or a personal image collection (paragraph [0191]).
After being captured, the image and other information, including the identification of the patron obtained by remote identification, are stored (paragraphs [0111] and [0112], Figure 2).
Different techniques of "remote identification" are disclosed in document D1. One of the techniques is face recognition used in direct subject recognition (see e.g. paragraphs [0071] and [0099] to [0102]). The image of a patron's face is analysed "for features particular to that face, including the distances between different features (such as eyes, base of the nose, and the center of the mouth), and more abstract metrics such as the eigenface and eigenfeature decomposition of the face" (paragraph [0101]).
3.4 In several embodiments of D1, a facial identifier or facial ID (for example, one corresponding to the Facial Identification Record (FIR) of Cognitec GmbH, Dresden), which is a numerical or complex description of the face, is generated during the process of facial recognition. When a new image is to be tested in a facial recognition program, the facial ID for a face in the new image is numerically compared with the facial IDs from the reference images in the database (paragraph [0136]). A facial ID corresponds to a "face descriptor" within the meaning of claim 1.
In several embodiments of document D1 based on face recognition, a database as shown in Figure 24B is used in which the name of the patron and a facial ID are stored (paragraphs [0137], [0141], [0149], [0177]).
In the board's opinion, the skilled person understands from document D1 that the ideas described in the first part of the description, as explained in point 3.3 above, equally apply to embodiments described in other parts of document D1 which are based on facial recognition and the use of facial IDs, such as the management and distribution of photos captured at events or used in a personal image collection.
3.5 Therefore, document D1 discloses in a single embodiment a system for saving digital content classified on the basis of face recognition comprising several features of claim 1. In particular, the system of D1 includes a database to save a plurality of digital content classified on the basis of face recognition and a "face recognition unit [...] to extract a face descriptor of new content to be saved in the database", as well as a classification unit to classify the digital content on the basis of face recognition using the extracted face descriptor. It also discloses a data structure update unit to update the data structure according to the classification.
The board further notes that those features are also disclosed in combination on page 27, claim 1, of document D1. In addition, document D1 discloses storing the electronic images in association with the facial identifier of each person represented within the electronic image, and using clustering for classifying images on the basis of face recognition (see paragraphs [0186] to [0190], and page 27, claim 6). In particular, document D1 teaches in paragraph [0188] that clustering techniques provide automated means for classifying images according to the people within the images.
In the board's opinion, it is clear from these passages that the clusters are generated and updated on the basis of face recognition using the extracted face identifiers. On page 27, document D1 also discloses joining clusters using manual input from the user (claim 7).
3.6 Claim 1 differs from that embodiment of D1 in that
(a) the data structure is a linked list that comprises person nodes linked together and face nodes, wherein the face nodes are linked to the person nodes to save face information of the digital content that is determined to be the same face;
(b) updating comprises applying to the data structure the cluster generation, deletion or combination performed by the cluster classification unit.
3.7 The distinguishing features solve the problem of implementing the automatic update of the digital content clusters of the system of D1 as images are input into the system.
3.8 As the cluster classification unit contains the logic for classifying content, it would have been obvious to use the unit's functionality for updating clusters as reflected in feature (b). Linked lists are a well known way of organising data records which are related to each other and in the system of D1 the content is classified into clusters of images corresponding to different persons. The skilled person would therefore implement the automatic update of the content clusters in the way specified in features (a) and (b). The board further notes that claim 1 does not explain in detail how the update operations of feature (b) are implemented, nor how the data structure of feature (a) is used in the claimed system.
3.9 The appellant did not contest the board's preliminary opinion nor provided any further arguments dealing with the board's preliminary reasoning, which essentially corresponds to the reasons given in this decision.
3.10 Therefore, the subject-matter of claim 1 is not inventive (Article 56 EPC).
Concluding remark
4. Since the sole request on file is not allowable, the appeal is to be dismissed.
Order
For these reasons it is decided that:
The appeal is dismissed.