International Business Machines Corporation

Patent Trials and Appeals Board

Dec 23, 2021

2020005550 (P.T.A.B. Dec. 23, 2021)

UNITED STATES PATENT AND TRADEMARK OFFICE
UNITED STATES DEPARTMENT OF COMMERCE
United States Patent and Trademark Office
Address: COMMISSIONER FOR PATENTS
P.O. Box 1450
Alexandria, Virginia 22313-1450
www.uspto.gov
APPLICATION NO. FILING DATE FIRST NAMED INVENTOR ATTORNEY DOCKET NO. CONFIRMATION NO.
15/964,687 04/27/2018 Eric W. Brown P201702862US01 1019
45725 7590 12/23/2021
Walder Intellectual Property Law PC
445 Crestover Circle
Richardson, TX 75080
EXAMINER
SONNERS, SCOTT E
ART UNIT PAPER NUMBER
2613
MAIL DATE DELIVERY MODE
12/23/2021 PAPER
Please find below and/or attached an Office communication concerning this application or proceeding.
The time period for reply, if any, is set in the attached communication.
PTOL-90A (Rev. 04/07)
UNITED STATES PATENT AND TRADEMARK OFFICE
BEFORE THE PATENT TRIAL AND APPEAL BOARD
Ex parte ERIC W. BROWN, MARIA ELEFTHERIOU, ANCA SAILER,
and CHING-HUEI TSOU
Appeal 2020-005550
Application 15/964,687
Technology Center 2600
Before CARL W. WHITEHEAD JR., GREGG I. ANDERSON, and
PHILLIP A. BENNETT, Administrative Patent Judges.
BENNETT, Administrative Patent Judge.
DECISION ON APPEAL
STATEMENT OF THE CASE
Pursuant to 35 U.S.C. § 134(a), Appellant1 appeals from the
Examiner’s decision to reject claims 1–9, and 11–20. We have jurisdiction
under 35 U.S.C. § 6(b).
We affirm.

1 “Appellant” refers to “applicant” as defined in 37 C.F.R. § 1.42(a).
Appellant identifies the real party in interest as International Business
Machines Corporation. Appeal Br. 2.
Appeal 2020-005550
Application 15/964,687

2
CLAIMED SUBJECT MATTER
The claims are directed to “presenting an augmented reality
representation to a medical professional associated with a patient’s medical
condition and/or treatment.” Spec. ¶ 1. Claim 1, reproduced below, is
illustrative of the claimed subject matter:
1. A method in a data processing system comprising at least
one processor and at least one memory, the at least one memory
comprising instructions executed by the at least one processor to
cause the at least one processor to implement a cognitive
healthcare system, wherein the cognitive healthcare system
operates to:
capturing, by a capturing mechanism of the cognitive
healthcare system, a real-time image of an area of a patient's body
being viewed by a medical professional via a head mounted
display (HMD) system;
identifying, by the cognitive healthcare system, one or
more body parts of the patient within the real-time image;
correlating, by the cognitive healthcare system, the one or
more identified body parts with the patient's electronic medical
records (EMRs) indicating the medical condition and/or
treatments associated with the patient; and
generating, by the cognitive healthcare system, an
augmented reality display in the HMD system, of one or more
areas of the patient’s body corresponding to the medical
condition and/or treatment of the patient overlaying the real-time
image of the area of the patient’s body, wherein a level of
information displayed in the augmented reality display is based
on a schedule of the medical professional such that the cognitive
healthcare system:
accesses a schedule of the medical professional
through a medical professional corpus or corpora of data;
determines an amount of time the medical
professional has to spend with the patient; and
Appeal 2020-005550
Application 15/964,687

3
displays the level of information in the augmented
reality display commensurate with the amount of time the
medical professional has to spend with the patient.
Appeal Br. 15–16 (Claims Appendix).
REFERENCES2
The Examiner relies on these references:
Name Reference Date
Ribble US 2014/0145915 A1 May 29, 2014
Balram US 2015/0088546 Al Mar. 26, 2015
Devam US 2016/0249989 Al Sept. 1, 2016
Robaina US 2018/0197624 Al July 12, 2018
Van Halteren US 2019/0013092 Al Jan. 10, 2019
REJECTIONS
Claims 1–6, 9, 11–14 and 16–19 stand rejected under 35 U.S.C. § 103
as being unpatentable over Ribble, Robaina, and Devam. Final Act. 3.
Claims 7, 15, and 20 stand rejected under 35 U.S.C. § 103 as being
unpatentable over Ribble, Robaina, Devam, and Van Halteren. Final Act.
16.
Claim 8 stands rejected under 35 U.S.C. § 103 as being unpatentable
over Ribble, Robaina, Devam, and Balram. Final Act. 19.
ISSUES
First Issue: Has the Examiner erred in finding Ribble, Robaina, and
Devam teach or suggest:
wherein a level information of displayed in the augmented
reality display is based on a schedule of the medical professional
such that the cognitive healthcare system:

2 Citations to the references are to the first named inventor/author only.
Appeal 2020-005550
Application 15/964,687

4
accesses a schedule of the medical professional
through a medical professional corpus or corpora of data;
determines an amount of time the medical
professional has to spend with the patient; and
displays the level of information in the augmented
reality display commensurate with the amount of time the
medical professional has to spend with the patient,
as recited in claim 1?
ANALYSIS
First Issue
The Examiner rejects claim 1 as obvious over the combined teachings
of Ribble, Robaina, and Devam. The Examiner relies on Ribble’s
augmented reality patient care system as teaching most of the claim
limitations. Final Act. 3–8. Relevant to the issues here, the Examiner finds
that Ribble’s description of scheduled tasks being performed by a care giver
using the augmented reality system teaches “wherein a level of information
displayed in the augmented reality display is based on a schedule of the
medical professional.” Final Act. 7 (citing Ribble ¶ 15). The Examiner
acknowledges that although Ribble teaches basing the level of information
displayed on a schedule, Ribble does not teach doing so by performing the
“accesses,” “determines,” and “displays” steps recited in claim 1.
The Examiner introduces Devam, finding that “Devam teaches that
the procedure to be performed and the virtual information provided in the
display is based on “procedural data” including “procedure time/date” such
that the user is presented with content commensurate with this procedure
time.” Final Act. 10 (citing Devam ¶¶ 35–42). The Examiner notes that
Devam “does not explicitly teach accessing a schedule of the medical
Appeal 2020-005550
Application 15/964,687

5
professional through a medical professional corpus.” Final Act. 10–11. The
Examiner cites Robaina, finding that
Robaina teaches using context information such as which
physician is performing a type of procedure “based on the
scheduling information of the operating room” where such data
can be “accessed from the remote data repository 280 or the
healthcare database system 1220” and level of information
corresponds to such “virtual content based on contextual
information” such as for example identifying “relevant regions
which the surgeon should operate on.”
Final Act. 11 (citing Robaina ¶¶ 310–316). The Examiner explains:
Thus Robaina teaches that a level of information displayed in the
AR display is based on a schedule of the medical professional
such that the medical professional wearing the AR system sees
the level of information associated with the scheduling
information of the room they are scheduled to provide service in
which further determines an a mount of time the medical
professional has to spend with a patient such that they have been
scheduled an amount of time they have to perform the service or
procedure. Thus if the scheduling information in Robaina is for
an amount of time pertaining to a minimally invasive heart
surgery the level of information is commensurate with this
amount of time the medical professional has to perform such
surgery. Similarly, if the scheduling information indicates a
procedure on a leg for the medical professional to perform, the
level of information is pertaining to the leg surgery based on the
amount of time the medical professional has to perform the
scheduled procedure. Thus Robaina teaches a known technique
for accessing a schedule of a medical professional through an
appropriate database and determining an amount of time the
medical professional has to spend with the patient based on the
scheduled service and then displaying the level of information in
the AR display commensurate with the scheduled time to
perform the service.
Final Act. 11–12.
Appeal 2020-005550
Application 15/964,687

6
Appellant contends the Examiner has erred in several respects. First,
Appellant argues the cited references fail to teach “wherein a level of
information displayed in the augmented reality display is based on a
schedule of the medical professional.” Appeal Br. 8–10. Specifically,
Appellant argues “Ribble does not determine an amount of time the medical
professional has to spend with the patient and thus, could not display a level
of information in the augmented reality display commensurate with the
amount of time a medical professional has to spend with the patient.”
Appeal Br. 8 (emphasis omitted). Appellant further argues Devam is
deficient because “Devam only shows the date and time of the procedure but
not the length of the procedure,” and that Robaina does cure these
deficiencies because “[w]hile there is a portion [in] Robaina that discusses
displaying the doctor’s schedule of the day, nowhere does Robaina identify
an amount of time the medical professional has to spend with the patient.”
Appeal Br. 9–10 (emphasis omitted). Appellant further asserts that to the
extent that Robaina system relies on a schedule, it relies on the schedule of
an operating room, and not on the schedule of a medical professional.
Appeal Br. 11.
We are not persuaded of error. Ribble teaches an augmented reality
system which is “configured to assist a caregiver by displaying, among other
things, information, tasks, protocols, and control options associated with the
patient and/or equipment in the vicinity of the patient.” Ribble ¶ 13. Ribble
teaches that the augmented reality display is configured to provide medical
information in the augmented reality display that overlays the patient’s body.
Ribble ¶ 23 (“In another contemplated embodiment, one of the programs
displays a patient's EEG readings in a menu adjacent to their heart and the
Appeal 2020-005550
Application 15/964,687

7
user can select the menu to read the EEG chart.”). Ribble teaches that the
information that is displayed may be based on an assigned task or scheduled
procedure. Ribble ¶ 15 (“[T]he display device displays information about
the status of the medical equipment . . . , the physiological characteristics of
the patient . . . , medicine management tasks . . . , care plan tasks for the
caregiver (patient turn at 2:15, . . patient prep for surgery at 7:30).”). Thus,
Ribble demonstrates that it was known in the art to correlate and/or tailor the
information displayed in the augmented reality system to the nature of the
medical treatment that is given.
As the Examiner acknowledges, Ribble does not specifically teach the
level of information displayed is generated by the system accessing a
schedule . . . , determining an amount of time . . . , and displaying the level
of information . . . commensurate with the amount of time the medical
professional has to spend with the patient, and relies on Devam and Robaina.
Devam “relates to provision of a medical overlay in virtual reality,
augmented reality, projected or other virtual space.” Devam ¶ 2. Devam
teaches a medical augmented reality system in which “imaging data is
interpreted into an Augmented Reality (‘AR’) or Virtual Reality (‘VR’) view
of a patient, to be shown to a doctor, surgeon or other medical practitioner
during a procedure in order to enhance the accuracy and efficacy of the
procedure.” Devam ¶ 25.
Devam teaches that the information that is displayed is derived from
“procedural data” that is retrieved from a database, and specific to the
procedure to be performed on the patient. Devam ¶ 38. The procedural data
includes “procedural instructions [and] procedure time/date.” Devam ¶ 38.
Thus, in the case of a lengthy, complex, surgical procedure, a detailed set of
Appeal 2020-005550
Application 15/964,687

8
information is displayed to the user. See, e.g., Devam ¶¶ 69–71 (describing
information presented via the heads up display during medical procedures).
Devam further describes that in situations where the doctor has a more time-
limited interaction, more limited information may be presented. For
example, Devam teaches that when a doctor is doing rounds, the heads up
display may simply display patient vital signs. Devam ¶ 102. We agree
with the Examiner that a person or ordinary skill in the art, possessing these
teachings of Devam, would understand that the information presented to the
user in the heads up display should be commensurate with the complexity of
the patient encounter. Although Biddle and Devam do not directly base the
level of information presented on the amount of time, they do so indirectly
because a person of ordinary skill in the art would have understood that
different types of procedures require different amounts of time commitments
from medical professionals. Thus, the combined teachings of Biddle and
Devam demonstrate that it was known in the art to at least indirectly base the
level of information displayed in the augmented reality display on the
amount of time that the medical professional spends with the patient.
Although Appellant argues Devam describes only the time and date of a
medical procedure, but not the length of the procedure, we agree with the
Examiner that Devam’s use of procedural data to determine what
information needs to be displayed on the heads up display teaches to display
teaches a “level of information . . . commensurate with the amount of time
the medical professional has to spend with the patient,” as claimed. See
Ans. 24 (explaining that the augmented reality system displays “appropriate
information to guide the surgeon during the a mount of time the medical
professional has to perform the surgery and at all times the level of
Appeal 2020-005550
Application 15/964,687

9
information is commensurate with the amount of time they have to perform
. . . during a detailed scheduled surgery”).
Although Biddle and Devam do not expressly teach accessing the
schedule of the medical professional to determine the amount of time that
the medical professional has for the patient, we agree with the Examiner that
Robaina demonstrates that it was known in the art to access a scheduling
information from a remote repository such as a healthcare database system
in the context of augmented reality systems. See Robaina ¶¶ 311 (“The
wearable device can determine contextual information based on data
acquired by one or more environmental sensors, data accessed from the
remote data repository 280 or the healthcare database system 1220, in
combination or the like.”), 316 (“determine that the surgeon is performing a
minimally invasive heart surgery based on the scheduling information of the
operating room”), 318 (“[B]ased on the contextual information . . . . the
wearable device can access scheduling information of the operating room.”).
Thus, taken together, we agree with the Examiner that the respective
teachings of the references would have made it obvious to base the level of
detail displayed on the amount of time the medical professional anticipates
for the patient encounter. Appellant argues that Robaina teaches accessing
the schedule of the operating room, and not of the doctor. We disagree that
such a distinction is sufficient to avoid obviousness in this case. As
explained by the Examiner, a person of ordinary skill in the art would have
understood that when an operating room is scheduled for a specific
procedure, the operating surgeon would also be scheduled into that
procedure. See Ans. 26 (“While Robaina teaches a schedule of the operating
room is used to determine AR information, it must be understood that in the
Appeal 2020-005550
Application 15/964,687

10
system the doctor scheduled to perform some procedure at the operating
room which has a schedule meaning that the schedule of the operating room
at those points is the same as the schedule of the medical professional.”). As
such, we agree with the Examiner that a person of ordinary skill in the art
would have understood that the operating room schedule reflected the
schedule of the medical practitioner, and tailoring the information based on
that schedule would have been obvious to a person having ordinary skill in
the art. We, therefore, are not persuaded by Appellant’s arguments, and we
sustain the rejection of claim 1.
Remaining Claims
Appellant presents no separate arguments for patentability of any
other claims. Accordingly, we sustain the Examiner’s rejections of these
claims for the reasons stated with respect to the independent claims from
which they depend. See 37 C.F.R. § 41.37(c)(1)(iv).
CONCLUSION
We affirm the Examiner’s decision to reject the claims.
DECISION SUMMARY
Claims
Rejected
35 U.S.C.
§
Reference(s)/Basis Affirmed Reversed
1–6, 9, 11–
14, 16–19
103 Ribble, Devam,
Robaina
1–6, 9, 11–
14, 16–19

7, 15, 20 103 Ribble, Devam,
Robaina, Van
Halteren
7, 15, 20
8 103 Ribble, Devam,
Robaina, Balram
8
Overall
Outcome
1–9, 11–20

Appeal 2020-005550
Application 15/964,687

11
TIME PERIOD FOR RESPONSE
No time period for taking any subsequent action in connection with
this appeal may be extended under 37 C.F.R. § 1.136(a). See 37 C.F.R.
§ 1.136(a)(1)(iv).
AFFIRMED