Alan C. Cannistraro et al.

Patent Trials and Appeals Board

Dec 31, 2019

13267749 - (D) (P.T.A.B. Dec. 31, 2019)

UNITED STATES PATENT AND TRADEMARK OFFICE
UNITED STATES DEPARTMENT OF COMMERCE
United States Patent and Trademark Office
Address: COMMISSIONER FOR PATENTS
P.O. Box 1450
Alexandria, Virginia 22313-1450
www.uspto.gov
APPLICATION NO. FILING DATE FIRST NAMED INVENTOR ATTORNEY DOCKET NO. CONFIRMATION NO.
13/267,749 10/06/2011 Alan C. Cannistraro P10853US2/77870000120102 9258
150004 7590 12/31/2019
DENTONS US LLP - Apple
4655 Executive Dr
Suite 700
San Diego, CA 92121
EXAMINER
ADESANYA, OLUJIMI A
ART UNIT PAPER NUMBER
2658
NOTIFICATION DATE DELIVERY MODE
12/31/2019 ELECTRONIC
Please find below and/or attached an Office communication concerning this application or proceeding.
The time period for reply, if any, is set in the attached communication.
Notice of the Office communication was sent electronically on above-indicated "Notification Date" to the
following e-mail address(es):
dentons_PAIR@firsttofile.com
patents.us@dentons.com
PTOL-90A (Rev. 04/07)

UNITED STATES PATENT AND TRADEMARK OFFICE
____________

BEFORE THE PATENT TRIAL AND APPEAL BOARD
____________

Ex parte ALAN C. CANNISTRARO, GREGORY S. ROBBIN, CASEY M.
DOUGHERTY, RAYMOND WALSH,
and MELISSA BREGLIO HAJJ
____________

Appeal 2018-002673
Application 13/267,749
Technology Center 2600
____________

Before MAHSHID D. SAADAT, ERIC B. CHEN, and
CARL L. SILVERMAN, Administrative Patent Judges.

SILVERMAN, Administrative Patent Judge.

DECISION ON APPEAL
Appellant1 appeals under 35 U.S.C. § 134(a) from the Examiner’s
Final Rejection of claims 1–8 and 21–28, which constitute all pending
claims. We have jurisdiction under 35 U.S.C. § 6(b). Appellant waived an
Oral Hearing that was scheduled for November 5, 2019.
We REVERSE.

1 Throughout this Decision, we use the word “Appellant” to refer to
“applicant” as defined in 37 C.F.R. § 1.42(a). Appellant identifies Apple
Inc. as the real party in interest. Appeal Br. 3.
Appeal 2018-002673
Application 13/267,749
2
STATEMENT OF THE CASE
The invention relates to mapping audio and text. Abstract; Spec. ¶¶ 8,
23, Figs. 8A, 8B, 5B. Claim 1, reproduced below, is exemplary of the
subject matter on appeal (emphases added):
1. A method comprising:
obtaining an annotation that comprises text data or voice
data input from a user;
storing an association between the annotation and a
specified location within a textual version of a work;
inspecting a mapping between a plurality of audio
locations in an audio version of the work and a corresponding
plurality of text locations in the textual version of the work,
wherein the mapping is based on an audio-to-text analysis of
the audio version of the work, to:
determine a particular text location, of the
plurality of text locations, that corresponds to the
specified location, and
based on the particular text location, determine a
particular audio location, of the plurality of audio
locations, that corresponds to the particular text
location;
providing the annotation and the particular audio
location to a media player to cause the media player to
display or play the annotation during playback of the
audio version of the work at a particular time based on
the particular audio location;
wherein the method is performed by one or more
computing devices.

Appeal Br. 25 (Claims App.).

Appeal 2018-002673
Application 13/267,749
3
THE REJECTIONS2
Claims 1–8 and 21–28 are rejected under pre-AIA 35 U.S.C.
§ 103(a) as being unpatentable over Starmen (US 2010/0324709 A1; pub.
Dec. 23, 2010) ("Starmen") in view of Spielberg et al (US 2006/0143559
A1; pub. Jun. 29, 2006) ("Spielberg") and Murphy US 8,484,027 B1; iss.
Jul. 9, 2013) ("Murphy"). Final Act. 6–24.

REFERENCES
Name Reference Date
Spielberg US 2006/0143559 A1 Jun. 29, 2006
Rosenberg US 2006/0194181 A1 Aug. 31, 2006
Starmen US 2010/0324709 A1 Dec. 23, 2010
Murphy US 8,484,027 B1 Jul. 9, 2013

ANALYSIS
Appellant argues, inter alia, that the Examiner errs in finding the
combination of Starmen, Spielberg, and Murphy teaches the claim 13
limitations (also referred to as disputed limitations):
inspecting a mapping between a plurality of audio
locations in an audio version of the work and a corresponding
plurality of text locations in the textual version of the work,
wherein the mapping is based on an audio-to-text analysis of
the audio version of the work, to:
determine a particular text location, of the
plurality of text locations, that corresponds to the
specified location, and
based on the particular text location,
determine a particular audio location, of the

2 The 35 U.S.C § 112 rejections of claims 9–20 and 29–40 are withdrawn in
view of Appellant’s cancelling these claims. See Ans. 2; Advisory Action 2.
3 Appellant argues claims 1, 3–7, and 21–28 as a group, and we choose claim
1 as representative of the group. See 37 C.F.R. § 41.37(c)(1)(iv).
Appeal 2018-002673
Application 13/267,749
4
plurality of audio locations, that corresponds to
the particular text location.

Appellant also argues that the Examiner provides insufficient
motivation for one of ordinary skill in the art to combine Starmen, Spielberg,
and Murphy.

Appeal Br. 5–19; Reply Br. 3–14.
In particular, Appellant argues that the Examiner misconstrues claim 1
and errs in applying the teachings of Spielberg. Id. According to
Appellant’s arguments, the disputed limitations can be considered to
require4:
obtaining an annotation from a user;
storing an association between the annotation and a
specified location within a textual version of a work;
inspecting a mapping between audio locations in an
audio version and corresponding text locations in the textual
version of the work to:
(Determination 1)
determine a particular text location that corresponds to the
specified location; and
(Determination 2)
based on the particular text location, determine a particular
audio location that corresponds to the particular text location.

Id. (Emphasis added; the terms “Determination 1” and “Determination 2”
added).
In the Final Action, the Examiner finds that Spielberg teaches
mapping between audio locations in an audio version of the work and

4 The Examiner refers to Murphy for teaching the limitation, “wherein the
mapping is based on an audio-to-text analysis of the audio version of the
work.” See, for example, Final Act. 8–9.
Appeal 2018-002673
Application 13/267,749
5
corresponding text locations in a textural version and teaches the disputed
limitations of claim 1:
inspecting a mapping between a plurality of audio
locations in an audio version of the work and a corresponding
plurality of text locations in the textual version of the work to:
determine a particular text location, of the plurality of text
locations, that corresponds to the specified location (The server
computer comprises an annotator that is configured to associate
the audio comment file with a location in the document that
corresponds to the portion of the audio file playing when the
user provided the comment. . . . ., para. [0024]-[0026]; para.
[0028]-[0030]; para. [0138]; Navigator 1806 uses the line
ID/audio map 1504 to synchronize the current display line with
the audio playback . . . ., para. [0158]; In one embodiment,
selection of an annotation scrolls the display to the line
associated with the annotation and may initiate playback of the
audio annotation file . . . . , para. [0165],
synchronizing/associating audio comment/annotation with
location in text document of multiple locations that correspond
to audio location in file of multiple audio locations as implying
limitation), and based on the particular text location, determine
a particular audio location, of the plurality of audio locations,
that corresponds to the particular text location (para. [0024];
para. [0167]); providing the annotation and the particular audio
location, to a media player to cause the media player to display
or play the annotation during playback of the audio version of
the work at a particular time based on the particular audio
location (para. [0024]; para. [0071]; para. [0167]; para. [0185]).

Final Act. 7–8.
In the Appeal Brief, Appellant argues that, although Spielberg
describes determining a location in document, it does not do so by
“inspecting a mapping” as required by claim 1. Appeal. Br. 9. Appellant
argues that, even assuming Spielberg’s “line ID/audio map” is accepted as a
“mapping” of claim 1, Spielberg’s “line ID/audio map” is not inspected to
Appeal 2018-002673
Application 13/267,749
6
perform the two determinations5 required by claim 1. Appeal Br. 14 (citing
Spielberg ¶¶ 28, 71, 161, 164, 167, 185, 186). According to Appellant:
Thus, when a user listens to the audio file and initiates a
command to create an annotation, Spielberg's system uses the
“line ID/audio map” to determine a document line number to be
associated with the annotation. As such, Spielberg at most
discloses using its “line ID/audio map” to, based on a location
in the audio file, identify a text location (i.e., the document line
number) to be associated with the annotation. Spielberg's
implementation is therefore exactly the opposite of an
implementation involving “based on the particular text location,
determine a particular audio location,” as required by claim 1.
Accordingly, Spielberg does not teach or suggest the claimed
limitation.
Put another way, Spielberg converts a text file into an
audio file and allows the user to create annotations when
consuming the audio file (as opposed to the textual version).
As such, it naturally follows that Spielberg's system first
obtains an audio location in the currently-playing audio file and
then uses its “line ID/audio map” to identify a text location (i.e.,
the document line number). It is nonsensical for Spielberg's
system to “inspect[] a mapping . . . to: . . . based on the
particular text location, determine a particular audio location, of
the plurality of audio locations, that corresponds to the
particular text location” as recited in claim 1, since Spielberg
implements the exact opposite functionality of the claimed
method (annotations made in an audio context, rather than a
textual context).
Indeed, the Examiner's arguments demonstrate a
misunderstanding as to Appellants' position regarding
Spielberg. Appellants do not simply argue that Spielberg fails
to teach a mapping, or that the mapping is used to “synchronize

5 As discussed, supra: (Determination 1) determine a particular text
location that corresponds to the specified location; and
(Determination 2) based on the particular text location, determine a
particular audio location that corresponds to the particular text
location.

Appeal 2018-002673
Application 13/267,749
7
a current text display line with audio playback.” Rather,
Appellants argue that Spielberg fails to teach a mapping in the
manner claimed. Claim 1 explicitly requires, inter alia, that
inspection of the mapping results in determining “a particular
text location” and “based on the particular text location,”
determining “a particular audio location.” Such a feature is
simply not taught in Spielberg.

Appeal Br. 14–15.
In the Answer, the Examiner refers to the Specification use of
mapping audio to text, audio locations as time offsets, and text locations as
including line numbers. Ans. 12–13 (citing Spec. ¶¶ 23, 28, 29, 31, 74).
The Examiner finds:
Spielberg discloses a method of augmenting audio data
i.e. audio version of work with text data i.e. textual version of
work by the use of a map/mapping associating time codes for
the audio data with line numbers of the original text document,
where the line number-time code information in the audio file
is used to navigate within the audio file (Abstract) and where
the line numbers for the text in the display are synchronized
with audio through the use of the line identifier-time code
map 1504 (or line code data channel 1505) generated during
the text-to-speech conversion process (para. [0138]). Spielberg
further discloses enabling a user to add audio/verbal/comment
annotations to the textual document (para. [0019]) where an
annotator module associates the audio/verbal/comment
annotations file with a location in the text document (para.
[0023]; para. [0024]), and using the line number-time code
information to associate bookmark links and captured
audio/verbal/comment annotations with line numbers of the
original text document (Abstract).
Therefore,
1). Since Spielberg discloses using the line number-time code
information (i.e. text location-audio location map/mapping) to
associate captured audio/verbal/comment annotations with line
numbers of the original text document (Abstract), Spielberg
Appeal 2018-002673
Application 13/267,749
8
discloses inspecting a mapping to determine a text location/line
number among a plurality of text locations that corresponds to
the location/specified location of the annotation, i.e. limitation
“inspecting a mapping between a plurality of audio locations in
an audio version of the work and a corresponding plurality of
text locations in the textual version of the work . . . . to:
determine a particular text location, of the plurality of text
locations, that corresponds to the specified location”, and
2). Since Spielberg discloses using the line number-time code
information map/mapping 1504 to associate captured
audio/verbal/comment annotations with line numbers of the
original text document (Abstract), Spielberg discloses to
determine a particular audio location, of the plurality of audio
locations, that corresponds to the particular text location,
because the “line numbers” location of text in the line number-
time code information map/mapping 1504 include the
corresponding/associated “time code” or location of audio (see
Spielberg, para. [0135]).
Nevertheless, Spielberg further explicitly states “The
audio file 1500 provides the information used to determine the
respective audio time code for a given line number” (para.
[0158]), where the audio file 1500 (which includes the line
number-time code information 1504) provides the
information used to determine the respective audio time code
for a given line number, i.e. based on the particular text
location/given line number, determine a particular audio
location/audio time code, of the plurality of audio locations/
audio time codes, that corresponds to the particular text
location/ given line number
Therefore, Spielberg discloses “. . . . and based on the
particular text location, determine a particular audio location,
of the plurality of audio locations, that corresponds to the
particular text location”.
Furthermore, the line number-time code information
map/mapping 1504 used to associate captured audio
annotation files with line numbers of the original text document
(Spielberg, Abstract) obviates appellant's argument that it is
nonsensical for Spielberg's system to inspect a mapping, as the
inspected line number-time code information map/mapping
Appeal 2018-002673
Application 13/267,749
9
1504 is used obtain the location where the annotation is stored
i.e. the particular or specified location, and not the opposite of
the claim language like appellant is arguing.

Ans. 13–16.
In the Reply Brief, Appellant reiterates arguments presented in the
Appeal Brief and refers to the interpretation of the disputed limitations:
Claim l requires “determin[ing] a particular text location,
. . . ., that corresponds to the specified location [within a textual
version of the work]”6 1 and “"based on the particular text
location, determine a particular audio location, . . . . . ., that
corresponds to the particular text location.” Accordingly, claim
1 requires “determin[ing] a particular text location”
corresponding to “the specified [textual] location” and then
“based on the particular text location [corresponding to the
specified textual location], determin[ing] a particular audio
location.” Such feature is simply absent in the Spielberg
reference the Examiner relied upon for allegedly teaching the
instant claim limitation.

Reply Br. 3.
Appellant argues the Examiner improperly cited additional paragraphs
135 and 158 of Spielberg in the Answer, and the Examiner errs in finding
that the navigator 1806 of these paragraphs determines an audio time code
for a line number using the “lineID audio map” and teaches “based on the
particular text location, determine a particular audio location, of the plurality
of audio locations, that corresponds to the particular text location of claim
1.” Id. at 6 (citing Spielberg ¶ 158). Appellant argues the Examiner’s

6 According to Appellant, “[b]y virtue of antecedent basis, '"the specified
location" refers to '"a specified location within a textual version of a work"
in claim l. Reply Br. 3.
Appeal 2018-002673
Application 13/267,749
10
findings are based on failure to consider the antecedent basis required by
claim 1. Id. According to Appellant:
As discussed, by virtue of antecedent basis, claim 1
requires that "a particular audio location" is "determin[ed]"
"based on the particular text location [determined to correspond
the specified location within a textual version of a work]."7
Simply put, claim 1 requires “determin[ing] a particular audio
location” “based on” a “text location” previously determined
to correspond to a “specified” textual location. This
requirement of claim l is schematically illustrated in the chart
below (page 8). In contrast, even if Spielberg's audio time code
(corresponding to an annotation) were an “audio location” and
if Spielberg's line number were a “text location” (which
Appellant does not concede), Spielberg's line number used to
determine Spielberg's audio time code was not previously
determined to correspond to a “specified” textual location.
Rather, Spielberg's line number was previously determined to
correspond to an audio location (see the chart below). [The
chart below illustrates a flow chart of the method of claim 1 and
the teaching of Spielberg]

7 Appellant cites “Digital Biometrics, Inc. v. Identix, Inc., 149 F.3d 1335.
1345 (Fed. Cir. 1998). See also Process Control Corp. v. HydReclaim Corp.,
190 F.3d 1350, 1356 (Fed. Cir. 1999) (finding that the “identical” claim
language “clearly indicates that 'a discharge rate' in clause [b] is the same as
'the discharge rate' in clause [d]”).
Appeal 2018-002673
Application 13/267,749
11

[The chart above illustrates a flow chart of the method of
claim 1 and the teaching of Spielberg]

This is because, as established in the Appeal Brief,
Spielberg teaches that a user creates annotations while listening
to an audio file. Thus, Spielberg discloses first using an audio
file location to identify a line number associated with the
annotation.8 Accordingly, the line number associated with the
annotation in Spielberg is previously determined to correspond
to an audio location, not to a text location. It follows that
when Spielberg subsequently determines the annotation audio
location (e.g., the audio time code) based on the line number

8 Appellant cites Appeal Brief at page 15.
Appeal 2018-002673
Application 13/267,749
12
associated with the annotation,9 the annotation audio location
is determined based on a line number previously determined
to correspond to an audio location. This is exactly the opposite
of claim 1, which requires that the “particular audio location” is
“determin[ed]” “based on the particular text location”
previously determined to “correspond to the specified [textual]
location.”
Thus, as explained in Appellant's opening brief:
Spielberg fails to disclose or suggest “based on the particular
text location, determine a particular audio location.”
Accordingly, the rejection of the Group A claims should be
reversed.

Reply Br. 7–8.

We are persuaded by Appellant’s arguments because, on the record
before us, the Examiner presents insufficient evidence that Spielberg teaches
the disputed limitation.
During prosecution, claims must be given their broadest reasonable
interpretation when reading claim language in light of the Specification as it
would be interpreted by one of ordinary skill in the art. In re Am. Acad. of
Sci. Tech. Ctr., 367 F.3d 1359, 1364 (Fed. Cir. 2004). While we interpret
claims broadly but reasonably in light of the Specification, we nonetheless
must not import limitations from the Specification into the claims. See In re
Van Geuns, 988 F.2d 1181, 1184 (Fed. Cir. 1993). Our reviewing court
states that “the words of a claim ‘are generally given their ordinary and
customary meaning.’” Phillips v. AWH Corp., 415 F.3d 1303, 1312 (Fed.
Cir. 2005) (en banc) (citations omitted). However, the broadest reasonable
interpretation differs from the broadest possible interpretation. In re Smith

9 Appellant cites Spielberg at ¶ 138.
Appeal 2018-002673
Application 13/267,749
13
Int’l, Inc., 871 F.3d 1375, 1383 (Fed. Cir. 2017). The correct inquiry in
giving a claim term its broadest reasonable interpretation in light of the
specification is “an interpretation that corresponds with what and how the
inventor describes his invention in the specification, i.e., an interpretation
that is ‘consistent with the specification.’” Id. at 1382–83 (quoting In re
Morris, 127 F.3d 1048, 1054 (Fed. Cir. 1997)).
Here, the Examiner’s claim interpretation of the disputed limitation is
unreasonably broad, and fails to address the antecedent basis for the
“specified location.” Instead, we agree with the Appellant’s claim
interpretation that requires “a particular text location that corresponds to the
specified location” (Determination 1) to be the “specified location” recited
in “storing an association between the annotation and a specified location
within a textural version of a work.”
Although Spielberg teaches mapping audio and text, on the record
before us, the Examiner has not sufficiently shown that Spielberg teaches the
disputed limitation. We agree that the Examiner has not shown that
Spielberg teaches the claimed relationship between the specified location in
“storing an association between the annotation and a specified location
within a textural version of a work” and “a particular text location that
corresponds to the specified location” (Determination 1). In particular,
Spielberg’s annotation is determined to correspond to a line number
associated with an audio location, not to a text location, and the annotation
audio location is determined based on a line number previously determined
to correspond to an audio location. See Spielberg ¶ 138; Reply Br. 7–8.
In view of the above, we do not sustain the rejection of claim 1, and
dependent claims 2–18, and 21–28. Cf. In re Fritch, 972 F.2d 1260, 1266
Appeal 2018-002673
Application 13/267,749
14
(Fed. Cir. 1992) (“[D]ependent claims are nonobvious if the independent
claims from which they depend are nonobvious . . . .”).
Because our decision with regard to the disputed limitation is
dispositive of the rejections, we do not address additional arguments raised
by Appellant.

DECISION

Claims
Rejected
35 U.S.C. § Reference(s)/Basis Affirmed Reversed
1–8, 21–28 103(a) Starmen,
Spielberg, Murphy
1–8, 21–28
Overall Outcome 1–8, 21–28

REVERSED