A new Apple patent (number 201110154200) for "enhancing media content with content-aware resources" has appeared at the US Patent & Trademark Office. It shows what Apple is considering for future versions of the Apple TV (or perhaps an Apple-branded television set, if you buy the latest rumors).

Here's Apple's background and summary of the invention: "In the realm of computer software operating systems and application programs, light-weight, single-purpose applications referred to as 'widgets' or 'gadgets' have gained some prominence as useful resources with which users can interact to obtain information (e.g., weather, stock ticker values), perform a particular function (e.g., desktop calculator, web search interface) or interact with others (e.g., send messages back and forth among friends on a social networking website).

"Apple Inc., for example, provides an environment known as 'Dashboard' that enables users to choose from among a wide assortment of widgets, which can be installed and execute locally on a user's computer. Generally speaking, the basic components of a widget include a graphical user interface (GUI) for communicating with a user and a single-purpose functionality that responds to user input and which represents an available resource. The types and functionality of such widgets are limited largely only by the widget developer's creativity.

"Recently, a few consumer electronics companies have extended the widget paradigm to television (TV). For example, while watching TV programming on a widget-enabled TV set, the viewer can manipulate the TV remote control to interact, for example, with a 'chat' widget displayed on the TV screen to send text messages back and forth with others connected to a common chat network.

"The present inventors recognized a limitation of existing widget technology as applied to TV environment in that conventional widgets, while often useful resources standing alone, nevertheless are unaware of the media content that the TV set was currently presenting. For example, such conventional TV widgets are unaware of what particular television program the user is presently watching on the TV.

"Accordingly, the present inventors envisioned and developed an enhanced TV widget paradigm in which widgets are capable of being content-aware and thus capable, among other things, of automatically (i.e., without intervening user input) providing the user with access to information or other resources that are complementary or otherwise relevant to the media content currently being presented by the TV set to the user.

"In general, in one aspect, the subject matter can be implemented to include methods, systems, and apparatus for making enhanced media content available to a viewer of a media device in which data packets are received via a packet-switched network, the received data packets including (i) media content for presentation to a user, (ii) location data specifying a resource that is complementary to the media content, and (iii) state data relating to a state of the complementary resource (e.g., corresponding to one or more of the following states: visibility/invisibility, activate/deactivate, change functionality, change appearance, and change position); based at least in part on the received state data, a determination is made whether the state of the complementary resource is to be changed; and based on a result of the determination, operations are selectively performed including using the received location data to communicate with, and retrieve complementary content from, the complementary resource; and presenting the complementary content to the user in synchronization with the media content.

"In general, in an aspect, methods, systems, and computer program products for making enhanced media content available to a viewer of a media device may include receiving data packets via a packet-switched network, the received data packets including (i) media content for presentation to a user, (ii) location data specifying a resource that is complementary to the media content, and (iii) state data relating to a state of the complementary resource; determining, based at least in part on the received state data, whether the state of the complementary resource is to be changed; and based on a result of the determination, selectively performing operations including using the received location data to communicate with, and retrieve complementary content from, the complementary resource; and presenting the complementary content to the user in synchronization with the media content, optionally also formatting the received information based on an output device with which the user is accessing the media content.

"In addition, input may be received from the user relating to a requested interaction with the complementary resource, in which case the received input may be delivered to the complementary resource. Information may then be received from the complementary resource responsive to the received user input, and presented to the received information to the user.

"Further user input specifying a second resource of the user's choosing may be received and used to retrieve second content from the second resource based on location information corresponding to the second resource. The retrieved second content may be formatted relative to the media content and relative to the complementary content, and the formatted second content, the complementary resource and media content may be presented to the user.

"The data packets received may further include one or more markers identifying one or more events that trigger communication with the complementary resource or the user or both. Such markers may be presented to the user and the user may be enabled to interact with the tags to alter one or more of timing, behavior and complementary content.

"The user may be presented with one or more user interface mechanisms to enable the user to modify behavior of a complementary resource, to access an online repository of complementary resources available for download, and/or to enable the user to generate complementary resources.

"In another aspect, an enhanced media content development system includes a computer system having a processor, memory, and input and output devices. An application configured to execute on the computer system may enable a user of the computer system to build an item of enhanced media content by specifying complementary resources that will be presented to an audience member along with an item of primary media content.

"The application may include a user interface configured to provide a user of the enhanced media content development system with mechanisms to synchronize one or more complementary resources with corresponding portions of the item of primary media content. The application may be configured to generate an enhanced media file that includes the primary media content and metadata specifying locations at which the one or more complementary resources are to be accessed by a media presentation device when the corresponding portions of the primary media content item are being presented to the audience member.

"The user interface may further be configured to provide the user of the enhanced content development system with one or more mechanisms to synchronize one or more events with corresponding portions of the item of primary media content.

"The user interface may include a film strip region that provides the user with access to the primary media content item, a complementary resource region that provides the user with access to one or more complementary resources available for synchronization with the primary media content item, an event region that provides the user with access to one or more events available for synchronization with the primary media content item, and a timeline region that enables the user to synchronize one or more of the complementary resources with corresponding portions of the item of primary media content. The timeline region may include a plurality of individual timelines each of which corresponds to a different presentation platform for which the enhanced media file is optimized.

"The subject matter described in this specification can be implemented to realize one or more of the following potential advantages. For example, the subject matter can be implemented to create an enhanced and richer TV viewing experience in which complementary resources (e.g., background information, webpages, supplemental media content, executable applications, utilities and the like) that are guaranteed to be relevant to the media content being presented can be caused to automatically appear on the user's TV screen at an appropriate time and/or in synchronization with presentation of the media content.

"Similarly, these same resources can be caused to automatically disappear when they are no longer relevant or useful based on the currently presented portion of the media content, thereby minimizing confusion and screen clutter. As a result, the user will tend to have a more enjoyable and fulfilling viewing experience and will be spared the trouble of having to manually locate and access resources that may or may not be relevant to the content presently being presented.

"The details of one or more implementations are set forth in the accompanying drawings and the description below. Other features and potential advantages will be apparent from the description and drawings, and from the claims."

The inventors are Daniel Davis, Garrett Groszko and Alan Cannistraro.

-- Dennis Sellers