[0039] As shown in FIG. 2, the predicted quality values 160 resulting from multiple ad selections may be aggregated and processed 200 (as described below with respect to FIGS. 14-20) to further provide ad quality parameters 210 that indicate that each respective advertisement is a good advertisement or a bad advertisement. The ad quality parameters 210 may be used to filter, rank and/or promote selected ones of the ads associated with the ad quality parameters 210. A set of ads 220 may first be determined that are relevant to a search query 224 issued by a user. An advertisement may be determined to be relevant to search query 224, for example, based on a comparison of the content of the advertisement with the terms of search query 224. The set of ads 220 may then be filtered 230 based on the ad quality parameter(s) 210, alone or in combination with CTR, associated with each of the ads. Filtering the set of ads 220 may disable selected ones of the advertisements such that they are not provided to the user that issued search query 224. Various techniques, that use the estimated advertisement qualities, may be used for filtering selected ads of the set of relevant ads 210 as further described below.
[0040] The set of relevant ads 220 may then be ranked 240 based on the ad quality parameter(s) 210. The set of relevant ads 220 may be ranked 240 subsequent (or prior) to filtering 230, or without filtering 230 being performed. Ranking 240 the set of relevant ads 220 determines a selected order for providing each of the relevant ads 220 to the user that issued the search query 224.
[0041] One or more ads of the set of relevant ads 220 may further be promoted 250 based on the ad quality parameter(s) 210. Promotion of an advertisment may include positioning the promoted ad at a different location on a document relative to unpromoted ads. For example, promoted ads may be placed in a highlighted (more prominent) position on a document (e.g., at a top of a document or a bottom of the document), whereas unpromoted ads may be placed in a different, non-highlighted or less prominent position on the document. The set of relevant ads 220 may be promoted 250 subsequent to filtering 230 and ranking 240, subsequent only to ranking 240, or without filtering 230 or ranking 240 being performed.
[0042] The filtered, ranked and/or promoted ads may be provided to the user that issued the search query 224. In some implementations, the filtered, ranked and/or promoted ads may be included in one or more documents provided to the user that include search results resulting from the execution of the user’s issued search query 224 by a search engine.
EXEMPLARY NETWORK CONFIGURATION
[0043] FIG. 3 is an exemplary diagram of a network 300 in which systems and methods consistent with the principles of the invention may be implemented. Network 300 may include multiple clients 310 connected to one or more servers 320-330 via a network 340. Two clients 310 and two servers 320-330 have been illustrated as connected to network 340 for simplicity. In practice, there may be more or fewer clients and servers. Also, in some instances, a client may perform a function of a server and a server may perform a function of a client.
[0044] Clients 310 may include client entities. An entity may be defined as a device, such as a personal computer, a wireless telephone, a personal digital assistant (PDA), a lap top, or another type of computation or communication device, a thread or process running on one of these devices, and/or an object executable by one of these devices. One or more users may be associated with each client 310. Servers 320 and 330 may include server entities that access, fetch, aggregate, process, search, and/or maintain documents in a manner consistent with the principles of the invention. Clients 310 and servers 320 and 330 may connect to network 340 via wired, wireless, and/or optical connections.
[0045] In an implementation consistent with the principles of the invention, server 320 may include a search engine system 325 usable by users at clients 310. Server 320 may implement a data aggregation service by crawling a corpus of documents (e.g., web documents), indexing the documents, and storing information associated with the documents in a repository of documents. The data aggregation service may be implemented in other ways, such as by agreement with the operator(s) of data server(s) 330 to distribute their hosted documents via the data aggregation service. In some implementations, server 320 may host advertisements (e.g., creatives, ad landing documents) that can be provided to users at clients 310. Search engine system 325 may execute a query, received from a user at a client 310, on the corpus of documents stored in the repository of documents, and may provide a set of search results to the user that are relevant to the executed query. In addition to the set of search results, server 320 may provide one or more advertising creatives, associated with results of the executed search, to the user at client 310.
[0046] Server(s) 330 may store or maintain documents that may be crawled by server 320. Such documents may include data related to published news stories, products, images, user groups, geographic areas, or any other type of data. For example, server(s) 330 may store or maintain news stories from any type of news source, such as, for example, the Washington Post, the New York Times, Time magazine, or Newsweek. As another example, server(s) 330 may store or maintain data related to specific products, such as product data provided by one or more product manufacturers. As yet another example, server(s) 330 may store or maintain data related to other types of web documents, such as pages of web sites. Server(s) 330 may further host advertisements, such as ad creatives and ad landing documents.
[0047] Network 340 may include one or more networks of any type, including a local area network (LAN), a wide area network (WAN), a metropolitan area network (MAN), a telephone network, such as the Public Switched Telephone Network (PSTN) or a Public Land Mobile Network (PLMN), an intranet, the Internet, a memory device, or a combination of networks. The PLMN(s) may further include a packet-switched sub-network, such as, for example, General Packet Radio Service (GPRS), Cellular Digital Packet Data (CDPD), or Mobile IP sub-network.
[0048] While servers 320-330 are shown as separate entities, it may be possible for one of servers 320-330 to perform one or more of the functions of the other one of servers 320-330. For example, it may be possible that servers 320 and 330 are implemented as a single server. It may also be possible for a single one of servers 320 and 330 to be implemented as two or more separate (and possibly distributed) devices.
EXEMPLARY CLIENT/SERVER ARCHITECTURE
[0049] FIG. 4 is an exemplary diagram of a client or server entity (hereinafter called “client/server entity”), which may correspond to one or more of clients 310 and/or servers 320-330, according to an implementation consistent with the principles of the invention. The client/server entity may include a bus 410, a processor 420, a main memory 430, a read only memory (ROM) 440, a storage device 450, an input device 460, an output device 470, and a communication interface 480. Bus 410 may include a path that permits communication among the elements of the client/server entity.
[0050] Processor 420 may include a processor, microprocessor, or processing logic that may interpret and execute instructions. Main memory 430 may include a random access memory (RAM) or another type of dynamic storage device that may store information and instructions for execution by processor 420. ROM 440 may include a ROM device or another type of static storage device that may store static information and instructions for use by processor 420. Storage device 450 may include a magnetic and/or optical recording medium and its corresponding drive.
[0051] Input device 460 may include a mechanism that permits an operator to input information to the client/server entity, such as a keyboard, a mouse, a pen, voice recognition and/or biometric mechanisms, etc. Output device 470 may include a mechanism that outputs information to the operator, including a display, a printer, a speaker, etc. Communication interface 480 may include any transceiver-like mechanism that enables the client/server entity to communicate with other devices and/or systems. For example, communication interface 480 may include mechanisms for communicating with another device or system via a network, such as network 340.
[0052] The client/server entity, consistent with the principles of the invention, may perform certain operations or processes, as will be described in detail below. The client/server entity may perform these operations in response to processor 420 executing software instructions contained in a computer-readable medium, such as memory 430. A computer-readable medium may be defined as a physical or logical memory device and/of carrier wave.
[0053] The software instructions may be read into memory 430 from another computer-readable medium, such as data storage device 450, or from another device via communication interface 480. The software instructions contained in memory 430 may cause processor 420 to perform operations or processes that will be described later. Alternatively, hardwired circuitry may be used in place of or in combination with software instructions to implement processes consistent with the principles of the invention. Thus, implementations consistent with the principles of the invention are not limited to any specific combination of hardware circuitry and software.
EXEMPLARY PROCESS FOR CONSTRUCTING A STATISTICAL MODEL OF USER BEHAVIOR ASSOCIATED WITH AD SELECTIONS
[0054] FIG. 5 is a flowchart of an exemplary process for constructing a statistical model of user behavior associated with the selections of multiple on-line advertisements. As one skilled in the art will appreciate, the process exemplified by FIG. 5 can be implemented in software and stored on a computer-readable memory, such as main memory 430, ROM 440, or storage device 450 of server 320, server 330 or a client 310, as appropriate.