Previous Next

Searching an Index

Building Queries

There are two ways to search the index. The first method uses query parser to construct a query from a string. The second is to programmatically create your own queries through the Zend_Search_Lucene API.

Before choosing to use the provided query parser, please consider the following:

  1. If you are programmatically creating a query string and then parsing it with the query parser then you should consider building your queries directly with the query API. Generally speaking, the query parser is designed for human-entered text, not for program-generated text.

  2. Untokenized fields are best added directly to queries and not through the query parser. If a field's values are generated programmatically by the application, then the query clauses for this field should also be constructed programmatically. An analyzer, which the query parser uses, is designed to convert human-entered text to terms. Program-generated values, like dates, keywords, etc., should be added with the query API.

  3. In a query form, fields that are general text should use the query parser. All others, such as date ranges, keywords, etc., are better added directly through the query API. A field with a limited set of values that can be specified with a pull-down menu should not be added to a query string that is subsequently parsed but instead should be added as a TermQuery clause.

  4. Boolean queries allow the programmer to logically combine two or more queries into new one. Thus it's the best way to add additional criteria to a search defined by a query string.

Both ways use the same API method to search through the index:

find($query);

The Zend_Search_Lucene::find() method determines the input type automatically and uses the query parser to construct an appropriate Zend_Search_Lucene_Search_Query object from an input of type string.

It is important to note that the query parser uses the standard analyzer to tokenize separate parts of query string. Thus all transformations which are applied to indexed text are also applied to query strings.

The standard analyzer may transform the query string to lower case for case-insensitivity, remove stop-words, and stem among other transformations.

The API method doesn't transform or filter input terms in any way. It's therefore more suitable for computer generated or untokenized fields.

Query Parsing

Zend_Search_Lucene_Search_QueryParser::parse() method may be used to parse query strings into query objects.

This query object may be used in query construction API methods to combine user entered queries with programmatically generated queries.

Actually, in some cases it's the only way to search for values within untokenized fields:

addSubquery($userQuery, true /* required */);
$query->addSubquery($pathQuery, true /* required */);

$hits = $index->find($query);

Zend_Search_Lucene_Search_QueryParser::parse() method also takes an optional encoding parameter, which can specify query string encoding:



            

If the encoding parameter is omitted, then current locale is used.

It's also possible to specify the default query string encoding with Zend_Search_Lucene_Search_QueryParser::setDefaultEncoding() method:



            

Zend_Search_Lucene_Search_QueryParser::getDefaultEncoding() returns the current default query string encoding (the empty string means "current locale").

Search Results

The search result is an array of Zend_Search_Lucene_Search_QueryHit objects. Each of these has two properties: $hit->document is a document number within the index and $hit->score is a score of the hit in a search result. The results are ordered by score (descending from highest score).

The Zend_Search_Lucene_Search_QueryHit object also exposes each field of the Zend_Search_Lucene_Document found in the search as a property of the hit. In the following example, a hit is returned with two fields from the corresponding document: title and author.

find($query);

foreach ($hits as $hit) {
    echo $hit->score;
    echo $hit->title;
    echo $hit->author;
}

Stored fields are always returned in UTF-8 encoding.

Optionally, the original Zend_Search_Lucene_Document object can be returned from the Zend_Search_Lucene_Search_QueryHit. You can retrieve stored parts of the document by using the getDocument() method of the index object and then get them by getFieldValue() method:

find($query);
foreach ($hits as $hit) {
    // return Zend_Search_Lucene_Document object for this hit
    echo $document = $hit->getDocument();

    // return a Zend_Search_Lucene_Field object
    // from the Zend_Search_Lucene_Document
    echo $document->getField('title');

    // return the string value of the Zend_Search_Lucene_Field object
    echo $document->getFieldValue('title');

    // same as getFieldValue()
    echo $document->title;
}

The fields available from the Zend_Search_Lucene_Document object are determined at the time of indexing. The document fields are either indexed, or index and stored, in the document by the indexing application (e.g. LuceneIndexCreation.jar).

Note that the document identity ('path' in our example) is also stored in the index and must be retrieved from it.

Limiting the Result Set

The most computationally expensive part of searching is score calculation. It may take several seconds for large result sets (tens of thousands of hits).

Zend_Search_Lucene gives the possibility to limit result set size with getResultSetLimit() and setResultSetLimit() methods:



            The default value of 0 means 'no limit'.
        

It doesn't give the 'best N' results, but only the 'first N'[1].

Results Scoring

Zend_Search_Lucene uses the same scoring algorithms as Java Lucene. All hits in the search result are ordered by score by default. Hits with greater score come first, and documents having higher scores should match the query more precisely than documents having lower scores.

Roughly speaking, search hits that contain the searched term or phrase more frequently will have a higher score.

A hit's score can be retrieved by accessing the score property of the hit:

find($query);

foreach ($hits as $hit) {
    echo $hit->id;
    echo $hit->score;
}

The Zend_Search_Lucene_Search_Similarity class is used to calculate the score for each hit. See Extensibility. Scoring Algorithms section for details.

Search Result Sorting

By default, the search results are ordered by score. The programmer can change this behavior by setting a sort field (or a list of fields), sort type and sort order parameters.

$index->find() call may take several optional parameters:

find($query [, $sortField [, $sortType [, $sortOrder]]] [, $sortField2 [, $sortType [, $sortOrder]]] ...);

A name of stored field by which to sort result should be passed as the $sortField parameter.

$sortType may be omitted or take the following enumerated values: SORT_REGULAR (compare items normally- default value), SORT_NUMERIC (compare items numerically), SORT_STRING (compare items as strings).

$sortOrder may be omitted or take the following enumerated values: SORT_ASC (sort in ascending order- default value), SORT_DESC (sort in descending order).

Examples:

find($query, 'quantity', SORT_NUMERIC, SORT_DESC);
find($query, 'fname', SORT_STRING, 'lname', SORT_STRING);
find($query, 'name', SORT_STRING, 'quantity', SORT_NUMERIC, SORT_DESC);

Please use caution when using a non-default search order; the query needs to retrieve documents completely from an index, which may dramatically reduce search performance.

Search Results Highlighting

Zend_Search_Lucene_Search_Query::highlightMatches() method allows the developer to highlight HTML document terms in the context of a search query:

find($query);
...
$highlightedHTML = $query->highlightMatches($sourceHTML);

highlightMatches() method utilizes Zend_Search_Lucene_Document_Html class (see HTML documents section for details) for HTML processing, so it has the same requirements for HTML source.

[1]Returned hits are still ordered by score or by the the specified order, if given.
Previous Next
Introduction to Zend Framework
Overview
Installation
Zend_Acl
Introduction
Refining Access Controls
Advanced Use
Zend_Auth
Introduction
Database Table Authentication
Digest Authentication
HTTP Authentication Adapter
LDAP Authentication
Open ID Authentication
Zend_Cache
Introduction
The theory of caching
Zend_Cache frontends
Zend_Cache backends
Zend_Captcha
Introduction
Captcha Operation
Captcha Adapters
Zend_Config
Introduction
Theory of Operation
Zend_Config_Ini
Zend_Config_Xml
Zend_Console_Getopt
Introduction to Getopt
Declaring Getopt Rules
Fetching Options and Arguments
Configuring Zend_Console_Getopt
Zend_Controller
Zend_Controller Quick Start
Zend_Controller Basics
The Front Controller
The Request Object
The Standard Router: Zend_Controller_Router_Rewrite
The Dispatcher
Action Controllers
Action Helpers
The Response Object
Plugins
Using a Conventional Modular Directory Structure
MVC Exceptions
Migrating from Previous Versions
Zend_Currency
Introduction to Zend_Currency
How to work with currencies
Migrating from Previous Versions
Zend_Date
Introduction
Theory of Operation
Basic Methods
Zend_Date API Overview
Creation of dates
Constants for General Date Functions
Working examples
Zend_Db
Zend_Db_Adapter
Zend_Db_Statement
Zend_Db_Profiler
Zend_Db_Select
Zend_Db_Table
Zend_Db_Table_Row
Zend_Db_Table_Rowset
Zend_Db_Table Relationships
Zend_Debug
Dumping Variables
Zend_Dojo
Introduction
Zend_Dojo_Data: dojo.data Envelopes
Dojo View Helpers
Dojo Form Elements and Decorators
Zend_Dom
Introduction
Zend_Dom_Query
Zend_Exception
Using Exceptions
Zend_Feed
Introduction
Importing Feeds
Retrieving Feeds from Web Pages
Consuming an RSS Feed
Consuming an Atom Feed
Consuming a Single Atom Entry
Modifying Feed and Entry structures
Custom Feed and Entry Classes
Zend_File
Zend_File_Transfer
Validators for Zend_File_Transfer
Zend_Filter
Introduction
Standard Filter Classes
Filter Chains
Writing Filters
Zend_Filter_Input
Zend_Filter_Inflector
Zend_Form
Zend_Form
Zend_Form Quick Start
Creating Form Elements Using Zend_Form_Element
Creating Forms Using Zend_Form
Creating Custom Form Markup Using Zend_Form_Decorator
Standard Form Elements Shipped With Zend Framework
Standard Form Decorators Shipped With Zend Framework
Internationalization of Zend_Form
Advanced Zend_Form Usage
Zend_Gdata
Introduction to Gdata
Authenticating with AuthSub
Authenticating with ClientLogin
Using Google Calendar
Using Google Documents List Data API
Using Google Spreadsheets
Using Google Apps Provisioning
Using Google Base
Using the YouTube Data API
Using Picasa Web Albums
Catching Gdata Exceptions
Zend_Http
Zend_Http_Client - Introduction
Zend_Http_Client - Advanced Usage
Zend_Http_Client - Connection Adapters
Zend_Http_Cookie and Zend_Http_CookieJar
Zend_Http_Response
Zend_InfoCard
Introduction
Zend_Json
Introduction
Basic Usage
JSON Objects
XML to JSON conversion
Zend_Json_Server - JSON-RPC server
Zend_Layout
Introduction
Zend_Layout Quick Start
Zend_Layout Configuration Options
Zend_Layout Advanced Usage
Zend_Ldap
Introduction
Zend_Loader
Loading Files and Classes Dynamically
Loading Plugins
Zend_Locale
Introduction
Using Zend_Locale
Normalization and Localization
Working with Dates and Times
Supported Languages for Locales
Supported Regions for Locales
Zend_Log
Overview
Writers
Formatters
Filters
Zend_Mail
Introduction
Sending via SMTP
Sending Multiple Mails per SMTP Connection
Using Different Transports
HTML E-Mail
Attachments
Adding Recipients
Controlling the MIME Boundary
Additional Headers
Character Sets
Encoding
SMTP Authentication
Securing SMTP Transport
Reading Mail Messages
Zend_Measure
Introduction
Creation of Measurements
Outputting measurements
Manipulating Measurements
Types of measurements
Zend_Memory
Overview
Memory Manager
Memory Objects
Zend_Mime
Zend_Mime
Zend_Mime_Message
Zend_Mime_Part
Zend_OpenId
Introduction
Zend_OpenId_Consumer Basics
Zend_OpenId_Provider
Zend_Paginator
Introduction
Usage
Configuration
Advanced usage
Zend_Pdf
Introduction.
Creating and loading PDF documents.
Save changes to the PDF document.
Document pages.
Drawing.
Document Info and Metadata.
Zend_Pdf module usage example.
Zend_Registry
Using the Registry
Zend_Rest
Introduction
Zend_Rest_Client
Zend_Rest_Server
Zend_Search_Lucene
Overview
Building Indexes
Searching an Index
Query Language
Query Construction API
Character Set
Extensibility
Interoperating with Java Lucene
Advanced
Best Practices
Zend_Server
Introduction
Zend_Server_Reflection
Zend_Service
Introduction
Zend_Service_Akismet
Zend_Service_Amazon
Zend_Service_Audioscrobbler
Zend_Service_Delicious
Zend_Service_Flickr
Zend_Service_Nirvanix
Zend_Service_ReCaptcha
Zend_Service_Simpy
Introduction
Zend_Service_StrikeIron
Zend_Service_StrikeIron: Bundled Services
Zend_Service_StrikeIron: Advanced Uses
Zend_Service_Technorati
Zend_Service_Yahoo
Zend_Session
Introduction
Basic Usage
Advanced Usage
Global Session Management
Zend_Session_SaveHandler_DbTable
Zend_Soap
Zend_Soap_Server
Zend_Soap_Client
WSDL Accessor
AutoDiscovery. Introduction
Class autodiscovering.
Functions autodiscovering.
Autodiscovering. Datatypes.
Zend_Test
Introduction
Zend_Test_PHPUnit
Zend_Text
Zend_Text_Figlet
Zend_TimeSync
Introduction
Working with Zend_TimeSync
Zend_Translate
Introduction
Adapters for Zend_Translate
Using Translation Adapters
Zend_Uri
Zend_Uri
Zend_Validate
Introduction
Standard Validation Classes
Validator Chains
Writing Validators
Zend_Version
Reading the Zend Framework Version
Zend_View
Introduction
Controller Scripts
View Scripts
View Helpers
Zend_View_Abstract
Zend_Wildfire
Zend_Wildfire
Zend_XmlRpc
Introduction
Zend_XmlRpc_Client
Zend_XmlRpc_Server
Zend Framework Requirements
PHP Version
PHP Extensions
Zend Framework Components
Zend Framework Dependencies
Zend Framework Coding Standard for PHP
Overview
PHP File Formatting
Naming Conventions
Coding Style
Copyright Information