Previous Next

Searching an Index

Building Queries

There are two ways to search the index. The first method uses query parser to construct a query from a string. The second is to programmatically create your own queries through the Zend_Search_Lucene API.

Before choosing to use the provided query parser, please consider the following:

  1. If you are programmatically creating a query string and then parsing it with the query parser then you should consider building your queries directly with the query API. Generally speaking, the query parser is designed for human-entered text, not for program-generated text.

  2. Untokenized fields are best added directly to queries and not through the query parser. If a field's values are generated programmatically by the application, then the query clauses for this field should also be constructed programmatically. An analyzer, which the query parser uses, is designed to convert human-entered text to terms. Program-generated values, like dates, keywords, etc., should be added with the query API.

  3. In a query form, fields that are general text should use the query parser. All others, such as date ranges, keywords, etc., are better added directly through the query API. A field with a limited set of values that can be specified with a pull-down menu should not be added to a query string that is subsequently parsed but instead should be added as a TermQuery clause.

  4. Boolean queries allow the programmer to logically combine two or more queries into new one. Thus it's the best way to add additional criteria to a search defined by a query string.

Both ways use the same API method to search through the index:

$index = Zend_Search_Lucene::open('/data/my_index');

$index->find($query);

The Zend_Search_Lucene::find() method determines the input type automatically and uses the query parser to construct an appropriate Zend_Search_Lucene_Search_Query object from an input of type string.

It is important to note that the query parser uses the standard analyzer to tokenize separate parts of query string. Thus all transformations which are applied to indexed text are also applied to query strings.

The standard analyzer may transform the query string to lower case for case-insensitivity, remove stop-words, and stem among other transformations.

The API method doesn't transform or filter input terms in any way. It's therefore more suitable for computer generated or untokenized fields.

Query Parsing

Zend_Search_Lucene_Search_QueryParser::parse() method may be used to parse query strings into query objects.

This query object may be used in query construction API methods to combine user entered queries with programmatically generated queries.

Actually, in some cases it's the only way to search for values within untokenized fields:

$userQuery = Zend_Search_Lucene_Search_QueryParser::parse($queryStr);

$pathTerm  = new Zend_Search_Lucene_Index_Term(
                     '/data/doc_dir/' . $filename, 'path'
                 );
$pathQuery = new Zend_Search_Lucene_Search_Query_Term($pathTerm);

$query = new Zend_Search_Lucene_Search_Query_Boolean();
$query->addSubquery($userQuery, true /* required */);
$query->addSubquery($pathQuery, true /* required */);

$hits = $index->find($query);

Zend_Search_Lucene_Search_QueryParser::parse() method also takes an optional encoding parameter, which can specify query string encoding:

$userQuery = Zend_Search_Lucene_Search_QueryParser::parse($queryStr,
                                                          'iso-8859-5');

If the encoding parameter is omitted, then current locale is used.

It's also possible to specify the default query string encoding with Zend_Search_Lucene_Search_QueryParser::setDefaultEncoding() method:

Zend_Search_Lucene_Search_QueryParser::setDefaultEncoding('iso-8859-5');
...
$userQuery = Zend_Search_Lucene_Search_QueryParser::parse($queryStr);

Zend_Search_Lucene_Search_QueryParser::getDefaultEncoding() returns the current default query string encoding (the empty string means "current locale").

Search Results

The search result is an array of Zend_Search_Lucene_Search_QueryHit objects. Each of these has two properties: $hit->document is a document number within the index and $hit->score is a score of the hit in a search result. The results are ordered by score (descending from highest score).

The Zend_Search_Lucene_Search_QueryHit object also exposes each field of the Zend_Search_Lucene_Document found in the search as a property of the hit. In the following example, a hit is returned with two fields from the corresponding document: title and author.

$index = Zend_Search_Lucene::open('/data/my_index');

$hits = $index->find($query);

foreach ($hits as $hit) {
    echo $hit->score;
    echo $hit->title;
    echo $hit->author;
}

Stored fields are always returned in UTF-8 encoding.

Optionally, the original Zend_Search_Lucene_Document object can be returned from the Zend_Search_Lucene_Search_QueryHit. You can retrieve stored parts of the document by using the getDocument() method of the index object and then get them by getFieldValue() method:

$index = Zend_Search_Lucene::open('/data/my_index');

$hits = $index->find($query);
foreach ($hits as $hit) {
    // return Zend_Search_Lucene_Document object for this hit
    echo $document = $hit->getDocument();

    // return a Zend_Search_Lucene_Field object
    // from the Zend_Search_Lucene_Document
    echo $document->getField('title');

    // return the string value of the Zend_Search_Lucene_Field object
    echo $document->getFieldValue('title');

    // same as getFieldValue()
    echo $document->title;
}

The fields available from the Zend_Search_Lucene_Document object are determined at the time of indexing. The document fields are either indexed, or index and stored, in the document by the indexing application (e.g. LuceneIndexCreation.jar).

Note that the document identity ('path' in our example) is also stored in the index and must be retrieved from it.

Limiting the Result Set

The most computationally expensive part of searching is score calculation. It may take several seconds for large result sets (tens of thousands of hits).

Zend_Search_Lucene gives the possibility to limit result set size with getResultSetLimit() and setResultSetLimit() methods:

$currentResultSetLimit = Zend_Search_Lucene::getResultSetLimit();

Zend_Search_Lucene::setResultSetLimit($newLimit);
The default value of 0 means 'no limit'.

It doesn't give the 'best N' results, but only the 'first N' [1] .

Results Scoring

Zend_Search_Lucene uses the same scoring algorithms as Java Lucene. All hits in the search result are ordered by score by default. Hits with greater score come first, and documents having higher scores should match the query more precisely than documents having lower scores.

Roughly speaking, search hits that contain the searched term or phrase more frequently will have a higher score.

A hit's score can be retrieved by accessing the score property of the hit:

$hits = $index->find($query);

foreach ($hits as $hit) {
    echo $hit->id;
    echo $hit->score;
}

The Zend_Search_Lucene_Search_Similarity class is used to calculate the score for each hit. See Extensibility. Scoring Algorithms section for details.

Search Result Sorting

By default, the search results are ordered by score. The programmer can change this behavior by setting a sort field (or a list of fields), sort type and sort order parameters.

$index->find() call may take several optional parameters:

$index->find($query [, $sortField [, $sortType [, $sortOrder]]]
                    [, $sortField2 [, $sortType [, $sortOrder]]]
             ...);

A name of stored field by which to sort result should be passed as the $sortField parameter.

$sortType may be omitted or take the following enumerated values: SORT_REGULAR (compare items normally- default value), SORT_NUMERIC (compare items numerically), SORT_STRING (compare items as strings).

$sortOrder may be omitted or take the following enumerated values: SORT_ASC (sort in ascending order- default value), SORT_DESC (sort in descending order).

Examples:

$index->find($query, 'quantity', SORT_NUMERIC, SORT_DESC);
$index->find($query, 'fname', SORT_STRING, 'lname', SORT_STRING);
$index->find($query, 'name', SORT_STRING, 'quantity', SORT_NUMERIC, SORT_DESC);

Please use caution when using a non-default search order; the query needs to retrieve documents completely from an index, which may dramatically reduce search performance.

Search Results Highlighting

Zend_Search_Lucene provides two options for search results highlighting.

The first one is utilizing Zend_Search_Lucene_Document_Html class (see HTML documents section for details) using the following methods:

/**
 * Highlight text with specified color
 *
 * @param string|array $words
 * @param string $colour
 * @return string
 */
public function highlight($words, $colour = '#66ffff');
/**
 * Highlight text using specified View helper or callback function.
 *
 * @param string|array $words  Words to highlight. Words could be organized
                               using the array or string.
 * @param callback $callback   Callback method, used to transform
                               (highlighting) text.
 * @param array    $params     Array of additionall callback parameters passed
                               through into it (first non-optional parameter
                               is an HTML fragment for highlighting)
 * @return string
 * @throws Zend_Search_Lucene_Exception
 */
public function highlightExtended($words, $callback, $params = array())

To customize highlighting behavior use highlightExtended() method with specified callback, which takes one or more parameters[2], or extend Zend_Search_Lucene_Document_Html class and redefine applyColour($stringToHighlight, $colour) method used as a default highlighting callback. [3] XHTML

View helpers also can be used as callbacks in context of view script:

$doc->highlightExtended('word1 word2 word3...', array($this, 'myViewHelper'));

The result of highlighting operation is retrieved by Zend_Search_Lucene_Document_Html->getHTML() method.

Note:

Highlighting is performed in terms of current analyzer. So all forms of the word(s) recognized by analyzer are highlighted.

E.g. if current analyzer is case insensitive and we request to highlight 'text' word, then 'text', 'Text', 'TEXT' and other case combinations will be highlighted.

In the same way, if current analyzer supports stemming and we request to highlight 'indexed', then 'index', 'indexing', 'indices' and other word forms will be highlighted.

On the other hand, if word is skipped by corrent analyzer (e.g. if short words filter is applied to the analyzer), then nothing will be highlighted.

The second option is to use Zend_Search_Lucene_Search_Query->highlightMatches(string $inputHTML[, $defaultEncoding = 'UTF-8'[, Zend_Search_Lucene_Search_Highlighter_Interface $highlighter]]) method:

$query = Zend_Search_Lucene_Search_QueryParser::parse($queryStr);
$highlightedHTML = $query->highlightMatches($sourceHTML);

Optional second parameter is a default HTML document encoding. It's used if encoding is not specified using Content-type HTTP-EQUIV meta tag.

Optional third parameter is a highlighter object which has to implement Zend_Search_Lucene_Search_Highlighter_Interface interface:

interface Zend_Search_Lucene_Search_Highlighter_Interface
{
    /**
     * Set document for highlighting.
     *
     * @param Zend_Search_Lucene_Document_Html $document
     */
    public function setDocument(Zend_Search_Lucene_Document_Html $document);

    /**
     * Get document for highlighting.
     *
     * @return Zend_Search_Lucene_Document_Html $document
     */
    public function getDocument();

    /**
     * Highlight specified words (method is invoked once per subquery)
     *
     * @param string|array $words  Words to highlight. They could be
                                   organized using the array or string.
     */
    public function highlight($words);
}
Where Zend_Search_Lucene_Document_Html object is an object constructed from the source HTML provided to the Zend_Search_Lucene_Search_Query->highlightMatches() method.

If $highlighter parameter is omitted, then Zend_Search_Lucene_Search_Highlighter_Default object is instantiated and used.

Highlighter highlight() method is invoked once per subquery, so it has an ability to differentiate highlighting for them.

Actually, default highlighter does this walking through predefined color table. So you can implement your own highlighter or just extend the default and redefine color table.

Zend_Search_Lucene_Search_Query->htmlFragmentHighlightMatches() has similar behavior. The only differenece is that it takes as an input and returns HTML fragment without <>HTML>, <HEAD>, <BODY> tags. Nevertheless, fragment is automatically transformed to valid XHTML.

[1] Returned hits are still ordered by score or by the specified order, if given.
[2]The first is an HTML fragment for highlighting and others are callback behavior dependent. Returned value is a highlighted HTML fragment.
[3] In both cases returned HTML is automatically transformed into valid .
Previous Next
Introduction to Zend Framework
Présentation
Installation
Zend_Acl
Introduction
Affiner les Contrôles d'Accès
Utilisation avancée
Zend_Amf
Introduction
Zend_Amf_Server
Zend_Application
Introduction
Zend_Application Quick Start
Théorie générale
Exemples
Fonctionnalités principales
Plugins de ressources disponibles
Zend_Auth
Introduction
Authentification avec une table de base de données
Authentification "Digest"
Adaptateur d'authentification HTTP
LDAP Authentication
Authentification OpenID
Zend_Cache
Introduction
Aspect théorique
Les frontends Zend_Cache
Les backends Zend_Cache
Zend_Captcha
Introduction
Opération Captcha
Adaptateurs CAPTCHA
Zend_CodeGenerator
Introduction
Exemples Zend_CodeGenerator
Zend_CodeGenerator Réference
Zend_Config
Introduction
Aspect théorique
Zend_Config_Ini
Zend_Config_Xml
Zend_Config_Writer
Zend_Config_Writer
Zend_Console_Getopt
Introduction
Déclarer les règles Getopt
Extraire les options et les arguments
Configurer Zend_Console_Getopt
Zend_Controller
Zend_Controller - Démarrage rapide
Fondations de Zend_Controller
Le contrôleur frontal (Front Controller)
L'objet Requête
Routeur Standard
Le distributeur
Contrôleurs d'action
Aides d'action (Helper)
Objet de réponse
Plugins
Utilisation de conventions de dossiers modulaires
Exceptions avec MVC
Migrer depuis des versions précédentes
Zend_Currency
Introduction à Zend_Currency
How to Work with Currencies
Migrer depuis des versions antérieures
Zend_Date
Introduction
Aspect théorique
Méthodes de base
Zend_Date API Overview
Créer des dates
Constants for General Date Functions
Exemples concrets
Zend_Db
Zend_Db_Adapter
Zend_Db_Statement
Zend_Db_Profiler
Zend_Db_Select
Zend_Db_Table
Zend_Db_Table_Row
Zend_Db_Table_Rowset
Relations Zend_Db_Table
Zend_Db_Table_Definition
Zend_Debug
Afficher des informations
Zend_Dojo
Introduction
Zend_Dojo_Data: dojo.data Envelopes
Les aides de vues Dojo
Les éléments de formulaire et les décorateurs Dojo
Zend_Dojo build layer support
Zend_Dom
Introduction
Zend_Dom_Query
Zend_Exception
Utiliser les exceptions
Zend_Feed
Introduction
Importer des flux
Obtenir des flux à partir de pages Web
Consommer un flux RSS
Consommer un flux Atom
Consommer une entrée Atom particulière
Modifier la structure du flux ou des entrées
Classes personnalisées pour les flux et entrées
Zend_Feed_Reader
Zend_File
Zend_File_Transfer
Validateurs pour Zend_File_Transfer
Filtres pour Zend_File_Transfer
Migrer à partir des versions précédentes
Zend_Filter
Introduction
Classes de filtre standards
Chaînes de filtrage
Écriture de filtres
Zend_Filter_Input
Zend_Filter_Inflector
Migrating from Previous Versions
Zend_Form
Zend_Form
Zend_Form démarrage rapide
Creating Form Elements Using Zend_Form_Element
Creating Forms Using Zend_Form
Créer un visuel personnalisé en utilisant Zend_Form_Decorator
Standard Form Elements Shipped With Zend Framework
Décorateurs standards fournis avec Zend Framework
Internationaliser un formulaire Zend_Form
Advanced Zend_Form Usage
Zend_Gdata
Introduction
Authentification par procédé AuthSub
Using the Book Search Data API
Authentification avec ClientLogin
Using Google Calendar
Using Google Documents List Data API
Using Google Health
Using Google Spreadsheets
Using Google Apps Provisioning
Using Google Base
Utilisation des albums Web Picasa
Using the YouTube Data API
Attraper les exceptions Gdata
Zend_Http
Introduction
Zend_Http_Client - Utilisation avancée
Zend_Http_Client - Adaptateurs de connexion
Migrer depuis les versions précédentes
Zend_Http_Cookie and Zend_Http_CookieJar
Zend_Http_Response
Zend_InfoCard
Introduction
Zend_Json
Introduction
Utilisation de base
Utilisation avancée de Zend_Json
XML to JSON conversion
Zend_Json_Server - JSON-RPC server
Zend_Layout
Introduction
Zend_Layout - Démarrage rapide
Zend_Layout options de configuration
Zend_Layout, utilisation avancée
Zend_Ldap
Introduction
API overview
Usage Scenarios
Tools
Object oriented access to the LDAP tree using Zend_Ldap_Node
Getting information from the LDAP server
Serializing LDAP data to and from LDIF
Zend_Loader
Charger les fichiers et les classes dynamiquement
L'autoloader
Autoloaders de ressources
Chargeur de Plugins
Zend_Locale
Introduction
Using Zend_Locale
Normalization and Localization
Working with Dates and Times
Supported locales
Migrer à partir des versions précédentes
Zend_Log
Présentation
Rédacteurs (Writers)
Formateurs (mise en forme)
Filtres
Zend_Mail
Introduction
Envoyer des émail en utilisant SMTP
Envoyer plusieurs émail par connexion SMTP
Utiliser différents transports
Émail HTML
Fichiers joints
Ajouter des destinataires
Contrôler les limites MIME
En-têtes additionnels
Jeux de caractères
Encodage
Authentification SMTP
Sécuriser les transports SMTP
Lire des émail
Zend_Measure
Introduction
Création d'une mesure
Récupérer des mesures
Manipuler des mesures
Types de mesures
Zend_Memory
Présentation
Manager de mémoire
Objet mémoire
Zend_Mime
Zend_Mime
Zend_Mime_Message
Zend_Mime_Part
Zend_Navigation
Introduction
Pages
Containers
Migrating from Previous Versions
Zend_OpenId
Introduction
Zend_OpenId_Consumer Basics
Zend_OpenId_Provider
Zend_Paginator
Introduction
Utilisation
Configuration
Utilisation avancée
Zend_Pdf
Introduction
Créer et charger des documents PDF
Sauvegarder les changement dans un document PDF
Les pages d'un document
Dessiner
Interactive Features
Informations du document et métadonnées
Exemple d'utilisation du module Zend_Pdf
Zend_ProgressBar
Zend_ProgressBar
Zend_Queue
Introduction
Example usage
Framework
Adapters
Customizing Zend_Queue
Stomp
Zend_Reflection
Introduction
Zend_Reflection Exemples
Réference de Zend_Reflection
Zend_Registry
Utiliser le registre
Zend_Rest
Introduction
Zend_Rest_Client
Zend_Rest_Server
Zend_Search_Lucene
Vue d'ensemble
Créer des index
Searching an Index
Query Language
Query Construction API
Jeu de caractères
Extensibility
Agir avec Lucene Java
Avancé
Bonnes pratiques
Zend_Server
Introduction
Zend_Server_Reflection
Zend_Service
Introduction
Zend_Service_Akismet
Zend_Service_Amazon
Zend_Service_Amazon_Ec2
Zend_Service_Amazon_Ec2: Instances
Zend_Service_Amazon_Ec2: Windows Instances
Zend_Service_Amazon_Ec2: Reserved Instances
Zend_Service_Amazon_Ec2: CloudWatch Monitoring
Zend_Service_Amazon_Ec2: Amazon Machine Images (AMI)
Zend_Service_Amazon_Ec2: Elastic Block Stroage (EBS)
Zend_Service_Amazon_Ec2: Elastic IP Addresses
Zend_Service_Amazon_Ec2: Keypairs
Zend_Service_Amazon_Ec2: Regions and Availability Zones
Zend_Service_Amazon_Ec2: Security Groups
Zend_Service_Amazon_S3
Zend_Service_Amazon_Sqs
Zend_Service_Audioscrobbler
Zend_Service_Delicious
Zend_Service_Flickr
Zend_Service_Nirvanix
Zend_Service_ReCaptcha
Zend_Service_Simpy
Introduction
Zend_Service_StrikeIron
Zend_Service_StrikeIron: Bundled Services
Zend_Service_StrikeIron: Advanced Uses
Zend_Service_Technorati
Zend_Service_Twitter
Zend_Service_Yahoo
Zend_Session
Introduction
Usage basique
Utilisation avancée
Gestion générale de la session
Zend_Session_SaveHandler_DbTable
Zend_Soap
Zend_Soap_Server
Zend_Soap_Client
WSDL
Auto découverte
Zend_Tag
Introduction
Zend_Tag_Cloud
Zend_Test
Introduction
Zend_Test_PHPUnit
Zend_Test_PHPUnit_Db
Zend_Text
Zend_Text_Figlet
Zend_Text_Table
Zend_TimeSync
Introduction
Utiliser Zend_TimeSync
Zend_Tool_Framework
Introduction
Using the CLI Tool
Architecture
Creating Providers to use with Zend_Tool_Framework
Shipped System Providers
Extending and Configuring Zend_Tool_Framework
Zend_Tool_Project
Introduction
Create A Project
Zend Tool Project Providers
Zend_Translate
Introduction
Adaptateurs pour Zend_Translate
Utiliser les adaptateurs de traduction
Creating source files
Additional features for translation
Plural notations for Translation
Migrer à partir des versions précédentes
Zend_Uri
Zend_Uri
Zend_Validate
Introduction
Classes de validation standard
Chaînes de validation
Écrire des validateurs
Validation Messages
Zend_Version
Lire la version de Zend Framework
Zend_View
Introduction
Scripts de contrôleur
Scripts de vue
Aides de vue
Zend_View_Abstract
Migration depuis les versions précédentes
Zend_Wildfire
Zend_Wildfire
Zend_XmlRpc
Introduction
Zend_XmlRpc_Client
Zend_XmlRpc_Server
Configuration système requise par Zend Framework
Introduction
Convention de codage PHP de Zend Framework
Vue d'ensemble
Formatage des fichiers PHP
Conventions de nommage
Style de codage
Zend Framework Documentation Standard
Overview
Documentation File Formatting
Recommendations
Recommended Project Structure for Zend Framework MVC Applications
Overview
Recommended Project Directory Structure
Module Structure
Rewrite Configuration Guide
Zend Framework Performance Guide
Introduction
Chargement des classes
Zend_Db Performance
Internationalisation (i18n) and Localisation (l10n)
View Rendering
Informations de copyright