Using Solr in Django for Full-Text Searching via Solango

I've been doing quite a bit of work with Solr lately, both at the office and at home and, by golly, I love it! It's very powerful and simple to integrate with regardless of your platform.

In this post I'll explain how to use Solr as a data-store independent search provider for Django projects. I'll assume that you have a functional Solr install and generally understand how to use it. If that's not the case Apache's Solr documentation can help.

Get Solango

The easiest way to get your project talking to Solr is via the Solango Django application. Grab the source from here and copy the solango sub-directory into your PYTHON_PATH.

Configure Solango

Solango must now be configured. Jump into the solango directory that you copied above and edit the settings.py file. Modify the SEARCH_UPDATE_URL, SEARCH_SELECT_URL, and SEARCH_PING_URLS settings to match your Solr environment. For example if your Solr instance was running locally on port 8080 your settings would look like:

SEARCH_UPDATE_URL = getattr(settings, "SEARCH_UPDATE_URL", "http://localhost:8080/solr/update")
SEARCH_SELECT_URL = getattr(settings, "SEARCH_SELECT_URL", "http://localhost:8080/solr/select")
SEARCH_PING_URLS =  getattr(settings, "SEARCH_PING_URLS", ["http://localhost:8080/solr/admin/ping",])

Configure Your Application

Solango configuration out of the way you can now configure your project's settings.py to include the solango app. For example:

INSTALLED_APPS = (
    'solango',
    # your other apps
)

Define the Model

Now I'll show you an example model that you could search upon. I'll create a standard model named Article and a Solr document type named ArticleDocument. Essentially ArticleDocuments are the Solr equivilent of Articles. The last line ties the model and the document together. Also note the copy setting for the document's fields. That instructs Solr to make them copy fields that target the default search field.

import solango

# regular model, lives relationally
class Article(models.Model):
    id = models.AutoField(primary_key=True)
    title = models.CharField(max_length=1024)
    content = models.TextField()

# SearchDocument, lives in Solr
class ArticleDocument(solango.SearchDocument):
    title = solango.fields.CharField(copy=True)    
    content = solango.fields.TextField(copy=True)

# tie them together
solango.register(Article, ArticleDocument)

Define the Schema

With the meta data defined at the Django level we must now define it within Solr. Luckily solango simplifies this. Drop to a shell and run the following command in your project's main directory:

python manage.py solr --fields

This will output field definitions that you'll have to add to Solr's schema.xml.

########## FIELDS ###########

<field name="title" type="string" indexed="true" stored="true" omitNorms="false"
    required="false" multiValued="false"/>
<field name="content" type="string" indexed="true" stored="true" omitNorms="false"
    required="false" multiValued="false"/>
<field name="model" type="string" indexed="true" stored="true" omitNorms="false"
    required="true" multiValued="false"/>
<field name="id" type="string" indexed="true" stored="true" omitNorms="false"
    required="true" multiValued="false"/>

######## COPY FIELDS ########

<copyField source="title" dest="text"/>
<copyField source="content" dest="text"/>

Index

Assuming you have article data in your main data store it's simple to get it indexed with the following command. Keep in mind that I've had to specify nothing about connectivity to the relational data source above. Rather than use a data store specific Solr data import handler solango will use the Django model.

python manage.py solr --reindex 

Pow, let it run and your data is now indexed in Solr for speedy and powerful searching!

Search

Well, none of this would be useful unless it was actually searched, no? Below is an abbreviated view that queries Solr for ArticleDocuments. Thanks to duct-typing they can fit right in most places an Article model is used.

import solango

def search(search_string):
    articles = solango.connection.select(q = search_string).documents

That's just a basic search. More advanced features like facets and highlighting are supported as well. For details check out this documentation.

Conclusion

Well, it might not be as seamless as Ruby ActiveRecord's acts_as_solr, but solango certainly makes using Solr as a search provider for Django manageable. It decouples Solr from the data store. It provides manage.py-based management of Solr. It supports high level Solr features. It's not too shabby at all, no sir.

This post was but a brief introduction. The Django Solr Documentation can help complete the picture for you.

Fri Jan 01 2010 00:01:00 GMT+0000 (UTC)