Quick Notes on ScriptTransformers in Solr DataImportHandlers - One thing that's impressed me with Solr is the flexibility of the Data Import Handlers (DIHs). When I was new to Solr there were several times I thought for sure I'd have to write my own extension of DataImportHandler. Every time that's happened I've been wrong. A transformer or something handled my needs. Sometimes it's wonderful to be wrong! Especially when it means less code I have to write myself! One of the aspects of DIH's that provide such great flexibility is transformers like RegexTransformer and TemplateTransformer. In this post, however, I'm going to *quickly* cover the ScriptTransformer wich allows you to employ your own custom JavaScript code in the processing of imports. Prerequisites Obviously you'll need a functional Solr instance. Also, ScriptTransformers require Java 6 due to JavaScript support. I'll...
Employing Solr/Lucene with SQL Server for Full-Text Searching - I've been fiddling with Lucene a good bit of late and have been quite impressed. It's more than just a "blazing fast" full-text indexing system, especially when implemented via Solr. With Solr it becomes an incredibly scalable, full-featured and extensible search engine platform. I had always assumed that the Lucene stack wasn't for me. For the most part I store my data either in SQL Server or MySQL, both of which have perfectly adequate full-text search capability. It turns out that I could have saved myself a few headaches and saved my employer some money by adopting Solr and not writing my own faceting, caching, etc. Naturally, Lucene/Solr isn't for everyone. If you just have a few hundred-thousand rows of text that you want to perform some basic searches on under...
Using Solr in Django for Full-Text Searching via Solango - I've been doing quite a bit of work with Solr lately, both at the office and at home and, by golly, I love it! It's very powerful and simple to integrate with regardless of your platform. In this post I'll explain how to use Solr as a data-store independent search provider for Django projects. I'll assume that you have a functional Solr install and generally understand how to use it. If that's not the case Apache's Solr documentation can help. Get Solango The easiest way to get your project talking to Solr is via the Solango Django application. Grab the source from here and copy the solango sub-directory into your PYTHON_PATH. Configure Solango Solango must now be configured. Jump into the solango directory that you copied above and edit the settings.py...
Monitoring Solr with LucidGaze - As a professional DBA I'm always interested in monitoring systems. I have to know what's going on with my systems. Even in a world with automatic scaling strategies and automatic tuning humans have to be in the loop. Let's face it, sometimes automatic things don't work. Worse yet they sometimes automatically do what you said, not what you meant:) As I've been moving more of my read-only data out of traditional relational databases and into highly scalable, simpler document-oriented systems like Solr I've been keeping an eye out for monitoring tools. Specially for Solr I found LucidGaze by Lucid Imagination, a Lucene/Solr specialty shop. What is LucidGaze? LucidGaze is essentially the combination of a Solr request handler and a web application that sits in your servlet container. The request handler gathers...
Solrnet, a Solr Client Library for .Net - One of the strength's of Solr is it's ease of consumption by other platforms due to its REST API and response writers which include XML, JSON, native Ruby and native Python code. If you're trying to consume a Solr service from .Net you could easily use a WebClient and parse the results with .Net's System.Xml namespace and perhaps even build an object wrapper on top of it. Luckily that work's already been done with the solrnet library. In this post I'll outline the fundamentals of solrnet usage. Prerequisites This article assumes you have a .Net development environment such as Visual Studio and a functional Solr install in servlet container. I'll also assume that you understand how to configure Solr's schema. If that's not the case please consult the official Solr wiki....
Faceted Queries on acts_as_solr Associations - Recently in a rails app that employs Solr (via the acts_as_solr plugin) I've had the need to produce aggregate counts of entities on the far end of a many-to-many relationship. Essentially a tag cloud. My first attempt was to keep it entirely in ActiveRecord which resulted in a proliferation of SQL command executions. Obviously that wasn't performant. Sure it looked elegant, but was slow and unsustainable. While I could have hand-crafted the SQL it was more performant still to retrieve the aggregations from Solr via facets. Hell, I had the data handy! Such a faceted query directly from Solr is qutie simple but it required a little research to get it done with acts_as_solr due to my unfamiliarity with it. In order to make it simple for others attempting to do...
