One of the strength's of Solr is
it's ease of consumption by other platforms due to its REST API and response
writers which include XML, JSON, native Ruby and native Python code.
If you're trying to consume a Solr service from .Net you could easily use a WebClient and parse the results with .Net's System.Xml namespace and perhaps even build an object wrapper on top of it. Luckily that work's already been done with the solrnet library.
In this post I'll outline the fundamentals of solrnet usage.
Prerequisites
This article assumes you have a .Net development environment such as Visual Studio and a functional Solr install in servlet container. I'll also assume that you understand how to configure Solr's schema. If that's not the case please consult the official Solr wiki.
Sample Schema
For demonstrative purposes I'll assume the following field declarations in schema.xml.
<fields> <field name="id" type="int" indexed="true" stored="true" required="true"/> <field name="title" type="text" indexed="true" stored="true" required="true"/> <field name="content" type="text" indexed="true" stored="true" required="true"/> <field name="tag" type="string" indexed="true" stored="true" multiValued="true"/> <field name="text" type="text" indexed="true" stored="false" multiValued="true"/> </fields> <copyField source="title" dest="text"/> <copyField source="content" dest="text"/> <defaultSearchField>text</defaultSearchField> <uniqueKey>id</uniqueKey>
Project Setup
With the basic system in place now it's time to download solrnet from its project site on Google Code then add references to SolrNet.dll and Microsoft.Practices.ServiceLocation.dll (included with SolrNet) to a project.
Model
Now let's write some bloody code! Consider the following class declaration which defines the document's we'll be working with. In this case it's an article, much like a blog post, with a key, title, textual content and a list of tags.
Notice the SolrUniqueKey and SolrField attributes decorating the properties. That facilitates the mapping of the properties to field's in Solr.
using System;
using System.Collections.Generic;
using SolrNet;
using SolrNet.Attributes;
using SolrNet.Commands.Parameters;
using Microsoft.Practices.ServiceLocation;
class Article {
[SolrUniqueKey("id")]
public int ID { get; set; }
[SolrField("title")]
public string Title { get; set; }
[SolrField("content")]
public string Content { get; set; }
[SolrField("tag")]
public List<String> Tags { get; set; }
}
Writing Data
Model defined we can now connect to Solr and save some articles. The following code locates our Solr instance (running locally on port 8080 in my case), creates some documents and commits them to the index.
// find the service
Startup.Init<Article>("http://localhost:8080/solr");
ISolrOperations<Article> solr =
ServiceLocator.Current.GetInstance<ISolrOperations<Article>>();
// make some articles
solr.Add(new Article()
{
ID = 1,
Title = "my laptop",
Content = "my laptop is a portable power station",
Tags = new List<string>() {
"laptop",
"computer",
"device"
}
});
solr.Add(new Article()
{
ID = 2,
Title = "my iphone",
Content = "my iphone consumes power",
Tags = new List<string>() {
"phone",
"apple",
"device"
}
});
solr.Add(new Article()
{
ID = 3,
Title = "your blackberry",
Content = "your blackberry has an alt key",
Tags = new List<string>() {
"phone",
"rim",
"device"
}
});
// commit to the index
solr.Commit();
Basic Querying
Of course the primary purpose of Solr is performing search queries. Consider the following examples which does a general full-text search on the word "power" and a tag search for "phone":
// fulltext "power" search
Console.WriteLine("POWER ARTICLES:");
ISolrQueryResults<Article> powerArticles = solr.Query(new SolrQuery("power"));
foreach (Article article in powerArticles) {
Console.WriteLine(string.Format("{0}: {1}", article.ID, article.Title));
}
Console.WriteLine();
// tag search for "phone"
Console.WriteLine("PHONE TAGGED ARTICLES:");
ISolrQueryResults<Article> phoneTaggedArticles = solr.Query(new SolrQuery("tag:phone"));
foreach (Article article in phoneTaggedArticles)
{
Console.WriteLine(string.Format("{0}: {1}", article.ID, article.Title));
}
which produces the following output
POWER ARTICLES: 1: my laptop 2: my iphone PHONE TAGGED ARTICLES: 2: my iphone 3: your blackberry
Faceting
One of my personal favorite features of Solr is faceting which enables aggregate counts to be returned along with query results. Faceting is well supported in solrnet.
The following example displays counts per tag of articles matching the "device" tag:
Console.WriteLine("DEVICE TAGGED ARTICLES:");
ISolrQueryResults<Article> articles = solr.Query(new SolrQuery("tag:device"),
new QueryOptions() {
Facet = new FacetParameters
{
// ask solr for facets
Queries = new[] { new SolrFacetFieldQuery("tag") }
}
});
foreach (Article article in articles)
{
Console.WriteLine(string.Format("{0}: {1}", article.ID, article.Title));
}
Console.WriteLine("\nTAG COUNTS:");
foreach (var facet in articles.FacetFields["tag"])
{
Console.WriteLine("{0}: {1}", facet.Key, facet.Value);
}
with the following output:
DEVICE TAGGED ARTICLES: 1: my laptop 2: my iphone 3: your blackberry TAG COUNTS: device: 3 phone: 2 apple: 1 computer: 1 laptop: 1 rim: 1
Conclusion
Solrnet makes consumption of a Solr service easy, but I've only covered the basic concepts here. Other features of Solr such as spell checking and match highlighting are also handled.

As a professional DBA I'm always interested in monitoring systems. I have to
know what's going on with my systems. Even in a world with automatic scaling
strategies and automatic tuning humans have to be in the loop. Let's face it,
sometimes automatic things don't work. Worse yet they sometimes automatically
do what you said, not what you meant:)
As I've been moving more of my read-only data out of traditional relational databases and into highly scalable, simpler document-oriented systems like Solr I've been keeping an eye out for monitoring tools. Specially for Solr I found LucidGaze by Lucid Imagination, a Lucene/Solr specialty shop.
What is LucidGaze?
LucidGaze is essentially the combination of a Solr request handler and a web application that sits in your servlet container. The request handler gathers and stores information about other handlers in the Solr instance and the web app provides the administrator with visualization capabilities.
It's downloadable free of charge from Lucid's website.
Installation
The readme enclosed with LucidGaze does a fine job of getting you up and running so I won't provide step-by-step instructions here. Suffice it to say that it's just copying some jar's into your Solr install and servlet container, then configuring the request handler in solrconfig.xml.
Configuration
After a service restart visit the gaze web application with a url like http://localhost:8080/gaze/index.html and you can configure it to provide visualization for the Solr instance you deployed LucidGaze into by entering its URL and selecting a time retention.


You can then select which handlers in the Solr instance you want to monitor.

Then sit back, relax and let LucidGaze collect metrics.
Visualization
With the configuration out of the way and some metrics collected you can view graphs of the activity in the web application.
- Overview - the main entry point is an overview of all selected handlers with
graphs in requests/sec and milliseconds/request.

- Drilldown - after clicking on a graph a detail graph is displayed that allows
you to adjust the time dimension.

A few information screens are also provided.
-
System Info - a comprehensive system info dialog is available

- Index Info - the index info dialog displays statistics about the index and its
schema

Conclusion
As I lean more and more on Solr it's becoming increasingly important that I understand the load on a given system at a moments notice. LucidGaze certainly seems to help me along the way to that goal.
Note that Lucid Imagination also has a version for LucidGaze for plain-old Lucene, and a certified Solr distribution named LucidWorks which contains LucidGaze.

Back in the 90's and early 2000's I played around with BeOS, an alternative
operating system developed by Be Inc. The intention of BeOS was to be a
desktop operating system that specialized in multimedia and competed with
Microsoft Windows and Mac OS. Be Inc. even attempted unsuccessfully to sell
out to Apple as the replacement for the classic Mac OS.
The proper BeOS is now long defunct but the effort is continued by way of an operating system called Haiku which is an open source continuation of BeOS.
I've been meaning to make some time to play with Haiku (which is currently in
alpha) for a few months now, but family, other projects and work have gotten in
the way. Thanks to some snow... Well, a whole lot of snow, I've had
a chance to get my feet wet with it. I'd now like to
provide some resources for those who are interested in doing the same, and I
definitely suggest that you do if you have an interest in alternative operating
systems.
What is Haiku?
As I stated above Haiku is an open source continuation of the BeOS effort, currently in the alpha phase. Right now only the x86 architecture is supported but future x64 support is likely.
One of the core design principals behind BeOS and Haiku is to keep the system simple. The belief of the designers is that operating systems like unix's and windows have had layer after layer added on over time as new needs arose. This layering resulted in inconsistency and complexity. BeOS and Haiku avoided this complexity by starting from scratch with modern needs in mind.
While it's not a unix (and sort of takes exception to unix) it is POSIX compliant, has a Bash shell and has ports of many typical open source unix-y programs. If you're familiar with unix you'll feel at home on BeOS/Haiku. If you're not familiar with unix it should still be easy enough to operate due to Haiku's user-friendly GUI.

Installation
Installation is pretty strait-forward and entirely GUI based. For most people it will be something like:
- Select your destination disk
- Click "Setup partitions..." and make a BeOS partition
- Click "Install" and watch status bar eagerly
- Click on "Write Boot Sector to <Disk Name>" button

Software
No matter how stable, fast, or easy to use an operating system is it's only as useful as the software it runs. Luckily plenty of apps exist or have been ported to BeOS. Developers are also porting new apps to Haiku all the time.
Right out of the box you'll notice familiar open source applications are included such as the nano text editor and the gcc C compiler. You'll also have the necessities like a paint program (WonderBrush), media player (MediaPlayer), PDF viewer (BePDF), web browser (BeZilla), and others.

Thus far I've relied on three main sites for additional software for Haiku:
- BeBits - A general BeOS software repository. Some apps aren't compatible with Haiku specifically but it's still a great resource due to its variety.
- Haiku Ports - A repository of open source software that's been ported to Haiku.
- HaikuWare - A vast Haiku-specific software site.
In general there's a little bit of something for everyone if you do some looking around: productivity, multimedia, software development and even games like Wolfenstein 3d.

Hardware Support
It's hard for me to speak with any authority about supported hardware but drivers do indeed seem to be supported for most popular hardware like good video cards from NVIDIA and ATI. BeBits and HaikuWare are great resources for drivers for audio, video and other devices but I suspect there will be gaps for more obscure devices.
Development
Most programmers will be pleased to know that Haiku ships with such necessities as Python and gcc. A myriad of other languages like BASIC, Ruby and Eiffel are also available from the sites outlined above as well.
The most fundamental way to write Haiku applications, however, is to use the
actual BeOS API from C++. While I'm not all that familiar with the API myself I managed
to rearrange the HelloWorld sample from the official BeOS R5 sample code into a
concise example here. It simply displays a new window with the classic
"Hello, World!" text.
Note that the code is arranged into a single unit for ease of posting. It's by no means intended to exhibit proper style.
#include <Application.h>
#include <Window.h>
#include <StringView.h>
class HelloWorldView : public BStringView {
public:
HelloWorldView(BRect rect, const char *name, const char*text):
BStringView(rect, name, text) {
SetFont(be_bold_font);
SetFontSize(24);
}
};
class HelloWorldWin : public BWindow {
public:
HelloWorldWin(BRect frame) :
BWindow(frame, "Hello", B_TITLED_WINDOW,
B_NOT_RESIZABLE | B_NOT_ZOOMABLE) {
HelloWorldView *view;
BRect rect(Bounds());
view = new HelloWorldView(rect, "HelloWorldView", "Hello, World!");
AddChild(view);
}
virtual bool QuitRequested() {
be_app->PostMessage(B_QUIT_REQUESTED);
return true;
}
};
class HelloWorldApp: public BApplication {
public:
HelloWorldApp() :
BApplication("application/x-vnd.Be-HelloWorld") {
HelloWorldWin *wnd;
BRect rect;
rect.Set(100, 80, 260, 120);
wnd = new HelloWorldWin(rect);
wnd->Show();
}
};
int main(int argc, char **argv) {
HelloWorldApp app;
app.Run();
return 0;
}
Assuming the code was placed in a source file named hello_world.cpp it could be compiled with:
~> g++ -lbe hello_world.cpp
Conclusion
Haiku is only in alpha. Pretty darn young. Be forewarned that any software beyond what's bundled may require some effort to get to work. Still, it's easy enough to use for a non-hacker, but powerful enough for a hacker.
It seems to support that which most people need to do and good bit of what they want to do. It's definitely worth a look.

While
working on a certain unnamed iPhone app lately I ran across the need to
use basic authentication in communication with REST services. For something
that seems to be such a fundamental need for mobile applications I figured most
of the work would be done for me. Turns out that's not the case. A few
details are left up to you, the Cocoa Touch programmer.
What Basic Authentication is
In order to implement basic authentication in Cocoa Touch it is important to understand how it works. Basic authentication tokens are essentially formatted into a string in the format:
username:password
They are then Base64-encoded and formatted into an "Authorization" HTTP header who's value looks like:
Basic c2NvdHQ6dGlnZXI=
where "c2NvdHQ6dGlnZXI=" is the encoded token pair and "Basic" is a hard-coded prependage.
Easy enough right? Encode username and password, slap it in the header and make the request.
What's not Provided by Cocoa Touch: Base64
I thought for sure I'd be able to leverage something out-of-the-box to handle the Base64 encoding. Surely I can do it by using some simple method or function somewhere. It'd just be a one-liner something-er-other, right? Right?? Wrong!
Nothing in Cocoa Touch natively provides you with Base64 encoding capabilities. You also don't have access to openssl on the iPhone via the SDK. In Cocoa Touch's defense I guess it never claimed to have the batteries included (my python & ruby soaked brain always expects everything to be done for me;)).
While I've seen suggestions to statically link openssl against your iPhone app it's not only overkill but presumably puts the responsibility on you to comply with U.S. export regulations (the cryptography in openssl is legally a munition in the states after all).
Besides, it's rather simple to implement Base64 yourself.
A Base64 Implementation
There are a number of implementations you can cherry-pick from elsewhere but in the spirit of demonstration I'll provide an example Base64 encoder that you can use in your project.
Contrary to the spirit of demonstration, however, I'm not going to explain it too much as it's not the subject of the post. If you need more background about the algorithm Ramkumar Menon wrote an excellent blog post about it.
static char *alphabet = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+-";
@implementation Base64
+(char *)encode:(NSData *)plainText {
// create an adequately sized buffer for the output. every 3 bytes
// become four basically with padding to the next largest integer
// divisible by four.
char * encodedText = malloc((((([plainText length] % 3) +
[plainText length]) / 3) * 4) + 1);
char* inputBuffer = malloc([plainText length]);
inputBuffer = (char *)[plainText bytes];
NSInteger i;
NSInteger j = 0;
// encode, this expands every 3 bytes to 4
for(i = 0; i < [plainText length]; i += 3) {
encodedText[j++] = alphabet[(inputBuffer[i] & 0xFC) >> 2];
encodedText[j++] = alphabet[((inputBuffer[i] & 0x03) << 4)
| ((inputBuffer[i + 1] & 0xF0) >> 4)];
if(i + 1 >= [plainText length])
// padding
encodedText[j++] = '=';
else
encodedText[j++] = alphabet[((inputBuffer[i + 1] & 0x0F) << 2)
| ((inputBuffer[i + 2] & 0xC0) >> 6)];
if(i + 2 >= [plainText length])
// padding
encodedText[j++] = '=';
else
encodedText[j++] = alphabet[inputBuffer[i + 2] & 0x3F];
}
// terminate the string
encodedText[j] = 0;
return outputBuffer;
}
@end
Creating and Using a Proper Request
Now that we're ready to speak the encoding that the webservers are expecting we can get down to business. Consider the following code which executes an authenticated request against a resource via a synchronous NSURLRequest. Adding an "Authorization" header with the appropriately formatted, Base64-encoded authentication tokens are all that's required to authenticate the request.
NSURL *url = [NSURL URLWithString:@"http://127.0.0.1/"];
NSString userName = @"scott";
NSString password = @"tiger";
NSError *myError = nil;
// create a plaintext string in the format username:password
NSMutableString *loginString = (NSMutableString*)[@"" stringByAppendingFormat:@"%@:%@", userName, password];
// employ the Base64 encoding above to encode the authentication tokens
char *encodedLoginData = [Base64 encode:[loginString dataUsingEncoding:NSUTF8StringEncoding]];
// create the contents of the header
NSString *authHeader = [@"Basic " stringByAppendingFormat:@"%@",
[NSString stringWithCString:encodedLoginData length:strlen(encodedLoginData)]];
NSMutableURLRequest *request = [NSMutableURLRequest requestWithURL: url
cachePolicy: NSURLRequestReloadIgnoringCacheData
timeoutInterval: 3];
// add the header to the request. Here's the $$$!!!
[request addValue:authHeader forHTTPHeaderField:@"Authorization"];
// perform the reqeust
NSURLResponse *response;
NSData *data = [NSURLConnection
sendSynchronousRequest: request
returningResponse: &response
error: &myError];
*error = myError;
// POW, here's the content of the webserver's response.
NSString *result = [NSString stringWithCString:[data bytes] length:[data length]];
Conclusion
Aside from rolling-your-own (or snagging-someone-elses) Base64 implementation this isn't too bad.
To take it further you might employ NSURLCredential for storage of your authentication tokens.
Also if an asynchronous NSMutableURLRequest is used you can easily handle a webserver issuing a challenge by implementing the didReceiveAuthenticationChallenge message.

It's unavoidable.
Pretty much anywhere you go as a programmer these days
concurrency is important. It's no less important on mobile platforms like the
iPhone. In this post I intend to outline the fundamental techniques necessary
for asynchronous programming on the iPhone. While these techniques could very
well apply to standard, desktop Cocoa I'll focus on Cocoa touch.
I'll cover three basic approaches: NSObject's performSelectorInBackground message, NSThread and NSTimer.
NSObject's performSelectorInBackground message
NSObject's performSelectorInBackground message can be used to easily farm off tasks that run asynchronously. Here's an example that could be placed in a view controller.
-(void) workerThread {
NSAutoreleasePool *pool = [[NSAutoreleasePool alloc] init];
NSString *str = [NSString stringWithContentsOfURL:[NSURL URLWithString:@"http://www.chrisumbel.com"]];
[pool release];
}
- (void)viewDidLoad {
[super viewDidLoad];
[self performSelectorInBackground:@selector(workerThread) withObject:nil];
}
performSelectorInBackground simply executed workerThread asynchronously. Not much to it.
Essentially that just loaded a string with the contents of a web page asynchronously in about the most raw form possible in Cocoa. Obviously this is far more simplistic than anything you'd do for production but it is a potentially long-running, I/O-bound operation which makes it a fine candidate for asynchronous execution. Note that it was unconcerned with synchronization and did not communicate back with the main thread.
Check out the the use of NSAutoreleasePool. It's imperative that you create an NSAutoreleasePool initially and release it before you exit the thread. It's responsible for memory management in the thread body.
NSThread Basics
The most obvious class involved in asynchronous operations is NSThread which, as you would guess, effectively owns a separate thread of execution. Consider the following functional equivalent of the example above:
-(void)workerThread {
// setup the thread's memory management.
NSAutoreleasePool *pool = [[NSAutoreleasePool alloc] init];
// grab the contents of a web page
NSString *str = [NSString stringWithContentsOfURL:[NSURL
URLWithString:@"http://www.chrisumbel.com"]];
[pool release];
}
-(void)viewDidLoad {
[super viewDidLoad];
// spawn a worker thread
[NSThread detachNewThreadSelector:@selector(workerThread)
toTarget:self withObject:nil];
}
Thread Communication
Now I'll alter the example slightly to communicate back to the main thread. This is useful because you nearly always want to execute UI code on the main thread. The following example simply outputs its results to a label.
-(void)updateContent:(NSString *)content {
[outputLabel setText:content];
}
-(void)workerThread {
NSAutoreleasePool *pool = [[NSAutoreleasePool alloc] init];
NSString *str = [NSString stringWithContentsOfURL:
[NSURL URLWithString:@"http://www.chrisumbel.com"]];
// send our results back to the main thread
[self performSelectorOnMainThread:@selector(updateContent:)
withObject:str waitUntilDone:NO];
[pool release];
}
-(void)viewDidLoad {
[super viewDidLoad];
[NSThread detachNewThreadSelector:@selector(workerThread)
toTarget:self withObject:nil];
}
It's NSObject's performSelectorOnMainThread message that facilitates the inter-thread communication. Arguments to the target message (updateContent in my case) are passed along via withObject.
Synchronization
Mutex behavior is accomplished via the NSLock class. The following example synchronizes appending the results of the URL-lookup to a local file.
NSLock *myLock = nil;
-(void)workerThread:(NSString *)urlString {
NSAutoreleasePool *pool = [[NSAutoreleasePool alloc] init];
NSString *str = [NSString stringWithContentsOfURL:
[NSURL URLWithString:urlString]];
// block other threads
[myLock lock];
FILE *f = fopen("content.txt", "a+");
fputs([content UTF8String], f);
fclose(f);
// stop blocking
[myLock unlock];
[pool release];
}
-(void)viewDidLoad {
[super viewDidLoad];
// create the lock object
myLock = [[NSLock alloc] init];
// spawn worker thread 1
[NSThread detachNewThreadSelector:@selector(workerThread:)
toTarget:self withObject:@"http://www.apple.com"];
// spawn worker thread 2
[NSThread detachNewThreadSelector:@selector(workerThread:)
toTarget:self withObject:@"http://www.gnome.org"];
}
NSTimer
The NSThread examples above were true, honest-to-goodness multi-threaded where each task was running in a separate NSRunLoop (Cocoa event handling loop). By contrast the following code uses an NSTimer running on the main run-loop to periodically execute a timed operation. NSTimers aren't perfectly accurate, however. You shouldn't rely on them being real-time.
NSTimer *timer = nil;
NSInteger timesExecuted = 0;
-(void)timerTick {
timesExecuted++;
[outputLabel setText:[timesExecuted stringValue]];
}
-(void)viewDidLoad {
[super viewDidLoad];
// create a timer that ticks every 10 seconds and executes timerTick
// which I defined above
timer = [NSTimer scheduledTimerWithTimeInterval: 10.0 target:self
selector:@selector(timerTick) userInfo:nil repeats: YES];
}
Conclusion
Well, there's the basics. For more depth you can check out Apple's iPhone Threading Programming Guide It's quite comprehensive and definitely worth the read.
Digg it
Reddit
Delicous
Facebook








