If you're trying to consume a Solr service from .Net you could easily use a WebClient and parse the results with .Net's System.Xml namespace and perhaps even build an object wrapper on top of it. Luckily that work's already been done with the solrnet library.

In this post I'll outline the fundamentals of solrnet usage.

Prerequisites

This article assumes you have a .Net development environment such as Visual Studio and a functional Solr install in servlet container. I'll also assume that you understand how to configure Solr's schema. If that's not the case please consult the official Solr wiki.

Sample Schema

For demonstrative purposes I'll assume the following field declarations in schema.xml.

<fields>
<field name="id" type="int" indexed="true" stored="true" required="true"/> 
<field name="title" type="text" indexed="true" stored="true" required="true"/>
<field name="content" type="text" indexed="true" stored="true" required="true"/>
<field name="tag" type="string" indexed="true" stored="true" multiValued="true"/>  
<field name="text" type="text" indexed="true" stored="false" multiValued="true"/>
</fields>

<copyField source="title" dest="text"/>
<copyField source="content" dest="text"/>
 
<defaultSearchField>text</defaultSearchField>
<uniqueKey>id</uniqueKey>

Project Setup

With the basic system in place now it's time to download solrnet from its project site on Google Code then add references to SolrNet.dll and Microsoft.Practices.ServiceLocation.dll (included with SolrNet) to a project.

Model

Now let's write some bloody code! Consider the following class declaration which defines the document's we'll be working with. In this case it's an article, much like a blog post, with a key, title, textual content and a list of tags.

Notice the SolrUniqueKey and SolrField attributes decorating the properties. That facilitates the mapping of the properties to field's in Solr.

using System;
using System.Collections.Generic;
using SolrNet;
using SolrNet.Attributes;
using SolrNet.Commands.Parameters;
using Microsoft.Practices.ServiceLocation;
  
class Article {
    [SolrUniqueKey("id")]
    public int ID { get; set; }

    [SolrField("title")]
    public string Title { get; set; }

    [SolrField("content")]
    public string Content { get; set; }

    [SolrField("tag")]
    public List<String> Tags { get; set; }
}

Writing Data

Model defined we can now connect to Solr and save some articles. The following code locates our Solr instance (running locally on port 8080 in my case), creates some documents and commits them to the index.

  
// find the service  
Startup.Init<Article>("http://localhost:8080/solr");
ISolrOperations<Article> solr =
     ServiceLocator.Current.GetInstance<ISolrOperations<Article>>();

// make some articles
solr.Add(new Article()
{
    ID = 1,
    Title = "my laptop",
    Content = "my laptop is a portable power station",
    Tags = new List<string>() { 
        "laptop", 
        "computer",
        "device"
    }
});

solr.Add(new Article()
{
    ID = 2,
    Title = "my iphone",
    Content = "my iphone consumes power",
    Tags = new List<string>() { 
        "phone", 
        "apple",
        "device"
    }
});

solr.Add(new Article()
{
    ID = 3,
    Title = "your blackberry",
    Content = "your blackberry has an alt key",
    Tags = new List<string>() { 
        "phone", 
        "rim",
        "device"
    }
});            

// commit to the index
solr.Commit();

Basic Querying

Of course the primary purpose of Solr is performing search queries. Consider the following examples which does a general full-text search on the word "power" and a tag search for "phone":

  
// fulltext "power" search  
Console.WriteLine("POWER ARTICLES:");
ISolrQueryResults<Article> powerArticles = solr.Query(new SolrQuery("power"));

foreach (Article article in powerArticles) {
    Console.WriteLine(string.Format("{0}: {1}", article.ID, article.Title));
}

Console.WriteLine();

// tag search for "phone" 
Console.WriteLine("PHONE TAGGED ARTICLES:");
ISolrQueryResults<Article> phoneTaggedArticles = solr.Query(new SolrQuery("tag:phone"));

foreach (Article article in phoneTaggedArticles)
{
    Console.WriteLine(string.Format("{0}: {1}", article.ID, article.Title));
}

POWER ARTICLES:
1: my laptop
2: my iphone

PHONE TAGGED ARTICLES:
2: my iphone
3: your blackberry

Faceting

One of my personal favorite features of Solr is faceting which enables aggregate counts to be returned along with query results. Faceting is well supported in solrnet.

The following example displays counts per tag of articles matching the "device" tag:

Console.WriteLine("DEVICE TAGGED ARTICLES:");
ISolrQueryResults<Article> articles = solr.Query(new SolrQuery("tag:device"),
  new QueryOptions() {
    Facet = new FacetParameters
    {
        // ask solr for facets
        Queries = new[] { new SolrFacetFieldQuery("tag") }
    }
});

foreach (Article article in articles)
{
    Console.WriteLine(string.Format("{0}: {1}", article.ID, article.Title));
}

Console.WriteLine("\nTAG COUNTS:");

foreach (var facet in articles.FacetFields["tag"])
{
    Console.WriteLine("{0}: {1}", facet.Key, facet.Value);
}

with the following output:

DEVICE TAGGED ARTICLES:
1: my laptop
2: my iphone
3: your blackberry

TAG COUNTS:
device: 3
phone: 2
apple: 1
computer: 1
laptop: 1
rim: 1

Conclusion

Solrnet makes consumption of a Solr service easy, but I've only covered the basic concepts here. Other features of Solr such as spell checking and match highlighting are also handled.

As I've been moving more of my read-only data out of traditional relational databases and into highly scalable, simpler document-oriented systems like Solr I've been keeping an eye out for monitoring tools. Specially for Solr I found LucidGaze by Lucid Imagination, a Lucene/Solr specialty shop.

What is LucidGaze?

LucidGaze is essentially the combination of a Solr request handler and a web application that sits in your servlet container. The request handler gathers and stores information about other handlers in the Solr instance and the web app provides the administrator with visualization capabilities.

It's downloadable free of charge from Lucid's website.

Installation

The readme enclosed with LucidGaze does a fine job of getting you up and running so I won't provide step-by-step instructions here. Suffice it to say that it's just copying some jar's into your Solr install and servlet container, then configuring the request handler in solrconfig.xml.

Configuration

After a service restart visit the gaze web application with a url like http://localhost:8080/gaze/index.html and you can configure it to provide visualization for the Solr instance you deployed LucidGaze into by entering its URL and selecting a time retention.

You can then select which handlers in the Solr instance you want to monitor.

Then sit back, relax and let LucidGaze collect metrics.

Visualization

With the configuration out of the way and some metrics collected you can view graphs of the activity in the web application.

• Overview - the main entry point is an overview of all selected handlers with graphs in requests/sec and milliseconds/request.

  

• Drilldown - after clicking on a graph a detail graph is displayed that allows you to adjust the time dimension.

  

A few information screens are also provided.

• System Info - a comprehensive system info dialog is available

  

• Index Info - the index info dialog displays statistics about the index and its schema

  

Conclusion

As I lean more and more on Solr it's becoming increasingly important that I understand the load on a given system at a moments notice. LucidGaze certainly seems to help me along the way to that goal.

Note that Lucid Imagination also has a version for LucidGaze for plain-old Lucene, and a certified Solr distribution named LucidWorks which contains LucidGaze.

The proper BeOS is now long defunct but the effort is continued by way of an operating system called Haiku which is an open source continuation of BeOS.

I've been meaning to make some time to play with Haiku (which is currently in alpha) for a few months now, but family, other projects and work have gotten in the way. Thanks to some snow... Well, a whole lot of snow, I've had a chance to get my feet wet with it. I'd now like to provide some resources for those who are interested in doing the same, and I definitely suggest that you do if you have an interest in alternative operating systems.

What is Haiku?

As I stated above Haiku is an open source continuation of the BeOS effort, currently in the alpha phase. Right now only the x86 architecture is supported but future x64 support is likely.

One of the core design principals behind BeOS and Haiku is to keep the system simple. The belief of the designers is that operating systems like unix's and windows have had layer after layer added on over time as new needs arose. This layering resulted in inconsistency and complexity. BeOS and Haiku avoided this complexity by starting from scratch with modern needs in mind.

While it's not a unix (and sort of takes exception to unix) it is POSIX compliant, has a Bash shell and has ports of many typical open source unix-y programs. If you're familiar with unix you'll feel at home on BeOS/Haiku. If you're not familiar with unix it should still be easy enough to operate due to Haiku's user-friendly GUI.

Installation

Installation is pretty strait-forward and entirely GUI based. For most people it will be something like:

1. Select your destination disk
2. Click "Setup partitions..." and make a BeOS partition
3. Click "Install" and watch status bar eagerly
4. Click on "Write Boot Sector to <Disk Name>" button

Software

No matter how stable, fast, or easy to use an operating system is it's only as useful as the software it runs. Luckily plenty of apps exist or have been ported to BeOS. Developers are also porting new apps to Haiku all the time.

Right out of the box you'll notice familiar open source applications are included such as the nano text editor and the gcc C compiler. You'll also have the necessities like a paint program (WonderBrush), media player (MediaPlayer), PDF viewer (BePDF), web browser (BeZilla), and others.

Thus far I've relied on three main sites for additional software for Haiku:

• BeBits - A general BeOS software repository. Some apps aren't compatible with Haiku specifically but it's still a great resource due to its variety.
• Haiku Ports - A repository of open source software that's been ported to Haiku.
• HaikuWare - A vast Haiku-specific software site.

In general there's a little bit of something for everyone if you do some looking around: productivity, multimedia, software development and even games like Wolfenstein 3d.

Hardware Support

It's hard for me to speak with any authority about supported hardware but drivers do indeed seem to be supported for most popular hardware like good video cards from NVIDIA and ATI. BeBits and HaikuWare are great resources for drivers for audio, video and other devices but I suspect there will be gaps for more obscure devices.

Development

Most programmers will be pleased to know that Haiku ships with such necessities as Python and gcc. A myriad of other languages like BASIC, Ruby and Eiffel are also available from the sites outlined above as well.

The most fundamental way to write Haiku applications, however, is to use the actual BeOS API from C++. While I'm not all that familiar with the API myself I managed to rearrange the HelloWorld sample from the official BeOS R5 sample code into a concise example here. It simply displays a new window with the classic "Hello, World!" text.

Note that the code is arranged into a single unit for ease of posting. It's by no means intended to exhibit proper style.

#include <Application.h>
#include <Window.h>
#include <StringView.h>

class HelloWorldView : public BStringView {
public:
    HelloWorldView(BRect rect, const char *name, const char*text):
    	BStringView(rect, name, text) {
    	    SetFont(be_bold_font);
    	    SetFontSize(24);
    }
};

class HelloWorldWin : public BWindow {
public: 
    HelloWorldWin(BRect frame) :
    	    BWindow(frame, "Hello", B_TITLED_WINDOW,
                B_NOT_RESIZABLE | B_NOT_ZOOMABLE) {
    	HelloWorldView *view;
	    BRect rect(Bounds());
	    view = new HelloWorldView(rect, "HelloWorldView", "Hello, World!");
	    AddChild(view);
	}	
	
	virtual bool QuitRequested() {
	    be_app->PostMessage(B_QUIT_REQUESTED);	
	    return true;
	}
};

class HelloWorldApp: public BApplication {
public:
    HelloWorldApp() :
	    BApplication("application/x-vnd.Be-HelloWorld") {
	HelloWorldWin *wnd;
	BRect rect;

	rect.Set(100, 80, 260, 120);
	wnd = new HelloWorldWin(rect);

	wnd->Show();
    }		
};

int main(int argc, char **argv) {
    HelloWorldApp app;
    app.Run();
    return 0;
}

Assuming the code was placed in a source file named hello_world.cpp it could be compiled with:

~> g++ -lbe hello_world.cpp

Conclusion

Haiku is only in alpha. Pretty darn young. Be forewarned that any software beyond what's bundled may require some effort to get to work. Still, it's easy enough to use for a non-hacker, but powerful enough for a hacker.

It seems to support that which most people need to do and good bit of what they want to do. It's definitely worth a look.

What Basic Authentication is

In order to implement basic authentication in Cocoa Touch it is important to understand how it works. Basic authentication tokens are essentially formatted into a string in the format:

username:password

They are then Base64-encoded and formatted into an "Authorization" HTTP header who's value looks like:

Basic c2NvdHQ6dGlnZXI=

where "c2NvdHQ6dGlnZXI=" is the encoded token pair and "Basic" is a hard-coded prependage.

Easy enough right? Encode username and password, slap it in the header and make the request.

What's not Provided by Cocoa Touch: Base64

I thought for sure I'd be able to leverage something out-of-the-box to handle the Base64 encoding. Surely I can do it by using some simple method or function somewhere. It'd just be a one-liner something-er-other, right? Right?? Wrong!

Nothing in Cocoa Touch natively provides you with Base64 encoding capabilities. You also don't have access to openssl on the iPhone via the SDK. In Cocoa Touch's defense I guess it never claimed to have the batteries included (my python & ruby soaked brain always expects everything to be done for me;)).

While I've seen suggestions to statically link openssl against your iPhone app it's not only overkill but presumably puts the responsibility on you to comply with U.S. export regulations (the cryptography in openssl is legally a munition in the states after all).

Besides, it's rather simple to implement Base64 yourself.

A Base64 Implementation

There are a number of implementations you can cherry-pick from elsewhere but in the spirit of demonstration I'll provide an example Base64 encoder that you can use in your project.

Contrary to the spirit of demonstration, however, I'm not going to explain it too much as it's not the subject of the post. If you need more background about the algorithm Ramkumar Menon wrote an excellent blog post about it.

static char *alphabet = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+-";

@implementation Base64
    +(char *)encode:(NSData *)plainText {
        // create an adequately sized buffer for the output.  every 3 bytes 
        // become four basically with padding to the next largest integer
        // divisible by four. 
        char * encodedText = malloc((((([plainText length] % 3) +
            [plainText length]) / 3) * 4) + 1);
        char* inputBuffer = malloc([plainText length]);
        inputBuffer = (char *)[plainText bytes];
    
        NSInteger i;
        NSInteger j = 0;
    
        // encode, this expands every 3 bytes to 4
        for(i = 0; i < [plainText length]; i += 3) {
            encodedText[j++] = alphabet[(inputBuffer[i] & 0xFC) >> 2];
            encodedText[j++] = alphabet[((inputBuffer[i] & 0x03) << 4)
                | ((inputBuffer[i + 1] & 0xF0) >> 4)];
    
            if(i + 1 >= [plainText length])
                // padding
                encodedText[j++] = '=';
            else 
                encodedText[j++] = alphabet[((inputBuffer[i + 1] & 0x0F) << 2)
                | ((inputBuffer[i + 2] & 0xC0) >> 6)];
    
            if(i + 2 >= [plainText length])
                // padding
                encodedText[j++] = '=';
            else
                encodedText[j++] = alphabet[inputBuffer[i + 2] & 0x3F];
        }
        
        // terminate the string
        encodedText[j] = 0;
    
        return outputBuffer;
    }
@end

Creating and Using a Proper Request

Now that we're ready to speak the encoding that the webservers are expecting we can get down to business. Consider the following code which executes an authenticated request against a resource via a synchronous NSURLRequest. Adding an "Authorization" header with the appropriately formatted, Base64-encoded authentication tokens are all that's required to authenticate the request.

NSURL *url = [NSURL URLWithString:@"http://127.0.0.1/"];
NSString userName = @"scott";
NSString password = @"tiger";

NSError *myError = nil;

// create a plaintext string in the format username:password
NSMutableString *loginString = (NSMutableString*)[@"" stringByAppendingFormat:@"%@:%@", userName, password];

// employ the Base64 encoding above to encode the authentication tokens
char *encodedLoginData = [Base64 encode:[loginString dataUsingEncoding:NSUTF8StringEncoding]];

// create the contents of the header 
NSString *authHeader = [@"Basic " stringByAppendingFormat:@"%@", 
    [NSString stringWithCString:encodedLoginData length:strlen(encodedLoginData)]];
NSMutableURLRequest *request = [NSMutableURLRequest requestWithURL: url
    cachePolicy: NSURLRequestReloadIgnoringCacheData  
    timeoutInterval: 3];   

// add the header to the request.  Here's the $$$!!!
[request addValue:authHeader forHTTPHeaderField:@"Authorization"];

// perform the reqeust
NSURLResponse *response;

NSData *data = [NSURLConnection  
    sendSynchronousRequest: request  
    returningResponse: &response  
    error: &myError];  
*error = myError;

// POW, here's the content of the webserver's response.
NSString *result = [NSString stringWithCString:[data bytes] length:[data length]];

Conclusion

Aside from rolling-your-own (or snagging-someone-elses) Base64 implementation this isn't too bad.

To take it further you might employ NSURLCredential for storage of your authentication tokens.

Also if an asynchronous NSMutableURLRequest is used you can easily handle a webserver issuing a challenge by implementing the didReceiveAuthenticationChallenge message.

I'll cover three basic approaches: NSObject's performSelectorInBackground message, NSThread and NSTimer.

NSObject's performSelectorInBackground message

NSObject's performSelectorInBackground message can be used to easily farm off tasks that run asynchronously. Here's an example that could be placed in a view controller.

-(void) workerThread {
	NSAutoreleasePool *pool = [[NSAutoreleasePool alloc] init]; 

	NSString *str = [NSString stringWithContentsOfURL:[NSURL URLWithString:@"http://www.chrisumbel.com"]];

	[pool release];  
}

- (void)viewDidLoad {
    [super viewDidLoad];

	[self performSelectorInBackground:@selector(workerThread) withObject:nil];
}

performSelectorInBackground simply executed workerThread asynchronously. Not much to it.

Essentially that just loaded a string with the contents of a web page asynchronously in about the most raw form possible in Cocoa. Obviously this is far more simplistic than anything you'd do for production but it is a potentially long-running, I/O-bound operation which makes it a fine candidate for asynchronous execution. Note that it was unconcerned with synchronization and did not communicate back with the main thread.

Check out the the use of NSAutoreleasePool. It's imperative that you create an NSAutoreleasePool initially and release it before you exit the thread. It's responsible for memory management in the thread body.

NSThread Basics

The most obvious class involved in asynchronous operations is NSThread which, as you would guess, effectively owns a separate thread of execution. Consider the following functional equivalent of the example above:

-(void)workerThread {
    // setup the thread's memory management.
    NSAutoreleasePool *pool = [[NSAutoreleasePool alloc] init]; 

    // grab the contents of a web page
    NSString *str = [NSString stringWithContentsOfURL:[NSURL
        URLWithString:@"http://www.chrisumbel.com"]];
	
    [pool release];  
}

-(void)viewDidLoad {
    [super viewDidLoad];

    // spawn a worker thread
    [NSThread detachNewThreadSelector:@selector(workerThread) 
	toTarget:self withObject:nil];
}

Thread Communication

Now I'll alter the example slightly to communicate back to the main thread. This is useful because you nearly always want to execute UI code on the main thread. The following example simply outputs its results to a label.

-(void)updateContent:(NSString *)content {
    [outputLabel setText:content];
}

-(void)workerThread {
    NSAutoreleasePool *pool = [[NSAutoreleasePool alloc] init]; 

    NSString *str = [NSString stringWithContentsOfURL:
        [NSURL URLWithString:@"http://www.chrisumbel.com"]];
    
    // send our results back to the main thread
    [self performSelectorOnMainThread:@selector(updateContent:)
        withObject:str waitUntilDone:NO];
    
    [pool release];  
}

-(void)viewDidLoad {
    [super viewDidLoad];

    [NSThread detachNewThreadSelector:@selector(workerThread) 
	toTarget:self withObject:nil];
}

It's NSObject's performSelectorOnMainThread message that facilitates the inter-thread communication. Arguments to the target message (updateContent in my case) are passed along via withObject.

Synchronization

Mutex behavior is accomplished via the NSLock class. The following example synchronizes appending the results of the URL-lookup to a local file.

NSLock *myLock = nil;

-(void)workerThread:(NSString *)urlString {
    NSAutoreleasePool *pool = [[NSAutoreleasePool alloc] init]; 

    NSString *str = [NSString stringWithContentsOfURL:
	[NSURL URLWithString:urlString]];

    // block other threads	    
    [myLock lock];
	
    FILE *f = fopen("content.txt", "a+");
    fputs([content UTF8String], f);
    fclose(f);
	
    // stop blocking
    [myLock unlock];

    [pool release];  
}

-(void)viewDidLoad {
    [super viewDidLoad];

    // create the lock object
    myLock = [[NSLock alloc] init];
	
    // spawn worker thread 1
    [NSThread detachNewThreadSelector:@selector(workerThread:) 
	toTarget:self withObject:@"http://www.apple.com"];
    
    // spawn worker thread 2
    [NSThread detachNewThreadSelector:@selector(workerThread:) 
	toTarget:self withObject:@"http://www.gnome.org"];
}

NSTimer

The NSThread examples above were true, honest-to-goodness multi-threaded where each task was running in a separate NSRunLoop (Cocoa event handling loop). By contrast the following code uses an NSTimer running on the main run-loop to periodically execute a timed operation. NSTimers aren't perfectly accurate, however. You shouldn't rely on them being real-time.

NSTimer *timer = nil;
NSInteger timesExecuted = 0;

-(void)timerTick {
    timesExecuted++;
    [outputLabel setText:[timesExecuted stringValue]];
}

-(void)viewDidLoad {
    [super viewDidLoad];

    // create a timer that ticks every 10 seconds and executes timerTick
    // which I defined above
    timer = [NSTimer scheduledTimerWithTimeInterval: 10.0 target:self 
	selector:@selector(timerTick) userInfo:nil repeats: YES];
}

Conclusion