Chris Umbel

Using MongoDB as a Backend for Django with django-mongodb-engine

MongoDB logoI've been pretty taken with MongoDB of late. It's nearly disgusting how productive it is. However, like all database systems it's only as productive as the higher-level systems that interface with it.

Personally I've used it primarily from Java and Ruby on Rails (via MongoMapper) and from Python via PyMongo.

PyMongo essentially exposes MongoDB via Python dictionaries. Sure, it's plenty elegant and plenty pythonic but when it came to django I wanted was something more MongoMapper-like, an honest-to-goodness Object-Document-Mapper.

django logoMonths and months ago when I looked into the existence of a MongoDB driver for django all that turned up was dead, false-start projects, but after revisiting it recently django-mongodb-engine came to my attention. django-mongodb-engine is pretty much exactly what I was looking for. The authors describe it as, "a database backend that adds mongodb support to django". In this post I intend to introduce it to you.

I'm going to assume you're comfortable getting a django application started. If that's not the case please check out the official getting-started docs.

Requirements

In order to leverage MongoBD from django you'll need the following software installed and operating:

  • Python - If you're reading this article odds are you already have it.
  • MongoDB - I guess this is somewhat self explanatory, but you'll need MongoDB itself.
  • django-norel - is a special version of django designed for use with non-relational database engines in general.
  • django tooblox - a general purpose utility library upon which django-mongodb-engine depends
  • mongodb-engine - the MongoDB driver for django.

Application

Infrastructure in place I'll go ahead and create a django project named "testproj" with an application named "testapp".

django-admin.py startproject testproj
cd testproj/
django-admin.py startapp testapp

Setup

Naturally the django project must be configured to talk to a specific database in settings.py.

DATABASES = {
    'default': {
        'ENGINE': 'django_mongodb_engine',
        'NAME': 'mydatabase',
        'USER': '',
        'PASSWORD': '',
        'HOST': 'localhost',
        'PORT': '27017',
        'SUPPORTS_TRANSACTIONS': False,
    },
}

Edit 2012-11-20: Older versions may require ENGINE to be 'django_mongodb_engine.mongodb'.

Models

In said application you could create a model in models.py like

from django.db import models

class Article(models.Model):
    title = models.CharField(max_length = 64)
    content = models.TextField()

It looks like a standard old django model, right? Nothing fancy here, just a plain old model with plain old fields.

Note that it's important to not create and AutoField named "id" or things will blow up when saving. That's because Mongo wants to put a proper ObjectId in there.

Saving

We can then save some model data strait away from a django view.

from django.http import HttpResponse
from models import *

def testview(request):
  article = Article(title = 'test title',
    content = 'test content')
  article.save()

  return HttpResponse("

Saved!

")

If you then peer into Mongo with a native javascript query query like

db.testapp_article.find()

you'll find your document returned

{ "_id" : ObjectId("4cb4f9a01a8ff904fa000001"), "content" : "test content", "title" : "test title" }

Querying

of course it's a simple matter to query Mongo from django to retrieve a list of Article objects just like you would with a relational store.

articles = Article.objects.all()

Embedding Documents

Many document-database-esque features are covered as well, but I'll just touch on one here. With a minor alteration to our model

from django.db import models
from django_mongodb_engine.mongodb.fields import EmbeddedModel
from django_mongodb_engine.fields import ListField

class Comment(EmbeddedModel):
  name = models.CharField(max_length = 160)
  content = models.TextField()

class Article(models.Model):
  title = models.CharField(max_length = 160)
  content = models.TextField()
  comments = ListField()  

then we can embed Comment documents into Articles.

article = Article(title = 'test title', 
  content = 'test content')

article.comments.append(Comment(name = 'alice', description = 'foo bar'))
article.comments.append(Comment(name = 'bob', description = 'fun baz'))

article.save()

Conclusion

Thanks to the hard work of others it's a simple matter for us to use MongoDB as a backend for django. To see some specific examples of non-relational-style features check out the tests from the django-mongodb-engine project.

Wed Oct 13 2010 01:14:48 GMT+0000 (UTC)

Comments

Ruby in my Enterprise with JRuby Thanks to JRubyConf

JRuby logo Last weekend I went to JRubyConf and had a blast. I left armed with some new knowledge, some new contacts and a new academic appreciation for /whiske?y/. One of the best parts is that I really only had to pay for the hotel because I won the ticket at a Pittsburgh Ruby Brigade meeting.

Now, even though I was attending a JRuby conference I wasn't really all that familiar with JRuby itself. Of course I understood that it was a port of Ruby to the JVM thus facilitating use of existing Java code. I understood that JRuby has favorable concurrency characteristics due to the lack of a global interpreter lock. But I always assumed it was generally immature and difficult to work with.

Wow, was I wrong. It turns out that it's robust, solves many of the Ruby (and Rails) problems I've been having and simplifies tasks I figured I'd have to simplify myself.

It only took two key advantages for me to turn our rails department on a dime and start JRuby adoption: great x64 windows support and .war file deployment of rails apps.

Excellent 64-bit Windows Support

This took me off guard and I felt very stupid for not thinking of it earlier. JRuby runs great on every meaningful, modern platform within my enterprise. This stems from the proliferation of perfectly solid 100% pure Java libraries and drivers that exist for nearly any task including rock solid JDBC drivers for database connectivity.

Personally, I only use 64-bit macs and 32-bit linux boxes at the office, but a number of people who contribute to my projects are using 64-bit Windows machines, and... well... aren't going to change. Even though the MRI's quite happy on the platforms I use personally many gems that involve native code have proven to be unstable on Windows x64 thus screwing my coworkers (therefore annoying me). For instance it took quite a bit of hacking around to get a working mysql gem on my boss's workstation even though other x64 windows machines seemed to be ok with the same libmysql.dll.

With JRuby I never had to worry about it. I used a rock solid JDBC driver with an ActiveRecord adapter and every machine everywhere was happy... Happy and FAST!

I guess I always operated under the assumption that the MRI was as portable as Ruby runtimes will get. At its core it's pretty portable I suppose, but it's crippled by severe reliance on native code extensions. JRuby, however, is largely free of such concerns as the Java extensions are plenty fast enough and incredibly portable.

Easy .war File Deployment of Rails Applications

I was also caught off guard by JRuby on Rails' capacity to deploy applications as .war files.

My surprise is somewhat more understandable here, though. I'm not really a Java guy despite having been immersed in it recently. Sure, I hacked out plenty of Java from 1995-ish to 2001-ish but my head was never in Java Servlet-based web apps. I've only deployed other people's .war files, namely Solr, which felt more third-party-appliance-like. It wasn't something I was looking for and therefore didn't know that I wanted it.

As soon as I heard a speaker indicate I could wrap up the JRuby runtime itself along with whatever gems my rails project needs with a single “warble” command I was sold. That's some easy deployment!

All you have to do is install the warbler gem

jruby -S gem install warbler

and then type

warble

in your rails project's directory. A .war file will then be produced that you can deploy into the servlet container of your choice.

Conclusion

These two pieces of knowledge have already had a wonderful impact on productivity at the office and it's barely been a week. Even if I had to pay for the ticket it would have been worth it.

Thu Oct 07 2010 04:36:20 GMT+0000 (UTC)

Comments

Rich-Style Formatting of an Android TextView

Even a developer-friendly mobile platform like Android can have a developer feeling a little lost when trying to perform simple tasks when you're unfamiliar with the platform.

One of these simple, however poorly documented, tasks is rich-style text formatting within a TextView.

SpannableString

While it's possible to set a TextView's text property to a simple String and configure the TextView to have the formatting you desire you're then limited in how granular you can control the formatting within the TextView itself. The SpannableString class allows you to easily format certain pieces (spans) of a string one way and other pieces another by applying extensions of CharacterStyle (i.e. ForegroundColorSpan) via the setSpan method.

In the end this isn't limited to formatting. It also allows the developer to add behaviors to spans such as reacting to click events.

Example

Here's an example onCreate method of an Activity. This assumes there's a main.xml layout with a TextView identified by "rich_text".

Essentially this code will set a TextView's text to the familiar, "Lorem ipsum dolor sit amet" and perform the following formatting:

  • Make "Lorem" red
  • Make "ipsum" a 1.5 times bigger than what the TextView's setting
  • Make "dolor" display a toast message when touched
  • Strike through "sit"
  • Make "amet" twice as big as the TextView's setting, green and a link to this site

 
@Override
public void onCreate(Bundle icicle) {
    super.onCreate(icicle);
    setContentView(R.layout.main);
    richTextView = (TextView)findViewById(R.id.rich_text);

    // this is the text we'll be operating on
    SpannableString text = new SpannableString("Lorem ipsum dolor sit amet");

    // make "Lorem" (characters 0 to 5) red
    text.setSpan(new ForegroundColorSpan(Color.RED), 0, 5, 0);

    // make "ipsum" (characters 6 to 11) one and a half time bigger than the textbox
    text.setSpan(new RelativeSizeSpan(1.5f), 6, 11, 0);

    // make "dolor" (characters 12 to 17) display a toast message when touched
    final Context context = this;
    ClickableSpan clickableSpan = new ClickableSpan() {
        @Override
        public void onClick(View view) {
            Toast.makeText(context, "dolor", Toast.LENGTH_LONG).show();
        }
    };
    text.setSpan(clickableSpan, 12, 17, 0);

    // make "sit" (characters 18 to 21) struck through
    text.setSpan(new StrikethroughSpan(), 18, 21, 0);

    // make "amet" (characters 22 to 26) twice as big, green and a link to this site.
    // it's important to set the color after the URLSpan or the standard
    // link color will override it.
    text.setSpan(new RelativeSizeSpan(2f), 22, 26, 0);
    text.setSpan(new URLSpan("http://www.chrisumbel.com"), 22, 26, 0);
    text.setSpan(new ForegroundColorSpan(Color.GREEN), 22, 26, 0);

    // make our ClickableSpans and URLSpans work
    richTextView.setMovementMethod(LinkMovementMethod.getInstance());

    // shove our styled text into the TextView        
    richTextView.setText(text, BufferType.SPANNABLE);
}

The results of which will look something like:

Note that we set the TextView's movement method to a LinkMovementMethod instance. Without that the ClickableSpan and URLSpans won't perform their intended actions.

Next Steps

This covers the fundamental concepts, but there are many extensions of CharacterStyle I haven't covered here. Check out the CharacterStyle documentation for more details.

Also note that a SpannableStringBuilder is provided for building large spannables from smaller pieces.

Sat Aug 28 2010 18:25:02 GMT+0000 (UTC)

Comments

MapReduce with MongoMapper

A number of rails projects I've been working on lately have used MongoDB for a back-end via MongoMapper. In general it seems to do pretty much anything I'd want to do in a typical web app but finding documentation on how to do it can be difficult.

One such task I came across recently was performing on-the-fly map-reduce. After implementing it myself I decided to share a simple example.

Blog Post Example

Consider the typical Article model which is essentially a blog post. A title, some content and a list of tags. What I'll do is produce aggregate counts that could be used to display a tag cloud.

class Article
  include MongoMapper::Document

  key :title, String
  key :content, String
  key :tags, Array
end

Sample Data

I'll throw in three sample articles from the rails console.

Article.new(:title => 'one', :content => 'article one', :tags => ['number', 'one']).save()
Article.new(:title => 'two', :content => 'article two', :tags => ['number', 'two']).save()
Article.new(:title => 'uno', :content => 'article uno', :tags => ['number', 'uno', 'one']).save()

Map Reduce

Here's the money. I'll just slap a string containing a map javascript function and one containing a reduce javascript function into the Collection.map_reduce method.

I'll encapsulate this into a TagCloud class to be tidy.

class TagCloud
  self.map
    <<-JS
    function(){
      this.tags.forEach(function(tag){
        emit(tag, 1);
      });
    }
    JS
  end

  self.reduce 
    <<-JS
    function(prev, current) {
      var count = 0;

      for (index in current) {
          count += current[index];
      }

      return count;
    }
    JS
  end

  def self.build
    Article.collection.map_reduce(map, reduce, :query => {})
  end
end

In this case my mapping function simply iterates all tags in each document and emits them with a value of 1. The reduce function in turn tallies them up.

Querying

Now in the Article's controller I'll query the map_reduce in the tag_cloud action. This is a perfect action to sit behind a partial.

  def tag_cloud
    # here's where you could also add some filtering or sorting
    @tags = TagCloud.build.find()
  end

Display

Our map_reduce returns a hash keyed on the tag name we can use to display a tag cloud. For simplicity's sake I'll just display counts here, but here's where fancy tag-cloud style formatting could occur.

<% @tags.each do |tag| %>
   <%= "#{tag['_id']}" %> (<%= "#{tag['value']}" %>)
<% end %>

Producing:

number (3.0) one (2.0) two (1.0) uno (1.0)

Sun Aug 01 2010 02:52:42 GMT+0000 (UTC)

Comments
< 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 >
Follow Chris
RSS Feed
Twitter
Facebook
CodePlex
github
LinkedIn
Google