Chris Umbel

Windows Services in Python

Any more it seems that virtually all the code I write on the windows platform ends up being a windows service. It's just the nature of the kind of work I do: the underappreciated guts that sit far beneath the software that users directly interact with. Obviously windows services are typically written in mainstream .Net languages like C# or VB these days but it's also easy if not easier to do in Python.

Requirements

  • A Python Interpreter - this one's pretty obvious. Can't run Python code without a Python interpreter.
  • Python for Windows Extensions - this is a wonderful, easy to install project that exposes the innards of windows to Python.
  • Administrative access - you must be logged in with administrative access in order to install your service.

The Code

Once your Python environment meets the requirements outlined above you're ready to write your service. All you have to do is write an extension of the ServiceFramework class that overrides the SvcDoRun and SvcStop methods. As you can guess SvcDoRun performs the core logic of your service and SvcStop shuts it down.

I've created a sample service below that simply writes some text to a file every five seconds until the service is stopped. Note that this example uses windows signal events to trigger service shutdown via the CreateEvent, WaitForSingleObject and SetEvent windows API calls. Thank you Windows Extensions!

import win32service
import win32serviceutil
import win32event

class PySvc(win32serviceutil.ServiceFramework):
    # you can NET START/STOP the service by the following name
    _svc_name_ = "PySvc"
    # this text shows up as the service name in the Service
    # Control Manager (SCM)
    _svc_display_name_ = "Python Test Service"
    # this text shows up as the description in the SCM
    _svc_description_ = "This service writes stuff to a file"
    
    def __init__(self, args):
        win32serviceutil.ServiceFramework.__init__(self,args)
        # create an event to listen for stop requests on
        self.hWaitStop = win32event.CreateEvent(None, 0, 0, None)
    
    # core logic of the service   
    def SvcDoRun(self):
        import servicemanager
        
        f = open('test.dat', 'w+')
        rc = None
        
        # if the stop event hasn't been fired keep looping
        while rc != win32event.WAIT_OBJECT_0:
            f.write('TEST DATA\n')
            f.flush()
            # block for 5 seconds and listen for a stop event
            rc = win32event.WaitForSingleObject(self.hWaitStop, 5000)
            
        f.write('SHUTTING DOWN\n')
        f.close()
    
    # called when we're being shut down    
    def SvcStop(self):
        # tell the SCM we're shutting down
        self.ReportServiceStatus(win32service.SERVICE_STOP_PENDING)
        # fire the stop event
        win32event.SetEvent(self.hWaitStop)
        
if __name__ == '__main__':
    win32serviceutil.HandleCommandLine(PySvc)

Service Installation

Now that the service itself is written we can install it as follows:

C:\Dev\Projects\PySvc> python.exe .\PySvc.py install

Service Control

That's it! The service can now be started from the command line by

C:\Dev\Projects\PySvc> NET START PySvc

or from the Service Control Manager

Mon Nov 02 2009 20:11:00 GMT+0000 (UTC)

Comments

My Tool List

One of the more popular thing's Scott Hanselman has done is maintain a tool List, essentially a list of applications and utilities that he's found useful. There are plenty of absolute gems in his list and rather than present my own I'd be very comfortable just pointing you to his... But what fun would that be? None! I also think there are a few more that I can add that are relevant to my personal experience.

The list is rather short and there's a bit of overlap with Scott's but if all goes as planned I'll expand it over time as old useful tools come to mind or I discover new ones.

Cloud

  • iAWSManager - This is an iPhone app that lets you manage many of the functions of Amazon Web Services such as S3, SimpleDB and EC2.

  • Rackspace Cloud Servers iPhone App - Here's an iPhone app that allows you to deploy and manage virtual servers hosted on Rackspace Cloud Servers.

  • Elasticfox - This is a firefox plugin that facilitates full management of Amazon's EC2 within firefox.

  • S3Fox Organizer - This is a firefox plugin that facilitates full management of Amazon's S3 within firefox.

  • SimpleDBExtension - OK, here's some shameless self promotion. SimpleDBExtension is a SQL Server Reporting Services data processing extension that facilitates using Amazon's SimpleDB as a data source. It's written by yours truly and is released under the MIT License.

  • SDB Tool - Here's a FireFox plugin that is like the query analyzer for Amazon's SimpleDB.

Editors

  • KomodoEdit - This is a great, multiplatform (Linux, Windows and Mac) and free text editor that's specifically targeted at dynamic language development. It's developed by ActiveState, a company who's bread and butter is dynamic languages.

  • jed - This is the text-mode editor I've been using under unixes for a LONG time. It's compatible with emacs commands, has syntax highlighting and friendly pull-down menus.

System

  • Virtual CloneDrive - This let's you mount ISO images in Windows and is compatible with Windows 7 64-bit. The price is right too... It's free.

  • Sun VirtualBox - VirtualBox is an open source desktop virtualization system (similar to VMWare Workstation) developed by Sun Microsystems. It's multi-platform, feature-rich and free.

  • Cygwin - This is one of my all time favorites. It's essentially a mini unix-like environment that runs under windows as an application. It's not a VM, more like unix shell for windows. It boasts an X server and gives you access to all the goodies unix guys are used to like sed, awk and grep.

  • putty - The defacto Telnet/SSH client for windows.

  • PowerShell - For years windows suffered from having an infirior shell in cmd.exe. Those days are decidedly gone with PowerShell. Although it's been around a while now it seems that broad adoption has only begun to happen. It's basically an operating system shell with full .Net Framework integration. Easily the most powerful shell ever invented. If you haven't used it before be prepared to read and experiment, however. There is a learning curve.

Development

  • NDepend - Here is a very impressive refactoring and code management tool for .Net. My favorite feature is it's Code Query Language (CQL) which allows a programmer to answer questions about their code with a programmer-friendly SQL like syntax.

  • DB40 - This is an object oriented database for Java and .Net. A shockingly easy way to persist data.

  • Modify Headers - Firefox plugin that allows you to manually set HTTP headers.

  • Reflector - .Net assembly analysis tool by Red Gate with Visual Studio integration.

  • ComponentOne Controls - Control suites for Win Forms, WPF, ASP.Net, Silverlight, iPhone and more.

  • MongoHub - Native Mac GUI management tool for MongoDB.

Graphics/Video

  • GIMP - GIMP is basically the open source answer to photoshop. It's quite powerful but a left-brainer like myself uses it more as a utility rather than a creative outlet. Still it solves all my image manipulation troubles across all the platforms I use.

  • MediaInfo - Handy, cross platform utility that can be used to extract details about the encoding of various media formats/containers/codecs.

Mon Oct 26 2009 20:10:00 GMT+0000 (UTC)

Comments

Groovy: Dynamic Language for the JVM... Groovy!

I'm continuously encouraged by the influence dynamic languages such as Ruby and Python have had on mainstream runtimes like the CLR and JVM. Direct ports like JRuby, Jython to the JVM and IronRuby and IronPython to the CLR are truly exciting. More exciting still are languages like Boo that are built from the ground up for mainstream runtimes.

I've finally had the chance to fool around with a language I've been dying to dig in to that was built specifically for the JVM: Groovy.

Groovy is an agile, dynamically-typed, well supported language that is perhaps most famous as the basis for Groovy on Grails, a rails-like web development framework.

Example

As I typically do I'll show you a quick twitter status example to get your feet wet. Consider the following class written in Groovy:

import java.io.BufferedReader
import java.io.IOException
import java.io.InputStreamReader
import java.net.URL
import javax.xml.xpath.*
import javax.xml.parsers.*

class TwitterUser {
    /*  definition of UserName property.  public accessors and mutators
        are automatically created. */
    def UserName

    /* constructor definition */
    def TwitterUser(userName) {
        this.UserName = userName
    }

    /* public method to query twitter for the users status */
    def getStatus()
    {
        /* pretty much just java code sans the types and semicolons */
        def builder = DocumentBuilderFactory.newInstance().newDocumentBuilder()
        def document = builder.parse("http://twitter.com/users/${UserName}.xml")
        def xpath = XPathFactory.newInstance().newXPath()
        return xpath.evaluate( '/user/status/text', document,
            XPathConstants.STRING)
    }
}

This Groovy class can now be consumed by Java very easily:

public class Main {
    public static void main(String[] args) {
        TwitterUser status = new TwitterUser("chrisumbel");
        /* see the accessor we didn't create? */
        System.out.println(status.getStatus());
    }
}

Conciseness

That's all fine and dandy, but it's practically Java. Consider the following ruby-like block which defines a list and iterates it:

[2.5, "yo", 5].each() {
    /* "it" is the current item */
    println it
}

Ranges are also supported:

(0..3).each { i ->
    println i
}

Ruby/Python-ish indexing as well:

def list = [2, 4, 6, 8]
println list[1..2]

which produces:

[4, 6]

and

def words = "Hack off a word"
println words[0..-6]

resulting in:

Hack off a

Nice maps too:

def map = ["name" : "Chris Umbel", "age" : 30]
println map["age"]

Real Life

Considering it fits right into NetBeans Groovy is an incredibly accessible language. Having a first class IDE never hurts. The fact that groovy classes can sit in the same project with Java classes removes pretty much all barriers (and excuses). The nature of the runtime and the tooling proves that Groovy is ready for prime time.

Fri Oct 23 2009 11:39:00 GMT+0000 (UTC)

Comments

Easy Concurrency with Stackless Python

Since messing around with Erlang over the last couple months I've been very impressed with it's simplicity in managing concurrency. Erlang's message passing infrastructure really allows developers to have the locking just work without doing anything unnatural in code for synchronization. You almost let the problems define the locking behavior for you.

Such elegance isn't limited Erlang, oh no. Python's also been shown some concurrency-love by way of a specialized implementation called Stackless Python.

Stackless Python avoids using C's call stack and relies on lightweight microthreads rather than operating system threads or processes for concurrency. Channels (similar to Erlang's message passing) are used for synchronization and communication between tasklets (the implementation of microthreads).

Note that stackless python is a cooperative multitasking framework so you do have the responsibility of explicitly relinquishing control back to the thread scheduler.

See the small demonstration below.

#!/usr/bin/env python
import stackless

log_file = open('log.txt', 'w+')

# function to perform a CPU bound process
def worker(log_channel, address):
    while True:
        results = [perform CPU bound logic here]
        
        # yield to other threads
        stackless.schedule()
        
        results += [maybe do some more CPU bound logic here]

        # queue up a message containing the results
        # of the computation
        log_channel.send(results)
        
# function to log results of pings
def logger(log_channel):
    while True:
        # block until a message is recieved
        message = log_channel.receive()
        # write message to disk
        log_file.write(message)

# create a channel for inter-tasklet messaging
log_channel=stackless.channel()

# fire up two worker tasklets
worker1 = stackless.tasklet(worker)(log_channel)
worker2 = stackless.tasklet(worker)(log_channel)

# fire up a tasklet to recieve messages from the workers
listener = stackless.tasklet(logger)(log_channel)

# main loop
stackless.run()

log_file .close()

A lot like Erlang's threading, no? This is clearly simpler than python's native threading and has far less overhead.

Note that this stuff is used in real life by various video game deployments such as EVE Online and the Sylphis3d game engine. You now, the kind of people who really know concurrency.

Sat Oct 03 2009 21:11:00 GMT+0000 (UTC)

Comments
< 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 >
Follow Chris
RSS Feed
Twitter
Facebook
CodePlex
github
LinkedIn
Google