Sunday, July 8, 2012

Last week’s post was only a prelude to what I really wanted to show you is possible using Go. Something that I call the Mach programming model.

In the Mach microkernel, core operating system functions such as memory management, disk management, file system handling, device management, and the like run as separate tasks. These tasks communicate with each other by sending messages to ports. The Go programming language uses a similar concept, and allows goroutines to communicate through channels. This allows you to write programs that work much alike the Mach microkernel.

The basic idea is that you break your program up into services. A service manages a resource. The service is being run by a manager. You can get service by issuing a request. The request is communicated through a port, or in the case of Go, a channel.

const (
    DoSomething = iota
    DoSomethingElse
    DoSomethingNice
    DoSomethingCool
)

type Message struct {
    Code  int
    Args  []interface{}
    Reply chan *Message
}

func DoRequest(manager chan *Message, code int, args ...interface{}) (reply *Message) {
    reqArgs := make([]interface{}, 0)
    for _, a := range args {
        reqArgs = append(reqArgs, a)
    }
    replyChan := make(chan *Message)
    manager <- &Message{code, reqArgs, replyChan}
    reply := <-replyChan
    return
}

func ManagerGoRoutine(in chan *Message) {
    for {
        req := <-in

        switch req.Code {
            // handle request
            // ...
        }

        req.Reply <- &answer
    }
}

So, what is going on here? A request is a message that has a request code, which is a constant (that is easily enumerated using iota). Additionally, the request may have a variable number of arguments. Arguments can be of any type you like. The request also includes a reply channel, on which the requestor will receive an answer. The request is written to the service manager’s channel. All the manager does is sit in an infinite loop in its main goroutine answering requests. It sends the answer to the Reply channel that is in the request. The reply is also of type Message, so that it can have all kinds of replies through the Args field.

Try doing this in plain C—it’s hard. In Go, you practically get everything for free. In good old C you can do this using pthreads and pipes. Pipes aren’t nearly as easy to use as channels. And that trick we pulled with the request’s arguments, that’s near impossible to do in C. Sure there is va_list but it’s limited; you can’t make an array with varying typed elements in C.
As a solution in C you might pass the address of the va_list through a pipe, which is scary because the va_list points into the stack memory of the requesting thread. It’s a recipe for stack corruption. Now, because the requesting thread immediately blocks and waits for a reply, this just might work, but it’s hackish. In Go however, the code is easy and clean, and note that all of this is possible even though it’s a strictly typed language.

In the above code, the manager switches on a request code. You might as well include a function pointer in the request and call that. Now the question rises, why not call the function directly anyway? You would have to use resource locking, and things would work the ‘traditional’ way.
The answer is that the Mach programming model evades the traditional approach on purpose. It is another way of looking at large codes in which a variety of resources are managed. It’s a microkernel design rather than a monolithic one. It models the program flow as an information flow.
This different way of thinking gives a level of abstraction that leads to easier to understand code, better code maintainability, and (hopefully) fewer bugs.

Ultimately, the Mach kernel for operating systems was considered a failure because it yields lower performance than its monolithic counterpart. Nevertheless it remains an interesting concept and you might as well use it in places where you don’t need that ultra-high performance. You can couple this model with asynchronous networking code and then it really starts to shine.
What used to be hard in C, in Go you get for free.