Building TyNETv5 Pt. 8 - Initialize!

2013.07.01 22:17:40

image About high time I write a blog post about this stuff again. Apologies in advance if I sound angry or in any way derogatory about things, I'm currently not in a very preferable state of mind. Still, Imma do this because then I'll at least have done something today. So, what's this all about then?

Well, after having finished lQuery, the next step was to finally get started on Radiance proper. And the first thing to that was to build the main framework infrastructure and the INIT sequence to be able to launch the entire thing. This is all done now and I'm here to tell you all about it.

I've changed things around a couple of times as I was working on it, but this is the setup I've ended up with now: There's two parts to Radiance's core, which is radiance-lib-core and radiance-mod-core. Compiling and loading everything is tied together with a couple of ASD files. To start a server, one has to simply load radiance (ql:quickload :radiance), invoke (radiance:manage #'radiance:start) and presto, a webserver should be started. Configuration of the server is set with the radiance.json file in the base directory. There's currently no support for multiple ports or multiple server instances, but that might be a thing I add at some point. Especially multiple ports could be useful for dedicated applications.

Radiance-lib-core is a very basic library that contains all required functions to… create modules, hooks and implementations, which are the essential parts of Radiance. Modules allow you to create components that encapsulate individual services. Hooks allow you to create abstract interfaces between services. Implementations allow for standardized and controlled .. well, implementations of various features. I'll go through this in a bit more detail in a second.

Radiance-mod-core contains the core “program” parts of Radiance. This includes standard interface definitions, the flash-dispatcher and the kickstart-dispatcher, a mongodb implementation and server management functions. At the time of writing, everything except for the kickstart-dispatcher and mongodb interface have been built and work fine. Now let's take a closer look at everything from ground up.

The main mechanic has always been the modules. This isn't any different in v5. Creating a new module is done through the defmodule macro, which requires you to specify a name, superclass list, docstring, metadata plist and optionally additional fields for the class. The superclass list is by default the standard module class, but this can be extended for other classes. This is for example used in the implementation interfaces. Docstring is a mandatory documentation string. I don't quite see the point in making that optional, as documentation should always exist even to a minimal degree. Now the metadata plist is the most interesting thing. It is basically a map of all module standard fields, such as fully qualified name, author, version, homepage, license, but also things that are useful for introspection, such as dependencies, collections (database), callables and if it's persistent or not.

Of these metadata options, none actually do anything yet. I will have to remove the dependencies option, as it does not have any purpose anymore with my change of plans. It make no sense as all modules are loaded and instantiated only once when radiance is loaded. As such, there's no dependencies to load because everything is always loaded. Persistent is not in place yet, but is no difficult extension to make to INIT, and collections and callables only make sense for other modules to use for introspection. Collections is a list of collection instances. Collection is a class that contains the name of the database collection, a UNIX access mode, a list of columns and an optional description. Columns are the same as collections except for the lack of a columns field. The UNIX access mode is used to make sure that outside modules can understand which data in the database should be public or accessible to whom. This whole ordeal also allows one to write abstract modules that can parse pretty much any data available in the database and output it properly without any security concerns.

The trigger system is quite simple. There's only two functions to worry about, the first being defhook and the second being trigger. Defhook expects a name, module and function. It then simply maps the name to the module and its function. Once a trigger call is placed on that name, the function is invoked with the module as argument, as well as any additional arguments passed to trigger. Simple and powerful.

Even more powerful and neat is the whole thing if implementations are taken into account. Implementations work with three main functions. The first being defimpl, which allows you to create an implementation interface. A call to defimpl expects an implementation name and a list of function instructions. A function instruction is simply a list with name and possible arguments to the function. In the background defimpl expands into a class definition and a generic and method definition. The class definition is a simple no-slot class inheriting from module with the implementation's name. The methods themselves are stubs that throw an error if called. This is a very simple mechanism to ensure that implementations always provide all necessary functions of an interface. The next thing defimpl (or rather, defimplclass) does is define two more methods. The first being a method specific to this exact implementation name and any module. This method always throws an error. In effect, this ensures that no module can provide an implementation if it is not a subclass of the defined implementation class. The second method (which is specific to the exact implementation and any module that is subclassed under the impl) then simply sets the provided module as the implementation for the name.

If one wants to implement a certain interface with a module, they need to extend that defined class and then call implement on the implementation name and the module instance to register it. The last function in the impls series is simply “implementation”, which returns the module object for a given implementation name.

…I don't think I've ever used the word “implementation” this often before.

And that's pretty much all there is to the core libraries. Now quickly on to the core program: Flash-dispatch is the first dispatcher I've talked about. It uses a static lookup table to decide which hooks to call. It implements the dispatcher interface and has therefore only two very simple functions: dispatch and register. Dispatch expects a request object as an argument and determines the trigger to call based on the subdomain. Register takes a hook name and optionally a subdomain to register that hook for. So it's all very simple key→value stuff.

The last thing in the series is the server functions. These allow you to start, stop, restart and reload Radiance, and they handle the most basic interaction with Hunchentoot. It also serves as an up-front parser of the request object to provide often used variables, such as domain, subdomain, port and path. The manage function is the main part of it and is a simple wrapper function around the main start, stop and restart functions. It makes sure that the provided values are correct and then just calls the given function. Start initializes the hunchentoot stuff such as acceptor, dispatch table and server start. Stop shuts hunchentoot down and resets all runtime values. The most interesting function in that set though is the handler, which extracts the request information and puts it into a new object called radiance-request. This object is then passed to the dispatcher implementation. Finally it returns either the result of the dispatch call or the value of “result” in the request object to Hunchentoot so that it may be forwarded to the end user.


Obviously there's still a lot to be done. I have not looked into mongo-db integration much, haven't done anything about upload handling nor static file serving. These are all core features that need to exist before I can start making actual modules. But, I'm coming along steadily and I'm constantly surprised by the really simple and elegant solutions I find with Common Lisp that would not be as easily possible in any other language I know, if at all.

Next up: More core features. image

Written by shinmera