When I initially picked up Common Lisp my intention was to rewrite TyNET in it, with the focus on making a properly usable and extensible web framework that other people could profit from as well. And while that did indeed happen to an extent it was not satisfying to me and I felt like the old approach of just going at it would not yield respectable results.
After I finished my last Radiance related project (Plaster), I felt it was time to freeze any development on it and get started on writing a full specification. This was partly inspired by my realisation of how great and useful the Common Lisp Hyperspec was and by the simple fact that a good documentation both serves as a reference and as an escape route of sorts, since any user won't be stuck with a single implementation.
Another benefit of writing a specification is that you can figure out and solve a lot of design and structural problems beforehand. You won't be stuck in the same loop of implementing a feature, later realising it's bad, changing it, and then having to go through all the code that relies on it to fix that up again. Of course, writing a spec yields other problems such as missing extensions, bad foresight in how features would be used and so on. Either way, I felt like writing up a spec would be the right thing to do, especially as I already had a great deal of experience with the intended product and Common Lisp.
However, writing a spec on something is a difficult and long-winded task, especially when the project is as huge as Radiance is. Radiance is not simply a single library, it's a collection of libraries, technologies and programs that form a complete stack from a web-framework toolkit through a standardised framework implementation up to actual web-services. Hampered by the scale of this undertaking and distracted by other interests I did not do anything for quite a long time.
Recently however I've returned back to this and started designing things from the core out. As soon as I began, a multitude of questions and possible problems popped up and the more I answered and fixed, the further away from my initial prototype the design went. Sure, some main ideas and elements remain still, but the circumstances are wildly different now.
I still haven't finished working through the core library, but there isn't all too much left to design. Once this phase is done the specification process can begin and I'll have to find a way to precisely formalise my thoughts and ideas such that anyone, and not just me, could write an implementation that could support the entire stack that is built on top of it. Just like the hyperspec (not counting the places where implementations do differ).
Right now the core Radiance lib has been split into four parts. The first part is a framework that enables modularisation. That is to say, it provides mechanisms that help you build a project with different components split into modules. The core lib itself will be declared as such a module. The three remaining parts are declared as module extensions to the core. The first of these adds the interface mechanism that allows coders to specify interfaces that other modules can implement, depend on, and use without having to know what module in specific provides the functionality (a form of specification in-code, I guess). While interfaces enable a form of interaction that is necessary with minimal information, the second adds a hooks and triggers system that allows specific interaction between modules for tight interoperability and extension. The last extension adds what's necessary for it to become web-centric, a system that handles request dispatch and allows the distribution and registration of url-address-space to the differing modules. The web extension also provides a list of standard interfaces that a basic Radiance stack is built on top of.
These standard interfaces will be a specification process of their own. Mostly the focus will be on database interaction, webserver handling, authentication, and session/user handling. However, this is just a base laid out to provide a convenient framework to build web-apps with. It is extensible in every regard; modules can define their own interfaces, interfaces themselves can be extended with additional functionality, modules can act as libraries to build more complex services or provide more convenient abstractions, and so on. I'm trying my best to design every part of this core system to be as extensible and flexible as possible so that any part could be easily adapted into whatever form is required.
Another effort that came into play as I designed this and which ultimately resulted in the ‘core extensions’ model was that a lot of the parts of this system could be useful in a context outside of a web framework, especially the interfaces and triggers systems. Hopefully these libraries will be useful to someone that doesn't care about web development too.
For now I'll get back to designing and if all goes well I'll soon be ready to publish a first specification draft of the Radiance core module lib.
Written by shinmera