I can't deny that one of the things I love is taking on large tasks. Big projects have a lot to offer and promise a much more interesting time than smaller ones do. This leads me to mostly take on things that are much too large for me. One of the consequences of this is that I hardly ever really finish anything. The other however is much more problematic.
Small projects are easy to manage. There's a few straightforward things you want to achieve, so you quickly put together in what order they need to fit and then follow that order. Of course, there'll be changes and adaptions along the way, but it is nevertheless rather simple. The more the complexity and size of a project increases though, this path becomes less and less clear. Often enough, the only thing you know are certain stepping stones and main pillars that need to be finished, but the connection between the the current thing you're working on and the next is not obvious.
As you might have picked up by previous blog entries in this series, this too is a frequent problem for me. I know the approximate checkpoints I need to reach and I know my final goals, but what route through the project I should take is quite difficult for me to figure out. There's just too many possible things I could improve and work on, so many parts I should put more effort into but don't want to and so many that I would love to start, but can't because they aren't important enough at the time.
Being able to focus on what's important right here and now is a very valuable skill or trait that I sadly have not trained very well. It happens all too frequently that I sidestep the issues at hand for something else that I enjoy doing, which simply leads me to gradually abandon the project I got stuck on. This in turn means I get depressed about having yet another unfinished project on my record.
Given that I am writing this right now probably already made you realise that I am once more stuck and don't know where to go next. In this particular scenario, I am left with a couple of choices none of which I really want to follow. I will go through those one by one, hopefully writing this up will either make me realise the right choice by myself or someone else will be able to point it out to me.
Option one. This option arises due to a problem that I anticipated earlier but hoped I would not encounter. It has to do with a performance issue I have with the way I'm constructing pages. In order to avoid the HTML templating problem, I created lQuery, which allows me to dynamically manipulate HTML documents without having to rely on special tags or control constructs. This is a very, very nice thing that I think should be the only real way to build pages. It neatly separates programming logic and template, just the same as HTML and CSS separate layout and content (for the most part). The way I manipulate the document is by using CSS selectors and then modifying DOM nodes that match this. Sadly it appears that performing a CSS select is rather slow and a high amount of these selects leads to almost unbearable performance. I'm talking 0.045s for a rather standard page with nested iteration. This might not seem bad to you, but it is to me as it could very well mean even worse performance on sites that also include database interaction and a generally more complex page structure.
The core of the issue lies within the iteration here. I'm using a toolbox that allows me to automatically fill data into DOM nodes according to node attributes. This tool selects a “root” node out of the document that is then copied n amount of times and for each copy the data from the associated data model is filled in. Sadly each copy also requires a number of repeated CSS selects. With a lot of iteration and a lot of data, this leads to a lot of selects. This is especially stupid because the structure of each copy is always exactly the same, meaning every select leads to the exact same nodes in the position of the DOM tree. This could be sped up by a large factor by simply caching the position of the selected nodes.
Another issue are the CSS selectors themselves. The way they are constructed is almost inherently slow. The amount of queries performed on each node can be astronomical if the selector is even a bit complicated. In order to get rid of all these issues I would have to write my own DOM tree querying engine, which could take me a couple of weeks to do and even if I did there is no promise that it will lead to better results. I simply can't trust myself to be able to pull off enough wizardry to make this truly fast. So this option is risky because it could mean spending a lot of time without necessarily solving the problem.
Option two. Quite simply put this option would be to further develop the framework, make it easier to use, less complex, faster and add more functions to its toolchain to aid website construction. This could lead to a much better product overall and it could make it much easier to do what I did all of this for: Making sites. The problem here is that I don't quite know yet what would really be worth to put my focus on. There's still a lot of places that seem rough, unfinished or improvable. And I am certain that there's also a lot of things still blatantly missing. Not knowing what is necessary could be solved by option three.
Option three is to write an actual site already. I really want to have at least some form of end-product to show off before I go to University and have to lay off this project for a while. The issue here is intertwined with option two. While I could certainly start working on a site now, there's bound to be things along the way that I will have to rework about the framework, which in turn means I'd have to remake these parts of the site. I already had to go through with this in the verify module, which was a great deal of pain. To be more literal about it, I am afraid that the framework is not mature enough yet and starting something serious now would lead me to too much back-and-forth. Simultaneously though it is really hard to figure out what needs to be changed, added or removed without actually having a specific project to put this into relation to.
This is accompanied by the issue that a lot of essential parts of the site (such as user profiles and settings, administration and registration) are either not built yet at all or largely unfinished. Making an actual site without these rather vital components ready at hand is very risky as integrating these and achieving good interoperability is of vital importance to maintainability. Not to mention that the site would be practically unusable for production without proper user interfaces for settings and so on.
All in all I am still rather clueless as to what to do. #1 seems like it isn't of vital importance right now. Optimisation gambling can still be done later and my sites aren't popular enough to crash my servers, even if some page calls are a bit slow. I also still cannot really benchmark this as I do not have any actual, real complex pages to test it with. #2 seems like the sensible choice, but as I mentioned I am rather lost as to what to really work on or how to improve it. The best way to get to this would be through code review, but I am too scared of that and don't really have anyone I do trust who'd have time or who would be willing to look through it (if you are, do please let me know, I would be forever thankful). #3 is again really risky as, while it would potentially produce an actual thing to show, it means I'd have to take a lot of guesses and chances that might screw me over royally later.
I think I'll mostly have to do some code review myself and try to imagine more use cases so I can employ my interface-first methodology to figure out what to change about the framework and how. I will most likely write up another blog entry rather soon to count up all the issues and things I encountered. I promised in the last blog to talk about the dispatch system, but as that is one of the issue spots, I suppose I will briefly touch on it in the next one.
Written by shinmera