Using CL+Qt - Confession 49

2015.02.12 22:33:19

header Some deem it unfortunate, others are not bothered by it at all, but the fact remains that Common Lisp does not have a standard GUI toolkit. It does have a native toolkit called McCLIM, but due to general outdated-ness it is not a very attractive choice. Generally I'm not one to linger long on decisions when it comes to learning something, so after quickly evaluating the options I chose to try CommonQt, a library to allow using the Qt framework with CL.

The first thing I wrote with it was a primitive GUI for a chat client, but while I did finish it, I never went far with it. That is, until Parasol came along. Parasol makes heavy use of Qt, and unfortunately working with CommonQt forces you to write in a rather un-lispy style. This isn't surprising, since Qt itself is a C++ framework and thus matching idioms probably isn't as easy.

Fortunately for us, CommonQt already goes a long way of bridging the gap, but not quite far enough. In an effort to bring GUI writing with Qt closer to home, I created Qtools. In this entry we're going to make use of CommonQt and Qtools to show off what writing a basic GUI in CL can look like.

What we're going to do for this mini project here is write a primitive Twitter client. It'll have a dialog to let users log in via twitter and a main window to display new statuses, as well as let you post some. To make this all possible we'll make use of Chirp and the aforementioned Qtools. In order to understand this tutorial you'll need a moderate understanding of Common Lisp, some prior knowledge of UI programming, and a lack of fear to look things up in the hyperspec, Qt docs, and other documentation. Let's get to it.

(ql:quickload '(:chirp :qtools))

This month (February 2015), you'll want to get Qtools from git (version 0.4.2+) as the Quicklisp version is too outdated. In case the CommonQt loading fails, refer to the CommonQt homepage.

Now, as usual we'll create a new package for ourselves to live in.

(defpackage #:titter
  (:use #:cl+qt)
  (:export #:main))
(in-package #:titter)
(named-readtables:in-readtable :qtools)

Here you'll notice two deviations from the norm. First, we're not :use-ing the standard CL package, but rather CL+QT, which is a package from Qtools that provides convenient access to CL as well as Qt functionality. Second, we need the in-readtable statement to make use of CommonQt's reader extension for Qt methods.

Now we'll finally start with writing our own UI. Defining top-level widgets happens with define-widget, which exactly mirrors defclass, with the exception of some extensions that are irrelevant for this tutorial.

(define-widget login (QDialog)

This will be our dialog to log in with. You can already test it now, but you won't get much beyond a blank window.

(with-main-window (w (make-instance 'login)))

Time to get on to the meat of a widget, its contents. Logging in to twitter can't happen via password anymore unless you get special permission from twitter to do so. We'll instead use twitter's oAuth PIN method. To give that to the user, we'll need to show them a link, let them type in a PIN and have a button to confirm or something.

(define-subwidget (login url) (#_new QLabel login)
  (#_setTextFormat url (#_Qt::RichText))
  (#_setTextInteractionFlags url (#_Qt::TextBrowserInteraction))
  (#_setOpenExternalLinks url T))

That's quite a few new things here so let's go through them. define-subwidget as you probably expect defines a widget on our login widget, called url. This initializes to a QLabel instance with our main widget set as parent. #_new is the CommonQt equivalent to the new operator in C++. While widgets defined on the CL side need to be initialised as usual using make-instance, Qt-native classes need to be instantiated using #_new. Next in the body we set a couple of properties of our label using C++ methods with the #_ reader macro. Make sure to type the method names in their exact case or CommonQt won't be able to find them. These property changes are necessary to allow clickable URLs.

Don't launch your widget quite yet or you'll be disappointed to find it as bleak and empty as before. We'll get to that in a minute, but first let's define the rest of our components real quick.

(define-subwidget (login pin) (#_new QLineEdit login)
  (#_setPlaceholderText pin "PIN"))

(define-subwidget (login go) (#_new QPushButton "Login" login))

Alright, that was easy. Now, the subwidgets won't appear on your main widget magically as the system could not have any idea how you want them to be placed. For this we need layouts.

(define-subwidget (login layout) (#_new QVBoxLayout login)
  (#_setWindowTitle login "Login to Twitter")
  (#_addWidget layout url)
  (let ((inner (#_new QHBoxLayout)))
    (#_addWidget inner pin)
    (#_addWidget inner go)
    (#_addLayout layout inner)))

Rather simple layout stuff by GUI standards. A vertically oriented layout to hold our label and a horizontal layout that holds the PIN text field and button. Now you may launch your widget again and marvel at the impressively unexciting UI.

In order to make things react in Qt you need to employ their system of slots and signals. Slots are signal receptors and signals are identifiers as well as data-carriers for events. So, when a button gets clicked a signal is fired. Whatever slot is connected to the button on that signal then gets called with the signal properties for arguments. Since we have a button in our form, let's make a slot for it.

(define-slot (login done) ()
  (declare (connected go (released)))
  (#_QMessageBox::information login "OOoOo" \(°_o)/¯"))

What we've done here is defined a slot on our widget called done, which takes no arguments and is connected to the go button's released signal (which provides no properties). You'll notice here that Qtools uses declarations like a sly fox in order to make things a bit easier and lispier. Firing up the widget now will already give you the expected effect.

This is all good and well, but it has rather little to do with Twitter, so we'll change that. First, we need to fetch the URL to have the user authenticate with and display it on the label.

(defun set-url (widget)
  (let ((url (chirp:initiate-authentication
              :api-key "D1pMCK17gI10bQ6orBPS0w"
              :api-secret "BfkvKNRRMoBPkEtDYAAOPW4s2G9U8Z7u3KAf0dBUA")))
    (#_setText widget (format NIL "Please enter the pin from <a href=\"~a\">twitter</a>." url))))

Then we need to change our login slot definition to actually make use of this function.

(define-subwidget (login url) (#_new QLabel login)
  (#_setTextFormat url (#_Qt::RichText))
  (#_setTextInteractionFlags url (#_Qt::TextBrowserInteraction))
  (#_setOpenExternalLinks url T)
  (set-url url))

But, we're only half-way there. We still need to actually evaluate the PIN that the user passes back to get the proper authentication credentials. We'll do that in our done slot.

(defvar *logged-in* NIL)

(define-slot (login done) ()
  (declare (connected go (released)))
  (setf *logged-in* NIL)
  (#_setCursor login (#_new QCursor (#_Qt::WaitCursor)))
      (chirp:complete-authentication (#_text pin))
    (error (err)
      (#_QMessageBox::critical login "Error!" "Failed to login.")
      (#_setText pin "")
      (set-url url)
      (#_setCursor login (#_new QCursor (#_Qt::ArrowCursor))))
    (:no-error (&rest args)
      (declare (ignore args))
      (setf *logged-in* T)
      (#_close login))))

So, what happens here? First we have a variable to keep track of the login status and then we do some cursor displaying to let the user know that stuff is happening the back. Next we have error handling in case our authentication fails for some reason, which just resets things to let the user try again. However, if we succeed the widget closes itself and thus returns. To verify that everything logged in smoothly after you've tried it, you can use


So in, little under 50 lines we wrote a complete login dialog for our application. While we're fired up like that, let's move on to writing the actual client. We'll want a field to type new status updates into, a button to submit the tweet, and a list to hold new tweets from our home timeline.

(define-widget client (QWidget)

(define-subwidget (client status) (#_new QLineEdit client)
  (#_setPlaceholderText status "What's old?.."))

(define-subwidget (client tweet) (#_new QPushButton "Tweet!" client))

(define-subwidget (client timeline) (#_new QListWidget client)
  (#_setWordWrap timeline T)
  (#_setTextElideMode timeline (#_Qt::ElideNone)))

(define-subwidget (client layout) (#_new QVBoxLayout client)
  (#_setWindowTitle client "Titter")
  (let ((inner (#_new QHBoxLayout)))
    (#_addWidget inner status)
    (#_addWidget inner tweet)
    (#_addLayout layout inner))
  (#_addWidget layout timeline))

Mostly similar to what we had before, modulo widgets and properties. Now we need another big function to take care of submitting a tweet. This happens as before in a slot since we need to handle a button press.

(define-slot (client tweet) ()
  (declare (connected tweet (released)))
  (cond ((<= 1 (chirp:compute-status-length (#_text status)) 140)
         (#_setCursor client (#_new QCursor (#_Qt::WaitCursor)))
             (chirp:statuses/update (#_text status))
           (error (err)
             (#_QMessageBox::critical client "Error!" (format NIL "Failed to tweet: ~a" err)))
           (:no-error (&rest args)
             (declare (ignore args))
             (#_setText status "")))
         (#_setCursor client (#_new QCursor (#_Qt::ArrowCursor))))
         (#_QMessageBox::information client "Huh?" "Tweet must be between 1 and 140 characters long!"))))

Here we have a simple check to make sure the status has the allowed length (chirp takes care of URLs for us), sends out a new status update, and handles the potential errors. Simple, verbose stuff. Looking at our main window now

(with-main-window (w (make-instance 'client)))

We'll be able to send tweets, but nothing appears in the list. For that we need to cast some more advanced spells. To handle adding new items to our list we'll define our own signal and slot.

(define-signal (client new-tweet) (string string))

(define-slot (client new-tweet) ((user string) (status-text string))
  (declare (connected client (new-tweet string string)))
  (format T "~&Got new tweet from ~a: ~s" user status-text)
  (#_addItem timeline (format NIL "@~a: ~a" user status-text)))

As you can see, the signal definition holds a type argument list. We'll want to transmit the username and the status text and connect the slot to the widget itself. We'll use that to emit the signal once we get new tweets.

Since the main thread will be occupied with the UI we need to launch an additional thread to take care of incoming tweets. However, we also need to make sure that the thread shuts down with the UI as well and only launches after the UI is already available. To do this we'll define a general launch function.

(defun main ()
  (let ((thread))
    (with-main-window (w (make-instance 'client))
      (setf thread
             #'(lambda ()
                  :user #'(lambda (message)
                            (when thread
                              (process-message message w) T)))
                 (format T "~&Shutting down tweet stream"))
             :initial-bindings  `((*standard-output* . ,*standard-output*)))))
    (setf thread NIL)))

Aside from the with-main-window form, the guts here is the start-stream chirp function which will handle stream communication for us for as long as messages come through and our handler function returns with a non-NIL value. Thus we can check for thread termination and let everything clean up nicely once the UI exits. However, this makes use of one function we haven't defined yet, process-message. Let's change that.

(defun process-message (message client)
  (format T "~&Message: ~a" message)
  (when (typep message 'chirp:status)
    (signal! client (new-tweet string string)
             (chirp:screen-name (chirp:user message))
             (chirp:xml-decode (chirp:text-with-expanded-urls message)))))

Here we emit a signal to our client using the new-tweet signal and the mentioned arguments. Chirp takes care of URLs and entities. If you launch the client now using the main function, you should see your own status update, as well as everything that happens on your home timeline. That means we're pretty much done already! As a final addition, let's make the main also handle logging in.

(defun main ()
  (unless *logged-in*
    (with-main-window (w (make-instance 'login))))
  (when *logged-in*
    (let ((thread))
      (with-main-window (w (make-instance 'client))
        (setf thread
               #'(lambda ()
                    :user #'(lambda (message)
                              (when thread
                                (process-message message w) T)))
                   (format T "~&Shutting down tweet stream"))
               :initial-bindings  `((*standard-output* . ,*standard-output*)))))
      (setf thread NIL))))

Aaand done, ship it.

There isn't much else to the general concepts of UI programming with Qt other than widgets, signals, and slots. Everything else lies in knowing about the respective classes and methods, which is more vocabulary than concept. However, I hope that this quick introduction proved interesting and neat enough for you to take making UIs with Common Lisp into your list of feasible things.

I'd always welcome suggestions and ideas for extensions or modifications to Qtools to make working with Qt even more lispy than it is currently.

Thank you for your time.

You may read the source code in one piece here.

Additional note for the curious: You might be wondering how this all works in combination with Qt. As you know from your C/C++ experience, it uses different method naming conventions and types and all that wahoo. And indeed, the culprit for hiding this from you is Qtools. It translates types and method names into their C++ equivalents behind your back. This goes a long way towards bridging the gap. As an exercise, we'll take a look at the entire transformation sequence of a simple slot definition.

(define-slot (widget foo) ((text string))
  (print text))

The first thing that happens is that Qtools translates this into (surprise!) a method definition:

(defmethod %widget-slot-foo ((widget widget) (text string))
  (declare (slot foo (string)))
  (with-slots-bound (widget widget)
    (print text)))

Here we see another instance of using declarations to bridge the gap. You can of course also use defmethod directly if you prefer, and for some scenarios you really might. This also reveals why we need to :use cl+qt rather than cl, since Qtools needs to shadow the default defmethod. However, no worries, you can still use it as normal, the only difference is the extra declaration handling. Now, this method definition needs to be purified, as CL itself won't accept the slot declaration:

  (eval-when (:compile-toplevel :load-toplevel :execute)
    (progn (set-widget-class-option 'widget :slots '("foo(const QString&)" %widget-slot-foo))))
  (cl:defmethod %widget-slot-foo ((widget widget) (text string))
    (with-slots-bound (widget widget)
      (print text)))

And even more interesting things happened now! First what you see is Qtools' widget external redefinition capabilities. Using set-widget-class-option we can change the class definition form of the widget outside of its define-widget form. In this case we set a new :slot value (which is a CommonQt qt-class option). Here we also see that Qtools correctly translated the name and arguments of our slot definition into the equivalent name for the C++ side and links it to the method we define. The method that remains is a standard CL method definition. The with-slots-bound is a special form that performs a with-slots on all available slots of the class. Subwidgets get translated to class slots and using with-slots-bound they become automatically available through their respective symbols. This was added mostly because using accessors to refer to subwidgets becomes so ludicrously tedious, repetitive, and verbose that binding them all by default is the much less painful alternative.

Qtools offers quite a bit more than is outlined here such as additional type translation, menu definition, and finalization to name some. Take a look at the docs to see what it has in store.

Written by shinmera