Not long after Brad blogged about multi-touch in May, I got my hands on a brand new MacBook Pro under the excuse of implementing multi-touch support for Qt on Mac. This machine has a big and sexy trackpad (excuse my sexual orientation) that goes beyond normal mouse control to also include multi-touch- and gesture support.

After jumping on to the implementation, my first thought was that this was going to be a fairly quick thing to do. The API from Apple is rather straight forward. But after a short while, we started to realize that a touchpad (or trackpad in the case of a Mac) is more different from a touch screen than originally anticipated. The main difference being that on a touch screen, you place your finger directly on the widget you want to interact with. Qt then identifies that widget as the receiver of any following touch events. With a touchpad, you cannot place your finger directly on any widget in a similar fashion. You need to move the mouse to the widget first using the touchpad. This means that your first touch (and all following touches) will go to the widget under the cursor when the move starts, and not to the widget you are navigating to. So you would then need to do an extra step of lifting all your fingers and put them back-on the pad once the cursor is over the correct widget. In practice, this feels wrong and unintuitive.

After some hallway discussions and try-outs, we found that which widget to send touch events to should be decided after the second finger has been pressed. This is almost the same as saying that you will only get touch events when you are not controlling the mouse. This will normally be enough for applications that uses multi-touch for implementing gestures. If you are implementing single touch gestures, or otherwise need to handle the first touch, we added a flag you can set to override this default behavior. On a touch screen, you will always get all touch events.

A second problem regarding single touch (on a touchpad only) is the process of mapping touchpad position to screen position. When the touch begins, it is easy: screen position should be where the mouse is. But then it gets more blurry. The mouse will now move on the screen with a user defined speed and acceleration. So when your finger moves two centimeters on the touchpad, the mouse might move ten centimeters on the screen. In that case, should the screen position of the touch move with two centimeters? Or should the touch follow the mouse? The user would normally expect the latter (since there exist a logical connection between the touchpad and the mouse), especially if there are no visual clues on screen to mark the touch. However, implementing a strategy like this will grow in complexity (and inconsistency) once you start moving more fingers on the touchpad. And besides, recognizing gestures with mouse acceleration applied is not for the faint of heart. Conclusion: a touch will move on screen 1:1 with the touchpad. If you need to know where the mouse is, use mouse events.

In some cases, it might be more useful to know where the fingers are on the touchpad, rather than on the screen. To accommodate for this, we added to the touch events what is called a normalized position. This position range from zero to one, with the upper left corner of the touchpad being (0, 0), and lower right being (1, 1). With this information you can implement your own screen mapping, if you like.

Finally, there are certain limitations for using multi-touch. The API available from Apple is only for Cocoa, and only for Mac OS 10.6 (Snow Leopard). As a result, multi-touch for Qt/Mac will only be available for the Cocoa port running on OS 10.6. On the upshot, gestures are supported starting with OS 10.5.2, both in Carbon and Cocoa. Qt will of course mirror the same support.

Remember the days when all the tools at hand looked like a Basic interpreter with 30 commands, the most usable being PEEK, POKE and PRINT? Yes, that’s been a while…

Nevertheless, a few months ago I found myself doing lots of “printf-style debugging” again. Even with some basic IDE support that nicely displays QString and QByteArray, digging into more complex structures was just no fun. One of the reasons was certainly that we humans like to think of a QHash at a “higher level” of abstraction, in a way that’s fairly different from the actual implementation - and it’s only the low level implementation a debugger usually sees.

One solution is to bridge the gap between the low level (pointers, anonymous unions, …) and the high level (a bunch of keys with associated values) externally. This is done by asking the debugger for the type of an object only, and then - instead of asking for the low level data - tell it what part of the application’s memory is a hash key and what part is the corresponding value. This information is then transfered to Qt Creator’s “Locals & Watchers” view.

Below is the result for some nested containers and two QObjects with a parent-child relationship:

In the latter case the “more traditional” approach of displaying low-level data would produce:

The low-level display of a QHash is even less impressive. Of course, Qt Creator can easily toggle between the views, but it’s rarely needed.

For the convenience of the less Qt-centric audience similar displays have been added for a few widely used containers from the C++ Standard Library, proof coming here:

The display of aforementioned types (and a few more) works already with the Creator 1.1 release from April on Linux, Windows and Mac. All that’s needed is a not-too-old version of the GNU debugger. Gdb version 6.8 is just fine and widely available, on Mac Apple’s gdb will do.

As time goes on work continues, too. Qt Creator 1.2 will be able to show QAbstractItem, QAbstractItemModel, QByteArray, QDateTime, QDir, QFile, QFileInfo, QHash, QImage, QImageData, QLinkedList, QList, QLocale, QMap, QModelIndex, QMultiMap, QObject, QSet, QSharedPointer, QString, QStringList, QTextCodec, QVariant, QVector, QWeakPointer, std::list, std::map, std::set, std::string, std::vector, std::wstring as well as a few helper types (QMapData and the like). If you miss something desperately, just tell me

Last but not least, some special news for the people on the Windows side of the world: Starting with Creator 1.2 the debugger frontend will also support Microsoft’s debugger, even featuring the same friendly display of complex data.

André

Today’s example shows how to reuse the statechart from the flippin’ widgets example to create a widget that spins. The idea is to create a spin state that repeatedly enters a flip state until the speed is sufficiently slow (i.e. the widget should stop spinning). The spin state doesn’t know the internals of the flip state; the only thing it needs to do is enter the flip state, and have a state transition that’s triggered by the flip state’s finished() signal (at which point a single flip has been performed). In other words, the SpinWidget “embeds” the FlipWidget’s statechart in its own and builds more behavior on top. The spin state emits the finished() signal when its work is done, so now we can build new behavior on top of that state, and so on; indeed, the top-level states of the SpinWidget’s statechart are simply an idle state and a spin state. Also worth noting is that all states are composed of standard QState objects (no subclassing needed).

For fun I threw in a “pulse” effect for the item under the cursor. Video:

Download the example code here (qt/master required).

For those of you that enjoy writing relatively complex yet manageable apps without needing too much native code, the SCXML enabler that works on top of the Qt State Machine Framework is now active on public Gitorious, at http://qt.gitorious.org/qt-labs/scxml. We’re opening the repository after a few bug-fixes and improvements we found in other Git repositories - we want to enable people to contribute them back!

The old examples still work, and the code gets smaller and smaller as most of the functionality is already in Qt, under a C++ API; since QStateMachine was developed using the SCXML algorithm and semantics, the QScxml class mainly includes  some XML parsing and QtScript preparations - definitely not rocket science 

The SCXML enabler on top of QStateMachine adds the following functionalities:

• Having QtScript-based guard conditions for transitions. For example, this simple log-in functionality, which (hopefully) explains itself:<transition event=”q-signal:loginButton.clicked()” cond=”passwordEdit.text == currentUserInfo.password” target=”logged_in” />
• Evaluating scripts on entry/exit/transition, and in general utilizing a single QScriptEngine for the state-machine which is a small detail that makes a big difference.
• Anchors for generic “go back” functionality, which is different from history states, that don’t maintain a stack (see mediaplayer example).
• Using <invoke> to extend the functionality, like the message-boxes in the blackjack example.
• Assigning dynamic property values to states, based on ECMAscript, as the C++ QStateMachine only allows for static assignments.
• State-machine timers using the <send> and <cancel> tags (see blackjack example).
• Conformance with the SCXML draft spec.

The SCXML enabler works very nicely with the QtScript Binding Generator, and an interesting demo is coming up soon…

We’ve been hearing from people that they’ve been able to utilize this (tiny) library to develop QA solutions, and to model car infotainment systems. We’d like to know if this library is useful for you, and how!

As Ariya mentioned in one of his previous posts, we’ve been doing some work on using JavaScriptCore (JSC) as the QtScript back-end. The whole idea is that you’ll get the same QtScript API, but with JSC performance. There’s also the prospect of being able to use the QtScript API together with QtWebKit, giving a smooth integration (e.g. having a function similar to QWebFrame::evaluateJavaScript() that returns a QScriptValue rather than a QVariant, and being able to use QScriptValue::call() and QScriptValue::setProperty() and friends; in general, having such API will just make it much easier for the C++ side and JS side to communicate).

The main challenge we’re facing is keeping the QtScript API behave 100% as it does today; it’s not acceptable that the behavior changes just because us speed freaks decide to replace the back-end, of course. The main aid we have in this regard is the set of QtScript autotests; assuming the tests are all-encompassing (which, let’s face it, is hard to guarantee, but let’s pretend…), after completing the back-end swap, all the tests should still pass. Once this goal is reached, some brave soul would step forward, brown paper bag in hand, asserting that except for performance improvements, it’s not possible for the application developer to tell that the QtScript runtime was replaced. OK, so we’ll probably have to do some real-world testing as well.

Work on the back-end had been put on hold for some time due to various other projects, but now we’re slowly getting back in business again. And the best news is that the ongoing work is now out in the open (qtscript-jsc-backend branch)! Now before you start doing the Qt dance, just keep in mind that the branch is not currently in a state where it can be used to run new or existing apps against the back-end, because not all the API and/or behavior is implemented yet. Currently we’re working on bringing it up-to-date with the last few months of changes in WebKit trunk, then we can focus on remaining API. In case you’re interested in helping out hardcore hacker-style, though, there’s an easy recipe that we all follow: Run one of the qscript* autotests, look for failures or asserts that say “Implement me”, and hack on the implementation until the test passes. Test statement by test statement, the compatibility will get there. Happy hacking!

Remember the times when the QtScript debugger was nothing but a maverick project on Trolltech Labs, and you could run the debugger as a stand-alone application? When the debugger was finally integrated into Qt for the 4.5 release, that feature had mysteriously disappeared. Well, today I’m happy to tell you it’s been resurrected — not as part of Qt (yet, at least; that depends on the interest), but right here on the Labs. It needs to be compiled against the latest and greatest qt/master sources, and getting it to link can be quirky (it needs some symbols that aren’t exported from Qt by default — see the project page for details)… Let me know how you get on.

The debugger application embeds a QScriptRemoteTargetDebugger object.

QScriptRemoteTargetDebugger debugger;
debugger.attachTo(host, port);

The debugger emits an attached() signal when it’s attached to the target, or an error() signal if it fails to connect. Other than that, the API mirrors that of QScriptEngineDebugger; by default, the standard debugger window is shown when the target’s script evaluation is suspended, but you also have access to the individual debugger widgets and actions if you want to create a custom configuration. A “reference implementation” of a debugger can be found in the examples/debugger directory.

So how does your application’s script engine become a target that can hook up with a remote debugger? By adding lines like these:

QScriptDebuggerEngine debuggerEngine;
debuggerEngine.setTarget(scriptEngine);
debuggerEngine.listen();

The listen() function tells the debugger engine to wait for the debugger (UI) to initiate the connection; you can do it the other way around (have the debugger wait for the target to connect) by calling listen() on the debugger side and connectToDebugger() on the target side. The debugger engine emits a connected() signal when a connection has been established, or an error() signal if there’s a problem.

Once you’re connected, your application can start evaluating scripts as usual; the debugger engine will suspend evaluation and communicate with the debugger as necessary. There’s an example application in the examples/debuggee directory that demonstrates this. To try it, first start examples/debuggee, then start examples/debugger.

The current implementation uses QDataStream with the debugger’s own internal types to transport data from one end to the other; I’d like to also add support for protocols like those of GDB and Xdebug, so that other debugging tools could be used. It would be fun to see if Qt Creator’s existing GDB integration can be utilized for QtScript debugging too. Anyways, happy remote debugging! Comments and suggestions concerning API, functionality and protocols are welcome.

While planning for Developer Days 2008 I tried to wring something interesting out of my technically starved brain to show off to the crowds of hackers attending. At the time I had just noticed Twitter, a service that seemed to promise the value of Facebook without all the junk - seeing status updates from my friends and the ramblings from people I find interesting or useful from an academic perspective.

I started looking into the Twitter API and found that using Qt’s QNetworkAccessManager I could quickly create something powerful that interoperated with URL API’s. Attempting to be different, I decided that it wouldn’t be cool to create a UI based on traditional QWidgets, and definately too boring to simply browse twitter.com. So, using the Twitter API which returns XML data, QtXmlPattern’s XSLT support and a little bit of creative tinkering with exposing Qt native objects to QtWebKit, I ended up with a useful Twitter client that converted the XML data to UI.

After returning from Developer Days, we enlisted the help of Girish - a(n ex )Troll and Generally Great Guy who helped polish my haphazardly written demo-code into something beautiful and functional - as well as write a completely new example on how to integrate with YouTube! Girish also made some nice screencasts for our Qt 4.5 launch (source follows after);

And now we have finally pushed the two cool examples of QtWebKit that shows how easy and elegant it is to mixing web programming paradigms with native code in Qt! If you want to try it out, play with it, modify it, please head over to the Graphics Dojo and look for twitterview and videofeed. Happy hacking!

Because of the asynchronous nature of X11, it has traditionally been hard to make sure that the contents of a window synchronizes well with the window border while resizing. To solve this, a window manager protocol was developed that allowed the application to inform the window manager about when it had finished dealing with the resize event. This is known as the _NET_WM_SYNC_REQUEST protocol.

Until now, Qt has not officially supported this protocol, although there was a patch that adds support for it in the KDE copy of Qt 3. What bugged me about this was mainly that it caused resizing to be unnecessarily slow on the GNOME desktop, since Metacity will only send resize events at a fixed slow interval for applications that do not support this protocol, no matter how simple your dialog. I never did any X11 programming before, so I joined forces with Matthias and Denis and together we worked towards this patch, which adds support for the _NET_WM_SYNC_REQUEST to Qt 4.6.

Demonstrating resizing got smoother on GNOME

As a bonus, this fixed the disturbing resize behaviour when running with Compiz (visual effects). As became clear, Compiz relies heavily on this protocol to provide perfect synchronization between the application and the window border.

Demonstrating smooth and solid resizing on Compiz

Finally, I will note that waiting on the application does of course not come without drawbacks. For complex applications like for example Qt Creator, we have noticed that the resizing can also feel slower. This is because the window frame is waiting on the application to catch up, which increases the desynchronization with the mouse cursor. We’re not sure what is better in this case. I know I personally would prefer my frame to resize as fast as possible, without taking into account the application at all. But that should in my opinion be a window manager option.

For those who would like to try it out live but don’t want to go through the trouble of compiling Qt themselves, I’ve built two static 32-bit binaries; one that has the patch and one that doesn’t. Even though they are built statically against Qt, they will still need some other libraries available on your system, just see if it works for you (tested on Ubuntu Jaunty).

In Qt 4.6, QGraphicsWidget is gaining some new properties for transforming items: {x,y,z}Rotation, {x,y}Scale, {horizontal,vertical}Shear and transformOrigin, to be specific. How does this fit with the 4.6 master plan, the almighty Qt Kinetic project? Did someone just say “Like a glove! (Ka-bling!)“?! Yes, that sounds about right. Naturally I set out to use and abuse that shiny new stuff by means of states and animations. Seeing as I’m born flippy, a basic use case that came to mind was to create a widget that can be flipped between a “front” side and “back” side. Such a “two-faced” widget can (for example) be implemented as a QStackedWidget that contains the “front” widget on page 0 and the “back” widget on page 1. Flipping to the other side is then a matter of toggling the QStackedWidget’s currentIndex property. But obviously we want to smoothly animate this change, as failure to do so would kinda take the “flip” out of “flippin’” (ya know?). One way of doing it is to first animate the item’s yRotation from 0 to 90 (at which point the item will effectively be invisible), then toggle the currentIndex property, and finally animate yRotation from -90 to 0. The visual effect is that the “two-faced” item appears to rotate 180 degrees, exposing its Good (Evil) side.

Today’s post is brough to you courtesy of programming languages of yore (either you’re with them, or you’re against them). Alas, here’s the C++ code that sets up the stacked widget and such:

FlipWidget::FlipWidget(QGraphicsItem *parent)
    : QGraphicsWidget(parent)
{
    // The front.
    QWidget *front = new QWidget();
    QPushButton *toBackButton = new QPushButton("To Back");
    QObject::connect(toBackButton, SIGNAL(clicked()), this, SIGNAL(flipRequest()));
    toBackButton->setStyleSheet("background: pink");
    {
        QVBoxLayout *vbox = new QVBoxLayout(front);
        vbox->addWidget(toBackButton);
    }

    // The back.
    QWidget *back = new QWidget();
    QPushButton *toFrontButton = new QPushButton("To Front");
    QObject::connect(toFrontButton, SIGNAL(clicked()), this, SIGNAL(flipRequest()));
    toFrontButton->setStyleSheet("background: yellow");
    {
        QVBoxLayout *vbox = new QVBoxLayout(back);
        vbox->addWidget(toFrontButton);
    }

    // The stacked widget.
    m_stack = new QStackedWidget();
    m_stack->addWidget(front);
    m_stack->addWidget(back);

    QGraphicsProxyWidget *stackProxy = new QGraphicsProxyWidget();
    stackProxy->setWidget(m_stack);
    QGraphicsLinearLayout *vbox = new QGraphicsLinearLayout(this);
    vbox->addItem(stackProxy);

The “front” and “back” widgets don’t contain anything interesting in this example, just buttons that flip the item. Clicking both the “front” and “back” button will cause the item’s own flipRequest() signal to be emitted; this is so that the same state logic can be used to handle flips from both “sides”. Which brings us to the interesting bit, which is the code that sets up the states and animations for doing the flip:

    QStateMachine *machine = new QStateMachine(this);
    QState *s0 = new QState(machine->rootState());
    s0->assignProperty(this, "yRotation", 0);

    QState *s1 = new QState(machine->rootState());
    s1->assignProperty(this, "yRotation", 90);

    QAbstractTransition *t1 = s0->addTransition(this, SIGNAL(flipRequest()), s1);
    QPropertyAnimation *yRotationAnim = new QPropertyAnimation(this, "yRotation");
    yRotationAnim->setDuration(250);
    t1->addAnimation(yRotationAnim);

    QState *s2 = new QState(machine->rootState());
    QObject::connect(s2, SIGNAL(entered()), this, SLOT(togglePage()));
    s2->assignProperty(this, "yRotation", -90);
    s1->addTransition(s1, SIGNAL(polished()), s2);

    QAbstractTransition *t2 = s2->addTransition(s0);
    t2->addAnimation(yRotationAnim);

    machine->setInitialState(s0);
    machine->start();

Note the use of the QState::polished() signal (name still subject to change); this signal is emitted when all properties associated with a state change have reached their target values (effectively waiting for property animations, if any, to finish). When that happens, the machine transitions to a state that toggles the currentIndex (togglePage()) and inverts the rotation, and then unconditionally transitions back to the initial state (yRotation = 0), again animating the change. So, just three states to handle the flip both ways. Maybe it’s the bias talking, but isn’t it nice?

Finally, let’s create a number of independently flippable widgets and put them in a grid:

class Window : public QWidget
{
public:
    Window(QWidget *parent = 0)
        : QWidget(parent) {
        QGraphicsScene *scene = new QGraphicsScene(this);
        QGraphicsWidget *widget = new QGraphicsWidget();
        QGraphicsGridLayout *grid = new QGraphicsGridLayout(widget);
        for (int i = 0; i < 6; ++i) {
            for (int j = 0; j < 6; ++j)
                grid->addItem(new FlipWidget(), i, j);
        }
        scene->addItem(widget);
        QGraphicsView *view = new QGraphicsView(scene);
        QVBoxLayout *vbox = new QVBoxLayout(this);
        vbox->addWidget(view);
    }
};

EDIT: Added video:

Man, those widgets they be flippin’ all over! Best not show the Widget Pimp, lest he be pimpin’ out!

The source code for this example can be grabbed here. You’ll need to build it against qt/master. Something that might be fun to do is try to make the animated transition more “dramatic”, for example by animating the scale of the item as well, experimenting with the duration (”How slow can it go?”), and setting some whacky easing curves (did I declare my love for InOutElastic already? Why yes, yes I did). Oh yeah, if you really want to differentiate yourself you can create a different animation for the back-to-front flip.

The FlipWidget class could be made more general-purpose by allowing you to pass in any widgets to use as the two sides, and adding a flip() slot (so an item can be flipped programmatically) and a flipped() signal. I’ll leave that as a small exercise for the reader (hint 1: the flip() slot is a one-liner due to the presence of the flipRequest() signal; hint 2: the flipped() signal can be provided based on the QState::entered() signal, but you’ll need to insert an extra state). Once you have flip() and flipped(), you should be able to add some logic (Qt State Machine-based, of course) to the basic example and turn it into a “Find-two-of-a-kind” memory game. (As I’ve forgotten to mention on numerous occasions, the criteria by which a Qt API will ultimately be judged is its ability to sustain the healthy development of silly games. Lots and lots of silly little games.)

You know, I’m starting to get the same “It’s all coming together”-feeling as I did back with 4.4. It’s a good feeling. Anyway, happy hacking, and don’t be afraid to share your ideas and code!

An interesting feature of a navigation system (so I was told, I don’t own a car) is the so-called "night mode". Basically the display is adjusted so that the text and the map become more readable under low ambient light condition. An easy trick to do this is by inverting the color, just like what Mac OS X can do for the whole desktop.

Thanks to QPainter, it is fairly easy to do that in just a few lines of code. Basically we need to use a proper composition mode. If you think composition modes are like black magic, here is a chance to understand a bit of it. As you read the documentation on QPainter::CompositionMode_Difference or CompositionMode_Exclusion, you will find that Painting with white inverts the destination color. Voila! That is exactly the cheating path that we have to use.

The super-short example for color inversion is in the Graphics Dojo repository under the webnightmode directory (available for both Qt/C++ and PyQt). Although the trick here is illustrated for QWebView, of course it is suitable for other widgets, too (see Henrik’s yesMan example). Note that not every paint engine/device supports QPainter::CompositionMode_Difference, hence why in the code I force the use of raster graphics system under X11.

Note that an actual implementation of a night-mode feature is likely more complicated than just a color inversion method (feel free to suggest or brainstorm the ideas for such an algorithm). The overall brightness and constrast should be taken into account. Likely this requires an adventure in the non-RGB color space, something that is outside the scope of this example. Most of the cases however, you can get away with the discussed trick.

Exercise for the brave readers: use selectors (CSS or jQuery) to find the map element, then invert that element only.