I’ve always had a dream that Qt’s widget system would be based on a powerful 2D, or possibly even 3D, graphics engine, reaping all the benefits and optimizations that make games run fast. The reason is, coming from a 3D graphics background originally (alright, I was 16 at the time), I’ve always been puzzled by how poor application UIs perform, and how constrained they are, compared to the most basic 2D and 3D graphics engines out there. I think there are many reason for why graphics toolkits provide limited capabilities, and performance, and I’ve been studying this, hoping to help find ways for Qt to be better than the rest. If you ask me why oh why, be warned, I will talk all night.

I think I could get shot for saying this, but IMO widgets are monolithic beasts. Input, painting, clipping, geometry, events and all are almost always packed into just one class. And that class plugs into a framework that works in only one way. It’s hard to change the way a widget clips without introducing rendering artifacts (draw outside and try to update with the rendered region - oops, the dirty region is autoclipped to the widget rect). It’s impossible to know what a widget looks like without calling paintEvent(), which is a virtual function that might do something different every time it’s called. Multithreaded painting is extremely hard. It’s hard to make the widget paint outside paintEvent() in general. Couldn’t the widget just say what it looks like instead?

The main reason it’s like this, I think, is that UI toolkits’ graphics capabilities are just a hurdle in the path to the ultimate goal, which is to pull together UIs with a nice tool, perhaps targeting your favorite language, and with a cool style and the perfect widget ;-). IOW modeled vertically, after the concrete problem to be solved (which is generally speaking a good approach), and perhaps constrained by the capabilities of the primary target platform. I still think we can learn a lot from looking at UIs as a specialization of a 2D graphics scene API, rather than 2D graphics being an extension to a UI toolkit. We need to model our graphics the way that graphics works (both software and hardware), and not cling so much to a particular problem space. Make sense?

Qt provides both a high-level widget API, a low-level graphics API, and a “mid-level” canvas API called Graphics View. Graphics View is an example of loosening up the constraints of the high-level API, without exposing too many low-level problems such as geometry and dirty region handling. In a way it’s more a 2D graphics engine than anything else. It manages surfaces in 2D (or 2.5D, quasi-3D) space. Originally, we meant for it to be different from QWidget. Obviously, it’s a framework that’s meant for something completely different than widgets (vector graphics, charts, maps, IC design, large scrollable scenes, and so on). What we’ve learned, however, is that the two aren’t really that different. Why can’t I have 1000 widgets in a QScrollArea, for example.

So looking back a few years, you’ll see that we’ve made changes to improve QWidget. Without breaking compatibility of course (which is amazing in itself, shows how Qt’s architecture allows for significant internal changes).

Before Graphics View came out, in Qt 4.1, we loosened up one constraint in QWidget by enabling automatic background propagation. Now, widgets no longer had any default background (remember this change? how about QWidget::autoFillBackground oh, OK now you remember ;-)), and we were one step out of the box-model that widgets traditionally represent (btw when I use the term “box-model” I refer to a widget representing an independent rectangular region of actual screen real estate). Then, in Qt 4.2, we introduced delayed widget creation (DWC), which allows a widget to be constructed without an actual window handle. As part of the DWC work Paul, Matthias and a few others did for 4.2, they had to ensure that widgets had enough local state to independently represent what it otherwise had with a window handle, as without. Well, this had a ripple-effect. During the Qt 4.2 and 4.3 maintenance cycles, we discussed the fact that the only widget that really needs a window handle, is the top-level. With Qt 4.4, Bjørn Erik did some tough refactoring work (which some of us, me included, thought wouldn’t really be feasible), and gave birth to what we call Alien Widgets, the invisible behind-the-scenes beauty. Because of this, in Qt 4.4, a window and a widget are different, despite being the same class, in that the window signs a “contract” with the windowing system to register some screen real estate. This is the same now on all platforms.

Still, after this, you could see some strings that pull QWidget down. Painting outside paint event - using QWidget for screen shot captures, for example, required painter redirection. Each widget still constructed its own QPainter inside paintEvent(), and with it a separate paint engine, despite how all painting ended up in the same paint device: the QPixmap backingstore, which was associated with the same top-level window. Now in Qt 4.4, QWidget has a render-function, much like QGraphicsItem has a paint-function, and the window is, for a subtree of QWidgets, essentially the same as a QGraphicsView is for a scene. Puh.

Background propagation. DWC. Alien Widgets. Shared Painter. You see what’s happening? We’re on a mission. We’re closing the gap between QWidget and Graphics View. And we’re not done, there’s still more to come. There are some things that we cannot easily change, like QWidget’s clipping model for one (it’s opposite from Graphics View) [*]. And that Graphics View can’t make windows like QWidget (arguably, this is solvable though). Plus all our widgets are QWidget-based. Embedding the QWidget-based widgets into Graphics View using WoC is cool, but it’s just not good enough for full-blown exploitation…

Feature by feature, I must say the situation today looks surprisingly good. I’m looking forward to the day when I can simply assign a QGLWidget viewport as QWidget’s window. Or when I can load UIs from Qt Designer into Graphics View. Or in Qt 5, maybe the two are the same thing (the latter is usually only mentioned between some specially interested devs in Trolltech social events after consuming large amounts of beer).

That’s enough blabber for one blog post. I just felt like sharing what’s on my mind these days. This is btw all part of the research we’re doing in Development / Trolltech to support next generation UIs.

–

[*] QWidget is by default clipped to the intersect of its rect() and the localized exposed region before paintEvent() is called. QScrollArea has no explicit clipping features. Because most widgets don’t draw outside their bounds, item-imposed clipping should have been off by default (obviously expose-clipping is unavoidable for viewports that allow partial updates). And scroll areas should instead explicitly clip the widgets that intersect its edges (widgets outside shouldn’t be drawn, you don’t need clipping for that) (most 2D and 3D graphics APIs use subdivision instead, essentially real-time retesselation of the intersecting primitives to avoid clipping altogether). It’s extremely hard to change this in QWidget today without breaking compatibility. QGraphicsItem has the preferred model in place. But all our standard widgets are written using QWidget, not QGraphicsItem/QGraphicsWidget.

Although the Nokia acquisition has sucked away some time from development and made it somewhat difficult to stay focused, it doesn’t mean we are halted. Our coffee machine is still up and running so there should be no need to worry.

I’ve been working on improving the painting performance lately, and I can tell you we’ve found nasty stuff. Resizing top-level windows has always been quite frankly horrible. With Qt3 it was snappy, but looking at it almost felt like staring at a strobe emitting a series of flashes. I’m pretty sure it was a useful feature at the discos back in the days. With Qt 4.1.4 and the backing store, we closed down that business and started focusing on usability and managed to get rid of most of the flicker at the cost of performance. We have continuously worked on improving the performance in the 4.2 and 4.3 series, but up until the alien technology, we struggled with flicker. With flicker-free child widgets in place, the only problem left was the sluggish top-level flicker, and that’s what I’ve been working on recently. Tests showed that in worst-case an application could easily repaint itself three times per resize. So, with 100 resize events you could actually get 300 repaint events, which is 200 more than needed! Now imagine how this appears with an application having lots of complex widgets with expensive paint events. It’s ridiculous!

This has now been fixed and reduced down to one repaint per resize, but the story doesn’t end here. Another nasty thing I found was useless QPainter redirection operating on a global list involving several mutex locks and for loop iterations. More precisely (per paint event); 2 + X mutex locks and 1 + X for-loops, where X is the number of painters created during the paint event.

In addition, Jens found out how we could easily get rid of the black background fill appearing on resize on Windows Vista, and it really makes things look much better!

Aah, yes, and there’s one more thing I’d like to mention. If we go few months back in time when the Widgets on The Canvas project was integrated, you can see how we were able to draw widgets directly onto the canvas with resolution independence. You might wonder how on earth we were able to do that. Well, there is an easy answer: The new QWidget::render overload taking an arbitrary QPainter I really had to wrap my brain around lots of pain in order to achieve that. As you probably know, creating a QPainter on a widget boils down to initializing the paint engine using the backing store as the paint device. It just means we’re doing exactly the same over and over again, which in turn means it’s possible to use a shared painter! Yes! Easy, then we simply pass the shared painter in QPaintEvent and everything is fine. Or wait… Damn, then we would have to change every single paintEvent in Qt and customers code. Bad idea! So what we ended up doing instead was to hijack sub-sequent QPainter constructions/initializations and set the d-pointer of the newly created QPainter to the shared painter’s d-pointer and push/pop the state accordingly. Everything happens behind the scene without requiring changes to existing code. When it comes to performance it means we don’t have to initialize the paint engine (which is an expensive operation) each time we construct a QPainter. Great, isn’t it?

Unfortunately, I don’t have any demo to show you this time, but I hope it encourages you to download the latest snapshot and try it with your application. If you’re brave, you could also run it against 4.1, 4.2 and 4.3 and see how we gradually become better and better. It clearly shows we’re going in the right direction.

Performance is really important and needs serious attention, so you can expect to see more action on this topic in upcoming releases of Qt.

Happy hacking!

We’ve gotten reports about how Graphics View’s performance drops in one special (yet quite common) case. It’s the case when the item is fairly thin, flat, non-square, or simply sparse, where QGraphicsItem’s bounding rect generates something close to worst-case updates that cause repaints of areas that really haven’t changed. A very helpful customer showed me a simple testcase that illustrates this problem, which is quite hard to work around, which again effectively makes it a bug. Start the portedcanvas example, and clear the canvas (CTRL-E). Add a few meshes (ALT-M). Now click a node in the top left corner and drag it down towards the bottom right corner.

Performance degrades steadily as you get closer and closer to the bottom-right corner, until the example almost grinds to a halt. The example is very busy doing something close to nothing, at least it seems so. It’s redrawing the bounding rectangle of the edges to the node you’re dragging.

The yellow thing you get in debug mode by setting/exporting QT_FLUSH_PAINT=1 reveals what’s going on.

To finally hammer this bug down, we’ve added an opt-in feature, QGraphicsItem::boundingRegion(), and a configurable granularity. You can tune the granularity of the bounding region of your line / edge / connector items, and with the wave of a wand, QGraphicsView will magically generate an optimal update region for that item.

So with 4.4, you’ll instead get something like this:

What you’re seeing is that each of the two connector lines updates itself using a region consisting of 20-30 rectangles, a tunable granularity.

Should be in tonight’s 4.4 snapshots.

You have to admit it; you started reading this blog and fully expected a new years greeting. But HAH; I won’t do that in a blog. Find me in real life and I’ll add a hug or handshake to match. You know you want one!

Ok, with that out of the way, I want to talk about something that I find pretty darn exciting. Honestly, I think that a lot with KDE4 apps, they all have such very smart ideas. Well, this one is a bit different. I blogged about page sizes some 8 months ago, in that blog I said that I finished a feature that allows you to have a different page size for each page. There was just one tiny little detail that I forgot (for various definitions of forgetting); you need to be able to actually print those pages for this feature to come to its fullest potential.

I already blogged about me adding a method to QPrinter set a custom page size for printing. And that works great; but after implementing this stuff in KWord I found that it was not enough. I got the funny result that the page sizes for all pages were equal to the last set page size. Close, but no sigar.
So, over the holdays I spent some time reading up on the PDF spec and figured it was a simple bug that I could fix with some effort. The bugfix is in 4.4 and KWord can now honestly create proper PDFs with the pagesizes as you expect them. Some testing also showed that portrait/landscape *just works*. So I’m confident that a lot of new users are going to be happy with these capabilities.

While reading the PDF spec I noted that we can use some extra pdf tags to do things like bleeding better. We already do bleeding and it and probably *just* works for the majority of the printshops, but software can always get better. I’ll reserve that for Qt4.5 though.

As talking about graphics kind of makes a screenshot mandatory; here you go:

For the printer or printshop owners; here is the PDF itself (its uncompressed for debugging purposes) The pdf (pagesizes).

Since we just released the Qt/WinCE Technology Preview and it has been quite some time since our last Qt/WinCE blog, I think it is time for another chapter on how our brave Qt framework is discovering the land of Windows CE.

Those of you who downloaded Qt/WinCE and tested the technology preview might know that Qt/WinCE supports all Qt modules except Qt3Support and OpenGL.

Actually right now I am working on support for OpenGL ES for Qt/WinCE.

OpenGL ES is a specialized version of OpenGL for embedded devices. But it is still powerful enough for some really impressive demos and examples if you look at the SDK by PowerVR.

This is a small demo showing off OpenGL on the Dell Axim. For such a small device the OpenGL rendering is pretty fast I think. In this demo we already use QGLWidget for general rendering and QGLPainter for text rendering and the bubbles. So if everything works as planned our next Qt/WinCE release may contain support for OpenGL (ES).

In the meantime we are integrating Qt 4.4 into our Qt/WinCE branch.

Finally a few days ago I realized that our scribble example is really cool on a handheld, because it allows you to draw directly with a pen on the screen. To bad I have no real artistic talent.

I hope you guys will have as much fun developing cool and useful applications for Windows CE devices as we have porting the awesome Qt to Windows CE.

I just integrated a series of changes that adds support for doing multi-threaded text layout and printing. So it is now safe to use QFont and QFontMetrics outside the GUI thread. This means QPainter::drawText() works too (when painting on QImage, QPrinter, and QPicture). We’ve also done changes to QTextDocument that allow it to be cloned and passed off to a thread, so that all the layouting and printing happens without blocking the GUI.

For those interested, you can checkout tonight’s 4.4 snapshot. In tools/assistant/mainwindow.cpp, you’ll see code like this:

void PrintThread::start(QTextDocument *document)
{
    _document = document->clone();
    _document->moveToThread(this);
    QThread::start();
}

void PrintThread::run()
{
    _document->print(printer());
    delete _document;
    _document = 0;
}

void MainWindow::on_actionFilePrint_triggered()
{
    if (!QFontDatabase::supportsThreadedFontRendering()) {
        QPrinter printer(QPrinter::HighResolution);
        printer.setFullPage(true);

        QPrintDialog dlg(&printer, this);
        if (dlg.exec() == QDialog::Accepted) {
            qApp->setOverrideCursor(Qt::WaitCursor);
            tabs->currentBrowser()->document()->print(&printer);
            qApp->restoreOverrideCursor();
        }
        return;
    }

    PrintThread *thread = new PrintThread(this);

    QPrintDialog dlg(thread->printer(), this);
    if (dlg.exec() == QDialog::Accepted) {
        connect(thread, SIGNAL(finished()), SLOT(printingFinished()));
        connect(thread, SIGNAL(finished()), thread, SLOT(deleteLater()));

        qApp->setOverrideCursor(Qt::BusyCursor);
        thread->start(tabs->currentBrowser()->document());
    } else {
        delete thread;
    }
}

void MainWindow::printingFinished()
{
    qApp->restoreOverrideCursor();
}

The Thread Support in Qt documentation has also been updated to show what is supported (Note, at the time of writing, the snapshot documentation has not been updated, check back later if you don’t have a 4.4 snapshot after 27 Sep 2007).

And just to be clear, painting onto a QPixmap or a QWidget outside the GUI is not supported at all. We are looking at ways of making the GL paint engine safe for painting on to FrameBufferObjects and/or Pbuffers, which may or may not make it into 4.4.