Declarative UI is one of the big things on the Qt Roadmap for Qt 4.7 and 4.6.x. I already enjoyed working with the Qml language and the developers behind it for quite some time - and believe me, this one will fundamentally change the way slick Qt UIs are designed and look like! If you have not checked it out yet, do so … although it is not yet part of the Qt package, it is very mature, and just fun to experiment with

Anyhow, a technology like this of course does not come out of the blue, neither do the tools to support it - I think the first discussions about a Visual Editor for a language yet-to-be-invented on the basis of the QGraphicsView framework started in summer 2008! In early 2009 I first heard about the name “Qml”, and was soon the lead of a development project, with right now 5 people working full time in this project. One of the most important decisions we had to make early on was for a cool project name - we eventually settled for “Bauhaus”, as a reminiscence to the famous German school for design.

Back to the facts: What I want to share with you today are our plans for supporting Declarative UI / Qml in QtCreator. This is an important one, because with Declarative UI we target not only the traditional Qt/C++ developers, but also more design centric people - the goal is to let both share the same language, from early prototyping until the final product. We can only achieve this with good tools in place.

Here are the things we are currently working on for the next major version of QtCreator:

Qml Text Editor - We already have some basic syntax highlighting / formatting support in QtCreator 1.3, but in the QtCreator master branch we are right now working on really mature Qml/JavaScript support. This will include all the goodies you kind of expect these days: Code completion, context sensitive help, …

Visual Qml Editor - After all, we are talking about graphical user interfaces, and you will not make UI designers happy just with a text editor . Here we decided for a fresh start and developed the components for the Visual Editor from the ground up. Interestingly, we are using Qml heavily ourselves here - e.g. in the Property Editor and States View.

Seamless Integration - It is no either text, or visual editing: You want to use both, and quickly switch between both. This is why we deeply integrate the Visual Editor into QtCreator, share the same Undo/Redo history and are also preserving the Qml file formatting as much as possible when doing file changes.

Debugging - We integrate the Qml debugger, which allows you to inspect the qml item tree and its properties at runtime, to check framerates, to evaluate JavaScript expressions and so on inside QtCreator.

Enough said - Nigel recorded a nice video showing you how the support currently looks like:

If you want to try it out yourself, we also packaged a technical preview of creator, qmlviewer/qmldebugger and the declarative examples & demos in one installer:

qt-creator-win-opensource-1.3.80-qml-tp1.exe (Windows 32 bit)
qt-creator-linux-x86-opensource-1.3.80-qml–tp1.bin (Linux 32 bit)
qt-creator-mac-opensource-1.3.80-qml-tp1.dmg (Mac OS X)

Disclaimer: The binaries are unsupported, and are only meant for early testing. In fact they are based on an untested snapshot of Qt, and an (almost) untested version of QtCreator! If you want to use qml for production work, stay with QtCreator 1.3.1.

Of course we are interested in all kind of feedback, preferably via the bug tracker or on the qt-creator mailing list.

I’d like to share with the community a project I’m working on, while it’s still in its development phase (isn’t that what labs is for? :))
The goal of the project is to get CSS3 animations to a reasonable FPS performance, mainly on embedded hardware where it’s a pain.

See http://gitorious.org/~noamr/webkit/noamrs-webkit/commits/accel

The idea is to implement webkit’s GraphicsLayer concept, which allows platform-specific implementations of CSS transform and opacity animations, using the graphics-view and the Qt animation framework as a backend. This would only work for QGraphicsWebView and not for QWebView, as rendering a separate QGraphicsScene inside QWebView would probably not give us much performance benefit.

Preliminary results are very promising - The leaves demo, for example, runs 4 times faster on Maemo Fremantle than it does without the acceleration, and it looks graphically accurate.

The reason this gives us a performance benefit is mainly because of graphics-item caching: when a CSS animation occurs inside webkit, the item that’s being animated has to go through a re-layout and re-draw every so often, while with the accelerated approach we draw it once into a QPixmap (QGraphicsView takes care of that) and then it’s just a series of fast and furious pixmap blts. The hardware acceleration becomes relevant when the images are big and the blt itself becomes a bottleneck.

This project is not ready to go upstream, as it supports many delicate use-cases that need to be tested. But if you’re interested in participating (or to just comment!), this has so far been a fun project to hack on.

Instructions:

1. Get the Git repo from git://gitorious.org/~noamr/webkit/noamrs-webkit.git, branch accel
2. Build or get a relatively new version of Qt, possibly without building QtWebkit
3. Build Webkit from the downloaded Git repo:
   export QTDIR=[my-qt-4.6-root]
export PATH=$QTDIR/bin:$PATH
./WebKitTools/Scripts/build-webkit --qt
4. Run ./WebKitBuild/Release/bin/QGVLauncher --accel: This will enable the necessary web-setting for composite-layer acceleration. You can also create a small QGraphicsWebView example yourself, as long as you enable the new settings flag: QWebSettings::AcceleratedCompositingEnabled
5. Load a website with CSS transform/opacity animations: like this one or this one.
6. Hack. Most of the code is in WebCore/platform/graphics/qt/GraphicsLayerQt.cpp, or you could just search for the term USE(ACCELERATED_COMPOSITING)
7. Send merge requests through Gitorious or comments on Bugzilla

No’am

In previous posts in this series, Gunnar has described the design and performance characteristics of the painting system in Qt, and explored Raster in greater depth.  In this post, I’m going to talk about the unique features of the OpenVG graphics system.

Paint Engine

Unlike the other engines, the OpenVG paint engine was much easier to implement because the OpenVG API itself is very close in functionality to QPainter.  You can read all about the specification on Khronos’ OpenVG Page, but here are the high points:

• VGPath objects represent geometry made up of MoveTo, LineTo, and CubicTo elements.  This is a very close match for Qt’s QPainterPath and QVectorPath abstractions.
• VGPaint objects represent brushes and pens for filling paths with pixel patterns.  Solid colors, linear gradients, radial gradients, and pattern brushes are supported, but not conical gradients.
• VGImage objects represent pixmaps in a large variety of pixel formats.  OpenVG supports a lot more formats than OpenGL/ES which makes it a lot easier to convert QImage’s into VGImage’s.
• VGFont objects (OpenVG 1.1 only) store glyphs represented as VGImage’s or VGPath’s for quicker rendering of text items.  Under OpenVG 1.0, we fall back to path drawing at present.
• Scissor for rectangle-based clipping.
• Alpha mask for clipping to arbitrary shapes.
• Affine transformation matrices for path and glyph drawing, affine and projective transformation matrices for image drawing.

Transformation Matrices

OpenVG does not support projective transformation matrices for path drawing, which is annoying because QPainter allows any affine or projective QTransform to be used for any drawing operation.  There is a registered OpenVG extension called VG_NDS_projective_geometry but none of the OpenVG engines we have come across support it.  The reason why OpenVG doesn’t support it is because generating paint pixels in perspective can be quite difficult.  Projective matrices are supported for image drawing because drawing a simple image in perspective is a well-understood problem that OpenGL/ES systems do all the time.

When a projective transformation matrix is used for path drawing, we convert the path point-by-point using the QTransform and then draw it as a normal affine path using a default transformation for the window surface.  But what about the paint pixels?  Unfortunately, they won’t be perspective-correct.  In practice this isn’t a big problem because most paths are drawn with a solid color brush, and a solid color looks the same in perspective.

In general however, we discourage people from using projective transformations with paths.  If you really want to draw a scene in perspective, first draw it into a QPixmap and then draw the pixmap using a projective transformation.  You’ll probably want to do this anyway because perspective transformations mostly occur during “flip” animation effects - drawing every tiny path in perspective every frame during the flip would be too slow.

Path Transformation and Drawing

Most of the path transformation logic is done in vectorPathToVGPath() and painterPathToVGPath() in qpaintengine_vg.cpp. We detect the presence of affine vs projective transformation matrices and use an appropriate conversion. We convert both QVectorPath and QPainterPath using specialized routines. The other paint engines typically convert everything into a QVectorPath first. The QPainterPath conversion can improve performance slightly when arbitrary paths are drawn during SVG rendering and the like - there’s no point creating a QVectorPath if it is going to be quickly thrown away.

Path drawing takes a lazy update approach, attempting to minimize the number of OpenVG state changes from request to request:

• If the draw requires a pen, then the penPaint object is updated with the current QPen if it was different from last time.
• If the draw requires a brush, then the brushPaint object is updated with the current QBrush if it was different from last time.
• The path transformation matrix is updated if it has changed since the last path drawing operation.
• The path is drawn using vgDrawPath().

Most of the OpenVG state persists across paint events so if the same pen is used from one frame to the next, then it will be set once and never changed.  The state is also shared between all windows because there is only one OpenVG context for the entire system.

In an earlier version of the OpenVG paint engine, I just uploaded the state changes whenever they were made without trying to be lazy about it.  That was a mistake!  Applications that use QPainter, particularly those using QGraphicsView, can be very chatty - constantly saving and restoring the painter state.  It was quite common for brushes, pens, and transformation matrices to be changed, then changed again, without anything being drawn.  Now, it will only update the OpenVG state at the point where an actual drawing operation is about to happen.  This house-keeping does have a cost though, so if you can avoid unnecessary QPainter state changes in your application, then please do so.

Preallocated Paths

Rectangles, lines, points, and rounded rectangles feature quite heavily in many applications, with constantly changing co-ordinates.  OpenVG makes us create and destroy a VGPath every time.  To alleviate this, we’ve provided some pre-allocated paths for simple drawing operations, which we update with vgModifyPathCoords() rather than allocate GPU memory for a new path.  However, some chipsets can be slower at modifying a path than just making a new one!  On those chipsets, compile Qt with the QVG_NO_MODIFY_PATH macro.
Image Drawing

The best image drawing will be achieved with QPixmap rather than QImage.  With QPixmap, the image is converted into a VGImage once and then drawn multiple times.  With QImage,the image must be converted into a VGImage every time it is drawn.

The OpenVG drawing primitive vgDrawImage() is very primitive - it draws the selected VGImage at the origin of the current transformation.  There is no in-built support for sub-rectangle drawing. Fortunately, OpenVG has vgChildImage() which allows a sub-region to be quickly extracted, with the pixel data shared with the parent.  However, “quickly” is a very relative term - I’ve seen frame rates almost halve when using vgChildImage() compared to drawing a full image.  So if you can, draw entire QPixmap’s when using OpenVG and limit the use of sub-rectangles.

Another source of slowdown is drawing images with opacity.  OpenVG has a way to multiply a VGPaint object with a VGImage to produce a destination image.  This is a very cheap way to achieve opacity effects and is quite fast.  Except!  And there is always an Except!  Except when the image is drawn with a projective transformation matrix.  Remember - paint pixels cannot be generated in perspective - so we cannot use a paint object to generate an “opacity color” even though the solid opacity color will be the same in perspective from all angles!  This is very annoying - the OpenVG committee could have made a special exception for solid color VGPaint objects.

When the OpenVG paint engine draws an image with opacity, and a projective transformation is in effect, we have to generate a copy of the VGImage and use vgColorMatrix() to adjust the opacity.  This isn’t too bad if you are drawing the same image over and over with the same opacity, but it is very inefficient if you are animating the opacity.  So avoid opacity animations with OpenVG if you can.

Painting into a QPixmap isn’t currently accelerated with OpenVG - it uses the raster paint engine instead, so we recommend painting pixmaps once rather than constantly updating them.  We will be addressing this in future versions.  Even when we do implement painting into a pixmap, there will be a cost: switching rendering surfaces from a window to a VGImage and back again is not cheap - on some chipsets it can be as heavy as a full EGL context switch.  So try to avoid switching painting surfaces if you can.

Clipping

Clipping is the bane of my existence!  It seems so easy to application writers - set a clip rectangle and it will be efficient, drawing less pixels!  If only!

There are three techniques that can be used to achieve clipping with OpenVG:

• Scissor rectangle list.
• Alpha mask for arbitrary clip shapes.
• Scissor rectangle for simple clips and alpha mask for complex clips.

The last is the default in the OpenVG paint engine, and there are #define’s that can be used to enable the other modes.  However, on some PowerVR chipset versions there is a bug where if the scissor is combined with the alpha mask, performance drops off a cliff - down to 2 frames per second in some cases!  So on such devices you may want to turn on scissor-only or mask-only clipping.

Better is to not use clipping at all if you can avoid it.  Draw everything in your scene in bottom-up order and let the GPU do the heavy lifting.  Remember, modern OpenGL/ES GPU’s can crank out thousands of triangles per second, with clever algorithms for hidden-surface removal that are much cleverer than anything you can do by setting a clip.  OpenVG uses the same GPU in many cases.  If you set a clip, you may end up confusing the GPU into taking a slower path internally than it would otherwise.

If you must clip, try to use single-rectangle regions that can be set via the scissor.

Window Surfaces

Below the OpenVG paint engine is the window surface logic in the graphics system.  This is usually where platform-specific customizations are required to get pixels onto the screen as fast as possible.  The QVGWindowSurface class wraps a QVGEGLWindowSurfacePrivate object, which provides the heavy lifting.  The default EGL implementation is QVGEGLWindowSurfaceDirect which writes pixels into the window back buffer and calls eglSwapBuffers() to transfer it to the screen.  It is possible to enabled single-buffered operation with QVG_DIRECT_TO_WINDOW, but the cost may be tearing artifacts on-screen.

If your platform has some clever EGL extension mechanism for getting pixels onto the screen, then you will need to write a new graphics system plugin and implement your own QVGEGLWindowSurfacePrivate subclass.  The QtOpenVG module has been structured to make it relatively easy to do this without touching the core Qt code.

Memory Usage

Everything you do with graphics uses memory - in the CPU and in the GPU.  Window surfaces, VG rendering contexts, VGPath objects, VGImage objects, and so on.  It can get quite tight in the GPU on embedded systems.  We’ve taken some steps to manage this; e.g. destroying older VGImage objects when trying to upload a new QPixmap, and destroying all OpenVG objects when an application goes into the background to free up memory for foreground applications.

The more complex your application, the more likely it is that you’ll hit the GPU memory limit.  There’s only so much the QtOpenVG module can do for you.  We can take emergency measures to recover, but that’s about it.  So keep an eye on how many pixmaps and windows you have in use and see if you can simplify your application a little.  Definitely avoid uploading very large jpeg photographs as a single QPixmap - split them up into smaller “tiles” that can be released when GPU memory gets tight.

Summary

The following tips summarise the performance suggestions from the previous section:

• Avoid projective transformation matrices with drawing paths.
• Minimize state changes on pens, brushes, transforms, etc.
• Use QPixmap in preference to QImage where possible.
• Avoid drawing images using sub-rectangles.
• When drawing images with opacity, use an affine transformation matrix, or only a single opacity level.
• Avoid switching painting surfaces, particularly between windows and pixmaps.
• Don’t use clipping if you can paint your scene in bottom-up order instead.
• Split large images up into smaller pieces to avoid overloading GPU memory.

What’s Next?

There’s always more that can be done to improve any software system.  QtOpenVG is no different:

• Painting into QPixmap’s using OpenVG.
• Smarter VGImage pooling to deal with out of GPU memory conditions.
• Qt/Embedded and Lighthouse screen drivers.

Recently, Qt Declarative for Qt 4.6.0 got released. We got a lot of requests for Maemo 5 support (well - some people just wanted to play the SameGame on their N900 ). And voilà - declarative hit Maemo’s extras-devel repository. Obligatory screenshot:

Currently, the following packages are available:

• libqt4-maemo5-declarative: The QtDeclarative library
• libqt4-maemo5-declarative-dev: The development package, contains the headers for QtDeclarative
• libqt4-maemo5-declarative-dbg: The debugging symbols, for more meaningful backtraces
• qt4-maemo5-declarative-qmlviewer: The “player” for QML based apps
• qt4-maemo5-samegame: The SameGame demo

To just get the SameGame, all you need to do is

apt-get install qt4-maemo5-samegame

(provided that you enabled the extras-devel repository). And the neatest thing - with the N900’s text editor, you can program your own QML applications directly on the phone. Launch them from the xterm with /opt/qt4-maemo5/bin/qmlviewer ./myapp.qml

No need to stop hacking just because space is too constraint to get out your laptop

Obligatory note: Qt 4 for Maemo 5 is still in tech preview status, reports about molten N900s will be met with a compassionate “Told you”.

For a long time the standard layouts shipped with Qt’s have been built around the concepts of linear and grid layouts. In the QLayout regime they are represented as QBoxLayout and QGridLayout. In the QGraphicsLayout regime they are represented as QGraphicsLinearLayout and QGraphicsGridLayout. We therefore started early this year with research on a new layout. The efforts of that culiminated into the QGraphicsAnchorLayout.

Motivation

We have quite often seen that the existing layouts in Qt can not always do what you want with just one layout. Take for instance Qt Linguist’s “Find dialog”.
The dialog is made up of four box layouts and one grid layout for the checkboxes inside the groupbox.

(note that Qt Designer does not illustrate the layout inside the groupbox or the toplevel layout)

In addition, in order to achieve this simple arrangement it uses three levels of nested layouts.
Luckilly, if you design it in Qt Designer it is quite easy, but you might not get it right at first attempt. Of course, doing it programmatically makes it even worse. The result are probably far from how a UI designer would specify the arrangement. We think that an anchor layout will narrow that gap in a lot of cases.

Concept

The QGraphicsAnchorLayout is a very flexible layout, yet the concept is simple. By creating an anchor between two edges of two layout items you are giving the anchor layout a constraint to fulfill. In the simplest case you are just saying that the distance between the two layout edges should be a fixed distance. You can do this with the left, right, top, bottom and the “center edges” of any layout item. You can also have an edge that has several anchors connected to it. This is not possible with linear layout and it is partly possible with a grid layout. This allows you to build up the structure the layout should have. The layout will then try to fulfill the constraints you have given it by potentially growing or shrinking the layout items. You can also configure each anchor to grow or shrink if that is preferred.

The API for configuring the layout structure reflects this concept. The addAnchor() function is all that you need to configure the structure of the layout, but we added some convenience functions for anchoring corners (surprise, addCornerAnchors()) and for fitting the size of an item to another (addAncors()). Thus, the following code will ensure that the left edge of item findEdit is always connected to the right edge of the item findLabel. In this case it means that there will typically be a bit spacing between the items, and the spacing will always be the same.

   anchorLayout->addAnchor(findLabel, Qt::RightAnchor, findEdit, Qt::LeftAnchor);

You can also anchor an item to a layout edge. If the edge of an item is anchored directly or indirectly to a layout edge, it’s geometry will react to changes in the layout geometry:

   anchorLayout->addAnchor(anchorLayout, Qt::LeftAnchor, findLabel, Qt::LeftAnchor);
   anchorLayout->addAnchor(findLabel, Qt::RightAnchor, findEdit, Qt::LeftAnchor);

Of course, you can also anchor items the same way in the vertical dimension.

Now, with this brief introduction we can go back to Qt Linguist’s “Find Dialog” example as I gave above.
It turns out we can make the same arrangement with just one anchor layout.

This is how the code could look like:

    QGraphicsAnchorLayout *l = new QGraphicsAnchorLayout;
    QGraphicsWidget *w = new QGraphicsWidget(0, Qt::Window);
    w->setLayout(l);

    // label, line edit and "Find Next" button
    l->addCornerAnchors(l, Qt::TopLeftCorner, findLabel, Qt::TopLeftCorner);
    l->addCornerAnchors(findLabel, Qt::TopRightCorner, findEdit, Qt::TopLeftCorner);
    l->addCornerAnchors(findEdit, Qt::TopRightCorner, findButton, Qt::TopLeftCorner);
    l->addCornerAnchors(findButton, Qt::TopRightCorner, l, Qt::TopRightCorner);
    // group box
    l->addCornerAnchors(l, Qt::BottomLeftCorner, groupBox, Qt::BottomLeftCorner);
    l->addCornerAnchors(groupBox, Qt::TopRightCorner, findEdit, Qt::BottomRightCorner);
    // cancel button
    l->addCornerAnchors(findButton, Qt::BottomLeftCorner, cancelButton, Qt::TopLeftCorner);
    QGraphicsAnchor *anchor = l->addAnchor(cancelButton, Qt::AnchorBottom, l, Qt::AnchorBottom);
    anchor->setSizePolicy(QSizePolicy::Minimum);
    l->addAnchor(cancelButton, Qt::AnchorRight, l, Qt::AnchorRight);

    // Done!
    scene.addItem(w);

Note, for simplicity the code does not populate the group box with items, since that is easier done with a grid layout. Thus, the anchor layout reduced 4 layouts into one.

There’s a lot of interesting things I could mention here, but I would then be just repeating what’s already written in the documentation, so if you are interested I suggest you take a look at that.

Conclusion

The anchor layout has some attractive properties. For some setups it provides a more intuitive API that is closer to how the UI designer would specify it. It is also very flexible: You can construct free-form graphs of anchors where one edge might be anchored to several other edges. You can modify size policy of each anchor, which means there is little use for a spacer item. You can also modify the spacing to be both a positive and negative value.

Of course, the anchor layout is not the silver bullet and in a lot of cases you are better off with just using a linear or a grid layout. The anchor layout is a flexible layout, and there is a price you have to pay for that: For some arrangements it can be very verbose compared to a linear layout, since you don’t have to anchor each item together (the “anchoring” is implicit by the order of the items).

Read the rest of this entry »

For all you 3D and graphics hackers out there this will not be news:  writing OpenGL code is a pain.

Well, the Qt Graphics team is coming to save your sanity with the a new project called Qt/3D.

We teased about Qt/3D by putting a few of the foundations for it in Qt 4.6, which will be released very shortly. See the Qt/3D 4.6 features blog post for more details.

At some point Qt/3D will be available as part of Qt itself - exactly what sort of module or library we are not sure just yet - but for now you can try it out via Qt Labs!

With this post we’re pleased to announce that Qt/3D will be available for experimental use via the new Qt/3D labs repo.

Old School OpenGL code gets the Qt Treatment

The trusty old QGLWidget got you past first base: a nice window set up with a OpenGL context ready to go.

But from there you’re on your own with the OpenGL reference book, tearing your hair out writing code like

void My3DWidget::paintGL()
{
   QColor clearColor(palette().color(backgroundRole()));
   glClearColor(clearColor.redF(), clearColor.greenF(), clearColor.blueF(), clearColor.alphaF());
   glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);
   QColor color(170, 202, 0, 255);
   glColor4f(color.redF(), color.greenF(), color.blueF(), color.alphaF());
   static float vertices[] = {
      60.0f,  10.0f,  0.0f,
      110.0f, 110.0f, 0.0f,
      10.0f,  110.0f, 0.0f
   };
   glVertexPointer(3, GL_FLOAT, 0, vertices);
   glEnableClientState(GL_VERTEX_ARRAY);
   glDrawArrays(GL_TRIANGLES, 0, 3);
   glDisableClientState(GL_VERTEX_ARRAY);
}

just to paint a triangle on the screen.

But then if you want cross-platform code - something that you can try on your desktop, and then run on your device with OpenGL ES, it starts to look really horrible!

Macros everywhere to cope with the different function signatures and data types - not to mention shaders under ES 2.0 versus classic GL on the desktop, and a swath of other cross-platform difficulties.

With Qt/3D your code looks like this:

void My3DWidget::paintGL()
{
    QGLPainter painter(this);
    painter.setClearColor(palette().color(backgroundRole()));
    painter.clear();
    painter.setStandardEffect(QGL::FlatColor);
    painter.setColor(QColor(170, 202, 0, 255));
    QGLVertexArray vertices(QGL::Position, 3);
    vertices.append(60.0f,  10.0f,  0.0f);
    vertices.append(110.0f, 110.0f, 0.0f);
    vertices.append(10.0f,  110.0f, 0.0f);
    painter.setVertexArray(vertices);
    painter.draw(QGL::Triangles, 3);
}

And what’s more it runs the same on your OpenGL/ES device and your desktop.  (Note that I have elided the view and model transform setup code from both examples above for the sake of space).

As mentioned in the previous blog post Qt/3D has been in the wings for some time now, and the eagle-eyed might have notice math classes springing up in Qt’s GUI module.

These classes provide the basis for Qt/3D’s cross platform geometry abstraction: QGLVertexArray. This nifty class also dovetails into the QGLBuffer class to take care of uploading your geometry to VBO’s on the graphics adaptor, as well as coping with differences in platform on data member sizes.

Download the code from the labs repo and try out the examples - the code above comes from the tutorials directory, where you can find out more about writing your traditional OpenGL apps in the Qt cross-platform way.

Whats in Store with Qt/3D

There’s more to come from Qt/3D over and above the Portability tools mentioned in the example above.

With Enablers are included encapsulation classes like QGLMaterialParameters to encapsulate OpenGL materials in a cross platform and Qt’ish way.

One of the nicest enablers is the QGLView class and its friends.  Doing your GL painting into a view looks pretty much exactly the same as with an OpenGL widget, but a few more things are taken care of for you - no need to set up tricky viewing and model transforms (which is one reason why I elided them from the code above).  But even better as a bonus you get a pan-rotate-zoom view window for free.   Its customizable using the QGLCamera class, and with QGLLightParameters you can quickly set up your own lights too.

Then there’s Real 3D bringing basic but powerful geometry management, and model file import functionality. With this stuff we’re just dipping our toes into the world of 3D to allow coding up of basic applications using Qt style containers, QObject based memory management, and the kinds of abstractions you’ve come to expect from Qt. If you’re an Ogre programmer, or used to using Coin3D or CrystalSpace or other powerful 3D and modelling libraries - well, you’ll still need them. We’re not planning to go into competition with those established 3D toolkits.

Instead our aim is to deliver on the promise of Qt: do more with less.  It should be just as easy to use a 3D model file as it is to use a PNG file, and it should be just as easy to set up a cube with a texture on it as it is to create a Qt label. We call this component of Qt/3D Real 3D because it does start to provide functionality we’re used to seeing in 3D toolkits. But we’re working to be sure we do not go too far to go down this road, and thus to decide what will go in and what will left out - so please consider the stuff in our labs release as definite maybes.

QML and Qt/3D

There’s a lot of buzz around Declarative UI and its associated language QML.

Qt/3D will work with Declarative UI by providing QML bindings so 3D functionality can be easily used from Declarative UI programs. There’s a few demos of this in the source tree which can be tried out and you can see Henriks short video about QML and Qt/3D.

We’ll expand on the exciting possibilities of QML and Qt/3D in later posts.

We hope you like what we’re planning, and look forward to your feedback - keep tuned as there are more blog posts to follow, with some cool examples and things to try with Qt/3D.

Lately I’ve been working on a couple of exciting customer projects, involving HTML5-based UIs on embedded (TV) platforms. Due to the fabulous work some of my colleagues put into QtWebkit and QtDirectFB, and the ongoing work from MIPS Technologies on their Webkit JIT, the paint performance is very good, and Javascript is getting faster and faster.
However, while working on these projects, I’ve ran into a few issues that were not solvable by JIT-accelerated JavaScript or by hardware accelerated painting.

The main problem was that when you write a dynamic UI for a full application, the complexity starts to be difficult to manage. I started by dividing the app to a few HTML pages, but that of course created unacceptable delays when switching between the pages. So I looked online for dynamic web-app solutions. Those include the amazing Javascript libraries Dojo and JQuery. The problem with those is that they manage the complexity by selecting elements with custom-built CSS-like selectors (which are slow) and by modifying the HTML DOM tree in runtime (which is slower).

An alternate way to handle HTML performance is with the HTML5 canvas element (which is nicely accelerated) - the problem is that canvas API is currently too low level, and the widget libraries on top of it are not mature yet, definitely not in time for my customer projects.

So I asked myself: what components are really needed in order to make a dynamic UI?
Looking at the Qt Kinetic project, I need 4 main elements:

• Animations
• State Machine
• Graphic effects
• Declarative Syntax

Now HTML and CSS are already a declarative syntax, and CSS3 (supported in webkit) contains all the animations and graphic effects I currently need. The missing element, which is also my personal favorite, is the State Machine.

Why do I keep going on and on about state-machines?
If you think about it, both CSS, HTML, and Qt widgets are optimized to handle complexity in space. State machines, however, are optimized to handle complexity in time. An application can be dynamic only if it’s rich both in space and in time. That’s why to me state-machines are of outmost importance for dynamic UIs.

Enough philosophy, what’s this about?
Now I’ve had the pleasure to work with the SCXML standard in the last year, both by prototyping the Qt-SCXML library (which later helped with the evolution of the Qt State Machine Framework), and by joining the w3c group that defines the SCXML standard.
While working on my latest HTML-based project, I was thinking - what if I could define the flow of my web-app with an SCXML statechart? Wouldn’t that allow for managing the complexity of my web app, without incurring the performance overhead of modifying and re-modifying the DOM?

Introducing Statechartz
So I took the time to write an SCXML library for Javascript. I basically had to copy the SCXML algorithm to Javascript syntax, which took a while, but has already saved me hours of work by having the code of my web-apps cleaner and smaller.
The result can be found at the statechartz branch on the SCXML lab GIT repo., and the demo can be viewed online. Note that the library is currently only tested on webkit-based browsers (QtWebkit/Arora, Safari, Chrome), but porting it to other browsers shouldn’t be too difficult.

Looking a bit at the HTML code of the demo, you’ll notice the following:

<link rel="statechart" href="flow.scxml" />

This is the line that connects an HTML page with an SCXML statechart, defining the page’s flow. The state-machine would start running as soon as the page is loaded.
But hey! rel=”statechart” isn’t standard!

Well, I was surprised to learn that HTML5 allows you to add your own link types, which become legal if you register them in the WHATWG Wiki, which I did; so now <link rel=”statechart” href=”somefile.scxml” /> is completely standard
Though using the link tag would cover most cases, it’s completely optional and the Statechartz library can be used more dynamically, either by loading an SCXML file whenever wanted:

var myStateMachine = Statechartz.loadScxml(someURL);

var myStateMachine;
with (Statechartz) {
  myStateMachine = build(
    State("root",
        Entry(function() { doSomething(); }),
        Parallel("s1",Initial,
             Exit(function()(doSomethingElse(); }),
             Transition(Event("EVENT.QUIT"), Target("exit"))
        ),
        Final("exit")
    )
  );
}

<div id="screen_weather" class="screen">
      <script type="text/javascript" src="..."></script>
</div>
<div id="screen_shopping" class="screen">
    <iframe src="..." height="240" width="320" scrolling="no" style="overflow:hidden" frameborderwidth="0"></iframe>
</div>
<div id="screen_calc" class="screen">...</div>

#screen_calc {
  left: -350px;
  -webkit-transition-duration: 600ms;
  opacity: 0.5;
}
.state_calc #screen_calc {
  left: 0px;
  -webkit-transform: rotate(360deg) scale(1);
  opacity: 1;
}

The Demo, and performance
The demo that comes with the statechartz library demonstrates 3 uses of the Statechartz library:

• Dynamic UIs with states and transitions (switching between the 3 widgets)
• Managing UI complexity (e.g. the instructions popup)
• Business logic (the calculator ‘engine’)

Note that the demo specifically is too animation-heavy to run well on embedded platforms, and of course webkit could use some more optimizations. but the concepts (don’t mess with the DOM, use Statecharts to manage time) are the same, and are working on other webkit-based projects with a cvery nice performance gain.

The next step on the SCXML path
After this project, SCXML is now supported in 3 implementations: Qt C++ (with scc), QtScript (with QScxml), and web (with Statechartz). The next natural step is a statechart graphical tool (Creator plugin) that can help author SCXML files for all of those 3 implementations. Though it’s not on our roadmap, some of us (including the undersigned) were working on prototypes, and any help from the community would be appreciated!

Links
The code: http://qt.gitorious.org/qt-labs/scxml/trees/statechartz
The live demo (webkit browsers only): http://labs.trolltech.com/blogs/wp-content/uploads/2009/11/demo.html
Video capture of the live demo: http://labs.trolltech.com/blogs/wp-content/uploads/2009/11/statechartz-capture_0001.wmv

At the Munich DevDays we got a lot of questions about QML - first among them was where to download it. To celebrate DevDays in San Francisco we are releasing binary snapshots that you can download now for Windows, Mac and Linux.

For those of you who couldn’t attend DevDays, QML helps you create exciting UIs like this:

When Qt 4.6 is released, we’ll make another more up to date package. As we approach this milestone we’re focusing our attention on writing tests, squashing bugs, updating docs and giving the examples a little love - so by Qt 4.6 QML should be even better!

Do you know the main advantage of a Hummer? It can go pretty much everywhere, that’s why many armies are using it. If I talk about a Hummer it is because QGraphicsView can go pretty much everywhere too. Recently I was lucky enough to get a N900 (generously given by Jesper, don’t know for how long though) so I decided to test Qt on it. I mainly work on QGraphicsView here at Qt Development Frameworks so it was a way to test the speed of my toy (:p). Since I’ve also been a KDE user and developer for quite a bit, I thought I could try KDE on it and more precisely Plasma.

So I downloaded the Maemo 5 SDK and started building KDE and Qt with it.

First thing you have to know is that scratchbox is not easy to set-up because you always face problems and compiling inside scratchbox is really slow. The link step is a bit broken and you have to specify manually where are the libs you link with (apparently it’s expected to be like that). It seems odd… Anyways, after a couple of hours I had Qt 4.6 (Fremantle branch) up and running. I was quite happy to see that it performs well (if you are using the right things). Animated tiles are running smoothly and painting demos as well (using opengl es2 graphicssystem).

After this initial success I tried KDE. This was more painful than I originally thought. Broken packages (mysql-server), out-dated packages (shared-mime-info for instance) and the worst : CMake crashed during the configure step. I solved that by downloading the 2.8 RC version that I built myself. But after a couple of hours of fighting and debugging I got this:

Link to youtube video

As you can see Plasma is running fine on this device. Of course a lot of work is needed to make it “finger” enabled (the actual applet handles are not really appropriate) but it’s a good start and Plasma is flexible enough to allow that. I have created a separate “shell” called plasma-mobile (Plasma already has the desktop and netbook shell) and I pushed that on the KDE playground. It also needs a lot of work to integrate well with the device (battery, network, profiles and so on) but the goal is to invite people to contribute to Plasma in order to offer a real alternative to the hildon-desktop. Plasma already has many many features/applets that we can just use on the N900. This also comes with a global effort to bring KDE technologies to the device as Kevin said in his blog. You can also participate in discussions on the KDE mailing-list for Maemo.

So yes Plasma is also a hummer, it runs everywhere but it’s the luxury version.

So, are we in the future yet? It is 2009. Where are all the flying cars and personal jet-packs ? Surely we would all be wearing metallic outfits (complete with Employee Visualization Appendage) by now.

The Future Is A Canvas

Back in 2006 a little bit of the future arrived when QGraphicsView was introduced, we were aiming to replace our old QCanvas API with something more flexible and powerful.
It turns out that it is great for creating dynamic, smooth and animated GUIs. Over time it has become the backbone for next-generation user-interface projects like KDE’s Plasma and Qt Declarative UI.

Yet, for all its dynamic, smooth goodness, there are some missing pieces in the puzzle. People are creating full user interfaces in graphicsview, but Qt offers only the most basic widget types to help them out. In many cases it is enough to offer basic primitive types. But if you want to integrate with the platform you are running on or want to use more complex widgets, it becomes difficult.

The Shiny New Stuff

This is why we have started a research project aimed at providing a basic widget-set based on QGraphicsItem. It is a natural extension of the Next Generation Itemviews project. We will take the opportunity to look at how to improve the widget API, internal structure and how we do widget styling and theming.


HotWheels by Leap Kye on flickr

One of the lessons we have learned is that graphics hardware is generally happier when moving pixmaps around, than when drawing lines and shapes again and again. This has implications for the styling API, since our current QStyle is based around drawing complex controls imperatively. It would be better to build our widgets as a graph of (cached) primitive elements that can be transformed and updated individually if necessary.
Similarly it also makes sense to make the input regions in the widget into separate elements in that scene graph.

But, for all our fascination with the new and shiny, we should not forget that our existing widgets contain years of accumulated experience in the form of code that handles all sorts of weird and wonderful corner cases. We don’t want to go though the pain of re-discovering all these cases. It makes sense to re-use existing logic and behavioral code by moving this code into classes that can be used by both the old and the new widgets, when possible. This will also allow us to directly test and verify the internal widget logics independent of the widget front-end.

Still Kinkin’ It Old School ?

We are not forgetting our QWidget based widgets. They will get some initial benefit of better tested back-ends and hopefully some code cleanup. I also have some ideas that will drag QWidget kicking and screaming into the future, but I’ll leave those for another blog-post. Always leave the audience wanting more.

Talk To Us

Even though we are currently working full time on getting Qt 4.6 out the door, the widgets-ng repository is already public and we are already hanging out on #qt-widgets-ng on irc.freenode.net .
I will be attending the Qt Developer Days in München next week, and there will be several developers from the Qt Widgets Team attending the San Francisco event too. Feel free to discuss anything widget related (or any random topic, really) with us. See you there!