Some people have been asking how to embed Qt painted content (and especially Qt widgets) in a 3d scene. As I’ve been wanting to do this ever since we added QTransform supporting fully projective transformations, I sat down and wrote a small example demonstrating these capabilities. With no further delay, I present WolfenQt:
Here’s a screenshot as well for the Youtube-challenged:
A Wolfenstein like maze theme was chosen because of ease of implementation, you could of course embed a widget onto any 3d surface. The trick is to create a 3d transform mapping from a 2d plane (where the widget or other Qt drawn graphics resides) onto any quad in 3d space, and then do a perspective projection, before converting the 3d transform to a QTransform. Now, by making all the widgets and wall segments graphics items, you only need to use QGraphicsItem::setTransform() with the custom created transform and QGraphicsView will handle all event translation for you. The way the camera movement works is by recomputing and resetting every item’s transform on each frame. To avoid having to continuously redraw all the widgets if they’re not being updated, QGraphicsItem::setCacheMode(QGraphicsItem::ItemCoordinateCache) is the way to go.
I’ve embedded Qt’s media player demo to show the ability to embed videos in a QGraphicsView as tbastian blogged about just recently: Videos get pimped
I’ve also embedded the 3d .obj model viewer I blogged about earlier (Accelerate your widgets with OpenGL) to show how to mix in OpenGL content by setting up the correct projection and modelview matrices. Note that except from this 3d model everything else is drawn using QPainter with QTransform, both the walls, sprites, and widgets.
Now, this example might not be the most useful thing in the world, but it’s meant as a demonstration of how you can stretch the Qt API in new directions.
Want to play with the source? It’s available through our gitweb at http://labs.trolltech.com/gitweb?p=WolfenQt;a=summary
Note that the performance against Qt 4.5 is better than against Qt 4.4, so if you try it out you might want to use the 4.5 snapshots at the moment. For the very best performance run with the “-graphicssystem raster” option (So long and thanks for the blit!). Also, there seems to be some bugs regarding certain widgets not being shown on Windows/Mac. Left-clicking links in embedded QWebViews is also broken at the moment.
41 Responses to “Widgets enter the third dimension: WolfenQt”
My 2 cents:
I’d be very careful to use the *CoordinateCaches at the moment with Qt4.4 on X11 and OpenGL.
This way the widgets will end up being rendered with the old raster engine in XPixmap, that will be converted to QImage, format converted to GL format, mirrored (for no relevant reason), then uploaded to texture (mipmaps pulling the handbrake here). Texture cache cleaning always shows up in profiler, not the most efficient implementation either.
After all, it might be (and it is in my case) much much more efficient to disable every kind of caching. Really waiting for decent Qt4.5 
bence: Correct, which is why I recommend the raster graphicssystem where the default QPixmap backend is a QImage. We’ve improved texture upload performance in Qt 4.5 as well by using PBOs. Btw, we don’t do automatic mipmap generation when calling drawImage/drawPixmap, only when doing bindTexture(). Ideally we should make this optional at some point, both in the QPainter and the bindTexture() cases.
Um, isnt WolfenQt/mediaplayer/main.cpp omitted?
how about spending time making the widgets not suck
Uku: Nope, you’re not supposed to build mediaplayer separately, it’s used as part of WolfenQt
dsf: The widgets will look different based on which style you’re using, but yeah, they could need some more decoration (and be made to blend better in with the other artwork). Feel free to pimp them
@”dsfdsdsf”: How about spending time leaving more productive comments, for instance by actually describing what you think could be improved, rather than simply being offensive?
So the infinite viewing portal bit where you can create instance upon instance, you need to finish that by having the last portal be a view of a life feed web cam looking at you & your monitor.
Neat stuff : )
This is simply awesome.
That’s what I’ve been waiting for at least 2 years.
It opens the gates for tons of new stuff.
Now two questions remains:
- Are the performance good?
- Will this ever be possible in Direct 3D?
From my point of view that’s a new milestone for Qt.
Congrats sroedal.
PS: Someone has to fix the QWebView left click in QGraphicsView http://trolltech.com/developer/task-tracker/index_html?method=entry&id=209745
Benjamin: The performance is surprisingly good if you use the item coordinate cache mode and make sure that all the widgets aren’t updating all the time. Currently even the walls/floor/ceiling are graphics items, but in only the widgets would have to be. It could be possible to use pure OpenGL/Direct3D drawing and just use graphics view for event translation, or maybe do event translation on your own as well if you have specific requirements. I don’t see why you couldn’t do something like this in Direct3D but it might require more work and doing more stuff outside of graphicsview than with OpenGL.
Sroedal, this is awesome, it stilll needs a bit of tweaking but is already impressive, congrats
Great
Next step should be ability for the viewer to fire bullets that create events such as button presses upon hit on dialogs *hehe*
This rules. I can’t stop watching it
Holy Smokes
WOW !
That has blown me away.
Thanks mate !
I like the sentence: … , I sat down and wrote a small example demonstrating these capabilities.
BR and keep it up
paines
Nice work! We have been trying to do this and were having problems with the 2d->3d transformations. your example rocks!
Wow…
very nice
Inspiring.
maybe this will be the future of the OS
Wow, one of the coolest demos I have seen.
Damn you just raised the bar here for new demos Samuel! 
I love how the volume is controlled by the proximity to the mediaplayer.
It looks cool. Here is my (lame?) question: is 3D acceleration needed to run this at acceptable speed? If I’m right, a projective transformation adds only a constant factor of overhead to graphics instructions, so it should be feasible on today’s CPUs. I would be happy if it ran without a video card with 3D acceleration and (many times closed source) official driver for that card.
Thanks for all the nice comments guys
Grósz: If you turn off SmoothPixmapTransform and the linear gradient shading and tweak it a bit so that it doesn’t draw items that are hidden beneath other items it’s possible to get it to run at an acceptable speed, but you really want hardware acceleration for this so that your CPU isn’t completely swamped and is free to do other stuff.
> Also, there seems to be some bugs regarding certain widgets not being shown on Windows/Mac.
Looks like on Mac + Qt 4.4, it does not work totally, I have only Wolfen, but no embedded widgets. Need to wait 4.5 then. What is the especially about the Mac widgets?
Uku: We’ve pinpointed and fixed the missing widget problem on Mac and Windows by a workaround in the example code. It turned out that the proxy items got the position (0, 0) by default only on Linux.
Gah! That’s a bug; the position should be (0, 0) on all platforms!
Very nice indeed. This should definitely get added in to the demo apps that ship with Qt.
The bad 3d graphics, ridiculous music and video, and Wolfenstein. You’ve got to be kidding me! You made my day :).
Neat ideas though. Qt 4.5 is going to be excellent.
I have a question, when running the demo, I click on a link in a web widget and nothing happens… I can only open links by right clicking and selecting “Open Link” Is this a web widget issue, and if so, is there a workaround?
I’m glad y’all are working on this future stuff however, I need to make my Qt-4.4-Mac application fully compliant with Apple’s Human Interface Guidelines and a handful of known bugs makes this impossible. No one at Nokia is fixing these few bugs. How about working with QtDesigner or QtCreator getting it to follow the AHIG. How about a little love for the Mac.
This is great! Please keep developing this. Three suggestions: first, the ability to choose the look of the environment; a brick dungeon isn’t exactly what I would like to be in. Second, the ability to choose the character, other than a Nazi soldier. An animal would be nice (although if I remember correctly, the only animal in Wolf3D was a German shepherd). Third, it would be great if the character actually did something. It could “say” (via a speech bubble) pop-up messages that normally appear above the systray, or if you click on it it would display a menu, things like that.
Nice !
You can also find a firewall based on a modification of wolfotrack at http://software.inl.fr/trac/wiki/Wolfotrack
wes: It seems that the fix for the left-clicking in embedded web views was fixed in our mainline branch but not in Qt 4.5. I expect the fix to be in Qt 4.5 pretty soon though.
Peter: Good suggestions, the environment and character look is just about changing which texture images to load. It shouldn’t be too hard to implement speech bubbles for the character either, using QPainter::drawText() with a bounding rect
This is amazing, appears very intuitive. Could be the shopping interface of the future.
Although watch out for Microsoft Bob. 
I’ve working in a way to embed Qt widgets in a 3D scene since qt 3.3. When Qt 4 appears it was a great change and everithing starts working faster and better. You Qt guys have done an incredible job
If you want to have a look, i’ve developed a simple toolkit library to enable qt widgets in 3d scenes if you want some screenshots and videos you can found it at http://www.artificialideas.com/undefinedsymbol/node/12 (the screens are quite old). I will make a new release of the toolkit as soon as i can :P.
Actually i manage al the rendering of the 3D windows. Basicaly, I capture all the repaints events using an event filter to build a 2D opengl texture. As im working on 3D interaction all the interaction is done using a virtual ray instead of a 2D cursor. Notice than the rendering is not managed by a QGraphicsView it only needs a opengl display window.
This is quite cool. Actually, it would be interesting to monitor all people via maze >:)
or play different games… and you walk over to the next game hehe
btw the captcha here is very annoying 
Nice demo! Have a look at the scenarios at opencroquet and qwaq what they are already doing using 3D-environments. While the do have a diffrent technical approach (Smalltalk/Squeak) the developers at qwaq offers a nice OpenOffice-remote-screen inside Croquet..
just my 2 cent
That’s really awesome… Qt is sooooo powerfull!!!
I got the qt-snapshot from git
git clone –depth 1 git://labs.trolltech.com/qt-snapshot
then enabled the opengl
./configure -v -no-accessibility -no-exceptions -no-qt3support -nomake examples -nomake demos -opengl
/usr/bin/ld: cannot find -lGL
collect2: ld returned 1 exit status
I saw an error and i needed to get the nvidia dev files
sudo apt-get install nvidia-glx-177-dev
all worked ok
make ; sudo make install
git clone git://labs.trolltech.com/WolfenQt
cd WolfenQt
export PATH=”/usr/local/Trolltech/Qt-4.5.0/bin:$PATH”
/usr/local/Trolltech/Qt-4.5.0/bin/qmake
make ; sudo make install
All went ok on intrepid and even tried to record it
with recordmydesktop gtk but didn’t worked
what did you used to record the movie ?
mariuz: I used recordmydesktop but you have to enable “Full shots at every frame” for it to work with OpenGL windows.
Thanks now i record
One note the flash plugin is not enabled
in arora for example works (using webkit from qt4.5)
mariuz: Flash won’t work in graphics proxy widgets at the moment. The flash plugin works by creating a separate native window and drawing into it, thus it won’t work with the kind of painting redirection we are doing here. Windowless plugins might work, though I haven’t tried it.