For the last year, we have been investigating API’s that Qt needs to support 3D applications and clever 2.5D effects with OpenGL. When we started all this a year ago, the problem was broken down into three main areas:
- Enablers - Basic building blocks like matrices, shaders, vertex buffers, etc.
- Portability API - API’s that make it easier to write code that ports between desktop OpenGL and embedded OpenGL/ES. Particularly OpenGL/ES 2.0 which does not have a fixed function pipeline.
- Real 3D - API’s that take Qt into new application spaces beyond animations and 2D effects.
Obviously that covers a lot of ground, so in this post we will just focus on a few of the Enablers - specifically the ones that made it into 4.6 as the first taste of Qt/3D. In future posts, we’ll publish Qt/3D repository details and show you more of our plans for later Qt/3D releases.
Math3d
Traditionally, Qt has relied upon the OpenGL library to provide mathematical primitives, using functions like glOrtho(), glRotate(), and so on to manipulate matrices and vectors. However, with the advent of OpenGL/ES 2.0 it is no longer possible to rely upon the OpenGL library to do the heavy-lifting - the programmer has to do all the work. Also, the traditional OpenGL functions are really only useful when drawing objects - they aren’t of much use when building object meshes in memory and transforming them prior to uploading to the GPU.
So we really needed a hardcore 3D math library, just like the other 3D toolkits (Coin3D, Ogre, OpenSceneGraph, etc). But we didn’t want to go overboard - it is very easy to re-invent all of linear algebra and lose sight of the core goal: make typical 3D mathematical operations fast and elegant. We recognized that libraries like Eigen were very good at doing everything in mathematics, but our own goals were more focused. So what did we do?
The central workhorse is of course QMatrix4×4, which is highly optimized for 3D operations. Internally it keeps track of its “type” - whether it is a translation, scale, rotation, etc - so that it can more efficiently build up transformations than a naive “make matrices and multiply” implementation might. QTransform does the same thing for 2D transformation matrices. The following is an excerpt from the hellogl_es2 example in Qt 4.6 which builds up a modelview matrix and sets it on a shader program:
QMatrix4x4 modelview; modelview.rotate(m_fAngle, 0.0f, 1.0f, 0.0f); modelview.rotate(m_fAngle, 1.0f, 0.0f, 0.0f); modelview.rotate(m_fAngle, 0.0f, 0.0f, 1.0f); modelview.scale(m_fScale); modelview.translate(0.0f, -0.2f, 0.0f); program1.setUniformValue(matrixUniform1, modelview);
As can be seen, it is very similar to the traditional OpenGL functions:
glRotatef(m_fAngle, 0.0f, 1.0f, 0.0f); glRotatef(m_fAngle, 1.0f, 0.0f, 0.0f); glRotatef(m_fAngle, 0.0f, 0.0f, 1.0f); glScalef(m_fScale, m_fScale, m_fScale); glTranslatef(0.0f, -0.2f, 0.0f);
The choice to make the functions similar was deliberate: code that uses the existing OpenGL functions can be quickly converted into more portable code that uses QMatrix4×4.
The QGenericMatrix template is used for creating “other” matrix sizes that commonly crop up in OpenGL work: 2×2, 2×3, 2×4, 3×2, 3×3, 3×4, 4×2, and 4×3. It can do a lot more of course, being a template, although we did draw the line at supporting sparse matrices - the matrix sizes that occur in 3D code are rarely very large. A common question is why didn’t we make QMatrix4×4 an instance or subclass of QGenericMatrix. The main reason is performance - the 4×4 class needs to be very fast and it is easier to performance-tune a concrete class that isn’t at the mercy of the compiler’s template expansion system. The other reason is to reduce user confusion - the API’s for all QGenericMatrix sizes is exactly the same, but QMatrix4×4 is extremely rich in the additional operations it provides.
QVector2D, QVector3D, QVector4D provide vector classes of various sizes to complement QMatrix4×4. An interesting feature for the purposes of OpenGL is that these classes are guaranteed to use the same floating-point type internally as GLfloat on the system. QPointF wasn’t suitable for our 2D vector needs because it uses qreal, which can either be float or double depending upon the compilation flags passed to Qt’s configure. The GLfloat guarantee is very important when building large 3D object meshes: you want to get the vertex data into the most efficient format as early as possible. If we had made the internal type qreal, then Qt/3D would have needed to do a lot of floating-point conversions when uploading vertex data to the GPU.
The QQuaternion class is the last in our current math3d set. It provides an efficient implementation of rotations in 3D space for use with camera positioning, rotation, and animation.
Shader Programs
The fixed function pipeline in OpenGL is getting very “old school”. These days, OpenGL is all about shaders, shaders, shaders. But resolving the extensions and managing the compilation, linking, and use of shader programs can be quite daunting. In Qt 4.5, we had no less than three different internal shader program wrappers for pixmap filters, the OpenGL2 paint engine, and the boxes demo. So in Qt 4.6 we have merged all of these efforts and devised a new public API to wrap the extensions. The result is the QGLShader and QGLShaderProgram classes, which:
- Support the GLSL and GLSL/ES shader languages.
- Handle vertex and fragment shaders (geometry shaders are coming in future versions of Qt).
- Support writing portable shaders that work on both GLSL and GLSL/ES.
That last point is probably the most interesting for Qt. GLSL has a lot of built-in variables like gl_Vertex, gl_Normal, gl_ModelViewProjectionMatrix, etc that don’t exist in GLSL/ES. In turn, GLSL/ES has additional type qualifiers like highp, mediump, and lowp that are used to specify the desired precision. These issues can make it a pain to port existing shader code from desktop to embedded. We didn’t want to have to write two sets of shaders for the OpenGL2 paint engine, so a solution needed to be found.
The solution we chose was to use GLSL/ES as the primary language for writing shaders in Qt, and provide #define’s for the extra keywords to make the code compile on desktop GLSL systems. It is still possible to use the full GLSL language if you want to, but portability will suffer.
The following example demonstrates how to compile and link a simple shader program that can be used to draw triangles with a flat color:
program.addShaderFromSourceCode(QGLShader::Vertex,
"attribute highp vec4 vertex;"
"attribute mediump mat4 matrix;"
"void main(void)"
"{"
" gl_Position = matrix * vertex;"
"}");
program.addShaderFromSourceCode(QGLShader::Fragment,
"uniform mediump vec4 color;"
"void main(void)"
"{"
" gl_FragColor = color;"
"}");
program.link();
program.bind();
int vertexLocation = program.attributeLocation("vertex");
int matrixLocation = program.attributeLocation("matrix");
int colorLocation = program.uniformLocation("color");
The highp and mediump keywords are added to keep GLSL/ES happy - on desktop they #define to an empty string. Also, we have deliberately used user variables for the vertex position, matrix, and color rather than relying upon the desktop-specific gl_Vertex, gl_ModelViewProjectionMatrix, and gl_Color variables. We can then draw a green triangle as follows:
QVector3D triangleVertices[] = {
QVector3D(60.0f, 10.0f, 0.0f),
QVector3D(110.0f, 110.0f, 0.0f),
QVector3D(10.0f, 110.0f, 0.0f)
};
QMatrix4x4 pmvMatrix;
pmvMatrix.ortho(rect());
program.enableAttributeArray(vertexLocation);
program.setAttributeArray(vertexLocation, triangleVertices);
program.setUniformValue(matrixLocation, pmvMatrix);
program.setUniformValue(colorLocation, QColor(0, 255, 0, 255));
glDrawArrays(GL_TRIANGLES, 0, 3);
program.disableAttributeArray(vertexLocation);
Note the use of QMatrix4×4 above to create an orthographic projection matrix to pass to the vertex shader, and the use of QVector3D to build the vertex array. And that’s basically it! Shaders 101.
What’s Next?
Lots and lots of stuff. Wrapper classes for vertex buffers and textures will probably go into Qt in the near future. Geometry handling for building object models. Special-purpose 3D viewing widgets. Integration with Declarative UI for quickly building 3D applications. And the portability API. More to come on these in the next post …
32 Responses to “Qt/3D features in Qt 4.6”
great, was missing more elaborate information in the documentation and at the qt dev days.
You need to have a talk at the next dev days!
One other thing to think about for the future: there’s a few well established 3D APIs out there (Coin3D, VTK, …). Will Qt work with them? Replace them?
This looks very good. Where does OpenCL falls in all this ? You talk about hardcore 3Dmath library. GPU computing is more than hardware 
I believe it could really differentiate from all the other Toolkit.
respect!
From your triangle example in 3D:
QVector3D triangleVertices[] = {
QVector3D(60.0f, 10.0f, 0.0f),
QVector3D(110.0f, 110.0f, 0.0f),
QVector3D(10.0f, 110.0f, 0.0f)
};
Why are the vertices of the triangle called QVector3D, as they are not vectors, but points. So would a name like “QPoint3D()” be more meaningful, or have I missed something.
I know you want to keep the API small but perhaps offering a spherical coordinate system might be beneficial especially for GIS applications
–Why are the vertices of the triangle called QVector3D, as they are not vectors, but points?–
In computer graphics a vector changes it’s meaning depending on its context, points and normals for example. Also in openGL, points/normal/vectors are converted to 4 components: x, y, z and w, which is usually 1.
Is there any plans for supporting OpenGL 3.x anytime soon? I could see Qt/3D features playing very nicely with it.
Hi, What are the supported platforms for this Qt/3D? Will there be support for Windows Mobile / Windows CE?
mkrus: “One other thing to think about for the future: there’s a few well established 3D APIs out there (Coin3D, VTK, …). Will Qt work with them? Replace them?”
Many of the established toolkits already work with Qt (Coin3D has the SoQt wrapper for example). It would take a lot to replace them and we’re not sure we would want to. Qt needs better OpenGL support libraries than it currently has, but specialized 3D application domains will of course always need a specialized toolkit. Our biggest area of interest, being Nokia, is in the embedded space for OpenGL/ES, which is not well supported by the existing 3D API’s at the moment.
vpicaver: “Is there any plans for supporting OpenGL 3.x anytime soon?”
It’s certainly an area of research for us. Initially, the plan is to first capture the features that occur in both desktop and embedded (ES) OpenGL within the portability API. But there will be nothing stopping people mixing raw OpenGL with Qt/3D, and we encourage that for using specific features of OpenGL in their applications. We would certainly be interested in any specific areas of OpenGL 3.x you would like to see supported.
afriza: “Hi, What are the supported platforms for this Qt/3D? Will there be support for Windows Mobile / Windows CE?”
The goal would be every Qt platform that supports OpenGL or OpenGL/ES. We will be dropping support for OpenGL/ES 1.1 CommonLite (the fixed-point variant, not the floating-point Common profile) from Qt 4.7, but that is the only environment we don’t have immediate plans for.
I could be completely wrong and but I believe the programmer has to request 3.x specific rendering context, this is different then creating a 2.x context. Requesting the context is platform specific task. It would be nice if Qt gave the option for initializing a QGLWidget as a 3.x context or a 2.x context. Then like you said, the user can call opengl 3.x commands directly. Also, it looks like the Qt/3d api will be compatible with opengl 3.x. The removal of fix function pipeline in opengl 3.x is biggest difference between it and 2.x. I don’t believe you can use 3.x in the current QGLWidget implementation, I could be wrong.
For example on windows:
http://www.opengl.org/wiki/Tutorial:_OpenGL_3.1_The_First_Triangle_(C%2B%2B/Win)
For example on Linux:
http://www.opengl.org/discussion_boards/ubbthreads.php?ubb=showflat&Number=262519
This is why i use Qt!
Nice news!!!
This API is rather strange for Qt. 
QVector2D is a copy of QPointF but only do float
QVector3D would be better called QPoint3D
The two classes take a qreal in the constructor, but store the data as a float internally.
Qt already uses FreeType, used by QPainter. But to create 3D text and other 3D text effects we need access to the font face data provided by freetype to render text strings as OpenGL primitives. Will Qt provide an interface to use freetype ?
What I miss in Qt/3D:
-QVector2D does provide length(), but not angle()
-There’s not built-in support for float/half-float/2 channels textures in QGL. You have to do it the old way for now.
-support for extensions other than framebuffer/shaders. Or maybe an easy way to invoke current and future OpenGL extensions ?
I think Qt needs to be more modular :s
Why all the 3D stuff isnt riped from the gui module to its own module?
On Linux it doesnt matter sice libraries are installed system wide.
But lets say i am doing a small and tradicional app using qt for windows, and i need to bundle the runtime libraries?
My program will get allot big, but i am not using most of the functionality included in the libraries!
I could compile Qt myself and remove many stuff on configure but thats not the way to go
@porfirio: If you are concerned of the final size of your shipped application, static linking is a viable option in many cases. Of course that requires configuring and building Qt, yourself. But it’s worth it, imho. On Win32, you can also use UPX on the final .exe which compresses it a lot. Have a look at these relatively compact Qt based apps on Win32 and OSX: http://code.google.com/p/speedcrunch/ http://code.google.com/p/fotowall/ http://code.google.com/p/qtlinguistdownload/
But going forward static linking on OSX isn’t feasible (aka with Cocoa). I used to static link for years and only recently transitioned to frameworks only to see my application balloon in size (although that’s mainly because QWebKit.framework is HUGE). Why is it Chrome for Mac, which is based on webkit, is so much smaller than the webkit I build using Qt?
Thanks for reminding me about UPX. I didn’t know you could use that on Windows these days. I might check that out.
What about SSE and memory alignment in the 3D and 2D classes (vectors and matrices)?
one major consideration about mathematical types and functions for 3D and 2D stuff is speed. Since you pointed out, rightly so, that these new types are for the usage on the CPU side (e.g. to calculate and prepare the data before it’s sent to the graphics card), the two major speed factors on the CPU side are threading and vectorizing (e.g. SSE). While threading requires care at the function level (to keep them thread safe), vectorizing also requires the data to be aligned in a particular way.
It would be nice of you to
a) make sure that your 2D/3D vector and matrix functions are thread safe and use vectorization when available (e.g. SSE or similar extensions)
b) make sure that your 2D/3D types contain what’s required to keep them aligned as required by SSE
Keep in mind, that executables which are packed, like with UPX, will be recognized from anti virtus tools as trojan and will be deleted.
Will there be a QtSL and QtGPGPU ?
Alessandro: but static linking is illegal for closed source programs using the lgpl version of qt, isn’t it?
Laurent: “QVector3D would be better called QPoint3D”
It would then be a bit weird when it is used for representing normal vectors. Since the representation for “points” and “vectors” is the same, I decided to avoid duplication and class bloat and went with the most common name used in other 3D toolkits, which is “vector”.
divide: “support for extensions other than framebuffer/shaders. Or maybe an easy way to invoke current and future OpenGL extensions ?”
Definitely in our plans. Vertex buffers are nearly ready to go and we will show more of that when we release the Qt/3D repository in the next post.
I will make a note of the other requests (OpenGL 3.x context creation, FreeType, textures, SSE, etc) so that they aren’t forgotten - very good ideas all.
Another serious thing to consider is making QGraphicsView threadsafe so it can be used in multi-threaded rendering. One example, is putting the graphics rendering code into its own thread that updates the scene at an exact FPS speed, and a GUI thread that interacts with the user but does not synchronously lock up the scene updates. Right now, I have to dump QGraphicsView and do it all myself through raw OpenGL calls in order to have usable UI interaction while having exact (to the millisecond) FPS updates.
Another improvement is to fix MOC so it’s multi-core compatible. Right now, even on this 4-core i7, the slowest part of compilation is waiting for single-threaded moc’ing to finish while the rest of the normal compiles can be passed off to multiple cc instances.
Also, since qreals are doubles I’m not sure how SSE optimal they are especially since most code does not require double precision, while x87 compatibility is being heavily phased out by compilers.
@Kai: Exactly. That’s why I’m using frameworks now since I’m using Qt under the LGPL. I’m honesty surprised this hasn’t come up before since QtWebKit has really gotten HUGE, it makes my application and the other Qt dll’s on Windows look pathetic in size. Ditto for the frameworks and application binary on Mac.
You are right about static linking, it can really make the application smaller but its a bit pain
I still think some stuff could be reorganized and removed from gui and core libs…
For high performance GL usage, Qt’s GL support should have:
1. support for (vertex) buffer objects, accessed via the GL routines: glBindBuffer, glGenBuffers, glBufferData, glBufferSubData
2. such support into QGLShaderProgram’s in setting attributes to buffer objects.
3. in GL one can set an array of uniforms with one call via the family of functions glUniformNfv, glUniformMatrix*fv, support for these should also be added in QGLShaderProgram. Unfortunately, a QMatrix4×4 is not just the 16 floats that comprise it, it also includes a bitfield hinting at the matrix “type”: Identity, General, Translation, Scale, Rotation
4. Qt still does not support creating a GL3 context via the new wgl and glX entry points. Currently nVidia hardware gives a 3.x compatibility context anyways, but other hardware may or may not. Actually adding this is not hard.
5. Geometry shader support: available via an extension and in GL 3.2
6. It is good that the QMatrix4×4 class includes an inverse function which is “orthogonal matrix aware”, i.e. if the matrix’s upper 3×3 corner is known to be orthogonal to then just compute the inverses upper 3×3 chunk as the transpose. Unfortunately, this does not at all address round off errors: if one were to create such a matrix M, and compute its inverse via the above into say N and then compute: for(int i=0;i
I wonder why there is so much code duplication in those classes. Just two examples:
- “qReal QVector4D::length()” could just return qSqrt() of lengthSquared()
- “QVector4D QVector4D::normalized()” could just return QVector4D(*this).normalize()
There are quite a few more opportunities to avoid code duplication. If you are concerned about performance, just make chosen methods inline.
Great news! From a hefty graphicsview fan:
- Plan neat way to have the 3d scene graph exportable to svg (the beauty of graphicsview, smoothly switch QWidget versus QGLWidget).
- 1:1 interface for 2d graphicsview concepts, say selection for a start, for 3d.
so that existing graphicsview code can be trivially ported to 3d. Consider the NG widgets going 3d for instance.
@porfirio: We can’t do that in the Qt 4.x series, due to BC (Binary Compatibility).
@No: You should really use a better Virus checker then. Most of them recognise UPX-compressed binaries.
@Will: Are you bundling the debug version of QtWebKit too perhaps?
Why math3d classes are located in gui module? I would like to use them in command line apps as well. The footprint of math3d is really small. Wouldn’t be better to put it into the core module, just like QPoint, QVector, etc? IMO it is basic functionality unrelated to gui.
hi
i have a question and will thanks any one can help me
i have an image(QImage) that pixels on it have different opacity and i want to multiply this opacities by a fixed number.
how can i do that? with use of setOpacity(f) in the QPainter class the opacity of all pixels became a fixed number(f).but
i want that each pixel opacity multiplied by a fixed number. i can do this by reading all pixel and do the multiplication manually but this is more time consuming. is there a fast way to do that like the matrix used in csharp language?
I would be very interested in knowing the reasons why eigen was not chosen. Typically, eigen has specialization for both small, fixed-size matrices, and for geometry/3D stuff. It comes from a strong KDE background, so would naturally fit with Qt.
@Alessandro: static linking cannot be always used:
- Statically link a proprietary app to Qt puts some more constraints due to the LGPL (you must publish .obj files + other things)
- If your application is composed of several .dll/.exe that depend on Qt, then static link is not an option
IMHO Qt3D should be separated from QtGui and there is not yet BC on Qt3D so it’s time to do it if possible
First of all - many thanks for doing great work Tried out new Qt/3D classes, and they are really convenient.
Still, I wonder why QMatrix4×4 isn’t implicitly shared. Given that qreal is really 8-byte floating point type, sizeof(QMatrix4×4)==128 bytes - quite a lot, and it’s returned by value from operators. Another pro of implicit sharing is that it’s much easier to force 16-byte memory alignment without touching existing code. With such alignment 3d math could really benefit from SSE/SSE2 optimizations.