Arora

The demo browser in Qt 4.4 has obtained a lot of attention and interest. Now that 4.4 is almost out I have forked it and started a new project called Arora where development can continue, features can be added, and anyone who wants can contribute. The source code is in a Git repository and is currently hosted on github. Along with some improvements and features there are autotest and manual tests. Git (and GitHub) makes it pretty easy for new developers to a project to fork off a branch to develop features that can be later merged back into the project. Feel free to grab the source and add the feature that you miss the most or take one of the fun items off the TODO list. To any artists out there we need an icon to match the new name. Given that this is a fork of the demo future news on Arora will be on the project’s home page or on the Arora blog.

QtWebKit & WebKit trunk

In an effort to stabilize QtWebKit before the 4.4 release QtWebKit was forked from WebKit about six months ago. This is why the acid3 test score in Qt4.4 didn’t match the latest Safari. Now that 4.4 is almost out the door work has begun on integrating the changes into WebKit trunk for 4.5. Not everything is in yet, but already a lot works and it is very exciting. I’ll let this screen shot speak for itself:

The remaining failing tests mostly relate to SVG Fonts. If you are interested in helping out feel free to join #webkit or #qtwebkit on freenode. Instructions on how to build Arora with QtWebKit trunk can be found in the Arora source wiki page.

History in a web browser has a number of challenges that make it a good demonstration of Qt’s model/view. The history is used in the url completer, the history dialog, the history menu and lastly by WebKit itself for determining if you have been somewhere before. And just to add to the mix it need to be able to scale. A year of history could easily be 20,000+ entries. The History component of the demo browser is more advanced then the examples in Qt and shows off some more the cool aspects of model/view and it has been suggested to me that writing up a little description of how it works might be useful to some developers looking to use model/view in their own applications.

I have spent time working on item view and know it pretty well (I put together the Modeltest and ItemView Tips) so the history component of the demo browser was fun to write. The source code for all the classes can be found in demos/browser/history*.

The first class is the history manager who’s data structure is optimized for history, not for dealing with item views. The class loads and stores the history, emits signals when there is a new entry or one is removed and that is about it. This class should be able to stand on its own if it was pulled out and used without itemview. I wanted to keep the memory usage down as much as possible. For 20K entries it was only using around 10MB of ram when I was done.

On top of the HistoryManager is the first model, a QAbstractTableModel (HistoryModel) who’s only job is to wrap the history model. It doesn’t do anything other then implementing functions such as rowCount(), flags(), and data(), matching up the itemview functionality with the corresponding functionality in the history model.

On top of that is a QAbstractProxyModel (HistoryFilterModel) who job it is to filter out duplicates. Looking at browsers I noticed that if you visit a link twice the browsers always hide everything, but the most recent visit. A hash is used to determine if a link has been visited along with the most recent location of the visit. This model has a convenience function that is used by WebKit to determine if a link has been visited.

On top of HistoryFilterModel is another QAbstractProxyModel (CompleterModel) that adds a new row for every entry stripping the front of the url so the completer can complete on “http://trolltech.com” or “trolltech.com”. This model is used by the QCompleter in the url line edit.

For the history menu and the history dialog the browser wants to display a tree based structure where each top level node is a Date with history entries for that date as the children. So on top of HistoryFilterModel I wrote a new QAbstractProxyModel (HistoryTreeModel) that converted the table into a tree. Converting the table into a tree was a little tricky, but the modeltest was very useful for detecting problems.

Before putting the HistoryTreeModel into the dialog I wrapped it one last time with a QSortFilterProxyModel so that the user can do on the fly searching. The searching (really filtering) is very fast. This is put into the HistoryDialog that has a EditTreeView which is a normal QTreeView, but with added support for the delete key to remove one row from the model.

On top of the HistoryTreeModel there is another QAbstractProxyModel (HistoryMenuModel) that slightly modifies the tree moving the most recent history entries inside of the root rather the first date folder. To do the mapping I translate each node in the tree all the way up to the table and back again to convert to the source index. This class was just a big pain to code and took a solid day to do. Looking back adding a hack to the menu rather then writing this class seems like a reasonable option given the effort.

Both history and bookmarks show up in the top level menu so I wrote a subclass of QMenu (ModelMenu) which can take a QAbstractItemModel and on the fly the ModelMenu will populate it with actions from the model. This is pretty straight forward and works very well.

Overall the History classes show off how you can use item view to to write some pretty cool stuff that scales too. History’s classes contains a wrapper, some classes that filter out items, some classes that add new entires, modify or even move items around and lastly some views the present the data in very different ways.

Using my inkscape skills (or lack of) here is a diagram of the classes to help make better sense of the relationships.

In 4.4 Qt is getting a new module, qtwebkit. I have been watching it develop over the past year and have been eager to play around with it in Qt. Shortly after the module was merged into the main Qt branch I began working on a small browser in my spare time. Something I could use for at least some of my daily browsing. By eating my own dogfood I could catch errors in WebKit, the new networking code, and especially spot trouble areas in the API before the release.

*Disclaimer: This is just a little demo project of mine, not a FireFox replacement.*

Startup speed
One nice feature of Qt4 over Qt3 is the startup speed. I have seen reviews of KDE4 that even mention the nice bump that KDE applications received, even application have have just been porting and no extra refactoring was done. When Zack was first working on the webkit Qt integration I noticed that his test application would launch near instantaneous when Qt was in memory already. So if WebKit can launch fast, and Qt can launch fast I was definitely going to make sure that some silly demo code didn’t make it launch slow. So even with an icon database, history, cookies, tabs, etc the demo can still launches quickly. So I think it is fast enough.

Networking
In 4.4 there are some new networking class built up around the new QNetworkAccessManager class. Check out Thiago’s recent blog entry One more piece falling into place: Network Access for some more information about all the new goodies included with it. On top of networking classes I created some gui components including a minimal download manager with the usual suspects such as a progress bar, download speed, etc. A nice little demo for the download manager in its own right. Something new that QNetworkAccessManager uses that not mentioned before is the QCookieJar that is included with Qt4. Wrapping it I added saving/loading, and the usual web browser features.

User Agent
Once the browser was in good enough shape to be use every day I setup Gnome and KDE to choose the demo browser as my preferred browser. So anytime I clicked on links from e-mail or from #qt on irc the demo browser would be used and in the server logs a new user agent would show up for the mysterious demobrowser/0.1. The default qtwebkit user-agent is constructed from QCoreApplication::applicationName and the new QCoreApplication::applicationVersion properties. So if you integrate QtWebKit in your application it will automatically include the application name and version in the user-agent. This is of course configurable. Because it is just a demo and for simplicity sake the demo behaves like a single application, when launched it will contact the already running browser and tell it to open the url from the command line arguments if there is one (configurable to open in new window or tab of course).

Session Managment
Developing the demo I knew that I typically leave my browser open for days at a time and loosing your open tabs (or anything really) in a crash is never good. Also while developing the demo I would want to quickly restart it as changes were made. With that in mind the demo contains is a little watch dog class that makes sure that at most you will only loose the up to the last three seconds in the event of a crash. Not just tabs, but history, cookies, and everything else is protected. Restarting the browser selecting History/Restore Last session will quickly re-open all the top level windows and their tabs from last time.

x-qt-plugin
QtWebKit include a plugin framework and QWebPage has extra built in support for one specific plugin type, x-qt-plugin which the demobrowser implements (see QWebPage::createPlugin) The demo browser lets you add any Qt widget into the webpage. Just like with QScript, signals, slots, properties are all fully accessible from JavaScript. You could even embed QWebView and make a browser in the browser if you wanted to. With the demo browser if you load up a webpage with the following code you will find a loading QProgressbar.
<object type="application/x-qt-plugin" classid="QProgressBar" name="progressbar" height=30></object>
<script>
function display(){
if (++document.progressbar.value != 100)
setTimeout("display()", 50)
}
display();
</script>

More than for a browser
Already there have been Qt developers blogging about their interesting QtWebKit projects such as Amarok and KDE Plasmoids. I am looking forward to see what everyone does with QtWebKit.

FAQ:
The demo can be found in the demos/browser directory of the Qt package. The browser in the 4.4 snapshots has more then the beta including proxy configuration, in page search, improved keyboard shortcuts and some of the features mentioned here.

• This is only a demo, I tried to not do anything that couldn’t be done with Qt easily, so adding bunzip, tar, and bittorrent support to the download manager wasn’t on my todo. There are plenty of neat things to do first such as bookmarks.
• Although plugin support was added to QtWebKit for 4.4, the official Adobe flash plugin support will have to wait for 4.5. Or more to the point the netscape plugin support wont be added until 4.5
• gmail currently does not work.
• There are no known crashes so please report any that you find.
• I plan on continuing to develop this demo, for example I have bookmarks mostly done in a branch and am am working on adding more autotests before merging.
• Many cookie issues can be worked around by turning on “Accept All Cookies” in the preference, the cookiejar has a few fixes that are not in yet.
• Feel free to check out the code for the demo. The total size is quiet small and I tried to keep the code clean.

A blog about a browser is only complete with a screenshot so here it is running on Windows (Vista), Linux (Gnome), and OS X (Tiger)

Having a cross platform browser who’s code base is small and completely open kicks ass.

A big thanks to Jen, Jens, Holger, Morten, Thiago, Simon, Jesper, Thomas, Paul, and everyone else who gave feedback/patches.

One small feature that I have wanted Qt to have when writing applications was a way to get some common directories. Where is the desktop, the music directory, or even the location where I should store my data files? On Mac, Windows and Linux these have always been different. A big problem in adding this feature has been Linux with the lack of any sort of standard, but this past year with the xdg-user-dirs spec released and with distributions like Ubuntu and others adopting it I decided it was a good time to revisit the problem. In Qt 4.4 QDesktopServices got a new function, storageLocation that returns the default system directory where files of a certain type belong. The currently supported types are: Desktop, Documents, Fonts, Applications, Music, Movies, Pictures, Temp, Home, and Data. Note that storage location can return an empty string or even a directory that currently doesn’t exists such as where the Music directory should be, but the user deleted it.

In a media player you could set the initial directory of the file dialog to open the music directory.

    QString fileName = QFileDialog::getOpenFileName(this, tr("Open File"),
                                                     QDesktopServices::storageLocation(QDesktopServices::MusicLocation),
                                                     tr("Media Files (*.ogg *.mp3 *.flac *.wav)"));

Or when looking to store some application data, say palettes for an image editor.

    QString directory = QDesktopServices::storageLocation(QDesktopServices::DataLocation);
    // embedded platforms currently don't define this location and you don't want to write to /foo
    if (directory.isEmpty())
        directory = QDir::homePath() + "/." + QCoreApplication::applicationName();
    // the first time I am saving data
    if (!QFile::exists(directory)) {
        QDir dir;
        dir.mkpath(directory);
    }
    // save my data
    QFile file(directory + "/custompalettes.dat");
    ...

Finishing off IPC features in 4.4, two small classes that have been added are QLocalServer and QLocalSocket. On Windows this is a named pipe and on Unix this is a local domain socket. The API is almost identical to QAbstractSocket so switching from a local socket to a tcp socket and vice versa is very easy if you want to. For those KDE applications that are currently using KLocalServer/KLocalSocket with minor function name changes most of them should be able to switch and then they will get another feature working on Windows. If you have a Qt 4.4 snapshot in the examples/ipc/ directory you will find two new examples that are ports of the fortune server and client.

Traditionally when you wanted to map a file into memory on Unix this was done using mmap and on Windows with CreateFileMapping. Continuing the tradition of simple, consistent API’s, in 4.4 QFile will have two new functions: map() and unmap() that provide the ability to map files into memory.

An quick example:

QFile file("foo");
file.open(QFile::ReadOnly);
uchar *memory = file.map(0, file.size());
if (memory) {
// have some fun with the data
file.unmap();
}

You can check out more in the
map and unmap documentation.

One of Qt’s solutions is a class called QtSharedMemory. It lets you share and lock memory between two processes on the same system. Originally written for Qt3 and then ported to Qt4 I set aside some time and created the successors of this class which will be part of Qt 4.4. The locking capability found in QtSharedMemory is useful for cases other then just shared memory and it was taken out and made into its own class. A better API and a nice example the code is in 4.4 and can be seen in the snapshots today.

Here is a “Hello world” shared memory example.
First process A would do the following:

QSharedMemory sharedMemory("foobar");
sharedMemory.create(1024);
sharedMemory.lock();
char *to = (char*)sharedMemory.data();
char *text = "hello world";
memcpy(to, text, strlen(text)+1);
sharedMemory.unlock();

And then until process A destroyed its sharedMemory variable process B can access the “hello world” data.

QSharedMemory sharedMemory("foobar");
sharedMemory.attach();
sharedMemory.lock();
char *to = (char*)sharedMemory.data();
qDebug() << to;
sharedMemory.unlock();

QSharedMemory uses the second new class, QSystemSemaphore to provide system wide locking. This is useful when you have different processes accessing the same data. If you are using threads you should use QSemaphore which is much lighter and has more capabilities. Check out the shared memory example documentation which walks through more detailed usage of shared memory.

Due to the different ways that every platform implement these features there are a few exceptions to be away of and they are documented in the class description (for example on HP-UX only one attach to a shared memory segment is allowed per process). QtSharedMemory and QSharedMemory and incompatible to prevent accidentally using them together and it is advisable to stick with one or the other.

The API for these classes has gone through a number of revisions. Check out the class API’s and example and let us know what you think.

Documentation links:
SharedMemory
SystemSemaphore
SharedMemory Example

Here’s a follow up on the recent blog post which presented a new filedialog interface. Listening to the feedback it has been changed back to look very similar to the one in 4.2. The big differences are that the Save As collapsing functionality has been remove and the line edit is back and gets focus by default. The saving has been fixes so adding folders to the sidebar will be saved, along with history. Note that the painting issue on the cancel button is being looked into already and will be fixed shortly (it only occurs in Plastique style).

Feel free to provide feedback on what you think of this change.

Open dialog

Save dialog

P.S. A little hidden feature that has been in the file dialog sense at least 4.2 is that if you type “..” in the line edit and hit enter it will move to the parent directory. In 4.3 that has been extended to the completer so you can now type ../foo/bar and the completer will understand the relative path.

I have been working on a new file dialog for Qt recently and would like to get feedback. A few fun highlights and screenshots taken from Linux for you to enjoy:

File Open:
On Windows and OS X we now pull the system icons and on Linux we get it from the current icon theme.

Save As:
Starting out small and simple clicking on the down button will present a smooth animation to the large dialog.

SaveAs big:
When Typing in the view a line edit will popup similar to how Qt Assistant behaves. It has completion and support for operations such as “..” to move up a directory.

Behind the scene the file dialog no longer uses the QDirModel, but a new model which is much improved. It uses less memory, stats the system less, is aware when files change in the directory you are viewing, and much more.

You can check out the new file dialog which is in the Qt snapshots. Once you download and compile the source you can run the Standard Dialogs example to check out the new file dialog. On Windows and OS X you will need to modify the Standard Dialogs source to avoid showing the native file dialog.