One of the things I have done whilst working at Last.fm is create a simple system whereby critical monitoring is displayed on screens that we have hanging from the ceiling. There is one in each corner of the room, and opposite monitors display the same thing (e.g. two monitors display our key Cacti graphs, and two display Nagios monitoring output, so everyone in the room can see it). This is achieved through a simple dual output graphics card, and a couple of two-way monitor splitters (and a lot of cable!)
The software itself is simple: The data is displaying using some PHP scripts written by myself specifically for output on these 22″ screens, and are hosted on our servers, so all that is required to display them is a web browser.You can see these two pages in action here (Naglite2) and here (CactiView)
Very simple, or so you would think. The problem is, with the nature of this data, it needs to be refreshed constantly. The graphs are in a rotation controlled by a Javascript frame that changes to a new URL every 20 seconds, and the services/host up/down notification screen updates with a meta refresh every 5 seconds. Again, sounds pretty simple. Here are my findings:
Initial Configuration – Ubuntu Linux with Firefox 3
Being my browser of choice anyway, I set everything up in Firefox to start with. We figured Linux desktop would be more stable for hosting this rather than Windows. F11 to fullscreen mode on both the monitors, and off it goes. We didn’t notice it too much at the time, but it’s pretty annoying the way it deals with the refreshing of the images.. It clears the page, and loads the images one by one, leading to a noticable flashing of the screen every time it reloads the page. Not only that, it was the worst browser we used, leading to 90% RAM usage (on a 2gb machine) after just a day. At this point, not only did it become very sluggish, but it would stop displaying the graphs randomly, and eventually ending up in severe corruption of all the images, mixing them together in an interesting fashion. Connecting via VNC every day and restarting Firefox became a bit of a chore, so we decided to give up and try something else.
Second configuration – Ubuntu Linux with Opera
Straight away Opera was performing much better than Firefox. It seemed to almost pre-load the images for the next set of graphs before it refreshed the page, leading to no flickring of the screen, just seamless re-loading of the page. It also managed a week before showing any signs of slowing down, but after that point the graphs started disappearing again. Opera had suffered the same fate as Firefox… Using all the memory available on the machine.
We also had another little problem.. We have the time printed in the bottom right of the screen (as text rather than an image) and even by forcing cache control headers, Opera was caching the pages. The clock would move between 5-10 minutes as each graph appeared. I discovered that Opera has some advanced preferences that lets you disable the cache completely. Whilst this fixed the problem with the clock, it meant that it then only survived 2-3 days before exhausting the memory usage. We put up with this for a number of months, before deciding to move on.
Hello Webkit
At this point, Russ and I thought it was about time we gave a Webkit based browser a shot. Konquerer seemed a good choice.. We installed kubuntu-desktop, and got Konquerer running, but had trouble getting it in a proper full screen mode. Eventually we managed to hide the tab bar, but the status bar was still there. Although we found some hacks to remove it, we wanted to try something in particular, which ended up with a radical change…
Current configuration – Windows XP and Google Chrome
We really wanted to give Google Chrome (Chromium) a go on Linux, but unfortunately it’s not quite at it’s prime yet… More than anything, we couldn’t get the pages to load at all because the HTTP Auth dialog has yet to be coded. (it simply doesn’t appear. As a side note, using the user:password@ url notation makes it crash!)
After a quick hour of installation, drivers and updates, we had the screens back up and running with XP and Chromium. The nice points so far have been:
- Turning the two different pages we use into their own Apps using the Google Gears “Create application shortcut” menu option. Now we have a single icon to click to open one window, and another for the other.
- Separate processes – Now we can monitor which tab is using the RAM, and just restartthe offending process if it becomes a problem
- The biggest win by far – It leaks very little memory. So far after using it for a week, the process running the text only Nagios view has not used any more RAM than it did when we started it (35mb). The Cacti graphs screen, reloading graphs 24/7 for a week every 20 seconds has used just 80mb (40mb when it started). The reason for this is obvious; if you watch the usage, it loads the page, the memory increases by 5mb. After a few secnods, it drops by 5mb again. So there is a small memory leak somewhere but it seems Chrome is cleaning up after itself almost immediately, something which the other 2 browsers failed miserably at.
The overall functionality of the system is much the same.. I have compiled a couple of exe’s so that one switches off the displays and one turns them back on again (This combined with Task Scheduler means we save the planet whilst we’re not at work!) and VNC server functions actually better on Windows than on Linux (for some reason the secondary monitor displayed as a black screen on Linux, so you could control but not see it).
Downsides
The only downside of the Google Chrome based solution is: Webkit doesn’t support “text-decoration: blink”! In the image linked above, you can see we use the text CRITICAL for a service that is broken, and DOWN for a host that is having an issue. These used to blink, which was a nice touch to draw your eye to the issue. This is about the only valid use of “text-decoration: blink” I can think of, but unfortunately the webkit developers have chosen not to support it. Any support on this ticket would be appreciated!
We’re currently using the bleeding edge dev version, simply because it was the only version that had F11 Full screen mode in. This works very well, and it’s also very stable for a bleeding edge release (although obviously we aren’t using it like a regular browser).
Fin
If you’re after a browser that can handle sitting there all day and night happily refreshing a page, and you don’t mind running Windows (for now, anyway) then it seems Google Chrome may be your best bet. I will continue to evaluate it’s performance and maybe one day we can find something even better.
Any comments are welcome and we’re still open to suggestions, although I’m pretty happy I won’t have to restart Chrome for a few months if this trend continues!
just use javascript! e.g. with jquery http://ajaxian.com/archives/blink-at-the-marquee-one-more-time
Very interesting read. Thanks for the post. This is the type of thing I would get carried away with!
In my cross platform experience, when it comes to memory stuff, Windows wins. I’m not sure why, but it’s something to consider.
Max: It’ll be nice one day to have a crack at Chromium on Linux. When it’s ready, anyway 🙂
I set up a similar kiosk and also had leaking memory issues. After wrestling with several different browsers, I rewrote my code a fixed the problem. It turns out that while some browsers definitely have a smaller memory footprint (Google Chrome wins!), browsers only leak memory when the code they’re running makes them do it. In the case of Firefox, a plugin could be the culprit, but with no plugins installed Firefox, just like all the others browsers, will only leak memory if the page they’re on is causing them to do it. It’s true. Don’t believe it? Try loading a blank page and watch memory consumption.
Moral of the story… Your monitoring system is the one leaking memory, not the browser it runs in. This “JavaScript Frame” you speak of is probably adding a new head and doctype to the frame with every page load, causing the browser to pull double DOM duty (then triple, quadruple, etc.) or something.
Aaron: Thanks for the reply. I have no reason to doubt that you’re absolutely correct, and that’s definitely worth looking at, but it’s still weird that Chrome is so much better at cleaning up after itself even in that situation.
Hi Laurie,
Thanks so much for sharing your experience. I have been trying to build a Ubuntu box (to be an appliance later) that runs a browser listening to an internet radio station. As the box will be installed in the comms cupboard running 24/7 it needs to be no maintenance. So I have been working down the constant page display road and have found also the wonderful mem leaks. Strangely though my original config of Ubuntu 9.04 Seamonkey 1.1.17 and adobe flashplayer 10 was very solid. We ran it many times over a 24-36 hour period and witnessed a 30-40 MB mem reduction over a 15 min period, however it also released that mem on a 15 min cycle as well. There was a period of weeks where we have not had access to the target hardware and were having problems replicating our results on different hardware, we assumed it was the different h/ware causing the issues. We have been down a number of other roads that have been much less stable and recently with access again to the orig h/ware went back to the config that ran well. NOW this config bleeds mem at the rate of knots and eventually stops. The web page source I have been told has had little change however we cannot now make the orig config operate as before…… Obviously there has been a subtle change in the web page, however the end game being so sensitive makes it pretty much useless as a production device. I hate to say it however I am going to have to try your solution, for now.
Thanks again JK
You have the option in Opera to reload pages in a stipulated interval, Check right click > reload every
That way you only need to change the content of the page to reload, and get rid of the JavaScript
Please teach the rest of these internet holigaons how to write and research!