Logfiles Made Interesting with glTail

Fudgie writes "My boss claimed it was pretty much impossible to create an entertaining way to visualize server traffic and events in a short time frame, so of course I had to prove him wrong. A weekend of neglecting my family produced a small ruby program which connects to your servers via SSH, grabs and parses data from Apaches access log and Ruby on Rails production log, and displays your traffic and statistics in real-time using a simple OpenGL interface (tested under Linux and Mac OS/X). It's a bit hard to explain over text, so please have a look at fudgie.org for an example movie, and more information."

Oh dear...

[DamonHD (794830)]

...I'm afraid that's the nearest I've seen to a simulated pissing contest ever! B^>

Rgds

Damon

Re:

[Nimey (114278)]

I think those are blobs of spooge. Excitement from having such a large request, possibly.

Oh great...

[GodlikeDoglike (600594)]

...we just made his log screen look like a bukkake flick.

Just took a look at the video

[Centurix (249778)]

And it looks like lots of things taking a wee. Once the site is slashdotted, it'll be a veritable golden shower...

Nice work though.

engineering management 101

[by Anonymous Coward]

tell the engineer it can't be done

Re:

[H310iSe (249662)]

Just want to give props, very nice you made my morning. Now to convert this to a heads up display for my helmet and I'm 1 step closer to becoming the motorcycle hacker I always dreamed I could be. And 1 step closer to earning a darwin award...

Re:

[mattgreen (701203)]

It really is. Kudos to the original poster, I chuckled quite a bit.

Visitorville

[by Anonymous Coward]

The most entertaining way I ever saw to view logs was Visitorville [visitorville.com]-its kind of like SimCity meets web logging.

Not "Fudgie", glTail

[gumpish (682245)]

It's pretty obvious that fudgie.org is just the name of the site and glTail is the name of the program.

Re:

[JamesRose (1062530)]

On the main page it says glTail, and when you click the link to read it with comments it says Fudgie, so actually, it seems both are there.

Re:

[Bert64 (520050)]

Also, "slash" is also British slang for "urinate".

Re:

[Aladrin (926209)]

And American slang for 'fan-fiction with male-male relationships.' What's your point?

Re:

[nacturation (646836)]

Also, "slash" is also British slang for "urinate".

And backslash is what happens if you urinate onto a parabolic surface?
 

Wow !

[cheros (223479)]

Obligatory jokes about 'taking the piss' aside, that is brilliant. It's the ultimate 'machine that does ping' (to name an old sketch) to keep management amused, but also provides real data. I bet that screen will go ballistic when you get Slashdotted (also a good way to visualise DDoS, maybe?).

I was about to say that it's a sort of etherape on steroids, but I've just realised your visualisation could benefit etherape instead (if you don't know etherape, look it up. No tools identifies a virus infection quicker).

Class, I'm impressed.

Re:

[bughunter (10093)]

I bet that screen will go ballistic when you get Slashdotted

Look closer. It already is ballistic.

Re:

[Poromenos1 (830658)]

Man, capitalize names. I got all sorts of things in my mind when I read about your EtheRape program...

just a ploy to visualize the slashdot effect

[molo (94384)]

Notice in the movie that one of the sites being monitored is fudgie.org, which is what is linked to here. This looks like a ploy to visualize the slashdot effect. :) Wonder what that must look like. Might tax the renderer pretty hard. I guess that is one way to get load testing done!

-molo

Re:just a ploy to visualize the slashdot effect

[Fudgie (594631)]

Still running at 30 fps with ~25 requests / second.

User agents and OS

[Wabbit Wabbit (828630)]

This would be very cool indeed.

I guess we could download the source and do it ourselves!

I don't know why so many comments were hating on this tool. As a big fan of "visualization" (Tufte books, etc.) I find Fudgie easy to understand and useful. The possibilities here are amazing.

Kudos to you, Fudgie (er...that sounds kinda bad)

Re:just a ploy to visualize the slashdot effect

[Fudgie (594631)]

http://www.fudgie.org/slashdotted.jpg [fudgie.org] for how that looks.

Here's what it looks like when you're not ./-ed

[chipster (661352)]

http://chip.cuccio.us/gl_tail.png [cuccio.us]

Perhaps the parser doesn't like my Apache logs?

2437 frames in 5.000 seconds = 487.400 FPS
Elements[0], Activities[0]
2550 frames in 5.001 seconds = 509.898 FPS
Elements[0], Activities[0]
1182 frames in 5.002 seconds = 236.305 FPS
Elements[0], Activities[0]
987 frames in 5.001 seconds = 397.321 FPS
Elements[0], Activities[0]
2534 frames in 5.003 seconds = 506.496 FPS
Elements[0], Activities[0]
2506 frames in 5.000 seconds = 501.200 FPS
Elements[0], Activities[0]
2505 frames in 5.0

Re:

[chipster (661352)]

I figured it out.

My apache config has the "HostNameLookup" feature enabled for the logs.

The ruby script's apache log regex parser only allowed for IP's in the logs. I changed it from [\d.] to [a-z0-9.] (line 87).

Bingo.

PS: THis is a pretty neat script.

Re:

[chipster (661352)]

No worries! Ideally, hostname lookups introduce extra load and traffic anyway :) The a-z0-9 should capture IP's and hostnames.

Nice work.

Re:

[eclectro (227083)]

Wow, it's like slashdot hurls chunks.

Re:

[nacturation (646836)]

Very nice. One suggestion: rather than have each side's dots fall off at the bottom of the opposite side, how about matching up serving requests with the originating referral so that the dots go to the corresponding spot on the right? Also, if you're not familiar with Flight Patterns [ucla.edu] it's along the same lines. Borrowing from that, it'd be quite interesting to show a 2D map arranged in a hub and spoke model with the center being the site(s) and the spokes representing the top 10 (or 20... configurable) re

doom

[rucs_hack (784150)]

didn't someone once do a version of doom that displayed network activity?

I recall seeing screenshots, but that was years ago.

Re:doom

[xappax (876447)]

perhaps you mean this: http://www.cs.unm.edu/~dlchao/flake/doom/ [unm.edu]

Re:

[rucs_hack (784150)]

ah yes, that was it, not for networks then.

Re:

[aled (228417)]

It's a shame nobody did a newer version with Quake.

Re:

[lahi (316099)]

I can see why that approach didn't take off. (The idea seems to be 8 years old.)

However, the line you quote is quite satisfying: csh certainly deserves to be shot. Of course, so do users of csh. This also applies to tcsh of course.

-Lasse

Oh, Sweeeetness!

[avirrey (972127)]

You gotta add an 'Asteroids' ship on the screen that lets you shoot down connections!

"Oh, look! Bob just logged on... let's get 'em!"

...

"IT support. How can I help you?"

"Hi, this is Bob..."

--
X's and O's for all my foes.

Re:

[NFN_NLN (633283)]

We're finally catching up to movies now... you know the cheesy and disconnected from reality sequence where some hackers enters a system by navigating a 3D maze... and the firewall is a monster you have to literally kill. The movie Masterminds comes to mind.

Re:

[quanticle (843097)]

And maybe after that you can add a tool to allow you to kill "rabbits" with "flu shots" ;-)

Compiz for syslogs - ohmygawd !

[udippel (562132)]

Luckily, I saw the movie before the meltdown of the server. It always pays to be on time. ;)

For those unlucky and late, actually, you missed a competition of peeing coloured snowflakes from the right versus doing the same from the left.
Only, the sources on the left are much better at aiming.
Plus, you have some 'Login ...' scrolling top to bottom; like the cast of a movie.

Heads up, Fudgie, it is truely the most amazing display of log files ever creeping across my eyes.
Keep the good work up, and please post again when you have something actually useful for the sysadmin.

I declare you 'King of Log Candy' !

Ob quote

[Provocateur (133110)]

All I see now is blonde, brunette, redhead.

GNU GPL

[wikinerd (809585)]

#!/usr/bin/env ruby # gl_tail.rb v0.01 - OpenGL visualization of your server traffic # Copyright 2007 Erlend Simonsen # # Licensed under the GPLv2

Hey, this is not the correct way to apply the GNU GPL licence. I don't know whether you had very little time available or just don't care, but the correct way is to explain exactly what licence (full title) the program is under and enable the user to find the licence (provide a copy of it and explain that the author of the licence is FSF, giving their address). We nerds of course understand completely what you mean, but other people may have no idea what you are talking about. To learn how to apply GPL

Sorry, but the boss won this bet

[nurb432 (527695)]

its still NOT entertaining.. Its more bizzare then anything else.

Re:

[aftk2 (556992)]

I'm afraid I'd have to agree. The first thing I thought was: "Hmm..how about trying to make logfiles more readable/understandable instead?" I'm impressed by the technical acuity though.

syslog, not ssh+tail -f

[allenw (33234)]

Why use ssh + tail -f when one can send the output to a centralized syslog server? There isn't any need to setup an account, keys, etc. when you can have the individual servers consolidate the data for you.

seconded

[Cheesey (70139)]

Remote syslog also means that your servers are more secure: (a) because it is harder for crackers to falsify remote logs as they need to compromise two machines, not just one; and (b) because your visualisation program doesn't need access to SSH keys for all of the machines it monitors, so a compromise on the visualisation computer doesn't automatically mean that all of the servers can also be compromised. However, you could presumably adapt this tool to use syslog quite easily.

Re:

[MarkRose (820682)]

You could, like me, be using a shared host where you have access to the server logs, but not to the server configuration files. This is a fantastic way to monitor performance remotely.

Running glTail on Windows

[Mazin07 (999269)]

If you want to run glTail on Windows:

1. Use the One-click Ruby installer from rubyforge (not Cygwin ruby)
2. Make sure to `gem install net-ssh`
3. Change "require 'glut'" to "require 'glut_prev'" to enable legacy GLUT ruby bindings

Took me a while to figure this out.

Re:Looks promising

[Fudgie (594631)]

Anything put into a logfile could be parsed and shown. I've tried with emails, shoutcast listeners and server logins, but they're not as interesting to show in the movie as I don't have the kind of traffic to make it useful.

Re:

[MarkRose (820682)]

This would be really handy for MySQL queries. My shared MySQL server runs 10 to 200 queries per second for me alone. Finding a good way to represent the data could be interesting.

Re:Looks promising

[DudeTheMath (522264)]

So...how many hours of unpaid overtime did your boss get out of you?

I like getting paid for my awesome work. Kudos, though.

Re:

[Fudgie (594631)]

Shouldn't be too hard. I'll cook one up this evening.

Re:

[liquidpele (663430)]

Any idea on how many servers it can connect to before performance issues come up?

Re:

[fmobus (831767)]

I believe this sort of tool is useful for realtime monitoring of net resources utilization. It can assist you giving graphic clues when something goes out of the usual parameters, like DDoS, slashdotments (sp?), router failure, etc. Depending on information being monitored and how it is displayed, it could also be used for long-term decision like buying more hardware or switching software because the current setup is not handling the load.

One nice, but more local example is the "duck" activity monitor (a

Re:

[symbolic (11752)]

Agreed. I saw something similar a few years ago, but this seems a bit more refined. I think there's actually a lot you can do when combining a graphics rendering engine with something like network activity. All it takes is a little creativity, a little time, and a boss who says it can't be done.

Re:Wait, what... they're not interesting?

[Fudgie (594631)]

A lot of my time at work is spent looking at logfiles from webservers, applications servers, and databases looking for things about to break down, but after I introduced this I just need to glance at a screen to instantly see if some server has stopped answering, is taking too long to answer, or is generating way more exceptions than normal. I also add an event (the login text bouncing down the screen in the movie) on each money generating activity, which always amazes marketing people when they walk by.