Mouse Gestures in Javascript

christodd writes "I have become big fan of mouse gestures, a feature included in Opera, Mozilla, and MyIE2. There's even a plugin for IE. Other programs like StrokeIt and Cocoa Gestures are also based around the concept. I can't believe nobody else has thought of this before, but what about mouse gestures in javascript? Turns out that it is incredibly simple to implement, and really handy for those 'feature incomplete' web browsers. Unfortunately, for the total user experience, we'd have to upgrade the whole internet..."

FVWM

dont forget fvwm [fvwm.org], create your own mouse gestures, 'Strokes', and bind them to any action/command.

KICKS ASS. [fvwm.org]

Browser Level == Better

I'm all for the idea of faster, better, stronger ways of browsing. I happen to think that mouse gestures and browser level code should be based in the browser, and controlled by the browser. Like look at all the gestures [mozdev.org] you get with Mozilla.

(mo: Don't invent the wheel: we have it already)

The problem I forsee with the jscript use, is a misuse of the mouse gesture jscripts by unethical sites. Because it's the planet Earth, and The Internet, half of the sites will impliment this correctly, the other half will use it as a joke, or for annoying adverts (browser interstitials) and thus cause the whole thing to be crap.

If it's at a browser level, websites can't fuck with it. So ideally, browsers will want to add the ability to block javascript mouse control, and promptly add this cool feature at a browswer level. I'm all for the idea of mousegestures, but I'm against the ability to tell a website to fuck off using them. (mo: KISS).

Re:Browser Level == Better

The problem I forsee with the jscript use, is a misuse of the mouse gesture jscripts by unethical sites.

I think that is the problem with Jscript full stop. why netscape thought it a good idea to allow any site an almost arbitrary level of control over my browser is beyond me.

The idea of doing mouse guestures or any other browser extension in JScript, except as a demo is idiotic. The whole value of these systems comes from consistency. Apple do know some things about UIs, the value in the Apple UI is that every program work the same way and you don't have to spend lots of time relearning.

If I go to one site that has mouse guestures and then another that does not or worse implements them a different way ... yuk!

But back to the original issue, Jscript sucks. The command set should be partitioned according to the security considerations. Popping up a window has a significant security impact, it can be used to launch a trojan. The toolbars on the browser window are my toolbars, no web site should be able to disable them.

I use the feature of IE that allows Jscript to be turned off by default and enabled selectively site by site. But this is not as effective as it could be because you often come across idiotic sites using jscript for everything - including navigation. The idea being to force the site designers idea of a user interface down the user's throat.

Re:In other words == mo

<pedant mode=Latin>
The two aren't particularly equivalent - i.e. is short for id est - meaning "that is", or "that is to say".

Modus operandi, on the other hand, means "the way of working", not really applicable in your message.

Don't let Microsoft make you misuse your Latin abbreviations.
</pedant>

Re:In other words == mo

<pedant mode=Latin>

What the hell kind of pedant uses HTML 4 syntax? Quote all of your attribute elements! And don't forget your doctype either :)

Insert code with proxy...

Or you could have your filtering proxy (like Proxomitron or Privoxy) insert the JavaScript code on every page. Though personally, I'd just use a browser that suppots it.

Oh yay!

Another annoying feature that wannabe web designers can add! Oh well. At least I don't have to worry about it. *Makes sure Disable Javascript is checked*

Re:Oh yay!

I tried some of the mouse gestures on that page and they conflict with other things for which I use the mouse. For example, try selecting a paragraph of text on the page. It activates one of the mouse gestures causing the page to scoll.

Really annoying.

Re:Oh yay!

Sure, Javascript has some useful features. Such as:

-- Hiding the real URL of a link as the mouse hovers over it. Real useful. Thanks.

-- Drop down menus. There are plenty of ways that do this without hiding the submenues from browsers used by the handicapped. Check into CSS (or even plain HTML!) for alternatives.

-- Opening/resizing/closing browser windows. I've got the capability of doing that myself, thanks. If I want to open it within a new tab, this "feature" prevents me from doing that.

-- Playing MIDI files while I view photos of your pet dog. AAaaaaiiieeeeeeee! (that's me screaming as I hit Alt-F4.)

Now that we have those out of the way, I admit that there are some useful features. However, for each feature, there are alternatives that, in my mind, provide just as good or better ways to do it. The potential for abuse is too great, and some browsers provide too few abilities to limit abuse while retaining the usefulness. Mozilla and Privoxy in combination are doing a decent job for me for now.

In effect, your second statement is what I'm saying by simply "Voting with my Back Button." If your web site annoys me, sometimes I'll give you the courtesy of emailing to tell you why I moved on. More often, I'll just silently move on.. and my $$$ goes somewhere else than feeding your progeny.

Its hard to make something foolproof; fools are so ingenious! The advertisers/spammers will always figureout someway to screw it up..

Re:Oh yay!

As a web developer myself, I feel that javascript's time has come (one third of mozilla is written in Javascript), but users like you who view the world in only black and white are going to slow down its adoption and dampen the usefulness of javascript webpages. Certainly 1/3rd of Mozilla isn't gimmicky cruft...

Six months ago, I thought JavaScript was a joke, a toy scripting language that just pretended to have real capbilities. I am now FIRMLY convinced that JavaScript may well be the MOST important asset that we have in opposing anyone's efforts to take over, control, or "proprietize" the web, as Microsoft and Macromedia are rolling ahead to do, with .NET and the new Flash.

Several reasons why I think JavaScript is the best choice for much app development today:

• Ubiquitous environment: It's the closest thing we have to a universal platform. There is nothing else that even approaches its ubiquity and reach. Mac, Windows, Unix/Linux, whatever, it's there and it just works. No other environment is so pervasive. Like the Bourne Shell in the Unix world, you can count on its presence and rely on it to get the job done, regardless of the platform. No other environment can credibly make that claim today, and I see no other real challengers on the horizon from a cross-platform point-of-view.
  
  
• Capability: There is very little you can't do with it (except the few things network-delivered code has no right doing in the first place, and that's a good thing!) In general, it's safer than Java because it's "sandbox" restricts it to the browser, limiting damage even if something does go wrong. In the better implementations (like Mozilla's) it is capable of absolutely staggering things - but doing so requires a good understanding not only of JavaScript, but also the DOM, CSS, and possibly XML. In reality, you need to know these things anyway, as they ensure your app is platform agnostic.
  
  
• Compatibility: There are far fewer problematic incompatibilities across all the varying JavaScript/JScript/ECMAscript implementations than there are across different versions of the JRE, for instance. If Microsoft would pay some attention to web standards in IE, much of what's pain now would go away. I'm convinced this is why they refuse to fix many obvious bugs - it would weaken their efforts to force .NET on their customers.
  
  
• Object Orientation: While not as snazzy as some other environments, JavaScript does have real objects, and you can do real oo work with it. I suspect the reason we haven't seen more acknowledgement of this is that the elite types turn up their nose at it before they even bother to find out what it can do... As Mozilla has clearly shown, JavaScript is up to doing the heavy lifting, and it's time for the effete snobs that claim otherwise to reassess their own bigotries.
  
  
• Ease of use: JavaScript is not a hard language to get started in. It's easy to do many useful and interesting things with little effort. There is even a huge and rapidly growing base of JavaScripts to be leveraged out there - nothing as comprehensive as CPAN, but several that, taken together, are close.
  
  
• Momentum: JavaScript is finally being recognized for its real abilities, rather than it's image as somethign best suited for only toys. Combined with the factors above, I think JavaScript is the most important development environment in the world today, and the only one that has a real chance of helping make sure that the web stays based on open standards and protocols.

If you still think JavaScript is a steaming pile, commit to spending a few dozen hours cheking out what it can *really* do before giving up on what may well be the best hope for the open, interoperable future that is of the greatest benefit to us all.

Re:Oh yay!

Word. The dumbest features ever thunk up by man are all tied to Javascript...the images that swirl around your cursor, preventing you from clicking on links. The script which loads everything at the same time, which means you wait forever for the one broken image. Cascading menus that don't disappear. Or do disappear, just as you are about to click on them.

And worst of all, blocking the right mouse button (or as I like to call it, "the button i use to navigate the fucking internet") in the name of "copyright protection." Every time I see this monstrosity, I download all of the images from the site, stick them in a zip file, and email it to the webmaster. "Your copyright protection didn't work. Neither did the mouse button I use to open links in a new window. One of these things can be easily fixed."

Accessability

While mouse-gestures sounds really neat to a lot of folks, 'features' like this only means more and more websites that are less accessable to people that require assistive technology such as screen-readers (most Javascript features are notorious for being inaccessable).

It would be nice if, for once, web technology was developed that made content more accessable to people with disabilities instead of less.

RSI

I've never been much of a fan of mouse gestures. Whenever you see someone using them, there's a rapid flick of the wrist in some angled direction... that can't be good for you if repeated often...

In general I don't have much sympathy for RSI sufferers. (I was going to put sufferers in quotes, but thought better of it :-).

I use a keyboard something like 8 hours a day, and have done for the last 15 years, programming computers. If anyone is a prime candidate, it's me, and no RSI as yet. On the other hand, I'm reasonably careful - I don't hammer the keyboard, and I try to rest all my forearms on the desk in front of the keyboard. Sensible things to minimise the effect... unlike "gestures", which are just a disaster waiting to happen, IMHO.

Simon

You're right, but you're self-righteous

In general I don't have much sympathy for RSI sufferers. (I was going to put sufferers in quotes, but thought better of it :-).

You're right that these "gestures" we're talking about do sound like exactly what the medical literature says causes RSI problems. Wrist-turning moves, over and over, are the basic cause of computer-related RSIs.

But your sample of one is a crock when it comes to dismissing everyone who has pain from this. Extremely useful "knowledge," that -- except all it does is arm you to dismiss other people and feel smug about not having been unlucky yourself. I used to work in bookstores in college, and some of the older clerks had RSI pain from shelving. Not something they were privileged to avoid in their jobs.

To think people are submitting articles to JAMA (003 Jun 11;289(22):2963-9 -- "Computer use and carpal tunnel syndrome: a 1-year follow-up study") trying to figure out whether carpal tunnel is associated with keyboards or mice or a combination. All we had to do was ask you and you could tell us it was a matter of being "reasonably careful." (Note -- those are quotes.)

Hey, guess what that study (and others) have indicated? It's mouse use, not the keyboard, that seems to be a main culprit. RSIs from computer use are almost always related to wrist movement. Trackballs (with a wrist rest especially) seem to be less problematic. Hmm, maybe we could use this information to prevent other people from undergoing a lot of pain, encourage trackballs instead... Oh, sorry, we don't have any sympathy for those people, 'cause they injure themselves out of a lack of common sense. No need to publish medical recommendations to guide businesses in their purchasing, for example. Morons. Let 'em "suffer."

Is that site supposed to be a demo?

If that site is supposed to be a demo, it does not serve the purpose well; it doesn't work at all in Mozilla (1.5).

Pie menus

Actually ... mouse gestures are better implemented as Pie Menus [piemenus.com].
-russ

Re:Pie menus

And Mozilla users can get an excellent pie menu extention here [radialthinking.de] .

Browsing just doesnt feel right without it :)

KDE 3.2

In KDE 3.2 you can control the entire desktop with mouse-gestures, not just browser.

Why this is stupid.

Honestly, it's a pretty stupid idea. Let me illustrate why: scroller mice. Once you get used to a scroll mouse, then you have to use a computer that just has a normal mouse, it's a major pain.

If you depend on every web page to implement mouse gestures, then you'll get this effect from page to page while you're browsing! It would be annoying to no end. And it's not an easily visible thing you can check for, unless each web page also uses some kind of cheesy "Gestures Enabled" logo. And each site might implement it differently, so that strokes mean different things from page to page. I repeat: stupid idea.

A user interface tool should be just that: part of the user interface. Just like a keyboard or mouse, gestures take time to become accustomed to. A user interface feature needs to act the same way no matter what you're doing.

Re:Why this is stupid.

This is a user interface feature that randomly turns on and off or changes behavior, depending on which web page you're on. How popular do you think the mouse would be, if for random periods it switched directions, reversed buttons, or turned off completely?

My first ooops...

My first ooops with javascript gestures: I tried to select/copy the text to send it to a few pals so that in case (more like when) it get's slashdotted they can read it. I selected the text at the top, pulled down and to the right, and the window closed (as it should).

It only took a few seconds to notice the status bar at the bottom which indicates if a gesture will be activated when you release the click... keep an eye on that when using using these. You can see if the gesture is 'blank' = it's not going to run an action. Quite handy, pretty cool. I've already grabbed the .js file !

Actually...

Actually... Mozilla's gestures *are* implemented in javascript. Download the Optimoz MozGest .xpi file [mozdev.org] (or find it on your hard drive), open the .xpi file in winzip, and there's all the .js implementation for it.

It's been done!

I can't believe nobody else has thought of this before, but what about mouse gestures in javascript?

Well, if that were only the case.

A quick search through the USPTO database shows that in fact Amazon has already claimed the mouse-gestures patent, specifically referencing Javascript. Not only that, but they've also patented the one-gesture purchase, apparently to be implemented on their site at some point in the future.

It doesn't stop there, however. IBM claims that they patented this back in the 1980s, but didn't specifically mention a mouse but rather a generic input device. And SCO, in one of their counterclaims, says that gestures are part of the original UNIX and that in fact there are over a million instances of copyright infringement in both IBM and Amazon's patent filings.

And, if only that were the end of it. Disney has jumped into the fray with claims that Steamboat Willie has mouse gestures in it, reducing this to a boiling cauldron of copyright, patents, and trademark issues.

Perhaps the author of the Javascript code should look more carefully into possible IP infringement issues before posting what amounts to a boast on Slashdot about how novel and clever they've been.

Hope this helps.

No, no, no, no, no. STANDARDS.

This is inane.

Repeat after me:
"Web Standards."

It belabors the obvious to point out that this will never be implemented my more than a tiny fraction of sites, that it actively interferes with normal point/click/drag behaviors (like highlighting text? click, drag left->right?) and that learning PER-SITE navigation is simply ridiculous.

It's not that no-one's thought of it before, it's that it's a bad idea on the face of it.

Gestures... I don't get it

I don't get the fascination with mouse gestures. Is it really that much more convenient than clicking a button or pressing a key? From the Mozilla gestures page:

View Source - Left-Down-Right-Down-Left (draw a squarish S)

Is right-clicking and choosing "view source" such a chore that you'd rather draw "S" shapes instead?

Reload (bypass cache) - Up-Down-Up

I dunno, pressing "F5" always seemed to work for me.

Personally I think the obsession with mouse gestures boils down to the typical geek fascination with things that, impractical and useless they may be, are just "exciting" for some reason.

Hey look, Slashdot implemented gestures.

Submit post - Left-Right-Up-Down-Down-Down-Up-Left-Down-Right-Up -Right. How did I ever live without these things?

Stupid.

Note MozGest adds much more functionality besides "moving the mouse".
- Rockers: Hold one button, press other to perform action
- Wheel rockers: Hold a button and rotate wheel to perform action
- Custom gestures: You don't like some? Remove it! You'd prefer it done otherwise? Modify assignment. You have a new amazing idea? Write it, bookmarklet style in "custom gesture" field. Pissed off with LMB disturbing with selection? Switch to RMB!

Plus for those who protest against "flick of wrist" - I think moving your hand 2mm left to launch "back" is less stressing than moving it 5cm, to reach the "back" button.

Problem: Performance. With multiple heavy pages opening, on average hardware, it slows down seriously and sometimes gestures don't get recognised.

"next page" tag in HTML?

What I'd much rather see is a "next page in a series" tag in HTML. Imagine reading a multipage article. At the end of the page, instead of finding that stupid link (which often seems to be where you least expect it), you could just press ALT-RIGHT (or right-click/next, or mouse:right-down, or whatever) to go to the next page. Maybe I could even configure my browser to automatically preload that next page.

HTML and all its extensions should focus on providing the document's contents and structure. The method of navigation is entirely up to the browser application, and should not be decided by the web designer.

Re:"next page" tag in HTML?

Opera has already implemented this functionality in its newer versions. There's now a Fast Forward button on the toolbar. It's pretty good at figuring out what the next page should be. In Slashdot's site, it pulls up the comments for the next story; on Google it returns the next page of results. I don't know anything about the algorithm behind it, however.