![The Internet The Internet](http://a.fsdn.com/sd/topics/internet_64.png)
![Java Java](http://a.fsdn.com/sd/topics/java_64.png)
![Networking Networking](http://a.fsdn.com/sd/topics/networking_64.png)
![Programming Programming](http://a.fsdn.com/sd/topics/programming_64.png)
TinyDisk, A File System on Someone Else's Web App 188
Psy writes "I attended Phreaknic this weekend where Acidus released TinyDisk, a shared file system that runs on top of TinyURL or his own implementation NanoURL. TinyDisk compresses a file, encrypts it, and dices it into clusters. Each cluster is submitted to TinyURL as if it were a url. This clusters can be read back out of the database, making TinyDisk a global file system anyone can use. There are safeguards in the default config to prevent people from dumping gigs of MP3s into TinyURL. While file-system-on-web-applications are nothing new (GMail file system anyone?) this hack shows how easy it is to accidentally design a web application insecurely despite the default PHP protections. See his presentation for more info"
Nifty hack (Score:5, Interesting)
Nifty little program all the same and a nice hack ,
Having it running on his NanoURL implementation locally , could allow for a cool little service . Though there are better ways to offer web based file systems in the real world .
He does state in the FAQ that its not intended to pollute TinURL in any way
Perhaps it will give TinyURL a nudge to tighten up their security though .
It's simple. (Score:3, Insightful)
Re:It's simple. (Score:5, Insightful)
You wouldn't even need to do this with every URL added to the system. Spot-checking every 1 in 10 URLs or so will go a long way to preventing any sort of abuse.
=Smidge=
Re:It's simple. (Score:3, Informative)
Re:It's simple. (Score:3, Interesting)
Re:It's simple. (Score:4, Interesting)
(1) create one tinyurl which contains encrypted data
(2) create another tinyurl which contains the decryption key
Never access them from the same IP nor around the same time, and nobody will ever know what you're hiding.
Re:It's simple. (Score:2)
Re:It's simple. (Score:3, Funny)
Oh, great; now we're all gonna have to remember "http://tinyurl.nsa.gov" instead of "http://tinyurl.com".
Re:It's simple. (Score:2)
Re:It's simple. (Score:2)
Actually, that would only make things worse, as the filesystem code would simply have to resubmit 10% of queries, doing nothing but drive the load up further. Assuming, of course, that the trick suggested by the AC where all the data is stored in the query string is not used.
Re:It's simple. (Score:2)
Re:It's simple. (Score:2)
Re:It's simple. (Score:2)
In other terms: if you spot check 2500 URL's per month, you will catch at least one invalid URL each month, 99.98% of the time.
This isn't true. The base rate of 'false URLs' matters. e.g. if all 11 million URLs are valid, checking 2500 urls finds you an invalid URL 0% of the time. If all are invalid, checking 1 finds you an invalid URL 100% of the time. If half are invalid, checking 1 finds you an invalid url 50% of the time, checking 2 finds you at least one invalid url 75% of the time, etc
Re:It's simple. (Score:4, Informative)
Further, even the best visual captchas are easily overridden if the attacker is motivated enough; a common means to perform this action is to get other humans to voluntarily solve the captchas as they are encountered by offering, eg, free porn.
Basically, captchas aren't really the solution to preventing bots (there are no good solutions for this), they only deter casual botters.
Re:Nifty hack, or antisocial behavior? (Score:5, Insightful)
At its core, Tinyurl is just a write-once database. You add data and get back a key/pointer to said data. As with typical databases, the size of the pointer is logarithmic in the size of the input (* number of keys stored, not bytes; however, the number of bytes/key is bounded under some constant, so it's effectively the number of bytes).
This gives us a logarithmic compression scheme, where our compression ratio (N-logN)/N approaches 100% as N gets large. This kind of "infinite compression" is what makes the method attractive: you put in say a kilobyte of data and get out a (currently) 5 byte key. All you have to do is keep an index of the keys.
TinyDisk doesn't seem to do this, but you could then turn around and store the index as a key. Take 1000/5 = 200 keys and get back one key. Lather. Rinse. Repeat. In the end, you have a single key that points to the backup of your mp3 collection, all in one TinyUrl! Not too shabby. After all, it's free storage, right? Wrong. Someone ends up paying for the infinite compression. In this case, it's Tinyurl. If this kid had stopped to think for a few minutes before publishing his hack, he would have realized that he's actually doing a malicious, antisocial thing. I suspect there will be a dozen copycats in the wild before the end of the day.
Farewell TinyUrl, we knew ye well.
Re:Nifty hack, or antisocial behavior? (Score:2)
I don't really see this as abuse as much as the fundamental flaw in providing free services. If the cost to support the service is higher than the cost to the recipient, it's just a matter of time before someone finds a way to cut their co
Re:Nifty hack, or antisocial behavior? (Score:3, Insightful)
Sure, but I think it's a pretty dumb idea because of the large overhead (in time and data) of actually retrieving that data.. http request and response, encoding, etc. And the fact that tinyurl will (rightly) kick your ass off the service once he's on to you.
Re:Nifty hack, or antisocial behavior? (Score:2)
After all, it's free storage, right? Wrong.
He wasn't suggesting it was a good idea to do it - he was giving a sample mindset of someone who would use TinyDisk to do stupid/malicious things.
Re:Nifty hack (Score:2)
damn that's a lame filter.
-nB
Problems ahead? ;) (Score:5, Funny)
Re:Problems ahead? ;) (Score:2)
From what I understand (Score:4, Informative)
1- Open a meta file
2- Retrieve and concatenate all the clusters from TinyURL in the order
specified in the meta file.
2- Base64 decode the file
3- Decrypt the file with the algorithm and key in the meta file
4- Decompress the file with the algorithm in the meta file.
5 - Verify the file size given in the meta file is correct for the
decoded/decrypted/decompressed file
6- Verify the checksum with the algorithm and value in the meta file matches
for the decoded/decrypted/decompressed file
7- Set the filename of the decoded/decrypted/decompressed file to the
filename specified in the meta file.
Hope that helps somebody
Solution for which problem? (Score:4, Interesting)
I adore the ingenuity (correct spelling?) of the hack but... I can't really find a problem this hack is a solution for.
As a way to distribute files, it's probably too slow. The pro's I see here: the file is not stored as one single file but it's stored as a distributed file (a set of Base-64 encoded clusters), making removal of the file hard. On the other hand, if one single segment drops out, the file will be destroyed (except if some redundancy exists, of which I did not find evidence).
If you want to send attachments in an e-mail, this is a very complicated way to do it. Every receiver must have the decoder program to re-assemble the file.
Moreover, if tinyURL builds in a check to see whether the submitted URL exists (not just some 404 page), the whole concept would probably break.
Anyways, very clever hack!
Re:Solution for which problem? (Score:3, Interesting)
Also, it could be used for distributing small text files containing reports from warzones and other heavy-censored countries. EFF should have a blast on this one.
Re:Solution for which problem? (Score:2)
Yes yes, and thermonuclear weapons were only invented to deter the Soviets.
Sorry, but moral questions DO come into this.
Insecure? Really? (Score:5, Insightful)
Insecure? Rancid tabloid hyperbole more like.
Re:Insecure? Really? (Score:2)
Well, your reaction is not very restrained either.
Hang on. Is it hyperbole day on Slashdot and no one told me? obBart: "This is the greatest injustice in the history of mankind!"
Re:Insecure? Really? (Score:5, Funny)
Hyperbole day? That's the most ridiculous thing I've ever heard in my entire life!
Re:Insecure? Really? (Score:2)
Of course, on Slashdot, every day is hyperbole day!
Re:Insecure? Really? (Score:2)
Re:Insecure? Really? (Score:2)
Re:Insecure? Really? (Score:4, Insightful)
It would also prevent tinyurl being useful for private URLs (e.g. those behind firewalls which only allow connections from known IP addresses). You can also use currently use tinyurl with protocols that the tinyurl server knows nothing about, e.g. ed2k: or magnet:.
The better solution is just to disallow any single IP from creating more than, say, 10 URLs in an hour. This would make such a filesystem implementation useless without overly restricting legitimate users.
Re:Insecure? Really? (Score:2)
What about set-ups where a large number of users (say > 1000) are masqueraded behind one IP address?
Re:Insecure? Really? (Score:2)
Valid URLs can be used (Score:2)
http://www.google.com/search?hl=en&q=four+score+an d+seven+years+ago [google.com]
http://www.google.com/search?hl=en&q=our+fathers+b rought+forth+on+this+continent [google.com]
Oh my god! (Score:2)
NanoURL review (Score:5, Funny)
Re:NanoURL review (Score:2)
I'm sure this won't be abused (Score:3, Interesting)
Pretty soon you'll see someone trying to use this as their backup system for 30gb of pr0n. Will large files kill TinyURL? What kind of latency is this going to introduce? If nothing else, this might constitute a DoS attack on TinyURL.com (which would be illegal.
It's still interesting work.
Re:I'm sure this won't be abused (Score:3, Insightful)
Even more interesting would be something which encrypts your files and spreads them around in various free storage media (slashdot trolls?) in such a way that they can not be easily correlated with each other.
Cramming all this stuff into tinyurl is bound to be noticed, but if it is a couple of dozen bytes here and there it might be possible to store lots of stuff with a reasonable degree of safety.
Re:I'm sure this won't be abused (Score:2)
Default PHP protections? (Score:4, Funny)
Re:Default PHP protections? (Score:3, Informative)
Really? So in what way is 'echo "hello world";' insecure? The only PHP scripts that are insecure are the ones where programmers made stupid decisions or wasn't thinking the design through, just like in any other language. 99% of these PHP problems are using external data without checking it. 99% of those cases are where the programm
Re:Default PHP protections? (Score:2)
And the main reason for that is the brain-dead magic quotes feature - i.e. not only is the grandparent wrong about no default security, but it's actually the default security that causes the problems he's complaining about.
not just like any other language out there (Score:2)
This would have been a straightforward feature to copy/adapt into PHP if anyone were interested in making it a decent server-side web language.
Don't say "just like in any other language" when you're unaware of lang
Re:Default PHP protections? (Score:2)
Re:Default PHP protections? (Score:2)
I haven't seen a whole lot of PHP 5 either, but from what i have seen they mostly concentrated on fixed a lot of the OO problems. Which is good but I was hoping they would address some of the more serious (IMHO) problems with the underlying language (adopting a standard naming scheme for functions, maybe creating some namespaces,
Re:Default PHP protections? (Score:2)
Re:Default PHP protections? (Score:2)
Perl allows you to use 'taint
Is it (Score:2, Funny)
No. (Score:2)
TinyDisk's inspiration (Score:3, Interesting)
You could do this with blogs or any CMS (Score:5, Insightful)
But overall 'WHY?' must be the question? Al Quaeda or The Real IRA? They still have their old working communication channels. Also who needs space like this? Space of this amount could be made redundant and available by using GoogleMail, Yahoo and Hotmail in synchrony. If none of those are available, presumably you'd have it on USB key as well.
Re:You could do this with blogs or any CMS (Score:3, Informative)
I've used something similar myself, and there are a few obscure reasons for hiding data in somebody else's web application.
For instance, Opera's UserJS (the inspiration for Greasemonkey) doesn't have a restriction-free XMLHttpRequest object, so the only information you can retrieve with it is from the original host. Stuffing data onto that host is sometimes the only way of making some features work.
Re:You could do this with blogs or any CMS (Score:2)
Re:You could do this with blogs or any CMS (Score:2)
Re:You could do this with blogs or any CMS (Score:2)
Re:You could do this with blogs or any CMS (Score:2)
I wouldn't trust tinyurl.com not to keep logs with enough info to identify me if somebody was that desparate to find me. Far better to go through a service that is, at least, supposed to be anonymous.
Furthur Compression (Score:5, Interesting)
Re:Furthur Compression (Score:2, Insightful)
Re:Furthur Compression (Score:2)
See here [wikipedia.org] if that was indeed simply a typo!
Re:Furthur Compression (Score:2)
Just had to try tinyurl, I think it was designed for uses just like this
Holy.. cool technology overload.. (Score:2)
This sounds like a very cool conference, are they going to distribute a conference program in pdf format, or is Phreaknic too underground for that, and require you to get it off torrent ??
What does PHP have to do with it? (Score:5, Informative)
article defends PHP; no bashing (Score:2)
The underlying message is that web application development is inherently difficult to secure, despite PHP's valiant attempts to protect programmers from themselves. This is the opposite of PHP bashing. It's PHP apologetics.
I disagree with the article's premise. It seems to me the same sort of mindset that attributes to "pilot error" aviation incidents that would better be attributed to poorly designed instrumentation.
Re:article defends PHP; no bashing (Score:2)
Security is layered. PHP components can be secure, but in providing a general purpose language, anything can be built from those components, including insecure web applications.
To create a programming language or environment from which you cannot build an insecure application would require seriously compromising the flexibility which gives the l
commentary defends PHP; no bashing (Score:2)
In my analogy the programmer is the pilot and the programming language is the instrumenta
The end of TinyURL. (Score:2, Insightful)
But this is a misuse of a really useful service.
When TinyURL's administrator has to either go out and buy his
second 2Terabyte disk array in a week or shutdown, which do
you think he will pick?
Re:The end of TinyURL. (Score:2)
Re:The end of TinyURL. (Score:2)
Seriously - this app encodes your data as URLs. Imagine splitting a DVD image into URL sized chunks and then submitting them one by one. Does that sound like a workable storage system to you?
As a fully-distributed system for illegally distributed or illegal materials?
Absolutely.
The reason an abuse shell script wouldn't be as bad is because of motive. This is a way to abuse the system which is useful.
Video/Overview of Acidus's presentation (Score:4, Informative)
Here [wilpig.org] is a video of Acidus's presentation. If you haven't seen him present before (At Hope, O'Reilly's E-Tech, Toorcon, Phreaknic, Interz0ne, etc, etc) he puts on a good show.
The presentation was called: Layer 7 Fun: Extending web applications in interesting ways. He discusses how traditional web applications work -vs- "new" web ppas that use AJAX. He talks about writing extensions to web apps using an API supplied (ala Housingmaps.com, or chicagocrime.org). Finally he talks about writing an extension to a web app where you don't have access to an API. TinyDisk was a case study for writes these so-called "non-sanctioned" extensions. He has a funny little slide he goes back to about how to properly implement a web app (which TinyRUL fails to do). Things like "don't wallow users to uploaded arbitrary amounts of data directly into your database."
Funny Stuff. His upcoming talk at Shmoocon [shmoocon.org] seems pretty cool too.
Book names - Recommended Reading (Score:4, Informative)
There are definitive works in certain fields that online guides and HOWTOs cannot even approach in terms of detail or quality. It's a class of books that are so familiar people refer to them by nicknames instead of by full title.
Well maybe so, but I did not know them all, and in the interest of helping people along the path here they are:
Books like:
K&R, The C Programming Language by Brian W. Kernighan and Dennis M. Ritchie
The Dinosaur Book, Operating System Concepts by Abraham Silberschatz
Knuth's never-ending story, The Art of Computer Programming, but Donald Knuth
The White Book, Introduction To Algorithms by Thomas H. Cormen, Charles E. Leiserson, Ronald L. Rivest, and Cliff Stein
P&H, Computer Organization and Design The Hardware/Software Interface David Patterson John Hennessy
The Illustrated's. TCP/IP Illustrated Series (The Illustrated's) - W. Richard Stevens
The Rainbow series. U.S. DOD Computer Security Series
Re:Book names - Recommended Reading (Score:2)
ISO 9945-1 Portable Operating System Interface (POSIX) -- Part 1: Base Definitions
"The Ugly Red Book That Won't Fit On The Shelf"
This one was not mentioned in the TinyURL Recommended reading. But there are many of these entries in the Jargon File [catb.org].
Security isn't the issue; resource exploitation is (Score:4, Insightful)
Re:Security isn't the issue; resource exploitation (Score:2)
Re:Security isn't the issue; resource exploitation (Score:2)
Why not go further? (Score:4, Interesting)
Take the list of cluster URLs [msblabs.org]. Concatenate them into a single URL. Submit it again. Thus compressing literally ANY file to five characters.
At least, as long as the possibility space of five-character URLs isn't exhausted. It's very much first come, first served.
Re:Why not go further? (Score:3, Informative)
We are simply addressing it. By your definition, a filesystem path (e.g.
Re:Why not go further? (Score:2, Funny)
Okay, back to working on my perpetual motion machine!
Re:Why not go further? (Score:2)
I can't be arsed to look it up but off the top of my head I think it's =256 characters.
Re:Why not go further? (Score:2)
Re:Why not go further? (Score:2)
The HTTP protocol does not place any a priori limit on the length of
a URI. Servers MUST be able to handle the URI of any resource they
serve, and SHOULD be able to handle URIs of unbounded length if they
provide GET-based forms that could generate such URIs. A server
SHOULD return 414 (Request-URI Too Long) status if a URI is longer
than the server can handl
Great Idea! (Score:4, Insightful)
I guess once this goes down, I'll have to go back to posting UUencoded files in peoples blogs.
Juggling With Packets (Score:4, Insightful)
Now make it RAID 10 (Score:2)
What does PHP have to do with it (Score:3, Interesting)
The basic functionality of TinyURL, NanoURL or any other service is to accept a string (presumably a URI) and return a shorter string that will serve as a pointer to it. If you want your application to accomplish that it doesn't matter what it was written in, people can store things other than URLs in your database. The protections against this sort of use/abuse suggested in the article are also language independent.
Greatest FAQ answer ever. (Score:5, Funny)
From the TinyDisk FAQ:
Q: This damn thing doesn't work on large files! #@%& You!
A: Did you not read the manual? Man I wish I could punch you in the face over TCP/IP! Change the config file's MaxSize line. By default the limit is 2 megs.
google fuck (Score:3, Interesting)
At the time, no one else had written about such things. I just never got around to automating the process, so it never really materialized. Maybe some brave and time-rich soul would like to give it a go?
took a bit of work on OS X (Score:2)
Re:took a bit of work on OS X (Score:2)
Re:took a bit of work on OS X (Score:2)
Re:TinyDisk? (Score:2)
Re:TinyDisk? (Score:2, Funny)
Re:TinyDisk? (Score:2, Funny)
Re:TinyDisk? (Score:2)
Mine is not so *Big*Disk
but it is certainly HardDisk permanent media!
Re:Google and banks (MODS ON CRACK!) (Score:3, Interesting)
And WTF is this modded 'offtopic'?
Re:This is simply vandalism (Score:2)
Re:It's not a file system (Score:3, Informative)
A filesystem stores and retrieves files.
Here are some exmaples of filesystems that undoubtably violate posix:
FAT as shipped in DOS 1.0
Had no subdirectories
Had no notion of users
Had no permissions
Limited filenames to 8.3
CD-R
Doesn't allow data to be modified
Re:It's not a file system (Score:2)
Whoa. In nearly the words of Babbage, "I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a statement." Only POSIX filesystems are filesystems?
Trivial Solution to TinyURL URL validation (Score:2)
Two flaws. (1) It is possible to create a loop of redirects. Of course, the solution on TinyURL's end would be to follow an arbitrary number of redirects, and declare anything that redirects