Making Freenet Find Stuff Faster 283
Sanity writes "Many probably saw the recent announcement of Freenet 0.5.2. This release represented a vast amount of work - primarily in reducing Freenet's CPU and memory requirements. However, streamlining Freenet's current functionality isn't all we've been working on. I just finished an article that describes the most fundamental improvement to Freenet's core algorithm since its original design over three years ago, it is called "Next Generation Routing" and has the potential to dramatically increase the speed with which Freenet retrieves information. It could even make Freenet faster than the World Wide Web in many circumstances, all without compromizing anonymity and while remaining immune to the /. effect."
Good. (Score:5, Interesting)
Freenet is an awesome idea, and very rapidly becoming one that is neccesary to ensure your protection. Although it is a double edged sword (It can help both good, and bad people), I think it's one that is neccesary. And, if it becomes speedier than the web at large, it'd be just freaking awesome. Now, no one needs to fear censorship, nor do they need to fear the government shoving them into a database.
Now if only I could get it running on my Mac OS X box...
Re:Good. (Score:5, Informative)
Re:Good. (Score:2)
It never does anything for me..
Re:Good. (Score:4, Informative)
Re:Good. (Score:3, Funny)
Dammit, I hate it when I miss things.
Re:Good. (Score:5, Informative)
Re:Good. (Score:2)
Re:Good. (Score:2)
It's possible to lose your anonymity with IE since it ignores MIME types.
Re:Good. (Score:3)
As the number of people on
Instant distributed mirroring.
Bloody cool.
Mirroring websites on Freenet. (Score:4, Informative)
The only problem is that there's no one-click tool to mirror a website into Freenet, yet. Freenet's gateway has an anonymity filter which prohibits out-of-freenet links, and it also disallows a lot of things. If someone wanted to write a simple tool to clean up a site and hack the links to work in Freenet, it would make this a lot easier.
By the way, using the http://127.0.0.1:8888/KEY@whatever style links is discouraged, because not everyone's freenet node is localhost, and not everyone runs it on port 8888! The preferred format is freenet:KEY@whatever which can then be handled appropriately by your browser.
Re:Good. (Score:5, Interesting)
I doubt that this will actually happen (Score:3, Interesting)
More likely, Congress will order the FBI to use Carnivore (or whatever it is called now) to track people downloading a particular file on Freenet, and to try and find out who they are. I don't remember how Freenet works, or how Carnivore works, but I'm sure with total control of the router infrastructure you could figure out who was downloading what, eventually..... although, every control message for freenet is encrypted, huh? Well, anyway they'd try.
Then, the RIAA will demand that co
Re:I doubt that this will actually happen (Score:2)
How is Carnivore supposed to decrypt the files now?
Carnivore is a waste of money, it doesnt even work for people who know how to properly defend from it.
Re:I doubt that this will actually happen (Score:3, Informative)
If the feds are tracking you, they'll do it by putting a microphone in your desklamp
Re:Good. (Score:3, Interesting)
Simple analytical reasoning will tell you that Freenet is not a good choice if you're looking for a relaxed low-profile cruise through an anarchical network. Either it works as advertised, raising the hackles of those who believe that networked anonymity offers an unreasonable risk (from RIAA to government,
Re:Good. (Score:3, Interesting)
The point about Freenet is that you cannot divide countries into democracies, where Freenet is unnecessary and dictatorships, where Freenet is impossible. There is a continuum of possible options in between. I think of Freenet as a probe that tells you where on this continuum your country really is and
Challange? (Score:5, Funny)
If this isn't a challange I don't know what is
-traskjd
Re:Challange? (Score:5, Funny)
Re:Challange? (Score:4, Funny)
A dare? (Score:4, Funny)
Said the author of the slashdotted article.
Hmm, sadly (Score:2, Flamebait)
That's not true anymore, communists Mozilla maintainers removed mng support to save a 'whopping' 100k download.
Will Oppenheim Eat His Words? (Score:5, Interesting)
Other than the fact that most infringers do not like to use Freenet because it is too clunky for them to get their quick hit of free music, it is no more of a threat than any of the popular P2P services.
Translation: "Oh Lord, I hope Freenet is inherently unable to have robust search functions, because if it ever develops these, we're hosed. But in the meantime, we can dismiss this software as being a big POS."
Now, less than two weeks after the interview, it seems the one aspect of Freenet that Oppenheim wanted to write off at is on the brink of being fixed.
It isn't search... (Score:5, Informative)
Re:It isn't search... (Score:2)
nothing about freenet prevents searching (Score:3, Informative)
Re:nothing about freenet prevents searching (Score:2)
Easy update for existing freenet users. (Score:5, Informative)
Windows : Right click the rabbit icon in your system tray, then click upate to latest snapshot build.
Linux : run update.sh in the freenet directory.
Re:Easy update for existing freenet users. (Score:2)
Debian GNU/Linux: point
Re:Easy update for existing freenet users. (Score:4, Funny)
Funny that Windows users have to click on rabbits while Linux users run a script.
Is it really that necessary to insult the Windows users' intelligence by not including a batch file?
Re:Easy update for existing freenet users. (Score:3, Funny)
Does matter, given that the intelligence of Windows users is insulted every day by Windows itself ?
Beware the Federation (Score:5, Funny)
Re:Beware the Federation (Score:2)
Distributed algorithm benefits Freenet again (Score:5, Interesting)
He didn't invent it... (Score:2)
Just like capitalism. (Score:2)
Its a good model, however I think GNUnets economic model is more advanced.
peekabooty anyone? (Score:4, Interesting)
Publicibooty (Score:3, Interesting)
They thought it would be cool to design a censorproof network. They weren't interested in supporting what was already in development, namely Freenet, after all - where is the publicity in being part of someone else's project?
The only problem was that they dramatically underestimated the difficulty of pulling it off - the result? Peekabooty was, is, and
Let it be a distributed proxy network (Score:2)
They can both become a proxy for someone else as well as hide behind someone else in a ring.
You connect to me, I connect to joe.
Joe wants to talk to you, he connects to me, and I connect to you.
This could be random, every time they log on new proxy rings.
Immune to /., perhaps (Score:5, Informative)
I was in the first
Things to do if you plan on playing with freenet:
1. Set it up properly.
1a Set your IP in the config file, read the site for details, but it's freenet.ini
1b Try to use DynDNS if you have a dynamic IP
2c Leave it up 24/7 for a few days before you judge speed. You need to let the blood circulate
2. Install a proper version of Java. I recommend the 1.4.2 beta. IBM may work better, I haven't tried.
3. Fix your browser.
3a Your browser will crash on some sites (even Mozilla not Opera) because of a GIF bug.. patch it.
3b Set your number of simultaneous connections up a lot. You request a file from your local store, then it downloads it. You need to request as many in parallel as possible.
Now, on to advice.
Get Frost! Frost is like the news groups of the freenet. It's a great place to read interesting ideas.
If you want to make a site, check out Fish tools, Fuqid and FIW.
Be aware that there are 3 different kinds of sites, and two modes of getting information
3 types include interval based, revision, and static. Static sites are one time shots. Revisions you create directories like
There are SSK and CHK linking methods, which I still don't know a whole lot about, but maybe someone will reply and explain them.
By
Get IIP, so you can realtime chat with people that run some sites on freenet. #freenet is dedicated to freenet chat and issues.
Have fun!
(Posting anonymously in respect of the freenet principals.)
Re:Immune to /., perhaps (Score:5, Funny)
Now the question all the new freenetters really want answered, is - after installing, configuring and letting run for a while.... How do I get some porn off the nextwork? Is there a cache of keys on the netsomewhere that I need to be able to find or what? Is there a crawler app that just keeps track of what it knows it's run across and builds it's own little directory??????
Re:Immune to /., perhaps (Score:2)
Once you have it up, open the browser proxy page. there are some default bookmarks there. Go to The Freedom Engine.
When that loads (it'll take a while... it's big...) go looking for porn links. There's lots of non-porn stuff, but just do Find in Page 'porn' or some such and you should find a few. ALternately, YoYo and some of the other default bookmarks have categorised stuff (including porn) but they might be harder to get to load / more out of da
Do not download porn from Freenet. (Score:4, Funny)
If you download porn, the spyware programmed into Freenet will foward your IP to the RIAA, FBI, NSA, and then post it to a few hacking/warez newsgroups and forums.
Freenet is NOT a pornster program.
Removing Porn from Freenet (Score:3, Interesting)
"If you make it, they will come" is all to important with Freenet.
Another point to make: If you view the porn and try to download it, you are also spreading this content to other nodes. If you don't want it on the network, don't view it or use it. Indeed, Freenet
Hmm.. (Score:4, Interesting)
Re:Hmm.. (Score:3, Informative)
It is possible to publish data without strong crypto (KSK keys, I think), and those are vulnerable to spoofing, but it also makes for a convenient anonymous feedback system.
(IANACryptorapher)
Re:Hmm.. (Score:2)
It seriously needs it.. (Score:2, Interesting)
Two minutes to load the WARNING page in front of the main 'search engine' of sorts that it has.
Its worse than being on dialup. I'm all for the anonymity, but I'm on broadband, and it CRAWLS.
Re:It seriously needs it.. (Score:4, Informative)
Re:It seriously needs it.. (Score:2)
You:
I just installed .52 and boy, is it unusably slow.
From a parent (and much more justifiably up-modded) post:
2c Leave it up 24/7 for a few days before you judge speed. You need to let the blood circulate :)
Make Freenet Free! (Score:3, Interesting)
Blame Kaffe - not Freenet (Score:2, Informative)
Freenet is about Freedom of Communication, not Free Software. Just because there
Re:Blame Kaffe - not Freenet (Score:2)
Well, the problem could have been avoided in the first place by choosing not to develop Freenet using Java. There's nothing so special about java that warrants abrogating your freedom. Seems like an easily avoided goof to me. Probably someone knew java, wanted to learn java, or some silly thing, and now we're stuck w/ a dependancy on non-free software. Too bad, because now
Re:Blame Kaffe - not Freenet (Score:2)
And don't go around talking about what I "expect" people to do. I don't "expect" people to do anything. But I'm free to speak my mind, and do. And you're free to disagree with me. But it would really be a lot more productive if you stuck to the point, rathe
Re:Make Freenet Free! (Score:4, Informative)
> doesn't run on a free system like Debian
> GNU/Linux.
Package: freenet-unstable
Priority: extra
Section: contrib/net
Installed-Size: 1532
Maintainer: Robert Bihlmeyer
Architecture: all
Version: 0.6+20021221-1
Depends: kaffe (>= 1:1.0.6-4) | java-virtual-machine, adduser, debianutils (>= 1.6), net-tools, debconf (>= 1.2.9)
Conflicts: freenet
Filename: pool/contrib/f/freenet-unstable/freenet-unstable_
Size: 1273386
MD5sum: f1e9f4ae9949f77f618bd1ff6d7a5220
Description: A peer-to-peer network for anonymous publishing (unstable branch)
Freenet is a decentralised network of nodes designed to allow for efficient
distribution of information over the Internet. Freenet's goals are resilience
to censorship, and anonymity for producers and consumers of information
through plausible denyability.
This package provides the software necessary to run a Freenet node able to
take part in the network used by versions 0.4 to 0.6. Content can be inserted
and retrieved with a commandline tool, or via the HTTP gateway with any
browser.
This is a snapshot from the development branch.
plenty of room for future research/tuning (Score:4, Interesting)
Note: if you haven't read the article, this won't make much sense to you.
For one, the number of reference points doesn't have to be fixed; if/when memory and cpu power allows us, we could have variable number of reference points per node. This opens the door to other decisions, such as whether we encourage clustering reference points. If yes, we add new ref points closer to others. If not, we remove a ref point the density within some keyspace interval gets too big. Another option is to add a new ref point whenever the n previous estimates turn out to be more than x% correct, and remove one if otherwise.
Another direction to go into is curve fitting. If cpu power allows us, we could use various techniques of polynomial or Fourrier interpolation within the existing reference points to draw more accurate curve of time vs. keyspace.
neural nets (Score:2)
"remaining immune to the /. effect."? (Score:4, Interesting)
Transient Nodes and Permanent Nodes (Score:3, Informative)
I noticed the lastest versions default to permanent node and the Windows version also puts itself in your startup folder.
I don't think a few hundred or thousand transient nodes coming onto and off of Freenet would hurt it, but I think permanent nodes frequently hopping on and off will slow it down. I wonder why they changed the default to permanent?
If I understand correctly, a transient node doesn't store data, respond to dat
Re:Transient Nodes and Permanent Nodes (Score:3, Interesting)
So let's just wait and see if all these new non-permanent permanent no
The next level (Score:4, Insightful)
Who will take FreeNet to the masses?
In other words, who will make a simple, usable client/server program that works on FreeNet? (Think Napster/KaZaA/Gnucleus)
Will it be KaZaA? BearShare? Will it be some Open Source project?
How long until somebody with the right skill set takes this to the "next level" so that it's actually usable to people other than geeks?
Re:The next level (Score:4, Insightful)
So I have to explain that, well, there isn't really any "stuff" on Freenet at this point, and frankly if there were, it'd take forever and a day to complete, if you managed to find a node with all the data. But, like, there are all these sites that basically just link to each other, though occaisionally there's a site with some Dilbert cartoons that don't load. Oh, and did I mention browsing Freenet sites makes your $50/month broadband feel slower than a 14.4 modem?
OTOH, I'm all for the concept of Freenet. Every major release I set up a node and run it for a few days to see if it's gotten any better, but I end up shutting it off.
Well no wonder it sucks! (Score:5, Interesting)
You're part of the problem! The reason Freenet sucks for a little while after each release is that there's a huge influx of empty datastores joining the network. The network bounces back pretty quickly, as data gets passed around and as routing tables hone themselves, the network gets a lot better.
Then a day or two later, you and 90% of the other slashdotters drop off, and leave holes in everyone's routing tables. All the contribution that your nodes were just starting to make, gets undone. All the copies of content that got replicated into your datastores vanish. All the routing optimizations that were just sorting themselves out get broken again.
Tourists hurt the network. If you're judging Freenet based on it's performance the day after a slashdotting, you're not getting a full or fair picture. Come back and stay a while! Let your node run for a week and I think you'll be impressed.
When they say Freenet is slashdot-resistant, they refer to content within the network. Any piece of data, be it a single file or a whole freesite, will simply propagate more as more people request it. The network itself definitely labors a bit as empty datastores dillute it. The best way to improve Freenet's performance is to encourage those tourists to stick around, so they and the network will benefit the most.
Re:duh (Score:2)
That wasn't the point, was it?
It's not what *FreeNet* is trying to do, it's what KaZaA/Morpheous are doing!
FreeNet still requires geeks to run and operate. But, if I could download a FreeNet installer for my windows, double-click, and have an easy-to-use node in 10 minutes, it will *explode*.
So long as you are
Re:duh (Score:2)
I don't know you, but I downloaded the windows installed, and just clicked next, next, next... on the wizard and Freenet was up and running in less than 10 minutes. Hell, if you right click on the tray icon, there's even an option to "open the gateway" that launches your browser to the appropiate address.
The config dialog even has different tabs depending on your level
Freenet not a panacea (Score:4, Informative)
This might be able to be foiled with some kind of chaffing in which nodes respond even if they don't have a piece of the data in question, but that would introduce more inefficiency.
In particular, those who are "willfully blind" to infringement losing safe harbor provisions, I don't see how Freenet will survive as a means of propagating "questionable" material. And since that's it's raison d'être, then it probably won't survive at all in the U.S.
Re:Freenet not a panacea (Score:2, Informative)
You can't prove whether those nodes were sending you the material (thus hosting it) or simply forwarding it from another node.
Re:Freenet not a panacea (Score:3, Funny)
Re:Freenet not a panacea (Score:2)
Re:Freenet not a panacea (Score:2)
1. The DMCA is not legislate outside the US. Most people live outside the US.
2. You can apply the same reasoning that lead to the said legal safe hourbor for ISPs to those running Freenet nodes.
Re:Freenet not a panacea (Score:2, Informative)
2. The language in the DMCA about what constitutes a "service provider" is vague, but in order to be eligible for the safe harbor, the Freenet node operator would have to be determined to be an ISP. Even if that happens, the safe harbor is lost because the node operator is "willfully b
Re:Freenet not a panacea (Score:2)
Oh the irony. Read that three times and lament.
"willfully blind"
You keep saying this. Is this an actual legal term or did you just make it up. I would like to know what the legal definition of this is. It seems to make all proxy servers and all server who don't log illegal.
Re:Freenet not a panacea (Score:3, Informative)
Re:Freenet not a panacea (Score:2)
Re:Freenet not a panacea (Score:3, Insightful)
As for Freenet's stated goal being circumvention of laws, I don't remember having read that anywhere -- except circumvention of certain laws in certain totalitarian states. US officials, being such lovers of freedom, should have no problem with that goal.
Re:Freenet not a panacea (Score:2, Interesting)
If you request a key and my node hands you that file, there is no way for you to tell whether I had that file on my machine already and just sent it to you in response to your request, or whether my node went out and got that file from ANOTHER NODE in response to your request, and then passed it on to you, caching it on my node in case of further requests.
In other words, by trying to 'police' freenet in this fashion,
Re:Freenet not a panacea (Score:4, Interesting)
Re:Freenet not a panacea (Score:4, Informative)
On Freenet this becomes a non-trivial task.
First- all communication between nodes is encrypted. You'd need to do a real time decryption of the communication in order to spy.
Secondly, nodes will often respond even if they don't have the data- that's the point. Even with NG routing- it's still onion routing. A node that responds that it has a peice of data may just be lieing. And by requesting the data in the first place, due to agressive caching- you're spreading the data across the network.
As to then shutting down the nodes- you'd have to shut down nodes in places all over the world.
Lastly, you could just make a second copy of a given data, new key and then then your plan is foiled.
You should really read more of the Freenet docs- they explain all this.
Can it really be faster than WWW or not? (Score:2)
From the current announcement: It could even make Freenet faster than the World Wide Web in many circumstances.
From the Freenet FAQ [sourceforge.net]: While it is unlikely that freenet sites will ever load faster than regular websites, it does adapt to sudden surges of visitors (which will often occur when relatively unknown sites get linked to from a big site) better, and high download speeds for big files are feasible too. Just don't expect very low latency.
I'm about to
Re:Can it really be faster than WWW or not? (Score:4, Interesting)
Basicly, freenet latency is bad, freenet throughput is good. (and freenet reliability is different
meta data? (Score:3, Insightful)
Yes, I have heard about Frost. As far as I understand, it's some sort of anonymous newsgroup. I guess a search engine could harvest the keys posted on Frost, and index them after retrieving and analysing the content and possibly the meta tags. But then the question becomes: how do you host such a search engine anonymously? Aren't you liable/vulnerable if your search engine is known to help you retrieve questionable content? Can't Frost be attacked ultimately for that same reason? Or is it distributed/anonymous? Am I missing something? Should I RTFA?
Lets all subcribe to Freenet (Score:2, Interesting)
$10 a month to freenet and get all the music and movies you want,
Or pay the RIAA $100,000 dollars per song.
I think we dont have a choice but to make the logical business decision just like the RIAA made the logical business decision to sue 60 million people.
Here you go, Subscribe now FreenetSubscription
Freenet: just a few notes (Score:2, Informative)
2. Whatever connection you use give it time to integrate into the network.
3.Stuff you may not agree with can and probably will be stored on your node.
4. You cant be done for 3. Unless certain western goverments get really upset with freenet users.
5.Download it. Run it. Leave it as long as you can. Repeat. Eventually it will work ok.
6.Remember its worth it. Support this project you might need it.
This routing has its problems. (Score:3, Interesting)
2. It does not fit really well in the freenet sources. In the current freenet implementation the network layer and routing layer are split. Unless you develop it yourself this will not be implemented in freenet (soon).
DNF: estimate if they are legimate by estimating their time. This does not work on a saturated network. And freenet is always (by design i think ) full.
There are some asumptions here that do not work. Also there will be things in freenet that will try to hide the location
Inherited Knowledge:
Make nodes learn faste by assuming some kind (vague!) of trust between nodes. read: create trust by an estabished node and new (unreliable?) node. This is against the freenet paradigma and creates all kinds of security problems. This kind of thing should not be implemented in freenet where the 1st priority is security.
The only positive thing this article is suggesting is to time the data and so optimize the flow of messsages according to the internet structure. In freenet this is an implementation problem.
There were more of these kind of suggestions on the freenet tech mailing list. I unsubscribed it (too much spam, too much interesting ideas from people who had no clue)
If you write such articles please investigate other p2p solution as well! (gnet/gnunet india network and many others.)
Well there's three definitions of the word 'fixed' (Score:5, Funny)
Freenet is now being 'fixed' like a leaky faucet is fixed.
The RIAA wants the digitial audio/video market 'fixed' like a crooked horse race is fixed.
With the new Freenet the RIAA is about to be 'fixed' like your dog at the vet's is fixed.
I think that about covers it.
Re:Java can't be efficient (OT) (Score:5, Insightful)
- easier language to pick up / understand (this is a collective effort)
- little to no chance of buffer overruns, making the node much safer against attacks)
- runs on Linux, Apple, Sun, Windows, FreeBSD without - any - porting
- java was more or less created with projects like these in mind, so most functionality will be readily available in the default libraries
Nowadays CPU and memory are commodities that can easily be come by. I see it taking about 32 MB right now, but that is out of a single 512 MB pool that can be upgraded to 1 GB for virtually free. My processor usage is max 25%, but note that the freenet guys set the priority to low themselves.
Java means a shift to better programming, with better runtime information and safer programs. This will take CPU and memory, but this is an offer you should consider very well.
This same discussion went on between assembler and C programmers. Look at it now. I think the progress of object oriented, garbage collecting, more secure platforms are as important as that paradigm shift.
Warper
Re: (Score:2)
Re:Java can't be efficient (OT) (Score:3, Interesting)
IMHO, the Java VM should be loaded at startup, and a single VM should be used to launch multipl
Re:Java can't be efficient (OT) (Score:2)
And why kill them? Because most of the software gets stuck for some reason once in a while. Maybe not always because of the software itself, but because they're waiting for some network timeout or
Re:Freenet is under corporate control, not 100% fr (Score:2)
Larf (Score:2)
Uprizer hosts Freenet (Score:2)
Ian Clarke was founder of Uprizer and many other companies. Uprizer if I remember right pays for the bandwidth.
Re:Freenet is under corporate control, not 100% fr (Score:4, Informative)
Uh, your really off your mark here. The Freenet web interface thingy comes with it's own mini webserver and the functionality to turn any non-transient node into a freenet distribution center. From the Freenet web interface, there's a link called Spread Freenet [127.0.0.1]. (Link only works if you have Freenet installed and running.)
Even if the main Freenet site [freenetproject.org] got taken down, things would still be just peachy...
While we're at it, what's this about the Bittorent mainpage going down? I know that a few popular tracker sites went down, but I've never heard of the main BitTorrent site [bitconjurer.org] going down. Click the link; it's up right now.
Moderators: How the hell did the parent get modded +2 Insightful?
Re:ad for freenet? (Score:5, Insightful)
This is just the attitude that is delaying the adoption of many new technologies (IPv6, for one). "Early adopter, what's that? Just tell me when it's done!" How do you expect it to "get there" if no one uses it? Take a chance. You might be pleasantly surprised.
Re:ad for freenet? (Score:2, Interesting)
I'd love to. You show me where I can download the features that they're announcing, and I'll try it. That's the point: This stuff doesn't exist in Freenet yet; they're talking about their wishlist. This is news?
Its not ready yet. (Score:2)
I'm going to wait for NGrouting.
Re:Its not ready yet. (Score:3, Funny)
N Grouting? Does that mean I can seal my bathtub remotely? Cool feature!
Re:What's Your Specialty (Score:2)
Re:What's Your Specialty (Score:2, Informative)
No type of content is more likely to have keys starting with a certain prefix than any other. So you can't "specialize" in child porn, or any other content <i>type</i>.
Re:What's Your Specialty (Score:2, Informative)
This is good, since keys are a random sampling of content, so if a node goes down then no specific type of content is lost. (Not putting all your eggs in one basket idea.)
Re:good (Score:3, Insightful)