Making Freenet Find Stuff Faster

Sanity writes "Many probably saw the recent announcement of Freenet 0.5.2. This release represented a vast amount of work - primarily in reducing Freenet's CPU and memory requirements. However, streamlining Freenet's current functionality isn't all we've been working on. I just finished an article that describes the most fundamental improvement to Freenet's core algorithm since its original design over three years ago, it is called "Next Generation Routing" and has the potential to dramatically increase the speed with which Freenet retrieves information. It could even make Freenet faster than the World Wide Web in many circumstances, all without compromizing anonymity and while remaining immune to the /. effect."

Re:Good.

[Ryan_Singer]

it's running on my unmodified osx box. just use the unix version.-Ryan

Short explanation about what is the /. effect :

[BlueTrin]

From Slashdot FAQ [slashdot.org]:

What is the "Slashdot Effect?" When Slashdot links a site, often a lot of readers will hit the link to read the story or see the purty pictures. This can easily throw thousands of hits at the site in minutes. Most of the time, large professional websites have no problem with this, but often a site we link will be a smaller site, used to getting only a few thousand hits a day. When all those Slashdot readers start crashing the party, it can saturate the site completely, causing the site to buckle under the strain. When this happens, the site is said to be "Slashdotted." Recently, the terms "Slashdot Effect" and "Slashdotted" have been used more generally to refer to any short-term traffic jam at a website. We could conceivably cache pages, but that's a whole different ball of wax. Answered by: CmdrTaco Last Modified: 6/13/00

Easy update for existing freenet users.

[anonymous coword]

Instructions for windows and linux and linux compatables.

Windows : Right click the rabbit icon in your system tray, then click upate to latest snapshot build.

Linux : run update.sh in the freenet directory.

Re:Good.

[freedom_leffo]

Well, just wait half a minute or so and then point your favourite browser towards http://127.0.0.1:8888 - and off you go! The Freenet-thingie is running in the background.

It isn't search...

[Anonymous Coward]

...at least not keyword searching as you find in Google and Kazaa. When they refer to searching they mean given a key (a very large number), finding the corresponding data.

Immune to /., perhaps

[Anonymous Coward]

Immunity to ignorant masses of /. users it is not.

I was in the first /. crowd of joining, and here is the etiquet/advice I have.

Things to do if you plan on playing with freenet:

1. Set it up properly.
1a Set your IP in the config file, read the site for details, but it's freenet.ini
1b Try to use DynDNS if you have a dynamic IP
2c Leave it up 24/7 for a few days before you judge speed. You need to let the blood circulate :)
2. Install a proper version of Java. I recommend the 1.4.2 beta. IBM may work better, I haven't tried.
3. Fix your browser.
3a Your browser will crash on some sites (even Mozilla not Opera) because of a GIF bug.. patch it.
3b Set your number of simultaneous connections up a lot. You request a file from your local store, then it downloads it. You need to request as many in parallel as possible.

Now, on to advice.
Get Frost! Frost is like the news groups of the freenet. It's a great place to read interesting ideas.

If you want to make a site, check out Fish tools, Fuqid and FIW.

Be aware that there are 3 different kinds of sites, and two modes of getting information
3 types include interval based, revision, and static. Static sites are one time shots. Revisions you create directories like /1/ /2/ /3/ and link to images from the future. If the image loads, you know there is a more recent revision. date based must be activated every time interval, or they die. Be very careful with these.

There are SSK and CHK linking methods, which I still don't know a whole lot about, but maybe someone will reply and explain them.

By /. effect immunity, they mean linking to a site will only make it stronger. Everyone on /. joining freenet is just going to slow it down, because basically, you are creating a great suction on the net without any data to give back. Even worse, when you quit off of freenet, everyone will be looking for you from their cache and not finding you. This is going to cause the most problems, but surely not everyone on /. are going to quit on the same day. ;)

Get IIP, so you can realtime chat with people that run some sites on freenet. #freenet is dedicated to freenet chat and issues.

Have fun!
(Posting anonymously in respect of the freenet principals.)

Re:Freenet is under corporate control, not 100% fr

[Anonymous Coward]

Don't believe me? Look at its whois DNS founding; regulated by a corporation (DNS Trust) that is in-turn regulated by the FCC.

What are you talking about?

Once the user has a copy of Freenet, there is no reliance on DNS. Further-more, Freenet is designed to be propagated through means other than via the Freenet website. Google for "Distribution Servlet" and "Freenet".

Re:It seriously needs it..

[yarbo]

It takes some time to build up information on how to get around. It gets faster, be patient

Re:Good.

[Anonymous Coward]

I recommend using Frost [sf.net] for file transfers. The only thing I've ever successfully downloaded from a regular freesite (apart from graphics) is the Freesite Insertion Wizard.

Blame Kaffe - not Freenet

[Anonymous Coward]

From what I can see the project does try to stick to Java APIs for which free implementations are available, but it does not hold itself hostage to their bug-fixing schedules. If Kaffe can't run Freenet because it isn't meeting its API obligations, then blame the Kaffe team and pester them to fix the problems, don't blame the Freenet developers, or expect those that donated money to Freenet to have it spent debugging Kaffe.

Freenet is about Freedom of Communication, not Free Software. Just because there is significant overlap between those that advocate each - does not mean that Freenet should spend its resources advancing the Free Software/Open Source agenda at the expense of its own.

Re:Freenet is under corporate control, not 100% fr

[Anonymous Coward]

The Bittorent mainpage is shut down from a DOS, then shutdown by its chosen government (FCC), and now its in shambles. Freenet can have the same thing happen.

Uh, your really off your mark here. The Freenet web interface thingy comes with it's own mini webserver and the functionality to turn any non-transient node into a freenet distribution center. From the Freenet web interface, there's a link called Spread Freenet [127.0.0.1]. (Link only works if you have Freenet installed and running.)

Even if the main Freenet site [freenetproject.org] got taken down, things would still be just peachy...

While we're at it, what's this about the Bittorent mainpage going down? I know that a few popular tracker sites went down, but I've never heard of the main BitTorrent site [bitconjurer.org] going down. Click the link; it's up right now.

Moderators: How the hell did the parent get modded +2 Insightful?

Re:Hmm..

[MyHair]

There's no login or password to publish data on Freenet. Sites are inserted with private/public key combinations. As long as you never let your private key out in the open no one should be able to impersonate you.

It is possible to publish data without strong crypto (KSK keys, I think), and those are vulnerable to spoofing, but it also makes for a convenient anonymous feedback system.

(IANACryptorapher)

Freenet not a panacea

[acceleriter]

A "rights" holder knows the freenet key of certain material. Can the holder not hust write a script hop onto Freenet, request that key (and only that key), and fire off C&Ds to all the ISPs whose allocations include the addresses that respond? Seems simple enough--even with blinding of requests, the intellectual "property" holder can still point to the nodes that respond as having distributed the material--just as the exit server from Mixmaster (or freedom.net, before it became a casualty of 9/11 hysteria), etc. is vulnerable to legal attacks.

This might be able to be foiled with some kind of chaffing in which nodes respond even if they don't have a piece of the data in question, but that would introduce more inefficiency.

In particular, those who are "willfully blind" to infringement losing safe harbor provisions, I don't see how Freenet will survive as a means of propagating "questionable" material. And since that's it's raison d'être, then it probably won't survive at all in the U.S.

Re:Freenet not a panacea

[Anonymous Coward]

..can still point to the nodes that respond as having distributed the material..

You can't prove whether those nodes were sending you the material (thus hosting it) or simply forwarding it from another node.

Transient Nodes and Permanent Nodes

[MyHair]

When I first tried Freenet a year or so ago it defaulted to be a transient node.

I noticed the lastest versions default to permanent node and the Windows version also puts itself in your startup folder.

I don't think a few hundred or thousand transient nodes coming onto and off of Freenet would hurt it, but I think permanent nodes frequently hopping on and off will slow it down. I wonder why they changed the default to permanent?

If I understand correctly, a transient node doesn't store data, respond to data requests from other nodes or get put in the routing table, while a permanent node does. A full-time permanent node will make your local browsing faster as well has help out Freenet, but a sporadically on permanent node would cause delays I suspect.

The reason that Freenet is supposedly free from the Slashdot effect is because a greater demand for a freesite naturally causes it to be available from more nodes. The supply scales to demand.

Re:Make Freenet Free!

[John Hasler]

> It is a bit ironic that the Freenet project
> doesn't run on a free system like Debian
> GNU/Linux.

Package: freenet-unstable
Priority: extra
Section: contrib/net
Installed-Size: 1532
Maintainer: Robert Bihlmeyer
Architecture: all
Version: 0.6+20021221-1
Depends: kaffe (>= 1:1.0.6-4) | java-virtual-machine, adduser, debianutils (>= 1.6), net-tools, debconf (>= 1.2.9)
Conflicts: freenet
Filename: pool/contrib/f/freenet-unstable/freenet-unstable_0 .6+20021221-1_all.deb
Size: 1273386
MD5sum: f1e9f4ae9949f77f618bd1ff6d7a5220
Description: A peer-to-peer network for anonymous publishing (unstable branch)
Freenet is a decentralised network of nodes designed to allow for efficient
distribution of information over the Internet. Freenet's goals are resilience
to censorship, and anonymity for producers and consumers of information
through plausible denyability.
.
This package provides the software necessary to run a Freenet node able to
take part in the network used by versions 0.4 to 0.6. Content can be inserted
and retrieved with a commandline tool, or via the HTTP gateway with any
browser.
.
This is a snapshot from the development branch.

nothing about freenet prevents searching

[Vitriolix]

freenet is a *protocol*, not a client (though they do ship the http proxy client with the main distro). just like the http protocol doesnt have any search functionality built into it, neither does freenet. you can, and people do in fact use regular old web spiders to create searchable indexes of freenet.

Re:Freenet not a panacea

[RPoet]

We're not talking about "taking down" anything, we're talking about routing data. A Freenet node will route data on Freenet, just as any tcp/ip router will route data on the internet. They are quite analogous.

Re:Freenet not a panacea

[acceleriter]

1. I specifically referred to the U.S. in my original post. And the Berne Convention is causing similar laws to be passed in the EU. Freenet nodes outside the civilized world of North America and Europe could be safe, I guess.

2. The language in the DMCA about what constitutes a "service provider" is vague, but in order to be eligible for the safe harbor, the Freenet node operator would have to be determined to be an ISP. Even if that happens, the safe harbor is lost because the node operator is "willfully blind" to any infringement. And the DMCA only provides a safe harbor with respect to copyright infringement--not for obscenity or terrorist communication which could also be carried upon it.

Re:What's Your Specialty

[Anonymous Coward]

Data in Freenet is split into small pieves, and those pieces are hashesd to make the keys. When you specialize, you are more likely to store keys in some short range of prefixes than you are for any keys outside your specialization.

No type of content is more likely to have keys starting with a certain prefix than any other. So you can't "specialize" in child porn, or any other content <i>type</i>.

Re:What's Your Specialty

[Famatra]

Yes this is right, specialization occurs via key names, not content.

This is good, since keys are a random sampling of content, so if a node goes down then no specific type of content is lost. (Not putting all your eggs in one basket idea.)

Re:Java can't be efficient (OT)

[Anonymous Coward]

Just to clear up, Entropy is not a port of freenet. It shares many of its ideas and even uses the Freenet Client Protocol (The protocol applications use to talk to a node), but it does not use the Freenet Network Protocol (The protocol nodes use to talk to each other), and is hence incompatible with the original Freenet network.

Freenet is a fast moving target - anyone moving to implement a sister to FRED is going to be spending a lot of time playing catch-up.

Freenet: just a few notes

[Anonymous Coward]

1. Freenet does work, its slow, but it works, I run it on dialup, all you people with bband stop moaning.
2. Whatever connection you use give it time to integrate into the network.
3.Stuff you may not agree with can and probably will be stored on your node.
4. You cant be done for 3. Unless certain western goverments get really upset with freenet users.
5.Download it. Run it. Leave it as long as you can. Repeat. Eventually it will work ok.
6.Remember its worth it. Support this project you might need it.

Mirroring websites on Freenet.

[Myself]

I've said [slashdot.org] this [slashdot.org] before [slashdot.org]!

The only problem is that there's no one-click tool to mirror a website into Freenet, yet. Freenet's gateway has an anonymity filter which prohibits out-of-freenet links, and it also disallows a lot of things. If someone wanted to write a simple tool to clean up a site and hack the links to work in Freenet, it would make this a lot easier.

By the way, using the http://127.0.0.1:8888/KEY@whatever style links is discouraged, because not everyone's freenet node is localhost, and not everyone runs it on port 8888! The preferred format is freenet:KEY@whatever which can then be handled appropriately by your browser.

Re:Freenet not a panacea

[SWroclawski]

Your argument, if I understand it, is that given key A, then you find find the nodes that have it and shut them down.

On Freenet this becomes a non-trivial task.

First- all communication between nodes is encrypted. You'd need to do a real time decryption of the communication in order to spy.

Secondly, nodes will often respond even if they don't have the data- that's the point. Even with NG routing- it's still onion routing. A node that responds that it has a peice of data may just be lieing. And by requesting the data in the first place, due to agressive caching- you're spreading the data across the network.

As to then shutting down the nodes- you'd have to shut down nodes in places all over the world.

Lastly, you could just make a second copy of a given data, new key and then then your plan is foiled.

You should really read more of the Freenet docs- they explain all this.

Re:I doubt that this will actually happen

[Elwood P Dowd]

Actually grabbing every packet being sent to a client (and not through it) and also decrypting it would be difficult. However. A few months ago, /. had a story on some FBI report on wiretaps. They explained that out of all the wiretaps they did one year, 40 of them were obscured by encryption. In *none* of those cases, did the encryption get in the way of them getting the messages, and they didn't have to decrypt anything.

If the feds are tracking you, they'll do it by putting a microphone in your desklamp by your phone, and a bug in your computer keyboard. PGP doesn't help as much when you're on camera.