TweetFollow Us on Twitter

May 01 MacTech Online

Volume Number: 17 (2001)
Issue Number: 05
Column Tag: MacTech Online

by Jeff Clites online@mactech.com

The users of Napster couldn’t possibly have known what they were really doing. They thought they were just trading some music. The recording industry thought they were stealing. But what they were really doing, of course, was networking. Peer-to-peer networking, actually. In truth, peer-to-peer (or “p2p”) networking is nothing new, but it has taken a product like Napster, with its popularity and legal controversy, to bring it into the limelight. Now, not only are there new p2p applications, but there are articles, web sites, books, and conferences springing up. Like other “hot” technologies (Java and XML come to mind), there is a great deal of hype surrounding p2p — but there is also substance, if you dig deep enough.

As a side effect of the recent interest in p2p, there has also been a great deal of focus on defining what exactly constitutes a p2p technology. At its most basic, p2p networking is just client-server networking in which the machines involved act as clients sometimes and as servers other times. As a trivial example, two machines running FTP servers can act as peers. Some feel that “true” p2p application, in the modern sense, must explicitly allow for the involvement of computers which are not always connected to the internet and whose IP addresses change. But in truth, a precise definition is beside the point, as is the issue of whether a given application qualifies as p2p. What’s more important are the systems and technologies that are being developed, and the uses to which they are being put. At its core, the interesting part of p2p isn’t that it’s peer-based; the interesting aspect is content distribution, and putting this into the hands of end users.

Content Distribution

From its inception, the web has been about content distribution — it was originally designed as a mechanism for sharing research papers. But end users in today’s web are largely passive — most users don’t produce web content, but only view it. But p2p technologies are changing this, and turning end users into active participants, and they’re doing this as a matter of course — for instance, Napster users are automatically servers of MP3 content. This is a theme which recurs in many p2p applications: users play the role of content providers without needing to make an extra effort to do so, and while they are becoming more involved in the Internet they are not necessarily participating in the web, per se — users are more often sharing files than creating web pages.

This month we are going to take a look at Gnutella, one of the many interesting p2p applications springing up today, both for its technological features and for the insights it gives into the p2p “movement”. It’s a file-sharing-focussed p2p application, and a contender to be the successor to Napster. But first we are going to look at Napster itself, and find out why it has turned out to be such an important technology.

Napster as a Paradigm

The most important technological insight of Napster was its judicious mixing of centralization and decentralization. In a nutshell, what Napster adds to the “two FTP servers” scenario is integration with a search engine. (From a sociological point of view, Napster also adds ease of use, and a focus on a specific type of content.) Searches in Napster are preformed via a central server which indexes the music on each of the connected Napster clients. This allows for fast and efficient searching. But actual file transfers are conducted directly between two client machines, without passing through any centralized server. This is key, because it allows the Napster system as a whole to effectively take advantage of the aggregate bandwidth and disk space of all of its users — there is no need for a huge disk farm to store the content, and an unlimited number of transfers between clients can happen simultaneously without impacting the performance of the system as a whole. The decentralized aspects are in a sense the most inspired strength of Napster, and they are really what has sparked interest in peer-to-peer technologies in themselves. The centralized search facilities are in a sense the greatest weakness, but more from a political or legal point of view than from a technological one, because they necessitate a single authority which can be targeted with legal action, and an essential linchpin which can be “attacked” to shut down the entire system. Unsurprisingly, this has led to a trend in new p2p technologies to try to avoid any centralization at all, in order to provide both legal and technological robustness against attempts to shut down any systems developed. This backlash is a mixed blessing, because centralization in and of itself it not always a flaw — some operations are appropriate to centralize, such as searching, and in the absence of worries about “social attacks” this is often the best approach. At the same time, a community obsessed with maximizing the “p” in p2p is producing some interesting systems which are pushing the limits of what would have been thought to be possible in a purely distributed peer system, and is giving us greater insight into what can and cannot be done without centralization. After the legal controversy dies down I think that we will again be designing systems which are a sensible mix of p2p and traditional client-server, but they will be better than they would have been without the war stories from today’s p2p innovators.

Gnutella

Gnutella began at AOL with developers from Nullsoft, the company behind the popular Winamp MP3 player for Windows, and it quickly became on open source project being developed outside of the company. It has been described as the open-source Napster. At the surface this is true, but Gnutella differs from Napster in several important ways.

To begin with, Gnutella is actually a protocol, rather than an application, and there are several different applications for multiple platforms which speak the Gnutella protocol. Like Napster, Gnutella software allows users to perform searches and transfer files, and the same software plays the role of both client and server, both downloading and vending content. The key difference is that Gnutella is completely decentralized — there is no central entity to service queries. When a Gnutella application starts up, it joins the Gnutella network by connecting to another “node” — to any other machine running Gnutella software. In the beginning, users would find another node to connect to through word-of-mouth, or from web sites which posted lists of IP addresses of nodes, but today there are nodes called “host caches” which are set up to vend the locations of active nodes to new users joining the network (one point of slight, and optional, centralization in Gnutella). Once a node has connected to a handful of other nodes, it is part of the network, and equivalent to every other node. This has been described as a software-based network on top of the physical network, and it’s in a constant state of flux as new nodes join and others drop out — as new connections form and old ones are severed, from one minute to the next. When a user want to search the network for a piece of content, his node sends his query to each of its “adjacent” nodes (the nodes to which it is connected), which in turn forward the query to the nodes that they know about, and so on. Each node can then respond to the query by sending its response back along the network to the node which originated the query. In case you missed it, Gnutella queries happen via a broadcast across the network of users, rather than through a central server which indexes all of the content. More about this in a minute. As responses to a query trickle back to a node, if any of them contain the URL of a file that the user want to download, he can download it directly from the machine which sent that response. This is the same strategy employed by Napster, in that the (potentially large) file transfers bypass the greater Gnutella network completely.

In addition to the search strategy, Gnutella differs from Napster in the search content. A Gnutella query is free-form, and each node is independent and free to interpret a query however it likes. Napster treats queries as searches for MP3s, but a Gnutella query could be looking for music, some other type of file, or a different kind of information entirely. Most commonly, a node will treat a query as a request for a file, and will match the query against either file names or file content (the former would be appropriate for an MP3, and the latter for a news article). But to demonstrate the flexibility of this approach, Gnutella developers set up a small network with nodes which each handled queries in a different way — one interpreted them as requests for stock quotes, one matched them against news headlines, and another evaluated them as arithmetic expressions — and provided a web-based search interface to the network. This demonstrated the flexibility of the notion of a query along the network, and also pointed out that private Gnutella networks could easily be set up to serve novel functions. (This type of private Gnutella network is trivial to set up — it’s just a set of nodes which don’t happen to know about any nodes outside of a small set.)

As mentioned above, Gnutella uses a broadcast-based approach to searching. This strategy is a key feature of Gnutella and its decentralization, and it’s arguably its fatal flaw. It’s a flaw because queries take a significant amount of bandwidth across the network, and this poses a direct barrier to scalability. Gnutella’s approach is actually quite strange, once you begin to analyze it, and it’s difficult to decide whether it’s clever or pathological. Although logically a broadcast, queries are passed from node to node via direct TCP/IP connection, rather than via UDP broadcast. Also, despite being TCP-based, the queries are lossy — nodes will discard their responses if querying nodes are reacting too slowly. This saves things somewhat, as it prevents a few slow nodes from clogging the network. Also, queries have a time-to-live (TTL), just like IP packets — typically, Gnutella queries have a TTL of 7, meaning that queries are only transmitted to nodes 7 hops away. On average, a query reaches about 10,000 other nodes.

The Gnutella community is quick to point out that this “distributed query” approach models the way humans interact, transferring knowledge in an ever-changing network of interactions. On the other hand, as nice as the analogy may sound, it might not be the best design strategy, and the comparison breaks down in the details. If you do the math, even with TTLs and lossy responses, the performance of Gnutella networks will degrade and hit a bandwidth wall when the network grows above a certain number of nodes. Steps are being taken to address the “bandwidth issue”, and more sophisticated software is being developed to implement various strategies. Interestingly, this problem is no surprise to Gnutella developers — it was known at the very beginning, and is an inherent problem with broadcasts. Nevertheless, developers persisted in this approach, demonstrating an enthusiasm (or dementia) which has been attendant to p2p development, characterized as a willingness to go ahead and try things which seem “theoretically impossible but practically possible”. This sounds like a rather irrational philosophy for a community of developers, but despite this Gnutella has actually performed better than the naysayers predicted, and has given us a better idea of what is possible with this type of approach. And although the Gnutella community is now forced to do some technological clean up, they are able to address the real problems that have actually arisen, at a point where they know the strengths and weaknesses of their system better than they would have to start out with, and so are in a better position to solve them.

Further Reading

If you want to learn more about the ever-changing p2p landscape, and about Gnutella in particular, there are several great places to start. O’Reilly’s OpenP2P portal site contains original content as well as pointers to articles on other sites, and has a Gnutella section which can help you keep up to speed as the technology develops. O’Reilly also has a recently-published book, Peer-to-Peer: Harnessing the Power of Disruptive Technologies (ISBN: 0-596-00110-X), with articles on specific p2p technologies as well as articles covering general topics which are relevant across applications. In particular, check out “Gnutella: Alive, Well, and Changing Fast” and “Gnutella and the Transient Web” here, and “The Gnutella paradox” on Salon.com. Also, the Gnutella web site at wego.com is a portal for both Gnutella developers and users, and Gnutelliums maintains information on all of the Gnutella clients available for various platforms. If you are interested in developing p2p software, or just in figuring out how certain things are done, the source code to these clients (often licensed under the GPL) is a great resource. It’s often included with the client distributions, or you can find much of it at GnutellaDev. And finally, if you plan on doing any Gnutella development, you’ll need to know the details of the protocol, which you can find at Gnutelliums and at Clip2.

OpenP2P.com
http://www.openp2p.com/

OpenP2P.com: Gnutella
http://openp2p.com/topics/p2p/gnutella/

Gnutella: Alive, Well, and Changing Fast
http://www.openp2p.com/pub/a/p2p/2001/01/25/truelove0101.html

Gnutella and the Transient Web
http://www.openp2p.com/pub/a/p2p/2001/03/22/truelove.html

The Gnutella paradox
http://www.salon.com/tech/feature/2000/09/29/gnutella_paradox/

Gnutella portal page
http://gnutella.wego.com/

Gnutelliums
http://www.gnutelliums.com/

GnutellaDev
http://www.gnutelladev.com/

The GnutellaNet protocol at MROB
http://www.gnutelliums.com/linux_unix/gnut/doc/gnutella-prot.html

The Gnutella Protocol Specification v0.4
http://dss.clip2.com/GnutellaProtocol04.pdf

Lessons from Gnutella and Napster

The genesis and growth of Gnutella and Napster have a lot to tell us about both the technological and the sociological aspects of new technologies. In particular, Napster is popular not because it is p2p, but because people really like music — it's the content that's really important to the end user, not the means to get there, and it just happened that in Napster's case p2p was the best way to deliver this. I also find it telling that Napster isn't web browser based, but instead uses its own standalone client. This confounds the prevailing wisdom today that everything needs to live in a browser or the public won't use it. In Napster's case, it really wasn't technologically feasible to work inside of a browser, and I think it would have been a disaster if they had tried. The important lesson here is that if the content is compelling enough, and the technology delivers it well, then users are perfectly happy downloading a new piece of software to use. They really don't need to use Netscape or IE for everything, and I hope that developers realize this as they are planning for the next generation of applications.

In a way, Gnutella is a result of embracing the technology of Napster and inferring a set of ideals from it, rather than understanding what really has made it popular. On the other hand, there's really nothing wrong with that, and I don't think the Gnutella community would mind being accused of putting the cart before the horse. Not everything has to be the basis for a business plan, after all. Enthusiasm for the idea of p2p and a willingness to experiment are improving the underlying technology, and showing us what is possible, and in some cases how simply things can be done. And in the end, people are having fun with it — both developers and users. The "let's just try it and see what happens" approach is working out pretty well in fact. So when a business plan does come along, they'll be ready.

 

Community Search:
MacTech Search:

Software Updates via MacUpdate

Latest Forum Discussions

See All

Six fantastic ways to spend National Vid...
As if anyone needed an excuse to play games today, I am about to give you one: it is National Video Games Day. A day for us to play games, like we no doubt do every day. Let’s not look a gift horse in the mouth. Instead, feast your eyes on this... | Read more »
Old School RuneScape players turn out in...
The sheer leap in technological advancements in our lifetime has been mind-blowing. We went from Commodore 64s to VR glasses in what feels like a heartbeat, but more importantly, the internet. It can be a dark mess, but it also brought hundreds of... | Read more »
Today's Best Mobile Game Discounts...
Every day, we pick out a curated list of the best mobile discounts on the App Store and post them here. This list won't be comprehensive, but it every game on it is recommended. Feel free to check out the coverage we did on them in the links below... | Read more »
Nintendo and The Pokémon Company's...
Unless you have been living under a rock, you know that Nintendo has been locked in an epic battle with Pocketpair, creator of the obvious Pokémon rip-off Palworld. Nintendo often resorts to legal retaliation at the drop of a hat, but it seems this... | Read more »
Apple exclusive mobile games don’t make...
If you are a gamer on phones, no doubt you have been as distressed as I am on one huge sticking point: exclusivity. For years, Xbox and PlayStation have done battle, and before this was the Sega Genesis and the Nintendo NES. On console, it makes... | Read more »
Regionally exclusive events make no sens...
Last week, over on our sister site AppSpy, I babbled excitedly about the Pokémon GO Safari Days event. You can get nine Eevees with an explorer hat per day. Or, can you? Specifically, you, reader. Do you have the time or funds to possibly fly for... | Read more »
As Jon Bellamy defends his choice to can...
Back in March, Jagex announced the appointment of a new CEO, Jon Bellamy. Mr Bellamy then decided to almost immediately paint a huge target on his back by cancelling the Runescapes Pride event. This led to widespread condemnation about his perceived... | Read more »
Marvel Contest of Champions adds two mor...
When I saw the latest two Marvel Contest of Champions characters, I scoffed. Mr Knight and Silver Samurai, thought I, they are running out of good choices. Then I realised no, I was being far too cynical. This is one of the things that games do best... | Read more »
Grass is green, and water is wet: Pokémo...
It must be a day that ends in Y, because Pokémon Trading Card Game Pocket has kicked off its Zoroark Drop Event. Here you can get a promo version of another card, and look forward to the next Wonder Pick Event and the next Mass Outbreak that will be... | Read more »
Enter the Gungeon review
It took me a minute to get around to reviewing this game for a couple of very good reasons. The first is that Enter the Gungeon's style of roguelike bullet-hell action is teetering on the edge of being straight-up malicious, which made getting... | Read more »

Price Scanner via MacPrices.net

Take $150 off every Apple 11-inch M3 iPad Air
Amazon is offering a $150 discount on 11-inch M3 WiFi iPad Airs right now. Shipping is free: – 11″ 128GB M3 WiFi iPad Air: $449, $150 off – 11″ 256GB M3 WiFi iPad Air: $549, $150 off – 11″ 512GB M3... Read more
Apple iPad minis back on sale for $100 off MS...
Amazon is offering $100 discounts (up to 20% off) on Apple’s newest 2024 WiFi iPad minis, each with free shipping. These are the lowest prices available for new minis among the Apple retailers we... Read more
Apple’s 16-inch M4 Max MacBook Pros are on sa...
Amazon has 16-inch M4 Max MacBook Pros (Silver and Black colors) on sale for up to $410 off Apple’s MSRP right now. Shipping is free. Be sure to select Amazon as the seller, rather than a third-party... Read more
Red Pocket Mobile is offering a $150 rebate o...
Red Pocket Mobile has new Apple iPhone 17’s on sale for $150 off MSRP when you switch and open up a new line of service. Red Pocket Mobile is a nationwide MVNO using all the major wireless carrier... Read more
Switch to Verizon, and get any iPhone 16 for...
With yesterday’s introduction of the new iPhone 17 models, Verizon responded by running “on us” promos across much of the iPhone 16 lineup: iPhone 16 and 16 Plus show as $0/mo for 36 months with bill... Read more
Here is a summary of the new features in Appl...
Apple’s September 2025 event introduced major updates across its most popular product lines, focusing on health, performance, and design breakthroughs. The AirPods Pro 3 now feature best-in-class... Read more
Apple’s Smartphone Lineup Could Use A Touch o...
COMMENTARY – Whatever happened to the old adage, “less is more”? Apple’s smartphone lineup. — which is due for its annual refresh either this month or next (possibly at an Apple Event on September 9... Read more
Take $50 off every 11th-generation A16 WiFi i...
Amazon has Apple’s 11th-generation A16 WiFi iPads in stock on sale for $50 off MSRP right now. Shipping is free: – 11″ 11th-generation 128GB WiFi iPads: $299 $50 off MSRP – 11″ 11th-generation 256GB... Read more
Sunday Sale: 14-inch M4 MacBook Pros for up t...
Don’t pay full price! Amazon has Apple’s 14-inch M4 MacBook Pros (Silver and Black colors) on sale for up to $220 off MSRP right now. Shipping is free. Be sure to select Amazon as the seller, rather... Read more
Mac mini with M4 Pro CPU back on sale for $12...
B&H Photo has Apple’s Mac mini with the M4 Pro CPU back on sale for $1259, $140 off MSRP. B&H offers free 1-2 day shipping to most US addresses: – Mac mini M4 Pro CPU (24GB/512GB): $1259, $... Read more

Jobs Board

All contents are Copyright 1984-2011 by Xplain Corporation. All rights reserved. Theme designed by Icreon.