brainscat.com

Within an hour of my post, Mark Fletcher, CEO of Bloglines, left a comment saying all will be well with regards to the speed of the Bloglines service. I’d like to clarify that the fairy-tale post is somewhat embellished as all fairy-tales are.

I’m a huge fan of Bloglines, they’re the only aggregator I use currently. They’re without peer when it comes to having a web-client that deals in traditional syndication. I can see their technology as possibly supplementary to the new aggregator I’m going to take a poke at writing, as the new client will have a penchant for distribution of large media files, a task that seems incompatible with the Bloglines model.

Let these events stand as proof that pingbacks are useful, and allowed a Mark to know about (some small) buzz regarding his business and react quickly. While it’s refreshing to see companies that are accessable and have a more personal public face, it’s amazing to see a CEO respond so quickly in a blog. It’s really cool to see technology at work in a practical way.

As an aside, it’s interesting to see Mark’s blog about some of the recent hurdles Bloglines has had to jump.

I need help from other developers! I’m creating an aggregator that does the following.

1. Keeps an OPML file of all your content in a centralized location.
2. Checks with a server that keeps track of when your blogs were updated, and when you read them.
3. When fetching content, privately looks to peers for content. If it doesn’t find it, grabs it from the originating site.

This basically solves all of the bandwidth problems with the current syndication scheme. The service provided by the centralized server will be much more lightweight than Bloglines, and will serve as a hybrid news notifier and Bittorrent tracker server. Aggregators would all be Bittorrent clients that did the downloading seamlessly in the background. As far as outward appearances go, it will breathe and act like a normal aggregator.

If so inclined it will handle different mime-types accordingly, and put different types of content (even depending on the channel) in user-defined folders. A plugin system will let people handle the audio in iTunes, Windows Media Player, ZINF, or wherever they like once the content is downloaded.

Once upon a time, people got tired of having to check news sites many times a day to see if something new was available. To fix this, some smart people made aggregators who brought the news to you by getting an extra special page, that websites made available. This page could be lightweight and act as a notifier, only giving summaries, and telling you that new content was ready. Some were far more robust and brought the full content of the site to your screen. Sadly, having all these extra aggregators about created strain on the website; now they had to serve their old content and the new special version of the content. It got worse because the content was served multiple times, over and over again to users throughout the day, every time something new was added. Some people would download overlapping content as often as once every half an hour! This was even worse when nasty bad people used aggregators that were impolite piggies. These bad aggregators gobbled up content that they didn’t need because they didn’t respect the oh-so-important HTTP 304 response, a little message that said, “Nothing new to see here, move along.”

Things got worse when Adam, Dave, and a bunch of other creatures called web-casters and pod-casters wanted to serve large audio files of news and talk. Other people wanted to let their friends know when they had taken new pictures or drawn a new funny comic-strip. These files were big, and took even more time and work for the websites to serve.

Aside from all this, as long as the host could afford to serve all the content, aggregators were well and good if you only used one. A problem was that people wanted to use one at home, but the aggregator at work didn’t know what you had read at home. Smart people created programs like AmphetaDesk or Bloglines to solve this problem and all was well, at least, until so many people started using Bloglines that it slowed down to a crawl at times. You see, all the hosts were happy because they were serving less bandwidth, but more and more people used Bloglines, and Bloglines was serving the content of all the other websites combined!

All kinds of problems were showing up with everybody wanting to know what was new with news, pictures, or computer-radio. Adam became so popular, and his files were so large, that Adam wasn’t allowed to post his audio files as fast as he used to, and had to look for someone else to serve his audio. Bill had to cut back and only summarize the news on his website. Bloglines got slower and slower as more people used it.

Can peer-to-peer save the day?

circle.ch reminds me that the W3C does indeed provide a service to make sure all your links are pointing in the right direction.

My stomach just turned inside out and dropped to my feet. Good bye to my local cvs changes to WordPress.

I recently patched the latest cvs of WordPress to keep your tags when using rss summaries, so that links show up in people’s clients. Thankfully, I uploaded it to Mosquito, and it’s now safe there. I thought it would be a good idea to apply this against the latest 1.2.1 for people who can’t wait. Here’s what I did, see if you can figure out where I went wrong.


virgil2:Documents] bjornrud% wget http://wordpress.org/latest.tar.gz
--10:47:06-- http://wordpress.org/latest.tar.gz
== `latest.tar.gz'
Resolving wordpress.org... done.
Connecting to wordpress.org[67.19.16.238]:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [application/force-download]
10:47:07 (226.26 KB/s) - latest.tar.gz saved [238639]
virgil2:Documents] bjornrud% mkdir t
virgil2:Documents] bjornrud% mv latest.tar.gz t/
virgil2:Documents] bjornrud% tar zxvvf t/latest.tar.gz


You see, I didn’t change my working directory to the directory I wanted to unzip WordPress into (t/). As you might have guessed, I had a directory named WordPress already, and it over-wrote the files in it.

Woe is me.