RSS shows its age in real-time web (SUP and XMPP to the rescue?)

December 21, 2008 By

The real time web is coming at us very quickly, but it exposes major problems in our RSS/Atom infrastructure.

What is the real-time web?

You can get a small taste of that by watching the 5,300+ people I’m watching in Real Time on friendfeed.

The first time I saw the real-time web, I saw it when my tweets showed up on Twitter search and friendfeed within minutes. Sometimes within seconds. Now, imagine a world where everything worked like that. That’s the real-time web.

The problem is that our blogs don’t participate in the real-time web. They publish via RSS. RSS is not real time. RSS only publishes when a service like Google Reader asks for it. It has no way to wave its hand and tell your reader “hey, there’s something new here for you to get.” So, most RSS aggregators just visit on a regular basis, looking every few minutes to see whether something new has shown up.

For blogs that’s just fine. After all, most blogs take a few minutes to a few hours to write and it won’t kill you if you don’t read my words here for 20 minutes or longer.

But there’s a new expectation that we’re having thanks to Twitter. We want everything now in real time. I want to see everything that was published now and respond to it now and I want to have conversations about all that in real time.

This works on Twitter and friendfeed, which were built on real-time principles (er, messaging principles) rather than Web principles.

But when you try to hook the real-time web up to the old creaky RSS web, well, you see that the two aren’t very compatible.

Today I tried to setup an ego feed where I could track stuff that uses my name from around the web in real time. It doesn’t work very well. It’s slow. And, worse, friendfeed can’t tell where the original item came from so it gives it a generic RSS icon. So, it’s not only not real time but it’s ugly as well. I talked more about that with a bunch of people on friendfeed today.

So, what’s the answer?

Well, the geeks are exploring two technologies.

The first is XMPP. This is protocol developed for instant messaging applications but Twitter and friendfeed and others have adopted it. This is why when you Tweet the message shows up in friendfeed so fast.

The second is SUP. This was designed by friendfeed to be more efficient, like RSS. But with the added benefit that the feed provider can raise its hand and say “I have something new for you.” This makes real-time feeding possible, as developer Jeff Smith demonstrates when he built a system that shoved data into friendfeed in just a microsecond.

The third is GNIP, which is trying to build a service that stands between all sorts of services that are supporting the real-time web.

The problem? Very few services that could help the real-time web evolve are using either of these two protocols.

In fact, I was shown a real-time news service that’ll come out in March that didn’t use either of these protocols. Why? They didn’t even know that a real-time web was evolving on Twitter and FriendFeed and that there are dozens of tools like Twhirl and TweetDeck that are built on top of those too. Which is why I’m writing this post.

If you’re a developer, are you thinking about how to make your feeds real time? Why not?

One reason I can see is that it increases the bandwidth needed, especially if you’re pushing out a lot of data. So, in this harsh economic times developers might be unwilling to spend more resources. But there are some things, like searches, that need real-time results. I’d love to hear what developers are thinking here about balancing the need for low-cost systems with real-time publishing.

More info on SUP and the real-time web:

Paul Bucheit, co-founder of friendfeed, started a whole discussion about it.
OurDoings, a photosharing service, was one of the first services that supported the real-time web on friendfeed and they wrote about their experiences with SUP here.
The friendfeed blog has more info on the release of SUP.
Derek van Vilet made a WordPress plugin for SUP and explains that here.

UPDATE: Mike Taylor says I should have mentioned some XMPP resources in this friendfeed conversation. Here’s the ones he recommended: http://xmpp.org
http://metajack.im/
http://stpeter.im/

http://ralphm.net/blog/

Jonathan Jesse, in same friendfeed thread, added: “Robert: on Leo Laporte’s FLOSS Weekly they covered XMPP with one of the developers and the guy who writes the documentation for Jabber is a great overview of XMPP and more info: http://twit.tv/floss49 ”

Filed Under: technology industry Tagged With: , , , , ,

The tale of 20 likes and its impact on news

December 21, 2008 By

I care about news. It’s why I love talking with Gabe Rivera, the guy who makes TechMeme and a bunch of similar sites, like I did in Paris France at the recent LeWeb Conference.

I told him that TechMeme has grown cold for me, which is why I wanted a new system — one where humans bring me the news instead of algorithms.

Rivera countered that he wanted a page that — no matter when you looked at it — would be filled with news from the most credible and authoritative sources from around the world.

And that nailed why it’s cold for me. He’s removed all noise from it. Well, except that the news is noise of a different sort.

But humans are noisy and TechMeme doesn’t include things like Tweets. Here, quick, can you find this Tweet on TechMeme:

“Holy f**king shit I wasbjust in a plane crash!”

Tweets are all about noise, aren’t they? Or was that just news? Hint: you need humans to find the news, algorithms that don’t count video, tweets, or FriendFeed posts as news won’t find them all.

Yes, that Tweet was actually from someone who was on the plane yesterday that crashed off of the runway in Denver.

But humans will. As I found this one pretty quickly after Mike Wilson posted it from the Denver Airport (the rest of his tweets are fascinating, too).

Now, this morning, I’m sure Rivera would say that this Tweet doesn’t belong on TechMeme, but belongs on his sister site, Memeorandum, which is where the world’s news goes. But it’s not there either.

In the past week I’ve read many thousands of items and have liked 757 of them. You can see all my likes on FriendFeed. Here I’m giving you a sample of just my latest 20. These are from 5,363 people who I’ve hand added based on their ability to participate and bring me stuff that makes me smarter.

When I click “like” on something it means I think it’s important enough for you to read. Sometimes I wish it said “share” instead of “like” because some news items aren’t likeable but they are important anyway.

I’m also going to compare to TechMeme so you can see how many of these items appear on TechMeme. They are listed in order from newest to oldest.

1.

I liked this item because Loic’s company, Seesmic, just did a major rearchitecture where they built in XMPP between all major components. What does this mean? Things on Seesmic will appear much faster now on its real time news reader, Twhirl, and potentially a lot faster on real time news services like FriendFeed. In 2009 I believe the real time web will be much more important than it is today, so this is an interesting trend to watch, along with articles about SUP, the protocol developed by FriendFeed to do similar things. Not on TechMeme yet.

2. Seesmic

Loic Le Meur - Seesmic
Some explanations about our new architecture
Some explanations about our new architecture
I liked this item for the same reason, except this one contains a video by Loic. I’ve never seen video like this appear on a top-level post in TechMeme.
3. Blog
make - MAKE Magazine
Take better digital photos
I’m a photography buff and a Make Magazine fan. Anytime they write about an issue it usually is interesting and teaches me something. Also not on TechMeme.
4. Blog
Fred Wilson - A VC
Disrupting Class and Playing Games
Fred is a VC in New York City that always teaches me something through his writings and this is no exception where he shows a crack in our education system and explores how we can get kids enthusiastic again about science and math studies. Not on TechMeme as of 11:37 a.m. when I typed this.
5. Blog
Brian Solis - PR 2.0
Fear Kills Businesses, Dead
Brian Solis is one of the best PR people I deal with and wrote this post for TechCrunch yesterday. I liked this one so that his blog would get traffic and that he’d get credit for writing this interesting post, which implores companies to fight through the fear and keep spending on marketing. It was on TechMeme yesterday because TechCrunch gets on TechMeme quite frequently.
6. Twitter
Louis Gray - Twitter
I shared this one because Louis Gray picks interesting guest posters for his blog and lots of new people are discovering Twitter and I thought this author had something interesting to say on the topic. Matches my own advice to Twitter users: choosing who you follow is the most important thing you’ll do.
7. Blog
slashdot - Slashdot
Warner Music Pulls Videos Off YouTube
I track how idiotic the music industry is and how they choose which services they support. Plus, Slashdot always has an interesting take on the news, especially the geekier stuff, and the commenters there always are both entertaining and smart. This was on TechMeme yesterday.
8. Blog
Loren Heiny - Incremental Blogger
Technology is not there yet
I’ve known Loren Heiny for years (since before I worked at Microsoft) and he’s one of the smartest developers in the Tablet PC world. This time, though, he wants a far better way to share with his readers his CES experience. Me too! Not on TechMeme, but this shows why I love FriendFeed: I can keep in touch with the smart people I know even if they aren’t making top-level news.
9. Blog
(jeff)isageek - (jeff)isageek.net
What SUP FriendFeed?
Jeff Smith is a developer I’ve been following for a long time because he always seems to keep up to date on the cutting edge. Here is no exception as he is testing out SUP, a new protocol that lets developers build systems that post to FriendFeed very fast. Geeky exploration of new technologies rarely gets to TechMeme, but you’ll probably read more about this protocol in first half of 2009. The real-time Web is going to be big next year and SUP will play an important part in it.
10. Twitter
Guy Kawasaki - Twitter
I’ve loved studying typography ever since I got into desktop publishing in the late 1980s (our school got a $5,000 laser printer and I proceeded to start collecting Adobe fonts — later I even beta tested fonts for Adobe and beta tested Acrobat 1.0 too). Anyway, we don’t think enough about the fonts that put type on your screen, so I instantly liked this post. Love how Guy markets his own services too, but you already knew that if you watched the interview I did with him on FastCompany.tv. This isn’t on TechMeme.
11. Blog
Dave Winer - Scripting News
Measuring the worth of a blogger
Dave is one of the first bloggers I started reading and I still read him every day. He has great worth to me and glad he’s played such a big role in my life the past eight years. This isn’t on TechMeme.
12. Blog
Loren Heiny - Incremental Blogger
CES 2009: A mobile showcase for Netbooks and MIDs
Loren again, this time telling us about what he thinks of the latest netbooks. Since he’s been developing software for years for the Tablet PC, I trust his input. His sister works on Microsoft’s Tablet PC team too. Not on TechMeme.
13. Google Reader
Louis Gray - Google Reader
Five things that probably won’t happen in social media in 2009
Notice that this article is not written by Louis Gray. He shared it with his readers in Google Reader. So, Louis is bringing me news I would never have known about. This is a really key role if you want to be a participant in the new live web. We can’t subscribe to it all, but if we help each other out by bringing everyone good new voices or news, then we all win. This isn’t on TechMeme yet. I disagree with the author, too, so that might turn into a separate blog post.
14. Blog
GigaOm - GigaOM
GigaOM Cheat Sheet for National Broadband Plans
Ever since visiting Washington DC I’ve become interested in our national broadband policy and this headline caught my eye. I haven’t seen this on TechMeme yet either.
15. identi.ca
Francine Hardaway - identi.ca
“Redenting @newsjunk: [Frank Rich]: Who Wants to Kick a Millionaire?. http://x.techwheat.com/3EZ A MUST READ FOR INTELLIGENT PEOPLE”
Francine is a friend who teaches entrepreneurship in Phoenix and invests in early-stage companies. Generally when she says to read something it’s well worth the time. This is no exception. Note that this came from her identi.ca account. Only weird geeks use that service. Which makes stuff that happens over there more interesting than on other services. Not on TechMeme, but shouldn’t be because it isn’t about tech.
16. Twitter
Dave Winer - Twitter
Disgusting story that Dave found about how bailed-out banks are rewarding their execs with our tax money. Did you see this on TechMeme, now, of course not. It’s not tech, but you should know about it. Not on Techmeme, but is on sister site Memeorandum.
17. Blog
engadget - Engadget
Cellphone hacked to analyze blood, detect diseases on the spot
Engadget usually has stories about the latest gadgets, but this one caught my eye because of the impact it could have on medicine aroundd the world. You should read the comments over on FriendFeed, though. They think it’ll be used by insurance companies to spy on us.
18. Blog
Om Malik - GigaOM
To Fix Itself, Electronic Arts More Than Just Job Cuts
I visited EA earlier this year, so have been interested in how the company will do through the economic downturn. It totally missed the iPhone app store paradigm shift. You should see my son and how many games he plays on the iPhone. Of course, right now he’s playing Call of Duty 4. That’s not an EA game. And he’s played through to level 80 in World of Wacraft. Not an EA game either.
19. Blog
Louis Gray - louisgray.com
Will Twitter Go Mainstream? The Debate.
This is like arguing Mac. vs. PC. It’ll be a debate for a while, but it was an entertaining post to read on a rainy Sunday morning.
20. Twitter
Ben Hedrington - Twitter
I love developers who try new things out. Check this out. Is this on TechMeme? No. Plus using Google’s App Engine, which is another trend we’re tracking: cloud computing.
Well, I could keep going, but why don’t you just visit my like page and join in? Oh, and please share your FriendFeed account with me so I can track your news too. Thanks!
UPDATE: Of course we’re talking about this over on FriendFeed now, join in!
Filed Under: technology industry Tagged With: , , ,

Tech news in real time

December 15, 2008 By

Faster than a flying TechMeme. Deeper than a Reddit. More comments than a Digg post at Jr. High.

OK, I’m just having some fun, but I just put a link to Facebook Connect Coming to Twitter on my FriendFeed like page. If you haven’t watched that lately you’ll see a ton of stuff you won’t see anywhere else.

Watching 5,000 people tell me the latest news is changing everything about what I think news in the future will look like.

Your friends will bring you the news. Just like here.

Then we’ll talk about it. That’s my comment feed so you can see what I’m talking about.

Then we’ll blog about it, or do our own reporting.

And the cycle will start all over again.

Do you think this has anything to do with why FriendFeed is growing faster than TechMeme is?

And, want to see the news happen in real time without any filtering? Here’s the 5,230 people I’m listening to in real time. Wild.

Filed Under: technology industry Tagged With: ,

Guy Kawasaki says outrageous things about Twitter

December 12, 2008 By

Guy Kawasaki is a pretty influential guy and when he says “Twitter is a weapon,” in an interview I did with him my ears perk up.

But he got more outrageous from there. He took on TechCrunch’s Mike Arrington with a challenge. Guy would rather give up his cell phone for a week than give up Twitter for a week.

Oh, Guy said a few other fun things. Which is what you’d expect from the first technology evangelist (a role he held at Apple back in the early 1980s).

Not to mention that he started a cool social media directory, AllTop, and promptly put his own name at the top of the ego page. Oh, he’s not just outrageous about Twitter. In reality we were there to discuss his new book, Reality Check: the Irrreverent Guide to Outsmarting, Outmanaging, and Outmarketing Your Competition.

“I’m so sold on Twitter.”(1 minute in)

“I think Twitter is, arguably, the most powerful branding mechanism since television.” (2 minutes in)

The essence of a good pitch (3 minutes in)

“I would sell to Toyota,” when I asked him what he’d do about GM (4 minutes in)

Twitter should be bailed out by the government (4:45 in)

“The thing that drives your competition crazy the most is when you are successfull and they are not.” (7 minutes in)

“Is your boss an asshole?” (10 minutes in). Leads to a fun discussion about Steve Jobs. Funny how that happens!

Why your company can’t be successful like Apple or Google just buy spending money on goodies (14 minutes in)

“For me, Twitter is more important than a cell phone.” (16 minutes in)

“Twitter is a weapon.” (17 minutes in)

Thanks Guy for inviting me over for a fun conversation. Now back to Twitter. Follow Guy @guykawasaki and follow me @scobleizer.

UPDATE: over on FastCompanyTV this video is getting more comments than any other in recent memory, which brings on this outrageous Scoble rule: get a famous Twitterer to say outrageous things about Twitter and you’ll get more comments on your blog.

Filed Under: technology industry Tagged With: , , , , , , ,

10 Reasons why Twitter is for you and FriendFeed is not

December 8, 2008 By

Dave Winer talks about why FriendFeed hasn’t gotten super popular yet. He thinks there’s space for a new service between Twitter and FriendFeed. Everyone knows I’m FriendFeed’s #1 customer, but I’ve been studying the two services for a while and have found 10 reasons why FriendFeed isn’t right for you.

1. Twitter has one way to get content into the system. You see a box. You type. You push a button that says “update.” Compare this to FriendFeed. Let’s try to count the ways you can get content into the system.

  • a. the standard way on the top of the page, but there you have to choose whether you are putting in a message, a link, a photo.
  • b. Import your site. Or your Flickr. Or your YouTube. Or your Facebook. Or your Twitter stream. Or your blog. Or your Disqus comments. Or your Upcoming.org stream. Etc. Etc. There are 59 services that can be brought into FriendFeed. Very few services do the same for Twitter.
  • c. You can “Like” an item. Here’s all my likes. There’s 11,500 of them so far, which shows another problem: too much content to go through on FF. Twitter actually has a similar feature, called Favorites, but no one actually uses that.
  • d. You can “Comment” on an item. Here’s my comments. More than 6,000 of them. You really don’t want that kind of distraction. You might have to participate and that wouldn’t be good.
  • e. Then there’s the FriendFeed toolbar bookmarklet.
  • I’m probably missing five other ways you can get content into FriendFeed. Like emailing in items.

2. Twitter has one display of messages that are 140 characters long. You can’t handle the responsibility of longer messages. Plus FriendFeed’s messages include photos. YouTube videos. Play inline audio links. And more. You can’t handle those distractions. Twitter is for you.

3. FriendFeed has a search engine that’s just like Twitter’s search engine. Except you can use that search engine to only search certain data types. On Twitter you only have one datatype. So much less confusing.

4. On Twitter you follow people by your friends telling you their Twitter address. Mine is http://www.twitter.com/scobleizer or you will see someone replying to me by using @scobleizer and you’ll click that link and then click “follow.” FriendFeed has a whole list of recommended users that you’ll need to consider when first signing up for the service. The recommendations, if you’ve followed me, will include such people like Mike Arrington, Charlene Li, Corvida, Om Malik, Steve Gillmor, Jason Calacanis, David Sifry, and many other people. Now you’ll need to consider how those people got recommended to you and that might hurt your brain. On Twitter it just shows you who I’m following on the right side of the page — members who’ve been on longer are on top of that list.

5. Your friends are on Twitter, they don’t yet know what FriendFeed is. So, FriendFeed is totally lame. Of course, if you aren’t in the tech industry, or a US President Elect (er, his staff), or an NBC Camera Man, a well-known wine seller, a supply-chain manager in China, or a well-known newsman on CNN, you probably are actually on Facebook, but that’s a whole nother blog post.

6. FriendFeed doesn’t have Direct Messaging. Twitter does. Twitter FTW!

7. FriendFeed has rooms. Twitter doesn’t. Rooms seem a lot like old-school mailing lists. Makes your head hurt, so they can’t be a good thing.

8. If you don’t want to see someone on Twitter anymore, you unfollow them. If you don’t want them to see you either, you block them. But that’s about it. FriendFeed does both of those but then also lets you hide posts. Did you know you can hide just someone’s Flickr photos? Or her Tweets? Or both? Too confusing. You can’t handle that, which is why Twitter is for you. Any service that has a tutorial on how to use a feature is just not for you.

9. FriendFeed has this Real Time feature. Here’s my RealTime Feed which shows 5,000 people being aggregated together. Twitter doesn’t do that, although Twitter search gets close. You just can’t handle that kind of distraction. Did you know you can put that real time stream on the sidebar on Firefox? You really can’t handle that. It’ll distract you to no end.

10. Twitter has more apps like TweetDeck, Twhirl, a ton of iPhone apps, etc. FriendFeed doesn’t have nearly as many third-party apps, so it can’t be as useful.

I could keep going, but that’s why there are millions of people on Twitter and hundreds of thousands of people on FriendFeed.

Let’s meet in six more months and see if anything has changed. Until then, FriendFeed is just not for you. Sorry.

Filed Under: technology industry Tagged With: ,

The Twitterization of Conversations

November 24, 2008 By

Yesterday I filmed a video about the half-life of conversations. When I started blogging back in 2000 a blog conversation could go for a week or more. Those days are long gone. In this video I cover why, and show you some ways that tools can be used to lengthen the conversation’s half life (which, on Twitter, can be as short as five minutes).

This video caused a conversation to break out on FriendFeed.

[kyte.tv appKey=MarbachViewerEmbedded&uri=channels/6118/277917&embedId=49424382&premium=true&height=500&width=425]

Filed Under: technology industry Tagged With: , , ,
« Older Posts
Newer Posts »