RSS feed or reader broken by long thread?

November 8, 2009 10:58 AM

The November Experiment thread has broken something RSS-related.

I'm trying to follow that monster thread via RSS and I can't get the feed to load in my reader (RSSOwl 2.0.0 build 2009-10-24). Checking the log file, I'm seeing this:

MESSAGE Error loading 'Comments on: November is National Let's Try Obscuring Favorite Counts Month'
Problem: Error on line 26308: The character reference must end with the ';' delimiter.
Link: feed://metatalk.metafilter.com/18396/November-is-National-Lets-Try-Obscuring-Favorite-Counts-Month/rss

Is it the RSS feed that's broken, or can my reader just not handle it? Is it working for anyone else?

posted by FishBike to Bugs at 10:58 AM (31 comments total)

I'm guessing there are going to be lots of problems with that feed. Most feed readers aren't expecting feeds to have over 2,000 items in them. Even though your reader is reporting a parsing problem, I'm guessing that the real problem is related to the length of the feed.

I'm not sure what we could do to get around this. 99% of the time, the feeds work well for threads. But the system doesn't work for these outliers, and I'm not sure we should change everything just to handle them. I think you'll probably have to visit the thread in your browser to keep up with it.
posted by pb (staff) at 11:07 AM on November 8, 2009

It's working for me using Google Reader, Firefox 3.5.5, Windows 7.
posted by zinfandel at 11:08 AM on November 8, 2009

Since we're on the topic, is there anything I can do on my end to get the thread to load faster, or remember it's already loaded content? My browser is tied up for about two minutes if I interact with that thread at all. The good news is, I go refill my tea often.
posted by iamkimiam at 11:10 AM on November 8, 2009

No, there's not much to do to speed it up. It depends on your connection, your computer, and the MeFi Server. Luckily the MeFi server side of things has been handling the thread well (knock on virtual wood).
posted by pb (staff) at 11:14 AM on November 8, 2009

Would it be possible for long threads to be paginated?
posted by bigmusic at 11:16 AM on November 8, 2009

Or at least a link to the RSS feed outside of the thread itself?
posted by bigmusic at 11:17 AM on November 8, 2009

The topic of pagination has a long and contentious history on the site, so it wouldn't be a quick decision. The short answer is no. Here's the rss feed for that thread.
posted by pb (staff) at 11:23 AM on November 8, 2009

For what it's worth, I tried loading the feed directly in a browser to have a look at the XML, and it also is unhappy with it. Specifically, it complains that "An invalid character was found inside an entity reference." and it points at something that says "[ampersand]#8364," which I tracked down to this comment.

I figured it had to have something to do with the thread length, but now it looks like it's just an HTML entity entry mistake. Sorry about that.
posted by FishBike at 11:29 AM on November 8, 2009

I'm not sure what pagination means, but I do wish those extra long threads could just be continued as new threads after 200 or so comments. I have a pentium 3 with 256 meg running debian that sits beside my production machine, that is exclusively for metafilter. Those monster threads are out of bounds for me.
posted by StickyCarpet at 11:33 AM on November 8, 2009

The podcast is hosted on a completely different server/network, so that's not a good comparison. By things being good on the MeFi Server, I mean no errors, timeouts, etc. The baseline speed should be the same across all threads.
posted by pb (staff) at 11:43 AM on November 8, 2009

Thanks for tracking down that comment, FishBike. Looks like something isn't being escaped properly for the feed in that collection of characters. We'll get it patched up soon.
posted by pb (staff) at 12:01 PM on November 8, 2009

I'm not sure whether this would count as pagination, but it'd seem to me it could be good to cap longboats somewhere around 2000 comments and then open a related thread? This isn't something that comes up a whole lot...
posted by kittens for breakfast at 12:12 PM on November 8, 2009

I have a pentium 3 with 256 meg running debian that sits beside my production machine, that is exclusively for metafilter.

Seriously?
posted by empath at 12:30 PM on November 8, 2009

Why should the feed have the entire thread in it? That's not the way that RSS generally works.

Another option would be something like the following:

http://metatalk.metafilter.com/18427/RSS-feed-or-reader-broken-by-long-thread/rss?posts=200&content=brief

Dunno if rss feeds are dynamically generated or static, but if they're already dynamic, it shouldn't be too tough to give you some options for how you want to retrieve them.
posted by empath at 12:33 PM on November 8, 2009

Some people go out of their way to be annoyingly stupid.
posted by dead cousin ted at 2:00 PM on November 8, 2009

iamkimiam: "Since we're on the topic, is there anything I can do on my end to get the thread to load faster, or remember it's already loaded content?"

Monster threads kill my Firefox browser too, mainly on account of it having a few dozen add-ons as well as some Greasemonkey scripts which have to scan the whole page after it loads. I've found that these threads load a lot more swiftly in Google Chrome, since it's so pared down in comparison. It worked great for the 5,555-comment Palin thread.

(Also, cillit bang used to host a special website that would only display the last hundred or so comments of any given Mefi thread, but had to take it down for bandwidth reasons.)
posted by Rhaomi at 2:01 PM on November 8, 2009

empath writes "Seriously?"

This essentially describes my secondary PC except I'm running W2K on it. Don't know how much longer though as it is about to need a second replacement power supply and I'm not sure if I'm going to spend any more money on it.
posted by Mitheral at 2:17 PM on November 8, 2009

Metafilter: I have a pentium 3 with 256 meg running debian that sits beside my production machine, that is exclusively for metafilter.

Get an iPod touch. Much faster.
posted by fourcheesemac at 2:52 PM on November 8, 2009

Much more expensive.
posted by Mitheral at 3:23 PM on November 8, 2009

> I'm not sure what pagination means, but I do wish those extra long threads could just be continued as new threads after 200 or so comments.

Pagination means breaking a stream of content across multiple pages. If you're familiar with the common complaint on the blue about websites that force you to click "previous/next page" buttons for no apparent reason than to put more ads in your face, that's pagination in the service of... well, not good.
posted by ardgedee at 3:25 PM on November 8, 2009

Pagination means breaking a stream of content across multiple pages. If you're familiar with the common complaint on the blue about websites that force you to click "previous/next page" buttons for no apparent reason than to put more ads in your face, that's pagination in the service of... well, not good.

Well pagination in this context, pagination does serve a purpose - letting more people be able to read the thread and join in the conversation. While in this specific thread, it may not further the conversation - there are instances that it would be helpful - especially on mobile devices. I think around 500 comments and I think most folks would see the benefit of pagination. As it stands, I can't participate in the thread on many platforms.
posted by bigmusic at 3:44 PM on November 8, 2009

FYI, the thread about the November favorites experiment started crashing Mobile Safari days ago. I'd started skimming it with Recent Activity just because of the time investment required anyway.
posted by Pronoiac at 3:48 PM on November 8, 2009

Oh yeah: considering that that thread is a browser crasher, we might want to link the relevant FAQ instead in the info bar banner.
posted by Pronoiac at 3:51 PM on November 8, 2009

I believe pagination is slated for the December experement.
posted by Sailormom at 5:34 PM on November 8, 2009

Since we're on the topic, is there anything I can do on my end to get the thread to load faster

Turn off greasemonkey and any addons (assuming you're running FF). It does load faster for me in Safari or Chrome, because I rarely use those and they don't have any addons or tweaks or anything.
posted by rtha at 5:35 PM on November 8, 2009

Found the problem and fixed it up. It turns out we weren't escaping the euro symbol (character 8364) properly and the comment FishBike linked to has one. That was probably causing the parsing error, but I still wouldn't be surprised if the size of the feed is causing problems as well.
posted by pb (staff) at 8:49 PM on November 8, 2009

Good point Pronoiac, moved the top banner link to the FAQ.
posted by pb (staff) at 9:24 PM on November 8, 2009

If any of you know StickyCarpet, you know he is almost certainly entirely serious about the computer mentioned above.
posted by ocherdraco at 11:46 PM on November 8, 2009

And I'm just teasing him about the iPod . . .
posted by fourcheesemac at 3:58 AM on November 9, 2009

pb: Found the problem and fixed it up. It turns out we weren't escaping the euro symbol (character 8364) properly and the comment FishBike linked to has one. That was probably causing the parsing error, but I still wouldn't be surprised if the size of the feed is causing problems as well.

The feed is loading fine for me this morning, so that was definitely the problem. Thanks, pb!
posted by FishBike at 5:29 AM on November 9, 2009

Sorry for breaking Metafilter; thanks for fixing it pb.
posted by Mitheral at 6:36 AM on November 9, 2009