Want to cut MetaFilter's bandwidth in half?

October 3, 2003 10:40 AM

Want to cut MetaFilter's bandwidth in half? Put the comment post function in a pop-up window. More inside.
posted by squirrel to Feature Requests at 10:40 AM (40 comments total)

Popups suck.

If your site requires a whole new window to work then I will never visit it again.

Besides, how the hell will I post from my Treo if the form is dependant on Javascript?
posted by bshort at 10:46 AM on October 3, 2003

The way my browsers work, they check for a page update and reload the whole thread with every comment preview. If this is a long thread, and it's a busy time of day, this can bottleneck at the server.

If there were a way to post a comment without loading the thread code twice, (with the option of refreshing the page right before posting, to preserve those important "on preview" comment opportunities) the overall demand on the MetaFilter server would be reduced. Would it not?
posted by squirrel at 10:47 AM on October 3, 2003

actually it would cut MetaFilter's bandwidth by more than that since I would end up using it half as much as I do now. Yuck.

Also, catching newly-added posts when you preview can be very useful. And the extra logic needed on teh server to remember your state and how many comments were there when you decieed to post would probably go a long way to eating any benefit one might get.
posted by Space Coyote at 10:51 AM on October 3, 2003

Here's some pseudocode that would work.

//If a user hits the comment page for the first time (not a preview) then
   //Take a timestamp and store it in a hidden variable on the client side.
//Present the comment form with the hidden timestamp.
//Only show the main topic, any comments that have occurred since the timestamp, and the comment form.
posted by bshort at 10:55 AM on October 3, 2003

oh come on. talk about knee jerk reactions. Every function has its use somewhere, and popups are no exception. Even the blink tag has a use (even if only very, very, very, very sparingly). Just because advertisers overused and abused the popup window function doesn't mean that there are absolutely no valid and good uses of it.

However, since most browsers block popups for that reason makes the valid use of them a bit problematic. Good thought, squirrel.
posted by crunchland at 10:57 AM on October 3, 2003

I didn't think bandwidth was the issue. I thought processor throttling and database grunt work was the bottleneck.

On my personal website I actually use the popup comment strategy. It sucks from a backend and maintenance standpoint, and I've actually gone the other way on the next version of the code. So, I've used and coded for both. Matt's way is better. IMHO.
posted by y6y6y6 at 10:58 AM on October 3, 2003

I could just cut all the comments from comment preview, but sometimes people like to look at old comments while they are writing.

How do I solve that problem? Just show the last few comments?
posted by mathowie (staff) at 11:16 AM on October 3, 2003

How about just not showing the entire thread on preview? It'll also eliminate the "(on preview) what XXXX said" non-sequiturs.
posted by mkultra at 11:17 AM on October 3, 2003

(on post) what mathowie said ;)
posted by mkultra at 11:18 AM on October 3, 2003

I agree with mathowie's point about being able to read (and copy-quote from) the preceding comments. This could be achieved by making the new comment model he describes a spawned window. I guess it doesn't have to be a pop-up, per se. Probably better if it ain't.
posted by squirrel at 11:28 AM on October 3, 2003

Can't you use DOM scripting or something to see what the previewed version of your post would look like without updating the thread?

That's still problematic though, because I see so many comments when people modify their thoughts after a first or second preview.
posted by insomnyuk at 11:28 AM on October 3, 2003

Popups suck. If your site requires a whole new window to work then I will never visit it again. Besides, how the hell will I post from my Treo if the form is dependant on Javascript?

The behavior of having a link open in a new window doesn't have to be done in Javascript, no sir. Plain HTML will suffice:

<a href="blahblahurl" target="newWindowName">anchor text</a>

This degrades well, too... if the UA doesn't understand the target attribute, it just follows the link.
posted by weston at 11:31 AM on October 3, 2003

matthowie: How do I solve that problem? Just show the last few comments?

I'd argue against that. Some threads launch into parallel discussions; the sub-threads get intertwined and the genesis of a particular thought can reach back to the thread's beginning. I personally find it valuable to scroll up to others' comments as I'm writing mine. Arbitrarily selecting some "number less than all" would limit that.

How about two functions instead of one (I'm brainstorming the labelling here): "Post as Reply" would display the entire thread and "Post as Comment" would not?
posted by JollyWanker at 11:34 AM on October 3, 2003

1. Pop-up blockers do not block voluntary javascript pop-up stuff, like Haloscan comments on people's blogs and such.

2. Since the comment would be in a seperate window, one could choose whether to refresh the other comments on the other page anyhow, right? It wouldn't limit "on preview" comments.
posted by Ignatius J. Reilly at 11:41 AM on October 3, 2003

Can't you use DOM scripting or something to see what the previewed version of your post would look like without updating the thread?

That's actually a terrific idea, and it's certainly possible for anyone with a modern browser to read, write a comment, preview it, preview it again, then post it, all with a single pageview.

....but I'm not much of a javascript programmer.

Anyone? Anyone feel like doing this? Has everyone seen kuro5hin's active javascript loading of threads before? It's pretty freaking cool, and kind of the same technology I have in mind here.
posted by mathowie (staff) at 11:50 AM on October 3, 2003

I can't see this having a huge effect on bandwidth. I would think most of the bandwidth is used by lurkers and non-members.
posted by IshmaelGraves at 11:50 AM on October 3, 2003

This might help... do a find on page for "Click here for example" for relevant text. So the preview button would not be a submit button, and its onClick event would set the textnode and append the child text--the document.form.comments.value. Viewing the source on that same page shows how easily this can be added to existing code--so it might be possible to add this to the comments div already in use. I could whip up the exact example code if this is what you were thinking--but I'm guessing I've missed the point since I'm unfamiliar with k5's js.

(on preview - I previewed this post 7 8 times)
posted by Ms.JaneDoe at 12:30 PM on October 3, 2003

I'd like to be able to see the last ten posts before I submitted it, but it's also not a big deal to tack-on another post afterwards if there's something else worth responding to.
posted by The God Complex at 12:44 PM on October 3, 2003

Hey Mr. Howie - got a quick snippet fix for you if you like it:

mefitestpage.html

Go to the page, enter a comment, hit preview, and see the changes happen inline. Works in IE4+ and Mozilla 1+, dunno about Safari or older browsers. If you like, I can add browser-sniffer JS, or that can be done on the server if it's easier for you - older browsers just get the regular old reload-preview. If you have text-massaging on the back-end, that would have to be replicated here somehow, shouldn't be too hard. Lemme know, or take and run with it. (posting here so others can comment as well)
posted by kokogiak at 12:47 PM on October 3, 2003

kokogiak: works in safari but not ns4.7(but what does?)--execellent work/example. Using innerHTML also fixes the html tags bug that my example has (which I realized post posting).
posted by Ms.JaneDoe at 12:57 PM on October 3, 2003

Yeah, NS 4 and all older browsers could be excluded from this "feature" pretty easily, just depends on Matt's preference - client or server-side. Users of older browsers would just get the same MeFi experience everyone gets today.
posted by kokogiak at 1:01 PM on October 3, 2003

kokogiak: works great in mozilla Firebird .7+
posted by monju_bosatsu at 1:24 PM on October 3, 2003

kokogiak - works great in firebird .6 and ie6 on win2k,
posted by rhapsodie at 1:55 PM on October 3, 2003

I'm guessing that proportionately few of MeFi's visitors ever post. The bandwidth taken up by the posters would be cut in half, but it might be a negligible effect overall.
posted by scarabic at 2:20 PM on October 3, 2003

I agree scarabic, but there have been countless times I've typed, previewed and edited many, many posts that I never had the guts to hit "post" for--preview page loads may be a small amount, but probably not as small as you are thinking.

But if cutting down on bandwidth/server resource use is really the ultimate goal, some mechanism for new post notification would probably be more successful. Maybe a "watch this thread" button that could launch a pop-up window where the only info would be the number of comments (or something) to that particular post. Users could spend their time pulling that tiny bit of HTML instead of the entire page. You'd still have the db connect, but I can't imagine how you'd avoid that--someone else might though, kokogiak?

(oh, and kokogiak, I just noticed the detail/simplicity with which you dealt with the post button--very nice)
posted by Ms.JaneDoe at 2:30 PM on October 3, 2003

Yeah, the 'on preview' experience would be different - you could concievably fetch the latest comments (since datestamp XXX) in a hidden iframe, and dynamically add them into the page in the same manner as above, if the mefi backend can take such granular requests.

Or, you could pop open a new window with the latest comments just for those who want to check the latest updates without altering their post. (not as elegant, but simpler to execute).
posted by kokogiak at 2:41 PM on October 3, 2003

Would there be any value in caching threads in a temporary table for 60 seconds, or some other database-level solution?

I'm not sure how many joins are involved but there may be some degree of processing reduced.

I don't know the db structure at all, so I'm stabbing blindly at field names etc. I think this solution's not optimal, but maybe a stronger db geek can run with it?

create table tblCacheTime( intThreadId int, datTimestamp datetime )
// stores the time a thread was cached

create table tblCachedThread (intThreadid int, intCommentId int, intUserId int, txtComment text, datTimestamp datetime/*, etc*/)
// stores all the comments in a thread

create procedure spReturnComments( @intThreadId int )

// get the time the thread was cached
declare @datCachedTimestamp datetime
select @datCachedTimestamp = datTimestamp from tblCacheTime where intThreadId = @intThreadId

// if the cached is older than a minute, recreate it
// I can never remember the datediff options, so I think ss is wrong, but I mean seconds
if( datediff( ss, @datCachedTimestamp ) > 60 )
begin
delete from tblCachedThread where threadId = @intThreadId
// load the cache table with all the comments
insert into tblCachedThread select * from tblComments where intThreadId = @intThreadID
end
else
select * from tblCachedThreads
posted by cCranium at 3:01 PM on October 3, 2003

Oh, alternatively, store the number of comments rather than the timestamp, and if it's changed refresh the cache table otherwise get the results from there.
posted by cCranium at 3:08 PM on October 3, 2003

Crunchland: "oh come on. talk about knee jerk reactions. Every function has its use somewhere, and popups are no exception."

Aaack!! HEATHEN! BLASPHEMER! ...where'd I put that crucifix...? BACK! BACK I SAY, YOU SPAWN OF SATAN!

*everbody looks at him funny. Zach dislodges the capslock key.*

Ah wuz jus' kiddin', gawrsh.
posted by ZachsMind at 3:26 PM on October 3, 2003

[this is good]
posted by eddydamascene at 3:39 PM on October 3, 2003

I can see scarabic's point about posters being perhaps a drop in the bandwidth bucket, but there's also the posting user's experience to consider. Everyone here knows how much longer it takes to post a comment to a thread when it's a busy day. Reducing the number of times required to refresh a page before posting a comment impacts the posting user's experience as well as reducing server load.
posted by squirrel at 5:02 PM on October 3, 2003

Troodat. Improving the latency would be worthwile. I think your proposed solution would be simple, pure, and uncomplicated. But to address the various issues people have raised, and arrive at a truly uncompromised solution, some tricky-ness beyond either of our ken would be required. I leave it to the code-meisters. Just be sure it still works on squirrel's Mac SE - or whatever he's using nowadays.
posted by scarabic at 5:33 PM on October 3, 2003

SE30, fool!
posted by squirrel at 5:43 PM on October 3, 2003

kokogiak, that's some kickass work, thanks.
posted by mathowie (staff) at 7:22 PM on October 3, 2003

kokogiak is my web-hero. But I've said that before.

Also, popups (even summoned ones) do suck, I think.
posted by stavrosthewonderchicken at 9:01 PM on October 3, 2003

This could save Matt a lot of bandwidth.

This could also keep us from gouging our eyes out while waiting for comments to reload when we are in remote areas and deprived of broadband access.

And it doesn't require any pop-ups!

[this is very good]

Kudos to squirrel for the great idea and kokogiak for the excellent work!
posted by cup at 9:34 AM on October 4, 2003

The biggest improvement in bandwidth and processing power would come from breaking the comments into pages, though it would would take away some essential Mefi goodness.

Here's why: Once a page is full (perhaps 25 comments), it's not going to change-- allowing both the client to cache it for return visits and the server can cache it for reduced database hits. The only page which is reloaded and regenerated constantly is the bleeding edge of the thread.
posted by 4easypayments at 10:35 AM on October 4, 2003

Kudos to squirrel for the great idea and kokogiak for the excellent work!



Rock onkokogiak! Rock on squirrel! On to the next party!
posted by squirrel at 8:07 PM on October 4, 2003

insomnyuk mentioned DOM scripting instead of a pop-up window in the first place. Go me!
posted by insomnyuk at 8:59 PM on October 5, 2003

Rock on, insomnyuk!
posted by squirrel at 10:58 PM on October 5, 2003