Categories October 12, 2001 10:08 AM Subscribe
So is it time for categories on MetaFilter, specifically to filter mainstream news feeds from press releases from more personal, indie items?
willnot, good comment, but I think that's too many. I'd say something like:
News and Major Media
Humor and Independent Web Stuff
Discussion: Politics, ideology, religion, etc.
posted by msacheson at 12:48 PM on October 12, 2001
News and Major Media
Humor and Independent Web Stuff
Discussion: Politics, ideology, religion, etc.
posted by msacheson at 12:48 PM on October 12, 2001
One concern is that by having such categories, MeFi would sort of be ratifying them, so more people would feel free to post stuff. Even though people would be able to filter better, the total volume would surely go up. So I'd suggest working out a solution to the overloaded-server problem first.
posted by rodii at 1:58 PM on October 12, 2001
posted by rodii at 1:58 PM on October 12, 2001
Ah, sorry for the redundancy then.
Yeah, I'm for simplicity too. Anything that can seperate the major media published items from the independent voices and found items.
posted by teradome at 2:12 PM on October 12, 2001
Yeah, I'm for simplicity too. Anything that can seperate the major media published items from the independent voices and found items.
posted by teradome at 2:12 PM on October 12, 2001
Oh, but the inevitable drift about how this post should really have been in some other category. Sigh.
posted by sudama at 2:14 PM on October 12, 2001
posted by sudama at 2:14 PM on October 12, 2001
I heard somewhere that the biggest hit to the server is calling the front page (I was surprised by that and it doesn't entirely make sense to me, so I don't know if it's true or not).
If the default display set didn't include all categories and you had to explicitly ask for them, then they could be seen as supported but not endorsed by MetaFilter. Because most people wouldn't display those categories, it might reduce some of the server strain.
Of course then you're dealing with fights along the lines of sure this didn't belong in category X, but nobody would see it if I put it into category Y so...
posted by willnot at 3:03 PM on October 12, 2001
If the default display set didn't include all categories and you had to explicitly ask for them, then they could be seen as supported but not endorsed by MetaFilter. Because most people wouldn't display those categories, it might reduce some of the server strain.
Of course then you're dealing with fights along the lines of sure this didn't belong in category X, but nobody would see it if I put it into category Y so...
posted by willnot at 3:03 PM on October 12, 2001
willnot:
the front page is larger than most other pages being served on mefi (i.e. comments), and because many people use different views via sorting, the strain is even worse.
to illustrate: if there are 100 threads on the front page, the database must walk over and htmlify each of the 100, or "touch" 100 threads. due to the complexities of the sorting process, when you change the view of how your threads are presented, the database may take at most 700 touches. it still only needs to htmlify and output the 100, which would be the final, sorted output.
(if you were wondering: one of the most common, efficient general sorting algorithms we are currently aware of -- the quick sort method -- has a complexity of O(n * log2 n). (i would be very surprised to learn that mSQL, matt's db, uses any other algorithm for sorting purposes.) that means that for n = 100 threads, the sort takes 100 * log2 100 where log2(100) = a number between 6 and 7 (2^6 = 64, 2^7 = 128). however, you always round up to the ceiling number, which in this case is 7; thus 100*7, which is 700.)
all this is not to discount the hit the server surely takes when it displays the information -- it's just that, as a matter of algorithmic complexity, sorting is the most strenuous thing you can do at metafilter. it won't hurt to serve this up a couple of times, but assuming mefi takes a lot of simulataneous db queries, i think the sorting could really give matt's computer a beating.
the other half of the story is RAM, of course; all the information being pushed through the database to your browser needs to be stored in RAM at some point, and when you've run out of that, you need to start using virtual memory (aka the hard drive). i needn't explain how RAM is many times faster than the hard drive in terms of data access. the more RAM the better. while sorting probably is the worst villain from an algorithmic standpoint, i wouldn't be surprised at all to find out worst load is simply from copying the link and comment text to RAM prior to sending it out into the internet. it is, in the end, probably a combination of the two... and maybe some other things i hadn't accounted for.
posted by moz at 3:34 PM on October 12, 2001
the front page is larger than most other pages being served on mefi (i.e. comments), and because many people use different views via sorting, the strain is even worse.
to illustrate: if there are 100 threads on the front page, the database must walk over and htmlify each of the 100, or "touch" 100 threads. due to the complexities of the sorting process, when you change the view of how your threads are presented, the database may take at most 700 touches. it still only needs to htmlify and output the 100, which would be the final, sorted output.
(if you were wondering: one of the most common, efficient general sorting algorithms we are currently aware of -- the quick sort method -- has a complexity of O(n * log2 n). (i would be very surprised to learn that mSQL, matt's db, uses any other algorithm for sorting purposes.) that means that for n = 100 threads, the sort takes 100 * log2 100 where log2(100) = a number between 6 and 7 (2^6 = 64, 2^7 = 128). however, you always round up to the ceiling number, which in this case is 7; thus 100*7, which is 700.)
all this is not to discount the hit the server surely takes when it displays the information -- it's just that, as a matter of algorithmic complexity, sorting is the most strenuous thing you can do at metafilter. it won't hurt to serve this up a couple of times, but assuming mefi takes a lot of simulataneous db queries, i think the sorting could really give matt's computer a beating.
the other half of the story is RAM, of course; all the information being pushed through the database to your browser needs to be stored in RAM at some point, and when you've run out of that, you need to start using virtual memory (aka the hard drive). i needn't explain how RAM is many times faster than the hard drive in terms of data access. the more RAM the better. while sorting probably is the worst villain from an algorithmic standpoint, i wouldn't be surprised at all to find out worst load is simply from copying the link and comment text to RAM prior to sending it out into the internet. it is, in the end, probably a combination of the two... and maybe some other things i hadn't accounted for.
posted by moz at 3:34 PM on October 12, 2001
moz has hit on some of it.
The front page view for members is what's killing the box. Everyone has different custom number of days shown, they have come and gone at different times, etc. Non-logged in users see a static page that is only rendered once every five minutes.
When you request the front page as a member, it's custom to just you, and storing it in memory is almost pointless. When 100s of members are hitting the front page at once, that's a lot of custom processing.
Thanks to a helpful member, I'm going to more than double the RAM in the machine next week, hopefully that will improve things around here. Next after that will be a processor/motherboard upgrade.
posted by mathowie (staff) at 3:44 PM on October 12, 2001
The front page view for members is what's killing the box. Everyone has different custom number of days shown, they have come and gone at different times, etc. Non-logged in users see a static page that is only rendered once every five minutes.
When you request the front page as a member, it's custom to just you, and storing it in memory is almost pointless. When 100s of members are hitting the front page at once, that's a lot of custom processing.
Thanks to a helpful member, I'm going to more than double the RAM in the machine next week, hopefully that will improve things around here. Next after that will be a processor/motherboard upgrade.
posted by mathowie (staff) at 3:44 PM on October 12, 2001
Suggested topic list:
Current Events
Arts & Entertainment
Perennial Debates
On The Web
Dude, Where's My Pony?
posted by kindall at 5:42 PM on October 12, 2001
Current Events
Arts & Entertainment
Perennial Debates
On The Web
Dude, Where's My Pony?
posted by kindall at 5:42 PM on October 12, 2001
Here I am!
Maybe allow users to use NON custom settings? I have NEVER set any settings in my lifetime here, Its just somethin I don't do.
Poor server.
posted by darkpony at 7:36 PM on October 12, 2001
Maybe allow users to use NON custom settings? I have NEVER set any settings in my lifetime here, Its just somethin I don't do.
Poor server.
posted by darkpony at 7:36 PM on October 12, 2001
You are not logged in, either login or create an account to post comments
So how bout it Matt? Did you already have a base set in mind? Are you looking to go with a very small number of categories that encompas a lot of posts or are you planning to try to anticipate every possible type of post?
Based on what I've seen recently:
Best of the Web
Wackiest of the Web
Web Design & Blogging News (possibly 2 seperate categories)
Breaking News
Politics
Discussion
Humor
posted by willnot at 12:19 PM on October 12, 2001