Idea: Metafilter Link Checker (Needs a Fancier Name) September 18, 2009 8:59 PM   Subscribe

Idea: the Metafilter link-checker. Take the part of the 'new post' process that double-checks against prior posts, add the part of Mefi that conjures up the related posts, put a link to the feature up on the two lines of links near the logo.

Maybe add auto-links to site-specific searches via Google and Yahoo just for convenience's sake.

Yes, there are other ways to do it, including prepping a new post. But it'd make it a few degrees easier: as it stands now, in order to get to the part of the Mefi posting process that double-checks your link against doubles, you essentially have to write it up first, and THEN get it rejected for doublehood.

This'd be a much easier way to quickly ascertain that something's a double before writing up a good post.

And if the "related post" stuff was added to it, hopefully whatever algorithm's in use there might be able to give a clue as to whether the subject matter was handled before, even if the link itself wasn't a double.
posted by WCityMike to Feature Requests at 8:59 PM (36 comments total)

This post was deleted for the following reason: Poster's Request -- loup



Not speaking to the widget idea at all, but I find that I can pretty much find anything by doing a tag search for the two or three most relevant tags for my post topic and Google gets the rest. And, sometimes I cheat and just put all my links in the box and press preview and see if they've been posted before.
posted by jessamyn (staff) at 9:03 PM on September 18, 2009


I'm not sure how the feature would differ from the existing site search. You can type www.example.com into the current search and get a list of posts that have that link. How would a link-checker be different from that?
posted by pb (staff) at 9:20 PM on September 18, 2009


Shit. There goes my dissertation on the history of www.example.com in popular culture.
posted by yhbc at 9:31 PM on September 18, 2009 [1 favorite]


hopefully this ease would lead to a reduction in dupes

It can't possibly lead to a reduction in dupes if it's using the same algorithm that is already is in effect when making a post. All posts already go through the existing dupe checker logic, so if it's a dupe which would have not been detected (e.g. non-canonical domain names, different query params, same-content-different-site) then it's not going to be magically detected by putting it through that same exact logic twice.
posted by Rhomboid at 9:53 PM on September 18, 2009


I see that it'd frame the search results in terms of looking for links, but I don't think the effect would be much different than the current search results. And even at the bottom of every current search results page is, "You can also try a Google search for www.example.com limited to MetaFilter" that links to the Google search. (Sticking with the example.com theme we have going.)

And I'm not sure that breaking this out from the posting process would help. You can run a quick link or two through the standard search before you start writing a post, and then you'll have the benefit of a search that checks every link in your post as you're submitting your post. If people won't use the standard search or check similar tags now, I don't think they would use yet another specialized search that just checks for duplicate links when they're going to get that as they post anyway.

We do get duplicate posts that waste a bit of time and I think it's good to think about ways to reduce them, but I think we have a number of options in place already.
posted by pb (staff) at 9:54 PM on September 18, 2009


I'm against anything that lowers the bar for effort needs expending when making a post to MeFi. This coming from a poster who expends the least amount of effort necessary to get something on the blue. Also, site search is well past the point of being difficult to use. As long as you're logged in, you don't even have to input the www. or the .wevs—just put the meat of the URL (eg. kernsholler) in the search box and keep an eye out for '[keyword in HTML]'. If it's harder for people who want to post links to frequently-linked sites, all the better.
How am I so socially and financially liberal, yet so MetaFilterally conservative?
posted by carsonb at 9:58 PM on September 18, 2009


And, sometimes I cheat and just put all my links in the box and press preview and see if they've been posted before.

Bingo. Anytime I want to make a post, I do the above first, before even trying to coherent a write sentence.

I'm not sure what problem your solution would fix Mike.
posted by Brandon Blatcher at 10:11 PM on September 18, 2009


This idea kinda took off.
posted by tellurian at 10:51 PM on September 18, 2009 [1 favorite]


Dude, I looked at this and was like, well, I'm against it because you should just do your search anyway, you lazy bastard. And then I thought, well, it's kinda dumb to be against things just because they'd be more convenient, like somehow new members and posters wouldn't be working as hard at making their FPPs and it's effort that counts, dammit.

Then I was like, hey, it's OK as an idea, but y'know, if it were left up to me, we'd never get it because I don't know how to code this shit, and if I did, I'd imagine that I'd have, like, a million more things on my plate before I got to it. So if I really wanted it, I'd teach myself how to code it before getting all insistent and entitled, or more likely just say, Fuck it.
posted by klangklangston at 11:37 PM on September 18, 2009


In most software projects, the default answer to feature requests is and should be 'no'.
posted by empath at 11:39 PM on September 18, 2009 [4 favorites]


The core of Mike's idea might be good i.e. the part about having an explicit way for a user to search for duplicate links.

As to the rest of your long winded thoughts about the site and whether it accepts user input, I found to be so full of half baked, navel gazing assumptions it instantly snapped me awake this morning, so thank you for that.
posted by Brandon Blatcher at 5:30 AM on September 19, 2009


This idea kinda took off too. (One MeTa post, one pony request, one successful implementation - batting 1.000). So kids if you're reading at home, remember - whether you're asking for an extra piece of cake, begging to stay up to watch a movie or requesting a new feature on metafilter, always put sugar atop your pretty please.
posted by cashman at 7:07 AM on September 19, 2009


I would honestly be very curious to know two statistics about the "feature requests" subcategory of Metafilter. The statistics would be: (a) how many of them have, within say 1-5 posts of its start, a unfavorable comment from an admin suggesting the feature won't be implemented

You act as though not implementing stupid requests is a problem. If the default answer was yes, we'd have favorites removed this week, and then put back next week.

Then we'd have a digg-style ranking algorithm for posts, then customizable profiles that make even myspace want to puke.

Next we'd have a full gmail interface recreated in mefi mail, which only 17 people would use, but would require 3 months of developer time.

Next we'd be able to sort threads by upvotes which would result in most people missing the whole conversation and chiming in with more asinine comments than ever.

Look you've got a site here that has one full time developer (pb) and probably another half-time developer in mathowie. That's not a lot of coding to go around. I don't think you have any idea how difficult it is to implement features, test them exhaustively, troubleshoot, and fix the inevitable bugs, all on top of the everyday day-to-day stuff that goes into keeping a website running smoothly. They've got a list of plans for the site, and they're plugging away at it, but no, they won't jump at your every whim.

Your five bucks entitles you to contribute to the site, not to have the site developers change things around according to your whims. Deal with it.
posted by chrisamiller at 7:20 AM on September 19, 2009


Man, I'm getting tired of whiny complainers.
posted by languagehat at 7:56 AM on September 19, 2009


i could just perpetually have sucky ideas ...

Actually, you just have a lot of ideas. And MeFi is a site that doesn't implement a lot of new features regularly. That's totally fine but realistically we implement a small subset of feature requests which means that if you put in a lot of feature requests very few of them will be implemented. This has always been true.

maintaining a fiction that ideas are welcome and given full consideration is bad. Because, even if that's what you guys will go on to say, I just don't think that's actually the case.

The site's only programmer responded to you with interest within 25 minutes of you making this request past midnight on a Friday.

Look WCityMike, I know you're frustrated with how the site responds to you, and I know this is true for other MeTa threads you post. We run a really lean site here and one of our mods is on vacation for the entire month. Every new thing that we roll out requires implementation, testing, FAQ-writing, and then the inevitable debugging and "I liked this site better before..." stuff. It's not just a yes/no "will this feature make the site better/worse?" assessment.

If the community is clamoring for a feature, we often try to make that work. If a few individual users want a feature that other people don't see the benefit of, we're less excited about it. The info dump is public, you're more than welcome to run the MeTa threads through some assessment to see how we roll vis a vis feature implementation, but MeTa is really, as we see it, for suggesting features to the community to see what's what and if people are all like "hey yeah!" then we move forward.

It's not that you don't have a good idea or that we haven't thought about it [in this case, as well as in other cases] but that getting site-wide buy in is a tall hurdle, and one that's infrequently met.
posted by jessamyn (staff) at 8:03 AM on September 19, 2009


MeTa is really, as we see it, for suggesting features to the community to see what's what and if people are all like "hey yeah!" then we move forward.

We were all like "hell no!" to the removal of the colour shifting feature, but you people didn't give a shit. Did you? You don't care about us. You only care about yourselves.
posted by gman at 8:37 AM on September 19, 2009


Jessamyn always had a way of explaining things so I could understand them...

WCityMike, I think the short explanation here seems to be that it ain't really broke, so why fix it?
posted by runningwithscissors at 8:42 AM on September 19, 2009


Jessamyn always had a way of explaining things so I could understand them...

Yeah, what happened to that?

Just kidding, the mods here are exceptionally well-spoken. I always look forward to their responses to just about anything.
posted by owtytrof at 9:52 AM on September 19, 2009


You know, it might not feel as frustrating if you didn't make so many feature requests. Not to be an asshole or anything. Again. Just saying.
posted by katillathehun at 9:58 AM on September 19, 2009


*cuts open WCityMike's skull, pokes around in his ideameats*
posted by Eideteker at 10:35 AM on September 19, 2009


*bids goodbye to eide, forever*
posted by special-k at 11:27 AM on September 19, 2009


Why you all up in his cortex?!
posted by Brandon Blatcher at 11:29 AM on September 19, 2009


How many times have I told you not to drink WCityMike's koolaid?
posted by special-k at 11:43 AM on September 19, 2009


Look WCityMike, I probably came off as a little harsh there, but in my defense, so did you. I'm sorry, you're sorry - let's call it a wash and get back to snark-as-usual.
posted by chrisamiller at 3:25 PM on September 20, 2009


One of the things that this conversation pointed out to me is how fundamentally different search on MeFi is from a Google search of the site. When pb said "You can type www.example.com into the current search and get a list of posts that have that link," I was actually really surprised. Sure enough, a site search for the main link of my most recent post pops that post right up, where, if I do a Google search, I just get the search engine equivalent of a blank stare.

Because I (like many people, I imagine) am so used to using Google (and other search engines that work similarly), it never occurred to me that MeFi search might work this way. I think one of the reasons why WCityMike's proposal seems like a good idea at first blush (at least to me) is that the unique aspects of MeFi search aren't made explicit. You have to be really paying attention to notice the grey "(keyword in HTML)" notation after a search result to realize what you're getting. Is there some way to make it clear how search works here, so people will know that they essentially have a "link checker" at the bottom of every page, and will understand better which searches work better on the site as opposed to Google?
posted by ocherdraco at 9:30 PM on September 20, 2009


I use the the New Post page as WCity Mike's link checker from time to time. I find something I think is interesting and wonder what MeFi had to say about it, so I go to New Post, plug in the link and voilĂ  -- there's the MeFi post about my link. It works fine.
posted by Methylviolet at 11:16 PM on September 20, 2009


Good point, ocherdraco. MeFi Search is second nature to me, but we don't even have a FAQ entry about it. Our current FAQ about double posts suggests using the New Post form to search, but the site search would be much easier. (I'm guessing that was written before we had a local site search.) So updating that entry and adding a new one with a more detailed description of the site search could help.
posted by pb (staff) at 9:39 AM on September 21, 2009


I can totally do that and make it make sense and explain how our search really works.
posted by jessamyn (staff) at 9:55 AM on September 21, 2009


Excellent. Thanks, guys.
posted by ocherdraco at 10:17 AM on September 21, 2009


Yeah, I've dropped an occasional comment explaining this or that difference between google and site search—they each work well for specific tasks, less well for others—and codifying some of that in an easily-accessible place seems like a good plan.

To sort of reply (and reiterate) late to a couple specific points WCityMike was making in argument for the original proposal here:

1. We delete a fair number of doubles, but we don't spend much time deleting them, because they tend to be very straightforward. Maybe 1% of the doubles we delete require more than a glance at the flag queue and in the thread for a "previously" link. So further reducing the double load isn't a big priority; the time savings would be pretty minimal.

2. The doubles we do delete mostly wouldn't show up in a link checker that does what the posting form does now; they tend to be doubles-in-content at different urls. We've thought before about ways to make the existing link-checking a little more robust in its dealings with urls that can be similar but not identical (e.g. the various cruft at the end of a youtube link), but while that might get done at some point it only deals with a very specific subset of these, uh, heteroduples.

3. Generally speaking, "it'd be easy, you just need to combine some code right" isn't a good way to go in arguing for implementation. It might be a snap to do initial implementation, it might be a lot less of a snap, but in any case asserting that it just won't be hard to implement ergo we should implement it is making some pretty big assumptions about the amount of work that might be involved in planning and coding and testing and documenting something.
posted by cortex (staff) at 9:43 AM on September 22, 2009


I don't think this idea is worth its own MeTa thread*, and since it's tangentially related to this, I'm asking it here: in addition to checking a post for previously posted links, would it be possible for MeFi post preview to tell us if we've duplicated links within our posts? I'm sure that sometimes it's done on purpose, but when I'm building a post (particularly one dealing with a lot of similar URLs, like YouTube links, etc.) it would be nice if the page alerted me that I'd done it.

(I'd also love it if preview would alert me to malformed links—the kind that when you click on them in a post they just lead you back to the post page—which occasionally happen when previewing if a tag was left open or there were other html errors.)

*and besides, someone called me "the pony queen" at the last meetup, so I'm hesitant to reinforce that idea
posted by ocherdraco at 11:17 AM on October 12, 2009


Short of implementing a full-blown HTML validator, there isn't a good way to let you know when you have broken HTML. We're hoping that you check out links in preview before you post, and we're happy to clean up HTML problems that slip through on this end.

And I think double links within a post are rare enough that we don't need to check specifically for that. We can definitely remove a link here or there after posting if it happens.
posted by pb (staff) at 11:26 AM on October 12, 2009


Cool, I'll file that one away as unnecessary. I do check things in preview; I think my penchant for ~25 link posts is more to blame for my borky links than anything else.
posted by ocherdraco at 11:57 AM on October 12, 2009


Yeah. Edge case. That's what we keep boxcutters around for; no need for a hydraulic Link De-Duper 3000.
posted by cortex (staff) at 1:08 PM on October 12, 2009


a hydraulic Link De-Duper 3000 (cortexAdmin)

I am totally asking for one of those for Christmas, along with an Official Red Ryder Carbine-Action Two-Hundred-Shot Range Model Air Rifle.
posted by ocherdraco at 1:36 PM on October 12, 2009


You'll dupe your eye out.
posted by cortex (staff) at 4:07 PM on October 12, 2009 [1 favorite]


« Older So long, and thanks for all the fishpants.   |   OpenDNS not properly resolving ask.mefi? Newer »

This thread has been archived and is closed to new comments