search for URLs for FPPs

January 12, 2006 4:17 AM

In response to the inordinate number of doubles recently (e.g.) which begin:
MeFite A: Look! This is my link!
MeFite B: This is nice
MeFite A: sorry, double, delete

[mi]
posted by NinjaPirate to Feature Requests at 4:17 AM (28 comments total)

This is almost certainly because, since the dedicated search was removed for the health and wellbeing of the server, it's become a good deal harder to find out if your snazzy, gleaming link is actually yesterday's haircut. This is down to the generic search engine matches looking through both the initial links and the responses to them, meaning that "elephant spray deterrent" picks up both the first time your "new" link was posted, but also picks up the 250 times quonsar has visited a thread with a pissing pachyderm and someone objected. This makes pulling the answer you want out of the pile very laborious and clearly people aren't taking the (significant) time required to sift through until after posting their link.

Matt, could you develop a template not unlike the front page, but which only contains the text and links for the FPPs?
No links to flags, no links to users, no links to the ensuing discussion/inline images? Maybe one page per day of links, generating a new page once per day of the latest links?
Perhaps then you could put these pages under a new domain, then attach another Google/Yahoo! box to the search page which specifies "only search within this domain" and that domain, I might humbly suggest, would be NinjaTadpole.metafilter.com posts.metafilter.com (or similar).

Suddenly (or eventually) there would be a replacement dedicated FPP search again - not quite so good as the original, but it will certainly help to limit the deluge of results returned when scouting for similar posts and help prevent some of these doubles.
posted by NinjaPirate at 4:18 AM on January 12, 2006

Why do you hate quonsor?
posted by srboisvert at 4:52 AM on January 12, 2006

flippant response to srboisvert: He can't spell.

serious response to NinjaPirate: Nice idea.
posted by handee at 4:55 AM on January 12, 2006

I love quonsar and have the receipts to prove it.
posted by NinjaPirate at 5:02 AM on January 12, 2006

Better proposal: Quiz anyone trying to make an FPP about recent FPPs. Similar to what the free credit report people do to make sure that you're really you intra-session. They ask you questions like "From which of the following banks do you have a credit card", and since they (obviously) know the answer, they can present you with sensible answers.

1. What spacecraft will land in Utah on Sunday?

A. Voyager I
B. The Hubble
C. Stardust
D. Mathowie

2. Which muppet's head's aerodynamics have been discussed recently in the blue?

A. Animal
B. Statler/Waldorf
C. Mathowie
D. Bert

posted by Plutor at 5:16 AM on January 12, 2006

Yeah, I was actually just confused with the new posting system, and that sort of threw me off. I normally do a better job of searching for repeat posts. What I would really like is to be able to delete my own post if I realize something like that happens, but I could see how that could become problematic.
posted by sourbrew at 5:43 AM on January 12, 2006

Excellent idea, NinjaPirate. The Yahoo and Google searches are almost worthless for many things. This would help a lot.
posted by languagehat at 6:33 AM on January 12, 2006

The "preview" method (detailed in the "Don't double post" section of crunchland's How to Make a Better Front Page Post) works really well, though. The only problem with it is when the site you're looking to link to has moved to a new domain.
posted by Gator at 6:43 AM on January 12, 2006

What I've done is to call up the FPP form, and then just enter my main link in the URL field and hit "preview". If it's a double, the next screen shows what posts it appeared in and when.

It doesn't work for other links in the post, though, obviously.
posted by Miko at 6:43 AM on January 12, 2006

or, yeah, Gator.
posted by Miko at 6:44 AM on January 12, 2006

Plutor: Better proposal: Quiz anyone trying to make an FPP about recent FPPs. Similar to what the free credit report people do to make sure that you're really you intra-session. They ask you questions like "From which of the following banks do you have a credit card", and since they (obviously) know the answer, they can present you with sensible answers.

I guess I'll assume this was a joke and save myself the trouble of pointing out that work involved in maintaining such a quiz would be quite prohibitive, and that the quiz being restricted to "recent" fpps owuld by that token be useless for the stated purpose....

.... that said, those "personal information" questions are a real pain in the ass. Because of them, I can't change informaton on my credit report. See, the credit reporting company keeps askign me to verify who I am by verifying that I worked for a certain employer in 1999 for three months, in a city I've neither lived in nor visited at a place I'd never heard of before seeing my credit report.

I.e., someone tried to steal my identity, discovered it wasn't worth stealing (I had, literally, no credit at the time), and aborted the attempt -- but left garbage on my record in the process. Garbage which is now taken as true, and which I can't expunge because said garbage is part of the routine I'm requierd to go through in order to verify my identity....
posted by lodurr at 6:45 AM on January 12, 2006

... sorry, I just got lost in a furious reverie of reflection, there....

The link duplicaton check is a good thing, but it's not sufficient. Due to the contorted nature fo many URLs and the fact that many things can be multiply sourced, it's really really easy to do a double without duplicating a URL.

I don't think there really is a really strong solution to the double problem. There are just too many different ways to say and categorize things -- especially in a culture that puts a premium on clever and obscure FPP formulatons.
posted by lodurr at 6:49 AM on January 12, 2006

wow lodurr - thats sound like a major pain and like something that has to be fixed somehow.

/derail
posted by dabitch at 7:23 AM on January 12, 2006

lodurr: "I guess I'll assume this was a joke and save myself the trouble of pointing out that work involved in maintaining such a quiz would be quite prohibitive, and that the quiz being restricted to "recent" fpps owuld by that token be useless for the stated purpose."

Yeah. 'Twas. But I think the end result of such a hypothetical quiz would do more to prevent dupes than most other solutions: get people to pay attention to the front page. I think that too many people think of MeFi as a place to show off their NewsFilter or MemeWatch acumen, so they rush to post something cool without even searching or reading.
posted by Plutor at 7:27 AM on January 12, 2006

I just re-read and it sounds like I'm limply trying to call out sourbrew, which I wasn't.
Sorry, sourbrew, if you felt glared at.
posted by NinjaPirate at 7:33 AM on January 12, 2006

Paralepsis!
posted by cortex at 7:45 AM on January 12, 2006

What if, upon posting, MeFi's code parsed the entire post for URLs, and threw the URLs in a mysql (or whatever database you use) table like such:

PostID bigint unsigned,
url varchar(255)

This would mean it's possible to store multiple links per post, of course...

Then, you could pretty easily do a URL-specific search to see if something's been linked in a FPP before, and the database wouldn't get its ass handed to it.

Bonus points: WHERE url LIKE '$thisurl%', and other such smartness (i.e. removing well known meaningless URL parameters such as cid, sessionid, etc)
posted by twiggy at 8:21 AM on January 12, 2006

I'll just add a URL search in a few days, after a hardware upgrade that should take place in the next few hours.

I found this problem myself this week when I almost posted something that was a week old.
posted by mathowie (staff) at 8:33 AM on January 12, 2006

I like kuro5hin's method of stripping the links and link text out of an article. Perhaps we can have a page where it's just the links from each FPP?

Looking at the blue right now, it would look like this:

<h3>Jan 12</h3>
Clark and Dagger
For only $89.95 - http://americablog.blogspot.com/2006/01/americablog-just-bought-general-wesley.html
four-star general - http://en.wikipedia.org/wiki/Wesley_Clark
internets - http://www.locatecell.com/ [note this particular link is a dupe]

See You on the Other Side
Mark Spoon - http://www.discogs.com/artist/Mark+Spoon
Jam & Spoon - http://www.discogs.com/artist/Jam+%26+Spoon
found dead - http://www.undercover.com.au/news/2006/jan06/20060112_markspoon.html
"Stella" - http://mp3.juno.co.uk/MP3/SF49429-01-01-02.mp3
"Age of Love" - http://mp3.juno.co.uk/MP3/SF193378-01-02-01.mp3

Seeing the other side
Why does the Supreme Court Make Justices More Liberal? - http://bostonreview.net/BR31.1/hansonbenforado.html

I think actually printing out the URLs makes it harder to dupe. You can just do a search on that page (Ctrl-f or use Find-as-you-type) to see if it's a recent dupe. I think it'll also improve the quality of the link text; people will use more descriptive language in order to make the synopsis clearer. Lastly, if the link is directly to video, flash, or mp3 (as above), it's blatantly obvious, even to people like myself who browse with a PDA and therefore cannot hover over URLs. I'd recommend a link to the synopsis page from the "Post a Thread" page, prominently, so it's used.
posted by Eideteker at 11:31 AM on January 12, 2006

Eideteker: I, personally, think that's a great idea. But I'm betting that a lot of MeFites wouldn't agree with anything that makes MeFi look liek K5 or Plastic or some random PostNuke site.

Also, what you praise as clarity (and I agree), others would see as "boring" or uncreative. The oblique link text is a high-status style in certain circles; I heard some high-end design geek from Apple praise it to high heaven at a conference a few years back, tracing its first popular efflorescence to Suck.com. I have to say, when done well it can be powerful; it's often done poorly, though, in much the same manner as when people mis-use charts to illustrate a point, or when peopel just toss weighted-emphasis "tag clouds" out there as though they're good interaction design, instead of just something you do for cool effect. (For the record, I think that kind of visualization has value, but not as a piece of interaction deisgn. As interaction design, it's often worse than useless.)
posted by lodurr at 11:39 AM on January 12, 2006

Also, obligatory: If it's "about the links", Eideteker's proposal is clearly a winner.
posted by lodurr at 11:39 AM on January 12, 2006

lodurr: This is not for the Front Page; it's a separate index (posts.metafilter.com or metafilter.com/summary.mefi) that can be found either at the top/bottom of the page and on the "Post a Thread" page. I'm not suggesting we replace the current design (is that how you read me?).
posted by Eideteker at 11:56 AM on January 12, 2006

the 250 times quonsar has visited a thread with a pissing pachyderm

NinjaPirate is an ignorant douchebag. he cannot count. he cannot apprehend reality. would you discuss politics with such a man?
posted by quonsar at 12:27 PM on January 12, 2006

kidding, NP, just kidding.

certainly server-based search is not undoable. i had good results with swish-e on a large base of html. just gotta run the index build periodically to pick up new stuff. i don't know if it's available for windows, but if not there must be something similar.
posted by quonsar at 12:31 PM on January 12, 2006

♥
posted by NinjaPirate at 12:32 PM on January 12, 2006

swish-e is available for windows!

"Swish-e is a fast, flexible, and free open source system for indexing collections of Web pages or other files. Swish-e is ideally suited for collections of a million documents or smaller... Swish-e is also often used to supplement databases... for very fast full-text searching."
posted by quonsar at 12:37 PM on January 12, 2006

peps-e blue?i keed, i keed
posted by duende at 12:54 PM on January 12, 2006

I'd tell you to shut up, Kevin, but for once in your life, it looks like you're trying to be helpful.
posted by crunchland at 3:01 PM on January 12, 2006