Preview breaks html character entities.

April 27, 2005 5:39 PM

HTML entities, like Π or Σ get converted in the input area to the actual displayed characters after a preview.

So when the comment is posted, the displayed characters -- not the entities -- are actually submitted, and the submitted displayed characters are then converted to question marks in the posted comment.

The result is that a preview that looked fine turns into a comment that doesn't.

Also: using the

 tag (say to preserve original whitespace) results in double-spaced lines, taking up too much vertical space. Using the  tag prevents this, but whitespace is not preserved. Adding   appears to be a (n ugly and hard to edit) work-around, but while leading non-breaking spaces within  tags are retained in previews, they are stripped in the posted comment.



As a number of askMefi's ask for program code, and as even some FPPs may contain code fragments, this makes the code unreadable or even syntactically incorrect in languages than foolishly assign meaning to whitespace.



And it's a pure pain to spend the time to carefully add the whitespace and to Google the correct HTML entities for a post, only to see them stripped out.



(Browser: Firefox 1.0.3)

posted by orthogonality to bugs at 5:39 PM (70 comments total)

And apparently < and > too.
posted by orthogonality at 5:40 PM on April 27, 2005

This has been mentioned precisely 327, 408 times by my reckoning. It worked on preview! is an ancient Mefi-injoke that I'm surprised you haven't seen. So you get a spanking for being repetetive, and breaking the front page.

*spanks orthogonality*
posted by stavrosthewonderchicken at 5:47 PM on April 27, 2005

Spankfest! Woo!
posted by rhapsodie at 5:49 PM on April 27, 2005

i think i have had two threads in meta on this :o)
posted by andrew cooke at 5:56 PM on April 27, 2005

repetitive ^ ;-P
posted by mischief at 5:56 PM on April 27, 2005

stavrosthewonderchicken writes "So you get a spanking for being repetetive (sic), and breaking the front page."

It's always the messenger who gets shot. But seriously, I think this is new (either matt made a code change, or Firefox 1.0.3 is different), because I've used entities before without seeing this problem.
posted by orthogonality at 5:57 PM on April 27, 2005

the line spacing has certainly always been broken special.
iirc, some entities used to work and others didn't - perhaps exactly which (none work now?) has changed. < is preserved by preview for me.
posted by andrew cooke at 6:03 PM on April 27, 2005

*shoots the messenger*

I seem to recall that for the non-standard characters and stuff it's a mostly ColdFusion issue, Matt may have said before, and the last update he did to ColdFusion quite a while back fixed some of the problems. I may have dreamed that, though.

<blockquote>, for example, has always been a bit weird though, which has just been a stylesheet thing, I think, and unrelated.
posted by stavrosthewonderchicken at 6:31 PM on April 27, 2005

*waits for the oral sex to begin*
posted by dg at 6:48 PM on April 27, 2005

I ? orthogonality

In in preview mode hovering over a user's name, I see their name not their number in the url.
posted by Feisty at 6:55 PM on April 27, 2005

.
posted by stavrosthewonderchicken at 7:09 PM on April 27, 2005

In in preview mode hovering over a user's name, I see their name not their number in the url.
Correct.
posted by dg at 7:27 PM on April 27, 2005

what does the '.' mean, stav?
posted by quonsar at 8:38 PM on April 27, 2005

You're entitled to bring up this old chestnut, ortho, but you broke the front page. Next time, use "more inside", eh?
posted by squirrel at 8:47 PM on April 27, 2005

I seem to recall that for the non-standard characters and stuff it's a mostly ColdFusion issue, Matt may have said before, and the last update he did to ColdFusion quite a while back fixed some of the problems. I may have dreamed that, though.

cries a little...

Dear Matt:

We know, we know, JDBC doesn't like utf-8. That's why you say this high-ascii characters get sent improperly. Fine. No one cares. That's not the problem we want you to fix.

The problem is when you regurgitate the string for the textarea. All you have to do is a simple search and replace. Every ampersand -- regardless of context -- should be replaced with &amp;. This is always correct. That's all! What a wonderfully small fix! It won't fix the high-ascii issue, but it will stop people from beating their heads everytime they preview.

Of course, I know I'm talking to myself here because other people have said this time and time again, yet preview is still broken.
posted by sbutler at 9:14 PM on April 27, 2005

C'mon guys. You piled on me when I f*cked up my MeTa post tags, so give ortho the same treatment. Fair is fair.
posted by AlexReynolds at 9:23 PM on April 27, 2005

*piles on ortho, dutifully*
posted by soyjoy at 9:30 PM on April 27, 2005

All you have to do is a simple search and replace. Every ampersand -- regardless of context -- should be replaced with &. This is always correct.

Nuh uh. What if you preview three times? then you'd get &amp;&amp;&amp;&amp;
posted by mathowie at 9:36 PM on April 27, 2005

By request: Ortho, you dr*ma qu**n.
posted by Bugbread at 11:39 PM on April 27, 2005

Let me modify what sbutler said. Every ampersand should get replaced by &, unless it's in the context of a string without spaces followed by a semicolon.
posted by SpecialK at 11:41 PM on April 27, 2005

goddamn it. There, it just bit me. Every ampersand should get replaced by &amp;, unless it's in the context of a string followed by a semicolon.
posted by SpecialK at 11:41 PM on April 27, 2005

.... *turns slightly red in the face and steam starts to boil out of his ears* ... let's put it this way: The way it works now is counter-intuitive, especially for ampersands, and should probably get fixed.
posted by SpecialK at 11:42 PM on April 27, 2005

Nuh uh. What if you preview three times? then you'd get &&&&

No, you wouldn't. Give it a shot. I dare you.

It's because when your browser sees the "&" it displays and transmits¹ it as "&". It doesn't send the original "&" sequence again. Just in case you don't believe me, I wrote up a sample: test.cgi.

1. well, after POST encoding, but I'm assuming that CF has something that decodes that string automatically for you.
posted by sbutler at 12:00 AM on April 28, 2005

Goddamnit... I was foiled by the encoding while trying to post about the encoding.
posted by sbutler at 12:02 AM on April 28, 2005

?
posted by grouse at 12:22 AM on April 28, 2005

I don't underst&.
posted by shmegegge at 12:27 AM on April 28, 2005

What'ç the p®oblem, & how would the p®oposed çolution fÏx Ït¿
posted by shmegegge at 12:31 AM on April 28, 2005

Ah, I see this is the pants-are-optional but spanking-is-mandatory room! Good thing I refused to wear pants today.
posted by loquacious at 12:41 AM on April 28, 2005

Hahahaha.

This is like watching the best of Laurel and Hardy. But better. keep going guys.

shmegegge, don't spoil it by looking competent, bro.
posted by NinjaPirate at 1:49 AM on April 28, 2005

What sbutler said. His test program works. mathowie, please listen to him! :-)

This would be very useful for entering non-western, or iso-latin or whatever, chars: ??, though I just now realized the situation has improved, since on preview they don't show up converted to entities anymore. So thanks for that!

The problem with the use-one-time-only entities remains, however, and sbutler has the simple fix.
posted by Turtle at 3:02 AM on April 28, 2005

> ??, though I just now realized the situation has improved

Gah! Nope, got bitten, again...

andrew, I think you're right: none work now. :-(

It used to be, to enter Chinese (or other) characters, you had to:
1. Convert them to HTML entities
2. Not post directly after preview
3. There is no other way

Now it's:
1. You can't.

IMHO, this is kind of a bad bug, not a pony. Wake up America! ;-)
posted by Turtle at 3:22 AM on April 28, 2005

I ? sbutler.

I also ? mathowie, however.

I don't know folks. My only recommendation would be the following:

un(til/less) this gets fixed, just before you hit preview:

1. copy the entire entry you just typed to the clipboard.
2. Hit preview.
3. Paste the clipboard over the new text entry box text.

If everything's fine:

4. hit post.

If not:

4. make corrections.
5. go back to 1.
posted by shmegegge at 3:56 AM on April 28, 2005

NEIN! EET EEZ NOT POZZIBULL!

EYE ♥ MATHOWIE!!!
posted by shmegegge at 3:57 AM on April 28, 2005

whew!
posted by shmegegge at 3:58 AM on April 28, 2005

that's incredible, I failed to take my own advice while posting said advice.

I'm going to go hold my head under the bathwater until I stop struggling, now.
posted by shmegegge at 3:59 AM on April 28, 2005

See how well you blend in now? Good work, fella.
posted by NinjaPirate at 4:28 AM on April 28, 2005

&#19968;&#30693;&#21322;&#35299;
posted by Turtle at 4:59 AM on April 28, 2005

Nope, shmgegge's technique doesn't work for me. Oh wait, maybe it works in hex?

一知半解 (produced by &#x4E00;&#x77E5;&#x534A;&#x89E3;)

Success!

So I'll add to shmgegge's excellent advice: use hex-based character entities, not decimal ones. I'm used to using Outlook Express's "Source" mode as a quick-and-dirty way of converting non-ANSI characters to entities, but it uses decimal, and MeFi only likes hex, so I guess I'll need to find another trick.

Isn't this fun?
posted by Turtle at 5:17 AM on April 28, 2005

Well, this is another fine mess you've gotten us into, NinjaPirate.
posted by shmegegge at 5:42 AM on April 28, 2005

I'm going to go hold my head under the bathwater until I stop struggling, now.

I'm going to hold my head under all the stuff in this thread that I can't understand until I stop struggling.
posted by OmieWise at 6:12 AM on April 28, 2005

sbutler's solution does indeed work, is easy to implement, and most important of all, works the way the user expects it to: it doesn't eat encoded characters.
posted by zsazsa at 6:50 AM on April 28, 2005

That's all well and good, but I think we should continue piling on orthogonality.
posted by languagehat at 6:54 AM on April 28, 2005

I have to think that this whole rigmarole of trying to communicate about the bug while the bug is operative on our communication is something Kurt Gödel would've loved.
posted by soyjoy at 6:58 AM on April 28, 2005

*jumps onto orthogonality-pile*

*wiggles a little, discovers blood on hands, jumps up*

uhh... guys? Has anybody else noticed that he hasn't said anything since thewonderchicken shot him?
posted by raedyn at 7:30 AM on April 28, 2005

Ah, I see this is the pants-are-optional but spanking-is-mandatory room! Good thing I refused to wear pants today.

Just don't go on an arson spree and leave your pants behind.
posted by kirkaracha at 7:40 AM on April 28, 2005

[this is good]
posted by NinjaPirate at 8:00 AM on April 28, 2005

?????

?????
posted by Mayor Curley at 9:22 AM on April 28, 2005

See, now I don't know if the Mayor was massively confused, or actually trying to share some special character message with us all. NOW WE'LL NEVER KNOW!

*pout*
posted by raedyn at 9:43 AM on April 28, 2005

Once you hit preview all you need to do is replace the character with the original "code" and hit post. It'll show up correctly.

I ♥ MetaFilter

Oh, and:


posted by deborah at 9:57 AM on April 28, 2005

[LOL]

*tears squirt from eyes*

[deep breaths]

I ? Metafilter
posted by schyler523 at 10:51 AM on April 28, 2005

It seems i'm unsure about Metafilter...but im not...

I'll have to go old school...

I heart Metafilter
posted by schyler523 at 10:52 AM on April 28, 2005

For the life of me, I can't find the character sequence that lets me write

I [DQ] MetaFilter
posted by anapestic at 11:07 AM on April 28, 2005

This thread deserves a puppy:


posted by AlexReynolds at 11:11 AM on April 28, 2005

languagehat writes "That's all well and good, but I think we should continue piling on orthogonality."

Dreamghost, in another thread, asked "Matt is there anyway you can ban orthogonality?"

You can use this thread to get signatures on that petition. ;)
posted by orthogonality at 11:51 AM on April 28, 2005

Nah, I have no desire to see you banned. I do wish you'd cut down on opaque snarkery and up your intelligibility quotient, however. And not break the front page.
posted by languagehat at 11:55 AM on April 28, 2005

Ditto with languagehat. I like some of the stuff you write. I like some of it a whole lot, even when I disagree with it. On other days, though, the snarkery and wind-ups can be annoying. Not that it's a call-out; I know my long-windedness annoys some people quite a bit. It's just a matter of druthers, I guess. I'd like you more if you skipped the quonsarness, but your job isn't to make me like you, so that's that.
posted by Bugbread at 12:00 PM on April 28, 2005

Dreamghost, in another thread, asked "Matt is there anyway you can ban orthogonality?"

I believe that what dreamghost was objecting to in that thread was the tyranny of the right angle, rather than you personally. This is also the case whenever someone calls you "obtuse."

I'd like you more if you skipped the quonsarness

I suppose it's a hopeless cause, but I would be eternally grateful if people could cease further coinage of words with the q-word as a root.
posted by anapestic at 12:10 PM on April 28, 2005

anapestic : " I suppose it's a hopeless cause, but I would be eternally grateful if people could cease further coinage of words with the q-word as a root."

Quonsarctic circle?
QuonSARS?
Quonsa(a)rdvark?
Quonsarpeggio?
or maybe even Quonsarrested Development?
sorry
posted by Bugbread at 12:18 PM on April 28, 2005

I'm going to have nightmares.
posted by anapestic at 12:29 PM on April 28, 2005

bugbread, cut it out. That's absolutely too much quonsarination.
posted by soyjoy at 12:52 PM on April 28, 2005

Quonsarnit.
posted by kenko at 12:59 PM on April 28, 2005

Quonsarnit!

or maybe even Quonsarrested Development?
Man, that's my favorite TV show!
posted by graventy at 1:28 PM on April 28, 2005

bugbread writes "I'd like you more if you skipped the quonsarness, but your job isn't to make me like you, so that's that."

Yeah. As I noted in by profile bio when I joined, I initially thought quonsar was a troll, but soon realized he has a trenchant ability to let the air out of humbug claims. Frankly, I'm not as good or as funny or as pithy at doing that as he is, but I do like to try to hone my ability at it, in hopes of achieving his level of wit.

On the other hand, writing intelligent, non-snarky posts is hard, and generally I have to be provoked into the effort.

You make a very good point about my job not being to make you like me. I hope that you do like me (and I like you), but in general, more Mefites should have your attitude, that we're here to have cogent discussions, not love-fests, and that we can respect even those we don't like.

"and wind-ups can be annoying"

What's a wind-up mean in this context?


languagehat writes "I do wish you'd.... not break the front page."

Damnit, I previewed about five times and everything looked fine. Even the entitles were showing as entities because I'd escaped the ampersands. It was only when I hit post that the breakage occurred. The one consolation is that it was that very sort of breakage the post was meant to call attention to, but it wasn't my intention to ironically break the page -- again, I tried to ensure that wouldn't happen by going through several previews.
posted by orthogonality at 1:33 PM on April 28, 2005

orthogonality : " What's a wind-up mean in this context?"

Saying something in an especially provocative manner in order to elicit a strong reaction. Not quite trolling (trolling is saying something you don't even believe just for the reaction), but something softer along similar lines.
posted by Bugbread at 2:04 PM on April 28, 2005

Frankly, I'm not as good or as funny or as pithy at doing that as he is, but I do like to try to hone my ability at it, in hopes of achieving his level of wit.

You should maybe set the bar a little higher.
posted by anapestic at 2:48 PM on April 28, 2005

can anyone wreck this place?
posted by fire&wings at 3:18 PM on April 28, 2005

On the other hand, writing intelligent, non-snarky posts is hard, and generally I have to be provoked into the effort.

*provokes orthogonality*

Seriously, when you make the effort, you're a valuable contributor here. That's why I get so impatient with your lazy quonsarkery. It should be noted that there is but one quonsar, and quonsar is his profit; attempts to imitate the one holey and episodic quonsar never end well.
posted by languagehat at 5:46 PM on April 28, 2005

languagehat writes "Seriously, when you make the effort, you're a valuable contributor here. That's why I get so impatient with your lazy quonsarkery."

I dunno. I guess to me, some things are obvious or close enough to obvious, and the snark is shorthand, alluding to what seems obvious. When it's apparent that what seemed obvious to me isn't, I'll put more effort in. But snark I don't have to check for correctness; a longer comment that isn't snarky means I have to fact check, googler for supporting links, revise, etc.

Here's an example, specifically picked to be non-political, from the "Scientific American to stop reporting science, more creationism" thread. Since to me the truth of evolution and the vacuity of creationism or "intelligent design" are both pretty much self-evident, my first comment was pure snark, snark that alluded to what I believed to be obvious:

Pretty_Generic writes "I'd like any creationist to explain something as simple as the human appendix."

If God didn't like to trick us, what would be the point of faith?

When it was apparent that mere allusion to the obvious wasn't going to do it, I troubled myself to come up with a more serious comment:

...it's not just that evolution is consistent with what we know, it's also consistent with what we don't know.

It has predictive and explanatory power: using evolution, we are able to say, "assuming evolution is correct, we ought to see this", and then when we do look, we see what evolution predicts....

I don't snark to troll or just to be clever: I pretty much always think there's some truth in what I'm saying -- even if sometimes the reader might have to dig a bit to find it. But, honestly, MetaFilter a recreation for me, not a job. When I write a serious post, I try to "connect the dots" and to be precise in my language. But sometimes I don't have the time or the inclination to be precise or to connect all the dots, and those times I just make a comment full of dots and rely on the reader to fill in the lacunae. Sometimes that means an opaque comment or one that can be read more than one way. And sometimes I mean to imply one thing, or I try for humor, and I miss the mark. Regardless, I don't have time to annotate my comments, nor is anyone interested in reading annotated comments when there are so many other good comments here to read.

And as bugbread pointed out, I'm also not here to get people to like me. It's great if they do, but I'm not trying, nor do I wish to try, to be the poster who is most appreciated by languagehat or even a poster who doesn't make languagehat "impatient with [my] lazy quonsarkery". languagehat has a much better chance of being, and opportunity to be, the poster most appreciated by languagehat. I'm here for my recreation; if you get something out of that, that's great!, but if you don't, I'm not going to spend time worrying that I don't quite measure up on your scale -- anymore than I spend time worrying whether or not languagehat measures up to my expectations.
posted by orthogonality at 8:45 PM on April 28, 2005

Fair enough, and I'm certainly not expecting you to try to be the poster most appreciated by languagehat, any more than I try to be the poster most appreciated by orthogonality. I was just expanding on my earlier comment; I didn't want you thinking I was in the Orthogonality Must Die! camp.
posted by languagehat at 7:50 AM on April 29, 2005

Orthogonality Must Die.

But only for his bad formatting, not the pseudo-quonsitrarian content.
posted by soyjoy at 8:40 AM on April 29, 2005