Comments on: Special characters

Special characters

mathowie — Tue, 08 Oct 2002 13:18:48 -0800

I "think" spéciàl chàràctérs shöuld wörk nöw, thöugh it's pröbably töö éàrly tö sày the böxés and quéstiön marks will be göné. That'll be £5 or 1000¥ or €3.5.

By: mathowie

mathowie — Tue, 08 Oct 2002 13:22:00 -0800

hmm. everything seems to be working.

Here's some hebrew:
?? ???? ?? ??????? ??

Here's a trademark symbol: ™

By: thewittyname

thewittyname — Tue, 08 Oct 2002 13:22:09 -0800

Ì ßènt ?? Woo??e.

Excellent.

By: mathowie

mathowie — Tue, 08 Oct 2002 13:22:24 -0800

dang, is hebrew beyond utf-8?

By: thewittyname

thewittyname — Tue, 08 Oct 2002 13:22:58 -0800

Uh....fill in the blanks

By: dayvin

dayvin — Tue, 08 Oct 2002 13:27:16 -0800

Let's see if my favorite — the emdash (the  version) — works. How about the correct version (the — one) — does it work?

By: dash_slot-

dash_slot- — Tue, 08 Oct 2002 13:29:51 -0800

Do you prefer ¢ or £?

[I prefer £, I have enough sense already!]

By: mathowie

mathowie — Tue, 08 Oct 2002 13:33:30 -0800

old metatalk threads seem to work now, as do metafilter threads that had boxes in them.

By: timeistight

timeistight — Tue, 08 Oct 2002 13:34:55 -0800

Me – I like the en-dash – either version.

Looks good. I hope the fix isn't what's slowing down the site.

By: moz

moz — Tue, 08 Oct 2002 13:38:31 -0800

thanks, matt.

By: yhbc

yhbc — Tue, 08 Oct 2002 13:39:15 -0800

Sniff. Well, very good work anyway, Matt!

By: rocketman

rocketman — Tue, 08 Oct 2002 13:45:47 -0800

Excellent. Thanks for the improvement, Matt.

By: matteo

matteo — Tue, 08 Oct 2002 13:47:42 -0800

é è ç ì ù

By: bradlands

bradlands — Tue, 08 Oct 2002 13:49:00 -0800

[I prefer £, I have enough sense already!]

/me £'s dash_slot

By: Firefly

Firefly — Tue, 08 Oct 2002 13:50:09 -0800

Let's see if it's fixed what happened last time I tried to use typographer's punctuation.

By: joemaller

joemaller — Tue, 08 Oct 2002 14:07:56 -0800

Another attempt at hebrew: (just letters, works in preview)
??????

? Œ‰†‡æ¡™£¢8§¶£¢8°·?‡‹Ô?¯¿

Preview is working, at least in Mozilla/Chimera... here goes...

By: joemaller

joemaller — Tue, 08 Oct 2002 14:08:31 -0800

and failure.

By: joemaller

joemaller — Tue, 08 Oct 2002 14:09:46 -0800

Well, I'm glad the boxes are gone anyway, was it something you did or a patch to the server software? Any geeky details you'd care to share?

By: MrBaliHai

MrBaliHai — Tue, 08 Oct 2002 14:25:26 -0800

¡Excelente, puedo ahora utilizar la primera marca del exclamation!

By: mathowie

mathowie — Tue, 08 Oct 2002 14:45:03 -0800

I had to change the database field types to "n" types (nvarchar instead of varchar, ntext instead of text, etc), as well as force utf-8 encoding, and I also had to install the latest service packs on the application server. It seems to be mostly working, the only problem I've found in testing is hebrew.

By: Plunge

Plunge — Tue, 08 Oct 2002 14:58:54 -0800

Let's try Korean! :-)

? ?? ?? ???.

Works in the preview, now the moment of truth...

By: Plunge

Plunge — Tue, 08 Oct 2002 14:59:08 -0800

grrrrr...

By: mr_crash_davis

mr_crash_davis — Tue, 08 Oct 2002 15:07:02 -0800

¿Que?

By: chrismear

chrismear — Tue, 08 Oct 2002 15:10:18 -0800

Here is some Hebrew as character entities:
א ב ג
And just as characters:
? ? ?

By: puffin

puffin — Tue, 08 Oct 2002 15:13:56 -0800

Hmm..what about japanese?

?????????

looks ok in preview...

By: puffin

puffin — Tue, 08 Oct 2002 15:14:55 -0800

Nevermind, then..heh :)

By: mathowie

mathowie — Tue, 08 Oct 2002 15:15:00 -0800

ok, so language issues still persist. Most problems are solved though. If you're seeing the characters in preview, the problem lies with the database drivers still.

By: eyeballkid

eyeballkid — Tue, 08 Oct 2002 15:44:45 -0800

The difference in the character entity versions and the unicode is that your browser interprets the entities, SQL server and Cold Fusion have to interpret the unicode versions. Hence, the ???? when you try to do it without the entities.

By: Plunge

Plunge — Tue, 08 Oct 2002 15:46:39 -0800

I could make some snarky comment on how this must be a plot by mathowie against the Hebrew speaking Jews and the Asians (with Korean & Japanese not posting, Chinese most likely doesn't either) to keep us down, but instead I'll say good job for even getting it this far. I'm sure you'll get it working with all the different languages and then boy, are we in trouble.

:-)

By: eyeballkid

eyeballkid — Tue, 08 Oct 2002 16:04:25 -0800

UNICODE data types in MS SQL Server only support characters from the code page assigned at the SQL Server install (though, 2000 supports the use of seperate collations for each database and seperate collations for queries). If I'm right, the default install code page is 1252 . That page only supports, I believe, the english alphabet and variations therof (Italian, Spanish, etc.) I believe that UNICODE set required for multiple languages is 850.

Still, I can't find a listing for the 850 code page anywhere in the MSDN or in SQL's BOL, so I'm not positive that it's the right page. I do know that 1252 won't handle anything except the variants I mentioned above (and, I believe, the Greek alphabet).

By: dg

dg — Tue, 08 Oct 2002 16:48:10 -0800

Fantastic - but now all the apologies for weird characters appearing in posts are going to look silly :-)

By: XQUZYPHYR

XQUZYPHYR — Tue, 08 Oct 2002 18:43:47 -0800

Hey, it has the invisible characters working too now.

By: puffin

puffin — Tue, 08 Oct 2002 19:04:29 -0800

mathowie.. I wouldn't worry too much about getting the language characters to show up.

As nifty as it would be to post Japanese in hiragana, most users probably wouldn't be able to see the characters anyway because their computers don't have the language support installed. And hey -- this is an english-language web site, after all.

By: bobo123

bobo123 — Tue, 08 Oct 2002 19:07:17 -0800

I'm guessing this might not work: 日本語? א? ⇔?

By: bobo123

bobo123 — Tue, 08 Oct 2002 19:16:46 -0800

日朮語?

By: insomnyuk

insomnyuk — Tue, 08 Oct 2002 19:28:06 -0800

The first person that tries to post in Japanese is going to get a severe wood-shed-ass-hauled-into-metatalking-to.

By: amberglow

amberglow — Tue, 08 Oct 2002 19:30:19 -0800

conichi-wa, insomnyuk (couldn't resist!)

By: hama7

hama7 — Tue, 08 Oct 2002 20:58:05 -0800

bobo: that second character ain't quite right

By: reverendX

reverendX — Tue, 08 Oct 2002 21:21:25 -0800

???

By: stavrosthewonderchicken

stavrosthewonderchicken — Tue, 08 Oct 2002 21:21:58 -0800

Korean test :

?????!
?? stavrosthewonderchicken ???.

By: bobo123

bobo123 — Tue, 08 Oct 2002 21:22:47 -0800

I know. I meant 日本語 but I typed 672E instead of 672C.

私はガラスを食べられます。それは私を傷つけません。

By: Plunge

Plunge — Tue, 08 Oct 2002 21:25:23 -0800

Hmmm... so Japanese does work? But Korean is still a no show. *sigh*

/me ponders a Japanese conspiracy...

By: Yelling At Nothing

Yelling At Nothing — Tue, 08 Oct 2002 23:19:36 -0800

This is so great—dashes work now!

By: dydecker

dydecker — Tue, 08 Oct 2002 23:20:58 -0800

????????? ????

By: dydecker

dydecker — Tue, 08 Oct 2002 23:23:27 -0800

Regular Japanese input doesn't work.

By: chrismear

chrismear — Tue, 08 Oct 2002 23:41:05 -0800

No conspiracy, Plunge. bobo123 is using character entities, which means that instead of typing a character straight into the comment box, you type in the code for the character, in this format:

本

which would give you 本.

The database has no problem storing the character entities, because they're just numbers and punctuation, but the current software seems to choke if you enter the actual symbols, and produces all those ?s.

So if you're really desperate to put the odd foreign character in, that's how to do it (although remember it won't display in everyone's browser). Also watch out that when you type in a character entity, and then hit preview, it converts the code into a regular, typed-in character. You'll have to turn it back into a code before you post.

By: dagny

dagny — Wed, 09 Oct 2002 01:16:48 -0800

Åwøsæme!

By: chill

chill — Wed, 09 Oct 2002 02:09:55 -0800

UNICODE data types in MS SQL Server only support characters from the code page.
That's not true actually. In SQL Server, Unicode fields are Unicode, pure and simple. The collation refers to the character set/sort order of non-unicode fields (e.g. varchar instead of nvarchar).
Matt, question marks (as opposed to blocks), and the fact that the characters work in preview, indicate to me that the text has not made in into the database correctly. Can you see Hebrew characters in there?
I know nothing of Cold Fusion, but in ASP, the first thing to check would be that the server codepage is set to UTF8, this basically tells ASP that when it is response.writing unicode data from the database, it should convert it to UTF8, and that any input it receives from form data should be considered UTF8 and converted to Unicode accordingly.
I know nothing of your internationalization skills, so I apologize if a certain phrase about sucking eggs comes to mind.

By: andrew cooke

andrew cooke — Wed, 09 Oct 2002 06:01:07 -0800

this is all just just a cunning plan to reduce the maximum comment size from 8000 to 4000 characters... ;o)

By: Plunge

Plunge — Wed, 09 Oct 2002 09:31:01 -0800

No conspiracy, Plunge. bobo123 is using character entities, which means that instead of typing a character straight into the comment box, you type in the code for the character...

Oh sure, way to go and ruin a good conspiracy for me.

By: chipr

chipr — Wed, 09 Oct 2002 10:35:18 -0800

Wondering if this addresses the problem I posted last month.

&

That should display as ampersand-a-m-p-semicolon

By: wanderingmind

wanderingmind — Wed, 09 Oct 2002 12:11:52 -0800

I ? Unicode.

By: wanderingmind

wanderingmind — Wed, 09 Oct 2002 12:14:53 -0800

Dang. That looked fine on preview (and I even typed in the character-entry thingie, not copy-and-pasted from the character map).

By: stavrosthewonderchicken

stavrosthewonderchicken — Wed, 09 Oct 2002 15:24:47 -0800

An oldie but a goodie - Metafilter : That looked fine on preview!

By: MiguelCardoso

MiguelCardoso — Wed, 09 Oct 2002 16:39:25 -0800

Let's see! Here's a Portuguese sentence that uses all our accents (and which miserably failed the last test):

Os cães só têm maçãs à terça-feira.

[Dogs only have apples on Tuesday]

By: MiguelCardoso

MiguelCardoso — Wed, 09 Oct 2002 16:39:43 -0800

Whee!