Archives of the TeradataForum

Message Posted: Tue, 02 Mar 2004 @ 22:19:15 GMT


	<Prev	Next>		<<First	<Prev	Next>	Last>>

Subj:		Re: Storing Multiple Languages in Teradata

From:		Victor Sokovin

Duane,

First of all, thanks a lot for posting the update on "Concurrent Loads". It is good to see you got them working but I am concerned about the row level locking problems. Will have to read again the whole thread to understand them better.

We are beginning to get requests to store data with not only English characters, but also Greek, German, Latin, and Russian.

I'm wondering what changes have to be made in Teradata at a database and/or object level. We're having to change our Informatica to a Unicode data movement mode and implement UTF-8 as the server code page to accommodate these requirements.

I was interested in some of these languages and posted a question about them in September or October last year. You might be able to find the thread in the archives. There have been more threads on Unicode lately.

I'll just comment here that you don't have to do anything to implement German or Latin (do you really mean Latin, BTW?!). They are just a part of the standard "Latin" character set.

Greek and Russian are not a part of "Latin", as implemented in Teradata. They can be easily implemented in Unicode character set, though. The downside is that you will have to allocate two bytes for each character as Unicode on the server side is UTF-16, not UTF-8. It is a pity as these languages are essentially based on 1-byte encodings (or even 7-bit), so some prefer to use customized character sets to save space. It can be done but it is messy and is only worth it if you can save a lot on disk storage. If you plan to convert everything to Unicode, then storage costs are probably of no concern to you. You should not encounter any particular problems with these languages. After all, it's only a few dozens characters, counting upper and lower case.

Any information would be appreciated - or even a pointer to a good reference for this type of Teradata Information.

I found "SQL Ref - Vol 1 Fundamentals" useful for general information on multiple language support. Doc ID = B035-1101-061A (applies to R4.1). More information is scattered all over the place in many other documents, like Utilities manuals, ODBC and JDBC driver user guides etc. They are essential for client side settings. It is important to monitor the whole data flow chain. One unwanted implicit code page conversion during ETL can spoil the data.

Hope you can start from here.

Regards,

Victor


	<Prev	Next>		<<First	<Prev	Next>	Last>>

Attachments

Library

Quick Reference