Tracking Forums, Newsgroups, Maling Lists
Home Scripts Tutorials Tracker Forums
 
  HOME    TRACKER    MYSQL




Storing English/Japanese/Russian Characters


what is the best collation to be used for a field that will hold characters from various languages. It could be a set of characters from any of the following languages and different rows could hold different languages:

* English
* Japanese
* Russian
* Spanish
* Portugese
* German




View Complete Forum Thread with Replies

See Related Forum Messages: Follow the Links Below to View Complete Thread
Save Both Japanese And English Characters
I will try to explain you what is my trouble:

I am not be able to write in my database using asian characters:
1) メッセージ
2) 제외하고
3) анить пароль
4) etc.. .. ..

I have a trouble using in my database asian characters.


When I copy and past them on my forum, I see that:
1) 分対
2) じませ
3) にフォー


I think that I have to set up my mysql in a different way.

I would like to know that:

How I have to do for saving both european characters and asian characters?
What version of mysql I have to use?
What are the character_set and the collation that I have to use ?

How I Can Save Both Japanese And English Characters, In My Text Box?
I will try to explain you what is my trouble:

I am not be able to write in my database using asian characters:
1) メッセージ
2) 제외하고
3) анить пароль
4) etc.. .. ..

I have a trouble using in my database asian characters.


When I copy and past them on my forum, I see that:
1) 分対
2) じませ
3) にフォー

I think that I have to set up my mysql in a different way. I would like to know that:

How I have to do for saving both european characters and asian characters? What version of mysql I have to use? What are the character_set and the collation that I have to use ?

Russian Characters
my.conf:
[client]
...
default-character-set = cp1251

[mysqld]
...
default-character-set = cp1251

still russian characters are displayed as ?????
MySQL 4.1.14-max-log

Storing Russian Text In Table
I am using MySQL 4.0.20a. A russian user enters text in a HTML-textfield and POSTs & inserts through a PHP-script (php v.5). In what encoding is it stored in mysql? To me, it doesn't look russian but like this:

Âûïîëíèòü çàäàíèå

Is this format OK to store in? If I want to use the text into another page (and select from mysql), do I have to store it in any encoding?

Non-English Characters
My company's enterprise directory is supposed to store all data fields
in English.Before the data standards were implmented, many of the
HR systems that fed data into the ED were sending some fields with a
combination of English and non-English characters. I want to identify
records with erroneous characters so we can ask the HR systems to
re-send the correct English-only values.
So...
I've exported a list of all users in the ED into a MySQL table called
'users'.
I would like to write a query that will return all records where the
'givenname' field contains one or more characters outside of [a-z] and
[A-Z]. I know I have to use REGEXP, but I can't figure out how to
find any record where one or more non-English characters exists in the
'givenname' field.

Mysql Client Dont Write Russian Characters
I installed MySQL 5.0.24 and have so problem.I can't type russian characters in mysql client. Database has data in koi8 encoding (from dump, from php form...) and this data correctly shown.But when I switch encoding to russian in mysql client cursor stops moving independent from my keys pressing,when switch it return to English and typing continued.For example,I enter: select fld from tbl where fld="russ",if "russ" in koi8 I cann't type this characters.
It is possible type russian characters from OS command line: mysql -p -e "select fld from tbl where fld='russ'" dbname. In this case whole query text will be entered and correctly executed.
If anybody know how possibly decide this problem write me please.
My OS is ALTLINUX 3.0 : LANG=ru_RU.KOI8-R
MySQL's variables:
| character_set_client | koi8r
| character_set_connection | koi8r
| character_set_database | koi8r
| character_set_filesystem | binary
| character_set_results | koi8r
| character_set_server | koi8r
| character_set_system | utf8
| character_sets_dir | /usr/local/mysql-standard-5.0.24-linux-i686/share/mysql/charsets/ |
| collation_connection | koi8r_general_ci
| collation_database | koi8r_general_ci
| collation_server | koi8r_general_ci

Japanese Characters And Jsp
I am building a site for a client in both japanese and english. I was wondering if anybody could give me some basic pointers or can direct me to a tutorial online. I am using tomcat 5.0 and MySQL 4.02.

The research that I can find has let me to believe that I need 4.1 to support full unicode or there are hacks that will do it in lower versions such as defining varchar column types as binary. I've done this and still have problems viewing or returning data correctly. I wish that i could upgrade to 4.1 to make my life simpler, but my hosting company will not do the upgrade for some time.

I am wondering if there are things that i need to do to my java query? Is there a way around this where i can use the same query and dataconnector class that i have now to return japanese and english?

Storing Japanese Text In MySQL
I am developing a site for a Japanese jewerly business in PHP and I'm having a problem with getting the japanese text to store correctly in MySQL. I've been working on this problem for three days now and after an exhaustive internet search and countless coding/database collation changes, I'm still stumped. I've seen a number of forums suggesting the use of SET NAMES 'utf8' or SET CHARACTER SET 'utf8',

I've tried these and more with no success. Does anyone have any suggestions? One other thing I noticed - when I type japanese text directly into the database table field, the text gets converted to these: 水 - which looks odd but displays as the original japanese text when querried from the database and displayed in a web page. Is there a way I could convert my text to this format before inserting it?

MySQL Japanese Characters
I use a MySQL Database where I store Japanese characters in SJIS. I tested if they are in SJIS and they are.

I am using mysql 4.1.3b-beta, mysql-connector-java-3.0.14, JBuilderX (jdk 1.4.2), and OS XP.

I change the default encoding latin1 to sjis when I start MySQL. I use
mysqld-opt --default-character-set=sjis
mysql --default-character-set=sjis.

After starting mysql with the sjis character set I used:
create database japanese;
use japanese;
create table jp (jp1 varchar(100));

I use a file with sjis characters as input. And to see if the output is in sjis I write the values to an output file. The values I get are in sjis.

(I can't set the character set of the table to sjis. When I do that I only get garbage into the database and in the output file, maybe that's becuase I already changed the default character set at start-up?).

So I have no problem with inputting a file that contains sjis characters and I can also output data to a file in sjis when using only MySQL, but I can't do it with Java.

When I try to retrieve the data I only get a square, then a some character, then a square, then another character,... but not the actual kanji/kana. I have some latin characters in the same column as the kanji/kana and they are displayed correctly. I already tried only using kanji/kana in a column but it didn't work either.

I can display Japanese characters correctly in Java when I retrieve them from a file, so I don't think it's a problem with the font. I also tried outputting the data I retrieve through Java to a file but there I get the same symbols as in my JTable.

Have I done something wrong with creating the Database?
Is there something I need to consider when using the MySQL Connector?

On a side note as I am new to MySql after installing version mysql4.1.3b. I no longer have the mysqld.exe but only mysqld-opt.exe. So I am using this instead. But I don't think that can be responsible for my problem.

Entering Japanese Characters
Edit: Inserting it into the database manually works, so its entirely possible this is just a problem with php reading it and inserting it wrong. I'm just going to post this in the php section,

REGEXP Case Sensitivity With Non-English UTF8 Characters
I'm trying to use RLIKE or REGEXP with a utf_general_ci database/table but it always returns case sensitive results when using non-English characters. (MySQL: 5.0.21)
BTW database/tables are using UTF-8 charset and utf8_general_ci collate. Also, LIKE function works fine but all regex functions work case sensitivelly.

Japanese Characters In MySQL 4.0 - Character Sets?
How do I insert Japanese and other foreign characters into a MySQL 4.0 table? (4.0.27 to be specific) In other words, how do I specify a UTF-8 set for a database or table or column. MySQL 4.0 doesn't seem to support character set definitions the same way that 4.1 does, and I'm having trouble locating how to do it in 4.0.

Right now if I insert Japanese characters they show up as character codes like '& #25658; & #24111;' (without the spaces) in the table. While this displays properly if used to make content on a page, I'm not sure if a search engine will pick it up right.

Importing Japanese Characters From Text File
I have a tab-delimited text file I'm trying to import with Navicat Import Wizard. The first field of each line is in Japanese (SJIS format). Even though I've tried setting up the fields with Design Table for the field of the new database to be coded for SJIS (and every permutation I can think of), I only get question marks in the generated table.

MySQL LOad Data Infile Problem With Japanese Characters In Linux
I am trying to import a data file wich has japanese characters (sjis encoding). I am having problem loading the datafile into database using LOAD command in linux. The same command works on Windows though (Reagion set to Japan, Language Japanese without using any of the settings.

I have created the database using character set set to sjis. Table type is InnoDB and char set is sjis.

My MySQL server version is 5.0.16 on RedHat EL 3.0 AS (English and Japanese language support). I have set the following sys variables;

Unicode For Russian
I just wanted to insert Russian data in my database but while I insert it it get inserted easily, since I believe the data is inserted in the default format i.e. UTF-8 Unicode (utf8) i believe this is the default format but when i select it from database i could not get what was desired what is the solution for this, I am using Php as server side language.I get? as a result of select and while i look it in phpadmin the result is the same, is this the browser problem...which is not supporting the font.

Saving Text In Russian
I have a table with a field that is to save text in Russian. The text is
then retrieved by a PHP page. I run a query passing text in windows-1251
encoding, but when I retrieve it from the db, it shows up all wrong.
Does anyone know what the problem is?

Version 3.23 - Russian Character Set
Have issues inserting Russian charaters in a table. The array used for the insert appears to be fine. The supported character sets per Varibales are:

latin1 big5 czech euc_kr gb2312 gbk sjis tis620 ujis dec8 dos german1 hp8 koi8_ru latin2 swe7 usa7 cp1251 danish hebrew win1251 estonia hungarian koi8_ukr win1251ukr greek win1250 croat cp1257 latin5.

Russian Language (win-1251) At Export MySQL 4.1
I have a problem with mysql back-up of my site. At site is present the material in russian language (the coding WIN-1251). Presently, hosting company has done upgraid MySQL bases to version 4.1.

The Site works orderly, but when making back-up bases (Export), in the base are absent the symbols in russian language. At export of the base through PHPMYADMIN , the situation a bit better, some cyrilic symbols are seen, but not all. How it is correct to do the MySQL export to were seen cyrilic symbols?

Non-english Text
I'm working on application which stores web page content. Generally I'm
turning the whole page into base64 for ease of storage (into a TEXT field).

But I have another field which opens a socket to the page, sucks down
the HTML source, runs strip_tags and other PHP cleansing functions on
it, and inserts the remaining words into a mySQL TEXT column which is
straight text (not turned to base64).

I encounter a problem with foreign languages when I do a mysqldump.
Some of the characters are non-standard ASCII and I can't merely "cat"
the file back in to a mySQL database.

Creating A DB For Non-english Language
how can I create a database for a non-english language? Suppose the language is Hindi(Indian) or Chinese.

Database In Other Languages Except English
I am working on a cross language information retrieval system for hindi , tamil and english. For this i need to create a dtabase in all the above languages . How do i do that with MySql?

English Dictionary For Mysql
I'm trying to find an (english) english dictionary that I can install into mysql on an OS X server - does anyone know where I download a free one from?

Store Both English And Korean
I'd like to be able to store both English and Korean in my table.
Currently, it only supports English. I understand that I can use the
command:

Alter Table my_table [DEFAULT CHARACTER SET euckr] [euckr_korean_ci]

to alter my table to have Korean as my default character set. But
will I be able to store English? What if I want default to be English
and still be able to support other (more than 3) languages, how would
I alter my table?

Table Of English Words
does anyone know where i can find a mysql table definition or similar
that contains thousands of unique english words between 3-7 characters long?
don't have to be dictionary words, just as long as they are recognisable english words, nouns, adj, verbs, proper nouns, etc

English And Arabic Text Support?
How can I make a MySQL table accept both English and Arabic text?

So far, I only know how to create a normal MySQL table with the following query:

CREATE TABLE `items` (
`id` INT(7) UNSIGNED NOT NULL AUTO_INCREMENT PRIMARY KEY ,
`title` VARCHAR( 30 ) NOT NULL ,
`description` TEXT NOT NULL ,
`updated` INT( 10 ) UNSIGNED NOT NULL
);

What needs to be added to that to make it accept both English and Arabic inputs (for both title and description cols)?

DATE_FORMAT Result In Other Language Than English
I'm using DATE_FORMAT() function in SQL query with php but I can't dislay date in other language than english (for month and year), ie: Thu 1 Jun 2006.

I didn't find where to specify language.

during the call of query or connection ?

Mysql 4.1.14 With Japanese
I am using mysql on Gentoo Linux. Until yesterday I could use Japanese messages with mysql with phpBB without any problem, but since I upgraded mysql to 4.1.14, I cannot handle Japanese characters properly (maybe 4.0.xx or something..).

For example, if I try to enter Japanese strings into phpBB, it says:
-------------------
Could not insert new word matches

DEBUG MODE

INSERT INTO phpbb_search_wordmatch (post_id, word_id, title_match) SELECT 3, word_id, 0 FROM phpbb_search_wordlist WHERE word_text IN ('テスト')

Line : 251
File : functions_search.php
-------------------

I think this is due to lack of Japanese support on my server. How can I do it? I want to use ujis and preferably utf-8 also...

Japanese In MySQL
When I try to store Japanese characters in my mysql database, I do not
get the correct characters. The entry of the field is "???" instead of
the Japanese character. I tried to store the character with my java
application and with the MySQL Control Center. I know I have to
configure the mysql database. But I do not know how. I created my
database with the following sql statements:

CREATE TABLE vocab
(
Japanese VARCHAR(100) CHARACTER SET utf8,
German VARCHAR(100) CHARACTER SET utf8
);

But this did not work.

Japanese Character
In my mysql database i have stored accounts name in japanese character. All accounts has been displayed in a page sort by name. So i use "ORDER BY name" in my query. But my client complain that the names are not sorted correctly.
Is there anything i have to do in case of japanese or non-english character sorting?

Japanese Char
I am not able to store Japanese chars in mysql database. what should I do to work this out ?

How To Store Chinese Chars And Also Italian, French And English In Mysql ?
I'm not able to store chinese characters into a column in a table....

how should the db setted and the table created to be able to store
different chars like chinese, french, italian and english ?

Importing Japanese Characterset
I`m trying to get a mixed excel spreadsheet (English and Japanese) into mySQL. I thought phpMyAdmin could do it, but it can only export into an excel file.

The issue I have is, there is something weird about the csv file format in that it makes all my Japanese text become garbage. So I`m thinking I need a way to import directly from the xls format.

Does anyone have any experience inputting asian languages/charactersets into mySQL? I would prefer to use OSS or free software (it is a project for education = limited funds).

Can Input Japanese, Can't Pull It Out
I figured out how to store Japanese characters by setting the collation of a particular field to utf8_bin. I can input Japanese characters, and I can see them just fine under PHPMyAdmin.

The problem is that when I execute a query and print them to a web page, all I get is question marks instead of Japanese characters. This is while using the same browser that allows PHPmyAdmin to display the characters correctly.

I added the following metadata to the <head> portion of the document but it didn't help:

<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />

Sorting Chinese, Japanese
I have been asked to look into support for sorting non-latin 'alphabets' such as Chinese and Japanese. I already know that MySQL supports UTF-8, though my interest is in finding out how well MySQL support sorting of entries for these writing systems and what supplementary configuration I may need to do.

Unicode And Japanese Support
I am using cold fusion to make a small custom website using
mysql to store the information a user inputs. I want it to
be able to save japanese and unicode text, but whenever I
enter any it comes back as garbage. Is there anything
special I need to do to configure mysql to use unicode

Japanese Text Dump To Table
I'm trying to upload the content of a TXT file in Japanese (JSHIFT) into a table with the command LOAD DATA INFILE, but everytime I've tried, I receive a warning that the text is being cropped and then it doesn't display properly on my webpage (although it's encoding is properly set with :

<meta content="text/html; charset=Shift_JIS" http-equiv="content-type"/>).

I'm tired of trying, so I decided to ask in case anyone has experience with text in other encoding formats.

Japanese Table/Field Names In MySQL
I have come up against a problem when converting a Japanese database from MsAccess to MySQL.

Using the export function of MsAccess to transfer the data to an MySQL database, I find that when I view the MySQL table from the MySQL Control Center the Japanese double byte characters have been converted to gibberish. I have tried changing the display properties to display using Japanses fonts but the problem remains.

Is it possible to export tables/fields names which contain Double Byte Characters into MySQL and if so what do I need to do ?

Omitting Characters Between Characters?
This is what I wrote:

select f.faqdesk_id
, c.categories_id
, f.faqdesk_question
, c.categories_name
, f.faqdesk_answer_short
from faqdesk_description as f
join faqdesk_to_categories as f2c
on f.faqdesk_id = f2c.faqdesk_id
join faqdesk_categories_description as c
on f2c.categories_id = c.categories_id

It works just fine, but the column: f.faqdesk_answer_short has characters I want to omit.
For example, the results have html formatting.

<H1>Hello World</H1><br><b>How is your day?</b>

I want to perform the search without viewing the html. Is there a way to omit the brackets & anything between them?

Storing Pdb
I am in the process of creating a database that will store amongst otherthings, will need to store .pdb files and .vrml files. I am a bit confused as to go about it, as these files have to made accessible to the user- via a perl-cgi webpage. You get the idea- you search for something and you click on "get pdb file", and the user gets it. My question is this: Can i store files just in there .pdb form, or can a field only store the text that is in the pdb file. Is there a way to just put the file onto the web browser or will it have to be printed out via the cgi script onto the web browser line by line?

Storing 30.00
to store this in mysql, is this proper:

Code:

product_price float(4,2) unsigned NULL

Storing Images In The DB
I have a database that is a catalog of movies, books, etc and I would like to have a page that actually shows an image of the item along with the title, that's not the problem, the problem is I do not know how to go about storing images in the database and assigning them to catalog entries.

Storing User CSS
I'm working on a social networking site with a friend and am curious about something: how much bandwidth do mysql queries typically consume? I'd like to make the CSS of user profiles open for editting, but that would mean a lot of css settings would have to be stored and constantly queried. Does this take up a lot of bandwidth and are there any ways to open up the CSS without consuming too much?

Storing Keywords
I did a search on storing keywords and found a post where someone suggested storing keywords in a 1-to-many table scenario. I'm not sure I follow. Here's what i'm thinking of doing. I'll be storing a Word document that would be tied to various keywords. I want to be able to search by keywords and get back a query of various documents that match up to the keywords.
I figured i'd make a page to list the document and then input keywords and store them in the database for later searching.
What would be the best way to store these keywords for each document?
hope that wasn't to confusing. Thanks for the help.

Storing Numbers
How do I store a number in this format... 2.00


Storing Arrays
What is the best datatype/method of storing an array of data retrieved from PHP. And/or do I need to convert the array to a string before inserting?

Storing Time
I need to store time in several tables and im trying to find out the best way to do it. I need to store the time in this format HH:MM (or HH:MM:SS). Im thinking of using the TIME field in MySQL 4.1.

I need to show the time with AM/PM, so i guess i should store it in a 24hr format and use DATE_FORMAT to show it with AM/PM...right?

And i will need to show them chronologically...how would i do that?

Storing Day Of Week
I need to store the day of the week an item will occur on every week. I will then be able to sort by that column by the day. I was thinking about storing it in a datetime field while putting all items in the same week but changing the date so that it coincides with the proper day of the week. I'm not sure how to do this in either the mysql query or in php.

Storing And Using Images
I am planning an extension to my website that will be based on a template.
For the sake of argument let's suppose that the page caters for 5,000 clubs. While the general layout and options can all be based on the template, I want to put each clubs logo on their own page. I realise that there are two methods, either store images in a database or in a file and call them from a link in a database. Assuming I am uploading all the images, which would be the simplest solution. I use dreamweaver 8, mysql and php.
I can see that if they were in a database, putting them onto the page would be easy but uploading them would be difficult.
If I used a file based system, uploading would be easy but I don't know how to get them onto the page.

Storing Checkbox Value
How do I store the result of a checkbox (T/F) in the database? What type is best? I've tried enum & tinyint, but the error was 'Column count doesn't match value count at row 1'.


Copyright 2005-08 www.BigResource.com, All rights reserved