First I created database with utf8mb4_general_ci
collation and created table with same collation. Then I import csv file with
load data local infile '/mnt/c/Users/justi/Desktop/enml/enml.csv'
into table dict
CHARACTER SET utf8mb4
fields terminated by '\t'
IGNORE 1 ROWS;
Sample data
+--------+----------------+----------------+---------------------------------+
| # id | english_word | part_of_speech | malayalam_definition |
+--------+----------------+----------------+---------------------------------+
| 174569 | .net | n | പുത്തന് കമ്പ്യൂട്ടര് സാങ്കേതികത ഭാഷ |
+--------+----------------+----------------+---------------------------------+
| 116102 | A bad patch | n | കുഴപ്പം പിടിച്ച സമയം |
+--------+----------------+----------------+---------------------------------+
| 219752 | a bag of bones | phr | വളരെയതികം മെലിഞ്ഞ വ്യക്തി അഥവാ മൃഗം |
+--------+----------------+----------------+---------------------------------+
I check with
SELECT malayalam_definition from dict;
then var_dump($row);
gives
array(1) { ["malayalam_definition"]=> string(19) "ശരശയ്യ " }
array(1) { ["malayalam_definition"]=> string(22) "പൂമെത്ത " }
array(1) { ["malayalam_definition"]=> string(41) "സുഖകരമായ അവസ്ഥ " }
array(1) { ["malayalam_definition"]=> string(44) "അസുഖകരമായ അവസ്ഥ " }
array(1) { ["malayalam_definition"]=> string(22) "പൂമെത്ത " }
array(1) { ["malayalam_definition"]=> string(123) "സുഖകരമെങ്കിലും സ്വാതന്ത്യ്രമില്ലാത്ത അവസ്ഥ " }
...
You can find an unknown character after each word like "ശരശയ്യ "
.
I tried select trim(malayalam_definition) from dict
but gives same result. how to find out that character after each words?
Converting the string to hex is one way:
SELECT HEX(malayalam_definition),CONCAT("{",malayalam_definition,"}")
FROM dict
WHERE id=116102