How can I save Unicode characters in a .txt file?

Stephen Hughes
Posts: 3323
Joined: February 26th, 2013, 7:12 am

How can I save Unicode characters in a .txt file?

Post by Stephen Hughes » April 9th, 2015, 10:24 am

The .txt file editor changes this:
Polybius 5.1.1
τὸ μὲν οὖν κατὰ τὴν Ἀράτου τοῦ νεωτέρου στρατηγίαν ἔτος ἐτύγχανε διεληλυθὸς περὶ τὴν τῆς Πλειάδος ἐπιτολήν: οὕτως γὰρ ἦγε τοὺς χρόνους τότε τὸ τῶν Ἀχαιῶν ἔθνος.

διέρχεσθαι pass, elapse
ἄγειν reckon
to
Polybius 5.1.1
¦Ó? ¦Ì?¦Í ¦Ï?¦Í ¦Ê¦Á¦Ó? ¦Ó?¦Í ?¦Ñ?¦Ó¦Ï¦Ô ¦Ó¦Ï? ¦Í¦Å¦Ø¦Ó?¦Ñ¦Ï¦Ô ¦Ò¦Ó¦Ñ¦Á¦Ó¦Ç¦Ã?¦Á¦Í ?¦Ó¦Ï? ?¦Ó?¦Ã¦Ö¦Á¦Í¦Å ¦Ä¦É¦Å¦Ë¦Ç¦Ë¦Ô¦È?? ¦Ð¦Å¦Ñ? ¦Ó?¦Í ¦Ó?? ¦°¦Ë¦Å¦É?¦Ä¦Ï? ?¦Ð¦É¦Ó¦Ï¦Ë?¦Í: ¦Ï?¦Ó¦Ø? ¦Ã?¦Ñ ?¦Ã¦Å ¦Ó¦Ï?? ¦Ö¦Ñ?¦Í¦Ï¦Ô? ¦Ó?¦Ó¦Å ¦Ó? ¦Ó?¦Í ?¦Ö¦Á¦É?¦Í ?¦È¦Í¦Ï?.
when it is saved, uploaded, downloaded and displayed.

Is there something simple that I am not doing?
0 x


Γελᾷ δ' ὁ μωρός, κἄν τι μὴ γέλοιον ᾖ
(Menander, Γνῶμαι μονόστιχοι 108)

Eeli Kaikkonen
Posts: 411
Joined: June 2nd, 2011, 7:49 am
Location: Finland
Contact:

Re: How can I save Unicode characters in a .txt file?

Post by Eeli Kaikkonen » April 9th, 2015, 11:28 am

Most modern text editors have a way to choose the text encoding. The default may be some other than unicode, for example ISO-8859-something. You have to change it to unicode (usually UTF-8). UTF-8 and UTF-16 are actual low level encodings which implement the unicode standard, which itself is not an encoding.
0 x

Thomas Dolhanty
Posts: 400
Joined: May 20th, 2014, 10:13 am
Location: west coast of Canada

Re: How can I save Unicode characters in a .txt file?

Post by Thomas Dolhanty » April 9th, 2015, 11:34 pm

Stephen Hughes wrote:when it is saved, uploaded, downloaded and displayed.

Is there something simple that I am not doing?
Uploaded to ... downloaded from ... and displayed ... by what? That is, does it look right in your editor before you upload it? It is a bit nebulous, the way you describe it, where the text is getting jumbled.
0 x
γράφω μαθεῖν

Stephen Hughes
Posts: 3323
Joined: February 26th, 2013, 7:12 am

Re: How can I save Unicode characters in a .txt file?

Post by Stephen Hughes » April 10th, 2015, 12:34 am

Thomas Dolhanty wrote:
Stephen Hughes wrote:when it is saved, uploaded, downloaded and displayed.

Is there something simple that I am not doing?
Uploaded to ... downloaded from ... and displayed ... by what? That is, does it look right in your editor before you upload it? It is a bit nebulous, the way you describe it, where the text is getting jumbled.
In the folder, accompanying the sound files.

I'm not sure at which of the steps it turns to gibberish.
0 x
Γελᾷ δ' ὁ μωρός, κἄν τι μὴ γέλοιον ᾖ
(Menander, Γνῶμαι μονόστιχοι 108)

Louis L Sorenson
Posts: 706
Joined: May 5th, 2011, 9:21 pm
Location: Burnsville, MN, USA
Contact:

Re: How can I save Unicode characters in a .txt file?

Post by Louis L Sorenson » April 10th, 2015, 12:47 am

When you choose to save a file, say in an application like Windows Notepad. You will see three different fields:

(1) File Name
(2) Save as Type: txt (text); all files
(3) Encoding: ANSI; Unicode; Unicode Big Endian; UTF-8 (I'm using a MS Windows box)

When you save a file, you will need to set the encoding to UTF-8,. The reason for this is that texts which contain characters which are not Latin characters cannot be represented side by side with Latin characters if only 255 bits are allotted for any given character. (Google ANSI to understand this further). Unicode is a computer memory allocation that allows thousands of characters to be represented on screen. Older non-Unicode fonts used the ANSI (255 bits) set of characters to represent Greek/Hebrew and other character sets on a computer screen. Some of those fonts may have names such as GreekKeys. There are utilities which have been created to convert non-unicode fonts (which were frequently used by Biblical scholars in the 1990's to digitize Greek or Hebrew texts) into Unicode compliant fonts. Since the early 2000's, Unicode fonts have taken over, and almost no one now uses non-unicode fonts for Greek or Hebrew.

But a person who types Unicode text into a notepad file, textfile, or other text (html file), still has to specify if the font they have typed, they need to specify whether the file has to be saved into a 255 bit format or a unicode format (?255x255). If you save a unicode format into an ANSI format, you will get the following:

UNICODE TEXT:
Polybius 5.1.1.1
Τὸ μὲν οὖν κατὰ τὴν Ἀράτου τοῦ νεωτέρου στρατηγίαν ἔτος ἐτύγχανε διεληλυθὸς περὶ τὴν τῆς Πλειάδος ἐπιτολήν· οὕτως γὰρ ἦγε τοὺς χρόνους τότε (τὸ) τῶν Ἀχαιῶν ἔθνος.

If you do not save the file in UTF-8, but choose ANSI, you may get something like
Polybius 5.1.1
¦Ó? ¦Ì?¦Í ¦Ï?¦Í ¦Ê¦Á¦Ó? ¦Ó?¦Í ?¦Ñ?¦Ó¦Ï¦Ô ¦Ó¦Ï? ¦Í¦Å¦Ø¦Ó?¦Ñ¦Ï¦Ô ¦Ò¦Ó¦Ñ¦Á¦Ó¦Ç¦Ã?¦Á¦Í ?¦Ó¦Ï? ?¦Ó?¦Ã¦Ö¦Á¦Í¦Å ¦Ä¦É¦Å¦Ë¦Ç¦Ë¦Ô¦È?? ¦Ð¦Å¦Ñ? ¦Ó?¦Í ¦Ó?? ¦°¦Ë¦Å¦É?¦Ä¦Ï? ?¦Ð¦É¦Ó¦Ï¦Ë?¦Í: ¦Ï?¦Ó¦Ø? ¦Ã?¦Ñ ?¦Ã¦Å ¦Ó¦Ï?? ¦Ö¦Ñ?¦Í¦Ï¦Ô? ¦Ó?¦Ó¦Å ¦Ó? ¦Ó?¦Í ?¦Ö¦Á¦É?¦Í ?¦È¦Í¦Ï?.

So the bottom line: Specify the encoding when you save a text file.
0 x

George F Somsel
Posts: 172
Joined: May 9th, 2011, 10:11 am

Re: How can I save Unicode characters in a .txt file?

Post by George F Somsel » April 10th, 2015, 1:37 am

If you really do mean a text file, it is not possible to save Unicode in a text file since it allows only 128 differing characters (ASCII). You will need to choose some other format than a text file.
0 x
george
gfsomsel



… search for truth, hear truth,
learn truth, love truth, speak the truth, hold the truth,
defend the truth till death.



- Jan Hus

Jonathan Robie
Posts: 3489
Joined: May 5th, 2011, 5:34 pm
Location: Durham, NC
Contact:

Re: How can I save Unicode characters in a .txt file?

Post by Jonathan Robie » April 10th, 2015, 3:03 am

Stephen Hughes wrote:I'm not sure at which of the steps it turns to gibberish.
It happens when you save the file. It can apparently display Unicode fine.

What operating system are you on? When you run the text editor, can you choose "help" from the menu and tell us what the name of the text editor program is?
0 x
ἐξίσταντο δὲ πάντες καὶ διηποροῦντο, ἄλλος πρὸς ἄλλον λέγοντες, τί θέλει τοῦτο εἶναι;
http://jonathanrobie.biblicalhumanities.org/

Jonathan Robie
Posts: 3489
Joined: May 5th, 2011, 5:34 pm
Location: Durham, NC
Contact:

Re: How can I save Unicode characters in a .txt file?

Post by Jonathan Robie » April 10th, 2015, 3:12 am

George F Somsel wrote:If you really do mean a text file, it is not possible to save Unicode in a text file since it allows only 128 differing characters (ASCII). You will need to choose some other format than a text file.
A text file can contain Unicode in UTF8, but not all software supports this. I put Unicode in text files all the time. You do want an editor that knows what a byte order mark is ... Stephen, does your editor work correctly for Chinese?

But if you want to make sure everyone can read it, you're better off with a format that declares an encoding, such as HTML or XML. Proprietary word processor formats aren't so good for this.
0 x
ἐξίσταντο δὲ πάντες καὶ διηποροῦντο, ἄλλος πρὸς ἄλλον λέγοντες, τί θέλει τοῦτο εἶναι;
http://jonathanrobie.biblicalhumanities.org/

Thomas Dolhanty
Posts: 400
Joined: May 20th, 2014, 10:13 am
Location: west coast of Canada

Re: How can I save Unicode characters in a .txt file?

Post by Thomas Dolhanty » April 10th, 2015, 9:09 am

Jonathan Robie wrote:A text file can contain Unicode in UTF8, but not all software supports this. I put Unicode in text files all the time. You do want an editor that knows what a byte order mark is ... Stephen, does your editor work correctly for Chinese?
The simple, but quite elegant - and free - editor that I've used for these types of things for many years is Notepad++. It was designed by programmers for programmers and doesn't replace something like MS Word, but it is great for handling a large range of formatted files.
0 x
γράφω μαθεῖν

Stephen Hughes
Posts: 3323
Joined: February 26th, 2013, 7:12 am

Re: How can I save Unicode characters in a .txt file?

Post by Stephen Hughes » April 10th, 2015, 8:50 pm

Jonathan Robie wrote:What operating system are you on? When you run the text editor, can you choose "help" from the menu and tell us what the name of the text editor program is? ... Stephen, does your editor work correctly for Chinese?
Windows 7 Simplified Chinese. The name of the text editor is 记事本. My Chinese is not good enough for the help to be helpful. It has no problem handling Chinese characters.
0 x
Γελᾷ δ' ὁ μωρός, κἄν τι μὴ γέλοιον ᾖ
(Menander, Γνῶμαι μονόστιχοι 108)

Post Reply