Your comments
Yes, please do. Thanks, Alexander!
9 years ago
Alexander, I finally got around to analyzing this.
I ran Windows command prompt file compare (FC), followed by a file dump program. The "missing" three bytes turn out to be Unicode byte order mark (BOM) characters (0xEF, 0xBB, 0xBF) that Microsoft Notepad inserts at the very beginning of each Unicode text file. Apparently, the Textastic text editor strips them when it saves the file. Should it be doing this? I want to able to modify this (and other) files using both Notepad and Textastic interchangeably without problems.
UPDATE: I'm including an excerpt (below) from the Wikipedia article on "byte order mark." It states that, although the BOM is optional, the Unicode standard does NOT recommend removing it, if it is present.
EXCERPT:
The UTF-8 representation of the BOM is the byte sequence
The Unicode Standard permits the BOM in UTF-8,[2] but does not require or recommend its use.[3] Byte order has no meaning in UTF-8,[4] so its only use in UTF-8 is to signal at the start that the text stream is encoded in UTF-8. The BOM may also appear when UTF-8 data is converted from other encodings that use a BOM. The standard also does not recommend removing a BOM when it is there, so that round-tripping between encodings does not lose information, and so that code that relies on it continues to work.[5] [6] The IETF recommends that if a protocol either (a) always uses UTF-8, or (b) has some other way to indicate what encoding is being used, then it "SHOULD forbid use of U+FEFF as a signature."[7]
I ran Windows command prompt file compare (FC), followed by a file dump program. The "missing" three bytes turn out to be Unicode byte order mark (BOM) characters (0xEF, 0xBB, 0xBF) that Microsoft Notepad inserts at the very beginning of each Unicode text file. Apparently, the Textastic text editor strips them when it saves the file. Should it be doing this? I want to able to modify this (and other) files using both Notepad and Textastic interchangeably without problems.
UPDATE: I'm including an excerpt (below) from the Wikipedia article on "byte order mark." It states that, although the BOM is optional, the Unicode standard does NOT recommend removing it, if it is present.
EXCERPT:
The UTF-8 representation of the BOM is the byte sequence
0xEF,0xBB,0xBF. A text editor or web browser interpreting the text as ISO-8859-1 or CP1252 will display the characters  for this.The Unicode Standard permits the BOM in UTF-8,[2] but does not require or recommend its use.[3] Byte order has no meaning in UTF-8,[4] so its only use in UTF-8 is to signal at the start that the text stream is encoded in UTF-8. The BOM may also appear when UTF-8 data is converted from other encodings that use a BOM. The standard also does not recommend removing a BOM when it is there, so that round-tripping between encodings does not lose information, and so that code that relies on it continues to work.[5] [6] The IETF recommends that if a protocol either (a) always uses UTF-8, or (b) has some other way to indicate what encoding is being used, then it "SHOULD forbid use of U+FEFF as a signature."[7]
Thanks, Adrian and Alexander for your fast replies.
Alexander, the file is about 45KB in size, with about 1,440 lines, so the loss of three bytes is a bit strange. However, I had already decided to run the before-and-after files through the Windows command prompt File Compare (FC) program - and will do as soon as I get a chance. I'll reply here with the results once I've done that. Thanks again
Alexander, the file is about 45KB in size, with about 1,440 lines, so the loss of three bytes is a bit strange. However, I had already decided to run the before-and-after files through the Windows command prompt File Compare (FC) program - and will do as soon as I get a chance. I'll reply here with the results once I've done that. Thanks again
Okay, great. Thanks again!
Out of curiosity, is there any way to specify ASCII transfer mode?
That's what I needed to know. Thank you, Alexander! 😊
Customer support service by UserEcho
