<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML><HEAD>
<META http-equiv=Content-Type content="text/html; charset=us-ascii">
<META content="MSHTML 6.00.6000.16825" name=GENERATOR></HEAD>
<BODY><FONT face=sans-serif size=2>
<BLOCKQUOTE dir=ltr style="MARGIN-RIGHT: 0px">
<DIV><BR>4. Infoset codepage and encoding <BR><BR>The spec does not say what
codepage and encoding is used for string fields.
<BR><BR></DIV></BLOCKQUOTE></FONT>
<P><FONT face=Arial color=#0000ff size=2><SPAN class=953485712-05052009>I wanted
to comment on this.</SPAN></FONT>
<P><FONT face=Arial color=#0000ff size=2><SPAN class=953485712-05052009>There
are three choices here: </SPAN></FONT>
<OL>
<LI><FONT face=Arial color=#0000ff size=2><SPAN
class=953485712-05052009>unicode codepoints - we may need to preserve the
mapping table (from representation encoding to unicode) as part of the
infoset.</SPAN></FONT></LI>
<LI><FONT face=Arial color=#0000ff size=2><SPAN class=953485712-05052009>"As
Encoded" codepoints - we must add the encoding to the
infoset.</SPAN></FONT></LI>
<LI><FONT face=Arial color=#0000ff size=2><SPAN
class=953485712-05052009>Both</SPAN></FONT></LI></OL>
<DIV><FONT face=Arial color=#0000ff size=2><SPAN class=953485712-05052009>In
favor of unicode codepoints - simplicity. Minor issue is that some mappings will
lose information making perfect round-tripping of string contents
impossible.</SPAN></FONT></DIV>
<DIV><FONT face=Arial color=#0000ff size=2><SPAN class=953485712-05052009>E.g.,
EBCDIC has two different line-endings both of which normally are translated to
ASCII/Unicode linefeed. Hence, translating back is
ambiguous.</SPAN></FONT></DIV>
<DIV><FONT face=Arial color=#0000ff size=2><SPAN
class=953485712-05052009></SPAN></FONT> </DIV>
<DIV><FONT face=Arial color=#0000ff size=2><SPAN class=953485712-05052009>In
favor of "as encoded" - simplicity. We just add an encoding attribute to the
string infoset object which returns the information that the dfdl:encoding
representation property contained. Note that the encoding information really is
already available via the schema component associated with the string, so there
is some redundancy here. Also, there's the issue when dealing with this of
whether one wants codepoints, or raw access to the bytes. E.g., if the encoding
is UTF-8 or shifted JIS, then the characters take up 1 or more bytes. Do you
want the bytes, or the interpreted code points or both?</SPAN></FONT></DIV>
<DIV><FONT face=Arial color=#0000ff size=2><SPAN
class=953485712-05052009></SPAN></FONT> </DIV>
<DIV><FONT face=Arial color=#0000ff size=2><SPAN class=953485712-05052009>In
favor of "both" - complexity, but eliminates all the
ambiguity.</SPAN></FONT></DIV>
<DIV><FONT face=Arial color=#0000ff size=2><SPAN
class=953485712-05052009></SPAN></FONT> </DIV>
<DIV><FONT face=Arial color=#0000ff size=2><SPAN class=953485712-05052009>My
suggestion: keep it simple for v1.0 - Choose number 1 - because we can always
expand the capabilities later by providing access to the unencoded
representation one way or another. </SPAN></FONT></DIV>
<DIV><FONT face=Arial color=#0000ff size=2><SPAN
class=953485712-05052009></SPAN></FONT> </DIV>
<DIV><FONT face=Arial color=#0000ff size=2><SPAN class=953485712-05052009>If you
badly need infoset-level contents which expose the actual representation
character codes, you can always model this as an array of bytes instead of a
character string. </SPAN></FONT></DIV>
<DIV><FONT face=Arial color=#0000ff size=2><SPAN
class=953485712-05052009></SPAN></FONT> </DIV>
<DIV><FONT face=Arial color=#0000ff size=2><SPAN
class=953485712-05052009>...mike</SPAN></FONT></DIV>
<DIV><FONT face=Arial color=#0000ff size=2><SPAN
class=953485712-05052009></SPAN></FONT> </DIV>
<DIV><FONT face=Arial color=#0000ff size=2><SPAN class=953485712-05052009>
<P align=left><A name=""></A><?xml:namespace prefix = st1 ns =
"urn:schemas-microsoft-com:office:smarttags" /><st1:PersonName w:st="on"><SPAN
style="mso-bookmark: ''"><B><SPAN
style="FONT-SIZE: 10pt; COLOR: navy; FONT-FAMILY: Arial"></SPAN></B></SPAN></st1:PersonName><SPAN
style="mso-bookmark: ''"><B><SPAN
style="FONT-SIZE: 10pt; COLOR: navy; FONT-FAMILY: Arial">Mike Beckerle |
OGF DFDL WG Co-Chair | CTO | Oco, Inc.</SPAN></B></SPAN><BR><SPAN
style="FONT-SIZE: 10pt; COLOR: gray; FONT-FAMILY: Arial">Tel:
781-810-2125 | <st1:address w:st="on"><st1:Street w:st="on">100 Fifth
Ave., 4th Floor</st1:Street>, <st1:City w:st="on">Waltham</st1:City> <st1:State
w:st="on">MA</st1:State> <st1:PostalCode
w:st="on">02451</st1:PostalCode></st1:address> |</SPAN> <A
href="mailto:mbeckerle.dfdl@gmail.com"><SPAN
style="FONT-SIZE: 10pt; COLOR: gray"><FONT
face=Arial>mbeckerle.dfdl@gmail.com</FONT></SPAN></A><SPAN
style="FONT-SIZE: 10pt; COLOR: gray; FONT-FAMILY: Arial"> </SPAN></P></SPAN></FONT></DIV></BODY></HTML>