<html><body bgcolor="#FFFFFF"><div><br>How about we specify unicode codepoints but implementations can have limitations on the numeric range of codepoints. </div><div><br></div><div>Reason: keeps us out of the codepoints vs. encodings morass. </div><div><br></div><div>...mikeb<div><br class="webkit-block-placeholder"></div></div><div><br>On May 5, 2009, at 10:20 AM, Steve Hanson <<a href="mailto:smh@uk.ibm.com">smh@uk.ibm.com</a>> wrote:<br><br></div><div><span></span></div><blockquote type="cite"><div>
<br><font size="2" face="sans-serif">There is a 4th option - remain silent
and leave it up to the implementation.</font>
<br>
<br><font size="2" face="sans-serif">Reason: Within IBM we have different
products that will embed DFDL parser/unparser. WMB requires strings in
UTF-16, that is not always the case for others.</font>
<br>
<br><font size="2" face="sans-serif">Regards<br>
<br>
Steve Hanson<br>
Programming Model Architect<br>
WebSphere Message Brokers<br>
Hursley, UK<br>
Internet: <a href="mailto:smh@uk.ibm.com"><a href="mailto:smh@uk.ibm.com">smh@uk.ibm.com</a></a><br>
Phone (+44)/(0) 1962-815848</font>
<br>
<br>
<br>
<table width="100%">
<tbody><tr valign="top">
<td width="40%"><font size="1" face="sans-serif"><b>"Mike Beckerle"
<<a href="mailto:mbeckerle.dfdl@gmail.com">mbeckerle.dfdl@gmail.com</a>></b> </font>
<br><font size="1" face="sans-serif">Sent by: <a href="mailto:dfdl-wg-bounces@ogf.org"><a href="mailto:dfdl-wg-bounces@ogf.org">dfdl-wg-bounces@ogf.org</a></a></font>
<p><font size="1" face="sans-serif">05/05/2009 14:09</font>
</p><table border="">
<tbody><tr valign="top">
<td bgcolor="white">
<div align="center"><font size="1" face="sans-serif">Please respond to<br>
<a href="mailto:mbeckerle.dfdl@gmail.com"><a href="mailto:mbeckerle.dfdl@gmail.com">mbeckerle.dfdl@gmail.com</a></a></font></div></td></tr></tbody></table>
<br>
</td><td width="59%">
<table width="100%">
<tbody><tr valign="top">
<td>
<div align="right"><font size="1" face="sans-serif">To</font></div>
</td><td><font size="1" face="sans-serif">Alan Powell/UK/IBM@IBMGB, <<a href="mailto:dfdl-wg@ogf.org">dfdl-wg@ogf.org</a>></font>
</td></tr><tr valign="top">
<td>
<div align="right"><font size="1" face="sans-serif">cc</font></div>
</td><td>
</td></tr><tr valign="top">
<td>
<div align="right"><font size="1" face="sans-serif">Subject</font></div>
</td><td><font size="1" face="sans-serif">[DFDL-WG] Infoset codepage</font></td></tr></tbody></table>
<br>
<table>
<tbody><tr valign="top">
<td>
</td><td></td></tr></tbody></table>
<br></td></tr></tbody></table>
<br>
<br>
<br><font size="2" face="sans-serif"><br>
4. Infoset codepage and encoding <br>
<br>
The spec does not say what codepage and encoding is used for string fields.
<br>
</font>
<p><font size="2" color="blue" face="Arial">I wanted to comment on this.</font><font size="3">
</font>
</p><p><font size="2" color="blue" face="Arial">There are three choices here: </font>
<br><font size="2" face="sans-serif">1. </font><font size="2" color="blue" face="Arial">unicode
codepoints - we may need to preserve the mapping table (from representation
encoding to unicode) as part of the infoset.</font>
<br><font size="2" face="sans-serif">2. </font><font size="2" color="blue" face="Arial">"As
Encoded" codepoints - we must add the encoding to the infoset.</font>
<br><font size="2" face="sans-serif">3. </font><font size="2" color="blue" face="Arial">Both</font>
<br><font size="2" color="blue" face="Arial">In favor of unicode codepoints
- simplicity. Minor issue is that some mappings will lose information making
perfect round-tripping of string contents impossible.</font>
<br><font size="2" color="blue" face="Arial">E.g., EBCDIC has two different
line-endings both of which normally are translated to ASCII/Unicode linefeed.
Hence, translating back is ambiguous.</font>
<br><font size="3"> </font>
<br><font size="2" color="blue" face="Arial">In favor of "as encoded"
- simplicity. We just add an encoding attribute to the string infoset object
which returns the information that the dfdl:encoding representation property
contained. Note that the encoding information really is already available
via the schema component associated with the string, so there is some redundancy
here. Also, there's the issue when dealing with this of whether one wants
codepoints, or raw access to the bytes. E.g., if the encoding is UTF-8
or shifted JIS, then the characters take up 1 or more bytes. Do you want
the bytes, or the interpreted code points or both?</font>
<br><font size="3"> </font>
<br><font size="2" color="blue" face="Arial">In favor of "both" -
complexity, but eliminates all the ambiguity.</font>
<br><font size="3"> </font>
<br><font size="2" color="blue" face="Arial">My suggestion: keep it simple
for v1.0 - Choose number 1 - because we can always expand the capabilities
later by providing access to the unencoded representation one way or another.
</font>
<br><font size="3"> </font>
<br><font size="2" color="blue" face="Arial">If you badly need infoset-level
contents which expose the actual representation character codes, you can
always model this as an array of bytes instead of a character string. </font>
<br><font size="3"> </font>
<br><font size="2" color="blue" face="Arial">...mike</font>
<br><font size="3"> </font>
</p><p><font size="2" color="#000080" face="Arial"><b>Mike Beckerle | OGF DFDL
WG Co-Chair | CTO | Oco, Inc.</b></font><font size="2" color="#808080" face="Arial"><br>
Tel: 781-810-2125 | 100 Fifth Ave., 4th Floor, Waltham MA 02451
|</font><font size="2" color="blue" face="Arial"> </font><a href="mailto:mbeckerle.dfdl@gmail.com"><font size="2" color="#808080" face="Arial"><u>mbeckerle.dfdl@gmail.com</u></font></a><font size="2" color="#808080" face="Arial">
</font><tt><font size="2">--<br>
dfdl-wg mailing list<br>
<a href="mailto:dfdl-wg@ogf.org"><a href="mailto:dfdl-wg@ogf.org">dfdl-wg@ogf.org</a></a><br>
<a href="http://www.ogf.org/mailman/listinfo/dfdl-wg"><a href="http://www.ogf.org/mailman/listinfo/dfdl-wg">http://www.ogf.org/mailman/listinfo/dfdl-wg</a></a></font></tt>
<br><font size="2" face="sans-serif"><br>
</font>
<br><font size="2" face="sans-serif"><br>
</font>
</p><hr><font size="2" face="sans-serif"><br>
<i><br>
</i></font>
<p><font size="2" face="sans-serif"><i>Unless stated otherwise above:<br>
IBM United Kingdom Limited - Registered in England and Wales with number
741598. <br>
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6
3AU</i></font>
</p><p><font size="2" face="sans-serif"><br>
<br>
</font>
<br>
<br><font size="2" face="sans-serif"><br>
</font></p></div></blockquote></body></html>