<br><font size=2 face="sans-serif">Isn't choice 2 the most flexible? The
caller can convert to what they need.</font>
<br><font size=2 face="sans-serif"><br>
Alan Powell<br>
<br>
MP 211, IBM UK Labs, Hursley, Winchester, SO21 2JN, England<br>
Notes Id: Alan Powell/UK/IBM email: alan_powell@uk.ibm.com
<br>
Tel: +44 (0)1962 815073
Fax: +44 (0)1962 816898<br>
</font>
<br>
<br>
<br>
<table width=100%>
<tr valign=top>
<td><font size=1 color=#5f5f5f face="sans-serif">From:</font>
<td><font size=1 face="sans-serif">DFDL <mbeckerle.dfdl@gmail.com></font>
<tr valign=top>
<td><font size=1 color=#5f5f5f face="sans-serif">To:</font>
<td><font size=1 face="sans-serif">Steve Hanson/UK/IBM@IBMGB</font>
<tr>
<td valign=top><font size=1 color=#5f5f5f face="sans-serif">Cc:</font>
<td><font size=1 face="sans-serif">Alan Powell/UK/IBM@IBMGB, "dfdl-wg@ogf.org"
<dfdl-wg@ogf.org>, "dfdl-wg-bounces@ogf.org" <dfdl-wg-bounces@ogf.org></font>
<tr valign=top>
<td><font size=1 color=#5f5f5f face="sans-serif">Date:</font>
<td><font size=1 face="sans-serif">05/05/2009 15:35</font>
<tr valign=top>
<td><font size=1 color=#5f5f5f face="sans-serif">Subject:</font>
<td><font size=1 face="sans-serif">Re: [DFDL-WG] Infoset codepage</font></table>
<br>
<hr noshade>
<br>
<br>
<br><font size=3><br>
How about we specify unicode codepoints but implementations can have limitations
on the numeric range of codepoints. </font>
<br>
<br><font size=3>Reason: keeps us out of the codepoints vs. encodings morass.
</font>
<br>
<br><font size=3>...mikeb</font>
<br>
<br><font size=3><br>
On May 5, 2009, at 10:20 AM, Steve Hanson <</font><a href=mailto:smh@uk.ibm.com><font size=3 color=blue><u>smh@uk.ibm.com</u></font></a><font size=3>>
wrote:<br>
</font>
<br><font size=2 face="sans-serif"><br>
There is a 4th option - remain silent and leave it up to the implementation.</font><font size=3>
<br>
</font><font size=2 face="sans-serif"><br>
Reason: Within IBM we have different products that will embed DFDL
parser/unparser. WMB requires strings in UTF-16, that is not always the
case for others.</font><font size=3> <br>
</font><font size=2 face="sans-serif"><br>
Regards<br>
<br>
Steve Hanson<br>
Programming Model Architect<br>
WebSphere Message Brokers<br>
Hursley, UK<br>
Internet: </font><a href=mailto:smh@uk.ibm.com></a><a href=mailto:smh@uk.ibm.com><font size=2 color=blue face="sans-serif"><u>smh@uk.ibm.com</u></font></a><font size=2 face="sans-serif"><br>
Phone (+44)/(0) 1962-815848</font><font size=3> <br>
<br>
</font>
<table width=100%>
<tr valign=top>
<td width=48%><font size=1 face="sans-serif"><b>"Mike Beckerle"
<</b></font><a href=mailto:mbeckerle.dfdl@gmail.com><font size=1 color=blue face="sans-serif"><b><u>mbeckerle.dfdl@gmail.com</u></b></font></a><font size=1 face="sans-serif"><b>></b>
<br>
Sent by: </font><a href="mailto:dfdl-wg-bounces@ogf.org"></a><a href="mailto:dfdl-wg-bounces@ogf.org"><font size=1 color=blue face="sans-serif"><u>dfdl-wg-bounces@ogf.org</u></font></a><font size=3>
</font>
<p><font size=1 face="sans-serif">05/05/2009 14:09</font><font size=3>
</font>
<p>
<br>
<table border>
<tr valign=top>
<td bgcolor=white>
<div align=center><font size=1 face="sans-serif">Please respond to</font><font size=1 color=blue face="sans-serif"><u><br>
</u></font><a href=mailto:mbeckerle.dfdl@gmail.com></a><a href=mailto:mbeckerle.dfdl@gmail.com><font size=1 color=blue face="sans-serif"><u>mbeckerle.dfdl@gmail.com</u></font></a></div></table>
<br>
<td width=51%>
<table width=100%>
<tr valign=top>
<td width=13%>
<div align=right><font size=1 face="sans-serif">To</font></div>
<td width=86%><font size=1 face="sans-serif">Alan Powell/UK/IBM@IBMGB,
<</font><a href="mailto:dfdl-wg@ogf.org"><font size=1 color=blue face="sans-serif"><u>dfdl-wg@ogf.org</u></font></a><font size=1 face="sans-serif">></font><font size=3>
</font>
<tr valign=top>
<td>
<div align=right><font size=1 face="sans-serif">cc</font></div>
<td>
<tr valign=top>
<td>
<div align=right><font size=1 face="sans-serif">Subject</font></div>
<td><font size=1 face="sans-serif">[DFDL-WG] Infoset codepage</font></table>
<br>
<br>
<table>
<tr valign=top>
<td>
<td></table>
<br></table>
<br><font size=3><br>
<br>
</font><font size=2 face="sans-serif"><br>
<br>
4. Infoset codepage and encoding <br>
<br>
The spec does not say what codepage and encoding is used for string fields.
</font>
<p><font size=2 color=blue face="Arial">I wanted to comment on this.</font><font size=3>
</font>
<p><font size=2 color=blue face="Arial">There are three choices here: </font><font size=2 face="sans-serif"><br>
1. </font><font size=2 color=blue face="Arial">unicode
codepoints - we may need to preserve the mapping table (from representation
encoding to unicode) as part of the infoset.</font><font size=3> </font><font size=2 face="sans-serif"><br>
2. </font><font size=2 color=blue face="Arial">"As
Encoded" codepoints - we must add the encoding to the infoset.</font><font size=3>
</font><font size=2 face="sans-serif"><br>
3. </font><font size=2 color=blue face="Arial">Both</font><font size=3>
</font><font size=2 color=blue face="Arial"><br>
In favor of unicode codepoints - simplicity. Minor issue is that some mappings
will lose information making perfect round-tripping of string contents
impossible.</font><font size=3> </font><font size=2 color=blue face="Arial"><br>
E.g., EBCDIC has two different line-endings both of which normally are
translated to ASCII/Unicode linefeed. Hence, translating back is ambiguous.</font><font size=3>
<br>
</font><font size=2 color=blue face="Arial"><br>
In favor of "as encoded" - simplicity. We just add an encoding
attribute to the string infoset object which returns the information that
the dfdl:encoding representation property contained. Note that the encoding
information really is already available via the schema component associated
with the string, so there is some redundancy here. Also, there's the issue
when dealing with this of whether one wants codepoints, or raw access to
the bytes. E.g., if the encoding is UTF-8 or shifted JIS, then the characters
take up 1 or more bytes. Do you want the bytes, or the interpreted code
points or both?</font><font size=3> <br>
</font><font size=2 color=blue face="Arial"><br>
In favor of "both" - complexity, but eliminates all the ambiguity.</font><font size=3>
<br>
</font><font size=2 color=blue face="Arial"><br>
My suggestion: keep it simple for v1.0 - Choose number 1 - because we can
always expand the capabilities later by providing access to the unencoded
representation one way or another. </font><font size=3><br>
</font><font size=2 color=blue face="Arial"><br>
If you badly need infoset-level contents which expose the actual representation
character codes, you can always model this as an array of bytes instead
of a character string. </font><font size=3><br>
</font><font size=2 color=blue face="Arial"><br>
...mike</font><font size=3> <br>
</font>
<p><font size=2 color=#000080 face="Arial"><b>Mike Beckerle | OGF DFDL
WG Co-Chair | CTO | Oco, Inc.</b></font><font size=2 color=#808080 face="Arial"><br>
Tel: 781-810-2125 | 100 Fifth Ave., 4th Floor, Waltham MA 02451
|</font><font size=2 color=blue face="Arial"> </font><a href=mailto:mbeckerle.dfdl@gmail.com><font size=2 color=#808080 face="Arial"><u>mbeckerle.dfdl@gmail.com</u></font></a><font size=2 color=#808080 face="Arial">
</font><tt><font size=2>--<br>
dfdl-wg mailing list<br>
</font></tt><a href="mailto:dfdl-wg@ogf.org"></a><a href="mailto:dfdl-wg@ogf.org"><tt><font size=2 color=blue><u>dfdl-wg@ogf.org</u></font></tt></a><tt><font size=2><br>
</font></tt><a href="http://www.ogf.org/mailman/listinfo/dfdl-wg"></a><a href="http://www.ogf.org/mailman/listinfo/dfdl-wg"><tt><font size=2 color=blue><u>http://www.ogf.org/mailman/listinfo/dfdl-wg</u></font></tt></a><font size=3>
</font><font size=2 face="sans-serif"><br>
</font><font size=3><br>
</font><font size=2 face="sans-serif"><br>
</font>
<p>
<hr><font size=2 face="sans-serif"><i><br>
</i></font>
<p><font size=2 face="sans-serif"><i>Unless stated otherwise above:<br>
IBM United Kingdom Limited - Registered in England and Wales with number
741598. <br>
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6
3AU</i></font><font size=3> </font>
<p><font size=2 face="sans-serif"><br>
</font><font size=3><br>
<br>
</font><font size=2 face="sans-serif"><br>
</font>
<p>
<br><font size=2 face="sans-serif"><br>
</font>
<br><font size=2 face="sans-serif"><br>
</font>
<hr><font size=2 face="sans-serif"><br>
<i><br>
</i></font>
<p><font size=2 face="sans-serif"><i>Unless stated otherwise above:<br>
IBM United Kingdom Limited - Registered in England and Wales with number
741598. <br>
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6
3AU</i></font>
<p><font size=2 face="sans-serif"><br>
<br>
</font>
<br>
<br><font size=2 face="sans-serif"><br>
</font>