<br><font size=2 face="sans-serif">I forgot to clarify Simon's question
on sp165.</font>
<br>
<br><font size=2 face="sans-serif">This was the 'finalTerminatorCanBeMissing"
property. </font>
<br>
<br><font size=2 face="sans-serif">We considered the comment that this
might be unnecessary. </font>
<br>
<br><font size=2 face="sans-serif">Use case: file of text format. Each
"record" in the file is terminated by a CRLF so sez the user.
At the top level this file contains an array of these records. </font>
<br>
<br><font size=2 face="sans-serif">The file might or might not have a CRLF
at the end of the file because human beings might have edited the file
with a text editor, and either inserted or neglected to insert this final
CRLF.</font>
<br>
<br><font size=2 face="sans-serif">We want the file format to be legal
with or without the final CRLF; however, all prior CRLFs in the file must
be present.</font>
<br>
<br><font size=2 face="sans-serif">So how to express this:</font>
<br><font size=2 face="sans-serif">1) CRLF is a terminator of the record</font>
<br><font size=2 face="sans-serif">2) CRLF is an occursSeparator of the
enclosing array, records have no terminator. We enclose the array in a
sequence group where the array is followed by a hidden "optional"
(minOccurs=0 max=1) element of fixed="CRLF" string value.</font>
<br>
<br><font size=2 face="sans-serif">Choice (1) requires that we have finalTerminatorCanBeMissing</font>
<br>
<br><font size=2 face="sans-serif">Choice (2) is just modeling the behavior
that is required directly via hidden elements. This is tantamount to saying
that this keyword is not worth having because there is a way to model it
already. This is true of many keywords. If we deem this one too obscure,
then we need to revisit many others. (Leading/Trailing Skip Bytes is a
good example. Trivially represented by a hidden element). What are
our criteria for inclusion? Up until now our criteria have been to include
things that existing systems already have found a need for. However, existing
systems don't have hidden field capability.</font>
<br>
<br><font size=2 face="sans-serif">Note that this same missing final terminator
issue can come up not only with End-of-data, but with any bounded size
structure.</font>
<br>
<br><font size=2 face="sans-serif">E.g., suppose we say that an array has
occursUnits="bytes" and occursPath="874". Then it is
874 bytes long. The array elements can be terminated by a particular data.
E.g., semicolon. For the same reasons as the CRLF example above, we want
to be able to tolerate a missing final semicolon before the end of the
874 bytes. In effect the byte-length-limit creates an implicit "end-of-data"
for a sub-stream consisting of just those bytes. </font>
<br>
<br><font size=2 face="sans-serif">Conclusion: finalTerminatorCanBeMissing
seems to be useful enough and comes up often enough that I think the keyword
is worthwhile.</font>
<br>
<br><font size=2 face="sans-serif">Implication: we should create a list
of keywords or enumerated values for properties that we think are
in the grey area where perhaps we want to drop them. Here's some candidates:
byteOrderMarkPolicy, leading/trailingSkipBytes. Both these can be modeled
readily as hidden elements. There are probably others.</font>
<br>
<br><font size=2 face="sans-serif">Mike Beckerle<br>
STSM, Architect, Scalable Computing<br>
IBM Software Group<br>
Information Platform and Solutions<br>
Westborough, MA 01581<br>
direct: voice and FAX 508-599-7148<br>
assistant: Pam Riordan <br>
priordan@us.ibm.com
<br>
508-599-7046<br>
<br>
</font>
<br>
<br>
<br>
<table width=100%>
<tr valign=top>
<td width=40%><font size=1 face="sans-serif"><b>Mike Beckerle/Worcester/IBM</b></font>
<p><font size=1 face="sans-serif">08/14/2007 08:40 AM</font>
<td width=59%>
<table width=100%>
<tr valign=top>
<td>
<div align=right><font size=1 face="sans-serif">To</font></div>
<td><font size=1 face="sans-serif">"Simon Parker" <simon.parker@polarlake.com></font>
<tr valign=top>
<td>
<div align=right><font size=1 face="sans-serif">cc</font></div>
<td><font size=1 face="sans-serif">dfdl-wg@ogf.org</font>
<tr valign=top>
<td>
<div align=right><font size=1 face="sans-serif">Subject</font></div>
<td><font size=1 face="sans-serif">Re: [DFDL-WG] Minutes from 2007-08-08
Call</font><a href=Notes://D01ML259/85256FDB00077D54/DABA975B9FB113EB852564B5001283EA/BD9CFD7CA73D7AFD852573360052302A>Link</a></table>
<br>
<table>
<tr valign=top>
<td>
<td></table>
<br></table>
<br>
<br><font size=2 color=blue face="Arial">In conjunction with the annotated
document these notes are clear, except for 'sp165'. Perhaps someone will
recapitulate the discussion briefly at Wednesday's conference. I think
only three annotations remain:</font>
<br>
<br><font size=2 color=blue face="Arial"> sp167 Absent and
missing (expanded discussion on the wiki already)</font>
<br>
<br><font size=2 color=blue face="Arial">This will be a major topic on
a call.</font>
<br>
<br><font size=2 color=blue face="Arial"> sp172 separatorType="infix"</font>
<br>
<br><font size=2 color=blue face="Arial">I'm happy to drop this strange
stuff about separatorType=prefix or postfix and just say separator means
infix. However, I would note that at least two major integration products
(IBM WebSphere Transformation Extender - formerly Mercator, and Microsoft
Biztalk, have this concept, so we may end up putting it back in. Presumably
MS copied the earlier Mercator style, or both got it from common requirements
in some EDI standard.</font>
<br>
<br><font size=2 color=blue face="Arial"> sp173 defaultWhenMissing
(expanded discussion on the wiki already)</font>
<br>
<br><font size=3>Same topic as sp167 above. Will have a call topic to discuss.
</font>
<br><font size=3> </font>
<br><font size=2 color=blue face="Arial">I've added another contribution
to the wiki discussion on 'require'.</font>
<br>
<br><font size=2 color=blue face="Arial">This seems to be at resolution
I think, which is that we can express this using assertions. The general
style of using DFDL to describe what fixed-data syntactic constructs look
like is a good one.</font>
<br>
<br><font size=2 color=blue face="Arial">However, I've amended the Wiki
thread on this with a further issue for group consideration. See bottom
of page: </font>
<br><font size=2 color=blue face="Arial">https://forge.gridforum.org/sf/wiki/do/viewPage/projects.dfdl-wg/wiki/Require?_message=1187096164776</font>
<br><font size=3> </font>
<br><font size=2 color=blue face="Arial">The 'length and occurs' proposal
is an improvement, though I still have reservations to discuss; likewise
the 'opaque data' proposal.</font>
<br>
<br><font size=2 face="sans-serif">For a call, this week or soon. I will
send out an agenda.</font>
<br>
<br><font size=2 face="sans-serif">Mike Beckerle<br>
STSM, Architect, Scalable Computing<br>
IBM Software Group<br>
Information Platform and Solutions<br>
Westborough, MA 01581<br>
direct: voice and FAX 508-599-7148<br>
assistant: Pam Riordan <br>
priordan@us.ibm.com
<br>
508-599-7046<br>
<br>
</font>
<br>
<br>
<br>
<table width=100%>
<tr valign=top>
<td width=40%><font size=1 face="sans-serif"><b>"Simon Parker"
<simon.parker@polarlake.com></b> </font>
<br><font size=1 face="sans-serif">Sent by: dfdl-wg-bounces@ogf.org</font>
<p><font size=1 face="sans-serif">08/13/2007 10:56 AM</font>
<td width=59%>
<table width=100%>
<tr valign=top>
<td>
<div align=right><font size=1 face="sans-serif">To</font></div>
<td><font size=1 face="sans-serif"><dfdl-wg@ogf.org></font>
<tr valign=top>
<td>
<div align=right><font size=1 face="sans-serif">cc</font></div>
<td>
<tr valign=top>
<td>
<div align=right><font size=1 face="sans-serif">Subject</font></div>
<td><font size=1 face="sans-serif">Re: [DFDL-WG] Minutes from 2007-08-08
Call</font></table>
<br>
<table>
<tr valign=top>
<td>
<td></table>
<br></table>
<br>
<br>
<br>
<br><font size=3> </font>
<br><font size=2 color=blue face="Arial">In conjunction with the annotated
document these notes are clear, except for 'sp165'. Perhaps someone will
recapitulate the discussion briefly at Wednesday's conference. I think
only three annotations remain:</font>
<br>
<br><font size=2 color=blue face="Arial"> sp167 Absent and
missing (expanded discussion on the wiki already)</font>
<br><font size=2 color=blue face="Arial"> sp172 separatorType="infix"</font>
<br><font size=2 color=blue face="Arial"> sp173 defaultWhenMissing
(expanded discussion on the wiki already)</font>
<br><font size=3> </font>
<br><font size=2 color=blue face="Arial">I've added another contribution
to the wiki discussion on 'require'.</font>
<br><font size=3> </font>
<br><font size=2 color=blue face="Arial">The 'length and occurs' proposal
is an improvement, though I still have reservations to discuss; likewise
the 'opaque data' proposal.</font>
<br><font size=3> </font>
<br><font size=2 color=blue face="Arial">Regards,</font>
<br><font size=2 face="Arial"> Simon</font>
<br><font size=3> </font>
<br>
<br>
<hr><font size=2 face="Tahoma"><b>From:</b> dfdl-wg-bounces@ogf.org [mailto:dfdl-wg-bounces@ogf.org]
<b>On Behalf Of </b>Mike Beckerle<b><br>
Sent:</b> 08 August 2007 18:00<b><br>
To:</b> dfdl-wg@ogf.org<b><br>
Subject:</b> [DFDL-WG] Minutes from 2007-08-08 Call</font><font size=3><br>
</font>
<br><font size=2 face="sans-serif"><br>
MikeB, Geoff Judd, Alan Powell attended.</font><font size=3> <br>
</font><font size=2 face="sans-serif"><br>
Continued through SP's comments.</font><font size=3> <br>
</font><font size=2 face="sans-serif"><br>
sp37 - got it.</font><font size=3> <br>
</font><font size=2 face="sans-serif"><br>
sp45 - agree. This whole part to be rewritten.</font><font size=3> <br>
</font><font size=2 face="sans-serif"><br>
sp115 - ok. strict and "lax" as enums. No built-in default -
we never use defaults in the processor itself. Only in the predefined formats.</font><font size=3>
<br>
</font><font size=2 face="sans-serif"><br>
sp118 - ok</font><font size=3> <br>
</font><font size=2 face="sans-serif"><br>
sp123 - Proposal to simplify length, lengthKind, lengthUnits, and also
occursKind, occursPath, occursPathUnits needed. (along the lines of byteCount,
itemCount, length='delimited' enum, etc.)</font><font size=3> <br>
</font><font size=2 face="sans-serif"><br>
sp154 - Need specific proposal to eliminate hexBinary and use what for
opaque (consider also string with encoding='bytes'. ) Or introduce
a dfdl:byteString type or dfdl:opaque type. (derived type - just a standard
name).</font><font size=3> <br>
<br>
</font><font size=2 face="sans-serif"><br>
sp158 - see sp123</font><font size=3> <br>
</font><font size=2 face="sans-serif"><br>
sp165 - needed to have composition property for enclosing groups and or
end-of-data. Regexp doesn't fix this. </font><font size=3><br>
<br>
</font><font size=2 face="sans-serif"><br>
Mike Beckerle<br>
STSM, Architect, Scalable Computing<br>
IBM Software Group<br>
Information Platform and Solutions<br>
Westborough, MA 01581<br>
direct: voice and FAX 508-599-7148<br>
assistant: Pam Riordan <br>
priordan@us.ibm.com
<br>
508-599-7046<br>
</font><tt><font size=2>--<br>
dfdl-wg mailing list<br>
dfdl-wg@ogf.org<br>
http://www.ogf.org/mailman/listinfo/dfdl-wg</font></tt>
<br>
<br>