umouse

A0006.HTM (16288B)

---

 <title>ANS Forth RFI 0006: Writing to Input Buffers</title>
 
 This document is produced by TC X3J14 as its clarification of questions
 raised about ANSI X3.215-1994, American National Standard for Information
 Systems - Programming Languages - Forth.
 
 <p>
 The questions covered herein were raised by query
 
 <p>
 Q0006, regarding Writing to Input Buffers.
 
 <p>There are four parts in this document:
 
 <ol>
 <li><a href="#query">The original question as received.</a>
 <li><a href="#reply">The TC's reply.</a>
 <li><a href="#ballot">The Letter Ballot issued by TC Chair.</a>
 <li><a href="#results">TC Chair's statement of ballot results.</a>
 </ol>
 
 <h2><a name="query">Q0006 as received</a></h2>
 
 <pre>
 Subject: Q0006 Request Recognized
 To: X3J14@minerva.com
 Date: Thu, 05 Oct 95 09:23:38 PDT  
 </pre>
 
 The following query has been assigned number Q0005.
    - Greg Bailey, by direction  950913 0159Z
 <hr>
 
 Request from TC member Jonah Thomas (Jet.Thomas@minerva.com)
 
 <p>
 Here's a question about the Standard:
 
 <p>
 When is it allowed to write into input buffers?
 
 <p>
 On the surface this is simple:  "A program shall not write into the
 input buffer."
 
 <p>
 But what about this:
 
 <pre>
 {
 <a href=dpans6.htm#6.1.0450>:</a> CHANGE-TEXT <a href=dpans6.htm#6.1.2165>S"</a>  <a href=dpans15.htm#15.6.1.2465>WORDS</a> <a href=dpans15.htm#15.6.1.0220>.S</a> " <a href=dpans6.htm#6.1.2165>S"</a> <a href=dpans6.htm#6.1.2160>ROT</a> <a href=dpans6.htm#6.1.2260>SWAP</a> MOVE" <a href=dpans6.htm#6.1.1360>EVALUATE</a> <a href=dpans6.htm#6.1.0460>;</a>
 
  <a href=dpans15.htm#15.6.1.0220>.S</a> <a href=dpans15.htm#15.6.1.0220>.S</a> <a href=dpans15.htm#15.6.1.0220>.S</a> <a href=dpans15.htm#15.6.1.0220>.S</a> <a href=dpans15.htm#15.6.1.0220>.S</a> <a href=dpans6.htm#6.1.2216>SOURCE</a> <a href=dpans6.htm#6.1.1260>DROP</a> CHANGE-TEXT <a href=dpans15.htm#15.6.1.0220>.S</a>
 }
 </pre>
 
 Our input buffer starts out with
 
 <pre>
  <a href=dpans15.htm#15.6.1.0220>.S</a> <a href=dpans15.htm#15.6.1.0220>.S</a> <a href=dpans15.htm#15.6.1.0220>.S</a> <a href=dpans15.htm#15.6.1.0220>.S</a> <a href=dpans15.htm#15.6.1.0220>.S</a> <a href=dpans6.htm#6.1.2216>SOURCE</a> <a href=dpans6.htm#6.1.1260>DROP</a> CHANGE-TEXT <a href=dpans15.htm#15.6.1.0220>.S</a>
 </pre>
 
 When it executes CHANGE-TEXT it has ( ca ) the buffer address on the
 stack.  CHANGE-TEXT first provides a string ( ca ca' u ) then another
 string ( ca ca' u ca" u") and then evaluates the second string.  The
 original buffer is now not the input buffer, now the 2nd string is
 the read-only input buffer.  It does <a href=dpans6.htm#6.1.2160>ROT</a> ( ca' u ca ) 
 <a href=dpans6.htm#6.1.2260>SWAP</a> ( ca' ca u)
 <a href=dpans6.htm#6.1.1900>MOVE</a> and moves the first string into 
 the original input buffer, which now
 reads
 
 <pre>
  <a href=dpans15.htm#15.6.1.2465>WORDS</a> <a href=dpans15.htm#15.6.1.0220>.S</a> <a href=dpans15.htm#15.6.1.0220>.S</a> <a href=dpans15.htm#15.6.1.0220>.S</a> <a href=dpans15.htm#15.6.1.0220>.S</a> <a href=dpans6.htm#6.1.2216>SOURCE</a> <a href=dpans6.htm#6.1.1260>DROP</a> CHANGE-TEXT <a href=dpans15.htm#15.6.1.0220>.S</a>
 </pre>
 
 Then it reverts to the changed input buffer and does the final 
 <a href=dpans15.htm#15.6.1.0220>.S</a> .
 
 <p>
 I say that even if this is legal, it's bad practice.  If you want to 
 alter an input buffer, better to copy it into your own private area and
 change it however you like and then do <a href=dpans6.htm#6.1.1360>EVALUATE</a> .
 
 <p>
 My question is whether this _is_ legal, and why or why not.
 
 <p>
 I originally interpreted a previous dpANS document as saying that the
 user input buffer, the one pointed to by the obsolescent 
 <a href=dpans6.htm#6.2.2290>TIB</a> and <a href=dpans6.htm#6.2.0060>#TIB</a> ,
 is read-only.  That makes a different kind of sense to me.  In some
 systems it could be a read-only pipe from some other process, or it could
 be some special hardware read-only buffer.  It makes sense to me that the
 user input buffer should never be modified by a standard program.
 
 <p>
 So I can see 4 obvious choices, with perhaps others also available:
 
 <ol>
 
 <li>The current input buffer is read-only; strings which are not the
 current input buffer may be written to unless they're read-only for some
 other reason.
 
 <li>The current input buffer is read-only as is the user input buffer;
 any region of allotted memory is read/write except while it is the input
 buffer.
 
 <li>Any region of memory that is EVALUATEd is forever after read-only.
 
 <li>Any region of memory that is EVALUATEd is read-only until the
 evaluation is complete -- either by evaluation until the parse area is
 empty and return from <a href=dpans6.htm#6.1.1360>EVALUATE</a> or 
 <a href=dpans6.htm#6.1.1380>EXIT</a> found in the evaluated string, or
 <a href=dpans6.htm#6.1.2050>QUIT</a> or 
 <a href=dpans6.htm#6.1.0670>ABORT</a> performed before evaluation is 
 finished, or if an ambiguous
 condition exists due to the code evaluated.
 
 </ol>
 
 I don't like #3, since I sometimes want to evaluate text that's in block
 buffers or file buffers.  #4 could take a lot of figuring to tell whether
 it's being violated or not.  I like #1, but it seems to allow some
 atrocious practices in standard code.  #2 has the further problem that
 there is no way for a program to tell whether an address is in the 
 read-only user input buffer, except with 
 <a href=dpans6.htm#6.2.2218>SOURCE-ID</a> or 
 <a href=dpans6.htm#6.2.2290>TIB</a> , both of which 
 are in an optional extension wordset.
 
 <p>
 Any ideas?
 
 <h2><a name="reply">TC Reply to Q0006</a></h2>
 
 <pre>
 From: Elizabeth Rather
 Subject: Q0006R, Writing to Input Buffers, Official Response
 To: X3J14 Technical Committee
 Cc: lbarra@itic.nw.dc.us
 Date: Mon, 19 Feb 96 14:09
 
 
 Doc#:  X3J14/Q0006R
 Reference Doc#:  X3.215.1994 ANS Forth
 Date:  February 19, 1996
 Title: Response to Request for Interpretation Q0006, Writing to Input Buffers
 </pre>
 
 <h3>Q0006:  When is it allowed to write into input buffers?</h3>
 
 Request from TC member Jonah Thomas (Jet.Thomas@minerva.com)
 
 <i><blockquote>
 Here's a question about the Standard:
 
 When is it allowed to write into input buffers?
 </blockquote></i>
 
 When a standard program receives the address and length of the
 'input buffer' from the word  <a href=dpans6.htm#6.1.2216>SOURCE</a>  
 it *must* treat this information as though it describes a read-only region of memory.
 
 <p>
 This is one of the conditions that must be met in order to compose
 code whose processing of 'input buffers' is independent of the
 physical and logical characteristics of the diverse 'input sources'
 described in the Standard.
 
 <p>
 It is logically possible for an application to alter the 'input
 buffer' supplied to  <a href=dpans6.htm#6.1.1360>EVALUATE</a>  
 while that 'input buffer' is being
 processed.  This possibility exists as a special case because the
 application actually "owns" and has full control over the buffer
 in question.  As owner of the buffer, and both producer and 
 consumer of the data it contains, the application can alter this
 buffer and manipulate  <a href=dpans6.htm#6.1.0560>>IN</a>  with 
 deterministic results, assuming
 that it knows the buffer to exist in physically writable memory.
 
 <p>
 The Standard does not describe this possibility because the
 implied coding methods constitute a special case which is only
 usable with  <a href=dpans6.htm#6.1.1360>EVALUATE</a>  and can 
 lead to the development of source
 language syntax which is not processable from other 'input sources.'
 
 <h3>Relevant text from the Standard:</h3>
 
 <dl>
 
 <dt>
 <a href=dpans6.htm#6.1.1360>6.1.1360</a>  <a href=dpans6.htm#6.1.1360>EVALUATE</a>
 <dd>
     ... Make the string described ... both the
         'input source' and 'input buffer' ...
 
 <dt>
 <a href=dpans6.htm#6.1.2216>6.1.2216</a>  <a href=dpans6.htm#6.1.2216>SOURCE</a>
 <dd>
     ... c-addr is the address of, and u is the number of
     characters in, the 'input buffer'.
 
 <dt>
 <a href=dpans2.htm#2.1>2.1</a> Definitions of Terms
 <dd>
  <dl>
     <dt>input source: <dd> The device, file, block, or other entity
         that supplies characters to refill the 'input buffer'.
     <dt>input buffer: <dd> A region of memory containing the sequence
         of characters from the input source that is currently
         accessible to a program.
  </dl>
 
 <dt>
 <a href=dpans3.htm#3.3.3.5>3.3.3.5</a>  Input buffers
 <dd>
     The address, length, and content of the 'input buffer' may be
         transient.  A program shall not write into the 'input
         buffer'.  ... the 'input buffer' is either ... or a buffer
         specified by  <a href=dpans6.htm#6.1.1360>EVALUATE</a> .  ...  An ambiguous condition
         exists if a program modifies the contents of the 'input
         buffer'."
 
 </dl>
 
 <h3>Discussion of Technical Committee Intent</h3>
 
 <a href=dpans6.htm#6.2.2040>QUERY</a> and <a href=dpans6.htm#6.2.2290>TIB</a> 
 have been deprecated because while some systems have
 a discrete *place* called <a href=dpans6.htm#6.2.2290>TIB</a> that is 
 exclusively used for storing
 the results of <a href=dpans6.htm#6.2.2040>QUERY</a>, other existing systems 
 actually use <a href=dpans6.htm#6.2.2290>TIB</a> as a
 handle for the current line being interpreted from a file.  While
 applications exist making either assumption, these applications can
 obviously not work properly on both sorts of systems in general.
 
 <p>Storing into 'input buffers' is disallowed because we permit input
 sources to nest indefinitely and it is not practical for systems
 that conserve resources to guarantee unique concurrent addressability
 of all nested input sources, nor is it practical to create separate
 save areas for all current input buffers just in case someone stored
 into one of them.  The TC specifically intends that, when input is
 coming from refreshable sources, implementations may refresh their
 buffers on un-nesting to conserve resources, and that when logically
 possible implementations may use transient, shared buffers (as is
 common practice with  <a href=dpans7.htm#7.6.1.1790>LOAD</a>  on 
 multiprogrammed systems.)  Therefore,
 the results of storing into input buffers is stated as ambiguous,
 and may even be physically disallowed, as in the case of interpreting
 source from read only memory mapped files in some operating systems.
 
 <p>
 For similar reasons the address returned by  
 <a href=dpans6.htm#6.1.2216>SOURCE</a>  is transient,
 and there is specifically no guarantee that any 'input buffer' other
 than *the* (current) 'input buffer' is addressable at any time, nor
 that this address be valid after nesting or un-nesting.  (Indeed,
 in classical multiprogrammed systems the address returned by
 <a href=dpans6.htm#6.1.2216>SOURCE</a> is no longer valid after using 
 <a href=dpans6.htm#6.1.2450>WORD</a> .)
 
 <p>
 The TC expects all Systems to process buffers provided by  
 <a href=dpans6.htm#6.1.1360>EVALUATE</a>
 in place.  This is logically necessary, in our view, since there are
 no upper limits on the lengths of these buffers.  Since it is 
 semantically permissible to describe more than half of addressable memory
 in an <a href=dpans6.htm#6.1.1360>EVALUATE</a>  string it is not in 
 general *possible* to copy such a
 string elsewhere and address it consistently with the definition
 of  <a href=dpans6.htm#6.1.2216>SOURCE</a> .
 
 <p>
 Systems are not allowed to alter the contents of 'input buffers' 
 provided through  <a href=dpans6.htm#6.1.1360>EVALUATE</a> .  At the 
 same time, applications are responsible for guaranteeing that the 
 buffers they provide to  <a href=dpans6.htm#6.1.1360>EVALUATE</a>  are
 static in both address and content until the  
 <a href=dpans6.htm#6.1.1360>EVALUATE</a>  has completed.
 
 <p>
 <a href=dpans6.htm#6.1.1360>EVALUATE</a> is a special case since the 
 input buffer is literally
 the area of memory provided to <a href=dpans6.htm#6.1.1360>EVALUATE</a> and 
 is as such static.
 The application "owns" the memory it occupies, and the mechanism
 for messaging the interpreter via <a href=dpans6.htm#6.1.0560>>IN</a> 
 (as well as the syntax of
 Forth) implies no prefetching or preprocessing of input buffers
 is necessary or appropriate.
 
 <p>
 Given these conditions, it *is* deterministic for an application to
 store (with great care) into <a href=dpans6.htm#6.1.1360>EVALUATE</a> buffers 
 that it knows to be
 active, although such methods pertain exclusively to 
 <a href=dpans6.htm#6.1.1360>EVALUATE</a> and
 certainly not to any other input stream source.
 
 <p>
 However, as the Standard is written, any program that does this is
 not a standard program.
 
 <hr>
 
 It has been brought to the TC's attention that there exists at least
 one implementation which moves an <a href=dpans6.htm#6.1.1360>EVALUATE</a> 
 string to a separate work
 area for processing.  As a separate but related question we have
 been asked whether a system which does this, and which therefore
 fails John Hayes' test suite, is compliant.  (The test in question
 fails if the buffer address and length returned by 
 <a href=dpans6.htm#6.1.2216>SOURCE</a> occurring
 in an <a href=dpans6.htm#6.1.1360>EVALUATE</a> string differ from the 
 address and length of the
 argument provided to EVALUATE.)
 
 <p>
 The system in question is definitely not compliant since both the
 letter of the standard and the intent of the TC are that 
 <a href=dpans6.htm#6.1.1360>EVALUATE</a>
 strings be processed in place.  The fact that John's test suite is
 able to detect the deviation confirms its visibility to application
 code, assuming that any such code ever makes such a comparison.
 
 <p>
 The ambiguity documented for storing into 'input buffers' serves to
 mitigate the seriousness of this noncompliance since any program
 which stores into an active <a href=dpans6.htm#6.1.1360>EVALUATE</a> 
 string is environmentally dependent on the system's resolution of 
 the ambiguity.  Nevertheless, the
 Standard clearly states that the <a href=dpans6.htm#6.1.1360>EVALUATE</a> 
 string *is* the 'input
 buffer' (by definition a region of memory), and that 
 <a href=dpans6.htm#6.1.2216>SOURCE</a> returns
 the address and length of this region of memory.  This is an unqualified 
 promise to people writing standard programs, and any system
 which breaks that promise must document its nonstandard implementations 
 of <a href=dpans6.htm#6.1.1360>EVALUATE</a> and <a href=dpans6.htm#6.1.2216>SOURCE</a>.
 
 <h2><a name="ballot">Letter Ballot</a></h2>
 
 <pre>
 X3 Subgroup Letter Ballot
 Authorized by X3 Procedures - Distributed by X3 Subgroup X3J14
 Project: X3J14, ANS Forth
 Doc#:  X3J14/LB017
 Reference Doc#s:  X3J14/Q0006R, X3.215.1994 ANS Forth
 Date:  February 19, 1996
 Title:  Response to Request for Interpretation Q0006, Writing to Input Buffers
 Ballot Period:  30 Days
 Ballot Closes NOON DATE:  March 21, 1996
 Respond to:  greg@minerva.com
         or:  Elizabeth D. Rather, Chair
              FORTH, Inc.
              111 N. Sepulveda Blvd.  Suite 300
              Manhattan Beach, CA  90266
              (310) 372-8493    FAX (310) 318-7130
              erather@forth.com
 
 Statement:
     Document X3J14/Q0006R contains a proposed Response to Request for
     Interpretation Q0006.
 
 Question:
     Do you agree that this response represents the intended interpretation of
     X3.215.1994 ANS Forth?
 
 
 /------------------------  begin response area----------------------\
 |
 |  YES____ NO____ ABSTAIN____
 |
 |  Signature:  [not required for email ballots]
 |  Name:
 |  Organization:
 |
 |  Explanation (REQUIRED for NO or ABSTAIN votes):
 |    <none>
 \------------------------  end response area  ----------------------/
 
 INSTRUCTIONS:
 Please return the entire letter ballot with your response _either_ by email
 to greg@minerva.com _or_ by regular mail or fax or email to me at the above
 address, before the closing date & time.
 
    If replying electronically PLEASE edit only within the response area
    indicated above, inserting any explanatory text in place of <none>.
    Any changes made outside that area will likely be overlooked.
 
 All TC members must vote.  Failure to vote in two consecutive ballots may
 cause you to lose your voting rights in X3J14.
 
 Thank you for your participation.
 
 Elizabeth D. Rather, Chair, X3J14
 </pre>
 
 <h2><a name="results">Results of Letter Ballot</a></h2>
 
 <pre>
 Letter ballot 17 closed at noon March 21 with the following results:
 
         Y  N  A NV
 LB17:  12, 0, 1, 1
 
 Abstention from John Hayes.
 </pre>