\chapter{Library Utilities} \label{library_utilities}
%====================================================

In this chapter we introduce libraries of some useful predicates that
are supplied with XSB.  Interfaces and more elaborate packages are
documented in later chapters.  These predicates are available only
when imported them from (or explicitly consult) the corresponding
modules.


\section{List Processing}
%========================
The XSB library contains various list utilities, some of which 
are listed below.  These predicates should be explicitly imported from
the module specified after the skeletal specification of each predicate.
There are a lot more useful list processing predicates in various modules
of the XSB system, and the interested user can find them by 
looking at the sources.

\begin{description}
\ournewitem{append(?List1, ?List2, ?List3)}{basics}\index{\texttt{append/3}}
%\predindex{append/3~(L)}
    Succeeds if list {\tt List3} is the concatenation of lists 
    {\tt List1} and {\tt List2}.

\ournewitem{member(?Element, ?List)}{basics}\index{\texttt{member/2}}
%\predindex{member/2~(L)}
    Checks whether {\tt Element} unifies with any element of list 
    {\tt List}, succeeding more than once if there are multiple 
    such elements.

\ournewitem{memberchk(?Element, ?List)}{basics}\index{\texttt{memberchk/2}}
%\predindex{memberchk/2~(L)}
    Similar to {\tt member}/2, except that {\tt memberchk}/2 is
    deterministic, i.e.\ does not succeed more than once for any call.

\ournewitem{ith(?Index, ?List, ?Element)}{basics}\index{\texttt{ith/3}}
%\predindex{ith/3~(L)}
    Succeeds if the ${\tt Index^{th}}$ element of the list {\tt List} 
    unifies with {\tt Element}.  Fails if {\tt Index} is not a positive
    integer or greater than the length of {\tt List}.
    Either {\tt Index} and {\tt List}, or {\tt List} and {\tt Element}, 
    should be instantiated (but not necessarily ground) at the time of 
    the call.

\ournewitem{delete\_ith(+Index, +List, ?Element, ?RestList)}{listutil}
\index{\texttt{delete\_ith/4}}
%\predindex{delete\_ith/4~(L)}
    Succeeds if the ${\tt Index^{th}}$ element of the list {\tt List}
    unifies with {\tt Element}, and {\tt RestList} is {\tt List} with
    {\tt Element} removed.  Fails if {\tt Index} is not a positive
    integer or greater than the length of {\tt List}.

\ournewitem{log\_ith(?Index, ?Tree, ?Element)}{basics}\index{\texttt{log\_ith/3}}
%\predindex{log\_ith/3~(L)}
    Succeeds if the ${\tt Index^{th}}$ element of the Tree {\tt Tree}
    unifies with {\tt Element}.  Fails if {\tt Index} is not a
    positive integer or greater than the number of elements that can
    be in {\tt Tree}.  Either {\tt Index} and {\tt Tree}, or {\tt
    Tree} and {\tt Element}, should be instantiated (but not
    necessarily ground) at the time of the call.  Tree is a list of
    full binary trees, the first being of depth 0, and each one being
    of depth one greater than its predecessor.  So {\tt log\_ith/3} is
    very similar to {\tt ith/3} except it uses a tree instead of a
    list to obtain log-time access to its elements.

\ournewitem{log\_ith\_bound(?Index, ?Tree, ?Element)}{basics}
        \index{\texttt{log\_ith\_bound/3}}
%\predindex{log\_ith\_bound/3~(L)}
    is like {\tt log\_ith/3}, but only if the ${\tt Index^{th}}$ element
    of {\tt Tree} is nonvariable and equal to Element.  This predicate
    can be used in both directions, and is most useful with Index
    unbound, since it will then bind {\tt Index} and {\tt Element} for
    each non-variable element in {\tt Tree} (in time proportional to
    $N*logN$, for $N$ the number of non-variable entries in {\tt
    Tree}.)

\ournewitem{length(?List, ?Length)}{basics}\index{\texttt{length/2}}
%\predindex{length/2~(L)}
    Succeeds if the length of the list {\tt List} is {\tt Length}.
    This predicate is deterministic if {\tt List} is instantiated 
    to a list of definite length, but is nondeterministic if 
    {\tt List} is a variable or has a variable tail.  If {\tt List}
    is uninstantiated, it is unified with a list of length {\tt Length}
    that contains variables.

\ournewitem{same\_length(?List1, ?List2)}{basics}\index{\texttt{same\_length/2}}
%\predindex{same\_length/2~(L)}
    Succeeds if list {\tt List1} and {\tt List2} are both lists of
    the same number of elements.  No relation between the types or
    values of their elements is implied.  This predicate may be used
    to generate either list (containing variables as elements) given
    the other, or to generate two lists of the same length, in which
    case the arguments will be bound to lists of length $0,1,2,\ldots$.

\ournewitem{select(?Element, ?L1, ?L2)}{basics}\index{\texttt{select/3}}
%\predindex{select/3~(L)}
    {\tt List2} derives from {\tt List1} by selecting (removing) an 
    {\tt Element} non-deterministically.

\ournewitem{reverse(+List, ?ReversedList)}{basics}\index{\texttt{reverse/2}}
%\predindex{reverse/2~(L)}
    Succeeds if {\tt ReversedList} is the reverse of list {\tt List}.
    If {\tt List} is not a proper list, {\tt reverse/2} can succeed
    arbitrarily many times.  It works only one way.

\ournewitem{perm(+List, ?Perm)}{basics}\index{\texttt{perm/2}}
%\predindex{perm/2~(L)}
    Succeeds when {\tt List} and {\tt Perm} are permutations of each
    other.  The main use of {\tt perm/2} is to generate permutations
    of a given list.  {\tt List} must be a proper list.
    {\tt Perm} may be partly instantiated.

\ournewitem{subseq(?Sequence, ?SubSequence, ?Complement)}{basics}
\index{\texttt{subseq/3}}
%\predindex{subseq/3(L)}
    Succeeds when {\tt SubSequence} and {\tt Complement} are both
    subsequences of the list {\tt Sequence} (the order of corresponding
    elements being preserved) and every element of {\tt Sequence} which
    is not in {\tt SubSequence} is in the {\tt Complement} and vice
    versa.  That is,
    \[ length({\tt Sequence}) =
                length({\tt SubSequence})+length({\tt Complement}) \]
    for example, {\tt subseq([1,2,3,4], [1,3], [2,4]).}
    The main use of {\tt subseq/3} is to generate subsets and their
    complements together, but can also be used to interleave two lists
    in all possible ways.

\ournewitem{merge(+List1, +List2, ?List3)}{listutil}\index{\texttt{merge/3}}
%\predindex{merge/3~(L)}
    Succeeds if {\tt List3} is the list resulting from ``merging'' lists 
    {\tt List1} and {\tt List2},
    i.e.\ the elements of {\tt List1} together with any element of 
    {\tt List2} not occurring in {\tt List1}.
    If~{\tt List1} or~{\tt List2} contain duplicates, {\tt List3} may 
    also contain duplicates.

\ournewitem{absmerge(+List1, +List2, ?List3)}{listutil}\index{\texttt{absmerge/3}}
    Predicate {\tt absmerge/3} is similar to {\tt merge/3}, except that 
    it uses predicate {\tt absmember/2} described below rather than 
    {\tt member/2}.

\ournewitem{absmember(+Element, +List)}{listutil}\index{\texttt{absmember/2}}
    Similar to {\tt member}/2, except that it checks for identity
    (through the use of predicate {\tt '=='/2}) rather than unifiability 
    (through {\tt '='/2}) of {\tt Element} with elements of {\tt List}.

\ournewitem{member2(?Element, ?List)}{listutil}\index{\texttt{member2/2}}
    Checks whether {\tt Element} unifies with any of the actual elements 
    of {\tt List}.  The only difference between this predicate and 
    predicate {\tt member/2} is on lists having a variable tail, 
    e.g.\ \verb'[a, b, c | _ ]': while {\tt member/2} would insert 
    {\tt Element} at the end of such a list if it did not find it, 
    Predicate {\tt member2/2} only checks for membership but does not 
    insert the {\tt Element} into the list if it is not there.

\ournewitem{closetail(?List)}{listutil}\index{\texttt{closetail/1}}
    Predicate {\tt closetail/1} closes the tail of an open-ended list.
    It succeeds only once.

\end{description}

\subsection{Processing Comma Lists}
%===========================

It is often useful to process comma lists when meta-interpreting or
preprocessing.  XSB libraries include the following simple utilities.

\begin{description}

\ournewitem{comma\_to\_list(+CommaList,-List)}{basics}
\index{\texttt{comma\_to\_list/2}}
%
Transforms {\tt CommaList} to {\tt List}.

\ourrepeatnewitem{comma\_append(?CL1,?CL2,?CL3)}{basics}
\index{\texttt{comma\_append/3}}
%
\ourrepeatnewitem{comma\_length(?CommaList,?Length)}{basics}
\index{\texttt{comma\_length/2}}
%
\ourrepeatnewitem{comma\_member(?Element,?CommaList)}{basics}
\index{\texttt{comma\_member/2}}
%
\ournewitem{comma\_member(?Element,?CommaList)}{basics}
\index{\texttt{comma\_memberchk/2}}
%
Analogues for comma lists of {\tt append/3}, {\tt length/3}, {\tt member/2} and {\tt
memberchk/2}, respectively.

\end{description}
	
\section{Attributed Variables} \label{sec:attributed-variables}
%=============================

Attributed variables are a special data type that associates variables
with arbitrary attributes as well as supports extensible unification.
Attributed variables have proven to be a flexible and powerful mechanism
to extend a classic logic programming system with the ability of
constraint solving.  They have been implemented in SICStus \cite{sicstus-manual}, 
ECL$^i$PS$^e$ \cite{eclipse-manual} and hProlog \footnote{In XSB we try
to keep the implementation of attributed variables compatible with hProlog.} \cite{hprolog}.

In XSB attributed variabels now comes with a new interface.  It  allows the
programmer more control over structuring attributes than in SICStus.

\subsection{Lowlevel Interface}

Attributes of variables are pairs of module names and values. A module name can be any atom.
A value can be any XSB value (term, variable, atom, \ldots). Any variable has at most one 
attribute for a particular module.

Attributes can be manipulated with the following three predicates (\texttt{get\_attr/3},
\texttt{put\_attr/3} and \texttt{del\_attr/2}) defined in the module \texttt{machine}.

\begin{description}
\ournewitem{get\_attr(-Var,+Mod, ?Val)}{machine}\index{\texttt{get\_attr/3}}
   Gets the value of the attribute of \texttt{Var} in module \texttt{Mod}. Non-variable
   terms in \texttt{Var} cause a type error. \texttt{Val} will be unified with the
   value of the attribute, if it exists. Otherwise the predicate fails.

\ournewitem{put\_attr(-Var,+Mod, ?Val)}{machine}\index{\texttt{put\_attr/3}}
   Sets the value of the attribute of \texttt{Var} in module \texttt{Mod}. Non-variable
   terms in \texttt{Var} cause a type error. The previous value of the attribute is
   overwritten, if it exists.

\ournewitem{del\_attr(-Var, +Mod)}{machine}\index{\texttt{del\_attr/2}}
   Removes the attribute of \texttt{Var} in module \texttt{Mod}. Non-variable
   terms in \texttt{Var} cause a type error. The previous value of the attribute
   is removed, if it exists. 
\end{description}

One has to extend the default unification algorithm for used attributes by installing
a handler in the following way:

\demo{:- install\_verify\_attribute\_handler($+Mod, -AttrValue, -Target, +Handler$)}

\noindent The \texttt{install\_verify\_attribute\_handler/4}\index{\texttt{install\_verify\_attribute\_handler/4}} 
predicate is defined in module \texttt{machine}. $Mod$ is the attribute Module
and $Handler$ is a term with arguments $AttrValue$ and $Target$. The $Handler$
term has to correspond to a handler predicate that takes the value of the
attribute ($AttrValue$) and the term that the attributed value is bound to
($Target$) as arguments.

The handler is called after the unification of an attributed variable with a
term or other attributed variable, if the attributed variable has an attribute
in the corresponding module. The two arguments of the unification are already
bound at the time the handler is called, i.e. the handler is a post-unify handler.

Here, by giving the implementation of a simple finite domain constraint
solver (see the file \texttt{fd.P} below), we show how these lowlevel predicates
for attributed variables can be used.  In this example, an attribute in the module
\texttt{fd} is used and the value of this attribute is a list of terms.

\begin{small}
\begin{verbatim}
%% File: fd.P
%%
%% A simple finite domain constrait solver implemented using the lowlevel 
%% attributes variables interface.  

:- import put_attr/3, get_attr/3, del_attr/3, 
   install_verify_attribute_handler/4 from machine.
:- import member/2 from basics.

:- install_verify_attribute_handler(fd,AttrValue,Target,fd_handler(AttrValue,Target)).

fd_handler(Da, Target) :-
        (var(Target),                       % Target is an attributed variable
         get_attr(Target, fd, Db) ->            % has a domain
           intersection(Da, Db, [E|Es]),    % intersection not empty
           (Es = [] ->                      % exactly one element
              Target = E                    % bind Var (and Value) to E
           ;  put_attr(Target, fd, [E|Es]) % update Var's (and Value's)
           )
        ;  member(Target, Da)               % is Target a member of Da?
        ).

intersection([], _, []).
intersection([H|T], L2, [H|L3]) :-
        member(H, L2), !,
        intersection(T, L2, L3).
intersection([_|T], L2, L3) :-
        intersection(T, L2, L3).

domain(X, Dom) :- 
        var(Dom), !, 
        get_attr(X, fd, Dom). 
domain(X, List) :- 
        List = [El|Els],                     % at least one element 
        (Els = []                            % exactly one element
         -> X = El                           % implied binding 
        ;  put_attr(Fresh, fd, List),       % create a new attributed variable
           X = Fresh                         % may call verify_attributes/2
        ).

show_domain(X) :-                            % print out the domain of X
        var(X),                              % X must be a variable
        get_attr(X, fd, D),
        write('Domain of '), write(X),
        write(' is '), writeln(D).

\end{verbatim}
\end{small}

\section{Ground, Numbervars, Subsumption, Variant} \label{NumberVars}
%====================================================================

\begin{description}
\ournewitem{ground(+X)}{basics}\index{\texttt{ground/1}}
%\predindex{ground/1~(L)}
    Succeeds if {\tt X} is currently instantiated to a term that is 
    completely bound (has no uninstantiated variables in it); 
    otherwise it fails.  Predicate {\tt ground/1} has no associated 
    error conditions.

\ournewitem{numbervars(+Term, +FirstN, ?LastN)}{num\_vars}
\index{\texttt{numbervars/3}}
%\predindex{numbervars/3~(L)}
    This predicate provides a mechanism for grounding a (HiLog) term
    so that it may be analyzed.  Each variable in the (HiLog) term
    {\tt Term} is instantiated to a term of the form \verb|'$VAR'(N)|,
    where {\tt N} is an integer starting from {\tt FirstN}.  
    {\tt FirstN} is used as the value of {\tt N} for the first
    variable in {\tt Term} (starting from the left). The second distinct
    variable in {\tt Term} is given a value of {\tt N} satisfying
    {\tt "N is FirstN + 1"} and so on.  The last variable in {\tt Term}
    has the value {\tt LastN-1}.
% $ for prettyprinter....

\ournewitem{numbervars(+Term)}{num\_vars}\index{\texttt{numbervars/1}}
%\predindex{numbervars/1~(L)}
    This predicate is defined as:
    \begin{center}
    {\tt   numbervars(Term, 0, \_)}.
    \end{center}
    It is included solely for convenience.

%\ournewitem{varnumbers(Term, FirstN, Copy)}{num\_vars}
%\index{\texttt{varnumbers/3}}
%%\predindex{varnumbers/3~(B)}
%    This predicate is a partial inverse of predicate {\tt numbervars/3}.
%    It unifies {\tt Copy} with a copy of {\tt Term} in which subterms of
%    the form \verb|'$VAR'(N)| where {\tt N} is an integer not less than
%    {\tt FirstN} have been systematically replaced by fresh variables. 
%    Since 0 is the usual second argument of numbervars/3,
%    \begin{center}
%    {\tt   varnumbers(Term, Copy)}
%    \end{center}
%    is also provided.

\ournewitem{unnumbervars(+Term, +FirstN, ?Copy)}{num\_vars}
\index{\texttt{unnumbervars/3}}
%\predindex{unnumbervars/3~(B)}
    This predicate is a partial inverse of predicate {\tt
    numbervars/3}.  It creates a copy of Term in which all subterms of
    the form \verb|'$VAR'(<int>)| where \verb|<int>| is not less than
    {\tt FirstN} are uniformly replaced by variables.  \verb|'$VAR''|
    subterms with the same integer are replaced by the same variable.
    Also a version {\tt unnumbervars/2} is provided which calls {\tt
    unnumbervars/3} with the second parameter set to 0.

\ournewitem{subsumes(?Term1, +Term2)}{subsumes}\index{\texttt{subsumes/2}}
%\predindex{subsumes/2~(L)}
    Term subsumption is a sort of one-way unification.  Term {\tt Term1}
    and {\tt Term2} unify if they have a common instance, and unification
    in Prolog instantiates both terms to that (most general) common instance.
    {\tt Term1} subsumes {\tt Term2} if {\tt Term2} is already an instance of
    {\tt Term1}.  For our purposes, {\tt Term2} is an instance of {\tt Term1}
    if there is a substitution that leaves {\tt Term2} unchanged and makes
    {\tt Term1} identical to {\tt Term2}.  Predicate {\tt subsumes/2} does
    not work as described if {\tt Term1} and {\tt Term2} share common
    variables.

\ournewitem{subsumes\_chk(+Term1, +Term2)}{subsumes}
\index{\texttt{subsumes\_chk/2}}
%\predindex{subsumes\_chk/2~(L)}
    The {\tt subsumes\_chk/2} predicate is true when {\tt Term1} subsumes 
    {\tt Term2}; that is, when {\tt Term2} is already an instance of
    {\tt Term1}.  This predicate simply checks for subsumption and 
    does not bind any variables either in {\tt Term1} or in {\tt Term2}.
    {\tt Term1} and {\tt Term2} should not share any variables.

    Examples:
    {\footnotesize
     \begin{verbatim}
            | ?- subsumes_chk(a(X,f,Y,X),a(U,V,b,S)).

            no
            | ?- subsumes_chk(a(X,Y,X),a(b,b,b)).

            X = _595884
            Y = _595624
     \end{verbatim}}

\ournewitem{variant(?Term1, ?Term2)}{subsumes}\index{\texttt{variant/2}}
%\predindex{variant/2~(L)}
    This predicate is true when {\tt Term1} and {\tt Term2} are 
    alphabetic variants.  That is, you could imagine that {\tt variant/2}
    as being defined like:
    \begin{center}
    \begin{minipage}{3.5in}
    \begin{verbatim}
        variant(Term1, Term2) :-
             subsumes_chk(Term1, Term2),
             subsumes_chk(Term2, Term1).
    \end{verbatim}
    \end{minipage}
    \end{center}
    but the actual implementation of {\tt variant/2} is considerably more
    efficient.  However, in general, it does not work for terms that share
    variables; an assumption that holds for most (reasonable) uses of
    {\tt variant/2}.
\end{description}

\section{Formatted Output}

\begin{description}
\ournewitem{format(+String,+Control)}{format}\index{\texttt{format/2}}
\ournewitem{format(+Stream,+String,+Control)}{format}\index{\texttt{format/3}}

{\tt format/2} and {\tt format/3} act as a Prolog analog to the C {\tt
stdio} function {\tt printf()}, allowing formatted
output~\footnote{The {\tt format} family of predicates is due to
Quintus Prolog, by way of Ciao.}.

Output is formatted according to {\tt String} which can contain either
a format control sequence, or any other character which will appear
verbatim in the output. Control sequences act as place-holders for the
actual terms that will be output.  Thus
\begin{verbatim}
        ?- format("Hello ~q!",world).
\end{verbatim}

\noindent
will print {\tt Hello world!}.

If there is only one control sequence, the corresponding element may
be supplied alone in {\tt Control}.  If there are more, {\tt Contol}
must be a list of these elements. If there are none then {\tt Control}
must be an empty list. There have to be as many elements in {\tt
Control} as control sequences in {\tt String}.

The character \verb|~| introduces a control sequence. To print 
        a \verb|~| just repeat it:  
        \begin{verbatim}
        ?- format("Hello ~~world!", []).
        \end{verbatim}
      \noindent
        will output  \verb|Hello ~world!|.

%--------------------------------------------------------------------------------------------------
\comment{
% TLS: the \\c does not work properly.

A format may be spread over several lines. The control
        sequence {\tt \\c} followed by a @key{LFD} will translate to the 
        empty string:  
        \begin{verbatim}
        ?- format("Hello \\c
        world!", []).
        \end{verbatim}
        \noindent
        will result in {\tt Hello world!}.
}
%--------------------------------------------------------------------------------------------------

The general format of a control sequence is \verb|~NC|.  The character
{\tt C} determines the type of the control sequence.  {\tt N} is an
optional numeric argument.  An alternative form of {\tt N} is {\tt
*}. {\tt *} implies that the next argument in {\tt Arguments} should
be used as a numeric argument in the control sequence.  For example:

\begin{verbatim}
?- format("Hello~4cworld!", [0'x]).
\end{verbatim}

\noindent
and

\begin{verbatim}
?- format("Hello~*cworld!", [4,0'x]).
\end{verbatim}

\noindent
both produce

\begin{verbatim}
Helloxxxxworld!
\end{verbatim}

The following control sequences are available in XSB.

%TLS: printf-style integers and floats are not currently supported in XSB!

\begin{itemize}

\item \verb|~a|
The argument is an atom.  The atom is printed without quoting.  

\item \verb|~Nc|
(Print character.)  The argument is a number that will be interpreted as an
ASCII code. {\tt N} defaults to one and is interpreted as the number of
times to print the character.  

\item \verb|~f|
(Print float).  The argument is a float.  The float will be printed out by XSB.

\item \verb|~d|
(Print integer).  The argument is an integer, and will be printed out by XSB.

%-------------------------------------------------------------------------
\comment{
Various shadings dont work -- TLS
\item \verb|~Ne|
\item \verb|~NE|
\item \verb|~Nf|
\item \verb|~Ng|
\item \verb|~NG|
\begin{verbatim}
printf("%.@var{N}e", @var{Arg})
printf("%.@var{N}E", @var{Arg})
printf("%.@var{N}f", @var{Arg})
printf("%.@var{N}g", @var{Arg})
printf("%.@var{N}G", @var{Arg})
\end{verbatim}

If @var{N} is not supplied the action defaults to

\begin{verbatim}
printf("%e", @var{Arg})
printf("%E", @var{Arg})
printf("%f", @var{Arg})
printf("%g", @var{Arg})
printf("%G", @var{Arg})
\end{verbatim}

\item ~@var{N}d
(Print decimal.) The argument is an integer. @var{N} is interpreted as the
number of digits after the decimal point.  If @var{N} is 0 or missing, no
decimal point will be printed.  Example:  

\begin{verbatim}
?- format("Hello ~1d world!", [42]).
?- format("Hello ~d world!", [42]).
\end{verbatim}

\noindent
will print as

\begin{verbatim}
Hello 4.2 world!
Hello 42 world!
\end{verbatim}

\noindent
respectively.

\item ~@var{N}D
(Print decimal.) The argument is an integer.  Identical to @tt{~@var{N}d}
except that @tt{,} will separate groups of three digits to the left of the
decimal point.  Example:  

\begin{verbatim}
?- format("Hello ~1D world!", [12345]).
\end{verbatim}

\noindent
will print as

\begin{verbatim}
Hello 1,234.5 world!
\end{verbatim}

\item ~@var{N}r
(Print radix.) The argument is an integer. @var{N} is interpreted as a radix.
@var{N} should be >= 2 and <= 36.  If @var{N} is missing the radix defaults to
8.  The letters @tt{a-z} will denote digits larger than 9.  Example:  

\begin{verbatim}
?- format("Hello ~2r world!", [15]).
?- format("Hello ~16r world!", [15]).
\end{verbatim}

\noindent
will print as

\begin{verbatim}
Hello 1111 world!
Hello f world!V
\end{verbatim}

\noindent
respectively.

\item ~@var{N}R
(Print radix.) The argument is an integer.  Identical to @tt{~@var{N}r} except
that the letters @tt{A-Z} will denote digits larger than 9.  Example:  

\begin{verbatim}
?- format("Hello ~16R world!", [15]).
\end{verbatim}

\noindent
will print as

\begin{verbatim}
Hello F world!
\end{verbatim}

}
%-------------------------------------------------------------------------

\item \verb|~Ns|
(Print string.) The argument is a list of ASCII codes.  Exactly {\tt
N} characters will be printed. {\tt N} defaults to the length of the
string.  Example:

\begin{verbatim}
?- format("Hello ~4s ~4s!", ["new","world"]).
?- format("Hello ~s world!", ["new"]).
\end{verbatim}

\noindent
will print as

\begin{verbatim}
Hello new worl!
Hello new world!
\end{verbatim}

\noindent
respectively.

\item \verb|~i|
(Ignore argument.) The argument may be of any type.  The argument will be
ignored.  Example:  

\begin{verbatim}
?- format("Hello ~i~s world!", ["old","new"]).
\end{verbatim}

\noindent
will print as

\begin{verbatim}
Hello new world!
\end{verbatim}

\item \verb|~k|
(Print canonical.) The argument may be of any type.  The argument will be
passed to {\tt write\_canonical/2} ).  Example:  

\begin{verbatim}
?- format("Hello ~k world!", a+b+c).
\end{verbatim}

\noindent
will print as

\begin{verbatim}
Hello +(+(a,b),c) world!
\end{verbatim}

%------------------------------------------------------------------------------------------
\comment{
\item ~p
(print.) The argument may be of any type.  The argument will be passed to
{\tt print/2}.  Example:

\noindent
suposing the user has defined the predicate

\begin{verbatim}
portray([X|Y]) :- print(cons(X,Y)).
\end{verbatim}

\noindent
then

\begin{verbatim}
?- format("Hello ~p world!", [[a,b,c]]).
\end{verbatim}

\noindent
will print as

\begin{verbatim}
Hello cons(a,cons(b,cons(c,[]))) world!
\end{verbatim}
}
%------------------------------------------------------------------------------------------

\item \verb|~q|
(Print quoted.) The argument may be of any type.  The argument will be
passed to {\tt writeq/2}.  Example:

\begin{verbatim}
?- format("Hello ~q world!", [['A','B']]).
\end{verbatim}

\noindent
will print as

\begin{verbatim}
Hello ['A','B'] world!
\end{verbatim}

\item \verb|~w|
(write.) The argument may be of any type.  The argument will be passed
to {\tt write/2}.  Example:

\begin{verbatim}
?- format("Hello ~w world!", [['A','B']]).
\end{verbatim}

\noindent
will print as

\begin{verbatim}
Hello [A,B] world!
\end{verbatim}

\item \verb|~Nn|
(Print newline.) Print {\tt N} newlines. {\tt N} defaults to 1.
Example:

\begin{verbatim}
?- format("Hello ~n world!", []).
\end{verbatim}

\noindent
will print as

\begin{verbatim}
Hello
 world!
\end{verbatim}

\end{itemize}

%-------------------------------------------------------------------------------------------------------
\comment{
\item ~N
(Fresh line.) Print a newline, if not already at the beginning of a line.

% TLS: dont keep track of line_position.
The following control sequences are also available for compatibility,
but do not perform any useful functions.

\begin{itemize}
\item ~@var{N}|
(Set tab.) Set a tab stop at position @var{N}, where @var{N} defaults to
the current position, and advance the current position there.

\item ~@var{N}+
(Advance tab.) Set a tab stop at @var{N} positions past the current
position, where @var{N} defaults to 8, and advance the current position
there.

\item ~@var{N}t
(Set fill character.) Set the fill character to be used in the next
position movement to @var{N}, where @var{N} defaults to @key{SPC}.
\end{itemize}
}
\end{description}
%-------------------------------------------------------------------------------------------------------

%-------------------------------------------------------------------------------------
\comment{

\section{Lower-Level I/O}
\label{sec-low-level-io}
%======================

\begin{description}

%\ournewitem{current\_input\_port(-IOport)}{curr\_sym} \index{\texttt{current\_input\_port/}}
%   See {\tt current\_output\_port/1}.
%\ournewitem{current\_output\_port(-IOport)}{curr\_sym} \index{\texttt{current\_output\_port/}}
%   The above two predicates instantiate {\tt IOport} to the XSB I/O port
%   for the current user input and output ({\it i.e.}, the things 
%   that are manipulated through {\tt see/seen} and {\tt tell/told} predicates).
%   Once the I/O port is obtained, it is possible to safely use the
%   lower-level I/O predicates described below interchangeably with stream
%   I/O. 
    
%\ournewitem{file\_open(+FileName,+Mode,-IOport)}{file\_io} \index{\texttt{file\_open/3}}
%    Opens a file with name {\tt FileName} to be accessed
%    in mode {\tt Mode} and returns a file-descriptor in {\tt IOport} that
%    can be used to access the file.  If {\tt Mode} is atom ``{\tt r}'', the
%    the file is opened for reading; if it is ``{\tt w}'', the file is
%    opened for writing; if it is ``{\tt a}'', the file is opened for
%    appending.  If {\tt Mode} is ``{\tt sr}'', then the string making
%    up the atom {\tt FileName} is treated as the contents of the file, and
%    a descriptor is returned that allows ``file'' access to that string.
%    This is how one can use XSB's term I/O routines to build terms from
%    atoms. Mode ``{\tt sw}'' is reserved for ``open string for writing,''
%    but this has not been implemented as of yet.

%    The old-style mode specification, 0 ({\tt OREAD}), 1 ({\tt OWRITE}), 2
%    ({\tt OAPPEND}), or 3 ({\tt OSTRING}), is also supported.
%
%\ournewitem{file\_close(+IOport)}{file\_io}\index{\texttt{file\_close/1}}
%    Closes the file (or string) for descriptor {\tt IOport}.

%\ournewitem{ioport2iostream(+IOport, -Stream)}{file\_io}
%\index{\texttt{ioport2iostream/2}}
%Takes a valid open I/O port and returns an I/O stream. This stream can then
%be used by the standard I/O predicates, like see/1, tell/1, read/1, etc.
%The stream returns by this predicate is identified by a newly created atom,
%such as \verb|_$newstream_#123|. 

\ournewitem{file\_write(+IOport,+Term)}{xsb\_writ}
\index{\texttt{file\_write/2}}
    Writes the term {\tt Term} to the file (or string) with descriptor {\tt
    IOport}.

\ournewitem{file\_read(+IOport,-Term)}{xsb\_read}
\index{\texttt{file\_read/2}}
    Reads a term from the file (or string) with descriptor {\tt
    IOport} into {\tt Term}.  Note that the term must be terminated
    with a period (.) (whether it appears in a file or in a string.)

\ournewitem{file\_read(+IOport,-Term,-Vars)}{xsb\_read}
\index{\texttt{file\_read/3}}
    Reads a term from the file (or string) with descriptor {\tt
    IOport} into {\tt Term}, and returns in {\tt Vars} an open-tailed list of
    pairs of names of variables and the variables themselves that
    appear in Term.  For example, reading a term {\tt f(a,X,Y,X)}
    would result in {\tt term} being bound to {\tt
    f(a,\_25,\_26,\_25)} (for some internal variables) and {\tt Vars}
    being bound to {[vv('X',\_25) vv('Y',\_26) $\mid$ \_83]}.  Note that the
    pairing functor symbol is {\tt vv/2} and it must be imported from
    {\tt xsb\_read} along with this read predicate.  Also note that 
    {\tt Vars} is not a proper list, but has a free variable instead 
    of [] at its end.

\ournewitem{file\_read\_canonical(+IOport,-Term,-Psc)}{machine}
\index{\texttt{file\_read\_canonical/3}}
    Reads a term that is in canonical format from the the I/O port
    indicated by {\tt IOport} (as returned by {\tt file\_open/3} or
    by {\tt stat\_flag(10,IOport))}, and returns it in {\tt Term}.
    It also returns (in {\tt Psc}) the psc address of the main functor
    symbol of the term, if it is the same as that of the previously
    read term, and the current term is a ground (non 0-ary) fact.
    (This is used for efficiency in the implementation of {\tt
    load\_dync/1}).  Otherwise {\tt Psc} is set to 0.  To initialize
    its previous psc value to zero, this predicate can be called with
    {\tt IOport} of -1000.

\ournewitem{fget\_line(+Str,-Inlist,-Next)}{scrptutl}
\index{\texttt{fget\_line/3}}
    {\tt fget\_line/3} reads one line from the input stream {\tt Str} and
    unifies {\tt Inlist} to the list of ASCII integers representing the
    characters in the line, and {\tt Next} to the line terminator, either
    a newline ({\tt 10}) or EOF ({\tt-1}).
    
    This predicate is obsolete and
    \verb|file_read_line_list| should be used instead.

\end{description}

}
%-------------------------------------------------------------------------------------

%%$ -- so that font-lock-mode works in Emacs

\section{String Manipulation}
\label{sec-strings}

XSB has a number of powerful predicates that simplify the job of
string manipulation. These predicates are especially powerful when
they are combined with pattern-matching facilities provided by the
{\tt regmatch} package described in Chapter~\ref{chap-posix}.

\begin{description}
\ournewitem{str\_sub(+Sub, +Str, ?Pos)}{string}
\index{\texttt{str\_sub/3}}
\index{\texttt{str\_sub/2}}

Succeeds if {\tt Sub} is a substring of {\tt Str}. In that case, {\tt Pos}
unifies with the position where the match occurred. Positions start
from 0. {\tt str\_sub/2} is also available, which is equivalent
to having {\tt \_} in the third argument of {\tt str\_sub/3}.

\ournewitem{str\_match(+Sub, +Str, +Direction, ?Beg, ?End)}{string}
\index{\texttt{str\_match/5}}

This is an enhanced version of the previous predicate.
{\tt Direction} can be {\tt forward} or {\tt reverse} (or any abbreviation
of these). If {\tt forward}, the predicate finds the first match of {\tt
  Sub} from the beginning of {\tt Str}. If {\tt reverse}, it finds the
first match from the end of the string ({\it i.e.}, the last match of {\tt
  Sub} from the beginning of {\tt Str}). {\tt Beg} and {\tt End} must be
integers or unbound variables. (It is possible that one is bound and
another is not.)
{\tt Beg} unifies with the offset of the first character where {\tt Sub}
matched, and {\tt End} unifies with the offset of the next character to the
right of {\tt Sub} (such a character might not exist, but the offset is
stil defined). Offsets start from 0.

Both {\tt Beg} and {\tt End} can be bound to negative integers.
In this case, the value represents the offset from the \emph{second}
character past the end of {\tt Str}. Thus {\tt -1} represents the character
next to the end of {\tt Str} and can be used to check where the end of {\tt
  Sub}  matches in {\tt Str}. In the following examples
%%
\begin{verbatim}
    ?- string_match(Sub,Str,forw,X,-1).  
    ?- string_match(Sub,Str,rev,X,-1).  
    ?- string_match(Sub,Str,forw,0,X).  
\end{verbatim}
%%
the first checks if the \emph{first} match of {\tt Sub} from the beginning
of {\tt Str} is a suffix of {\tt Str} (because {\tt End} represents the
character next to the last character in {\tt Sub}, so {\tt End=-1} means
that the last characters of {\tt Sub} and of {\tt Str} occupy the same
position). If so, {\tt X} is bound to the offset (from the end of {\tt
  Str}) of the first character of {\tt Sub}. The second example checks if
the \emph{last} match of {\tt Sub} in {\tt Str} is a suffix of {\tt Str}
and binds {\tt X} to the offset of the beginning of that match (counted
from the beginning of {\tt Str}).  The last example checks if the first
match of {\tt Sub} is a prefix of {\tt Str}. If so, {\tt X} is bound to the
offset (from the beginning of {\tt Str}) of the last character of {\tt
  Sub}.

%-----------------------------------------------------------------------

\ournewitem{str\_length(+Str, ?Result)}{string}
\index{\texttt{str\_length/2}}

Unifies the {\tt Result}  with the length of {\tt Str}.

%%
\ournewitem{substring(+String, +BeginOffset, +EndOffset, -Result)}{string}
\index{\texttt{substring/4}}
%%
{\tt String} can be an atom or a list of characters, and the offsets must
be integers.  If {\tt EndOffset} is negative, endof({\tt String})+{\tt
  EndOffset}+1 is assumed. Thus, {\tt -1} means end of string.  If {\tt
  BeginOffset} is less than 0, then 0 is assumed; if it is greater than the
length of the string, then string end is assumed. If {\tt EndOffset} is
non-negative, but is less than {\tt BeginOffset}, then empty string is
returned.

Offsets start from 0.

The result returned in the fourth argument is a string, if {\tt String} is
an atom, or a list of characters, if so is {\tt String}.

The \verb|substring/4| predicate always succeeds (unless there is an error,
such as wrong argument type).

Here are some examples: 
%%
\begin{verbatim}
| ?- substring('abcdefg', 3, 5, L).

L = de

| ?- substring("abcdefg", 4, -1, L).

L = [101,102]
\end{verbatim}
%%
({\it i.e.}, L = ef represented using ASCII codes).

\ournewitem{string\_substitute(+InpStr, +SubstrList, +SubstitutionList, -OutStr)}{string}
\index{\texttt{string\_substitute/4}}

{\tt InputStr} can an atom or a list of characters.  {\tt SubstrList} must
be a list of terms of the form {\tt s(BegOffset, EndOffset)}, where the
name of the functor is immaterial.  The meaning of the offsets is the same
as for {\tt substring/4}. (In particular, negative offsets represent
offsets from the first character past the end of {\tt String}.)  Each such
term specifies a substring (between {\tt BegOffset} and {\tt EndOffset};
negative {\tt EndOffset} stands for the end of string) to be replaced.
{\tt SubstitutionList} must be a list of atoms or character lists.

Offsets start from 0, as in C/Java.

This predicate replaces the substrings specified in {\tt SubstrList} with
the corresponding strings from {\tt SubstitutionList}.  The result is
returned in {\tt OutStr}. {\tt OutStr} is a list of characters, if so is
{\tt InputStr}; otherwise, it is an atom.

If {\tt SubstitutionList} is shorter than {\tt SubstrList} then the last
string in {\tt SubstitutionList} is used for substituting the extra
substrings specified in {\tt SubstitutionList}. As a special case, this
makes it possible to replace all specified substrings with a single string.

As in the case of {\tt re\_substring/4}, if {\tt OutStr} is an atom, it is
not interned.  The user should either intern this string or convert it into
a list, as explained previously.

The \verb|string_substitute/4| predicate always succeeds.

Here are some examples:
%%
\begin{verbatim}
| ?- string_substitute('qaddf', [s(2,4)], ['123'] ,L).

L = qa123f

| ?- string_substitute('qaddf', [s(2,-1)], ['123'] ,L).

L = qa123

| ?- string_substitute("abcdefg", [s(4,-1)], ["123"],L).

L = [97,98,99,100,49,50,51]

| ?- string_substitute('1234567890123', [f(1,5),f(5,7),f(9,-2)], ["pppp", lll],X).

X = 1pppplll89lll

| ?- string_substitute('1234567890123', [f(1,5),f(6,7),f(9,-2)], ['---'],X).

X = 1---6---89---
\end{verbatim}
%%

\ournewitem{concat\_atom(+AtomList,?Atom)}{string}
\index{\texttt{concat\_atom/2 }}

{\tt AtomList} must be a list structure containing atoms, integers
and/or floats.  This predicate flattens {\tt AtomList} and
concatenates the atoms and integers into a single atom, returned in
{\tt Atom}.  Integers and floats are converted to character strings
using {\tt number\_codes/2}.

\ournewitem{concat\_atom(+AtomList,+Sep,?Atom)}{string}
\index{\texttt{concat\_atom/3 }}

{\tt AtomList} must be a list containing atoms, integers and/or
floats, and {\tt Sep} must be an atom.  This predicate concatenates
the atoms and integers into a single atom, separating each by {\tt
Sep}, return the resulting atom in {\tt Atom}.  Integers and floats
are converted to character strings using {\tt number\_codes/2}.
\end{description}

%-------------------------------------------------------------------------------------------
\comment{
% TLS: not needed with streams.
\ournewitem{atom\_to\_term(+Atom,-Term)}{string}
\index{\texttt{atom\_to\_term/2 }}

This predicate converts an atom (in {\tt Atom}) consisting of the
characters making up a valid term and converts it into that term,
placing the result in {\tt Term}.  This predicate is for convenience,
as any atom can be opened and read using XSB's streams.  The accepted
syntax is intended to be valid canonical form (with no trailing '.'),
extended by a treatment of the usual list syntax.  It should be the
inverse of {\tt term\_to\_atom/2}.  Floating point numbers are not
completely handled; only a fixed point representation is used.  If the
atom is not a syntactically valid term, the predicate fails, quietly.

\ournewitem{codes\_to\_term(+CodeList,-Term)}{string}
\index{\texttt{codes\_to\_term/2 }}

This predicate is used in the definition of {\tt atom\_to\_term/2} and 
converts a list of ascii codes consisting of a valid canonical term
into that term.  See {\tt atom\_to\_term} for details.

\ournewitem{read\_atom\_to\_term(+Atom,-Term)}{string}
\index{\texttt{read\_atom\_to\_term/2 }}

This predicate converts an atom {\tt Atom} whose characters make up a
valid term that can be read by {\tt read/1} into the term ({\tt Term})
it represents.  This predicate is for convenience, as any atom can be
opened and read using XSB's streams.  A minor difference with the
stream-based approach is that the atom should {\em not} contain a
terminating period ('.'). If the atom is not a syntactically correct
term, then this predicate fails, quietly.

\ournewitem{read\_atom\_to\_term(+Atom,-Term,-VarList)}{string}
\index{\texttt{read\_atom\_to\_term/3 }}

This predicate converts an atom {\tt Atom} whose characters make up a
valid term that can be read by {\tt read/1} into the term ({\tt Term})
it represents.  This predicate is for convenience, as any atom can be
opened and read using XSB's streams and {\tt read\_term/3}.  A minor
difference with the stream-based approach is that the atom should {\em
not} contain a terminating period ('.'). If the atom is not a
syntactically correct term, then this predicate fails, quietly.
}
%%
%-------------------------------------------------------------------------------------------

\section{Script Writing Utilities}
%========================

Prolog, (or XSB) can be useful for writing scripts.  Prolog's simple
syntax and declarative semantics make it especially suitable for
scripts that involve text processing.  There are several ways to
access script-writing commands from XSB.  The first is to execute the
command via the predicates {\tt shell/1} or {\tt shell/2}.  These
predicates can execute any command but they do not provide streamablity
across UNIX and Windows commands, and they do not return any output of
commands to Prolog.  Special predicates are provided to handle
cross-platform compatability and to bring output into XSB.  

\begin{description}
\ournewitem{expand\_filename(+FileName,-ExpandedName)}{machine}
\index{\texttt{expand\_filename/2}}
%
Expands the file name passed as the first argument and binds the
variable in the second argument to the expanded name. This includes
(1) expanding Unix tildas, (2) prepending {\tt FileName} to the
current directory, and (3) ``rectifying'' the expanded file name.  In
rectification, the expanded file name is ``rectified'' so that
multiple repeated slashes are replaced with a single slash, the
intervening ``./'' are removed, and ``../'' are applied so that the
preceding item in the path name is deleted. For instance, if the
current directory is {\tt /home}, then {\tt abc//cde/..///ff/./b} will
be converted into {\tt /home/abc/ff/b}.

Under Windows, this predicates does rectification as described above,
(using backslashes when appropriate), but it does not expand the
tildas.

\ournewitem{expand\_filename\_no\_prepend(+FileName,-ExpandedName)}{shell}
\index{\texttt{expand\_filename\_no\_prepend/2}}
This predicate behaves as {\tt expand\_filename/2}, but only expands
tildas and does rectification. It does not prepend the current working
directory to relative file names.

\ournewitem{parse\_filename(+FileName,-Dir,-Base,-Extension)}{machine}
\index{\texttt{parse\_filename/4}}
This predicate parses file names by separating the directory part, the base
name part, and file extension. If file extension is found, it is removed
from the base name. Also, directory names are rectified and if a directory
name starts with a tilde (in Unix), then it is expanded. Directory names
always end with a slash or a backslash, as appropriate for the OS at hand.

For instance, {\tt $\sim$john///doe/dir1//../foo.bar} will be parsed into:
{\tt /home/john/doe/}, {\tt foo}, and {\tt bar} (where we assume that    
{\tt /home/john} is what {\tt $\sim$john} expands into).  

\ournewitem{sleep(+Seconds)}{shell}
\index{\texttt{sleep/1}}
Put XSB to sleep for a given number of seconds.

\ournewitem{sys\_pid(-Pid)}{shell}
\index{\texttt{pid/1}}
Get Id of the current process.

\end{description}

%---------------------------------------------------------------------------------------
\comment{
TLS: Taken out of doc.  But I didn't touch the code, so as to not
introduce any incompatabilities.

\ournewitem{rm(+Path)}{shell}
\index{\texttt{rm/1}}
Remove file.

\ouritem{ls}\index{\texttt{ls/0}} 
    Under UNIX, this command lists in the current output stream the files 
    in the system's current directory if it can do so.  If so, it succeeds.
    It is the same as {\tt shell('ls -F', 0)}.  

\ouritem{cd(+Dir)}\index{\texttt{cd/1}}
    Under UNIX and Windows, this predicate changes the interpreter's
    working directory to {\tt Dir}.  If the directory specified does
    not exist or is not a directory, or the user does not have execute
    permission for that directory, predicate {\tt cd/1} simply fails
    raising no permission error.

    Exceptions:
    \begin{description}
    \item[{\tt instantiation\_error}]
        {\tt Dir} is not instantiated at the time of call.
    \item[{\tt type\_error}]
        {\tt Dir} is not an atom.
    \end{description}

\ouritem{edit(+Module)}\index{\texttt{edit/1}}
    Provided that the environment variable {\tt EDITOR} has been set,
    and the system is executing under UNIX, 
    a call to {\tt edit(foo)} will call the default editor on the file
    named {\tt foo.P}.  The argument to {\tt edit/1}, should be instantiated,
    it can be an absolute or a relative file name, but it should {\em not}
    contain the suffix {\tt .P}.  Users can also set their preferred options
    of calling the default editor by setting an environment variable named
    {\tt EDITOR\_OPTIONS}.

    Examples of possible uses of predicate {\tt edit/1} are:
    \begin{enumerate}
    \item If the environment variables have been set as follows:
          \begin{verbatim}
                setenv EDITOR /usr/ucb/vi
                setenv EDITOR_OPTIONS -l
          \end{verbatim}
          a call like:
          \begin{verbatim}
                | ?- edit(foo).
          \end{verbatim}
          will call the vi editor in the mode where left and right parentheses
          and curly brackets are checked for balance for the file {\tt foo.P}
          in the current working directory.
    \item If, on the other hand, they have been set as follows:
          \begin{verbatim}
                setenv EDITOR /usr/local/bin/emacs
                setenv EDITOR_OPTIONS -r
          \end{verbatim}
          a call like:
          \begin{verbatim}
                | ?- edit('~/foo').
          \end{verbatim}	
          will call the emacs editor in reverse video for the file {\tt foo.P}
          in user's home directory.
    \end{enumerate}

\ournewitem{file\_time(+FileName, -time(Time1,Time2))}{file\_io}
\index{\texttt{file\_time/2}}

Returns file's modification time. Because XSB steals 5 bits from each
word, time must be returned as two words: Time1, representing the most
significant digits, and Time2, representing the less significant
digits.

\ournewitem{file\_size(+FileName, -Size)}{file\_io}
\index{\texttt{file\_size/2}}

Returns file size.

\ournewitem{cwd(-Dir)}{shell}
\index{\texttt{cwd/1}}
Get current working directory.

\ournewitem{is\_absolute\_filename(+FileName)}{machine} 
\index{\texttt{is\_absolute\_filename/1}} Succeeds, if file name is absolute; fails
otherwise.  This predicate works also under Windows, {\it
i.e.}, it recognizes drive letters, etc.

\ournewitem{cd(+Path)}{shell}
\index{\texttt{cd/1}}
Change directory.

\ournewitem{rename(+Old,-New)}{shell}
\index{\texttt{rename/2}}
Rename file.

\ournewitem{directory(+Path,?Directory)}{directry}
\index{\texttt{directory/2}}

Unifies {\tt Directory} with a list of files in the directory specified by
path.  Information about the files is similar to that obtained by {\tt ls
  -l}, but transformed for ease of processing.  This currently works for
Unix only, is slow, and should be reimplemented in C using {\tt opendir()}
and {\tt readdir()}.

TLS: I think I put this in, and I think its overtaken by
file_read_line_list plus scan . 
\ournewitem{file\_to\_list(S, List)}{scrptutls}
\index{\texttt{file\_to\_list/2}}
%%
Read lines from an \emph{open} I/O port. Return a
list of terms, one per each line read. Each such term is a list of tokens
on the corresponding line.  Tokens are lists of characters separated by a
space symbol (space, newline, return, tabs, formfeed). For instance, if
{\tt IOport} 10 is bound to a file
%%
\begin{verbatim}
ads sdfdsfd ee
112 444
4555  
\end{verbatim}
%%
then
%%
\begin{verbatim}
| ?- file_to_list(10, L).  
L = [[ads,sdfdsfd,ee],[112,444],[4555]]
yes
\end{verbatim}
%%
Note: {\tt file\_to\_list/2} does not close the I/O port, so it is an
application program responsibility.


%% This predicate is superseded by spawn_process/5 and shell/5
%% Do not use it!
%%
%%\ournewitem{sysin(+Command,?Output)}{scrptutl}
%%\index{\texttt{sysin/2}}
%%
%%      Unifies {\tt Output} with the stdout of {\tt Command}
%%represented as a list of lists of tokens, where each element in the
%%outer list of {\tt Command} represents a line of stdout.
%%
%%Example:
%%{\footnotesize
%%\begin{verbatim}
%%                > uname -a
%%                Linux swiftlap 1.2.8 #1 Sun May 7 13:10:10 CDT 1995 i486
%%                | ?- sysin('uname -a',Output).
%%
%%                Output = [[Linux,swiftlap,1.2.8,#1,Sun,May,7,13:10:10,CDT,1995,i486]]
%%
%%                yes
%%\end{verbatim}}

\ouritem{ls}{shell}
\index{\texttt{ls/0}}
Does {\tt ls -F} in Unix, and a {\tt dir} in Windows.

\ournewitem{edit(+Path)}{shell}
\index{\texttt{edit/1}}
Edit file using your favorite editor (specified by the environment variable
{\tt EDITOR}. Unix only.
}
%----------------------------------------------------------------------------------------

\subsection{Communication with Subprocesses} \label{sec:subprocesses}

In the previous section, we have seen several predicates that allow XSB to
create other processes. However, these predicates offer only a very limited
way to communicate with these processes. The predicate
\verb|spawn_process/5| and friends come to the rescue. It allows a user to spawn
any process (including multiple copies of XSB) and redirect its
standard input and output to XSB streams. XSB can then write to the
process and read from it. The section of socket I/O describes yet
another mode of interprocess communication.

In addition, the predicate {\tt pipe\_open/2} described in this
section lets one create any number of pipes (that do not need to be
connected to the standard I/O streem) and talk to child processes
through these pipes.
%
All predicates in this section, except {\tt pipe\_open/2} and {\tt
fd2stream/2}, must be imported from module {\tt shell}.  The
predicates {\tt pipe\_open/2} and {\tt fd2stream/2} must be imported
from {\tt file\_io}.

\begin{description}
\ouritem{spawn\_process(+CmdSpec,-StreamToProc,-StreamFromProc,-ProcStderrStream,-ProcId)}\index{\texttt{spawn\_process/5}} 
Spawn a new process specified by {\tt CmdSpec}. {\tt CmdSpec} must be
either a single atom or a \emph{list} of atoms.
If it is an atom, then it must represent a shell command.
If it is a list, the first member of the list must be the name of the
program to run and the 
other elements must be arguments to the program. Program name must be specified
in such a way as to make sure the OS can find it using the contents of the
environment variable {\tt PATH}.
Also note that pipes, I/O redirection and such are not allowed in command
specification. That is, {\tt CmdSpec} must represent a single command.
(But read about process plumbing below and about the related predicate
{\tt shell/5}.)

The next three parameters of \verb|spawn_process| are XSB I/O stream
identifiers for the process (leading to the subprocess standard
input), from the process (from its standard output), and a stream
capturing the subprocess standard error output. The last parameter is
the system process id.
\end{description}
%%

\noindent
Here is a simple example of how it works.

%%
\begin{verbatim}
| ?- import file_flush/2, file_read_line_atom/2 from file_io.
| ?- import file_nl/1 , file_write/2 from xsb_writ.  

| ?- spawn_process([cat, '-'], To, From, Stderr, Pid),
     writeln(To,'Hello cat!'), flush_output(To,_), file_read_line_atom(From,Y).

To = 3
From = 4
Stderr = 5
Pid = 14328
Y = Hello cat!

yes
\end{verbatim}
%%

Here we created a new process, which runs the ``{\tt cat}'' program
with argument ``--''. This forces {\tt cat} to read from standard
input and write to standard output. The next line writes an atom and
newline to the XSB stream {\tt To}, which is bound to the standard
input of the {\tt cat} process (proc id {\tt 14328}). The {\tt cat}
process then copies the input to its standard output.  Since standard
output of the cat process is redirected to the XSB stream {\tt From}
in the parent process, the last line in our program is able to read it
and return in the variable {\tt Y}.
%
Note that in the second line we used {\tt flush\_output/2}. Flushing
the output is extremely important here, because XSB I/O pipe (file)
streams are buffered.  Thus, {\tt cat} might not see its input until
the buffer is filled up, so the above clause might hang. {\tt
flush\_output/2} makes sure that the input is immediately available to
the subprocess.

In addition to the above general schema, the user can tell
\verb|spawn_process/5| not to open one of the communication streams or to
use one of the existing communication streams.  This is useful when
you do not expect to write or read to/from the subprocess or when one
process wants to write to another (see the process plumbing example
below).
%
To tell that a certain stream is not needed, it suffices to bind that
stream to an atom.  For instance,
%%
\begin{verbatim}
| ?- spawn_process([cat, '-'], To, none, none, _),
     nl(To), writeln(To,'Hello cat!'), flush_output(To).


To = 3,
Hello cat!
\end{verbatim}
%%
reads from XSB and copies the result to standard output. Likewise,
%%
\begin{verbatim}
| ?- spawn_process('cat library.tex', none, From, none, _),
     file_read_line_atom(From, S).

From = 4
S = \chapter{Library Utilities} \label{library_utilities}
\end{verbatim}
%%
In each case, only one of the streams is open. (Note that the shell
command is specified as an atom rather than a list.) Finally, if both
streams are suppressed, then \verb|spawn_process| reduces to the usual
{\tt shell/1} call (in fact, this is how {\tt shell/1} is
implemented):
%%
\begin{verbatim}
| ?- spawn_process([pwd], none, none).

/usr/local/foo/bar
\end{verbatim}
%%
On the other hand, if any one of the three stream variables in
\verb|spawn_process| is bound to an already existing file stream, then the
subprocess will use that stream (see the process plumbing example
below).

One of the uses of XSB subprocesses is to create XSB servers that spawn
subprocesses and control them. A spawned subprocess can be another XSB
process. The following example shows one XSB process spawning another,
sending it a goal to evaluate and obtaining the result:
%%
\begin{verbatim}
| ?- spawn_process([xsb], To, From,Err,_),
     write(To,'assert(p(1)).'), flush_output(To,_),
     write(To,'p(X), writeln(X).'), flush_output(To,_), 
     file_read_line_atom(From,XX).  

XX = 1

yes
| ?-
\end{verbatim}
%%
Here the parent XSB process sends ``\verb|assert(p(1)).|'' and then
``\verb|p(X), writeln(X).|'' to the spawned XSB subprocess. The latter
evaluates the goal and prints (via ``\verb|writeln(X)|'') to its
standard output. The main process reads it through the {\tt From}
stream and binds the variable {\tt XX} to that output.

Finally, we should note that the stream variables in the
\verb|spawn_process| predicate can be used to do process plumbing, {\it
  i.e.}, redirect output of one subprocess into the input of another. Here
is an example:
%%
\begin{verbatim}
| ?- open(test,write,Stream),
     spawn_process([cat, 'data'], none, FromCat1, none, _),
     spawn_process([sort], FromCat1,Stream, none, _).  
\end{verbatim}
%%
Here, we first open file {\tt test}. Then \verb|cat data| is spawned.
This process has the input and standard error stream blocked (as
indicated by the atom {\tt none}), and its output goes into stream
{\tt FromCat1}.  Then we spawn another process, {\tt sort}, which
picks the output from the first process (since it uses the stream {\tt
FromCat1} as its input) and sends its own output (the sorted version
of {\tt data}) to its output stream {\tt Stream}.  However, {\tt
Stream} has already been open for output into the file {\tt
test}. Thus, the overall result of the above clause is tantamount to
the following shell command:
%%
\begin{verbatim}
        cat data | sort > test  
\end{verbatim}
%%

\paragraph{{\em Important notes about spawned processes\/}:}
\begin{enumerate}
\item Asynchronous processes spawned by XSB do not disappear (at least on
  Unix) when they terminate, \emph{unless} the XSB program executes a
  \emph{wait} on them (see {\tt process\_control} below). Instead, such
  processes become defunct \emph{zombies} (in Unix terminology); they do
  not do anything, but consume resources (such as file descriptors). So,
  when a subprocess is known to terminate, it must be waited on.
  
\item The XSB parent process must know how to terminate the asynchronous
  subprocesses it spawns. The drastic way is to kill it (see {\tt
    process\_control} below). Sometimes a subprocess might terminate by
  itself ({\it e.g.}, having finished reading a file). In other cases, the
  parent and the child programs must agree on a protocol by which the
  parent can tell the child to exit. The programs in the XSB subdirectory
  {\tt examples/subprocess} illustrate this idea. If the child subprocess
  is another XSB process, then it can be terminated by sending the atom
  {\tt end\_of\_file} or {\tt halt} to the standard input of the child.
  (For this to work, the child XSB must waiting at the prompt).
\item It is very important to not forget to close the streams that the
  parent uses to communicate with the child. These are the streams
  that are provided in arguments 2,3,4 of {\tt spawn\_process}. The
  reason is that the child might terminate, but these streams to the
  standard input of the child will remain open, since they belong to
  the parent process. As a result, the parent will own defunct I/O
  streams and might eventually run out of file descriptors or streams.
\end{enumerate}

\begin{description}
  \ouritem{process\_status(+Pid,-Status)}\index{\texttt{process\_status/2}}
  This predicate always succeeds. Given a process id, it binds the second
  argument (which must be an unbound variable) to one of the following
  atoms: {\tt running}, {\tt stopped}, {\tt exited\_normaly}, {\tt
    exited\_abnormally}, {\tt aborted}, {\tt invalid}, and {\tt unknown}.
  The {\tt invalid} status is given to processes that never existed or that
  are not children of the parent XSB process. The {\tt unknown} status is
  assigned when none of the other statuses can be assigned.
  
  Note: process status (other than {\tt running}) is system dependent.
  Windows does not seem to support {\tt stopped} and {\tt aborted}.  Also,
  processes killed using the \verb|process_control| predicate (described
  next) are often marked as {\tt invalid} rather than {\tt exited}, because
  Windows seems to lose all information about such processes. Process
  status might be inaccurate in some Unix systems as well, if the process
  has terminated and {\tt wait()} has been executed on that process.

\ouritem{process\_control(+Pid,+Operation)}\index{\texttt{process\_control/2}} 
    Perform a process control {\tt operation} on the process with the given
    {\tt Pid}. 
    Currently, the only supported operations are {\tt kill} (an atom) and {\tt
    wait(Code)} (a term).
    The former causes the process to exit unconditionally, and the latter
    waits for process completion. When the process exits, {\tt Code} is
    bound to the process exit code. The code for normal termination is 0.

    This predicate succeeds, if the operation was performed successfully.
    Otherwise, it fails. The {\tt wait} operation fails if the process
    specified in {\tt Pid} does not exist or is not a child of the parent
    XSB process. 
    
    The {\tt kill} operation might fail, if the process to be killed does
    not exist or if the parent XSB process does not have the permission to
    terminate that process. Unix and Windows have different ideas as to
    what these permissions are. See \emph{kill(2)} for Unix and
    \emph{TerminateProcess} for Windows.
    
    \emph{Note}: under Windows, the programmer's manual warns of dire
    consequences if one kills a process that has DLLs attached to it.
%
\ournewitem{get\_process\_table(-ProcessList)}{shell}
\index{\texttt{get\_process\_table/1}} 
    This predicate is imported from module {\tt shell}.
    It binds {\tt ProcessList} to the list of terms, each describing one of
    the active XSB subprocesses (created via \verb|spawn_process/5|).
    Each term has the form:
    %%
    \begin{center}
      \verb|process(Pid,ToStream,FromStream,StderrStream,CommandLine)|. 
    \end{center}
    %%
    The first argument in the term is the process id of the corresponding
    process, the next three arguments describe the three standard streams
    of the process, and the last is an atom that shows the command line used
    to invoke the process.
    This predicate always succeeds.

\ouritem{shell(+CmdSpec,-StreamToProc, -StreamFromProc, -ProcStderr,
  -ErrorCode)}\index{\texttt{shell/5}} 
%
The arguments of this predicate are similar to those of
\verb|spawn_process|, except for the following: (1) The first argument
is an atom or a list of atoms, like in \verb|spawn_process|. However,
if it is a list of atoms, then the resulting shell command is obtained
by string concatenation. This is different from \verb|spawn_process|
where each member of the list must represent an argument to the
program being invoked (and which must be the first member of that
list).  (2) The last argument is the error code returned by the shell
command and not a process id. The code -1 and 127 mean that the shell
command failed.
    
    The {\tt shell/5} predicate is similar to \verb|spawn_process| in
    that it spawns another process and can capture that process' input
    and output sreams.
%    
    The important difference, however, is that XSB will ways until the
    process spawned by {\tt shell/5} terminates. In contrast, the process
    spawned by \verb|spawn_process| will run concurrently with XSB.  In
    this latter case, XSB must explicitly synchronize with the spawned
    subprocess using the predicate \verb|process_control/2| (using the {\tt
      wait} operation), as described earlier.
    
    The fact that XSB must wait until {\tt shell/5} finishes has a very
    important implication: the amount of data the can be sent to and from
    the shell command is limited (1K is probably safe). This is because the
    shell command communicates with XSB via pipes, which have limited
    capacity.  So, if the pipe is filled, XSB will hang waiting for {\tt
      shell/5} to finish and {\tt shell/5} will wait for XSB to consume data
    from the pipe.  Thus, use \verb|spawn_process/5| for any kind of
    significant data exchange between external processes and XSB.
  
  Another difference between these two forms of spawning subprocesses is
  that {\tt CmdSpec} in {\tt shell/5} can represent \emph{any} shell
  statement, including those that have pipes and I/O redirection. In
  contrast, \verb|spawn_process| only allows command of the form ``program
  args''. For instance,
%%
\begin{verbatim}
| ?- open(test,write,Stream),
     shell('cat | sort > data', Stream, none, none, ErrCode)
\end{verbatim}
%%
As seen from this example, the same rules for blocking I/O streams
apply to {\tt shell/5}. Finally, we should note that the already
familiar standard predicates {\tt shell/1} and {\tt shell/2}
(documented in Volume 1) are implemented using {\tt shell/5}.

\paragraph{\em Notes:}
%%
\begin{enumerate}
  \item  With {\tt shell/5}, you do not have to worry about terminating
    child processes: XSB waits until the child exits automatically.
    However, since communication pipes have limited capacity, this method
    can be used only for exchanging small amounts of information between
    parent and child.
  \item The earlier remark about the need to close I/O streams to the child
    \emph{does} apply.
\end{enumerate}
%%

\ouritem{pipe\_open(-ReadPipe, -WritePipe)}\index{\texttt{pipe\_open/2}} 
Open a new pipe and return the read end and the write end of that
pipe.  If the operation fails, both {\tt ReadPipe} and {\tt WritePipe}
are bound to negative numbers.
%  
The pipes returned by the {\tt pipe\_open/2} predicate are small
integers that represent file descriptors used by the underlying
OS. They are {\bf not XSB I/O streams}, and they cannot be used for
I/O directly. To use them, one must convert them to streams using {\tt
open/3} or {\tt open/4}.~\footnote{
%% 
XSB does not convert pipe file descriptors into I/O streams
automatically.  Because of the way XSB I/O streams are represented,
they are not inherited by the child process and they do not make sense
to the child process (especially if the child is not another XSB
process). Therefore, we must pass the child processes an OS file
descriptor instead. The child then converts these descriptor into XSB
I/O streams.  
%% 
}

    The best way to illustrate how one can create a new pipe to a child
    (even if the child has been created earlier) is to show an example.
    Consider two programs, {\tt parent.P} and {\tt child.P}. The parent
    copy of XSB consults {\tt parent.P}, which does the following: First, it
    creates a pipe and spawns a copy of XSB. Then it tells the
    child copy of XSB to assert the fact {\tt pipe(RP)}, where {\tt RP} is
    a number representing the read part of the pipe. Next, the parent XSB tells
    the child XSB to consult the program {\tt child.P}. Finally, it sends
    the message {\tt Hello!}.

    The {\tt child.P} program gets the pipe from predicate {\tt pipe/1}
    (note that the parent tells the child XSB to first assert {\tt
    pipe(RP)} and only then to consult the {\tt child.P} file).
  After that, the child reads a message from the pipe and prints it to its
    standard output. Both programs are shown below:
    %%
    \begin{verbatim}
%% parent.P      
:- import pipe_open/2 from file_io.
%% Create the pipe and pass it to the child process
?- pipe_open(RP,WP),
   %% WF is now the XSB I/O stream bound to the write part of the pipe
   open(pipe(WP),write,WF),
   %% ProcInput becomes the XSB stream leading directly to the child's stdin
   spawn_process(nxsb1, ProcInput, block, block, Process),
   %% Tell the child where the reading part of the pipe is
   fmt_write(ProcInput, "assert(pipe(%d)).\n", arg(RP)),
   fmt_write(ProcInput, "[child].\n", _),
   flush_output(ProcInput, _),
   %% Pass a message through the pipe
   fmt_write(WF, "Hello!\n", _),
   flush_output(WF, _),
   fmt_write(ProcInput, "end_of_file.\n",_), % send end_of_file atom to child
   flush_output(ProcInput, _),
   %% wait for child (so as to not leave zombies around; 
   %% zombies quit when the parent finishes, but they consume resources)
   process_control(Process, wait),
   %% Close the ports used to commuicate with the process
   %% Otherwise, the parent might run out of file descriptors 
   %% (if many processes were spawned)
   close(ProcInput), close(WF).
    \end{verbatim}
    %%
    %%
    \begin{verbatim}
%% child.P
:- import file_read_line_atom/2 from file_io.
:- dynamic pipe/1.
?- pipe(P), open(pipe(P),read,F),
   %% Acknowledge receipt of the pipe
   fmt_write("\nPipe %d received\n", arg(P)),
   %% Get a message from the parent and print it to stdout
   file_read_line_atom(F, Line), write('Message was: '), writeln(Line).
    \end{verbatim}
    %%
    This produces the following output:
    %%
    \begin{verbatim}
| ?- [parent].                    <- parent XSB consults parent.P
[parent loaded]
yes
| ?- [xsb_configuration loaded]   <- parent.P spawns a child copy of XSB
[sysinitrc loaded]                   Here we see the startup messages of
[packaging loaded]                   the child copy
XSB Version 2.0 (Gouden Carolus) of June 27, 1999
[i686-pc-linux-gnu; mode: optimal; engine: slg-wam; scheduling: batched]
| ?- 
yes
| ?- [Compiling ./child]          <- The child copy of received the pipe from
[child compiled, cpu time used: 0.1300 seconds]     the parent and then the
[child loaded]                                      request to consult child.P
Pipe 15 received                  <- child.P acknowledges receipt of the pipe
Message was: Hello!               <- child.P gets the message and prints it
yes       
    \end{verbatim}
    %%
    
    Observe that the parent process is very careful about making sure
    that the child terminates and also about closing the I/O streams
    after they are no longer needed.
    
    Finally, we should note that this mechanism can be used to
    communicate through pipes with non-XSB processes as well. Indeed,
    an XSB process can create a pipe using {\tt pipe\_open}
    (\emph{before} spawning a child process), pass one end of the pipe
    to a child process (which can be a C program), and use {\tt
    open/3} to convert the other end of the pipe to an XSB
    stream. The C program, of course, does not need {\tt open/3},
    since it can use the pipe file handle directly. Likewise, a C
    program can spawn off an XSB process and pass it one end of a
    pipe.  The XSB child-process can then convert this pipe fd to a
    file using {\tt fd2iostream} and then talk to the parent C program.

\ouritem{fd2iostream(+Pipe, -IOstream)}\index{\texttt{fd2iostream/2}} 
Take a file descriptor and convert it to an XSB I/O stream. This
predicate should be used only for user-defined I/O.  Otherwise, use
{\tt open/\{3,4\}} when possible.


%------------------------------------------------------------------------------------------
\comment{
% TLS: Redefining ISO halt/1 as sys_exit/1.
\ouritem{sys\_exit(-ExitCode)}\index{\texttt{sys\_exit/1}} 
This predicate causes XSB subprocess to exit unconditionally with the exit
code {\tt ExitCode}. Normally {\tt 0} is considered to indicate normal
termination, while other exit codes are used to report various degrees of
abnormality.
}
%------------------------------------------------------------------------------------------
\end{description}



\section{Socket I/O}

The XSB socket library defines a number of predicates for communication
over BSD-style sockets. Most are modeled after and are interfaces to the
socket functions with the same name. For detailed information on sockets,
the reader is referred to the Unix man pages (another good source is
\emph{Unix Network Programming}, by W.  Richard Stevens).  Several examples
of the use of the XSB sockets interface can be found in the {\tt XSB/examples/}
directory in the XSB distribution.


XSB supports two modes of communication via sockets: \emph{stream-oriented}
and \emph{message-oriented}. In turn, stream-oriented communication can be
\emph{buffered} or \emph{character-at-a-time}.

To use \emph{buffered} stream-oriented communication, system socket
handles must be converted to XSB I/O streams using {\tt
fd2iostream/2}.  In these stream-oriented communication, messages have
no boundaries, and communication appears to the processes as reading
and writing to a file.  At present, buffered stream-oriented
communication works under Unix only.

\emph{Character-at-a-time} stream communication is accomplished using
the primitives {\tt socket\_put/3} and {\tt socket\_get0/3}. These
correspond to the usual Prolog {\tt put/1} and {\tt get0/1} I/O primitives.

In message-oriented communication, processes exchange messages that have
well-defined boundaries. The communicating processes use {\tt
  socket\_send/3} and {\tt socket\_recv/3} to talk to each other.
XSB messages are represented as strings where the first four bytes
({\tt sizeof(int)}) is an integer (represented in the binary network format
--- see the functions {\tt htonl} and {\tt ntohl} in socket documentation)
and the rest is the body of the message. The integer in the header
represents the length of the message body.

We now describe the XSB socket interface.  All predicates below must be
imported from the module {\tt socket}. Note that almost all predicates have
the last argument that unifies with the error code returned from the
corresponding socket operation. This argument is explained separately.

\paragraph{General socket calls.}
These are used to open/close sockets, to establish connections, and set
special socket options.
\begin{description}
\ouritem{socket(-Sockfd, ?ErrorCode)}\index{\texttt{socket/2}}
    A socket {\tt Sockfd}  in the AF\_INET domain is created.
    (The AF\_UNIX domain is not yet implemented). 
    {\tt Sockfd} is bound to a small integer, called socket descriptor or
    socket handle.

\ouritem{socket\_set\_option(+Sockfd,+OptionName,+Value)}\index{\texttt{socket\_set\_option/3}}
    Set socket option. At present, only the {\tt linger} option is
    supported. ``Lingering'' is a situation when a socket continues to live
    after it was shut down by the owner. This is used in order to let the
    client program that uses the socket to finish reading or writing
    from/to the socket. {\tt Value} represents the number of seconds to linger.
    The value -1 means do not linger at all.

\ouritem{socket\_close(+Sockfd, ?ErrorCode)}\index{\texttt{socket\_close/2}}
    {\tt Sockfd} is closed. Sockets used in {\tt socket\_connect/2}  should
    not be closed by {\tt socket\_close/1}  as they will be closed when the
    corresponding stream is closed.

\ouritem{socket\_bind(+Sockfd,+Port, ?ErrorCode)}\index{\texttt{socket\_bind/3}}
   The socket {\tt Sockfd}  is bound to the specified local port number.

\ouritem{socket\_connect(+Sockfd,+Port,+Hostname,?ErrorCode)}\index{\texttt{socket\_connect/4}}
    The socket Sockfd is connected to the address ({\tt Hostname}  and
{\tt Port}). If {\tt socket\_connect/4} terminates abnormally for any reason
(connection refused, timeout, etc.), then XSb closes the socket {\tt
    Sockfd} automatically, because such a socket cannot be used according
    to the BSD semantics. Therefore, it is always a good idea to check to
    the return code and reopen the socket, if the error code is not
    {\tt SOCK\_OK}.

\ouritem{socket\_listen(+Socket, +Length, ?ErrorCode)}\index{\texttt{socket\_listen/3}}
    The socket {\tt Sockfd}  is defined to have a maximum backlog queue of
{\tt Length}  pending connections.

\ouritem{socket\_accept(+Sockfd,-SockOut, ?ErrorCode)}\index{\texttt{socket\_accept/3}}
    Block the caller until a connection attempt arrives. If the incoming 
    queue is not empty, the first connection request is accepted, the call
    succeeds and returns a new socket, {\tt SockOut}, which can be used for
    this new connection.
\end{description}

\paragraph{Buffered, message-based communication.}
These calls are similar to the {\tt recv} and {\tt send} calls in C, except
that XSB wraps a higher-level message protocol around these low-level
functions. More precisely, {\tt socket\_send/3} prepends a 4-byte field
to each message, which indicates the length of the message
body. When {\tt socket\_recv/3} reads a message, it first reads the 4-byte
field to determine the length of the message and then reads the remainder
of the message. 

All this is transparent to the XSB user, but you should know these details
if you want to use these details to communicate with external processes
written in C and such. All this means that these external programs must
implement the same protocol. The subtle point here is that different
machines represent integers differently, so an integer must first be
converted into the machine-independent network format using the functions
{\tt htonl} and {\tt ntohl} provided by the socket library. For instance,
to send a message to XSB, one must do something like this:
%%
\begin{verbatim}
char *message, *msg_body;
unsigned int msg_body_len, network_encoded_len;

  msg_body_len = strlen(msg_body);
  network_encoded_len = (unsigned int) htonl((unsigned long int) msg_body_len);
  memcpy((void *) message, (void *) &network_encoded_len, 4);
  strcpy(message+4, msg_body);
\end{verbatim}
%%
To read a message sent by XSB, one can do as follows:
%%
\begin{verbatim}
int actual_len;
char lenbuf[4], msg_buff;
unsigned int msglen, net_encoded_len;  

  actual_len = (long)recvfrom(sock_handle, lenbuf, 4, 0, NULL, 0);
  memcpy((void *) &net_encoded_len, (void *) lenbuf, 4);
  msglen = ntohl(net_encoded_len);

  msg_buff = calloc(msglen+1, sizeof(char))); // check if this suceeded!!!
  recvfrom(sock_handle, msg_buff, msglen, 0, NULL, 0);
\end{verbatim}
%%
If making the external processes follow the XSB protocol is not practical
(because you did not write these programs), then you should use the
character-at-a-time interface or, better, the buffered
stream-based interface both of which are described in this section.
At present, however, the buffered stream-based interface does not work on
Windows.
%%
\begin{description}
\ouritem{socket\_recv(+Sockfd,-Message, ?ErrorCode)}\index{\texttt{socket\_recv/3}}
    Receives a message from the connection identified by the socket descriptor
    {\tt Sockfd}. Binds {\tt Message} to the message. {\tt socket\_recv/3}
    provides a message-oriented interface. It understands message
    boundaries set by {\tt socket\_send/3}.

\ouritem{socket\_send(+Sockfd,+Message, ?ErrorCode)}\index{\texttt{socket\_send/3}}
    Takes a message (which must be an atom) and sends it through the
    connection specified by {\tt Sockfd}. {\tt socket\_send/3} provides
    message-oriented communication. It prepends a 4-byte header to the
    message, which tells {\tt socket\_recv/3} the length of the message body.

\end{description}

\paragraph{Stream-oriented, character-at-a-time interface.}
Internally, this interface uses the same {\tt sendto} and {\tt recvfrom}
socket calls, but they are executed for each character separately.
This interface is appropriate when the message format is not known or when
message boundaries are determined using special delimiters.

{\tt socket\_get0/3} creates the end-of-file condition when it receives the
end-of-file character {\tt CH\_EOF\_P} (a.k.a. 255) defined in {\tt
  char\_defs.h} (which must be included in the XSB program). C programs
that need to send an end-of-file character should send {\tt (char)-1}.
%%
\begin{description}
\ouritem{socket\_get0(+Sockfd, -Char, ?ErrorCode)}\index{\texttt{socket\_get0/3}}
The equivalent of {\tt get0} for sockets.

\ouritem{socket\_put(+Sockfd, +Char, ?ErrorCode)}\index{\texttt{socket\_put/3}}
Similar to put/1, but works on sockets.
\end{description}
%%

\paragraph{Socket-probing.}
With the help of the predicate {\tt socket\_select/6} one can establish a
group of asynchronous or synchronous socket connections. In the synchronous
mode, this call is blocked until one of the sockets in the group becomes
available for reading or writing, as described below.  In the asynchronous
mode, this call is used to probe the sockets periodically, to find out
which sockets have data available for reading or which sockets have room in
the buffer to write to.

The directory {\tt XSB/examples/socket/select/} has a number of examples of
the use of the socket-probing calls.
%%
\begin{description}
  \ouritem{socket\_select(+SymConName,+Timeout,-ReadSockL,-WriteSockL,-ErrSockL,?ErrorCode)} \index{\texttt{socket\_select/6}}
  {\tt SymConName} must be an atom that
  denotes an existing connection group, which must be previously created with
  {\tt socket\_set\_select/4} (described below). {\tt ReadSockL}, {\tt
    WriteSockL}, {\tt ErrSockL} are lists of socket handles (as returned by
  {\tt socket/2}) that specify the available sockets that are available for
  reading, writing, or on which exception conditions occurred.  {\tt
    Timeout} must be an integer that specifies the timeout in seconds (0
  means probe and exit immediately). If {\tt Timeout} is a variable, then
  wait indefinitely until one of the sockets becomes available.

\ouritem{socket\_set\_select(+SymConName,+ReadSockFdLst,+WriteSockFdLst,+ErrorSockFdLst)} \index{\texttt{socket\_set\_select/4}}
Creates a connection group with the symbolic name {\tt SymConName}
(an atom) for subsequent use by {\tt socket\_select/6}.
{\tt ReadSockFdLst}, {\tt WriteSockFdLst}, and {\tt ErrorSockFdLst} are
lists of sockets for which {\tt socket\_select/6} will be used to monitor read,
write, or exception conditions.

\ouritem{socket\_select\_destroy(+SymConName)} \index{\texttt{socket\_select\_destroy/1}}
Destroys the specified connection group.

\end{description}


\paragraph{Error codes.}
The error code argument unifies with the error code returned by the
corresponding socket commands. The error code -2 signifies
\emph{timeout} for timeout-enabled primitives (see below). The error code
of zero signifies normal termination. Positive error codes denote specific
failures, as defined in BSD sockets. When such a failure occurs, an error
message is printed, but the predicate succeeds anyway. The specific error
codes are part of the socket documentation. Unfortunately, the symbolic
names and error numbers of these failures are different between Unix
compilers and Visual C++. Thus, there is no portable, reliable way to refer
to these error codes. The only reliably portable error codes that can be 
used in XSB programs defined through these symbolic constants:
%%
\begin{verbatim}
#include "socket_defs_xsb.h"  

#define SOCK_OK       0      /* indicates sucessful return from socket      */
#define SOCK_EOF     -1      /* end of file in socket_recv, socket_get0     */

#include "timer_defs_xsb.h"

#define TIMEOUT_ERR -2                 /* Timeout error code */
\end{verbatim}
%%

\paragraph{Timeouts.}
XSB socket interface allows the programer to specify timeouts for certain
operations. If the operations does not finish within the specified period
of time, the operation is aborted and the corresponding predicate succeeds
with the {\tt TIMEOUT\_ERR} error code. The following primitives are
timeout-enabled: {\tt socket\_connect/4}, {\tt socket\_accept/3}, {\tt
  socket\_recv/3}, {\tt socket\_send/3}, {\tt socket\_get0/3}, and {\tt
  socket\_put/3}.  To set a timeout value for any of the above primitives,
the user should execute {\tt set\_timer/1} right before the subgoal to be
timed. Note that timeouts are disabled after the corresponding timeout-enabled
call completes or times out. Therefore, one must use {\tt set\_timer/1}
before each call that needs to be controlled by a timeout mechanism.

The most common use of timeouts is to either abort or retry the operation
that times out. For the latter, XSB provides the {\tt sleep/1} primitive,
which allows the program to wait for a few seconds before retrying.

The {\tt set\_timer/1} and {\tt sleep/1} primitives are described below.
They are standard predicates and do not need to be explicitly imported.
%%
\begin{description}
\ouritem{set\_timer(+Seconds)}\index{\texttt{set\_timer/1}}
Set timeout value. If a timer-enabled goal executes after this value is
set, the clock begins ticking. If the goal does not finish in time, it
succeeds with the error code set to {\tt TIMEOUT\_ERR}. The timer is turned
off after the goal executes (whether timed out or not and whether it
succeeds or fails). This goal always succeeds.

Note that if the timer is not set, the timer-enabled goals execute
``normally,'' without timeouts. In particular, they might block (say, on
{\tt socket\_recv}, if data is not available).

\ouritem{sleep(+Seconds)}\index{\texttt{sleep/1}}
Put XSB to sleep for the specified number of seconds. Execution resumes
after the {\tt Seconds} number of seconds. This goal always succeeds.
\end{description}
%%
Here is an example of the use of the timer:
%%
\begin{samepage}
\begin{verbatim}
:- compiler_options([xpp_on]).
#include "timer_defs_xsb.h"

?- set_timer(3),  % wait for 3 secs
   socket_recv(Sockfd, Msg, ErrorCode),
   (ErrorCode == TIMEOUT_ERR
   -> writeln('Socket read timed out, retrying'),
      try_again(Sockfd)
   ;  write('Data received: '), writeln(Msg)
   ).
\end{verbatim}
\end{samepage}
%%

\noindent
Apart from the above timer-enabled primitives, a timeout value can be given
to {\tt socket\_select/6} directly, as an argument.


\paragraph{Buffered, stream-oriented communication.}
In Unix, socket descriptors can be ``promoted'' to file streams and the
regular read/write commands can be used with such streams. In XSB, such
promotion can be done using the following predicate:
%%
\begin{description}
\ournewitem{fd2ioport(+Pipe, -IOport)}{shell}\index{\texttt{fd2ioport/2}} 
    Take a socket descriptor and convert it to an XSB I/O port that can be used
    for regular file I/O. 
\end{description}
%%
Once {\tt IOport} is obtained, all I/O primitives described in
Section~\ref{sec-low-level-io} can be used. This is, perhaps, the easiest
and the most convenient way to use sockets in XSB. (This feature has not
been implemented for Windows.)

\noindent
Here is an example of the use of this feature:
%%
\begin{samepage}
\begin{verbatim}
:- compiler_options([xpp_on]).
#include "socket_defs_xsb.h"

?- (socket(Sockfd, SOCK_OK)
   ->   socket_connect(Sockfd1, 6020, localhost, Ecode),
        (Ecode == SOCK_OK
        -> fd2ioport(Sockfd, SockIOport),
           file_write(SockIOport, 'Hello Server!')
        ;  writeln('Can''t connect to server')
        ),
    ;   writeln('Can''t open socket'), fail
    ).
\end{verbatim}
\end{samepage}
%%


\section{Arrays}
%===============

The module {\tt array1} in directory {\tt lib} provides a very simple 
backtrackable array implementation.  The predicates through which the 
array objects are manipulated are:

\begin{description}
\ournewitem{array\_new(-Array, +Size)}{array1}\index{\texttt{array\_new/2}}
%\predindex{array\_new/2~(L)}
    Creates a one dimensional empty array of size {\tt Size}.  All the 
    elements of this array are variables.
\ournewitem{array\_elt(+Array, +Index, ?Element)}{array1}
\index{\texttt{array\_elt/3}}
%\predindex{array\_elt/3~(L)}
    True iff {\tt Element} is the {\tt Index}-th element of array 
    {\tt Array}.
\ournewitem{array\_update(+Array, +Index, +Elem, -NewArray)}{array1}
\index{\texttt{array\_update/4}}
%\predindex{array\_update/4~(L)}
    Updates the array {\tt Array} such that the {\tt Index}-th element
    of the new array is {\tt Elem} and returns the new array in 
    {\tt NewArray}.  The implementation is quite efficient in that it 
    avoids the copying of the entire array.
\end{description}

A small example that shows the use of these predicates is the following:
{\footnotesize
 \begin{verbatim}
           | ?- import [array_new/2, array_elt/3, array_update/4] from array1.

           yes
           | ?- array_new(A, 4), array_update(A,1,1,B), array_update(B,2,2,C),
                ( array_update(C,3,3,D), array_elt(D,3,E)
                ; array_update(C,3,6,D), array_elt(D,3,E)
                ; array_update(C,3,7,D), array_elt(D,3,E)
                ).

           A = array(1,2,3,_874600)
           B = array(1,2,3,_874600)
           C = array(1,2,3,_874600)
           D = array(1,2,3,_874600)
           E = 3;

           A = array(1,2,6,_874600)
           B = array(1,2,6,_874600)
           C = array(1,2,6,_874600)
           D = array(1,2,6,_874600)
           E = 6;

           A = array(1,2,7,_874600)
           B = array(1,2,7,_874600)
           C = array(1,2,7,_874600)
           D = array(1,2,7,_874600)
           E = 7;

           no
 \end{verbatim}
}


\section{The Profiling Library}
%==============================

XSB can provide Prolog-level profiling for Prolog programs, which
allows the Prolog programmer to determine what proportion of time is
spent executing code for each predicate.  To enable profiling, XSB
must be started with the command line parameter of -p.  The module
{\tt xsb\_profiling} contains the predicate {\tt profile\_call/1} that
invokes profiling.

\begin{description}
\ournewitem{profile\_call(+Goal)}{xsb\_profiling}\index{profile\_call/1}

Calls the goal {\tt Goal}, and when it first returns, prints out (on
userout) a table of predicate names indicating for each, the
percentage of time spent executing that predicate's code.  XSB must
have been started with the -p command line parameter to obtain this
table.  The percentages are accumulated by module.  {\tt Goal} may
backtrack, but profiling is done only for the time to the first
success.  So it is most appropriate to profile succeeding
deterministic goals.
\end{description}

Profiling works by starting another thread that interrupts every 100th
of a second and sets a flag so that the XSB emulator will determine
the predicate of the currently executing code.  The printout also
includes the total number of interrupts and for each predicate, the
raw number of times its code was determined to be executing.  A
predicate is printed only if its code was interrupted at least once.
The numbers will be meaningful only for relatively long-running
predicates, taking more than a couple of seconds.

When an interrupt occurs, the {\bf next} WAM call, execute, proceed or
trust instruction to be executed will log its associated predicate.
The system does not keep track of code addresses for tries (used to
represent the results of completed tables, and trie-indexed asserted
code), so for some interrupts the associated executing predicate
cannot be determined.  In these cases the interrupt is charged against
an ``unknown/?'' pseudo-predicate, and this count is included in the
output.

Profiling does not give the context from which the predicate is
called, so you may want to make renamed copies of basic predicates to
use in particular circumstances to determine their times.

Predicates compiled with the ``optimize'' option may provide
misleading results under profiling.  Note that all system predicates
(including those in {\tt basics}) are compiled with the ``optimize''
option, by default.  That option causes tail-recursive predicates to
use a ``jump'' instruction rather than an ``execute'' instruction to
make the recursive call, and so an interrupt in such a loop will not
be charged until the next {\bf logging} instruction is executed.  If
much time is spent in the recursion, this might not be for a long
time, and the interrupt might ultimately be charged to another
predicate.  (If an interrupt has not been charged by the time of the
next interrupt, it is lost.)

Profiling currently works under Windows, and somewhat under Linux.
For the profiling algorithm to work, the thread that wakes and sets
the interrupt flag must be of high priority and given the CPU when it
wants it.  I haven't figured out how to get this scheduling on some
machines, so if you want profiling to work somewhere it doesn't, maybe
you can help figure out how to get appropriate scheduling.

%----------------------------------------------------------------------
\comment{
%TLS: finally took this out...
%\input{tr_assert}
\section{Asserts/Retracts using Tries }

\index{\texttt{trie\_assert/1}}
\index{\texttt{trie\_retract/1}}
\index{\texttt{trie\_retractall/1}}
\index{\texttt{trie\_retract\_nr/1}}
\index{\texttt{abolish\_trie\_asserted/1}}
\index{\texttt{trie\_dynamic/1}}

In \version, trie asserted code has been merged with standard asserted
code.  If the user wishes to use tries for dynamic code, the
recommended programming practice is as outlined in the section {\it
Modification of the Database} in Volume 1.  For compatibility with
previous versions, the predicates {\tt trie\_assert/1}, {\tt
trie\_retract/1}, {\tt trie\_retractall/1}, {\tt trie\_retract\_nr/1},
{\tt abolish\_trie\_asserted/1} and {\tt trie\_dynamic/1} can be
imported from the module {\sf tables}.  However, if the current index
specification of these predicates is {\tt trie} (again, see the
section {\it Modification of the Database} in Volume 1, the predicates
are defined as {\tt assert/1}, {\tt retract/1}, {\tt retractall/1},
{\tt retract\_nr/1}, {\tt abolish/1} and {\tt dynamic/1} respectively.
If the index specification is other than {\tt tries}, the predicate
will issue a warning message and have no effect on the database.
}

\section{Low-level Trie Manipulation Utilities}

The following utilities are used to implement assert and retract, but they
can also be used to implement special purpose operations, like
backtrackable assert and retract. All the utilities in this section are
very low-level and require good understanding of the trie mechanism in XSB.

The module {\tt storage} described in Volume 1 provides a higher-level and
a more convenient interface to the XSB trie-based storage mechanism.

%%
\begin{description}
  %%
\ournewitem{newtrie(-Root)}{intern} \index{\texttt{newtrie/1}}
        Root is instantiated to a handle for a new trie.

\ournewitem{trie\_intern(+Term,+Root,-Leaf,-Flag,-Skel)}{intern}\index{\texttt{trie\_intern/5}}
%%
{\tt Term} is the Prolog term to be interned. {\tt Root} is the handle for
a trie. {\tt Leaf} is the handle for the interned {\tt Term} in the trie.
{\tt Flag} is 1 if the term is ``old'' (already exists in the trie); it is
0, if the term is newly inserted.  {\tt Skel} represents the collection of
all the variables in {\tt Term}. It has the form ret(V1,V2,...,VN), exactly
as in {\tt get\_calls}.


\ournewitem{trie\_intern(+Term,-Leaf,-Skel)}{intern}\index{\texttt{trie\_intern/3}}
%%
Interns {\tt Term} into the default trie. Does not return the new/old flag.


\ournewitem{trie\_interned(?Term,+Root,?Leaf,-Skel)}{intern}\index{\texttt{trie\_interned/4}}
%%
This predicate will backtrack through the terms interned into the trie
represented by the handle {\tt Root} if {\tt Leaf} is a free variable.
Otherwise, if {\tt Leaf} is bound, it will backtrack over the terms in
the trie that unify with the term pointed to by Leaf {\tt to}.  {\tt
Term} is the term to be retrieved; it can be either (partially) bound
or free.  {\tt Skel} is the collection of all the variables in {\tt
Term}; it has the form {\tt ret(V1,...,Vn)}.

\ournewitem{trie\_interned(?Term,?Leaf,-Skel)}{intern} \index{\texttt{trie\_interned/3}}
%%
Similar to {\tt trie\_interned/4}  but uses the default trie.


\ournewitem{trie\_unintern(+Root,+Leaf)}{intern} \index{\texttt{trie\_unintern/2}}
%%
Uninterns (deletes) a term from the trie indicated by root.  This predicate
has to be called with care. Uninterning can be done only when the trie from
which the term is being uninterned is not being actively accessed.

\ournewitem{trie\_unintern\_nr(+Root,+Leaf)}{intern}\index{\texttt{trie\_unintern\_nr/2}}
%%
This is a safe version of {\tt trie\_unintern/2}. The term pointed to by
{\tt Leaf} is marked as deleted, but is not deleted from the trie. This
permits an efficient implementation of backtrackable updates.

\ournewitem{unmark\_uninterned\_nr(+Root,+Leaf)}{intern}\index{\texttt{unmark\_uninterned\_nr/2}}
The term pointed to by {\tt Leaf} should have been previously marked for
deletion using
{\tt trie\_unintern\_nr/2}. This term is then ``unmarked'' (or undeleted)
and becomes again a notmal interned term.

\ournewitem{reclaim\_uninterned\_nr(+Root)}{intern}
\index{\texttt{reclaim\_uninterned\_rn/1}}
%%
Not yet implemented.\\
Runs through the chain of leaves of the trie {\tt Root} and
deletes the terms that have been marked for deletion by
{\tt trie\_unintern\_nr/2}. This is a garbage collection step that should
be done just before returning to the top level.

\ournewitem{delete\_trie(+Root)}{intern} \index{\texttt{delete\_trie/1}}
%%
Deletes all the terms in the trie pointed to by Root.


%%
\end{description}

\section{Gensym}
%================================

The Gensym library provides a convenient way to generate unique
integers or constants.  

\begin{description}
\ournewitem{prepare(+Index)}{gensym} \index{\texttt{prepare/1}}
%
Sets the initial integer to be used for generation to {\tt Index}.
Thus, the command {\tt ?- prepare(0)} would cause the first call to
{\tt gennum/1} to return {\tt 1}.  {\tt Index} must be a non-negative
integer.

\ournewitem{gennum(-Var)}{gensym} \index{\texttt{gennum/1}}
%
Unifies {\tt Var} with a new integer.

\ournewitem{gensym(+Atom,-Var)}{gensym} \index{\texttt{gensym/2}}
%
Generates a new integer, and concatenates this integer with {\tt
Atom}, unifying the result with {\tt Var}.  For instance a call {\tt
?- gensym(foo,Var)} might unify {\tt Var} with {\tt foo32}.
\end{description}

\section{Random Number Generator}
%================================

The following predicates are provided in module \texttt{random} to
generate random numbers (both integers and floating numbers), based on
the Wichmann-Hill Algorithm \cite{WicH82,McLe85}.  The random number
generator is entirely portable, and does not require any calls to the
operating system.  As noted below, it does require 3 seeds, each of
which must be an integer in a given range.


\begin{description}

\ournewitem{random(-Number)}{random} \index{\texttt{random/1}}
%
Binds \texttt{Number} to a random float in the interval [0.0, 1.0).
Note that 1.0 will never be generated.

\ournewitem{random(+Lower,+Upper,-Number)}{random} \index{\texttt{random/3}}
    Binds \texttt{Number} to a random integer in the interval
    [\texttt{Lower},\texttt{Upper}) if \texttt{Lower} and \texttt{Upper}
    are integers.  Otherwise \texttt{Number} is bound to a random float
    between \texttt{Lower} and \texttt{Upper}.  \texttt{Upper} will
    never be generated.

\ournewitem{genrand(?State)}{random} \index{\texttt{genrand/1}}
    Tries to unify \texttt{State} with the term \texttt{rand(X,Y,Z)}
    where \texttt{X},\texttt{Y},and \texttt{Z} are integers describing
    the state of the random generator.

\ournewitem{setrand(rand(+X,+Y,+Z))}{random} \index{\texttt{setrand/1}}
    Sets the state of the random generator.  \texttt{X},\texttt{Y}, and
    \texttt{Z} must be integers in the ranges [1,30269), [1,30307),
    [1,30323), respectively.

\ournewitem{randseq(+K, +N, -RandomSeq)}{random} \index{\texttt{randseq/3}}
    Generates a sequence of \texttt{K} unique integers chosen randomly
    in the range from 1 to \texttt{N}.  \texttt{RandomSeq} is not
    returned in any particular order.

\ournewitem{randset(+K, +N, -RandomSet)}{random} \index{\texttt{randset/3}}
    Generates an ordered set of \texttt{K} unique integers chosen
    randomly in the range from 1 to \texttt{N}.  The set is returned in
    reversed order, with the largest element first and the smallest
    last.

\end{description}

\section{Scanning in Prolog}

Scanners, (sometimes called tokenizers) take an input string, usually
in ASCII or similar format, and produce a scanned sequence of tokens.
The requirements that various applications have for scanning differ in
small but important ways -- a character that is special to one
application may be part of the token of another; or some applications
may want lower case text converted to upper-case test.  The {\tt
stdscan.P} library provides a simple scanner written in XSB that can
be configured in several ways.  While useful, this scanner is not
intended to be as powerful as general-purpose scanners such as {\em
lex} or {\em flex}.

\begin{description}

\ournewitem{scan(+List,-Tokens)}{stdscan} \index{\texttt{scan/2}}  

Given as input a {\tt List} of character codes, {\tt scan/2} scans
this list producing a list of atoms constituting the lexical tokens.
Its parameters are set via {\tt set\_scan\_pars/1}.

Tokens produced are either a sequence of {\em letters} and/or {\em
numbers} or consist of a single {\em special character} (e.g. {\tt (}
or {\tt )}).  Whitespaces may occur between tokens.

\ournewitem{scan(+List,+FieldSeparator,-Tokens)}{stdscan} 
\index{\texttt{scan/3}}  

Given as input a {\tt List} of character codes, along with a character
code for a field separator, {\tt scan/3} scans this list producing a
list of list of atoms constituting the lexical tokens in each field.
{\em scan/3} thus can be used to scan tabular information.  Its
parameters are set via {\tt set\_scan\_pars/1}.

\ournewitem{set\_scan\_pars(+List)}{stdscan} 
\index{\texttt{set\_scan\_pars/1}}  

{\tt set\_scan\_pars(+List)} is used to configure the tokenizer to a
particular need.  {\tt List} is a list of parameters including the
following:

\begin{itemize}

\item{{\tt whitespace}}.  The default action of the scanner is to return a
list of tokens, with any whitespace removed.  If {\tt whitespace} is a
parameter, then the scanner returns the token {\tt ''} when it finds
whitespace separating two tokens (unless the two tokens are letter
sequences; since two letter sequences can be two tokens ONLY if they
are separated by whitespace, such an indication of whitespace would be
redundant.)  Including the parameter {\tt no\_whitespace} undoes the
effect of previously including {\tt whitespace}.

\item{{\tt upper\_case}} The default action of the parser is to treat
lowercase letter differently from uppercase letters.  This parameter
should be set if conversion to uppercase should be done when producing
a token that does {\em not} consist entirely of letters (e.g. one with
mixed letters and digits).  Including the parameter {\tt no\_case}
undoes the effect of previously including {\tt upper\_case}.

\item{{\tt upper\_case\_in\_lit}} The default action of the parser is
to treat lowercase letter differently from uppercase letters.  This
parameter should be set if conversion to uppercase should be done when
producing a token that consists entirely of letters.  Including the
parameter {\tt no\_case\_in\_lit} undoes the effect of previously
including {\tt upper\_case}.

\item{{\tt whitespace(Code)}} adds {\tt Code} as a whitespace code.
By default, all ASCII codes less than or equal to {\tt 32} are
regarded as whitespace.

\item{{\tt letter(Code)}} adds {\tt Code} as a letter constituting a
token. By default, ASCII codes for characters {\tt a--z} and {\tt
A--Z} are regarded as letters.

\item{{\tt special\_char(Code)}} adds {\tt Code} as a special
  character.  By default, ASCII codes for the following characters are
  regarded as special characters:

\begin{verbatim}
| { } [ ] " $ % & ' ( ) * + , - . / : ; < = > ? @ \ ^ _ ~ `
\end{verbatim}
\end{itemize}

\ournewitem{get\_scan\_pars(-List)}{stdscan} 
\index{\texttt{get\_scan\_pars/1}}  
{\tt get\_scan\_pars/1} returns a list of the currently active
parameters.

\end{description} 

\section{XSB Lint}
%================================

The {\tt xsb\_lint\_impexp.P} file contains a simple tool to analyze
import/exports and definitions and uses of predicates.  It tries to
find possible inconsistencies, producing warnings when it finds them
and generating {\tt document\_import}/{\tt document\_export}
declarations that might be useful.  It can be used after a large
multi-file, multi-module XSB program has been written to find possible
inconsistencies in (or interesting aspects of) how predicates are
defined and used.

XSB source files that contain an {\tt export} compiler directive are
considered as modules.  Predicates defined in modules, but not
exported, are local to that module.  When compiling a module, the XSB
compiler generates useful warnings when predicates are used but not
defined or defined but not used.  All predicates that are defined in
source files that do not contain an {\tt export} directive are
compiled to be defined in a global module, called {\tt usermod}, and
no warning messages are generated.  The user may add {\tt
document\_export} and {\tt document\_import} compiler directives
(exactly analogous to the {\tt export} and {\tt import} directives) to
non-module source files.  These directives are ignored by the compiler
for its compilation, but cause the define-use analysis to be done and
any warning messages to be issued, if appropriate.  This allows a user
to get the benefit of the define-use analysis without using modules.
(See Volume 1, Chapter 3 for more details.)

The {\tt xsb\_lint\_impexp} utility processes both modules and regular
XSB source files that contain {\tt document\_export} statements.
%
{\tt xsb\_lint\_impexp} is not itself a module.  To use it, {\tt
[xsb\_lint\_impexp]} must be consulted, which will define the {\tt
checkImpExps/\{1,2\}} and {\tt add\_libraries/1} predicates in {\tt
usermod}.

\begin{description}
\ouritem{add\_libraries(+DirectoryNameList)}
%
{\tt add\_libraries/1} takes a list of directory names and adds them
to the {\tt library\_directory/1} predicate.  This causes the XSB
system to look for XSB source code files in these directories.  To use
{\tt checkImpExps/\{1,2\}}, all the directories that contain files (or
modules) referenced (recursively) in the files to be processed must be
in the {\tt library\_directory/1} predicate.  This predicate can be
used to add a number of directories at once.

\ouritem{checkImpExps(+Options,+FileNameList)}
%
{\tt checkImpExps/1} reads all the XSB source files named in the list
{\tt FileNameList}, and all files they reference (recursively), and
produces a listing that describes properties of how they reference
predicates.

{\tt Options} is a list of atoms (from the following list) indicating
details of how {\tt checkImpExps} should work.

\begin{enumerate}
\item
   {\tt used\_elsewhere}: Print a warning message in the case of a
   predicate defined in a file, not used there, but used elsewhere (in
   a file in {\tt FileNameList}). This can be useful to see whether it
   might be better to move the predicate definition to another file,
   but it produces many warnings for predicates in multi-use
   libraries.
\item
   {\tt unused}: Print a warning message in the case of a predicate that is
   exported but never used.  This can be useful to see if predicate is
   not used anywhere, and thus could be deleted.  Again this produces
   many warnings for predicates in multi-use libraries.
\item
   {\tt all\_files}: By default, only predicates in files that contain a {\tt
   :- document\_export} or {\tt :- export} declaration are processed.
   This option causes predicates of {\em all} files (and modules) to be
   processed.
\item
   {\tt all\_symbol\_uses}: Treat {\em all} non-predicate uses of symbols
   (even constants) as predicate uses for the purpose of generating
   imports.
\item
   {\tt no\_symbol\_uses}: Don't treat any non-predicate uses of symbols as
   predicate uses for the purpose of generating imports.
\end{enumerate}

We further explain the final two options, which allow the user to
determine more precisely what uses of a symbol are considered as uses
of it as the predicate symbol.  All uses of symbols that appear in a
``predicate context", i.e., in the body of a rule or in a
meta-predicate argument position of a use of a meta-predicate, are
considered uses of that predicate symbol.  The default is also to
allow nonconstant symbols appearing in any other context to also count
as uses of that symbol as that predicate symbol.  This is useful for
programs that define their own meta-predicates.

\ouritem{checkImpExps(+FileNameList)}
%
{\tt checkImpExps/1}  is
currently equivalent to {\tt checkImpExps([],FileNameList)}.
\end{description}

\section{Pretty Printing}

XSB contains a simple but useful pretty-printer~\footnote{This library
uses code both from hProlog and Ciao.}.

\begin{description}

\ournewitem{pretty\_print(+ClausePairs)}{pretty\_print}
\index{\texttt{pretty\_print/1}}
%
The input to {\tt pretty\_print/1}, {\tt ClausePairs}, can be either a
list of clause pairs or a single clause pair.  A clause pair is either
a Prolog clause (or declaration) or a pair:
%
\begin{center}
{\tt (Clause,Dict)} 
\end{center}
%
Where {\tt Dict} is a list of the form {\tt A = V} where {\tt V} is a
variable in {\tt Clause} and {\tt A} is the string to be used to
denote the variable \footnote{Thus the list of variable names returned
by {\tt read\_term/\{2,3\}} can be used directly in {\tt Dict}.}.

By default, {\tt pretty\_print/1} outputs atomic terms using {\tt
writeq/1}, but specialized output can be configured via asserting in
{\tt usermod} a term of the form
%
\begin{center}
{\tt user\_replacement\_hook(Term,Call)}
\end{center}
%
which will use Call to output an atomic literal $A$ whenever $A$
unifies with {\tt Term}.  For example, pretty printing weight
constraints in XSB's {\tt XASP} package is done via the hook
%
\begin{center}
{\tt user\_replacement\_hook(weight\_constr(Term),output\_weight\_constr(Term))}
\end{center}
%
which outputs a weight constraint in a (non-Prolog) syntax that is 
used by several ASP systems.

\end{description}

\section{Other Libraries}
%================================

Not all XSB libraries are currently documented.  We provide brief
summaries of some of these other libraries.

\subsection{AVL Trees}

AVL trees provide a mechanism to maintain key value pairs so that loop
up, insertion, and deletion all have complexity ${\cal O}(\log n)$.  This
library contains predicates to transform a sorted list to an AVL tree
and back, along with predicates to manipulate the AVL trees.  This
library was originally written for Sicstus by Mats Carlsson.

% Generic justifier for XSB programs; allows justification of T/F/U values
% NOTE : Any program predicate that depends recursively 
%        on itself through negation must be tabled.
%        This ensures that the delay list of any undefined 
%        atom can be accessed (say via get_residual/2).


\subsection{Justification}

Most Prolog debuggers, including XSB's, are based on a mechanism that
allows a user to trace the evaluation of a goal by interrupting the
evaluation at call, success, retry, or failure of various subgoals.
While this has proved an excellent mechanism for evaluating SLD(NF)
executions, it is difficult at best to use such a mechanism during a
tabled evaluation.  This is because, unlike with SLD(NF), SLG requires
answers to be returned to tabled subgoals at various times (depending
on whether batched or local evaluation is used), negative subgoals to
be sometimes be delayed and/or simplified, etc.

One approach to understanding tabled evaluation better is to abstract
away the procedural aspects of debugging and to use the tables
produced by an evaluation to construct a {\em justification} after the
evaluation has finished.  The justification library does just this
using algorithms described in \cite{GuRR01}.

\subsection{Ordered Sets}

{\tt ordset.P} provides an XSB port of the widely used ordset library,
written by Richard O'Keefe, whose summary we paraphrase here.  In the
ordset library, sets are represented by ordered lists with no
duplicates.  Thus {\em \{c,r,a,f,t\}} is represented as {\tt
[a,c,f,r,t]}.  The ordering is defined by the \verb|@<| family of term
comparison predicates, which is the ordering used by {\tt sort/2} and
{\tt setof/3}.  The benefit of the ordered representation is that the
elementary set operations can be done in time proportional to the sum
of the argument sizes rather than their product.  Some of the
unordered set routines, such as {\tt member/2}, {\tt length/2}, or
{\tt select/3} can be used unchanged.


%%% Local Variables: %%% mode: latex %%% TeX-master: "manual2" %%% End:
