![]() |
|
Published 1999-07-06 Printer-friendly version
(Part 3 of 3. Click to read Part 1 and Part 2)
Welcome to the third and final installment in the ABC FileManager Design series. In this article I'll get down to the meat of the class; the dictionary interaction and the file access itself. You may want to begin by reading the first and second installments if you haven't already done so.
Whilst initialisation and validation are logically distinct tasks they are grouped together here because in the fullness of time they will form the basis of what is really one concept: business rules. People can use business rules to mean just about anything they like. I use it to mean information about the data that is not contained explicitly within the data.
Some of these rules are already handled by the file driver. For example the DUP attribute in the Clarion language (upon a key) specifies something about the data you cannot get directly from the data (although you may be able to intuit it). The Clarion dictionary gains some of its power by specifying initialisation values and validation values within a single repository. In Clarion 2.003 this information was then scattered (at code generation time) throughout the application. Any form upon a file would contain code to initialise and then validate the record buffer. In ABC this code is contained in the (derived) FileManager object.
One of the ABC aims is to keep the initialisation and validation information in one place so that if the rule is dynamic (i.e. has to be done with an embed point) then that embed only needs to be placed once and all accesses to the data obey the new rule. This is particularly important as ABC was to have edit-in-place browses and automatic updates from drop-combos. In other words, we could no longer rely on forms being around to act as guardians of the file.
A consequence of the heavy dictionary tie to these routines and the need to pander to embed code is that a number of these methods are blank; they are placeholders for template generated code. In these instances I will describe the code that I envisage will go into the derived form of these methods (and which, purely coincidentally, the templates generate.)
CancelAutoInc PROCEDURE(<RelationManager RM>),VIRTUAL,BYTE,PROC
Initialising a
record with an autoincrement key results (in our implementation) in
a record being stored on disk to act as a placeholder for the
autoincrement value. If you cancel the operation then that record
needs to be deleted. The CancelAutoInc is the clue to
the FileManager that the record that
was initialised is about to be thrown away. The RelationManager parameter is a
solution to the problem often referred to as the "orphaned children
problem." In a 2.003 application if you insert a record with an
autoincrement key, go to the child tab, insert some child records
and then cancel the form, the child records persist. Of course you
can't see them until you insert a new record and find it
automatically gains some children! This is really scary on forms
with many children where the child tabs may not even be perused
during the insert. ABC gets around this problem in that the form
procedure passes the RelationManager in to the CancelAutoInc
and the CancelAutoInc undertakes to delete
any children (or refuse to cancel the autoincrement) as
appropriate.
The
implementation is much simpler than the description: if an
autoincrement has happened then either DELETE or the RI DELETE are called. In the
latter case the response is noted, as an RI relation of restrict
means that the autoincrement cannot be cancelled whilst the
children persist.
PrimeAutoIncServer
PROCEDURE(BYTE HandleErrors),BYTE,PROC,PRIVATE, PrimeAutoInc and TryPrimeAutoInc
are really just
two virtual hooks on the AutoIncServer. Non-zero for HandleErrors implies the server
will take all possible steps to ensure the autoincrement happens.
The TryPrimeAutoInc only is called
when TryInsert has been called. This
is never done with the shipping templates but has been added at the
request of a third party.
The routine
starts by checking the guard variables. PrimeAutoInc can be called multiple
times to allow for the cases where priming is done in the browse or
in the form. It also allows for inserts to be done without
pre-priming of the record. This is an instance of what I call
"objects with attitude." Basically the FileManager knows that priming
should be done once, and only once, and thus it does it at the
first opportunity it is asked, or at the last minute if no-one asks
it!
The algorithm for autoincrementing is essentially the one used to return a unique handle in SaveBuffer, find the last element and add on one. However, as you shall see, some of the little details make things a vast amount more complicated.
The
first LOOP is a loop to allow for
multiple reruns of the bulk of the procedure in the case where an
attempt to autoincrement failed. The failure is most likely to be
because between the reading (from disk) of the current highest
element, adding on one, and then ADDing the record, another
station got there first and ADDed the record with that
number! The simplest solution to this failure is therefore to try
again from the beginning, and the outer loop encodes that logic. Of
course you can't keep doing that for ever because something
may really be wrong. If you go to the end of the outer loop, after
the ADD(File) you will see logic to
trap the error.
If this is
the third failure the error manager is invoked to see if the user
wants to try again. If he does you simply try again (three times),
and if he doesn't you break out with a failure. If the add was
successful the method simply notes the autoincrement has been done
and then returns Level:Benign (which is zero and
means OK).
The second
(or inner) loop introduces a new complexity. Since it is possible
for there to be multiple autoincrement keys in one file, this code
has to find incremented values for each of them. So it loops on
each key, executing the body of the loop if it is autoincrement.
The SaveBuffer is important
because PrimeAutoInc should only change
those fields which are tagged in the dictionary as autoincrement
(or it won't be possible to support delaying the autoinc
allocation until the insert point). The code then splits into two
branches, the relatively easy one where the key has one component,
and the multi-component case.
One component
is handled by fetching the component any variable and assigning
to AutoIncField. The file is SET to key order and the
final record (or first for a descending key) is fetched. If no
record is found (i.e. the file is empty) the new autoinc value is
one, else it is the highest value plus one. It's necessary to
use AutoIncField/ AutoValue rather than just using
the underlying fields as the object will later restore the buffer
(which will corrupt the field values) and then perform the
autoincrement assignment.
The multi-component case uses exactly the same algorithm. The complexity is in finding the "final" record because you don't want the final record, you want the final record that matches the current record buffer in all components except the last.
ConcatGetComponents
is a simple (and ugly) way of snapshotting the leading components of a key to later
see if they have changed. The method then clears the minor-most
(and thus autoincrement) key high and does a SET(K,K)
followed by a Previous or Next as before. In the NoErrorcase it has to
check that the record fetched did match in the leading
components; if it did then it can use the AI value fetched and add
on one, otherwise it knows that no records currently match the
major components and thus can use one.
Having
computed the new field value (using either method) it restores the
buffer contents and then assigns the new autoincrement field value
into place. Once that is done that for all autoincrement keys it
can try ADDing the new
record.
If this
procedure looks long and horrible it is because it is. My general
rule of thumb is that any procedure more than a page long is a bug.
Again you can see engineering and efficiency overcoming science. I
could remove the "OneComponent" arm of the IF, and the only effect would
be a few more string compares (no big deal), but this also changes
this code:
CLEAR(OnlyKeyComponent) SET(K,K) PREVIOUS(K)
into this:
SET(K) PREVIOUS(K)
When you consider that the one component case is the standard third normal form case it was considered that the code verbosity was worth it. That said, the file drivers now spot the above optimisation in most instances so we may be able to simplify this code soon.
PrimeFields
PROCEDURE,PROC,VIRTUAL The default implementation
of this method is blank; in the derived form the templates insert
an assignment for each field that has a non-blank initialisation
value in the dictionary. The PrimeFields routine may not
assume that autoincrement has been done. Any required blanking will
have been performed.
PrimeRecord
PROCEDURE(BYTE SuppressClear = 0),BYTE,PROC,VIRTUAL This method is called
to prime the record whatever that means. In the current
implementation that involves calling PrimeFields to prime the field
values and then forcing the autoincrement to prime. Note that this
method overrides the attitude built into PrimeAutoInc; when PrimeRecordis called a new
autoincrement record will be made. The method has the
facility to clear out any fields it doesn't explicitly prime
or to leave them alone. This functionality is required to allow
the ViewManager to place extra priming
information into the record buffer (such as range-limited
components of keys) and is controlled by the SuppressClear flag.
The primary
case (where AliasedFile &=
NULL) is
quite straightforward. The interest comes when the file is an
alias. Here you don't want to call the priming
functions of the alias (because they don't have the required
embed code); you want to call them in the "real" file. The code for
this has changed in C5EEA; I am describing the new
code.
First the
"real" file has to be opened on this thread (it may not be), and
then the file contents/position have to be snapshotted so that
eventually the file can be restored. Now in the case where the
clear is to be suppressed the code has to assume that the record
(of the alias) contains interesting information that may be
required by the field priming or autoincrement. So it has to get
the information from the alias into the real file. This is done by
the devious device of snap-shotting the alias file buffer and the
restoring from the alias FileManager into the "real" file
buffer. Having done this it can perform the PrimeRecord on the "real" file; if
this is successful it needs to copy the result back to the alias
file, done using the save/restore trick again. Finally the "real"
file is restored to normality and closed. This may look odd: why
restore a file then close it? Simply because Open/Close only
increment/decrement counters. The Open only opens if this is the
first open, similarly the Close only closes if this is the
last close.
ValidateField
PROCEDURE(UNSIGNED Id),BYTE,PROC,VIRTUAL This function returns Level:Benign if the field is ok,
otherwise it returns an error level. The default implementation
only handles the alias case; the actual field validation is handled
in a derived method. The Id is the number that would
come back from a WHERE statement. The template code
simply generates a CASE statement on the field
number, it would be possible to produce a more sophisticated
version using WHAT. We went for simplicity as
this is a very common place to put embed code and also ValidateFieldis hit quite
frequently for control by control field validation and therefore
performance was an issue.
ValidateFields PROCEDURE(UNSIGNED Low,UNSIGNED High,
<*UNSIGNED Failed>),BYTE,PROTECTED,PROC,VIRTUAL
This method is simply an encapsulated way of calling a range of ValidateField
calls. It simply spools over the field numbers contained within the (inclusive)
range. If one fails then the failure number is assigned to Failed.
Again the alias case is handled by re-vectoring through the "real" file. It
should perhaps be noted that for efficiency the alias code does a SaveBuffer,
not SaveFile. This implicitly assumes that the field validation
code will not mess with the current file state (i.e. no file I/O will
be done on the primary file).
ValidateRecord
PROCEDURE(<*UNSIGNED Failed>),BYTE,VIRTUAL Another syntactic
short-hand, this simply calls ValidateFields to ensure that every
field in the record is validated.
These methods
are direct replacements for the equivalents in the file driver.
They are generally there to perform advanced error handling or to
ensure that other FileManager routines are called at
the appropriate moment.
BindFields
PROCEDURE,VIRTUAL This method is called at a
suitable point to bind the fields. By default the code simply
performs a bind on the record buffer. The templates further
override this to perform binds on any memos that are available. The
call can also be overridden by the user (in C5) to bind logical
names (as opposed to labels) as required.
Close
PROCEDURE,BYTE,PROC,VIRTUAL The close mechanism (tied to
the open mechanism) is designed to avoid the needless opening and
closing of files upon a thread. The FileManager therefore maintains a
count of the number of times a file has been opened and closed.
Upon a close it therefore decrements the counter. If this
close has closed the final remaining open then the close is
actually performed upon the file. The Used flag denotes if a file was
really forced open (by an implicit or explicit UseFile) as opposed to just
logically opened. Errors are not trapped by this routine as any
real problem with a close will be picked up again when the
file comes to be re-opened. For this reason ABC also eschews
the TryClose.
Fetch
PROCEDURE(KEY K),BYTE,PROC The Fetch routine is really just a
wrapper for a file GET. The error case results in
the buffer being cleared. Most of the work is done by re-vectoring
through TryFetch, and though this
doesn't save a great deal of code and is marginally less
efficient than inline coding it does result in greater code
integrity. Put another way, the code for fetching is in only one
place so only has to be fixed in one place.
InsertServer PROCEDURE(BYTE
HandleError),BYTE,PRIVATE Insert and TryInsert are just interface
maps of this procedure. InsertServer is a fairly good
illustration of the difference between the FileManager and the file driver
equivalents. It is only really trying to do an add; all of
the other code is there either to handle errors or to ensure
other ABC methods get called as appropriate.
First UseFileis called. This
registers that not only is this file logically open, it needs to be
actually open. Then comes a call to ValidateRecord. If the record is
not valid then the method returns. Note throughout ABC the
fact that Level:Benign is zero is assumed to
aid readability and brevity. There are then three cases:
PUT rather than ADD.ADD the record (remember PrimeAutoIncrementdoes not
corrupt the record buffer other than the autoinc components
themselves).There are then three error conditions to worry about:
NoError. In this case simply note
that any previously primed autoincrementing has now been used and
return. (Note this code assumes that PrimeAutoIncrementwill not have
left a value in ErrorCode if it was successful;
I suspect that technically this is a bug.)DupKey. This is the only error the
method attempts to recover from gracefully, stepping through the
keys and alerting the end user of any duplications that this record
causes.Everything else. Post a general (cryptic) error message to the user and return.
NextServer
PROCEDURE(BYTE HandleError,BYTE Prev),BYTE,PRIVATE Again Next and TryNext are just interfaces to
this method. Since C5EEA Previous and TryPrevious have also become
interfaces to this routine (the beauty of this method being
private!). The NextServer and PreviousServermethods of
earlier versions differed only in one line which has now been
parameterised with the Prev byte.
There are
only two real points of interest. Firstly the BadRecErr sets the EOF flag (see GetEOF). Secondly ABC has a
facility whereby a held record error can be treated simply as
a Skip rather than as an EOF (which it was in
2.003). You can argue back and forwards for hours as to whether it
is better to display a browse with information missing or to abort
the display. ABC takes the approach that that decision is best left
in the hands of the developer (as the real answer is probably
dataset specific) and so provides a property for her to register
her decision.
OpenServer PROCEDURE(BYTE HandleError,BYTE
IncrementUsage=True,
BYTE ForceOpen=False),BYTE,PROC,PRIVATE Open and TryOpen are interfaces to this
method. The second and third parameters are really for UseFile. They allow an actual
open to be forced without a logical open happening. Specifically,
setting increment usage to false means that a corresponding close
is not required. ForceOpen is used to force the
file open even when the LazyOpen status would suggest the
open can be deferred. Note that ForceOpen is quite safe as the
routine takes a failure to open because the file is already open as
a success. Again most of the work of the routine is in error
handling, and in this case some moderately sophisticated recovery
is allowed for.
NoError or FileOpen. Both treated as a success,
internal state variables cleared (on this thread).RecordLimitError. This code is just there for the evaluation
edition. An attempt has been made to open the file in read/write mode with
greater than the set number of records in the file. The code therefore sets
the OpenMode to read-only and cycles (the loop will then cause
another open attempt).NoAccessError. Read-write access
could not be acquired so the system tries to open the file in
read-only mode (having first warned the end user)NoFile. Provided the create mode
has been set the routine will attempt to create the file, if that
fails a fatal error is thrown.BadKeyErr. For those file drivers
with independent key files (notably Clarion) a corrupt key is
non-fatal and the system will try to rebuild the keys so that
processing can continue.The UNTIL 1at the end of
the loop means that any code falling through to the end of the loop
will cause the loop to terminate. The LOOP is not a real loop; it is
simply there to allow some of the recovery routines to attempt to
open the file again without a GOTO statement. It should be
noted that a CYCLE statement bypasses the loop
tail termination condition but not the loop top
termination condition.
Position
PROCEDURE(),STRING This method differs from the
file driver equivalent in that it will issue a UseFile and if a primary key is
available it will use that to form the position, or if there is no
primary key it will use the file itself. In general (and
increasingly) the ABC system assumes (and functions most
efficiently providing) that all files in the system have a primary
key. Note that this position string can only be used to perform
a TryReget; it is not as general as
the Clarion language Position. Whilst this functionality
restriction does not give us much presently it will eventually
allow extra efficiencies within the forthcoming FileClass.
TryFetch PROCEDURE(KEY K),BYTE,PROC TryFetch only really does
a UseFile before passing control to
the file system, although since C5EEA it has also performed
a SetKey in debug mode purely to
verify that the key passed in is valid for this file. (The ? is one of my favourite
C5 features; it allows you to write code very cleanly which will
not be executed when debug is turned off. This allows you to put in
quite a few safety checks with zero overhead in the final shipping
code.)
TryReget PROCEDURE(STRING Position),BYTE,PROCPerform a Regetfrom a string
provided by Position. The Reget/Position pair give the FileManager user one extra piece
of encapsulation: independence from key structure. Without them you
need to know from outside the class how to uniquely identify
a record. This illustrates an important aim: localising information
to reduce maintenance.
UpdateServer
PROCEDURE(BYTE HandleError),BYTE,PROC,PRIVATE InsertServer. The main extra comes from
concurrency issues. ABC implements a technique called optimistic
concurrency. Put simply, this means the algorithm assumes that no
one else will ever change the record being edited locally, and then
panics if they did. It relies upon a Update and TryUpdate are interfaces to this
procedure. Clearly this is similar toWATCH having been issued before
the record (now being updated) was fetched. In standard ABC usage
the WATCH is issued before the
view REGET in the browse
UpdateViewRecord method.
To handle
this the code first takes the position of the current record, then
it tries the PUT, if this returns a RecordChangedErrthen the user is
notified. (To help the end-user the form template passes in 2 as
the HandleError value which prompts
the use of a fairly verbose error message which tells the user of
such things as the history key). Then the record saved by the other
station is loaded (i.e. corrupting the local file buffer) and
control is handed back.
UseFile PROCEDURE(),BYTE,PROC The UseFile method
is there simply to perform a real open (using OpenServer) if lazy open is currently on and the file
is not open. This routine has one of the few bits of defensive coding in the
whole of ABC; it actually preserves the file buffers across the Open call just
in case the file driver (which could be supplied by a third party) corrupts
the file buffer upon the open.
I hope these articles on the FileManager have helped you understand
some of what we were aiming for (and have achieved) when we coded this class.
It is one of our largest and most complex, and it also presently forms the base
of what I call the spine of ABC : FileManager -> RelationManager
-> ViewManager -> BrowseClass. FileManager is also the class the
developer most frequently needs to interact with (at least as much as BrowseClass
and WindowManager). As such I believe an understanding of the principles involved
will send you well on your way towards mastery of the ABC system.
(Part 3 of 3. Click to read Part 1 and Part 2)
David Bayliss is a Systems Architect for The TopSpeed Development Center. He has worked upon TopSpeed's compiler and was the chief architect of the Application Builder Classes.
Copyright © 1999-2008 by CoveComm Inc. All Rights Reserved. Reproduction in any form without the express written consent of CoveComm Inc., except as described in the subscription agreement, is prohibited.
Clarion Magazine ISSN 1718-9942
One year: $189
(includes all back issues since '99)
Renewals from $139
Two years: $289
Renewals from $239