FATS - Fast Access Tree System
	Table of Contents
		Matchcode Commands

Matchcode Commands

Introduction
Mode of Operation
Features
Example

Introduction

The effort of modern application programs makes the collecting and processing of large amounts of data possible. For the user, it becomes more and more difficult to find the information relevant to him in available data resources.

As a solution to this problem, software programs normally offer indices based retrieval, that can be implemented by the programmer e.g. through a toolkit like FATS. It makes the look-up of specific data records possible for the user through the input of catchwords or indices those were assigned to the data records while its recording. Examples for this are article numbers for identification of an article record or the customers surname for look-up in the customer database.

Therefore, conventional search methods have the following disadvantages:

The user of the software must adapt to the specifications and to possibilities of the application program and has often to remember many numbers and/or terms for the successful research of his database.
The entered terms are matched with the keys of indices 1:1. As a result, different spelling can prevent the location of the searched data records. If the corresponding data are maintained by several users or inexperienced persons are searching for something, mistakes are made automatically.
During a linked look-up over several data fields or the look-up in not indexed fields, a sequential scan of the corresponding table or data file is unavoidable and mostly leads to not acceptable access times in the case of large databases or network applications.
Conventional indices are not suitable for a look-up in non-record structured data since these show no uniform structure. Therefore, much information in text- or memo fields is not traceable fast enough.
Through the logical order of an index file, character relation only occurs at the beginning of a search key, i.e. a search for "car" finds not "motorcar".

FATS Extended offers new, fascinating possibilities in order to overcome the mentioned problems. With the aid of the matchcode commands, you can extend your programs by a matchcode- resp. full text search within shortest time. This offers a natural retrieval function adapted to the human being using your applications.

The FATS matchcode offers the following advantages:

The content of any data fields can be indexed in the full text. The data records can be discovered by the specification of free terms which occur somewhere in the corresponding data fields very fast.
Studies of big Internet search services yielded that the users during research favour a simple input without boolean operators in the used search strings. Therefore, the retrieval function also returns optimal results without complex inputs and inexperienced persons reach the destination without training.
A associated look-up over several data fields is possible WITHOUT additional time required. During the implementation of the matchcode command in your programs, several data fields (perhaps of different tables or data files) can be combined into logical search groups. No access to the data file(s) is required for the location of the data records, i.e. look-up occurs exclusively in the matchcode index.
The matchcode search functions are also outstandingly suitable for look-up in non-record structured data such as texts and memo fields. You can find every word and every number in the corresponding texts in fractions of a second.
The technical realization of the matchcode index makes it possible to carry out the look-up after word parts (fragments) with the same rate as look-up according to a whole word.

Mode of Operation

The FATS matchcode commands create a fulltext index with the content of your data tables or -files. Every word and every number is incorporated into the index, an "inverted list" is generated.

This index enables FATS to find each data record by specification of any terms in fractions of a second.

The matchcode index was optimized on fragment search, i.e. in contrast to conventional fulltext retrieval systems the rate does not take off with the search for word fragments (word center, word ending ...).

A list of all retrieved records (record numbers and/or primary-keys) is available immediately after the search.

Features

High speed search even over extremely large databases. Even a several 100 megabyte large file is on the average searched in less than one second. The data file(s) are not needed for the search.
Extremely fast indexing (1 million data records on a pentium system in about 15 minutes).
An unlimited number of words, numbers and word fragments of any length are considered at the search. Even a word overall search is possible, i.e. a search for "motorca" finds "motorcar" and "motor caravan".
Optional sorting of the result set by means of hit quality. Alphanumeric sorting adjustable.
It is possible to connect fields and build logical search groups to do a joined search on up to 32 search groups.
Output of the number of hits already with the first result.
Max. 16 million data records per matchcode index file.
Max. field length variable, unlimited
Effective algorithms for the compression of the matchcode index file; space requirement for the index normally only approx. 1/2 of the input file.
Minimum memory requirement (the matchcode search already operates with 32 KB memory at full speed). The DOS versions can run in protected mode and therefore need only 800-2000 Bytes main memory.
Commands for subsequent adding, removing or modifying of data records.

Sample Program

The following Visual Basic example demonstrates the use of the FATS matchcode commands. Even if you operate with another programming language, you should read the following explanations. You find a individual description of the programming language interfaces in chapter 9. On the FATS diskette you additionally find a sample program to each programming language.

The matchcode commands are only supported by the extended version of FATS.

Create Matchcode-Indexfile

With the command Create Matchcode File (MC), the most important query facilities are determined already while creating the matchcode index file. With the search-group flag ("I#"), several logically related data columns can be registered in a common index so that query to this index resp. search group extends automatically over all these columns. A matchcode file manages up to 32 search groups that can be used for joined queries (using the "AND"-operator).

In this example some data fields of a customer table are to be indexed. The data record has the following structure:

Type kdat
Deleted As String * 1
Name As String * 25
Job As String * 25
Street As String * 25
Zip As String * 5
City As String * 20
End Type

The following program sequence creates the matchcode index file "customer.fms" with the search-groups Name (I1), Job (I2) and Zip/City (I3):

szCmnd = "MC\customer.fms\\1\I1\I2\I3"
dwRecno = FATSCall(szCmnd, nErrorcode, nKeylen, 0)

The syntax of the command:

CMND$ = "MC\FileName\Flags\FileNo\Col1def[\Col2def]"
FileName Filename, perhaps with an additional path
(i.e. C:/DATA/CUSTOMER.FMS or CUSTOMER.FMS)
Flags Reserved, not used at the moment
FileNo File number
Col#def Definition of data column # (flags, separated by comma).
The content of the corresponding data columns is transferred later to the commands "MB", "MI" and "MD" in the order determined by this command.
I# The content of the data column becomes part of search group #. You can combine several columns into a logical search group (e.g. first name, surname).
Further adjustments are possible and described in detail on the page page 4-55.

Build Matchcode (Indexing)

After the matchcode file was generated, the content of the data columns may be with the command Build Matchcode (MB) inserted into the matchcode index. The position of the data columns within the command string ("Col#data") corresponds to that with the call of command Create Matchcode File (MC) determined definition.

The syntax of the command:

CMND$ = "MB\FileNo\RecID\Col1data[\Col2data[\Col3data]]"
FileNo File number
RecID != 0 Record- resp. id-number
== 0 Stop Build, no more records
Col#data Content of data column #
[ ] Optional

The following sample program code indexes contents of the entire data file within a loop:

dwActRecno = 0

Do

' read record
Get #1, dwActRecno + 1, customers

If Not EOF(1) Then

dwActRecno = dwActRecno + 1

' create command string...

szCmnd = "MB\1\" & Str$(dwActRecno)

If customers.Deleted = " " Then

szCmnd = szCmnd & "\" & RTrim$(customers.Name)
szCmnd = szCmnd & "\" & RTrim$(customers.Job)
szCmnd = szCmnd & "\" & RTrim$(customers.Zip & " " & customers.City)
End If

dwRecno = FATSCall(szCmnd, nErrorcode, nKeylen, 0)

If nErrorcode <> 0 Then

' If an error occurred during the execution of the "MB" command,
' the matchcode index file is already closed by FATS.

Print "FATS Errorcode: "; nErrorcode; " (command: MB)"
Exit Do
End If

Else

' After the last record was inserted the creating process has to be
' terminated with the command "MB\{FileNo}\0".
' Because this command closes the matchcode index file you don't
' have to do a close command.

dwRecno = FATSCall("MB\1\0", nErrorcode, nKeylen, 0)

End If

Loop While Not EOF(1)

Open Matchcode Indexfile

With the command Open Indexfile (O) you open an existing matchcode index file with the opening flags defined with the command Auto Refresh (Y). After the file was opened it can be accessed under the file number you specified.

szCmnd = "O\customer.fms\1"
dwRecno = FATSCall(szCmnd, nErrorcode, nKeylen, 0)

Matchcode Query

The command Search in Matchcode (MS) searches the matchcode index and fills the result set with the record- resp. ID-numbers of data records which include the searched terms. If FATS finds the requested terms, it returns the record- resp. ID-number of the first hit in the "dwRecno" variable and a errorcode of 0. The result set can then be read out with the browser commands ( "MA", "ME", "MN", "MP" ...).

The following program transfers the content of the input fields szName (search group 1), szJob (group 2) and szZipCity (group 3) to the matchcode search command. FATS generates a result set that contains data records which include all of the searched terms:

szCmnd = "MS\1\\0\" & szName & "\" & szBranche & "\" & szPlzOrt
dwRecno = FATSCall(szCmnd, nErrorcode, nKeylen, 0)

The syntax of the command:

CMND$ = "MS\FileNo\Flags\Total\SearchGrp1[\SearchGrp2...]"
FileNo File number
Flags The search procedure can be adjusted by the specification of different flags. You find a description of the flags in the command reference.
Total Maximum number of hits in the result set:
0 All hits are set into the result set. This regulation works very fast, however, no sorting occurs by means of hit quality.
!= 0 It is attempted to put the specified number of hits into the result set. These hits are subjected to a sorting concerning their quality and then put into one of 7 sorting groups.
SearchGrp# The in search group # searched terms.

Reading the Result Table

Different FATS commands can be used for further processing of the result table, depending on the program logic. The following example printss all hits in a Visual Basic listbox:

' the record number of the first hit is already
' in the dwRecno-variable (see above)

Do

' load data record and add it to the list
Get #1, dwRecno, customers
Liste.AddItem customers.Name & " | " & customers.ZIP & " " & customers.City

' determine next hit with the "MN"-command (Next Result)
dwRecno = FATSCall("MN\1", nErrorcode, nKeylen, 0)

Loop While nErrorcode = 0

The following browser commands are supported by FATS:

"MF" Get First Result; page 4-65
"ML" Get Last Result; page 4-69
"MP" Get Previous Result; page 4-73
"MN" Get Next Result; page 4-71
"MA" Get Next Result After; page 4-50
"ME" Get Previous Result Before; page 4-63

Inserting Records

The command "MI" can be used to add a new record to the matchcode index:

CMND$ = "MI\FileNo\RecID\Col1data[\Col2data[\Col3data]]"
FileNo File number
RecID Record- resp. id-number
Col#data Contents of data column #
[ ] Optional

This command inserts the content of the data columns ("Col#data") of the specified data record ("RecID") into the matchcode index.

The position of the data columns within the command string ("Col#data") corresponds to that with the call of command Create Matchcode File (MC) determined definition. The syntax of the command corresponds to that of the Build Matchcode (MB) command.

Example:

szCmnd = "MI\1\" & Str$(dwActRecno)
szCmnd = szCmnd & "\" & RTrim$(customers.Name)
szCmnd = szCmnd & "\" & RTrim$(customers.Job)
szCmnd = szCmnd & "\" & RTrim$(customers.Zip & " " & customers.City)
dwRecno = FATSCall(szCmnd, nErrorcode, nKeylen, 0)

Deleting Records

When removing a data record from the data file the match-code index should also be updated. You can use the command "MD" to do this. The syntax of the command corresponds to that of the "MI"-command.

szCmnd = "MD\1\" & Str$(dwActRecno)
szCmnd = szCmnd & "\" & RTrim$(customers.Name)
szCmnd = szCmnd & "\" & RTrim$(customers.Job)
szCmnd = szCmnd & "\" & RTrim$(customers.Zip & " " & customers.City)
dwRecno = FATSCall(szCmnd, nErrorcode, nKeylen, 0)

Change Content of a Record

If you want to change the content of a data record that has already been indexed, its previous content (before modification) must at first be removed from the matchcode index with the Delete Matchcode (MD) command before the new content is inserted with the command Insert Matchcode (MI). Therefore, in the following example a command string is already formed before the modification of the data record:

' save actual content of the data record
szOld = RTrim$(customers.Name)
szOld = szOld & "\" & RTrim$(customers.Job)
szOld = szOld & "\" & RTrim$(customers.Zip) & " " & RTrim$(customers.City)

' change the contents of the data record
Call ChangeRecord

szNew = RTrim$(customers.Name)
szNew = szNew & "\" & RTrim$(customers.Job)
szNew = szNew & "\" & RTrim$(customers.Zip) & " " & RTrim$(customers.City)

If szOld <> szNew Then

szCmnd = "MD\1\" & Str$(dwActRecno) & "\" & szOld
dwRecno = FATSCall(szCmnd, nErrorcode, nKeylen, 0)
szCmnd = "MI\1\" & Str$(dwActRecno) & "\" & szNew
dwRecno = FATSCall(szCmnd, nErrorcode, nKeylen, 0)

End If

Close Matchcode-Indexfile

The following command closes the matchcode index file: "K" Close Indexfile.

dwRecno = FATSCall("K\1", nErrorcode, nKeylen, 0)

FileName	Filename, perhaps with an additional path (i.e. C:/DATA/CUSTOMER.FMS or CUSTOMER.FMS)
Flags	Reserved, not used at the moment
FileNo	File number
Col#def	Definition of data column # (flags, separated by comma). The content of the corresponding data columns is transferred later to the commands "MB", "MI" and "MD" in the order determined by this command.
	I#	The content of the data column becomes part of search group #. You can combine several columns into a logical search group (e.g. first name, surname).
	Further adjustments are possible and described in detail on the page page 4-55.

FileNo	File number
RecID	!= 0 Record- resp. id-number == 0 Stop Build, no more records
Col#data	Content of data column #
[ ]	Optional