CM Table Repair
Version 2010a
Copyright (c) 1995-2010 CompuMotor Company. All Rights Reserved.
Support site: www.cmstory.com

============================

QUICK START

Run from within Visual FoxPro cmrepair.app (this file have to be next to file you are reading now - readme.htm). Press <Open table> button and select a dbf table.
DESCRIPTION

This is check and repair utility for Visual FoxPro, FoxPro and other DBF-type tables. You can see and repair nearly all items in the header, structure and data of the table and memo files. It can be called from code to automatically repair bad headers or data. Preview and print reports for checked tables, folders and databases. Exporting capabilities.


REGISTRATION

Unregistered version is for testing purposes and works in development environment only. To use this utility you need to obtain (purchase) registered copy. Please contact through support web site (or send a message) and we will sent to you further details and license conditions. With registered version you can use full capacity of the utility, can include it in your end user software product to distribute in this way to your end users.


REQUIREMENTS

Application consists from one run-time file: cmrepair.app. This is a Visual FoxPro application, so you need VFP installed on your system. It was tested with VFP 7, 8 and 9. It works with VFP 6, but with some limitations.


WARNING

Please backup data before repairing. This utility can write in the tables and memo files at low level, so be careful.


WARRANTY

This program is provided "as is" without any kind of warranty. The author doesn't take any responsibilities for any damages or data loss.


MORE NOTES

It is well known, that if a FoxPro program (any version) appends/writes to a table and computer crashes, the table becomes bad (corrupted). Next time when FoxPro tries to open it, an error appears, like:

  • Not a table (Error number 15)
  • Table "name" has become corrupted. The table will need to be repaired before using again. (Error number 2091)
  • MEMO file is missing/invalid (Error number 41)

For several years of working with many and large data massifs, we discovered that in 99 % it is possible to recover tables and memo files with small or no loss of information.

One of the problems occurs when FoxPro appends records. FoxPro engine writes buffers on disk in this order: first changes record number in the header of the table, and afterwards changes the size of the files and writes data. Better way is to reverse order of this two operations, but it is embedded in the Foxpro engine.

During the years we added many more functionality to CM Table Repair, such as auto-repairing of errors when obtained, checking all tables in given folder, repairing tables over 2 Gb, backup and auto-backups, repairing memo file headers, examination of data in table checked at low level (that means independent from possibility of VFP to open the table), powerful Read/Write test with possibility to repair hidden errors in data, such as error 39 (numeric overflow) and 41 (invalid memo file addressing) and many more.

Many developers are using this utility with success for years.
Any comments, ideas, suggestions and bug reports are appreciated. Please send them through support web site (or send a message).
 

EXAMPLES FOR USAGE

In package are included some PRG files, example_1table_simple.prg , example_1table.prg, example_1table_all.prg, example_1folder.prg and CheckPathGiven.prg. You can explore and run them in Visual Foxpro to see how CM Table Repair works. You can also run directly cmrepair.app. Here are more examples to call cmrepair.app.

  • DO cmrepair.app
    Main interface window appears (modal mode). Press button <Open table> to select a table for checking or repairing.
  • DO cmrepair.app WITH .f., .f., .f., .f., -1
    As above, but interface window appears in modeless mode. That means you can start the app several times and work simultaneously, or you can start other application in same time.
  • DO cmrepair.app WITH 'c:\temp\'
    Main interface window appears. Initial folder to select the table is given to be c:\temp. Press button <Open table> to select a table for checking or repairing, or <Check folder> to check or repair all tables in given the folder.
  • DO cmrepair.app WITH 'c:\temp\baddata.dbf'
    The table to be checked is given with full path. In this case interface appears and you see results of the examination directly. Pres <Report> to preview or print report for structural information obtained.
  • DO cmrepair.app WITH 'c:\temp\baddata.dbf', .t.
    Auto-repair mode. If number of the records written in the header of the table does not match size of the file, then auto-repairing is performed. This mode auto-repairs corrupted memo file also (from version 2004c). There is a limitation within 5 records of difference: not to "auto- loss" more then 5 records in this mode. Interface window is not shown in case nothing to repair or repair process is successful.
     
  • DO cmrepair.app WITH 'c:\temp\', .t.
    Auto-repair folder mode. All tables in the given folder are checked for corruption and auto-repaired if needs..
  • DO cmrepair.app WITH 'c:\temp\baddata.dbf', .t., 1
    In this example is specified risk to be only one record in auto-repair mode.
     
  • DO cmrepair.app WITH 'c:\temp\', .t., .f., .f., 3
    Auto-repair folder mode. All tables in the given folder are checked for corruption an auto-repaired if needs. "Silence" mode (no interface window appears) and final report is displayed. (Parameters 3 and 4 are .f. to accept default values, see below).
  • DO cmrepair.app WITH 'DB:c:\temp\mydatabase.dbc', .t., .f., .f., 3
    Auto-repair database mode. All tables in the given database container are checked for corruption an auto-repaired if needs. "Silence" mode (no interface window appears) and final report is displayed.
  • DO cmrepair.app WITH 'DB:c:\temp\mydatabase.dbc', .t., .f., .f., 3, .f., .f., .f., .f., .f., .f., .t.
    As previous example, but all tables in database are also Read/Write tested and auto-repaired if needed (See 12-th parameter).
     

For testing purposes in sub-folder bad_dbf (after un-zipping CMrepair.zip) some damaged tables are provided.

CALLING INSTRUCTIONS

 

First of all be informed, that we are executing the following commands in the beginning of cmrepair.app:
SET CONSOLE OFF
 SET CPDIALOG OFF

SET PROCEDURE TO Moduls ADDITIVE

SET CLASSLIB TO CMrepair ADDITIVE

If you see any conflicts with your environment during cmrepair.app execution, please save SET("CONSOLE"), SET("PROCEDURE"), SET("CLASSLIB"), SET("CPDIALOG") before calling cmrepair.app, and restore them after finishing cmrepair.app usage.

 

In CM Repair we are using ON ERROR error handling. Because of possible conflicts, please DO NOT CALL Cmrepair.app within TRY...CATCH...FINALY statement.

Also, it is important calling command DO Cmrepair.app WITH ... not to be in an ON ERROR statement or call to procedure. This is because in Cmrepair.app we are trying to USE the table passed as parameter and to catch the eventual error. As Visual Foxpro can not nest ON ERROR commands, as described in the VFP help, if you call Cmrepair.app in ON ERROR statement or procedure, an error will happen.

If you have cmrepair.exe. simply execute it from operating system environment. Cmrepair.exe will search VFP runtime libraries.


CALLING PARAMETERS (there are no compulsory parameters)

Place

Parameter

Type

Defaults to

Description

1

cPathName

input

""

Variant 1. Valid table (full path and name) - table to check and repair.
Variant 2. Valid directory (full path with end slash) - folder to check or default directory for selection of table to check.

Variant 3. DB: prefix plus valid full path and name for database container (DBC file) - to check database and tables in it (see examples abouve).
If empty - user manually have to select table, folder or database for checking.

2

nAutoRepair

input

.f. or 0

.f. or 0 - interface mode (default).

.t. or 1 - auto-repair mode. Repairs corrupted table and memo files of the given table with cPathName (or entire folder/database). If all is OK, or corruptions are repaired with success - no interface appears, otherwise interface window appears or not according to tnSilence parameter. If no interface appears - results are returned in parameters 6, 7, 8, 9, 10. In this auto-repair mode Windows API functions are used for low level file manipulations, so table can be checked and repaired even same table is opened in share mode in the network.

2 - auto-repair mode forced to use build in Visual Foxpro low level functions for file manipulation. Than means table can not be opened if other user opened same table in share or exclusive mode in the network.

3

nAutoRiskRecords

input

5

Specifies how many records maximum can be truncated in auto-repair mode, or so called "risk records". If more then nAutoRiskRecords are damaged and lAutoRepair=.t.,   interface window appears or not according to tnSilence parameter.

4

cPathNameLogFile

input

AddBS(JustPath(SYS(16,0))) + "cmrepair.log"

Path and name of the log file to record any activity in auto-repair mode. If the file does not exists, it will be created first time CM Table Repair needs to write in it. If not given - default place for it is in same folder as main calling program.

5

tnSilence

input

0

-1 - modeless interface appears (calling program continue execution);

0 or .f. - modal interface appears (calling program stops execution while cmrepair.app is running);
1 or .t. - no interface, no progress messages;
2 - output to simple window, no interface, with progress messages;
3 - output to report (preview mode with print toolbar), no interface, with progress messages;
Notes:
1. In silence mode (no interface) you can get result info from a_TableCheckInfo and other output parameters.

2. In interface mode when lAutoRepair=.t. and given table is successfully auto-repaired, then no interface appears.

6

tnTablesToCheck

output

0

How many tables are given to check in auto-repair mode.
Equals to 1 if a file is given to be checked, and,
number of tables in the given folder/database if folder or database is given to be checked (or equals to ALEN(a_TableCheckInfo,1)).

7

tnTablesChecked

output

0

How many tables are checked from the given to check in auto-repair mode.

8

tnTablesCorrupted

output

0

How many tables are corrupted from the checked tables in auto-repair mode.

9

tnTablesAutoRepaired

output

0

How many tables are auto-repaired from the corrupted tables in auto-repair mode.

10

a_TableCheckInfo

output

array[tnTablesToCheck,8]

Array with info for tables given  to be checked.
a_TableCheckInfo[n,1] - name of the table
a_TableCheckInfo[n,2] - .t. if checked
a_TableCheckInfo[n,3] - .t. if corrupted
a_TableCheckInfo[n,4] - .t. if auto-repaired
a_TableCheckInfo[n,5] - messages, if any.
Read/Write data test results:
a_TableCheckInfo[n,6] - .t. if full RW test is performed
a_TableCheckInfo[n,7] - number of found RW errors
a_TableCheckInfo[n,8] - number of repaired RW errors

a_TableCheckInfo[n,9] - .t. if packing or other action (dependent on tnAutoActTables) is started for the table

a_TableCheckInfo[n,10] - .t. if this action is successful for the table

11

tlAutoBackup

 

input
output

.f.

If to make backup copy of table files before repairing. In auot-repair mode here on output - if backup is really created. Backup name is same as original table name, with "-backup" added (before extension dot).

12

tlAutoCheckdata

 

input
output

.f.

If to perform check for hidden errors in data of the table files after ordinary check or repairing. In fact errors in data like 39 (numeric overflow) and 41 (memo file is invalid) does not appear when table is opened, but appear when reading or writing data. It is recommended to perform this RW test with auto-repair possibilities.

13

tnAutoActTables

 

input

1

If to perform any service actions on the table(s) after all other checkings.

.f. or 0 or 1 – do nothing

2 - pack and reindex, 3 - zap, 4 - copy to folder, 5 - export as XLS, 6 - export delimited, 7 - export with tab.

14

tnAutoActTables

 

input

.f.

For now it is used to define output folder in case tnAutoActTables is 4, 5, 6 or 7 (copy or export).

15,16

taStruct, taIndex

 

input by ref

.f.

table structure arrays (also  indexes), in case table path is given - for detailed structure comparison of given structure and real structure of checked table.


HISTORY SINCE CM TABLE REPAIR IS PUBLISHED

Version

Date

Changes

 

1995
  • First Foxpro 2.6 versions.

 

1999
  • First Visual Foxpro version (VFP 6). Some interface changes and users suggestions.

 

2000-11-15
  • Some bugs removed (no more errors in case of any file checking). Some long file name problems decided

 

2002-12-08
  • Upgrading and support for VFP 7

2003a

2003-11-28
  • Added support for auto-incrementing in tables
  • Upgrading and support for VFP 8
  • Changes and improvements

2004a

2004-06-19
  • Added support for new data types
  • Upgrading and support for VFP 9
  • Removed limitation for number of records in advanced memo file analysis in case of running with VFP 9

2004b

2004-07-28
  • Auto-repair mode now repairs both record numbers and file size
  • Activity in auto-repair mode logs to a file with same folder and name as the utility, with .log extension
  • Introduced is new (third) calling parameter: number of risk records in auto-repair mode (defaults to 5)

2004c

2004-08-27
  • Auto-repair mode now repairs memo file headers in case of error 41 "memo file invalid" (next free block number and block size)
  • More precise handling of memo file checking process and offer to change memo file attributes
  • Checking given folder for corrupted tables, both manual and auto-repair mode
  • Introduced is new (fourth) calling parameter: path and name of log file for auto-repair activity
  • Introduced is new (fifth) calling parameter: silence mode (if .t.)
  • Introduced are new output parameters: info for performed work in auto-repair mode
  • Save and restore last directory where checking is performed (if not file or folder given as parameter)
  • Different style of starting: first screen appears, afterwards opening the table to check
  • Bugs are removed (refreshing in window "Memo file analyzing", appearing of the interface in auto-repair mode in case of not damaged table)
  • Improved documentation

2004d

2004-12-13
  • Some small bugs are removed
  • More support of SET TABLEVALIDATE command in auto-repair mode
  • In auto-repair mode, when the interface appears, information is shown about the reason

 

2005a

2005-07-04
  • Introduced is possibility to fix the problem if repaired table exceeds 2 Gb (the Foxpro limit) and becomes corrupted (that happens for example when error 1190 "File is too large" is not triggered in VFP7 and below).
  • Removed dependence on foxtools.fll. If you are using old versions of VFP (<7), please set library to foxtools.fll in advance.
  • Support for all known Foxpro table types: DBF, FRX, LBX, DBC, SCX, VCX, MNX, PJX and respective memo files.
  • Auto-backup parameter and separate button (parameter is 11-th and defaults to .f.!)
  • All reported bugs to the moment are removed
  • More interface improvements
  • Stand alone EXE version (for registered users only)

2005b

2005-12-09
  • Handling of error 13 (Alias is not found) when opening table in folder checking procedure. This error occurs if a table alias is used in index expression in CDX file, but not exist at that moment.
  • In case of folder checking all files with known table extensions are included in the process. The user can edit extensions list.
  • Procedure to read and show data from the table file is introduced. Reading is performed at low level, so it is independent from VFP possibility to open the table. For now we have 2 modes: "normal" - shows data with their data types, like in BROWSE window, and "raw" - you can see data from the file without any conversion, for example in Integer fields you see bytes, not digits, and in Memo type fields - bytes from memo block addresses. In both cases data are shown in columns, as specified in the table structure.
  • Read/Write test is introduced. This helps to find hidden errors in table data, such as numeric overflow and bad memo block addressing. This mode also have auto-repair possibility and one more parameter is added: tlAutoCheckdata. Columns of output array a_TableCheckInfo are also increased to hold results from new RW test.
  • Example functions are extended for checking given folder in auto-repair mode with displaying the result, ad for RW test.
  • Improved help.

2006a

2006-03-21
  • Handling read-only files - when program attempts to open in read/write mode to change them, first attempts to reset read-only attribute to make file writable.
  • Added total size info for table checked (table + memo and index files).
  • Reporting - preview and/or print results from checking table or folder with tables. Report FRX files are included in the app.
  • Corrected are some errors in RW test procedure connected with field validation expressions and other small bugs. More testing and improvements.
  • 5-th parameter tlSilence is changed to numeric (.t./.f. accepted anyway) and if tnSilence > 1 user can see new reports without displaying the interface See this parameter description for all valid values.

2006b

2006-09-04
  • Added possibility to perform various service actions on tables: pack/reindex, reindex only, zap, copy to folder, export as XLS, export delimited, export with tab.
  • This actions also have auto-possibilities and one more parameter is added: tnAutoActTables. Columns of output array a_TableCheckInfo are also increased to hold results from new actions.
  • Added one more parameter: tActParameters. For now it is used to define output folder in case tnAutoActTables is 4, 5, 6 or 7 (copy or export).
  • Provided is one more example program example_1table_all.prg - haw to call cmrepair.app for one table and after ordinary table check to test Read/Write, and also to perform given action (packing or other).

2007a

2007-11-20
  • Added possibility to run cmrepair.app in modeless mode (see additions to tnSilence parameter).
  • Added "Modify structure" possibility - sub-menu in "Use table" button.
  • Added new export format: simple HTML - data from the table are exported with no or minimum formatting.
  • More functionality in hole folder checking (all tables in it). Possibility to backup the folder. Interface changes - simplification, additions and more intuitive.
  • Preserve original filename and case in displaying results and creating files and folders.
  • VFP stand alone executable version is introduced - cmrepair.exe, requires VFP runtime libraries (for registered users only - on request, compiled with VFP version 9 only).
  • Added button for cleaning the folder (and all sub-folders) from files with extensions TMP, BAK or TBK.

2007b

2007-12-28
  • Resolved conflict when in table name are used special characters (not allowed in alias name).
  • Extended exporting capabilities:
    - in case tab or comma delimited export - replacing     the delimiter in all character fields with space - because VFP do not care to do this in COPY TO command and exported file can be truncated at some points (where the delimiter is contained in the data).
    - before export with COPY TO command - creating an intermediate cursor with transformed memo columns of the table - to C*254 columns (this is because VFP can not export memo fields and neglects them).
  • Possibility to select which columns of the table to export, and also their order (their succession in destination file).
  • Added new export format: MySQL - data from the table are exported in a file with sql extension. This file contains valid SQL commands for creating the table in a MySQL database and also INSERT commands to import data. The file can be used in an MySQL database utility such as phpMyAdmin.
  • In addition to above MySQL format: php variant. An file with php extension is created. If this file is uploaded on the server and HTTP accessed - table and data are imported in given database. To specify database and connection parameters, it is obligatory to specify them in the beginning of the file (please see variables $sqlhost, $sqllogin, $sqlpass, $dbname and init them).

2008a

2008-04-24
  • Auto-repair mode now changes table flag in case the table is marked to have CDX file, but CDX file is missing. This permits table to be opened with success.
  • New possibility to pass as parameters table structure arrays (also  indexes), in case table path is given, and new button and form with detailed comparison of given structure and real structure of checked table.
  • New possibility to use Windows API functions for low level file manipulation. This permits to check, repair and auto-repair tables even they are used from other users in the network in share mode. Now using API calls in CM Repair is default, but a parameter is provided if user like to force handling of files with Visual Foxpro low level file manipulation functions (see modified nAutoRepair parameter). In interface mode a checkbox is provided to control which functions to use.

2008b

2008-07-09
  • Introduced is database check. That means checking, testing and repairing DBC container and all tables in given database.
  • Auto-repair mode is extended over given database. See modification of 1-st calling parameter, and also calling example provided.

2009a

2009-05-09
  • Introduced is "Get healthy header" possibility.  This procedure gets header from other table with exactly same structure and stuff it in the examined table file. To be used when data in the table file seems to be in line, but header is severe demaged.
  • Pack test is introduced. This helps to find hidden errors in table data, such as bad memo block addressing (error 41). This mode also have repair possibility. The procedure is especially useful when error 41 is reported when packing only (not when open or read data in the table).

2010a

2010-08-11
  • A bug fix release:
  • Extensivelly tested with Visual Foxpro 6, 7, 8, 9 - proper usage of respective commands.
  • Siutable implementation in Windows 7 environment.