These feature only apply to SQL exports

Show export history

This command can be used to view the history of a particular export:

$ commcare-export-utils history --project X --query A.xlsx --output [SQL URL]


| Checkpoint Time            | Batch end date      | Export Complete | Project | Query Filename      | Query MD5                        | Key | CommCare HQ |
| -------------------------- | ------------------- | --------------- | ------- | ------------------- | -------------------------------- | --- | ----------- |
| 2018-08-29T15:18:04.299947 | 2014-08-30T04:30:48 | True            | X       | A.xlsx              | 9043f06f8cb4b26871ebd939609ca01e |     | prod        |
| 2018-08-29T15:18:01.553475 | 2014-08-29T09:45:32 | False           | X       | A.xlsx              | 9043f06f8cb4b26871ebd939609ca01e |     | prod        |
| 2018-08-29T15:17:59.300238 | 2014-08-28T05:13:17 | False           | X       | A.xlsx              | 9043f06f8cb4b26871ebd939609ca01e |     | prod        |
| 2018-08-29T15:17:57.151183 | 2014-08-27T14:36:38 | False           | X       | A.xlsx              | 9043f06f8cb4b26871ebd939609ca01e |     | prod        |
| 2018-08-29T15:17:54.914646 | 2014-08-26T15:41:11 | False           | X       | A.xlsx              | 9043f06f8cb4b26871ebd939609ca01e |     | prod        |
| 2018-08-29T15:17:52.771408 | 2014-08-25T21:46:47 | False           | X       | A.xlsx              | 9043f06f8cb4b26871ebd939609ca01e |     | prod        |


User defined export key for checkpointing

Normally the export tool uses the MD5 hash of the query file as the checkpoint key i.e. when starting up it will query the database using the MD5 hash to find the last checkpoint time.

This means that if the query file is changed the MD5 will also change the the tool will not be able to find the last checkpoint which will cause it to re-export all the data. In most cases this is what's desired since change to the query file require re-processing the data however in certain scenarios it may be more desirable to continue using the previous checkpoints.

For example, if a new column is added to a form export query file which represents a new field in the form, it is not necessary to re-process all the old data since that field doesn't exist anyway.

To accommodate these use cases the export tool can be made to ignore the MD5 hash of the query file and instead use a key supplied on the command line:

$ commcare-export --project X --query A.xlsx --output [SQL URL] --checkpoint-key my-key ...

Note that when running with a user supplied key the export tool will never consider the MD5 hash and therefore any re-processing of older data will need to be handled manually.

Converting a non-keyed export to a keyed export

To convert an export that has been using the MD5 hash as the key into one that uses a custom key run the following command:

$ commcare-export-utils set-key --project X --query A.xlsx --output [SQL URL] --checkpoint-key my-key

Note that this will use the MD5 hash of the query file to find the checkpoints that need to updated.

  • No labels