libesedb misses parts of certain columns

[jengelh]

MDB02.edb.gz

esedbexport MDB02.edb -T Folder_101

In table Folder_101, 2nd row, column SeparatedProperty01, esedbexport shows a 5171-byte value with start bytes 0x18 0xd6 0x1f. When uncompressed, this indeed looks like the random text I had previously input through MFCMAPI, but it is too short.

A hexdump of MDB02 reveals the lzxpress stream is located at byte 9830506 for 5171 bytes until 9835677. Following this is another lzxpress stream at 9835701 for 835 bytes until 9836536. When decompressing those two streams individually and concatenating the 0x1fd6 long first part and the 0x03ae second part, I get back the 9092 bytes I originally put in.

[joachimmetz]

I have a closer look when time permits

[jengelh]

I gdb'd this. In function libesedb_page_tree_get_leaf_value_by_key_from_page, there is:

        compare_result = libesedb_page_tree_key_compare(
                          leaf_value_key,
                          page_value_key,
                          error );

Inputs are:

(gdb) p leaf_value_key[0]
$31 = {type = 83 'S', data = 0x555555733980 "", data_size = 8}
(gdb) x/8bx leaf_value_key[0].data
0x555555733980: 0x00    0x00    0x00    0x01    0x00    0x00    0x14    0x33
(gdb) p page_value_key[0]
$32 = {type = 108 'l', data = 0x555555733c80 "", data_size = 4}
(gdb) x/4bx page_value_key[0].data
0x555555733c80: 0x00    0x00    0x00    0x01
(gdb) p/x page_flags
$33 = 0xa883

compare_result will be LIBFDATA_COMPARE_GREATER, but this is not handled in the subsequent if tree. The source code only tests for EQUAL under PAGE_FLAG_IS_LEAF.

        if( compare_result == -1 )
        {
                ...
                goto on_error;
        }
        if( ( page_flags & LIBESEDB_PAGE_FLAG_IS_LEAF ) != 0 )
        {
                if( compare_result == LIBFDATA_COMPARE_EQUAL )
                {
                        ...
                }
        }
        else if( compare_result != LIBFDATA_COMPARE_GREATER )
        {
                if( page_tree_value->data_size < 4 )
                {
                        ...
                }
                ...
        }

[jengelh]

Using this change to look at all invocations of libesedb_page_tree_key_compare made by libesedb_record_get_long_value_data_segments_list,

+++ b/libesedb/libesedb_page_tree.c
@@ -2801,6 +2801,13 @@ int libesedb_page_tree_get_leaf_value_by_key_from_page(
                                  leaf_value_key,
                                  page_value_key,
                                  error );
+               printf("PVI %d LEAF ", page_value_index);
+               for (size_t ii=0;ii<leaf_value_key->data_size;++ii)
+                       printf("%02x", (uint8_t)leaf_value_key->data[ii]);
+               printf(" PAGE ");
+               for (size_t ii=0;ii<page_value_key->data_size;++ii)
+                       printf("%02x", (uint8_t)page_value_key->data[ii]);
+               printf(" CMP %d\n", compare_result);
 
                if( compare_result == -1 )
                {

I find that there is a page_value_key with 1fd6 in it.

PVI 1 LEAF 0000000100000000 PAGE 00000001 CMP 3
PVI 2 LEAF 0000000100000000 PAGE 0000000100000000 CMP 2
PVI 1 LEAF 0000000100001433 PAGE 00000001 CMP 3
PVI 2 LEAF 0000000100001433 PAGE 0000000100000000 CMP 3
PVI 3 LEAF 0000000100001433 PAGE 0000000100001fd6 CMP 0

This suggests that the += operation in libesedb_record_get_long_value_data_segments_list needs to add the uncompressed size (0x1fd6), not just the (compressed) data_size (0x1433).

                        long_value_segment_offset += data_definition->data_size;