Skip to content

Fix bug #4#6

Open
rda0 wants to merge 9 commits intoelmar:masterfrom
rda0:master
Open

Fix bug #4#6
rda0 wants to merge 9 commits intoelmar:masterfrom
rda0:master

Conversation

@rda0
Copy link

@rda0 rda0 commented Dec 20, 2017

Hi Elmar

I would like to make some changes to ldap-git-backup suitable for
our new setup at the ISG D-PHYS as explained below. I also plan to
add another option to be able to exclude some attributes from the
backup. I am curious about your feedback.

If ldap-git-backup is used to backup the OpenLDAP configuration
stored in cn=config, the order of entries is critical for a restore
scenario. Therefore adding the possibility to have the entries' file
names prefixed with a zero-padded number will allow to concatenate
the files using their filename in the correct order. This may also
be useful if one needs to restore only some parts of the database
using ldapadd to a running slapd instance. It may also be useful to
have the DN of the entry in the filename to concatenate only parts
of the database backup.

If ldap-git-backup is used to backup the OpenLDAP configuration
stored in cn=config, the order of entries is critical for a restore
scenario. Therefore adding the possibility to have the entries' file
names prefixed with a zero-padded number will allow to concatenate
the files using their filename in the correct order. This may also
be useful if one needs to restore only some parts of the database
using ldapadd to a running slapd instance. It may also be useful to
have the DN of the entry in the filename to concatenate only parts
of the database backup.
@xtaran
Copy link
Collaborator

xtaran commented Dec 21, 2017

Hi Sven,

thanks for the suggestions and the patch. We'll have a look at it.

It's a little bit of a unlucky timing as I just released 1.0.8 yesterday despite not much (actually no code, just documentation, packaging and test suite) changed. (But that's not your fault, just bad luck.)

@rda0
Copy link
Author

rda0 commented Dec 21, 2017

Hi Axel

That is no problem at all, I just wanted to get some feedback. And maybe also know what the intentions were to use hashes as filenames. I am not sure anymore if it is wise at all to use DNs in the filenames, because of the maximum file size limitation. But I think for our database it will never be a problem most probably, we are at a max DN length of 131 at the moment.

It certainly needs some testing...

rda0 added 5 commits December 21, 2017 15:09
When the argument length in the git-add operation, resulting from
all the filenames of all ldif files is greater than ARG_MAX, an
error occurs similar as when trying to list these files in the
backup directory after that using a wildcard `ls /backupdir/*`:
  -bash: /bin/ls: Argument list too long
On my testsystem a `getconf ARG_MAX` reports 2097152, but the
actual limit was about 1880000. Maybe this value has to be deter-
mined dynamically during runtime. See also:
https://unix.stackexchange.com/a/120842/60293
@rda0
Copy link
Author

rda0 commented Dec 21, 2017

Hi Axel

I added a few more commits. Have a look at 55d5d7e which probably fixes the issue (error 7: very big file list to add to git). It now does multiple git add operations when a predefined limit arg_max is exceeded.

rda0 added 3 commits December 21, 2017 18:42
This reverts the following commits:

24642fd
8cd2f60
bbf3558
1c22e01
9b43379
1fc1fae

While just keeping the bugfix in the master branch, the removed
changes will be put into separate feature branches.
@rda0 rda0 changed the title Add options --dn, --sort and --sort-prefix Fix bug #4 Jan 4, 2018
@rda0
Copy link
Author

rda0 commented Jan 4, 2018

I reverted all the changes which add new features and moved them to separate branches.
This only leaves the changes for the fix of #4 in the master branch.

I will not make new pull requests for the new features at the moment:

The reason for this: I changed my mind and I now think it may be better to create a separate script for the sorting of the entries, to not bloat the backup code. In addition the new features could add other possible problems:

  • not allowed characters in filenames, which may appear in DNs (/)
  • max. allowed filename length

The filtering may be achieved by using an ldif-cmd, which pipes the LDIF to grep.

Please let me know if I should still make a pull request for these branches.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants