Skip to content

Conversation

@mmcnabb-vms
Copy link
Contributor

Summary of fixes on this PR:

  • The first three commits fix bugs in bcp in/out , which are actually discussed on Handling of NULL values in FreeBCP native file format with MSSQL #686 . With these fixes in place, data should round-trip correctly using {freebcp|MSbcp|ASEbcp} with character format interchangeably on the same data file. The roundtrip of freebcp -n with SQL Server has had some fixes but there are still outstanding issues regarding nulls in which are covered on 686 and I'll work on at a later date
  • Added two new tests and slightly expanded d_bcp test relating to the handling of NULL and Default Values in bcp
  • When talking to ASE: implement the feature of ASE bcp that if a column has a Default Value defined in the database, then the column can be omitted from the BCP format file, or have value specified as NULL in the data file where permitted, with the effect that the Default Value for the column will be inserted; even if the column was defined as non-nullable. (MS bcp does not allow omitting columns, and handles default values differently in a way that's already supported by freebcp).
  • Allow multiple Hints (that's the stuff in the WITH() clause). I also removed the code that validates the hints, since the database will reject an invalid hint and I don't see what we gain by rejecting the hint on client side; it seems more forwards-compatible to let the database handle this.
  • Implement the -k flag as supported by MSBCP (ASE bcp calls it --ignoreDefaults) which means that even if a column has a default value defined, BCP inserting a NULL will actually insert NULL and not the Default Value.

Most of the content is covered by the new test cases, because freebcp is a fairly thin wrapper around DBlib bcp API. However I also have made a local test script that does a few "front door" tests calling freebcp directly to test the null/default handling, and did ad-hoc testing of -k.

mmcnabb-vms and others added 10 commits January 26, 2026 12:14
Signed-off-by: Matt McNabb <matthew.mcnabb@vmssoftware.com>
…d from data file

Signed-off-by: Matt McNabb <matthew.mcnabb@vmssoftware.com>
Signed-off-by: Matt McNabb <matthew.mcnabb@vmssoftware.com>
Signed-off-by: Matt McNabb <matthew.mcnabb@vmssoftware.com>
Signed-off-by: Matt McNabb <matthew.mcnabb@vmssoftware.com>
Signed-off-by: Matt McNabb <matthew.mcnabb@vmssoftware.com>
	For bulk insert to possibly non-nullable datetime column
	with GETDATE() as default value.

Signed-off-by: Matt McNabb <matthew.mcnabb@vmssoftware.com>
Signed-off-by: Matt McNabb <matthew.mcnabb@vmssoftware.com>
Signed-off-by: Matt McNabb <matthew.mcnabb@vmssoftware.com>
Signed-off-by: Matt McNabb <matthew.mcnabb@vmssoftware.com>
@mmcnabb-vms
Copy link
Contributor Author

mmcnabb-vms commented Jan 27, 2026

Re. the old Ucko PRs - A large part of them was focused on having freebcp in reject malformed data . There are quite a few cases in freebcp in where it will just proceed and go off the rails, including reading the wrong number of bytes from the input file and getting out of sync, in response to various cases of malformed data. Ucko had to try and fix that for security reasons. That would be a separate topic to this PR though. I didn't see anywhere in those PRs that addressed ASE defaults.

mmcnabb-vms and others added 13 commits January 27, 2026 17:17
Signed-off-by: Matt McNabb <matthew.mcnabb@vmssoftware.com>
Signed-off-by: Matt McNabb <matthew.mcnabb@vmssoftware.com>
Signed-off-by: Matt McNabb <matthew.mcnabb@vmssoftware.com>
Signed-off-by: Matt McNabb <matthew.mcnabb@vmssoftware.com>
	To prepare for adding the Defaults processing

Signed-off-by: Matt McNabb <matthew.mcnabb@vmssoftware.com>
	Validate size is correct

Signed-off-by: Matt McNabb <matthew.mcnabb@vmssoftware.com>
Signed-off-by: Matt McNabb <matthew.mcnabb@vmssoftware.com>
Signed-off-by: Matt McNabb <matthew.mcnabb@vmssoftware.com>
    The validation code didn't support multiple hints,
    and it truncated hints with arguments (e.g. ORDER).

    The server will tell us anyway if a hint is invalid
    so it doesn't seem necessary to validate at compile-time.

Signed-off-by: Matt McNabb <matthew.mcnabb@vmssoftware.com>
Signed-off-by: Matt McNabb <matthew.mcnabb@vmssoftware.com>
Signed-off-by: Matt McNabb <matthew.mcnabb@vmssoftware.com>
Signed-off-by: Matt McNabb <matthew.mcnabb@vmssoftware.com>
Signed-off-by: Matt McNabb <matthew.mcnabb@vmssoftware.com>
@mmcnabb-vms
Copy link
Contributor Author

mmcnabb-vms commented Jan 27, 2026

Force pushed to fix linux build errors & missing Signoff message .

I noticed while fixing failure in t0016 that t0016 is actually a BCP test framework with 16 sub-cases of BCP tests. I should be able to replace my new test d_bcp_defaultdate with another case for t0016 , will do that now and force-push if you haven't started reviewing yet.

Am also working on a test that will read the same sources as t0016 but execute by calling freebcp (or even MS/ASE native bcp for reference) instead of using DBlib api

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant