helper modules inside execution context

[senny]

We have a lot of data normalization to do in our migration. It would be very helpful to be able to extend the ExecutionContext of the mappings. I think we should make this part of the DataImport API so that we are able to refactor the internals without having to change all the client code.

I'm thinking about something like:

module MyHelperModule
  def normalize(text)
    text.downcase
  end
end

DataImport.helper(MyHelperModule)

[senny]

@stmichael thoughts?

[stmichael]

Sounds interesting. I see the use of that. How about we make beside the global helper modules even definition specific helper modules like:

import 'Animals' do
  helper MyAnimalHelperModule

  mapping 'Name' do
    {:name => normalize_animal_name(arguments[:Name])}
  end
end

[andyt]

I'd find something like this useful. I'm currently struggling to find a way to get some per-import options into a mapping context, without resorting to globals of one form or another, although that's a slightly different use case.

Thanks for an awesome library - it's really proving useful.

[stmichael]

@andyt I'm very glad to hear that this library is of some use. Unfortunately I'm currently very short on time to work on this (as you might have noticed looking at the timestamp of the last commit 😉 ). Feel free to work on this issue if you find some time. You'd make me very happy 😄

[andyt]

One of the acceptance specs is failing (logger_spec.rb#41) due to the log format on my machine being inexplicably different. Ruby version is 1.9.3, which is passing on TravisCI. Are you happy for me to loosen off this spec a bit (check each expected line with a regex)?

[stmichael]

What exactly means "different"? Can you post an example of your log?

[andyt]

My log format includes the time and the level:

logger skip invalid records
Failure/Error: messages.string.strip.should == "Starting to import \"People\"\nRow {:Name=>\"Jack\", :Gender=>\"m\"} skipped since the gender is male"
-Starting to import "People"
-Row {:Name=>"Jack", :Gender=>"m"} skipped since the gender is male
+I, [2013-10-31T10:59:14.015288 #2584]  INFO -- : Starting to import "People"
+I, [2013-10-31T10:59:14.019618 #2584]  INFO -- : Row {:Name=>"Jack", :Gender=>"m"} skipped since the gender is male

One solution would be to explicitly define the formatter.