Import-O-Matic (WIP)

Import-O-Matic is a WIP Rails 4 gem for import data to active record models.

Features:

• Supports multiple formats (only csv now 😞).
• Clean configuration (external class).
• Text logs.
• Multiple actions per row (create, update, delete).
• Globalize support.

Import-O-Matic is in development for a ruby 2 and rails 4 project, so it is only tested in this environment at the moment.

💾 Install

Add the gem to your gemfile:

  gem 'import_o_matic', github: 'simplelogica/import_o_matic'

Add to Import-O-Matic your model:

  class MyModel < ActiveRecord::Base
    ...
    import_o_matic
    ...
  end

Ask your model:

  MyModel.importable?

Import data:

Add to Import-O-Matic your model:

  MyModel.import_from_file 'path/to/file'

Or use a default local file (see file_path option):

  MyModel.import_from_local

Take a look at log/imports for process information.

By default, Import-O-Matic creates a new instance of the model and try to map each column in the importation file with an attribute with same name.

The return of import_from_local and import_from_file will be a hash with the relative log path, counters of the actions processed:

{
  :log=>
  "log/imports/model_name/19000101T0000Z_model_name_import.log"
  :total => 100, # Total rows processed
  :create => 50, # Rows created
  :update => 40, # Rows updated
  :errors => 10, # Rows with errors (details in log file)
}

🎮 Configure

Create an import config class in app/imports:

  class MyModelImport < ImportOMmatic::Options
    ...
  end

And use it in your model:

  class MyModel < ActiveRecord::Base
    ...
    import_o_matic MyModelImport
    ...
  end

📖 Configure options:

Select format (default :csv):

  import_format :csv

Select format with options (default { headers: true }):

  import_format csv: { headers: true }

Set a default file for local import (with import_from_local method):

  file_path 'path/to/file'

Set the max logs stored in the log folder (default 10):

  max_logs 10

Strip blanks arround columns after read (default false):

  strip_values

Select column names from file to import with an array:

  import_matches [:attribute1]

Map model attribute names with column names with a hash:

  import_matches attribute1: :column1

Apply a function to values before update the attribute with a Proc:

  import_transforms attribute1: ->(value) { value.next }

Apply a function to a value before update the attribute with a method:

  import_transforms attribute1: :plus_one

  def plus_one value
    value.next
  end

You can assign multiple columns to an attribute, but it needs a transformation or it will use first column by default. Take care with the params order, it needs to be the same of the columns declaration:

  import_matches full_name: [:name, :last_name]
  import_transforms full_name: :join_name

  def join_name name, last_name
    "#{name} #{last_name}"
  end

Apply procs or methods (or both like the example) to the model before import action:

  before_actions :plus_one,
    ->(element) { element.string_attribute = 'after' }

  def plus_one element
    element.integer_attribute += 1
  end

Apply procs or methods (or both like the example) to an element after import action:

  after_actions :plus_one,
    ->(element) { element.update_attributes string_attribute: 'after' }

  def plus_one element
    element.update_attributes integer_attribute: element.integer_attribute.next
  end

Apply a named scope of the model when an item is search in incremental imports

  use_scope :custom_scope

Skip validations when model is created or updated (.save(validate: false))

  skip_validations

You can use different actions (create, update or delete) for incremental imports. You need a column for set the relation between import data and existing objects, and another column with the action. When the import can´t match an action, it uses create by default.

• Default relation column: id
• Default relation model attribute: id
• Default action column: action
• Default actions values: { create: "ADD", update: "UPDATE", destroy: "REMOVE" }

Set relation column:

  incremental relation: :column1

Set map relation column with relation model attribute:

  incremental relation: { column1: :attribute1 }

Set action column:

  incremental action_column: :action

Set action column values:

  incremental actions: { create: "ADD", update: "UPDATE", destroy: "REMOVE" }

You just need to add the action column to the import file when you want remove an element. Create and update actions are automatically used if the element is found by the incremental id (update) or not (create).

🌍 Globalize 4 support

  globalize

You can import globalize translated attributes with this option. It's not configurable at the moment.

NOTE: The gem will add 'accepts_nested_attributes_for :translations' to model if is not already set.

The gem gets automatically the model translated attributes and search for columns named "attribute-locale" for each available locale. For example:

  class MyModel < ActiveRecord::Base
    ...
    transtales :name
    ...
    import_o_matic MyModelImport
    ...
  end

  class MyModelImport < ImportOMmatic::Options
    ...
    globalize
    ...
  end

The import looks for name-en, name-es... columns in the file, one for each locale in I18n.available_locales.

✅ TODO:

• Better tests.
• Better logs.
• More ruby and rails versions.
• Configuration for globalize fields.
• Multiple assigns for globalize fields. ...
• Some cool stuff 😞.