cannot import name 'Hone' from 'hone'

[vagarwal77]

from hone import Hone

def process_mapper():
h = Hone()
schema = h.get_schema('curlim__table__mapping.csv') # returns nested JSON schema for input.csv
result = h.convert('curlim__table__mapping.csv') # returns converted JSON as Python dictionary

def main():
process_mapper()

if name == 'main':
main()

ERROR -
ImportError: cannot import name 'Hone' from 'hone' (/Users/user1/code-repo/test/datamapper/hone.py)

[chamkank]

Correct me if I'm wrong, but it looks like you copied the source code into /Users/user1/code-repo/test/datamapper/. Can you try installing the module via pip (pip3 install hone) instead?

[vagarwal77]

Thanks for pointing...I had not downloaded the code but gave my test file name as hone.py.

Changing it had solved the issue.

Now, i am seeing another issue while parsing attached csv -
Vivek-Agarwals-MacBook-Pro:datamapper vivekagarwal$ python3 hone_test.py
Traceback (most recent call last):
File "hone_test.py", line 15, in
main()
File "hone_test.py", line 12, in main
process_mapper()
File "hone_test.py", line 9, in process_mapper
result = h.convert('countries.csv') # returns converted JSON as Python dictionary
File "/usr/local/lib/python3.7/site-packages/hone/hone.py", line 18, in convert
json_struct = self.populate_structure_with_data(column_schema, column_names, data)
File "/usr/local/lib/python3.7/site-packages/hone/hone.py", line 36, in populate_structure_with_data
exec("json_row"+key_path+"="+"'"+cell+"'")
File "", line 1
json_row['euname']='COTE D'IVOIRE'
^
SyntaxError: invalid syntax

[vagarwal77]

Thanks for pointing...I had not downloaded the code but gave my test file name as hone.py. Changing it had solved the issue. Now, i am seeing another issue while parsing attached csv - Vivek-Agarwals-MacBook-Pro:datamapper vivekagarwal$ python3 hone_test.py Traceback (most recent call last): File "hone_test.py", line 15, in main() File "hone_test.py", line 12, in main process_mapper() File "hone_test.py", line 9, in process_mapper result = h.convert('countries.csv') # returns converted JSON as Python dictionary File "/usr/local/lib/python3.7/site-packages/hone/hone.py", line 18, in convert json_struct = self.populate_structure_with_data(column_schema, column_names, data) File "/usr/local/lib/python3.7/site-packages/hone/hone.py", line 36, in populate_structure_with_data exec("json_row"+key_path+"="+"'"+cell+"'") File "", line 1 json_row['euname']='COTE D'IVOIRE' ^ SyntaxError: invalid syntax From: Cham <notifications@github.com> Reply-To: chamkank/hone <reply@reply.github.com> Date: Monday, June 1, 2020 at 7:43 PM To: chamkank/hone <hone@noreply.github.com> Cc: Vivek Agarwal <VAgarwal@concertohealthai.com>, Author <author@noreply.github.com> Subject: Re: [chamkank/hone] cannot import name 'Hone' from 'hone' (#16) This message originated outside of Concerto Health AI , please use caution before opening attachments, clicking links and replying to this message. Correct me if I'm wrong, but it looks like you copied the source code into /Users/user1/code-repo/test/datamapper/. Can you try installing the module via pip (pip3 install hone) instead? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub<https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fchamkank%2Fhone%2Fissues%2F16%23issuecomment-637183451&data=02%7C01%7Cvagarwal%40concertohealthai.com%7C952169e6d58d42a97c6e08d806859c4f%7C42831aea6fc242368bfdcf4136c882fa%7C1%7C0%7C637266518217504393&sdata=7%2Fovbvzw%2F0V9K0GsnV%2BcBxelmhG%2FsI8NzWLMez0eVhc%3D&reserved=0>, or unsubscribe<https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FALCPUMFKWAE5WTN7NFJGIK3RUQ4KXANCNFSM4NQHCEVQ&data=02%7C01%7Cvagarwal%40concertohealthai.com%7C952169e6d58d42a97c6e08d806859c4f%7C42831aea6fc242368bfdcf4136c882fa%7C1%7C0%7C637266518217504393&sdata=9kSV1dPE%2BacjJP%2FGHrGXbxPNMSAtVqu8GLR4Q4xg58M%3D&reserved=0>.

[chamkank]

That's definitely a bug. Looks like the issue is that the single quote in 'COTE D'IVOIRE' isn't escaped properly. I'll get to fixing it later today, thanks for catching this!

[vagarwal77]

Thanks!!! I have removed single quotes in CSV and it is working like a charm.

I have a another question (and hopefully last)
I have a specific requirement like -

1. I have a CSV in-hand which user can import.. format of this CSV is not fixed/define.
2. I have a nested destination json schema which have dict, array, object, string act (sample as https://clinicaltrials.gov/api/query/full_studies?expr=NCT03322540&fmt=json)
3. I would like to have a mapper file (may be dictionary) which can be used to map CSV column to the destination json attributes... (This mapper file is required as each client will have its own CSV whose format (column names and all) is client specific). With this mapper, it can server as a generic solution for all the clients by just change the mapper file.

Can hone repo code be modified to accommodate this use case? or any other pointers for me to make progress?

Please suggest as it will be very helpful.

[chamkank]

I have a nested destination json schema which have dict, array, object, string act

• Hone currently only converts values to strings, but type casting to JSON data types is a planned feature (see Support for type casting #11).

I would like to have a mapper file (may be dictionary) which can be used to map CSV column to the destination json attributes...

• Hone generates the JSON structure automatically based on delimiters in the column names. so there's no real way to specify a strict schema. I think the best way to do this is to just read the CSV into a Python list, convert it to a dictionary, write code to make certain fields in the dictionary nested based on your mapper file, and then finally serialize the dict to JSON using json.dumps You could just swap the auto-generated schema with the schema from your user-defined mapper file. In the Hone.convert method, you just need to replace column_schema with your own schema.

  • def convert(self, csv_filepath):
        self.set_csv_filepath(csv_filepath)
        column_names = self.csv.get_column_names()
        data = self.csv.get_data_rows()
        column_schema = self.generate_full_structure(column_names)
        json_struct = self.populate_structure_with_data(column_schema, column_names, data)
        return json_struct
    

I'm not sure if that was helpful, but please let me know if you have any other questions. I'd be more than happy to help where I can!

[vagarwal77]

Thanks .. very helpful...
It is what i have created a mapper file
mapper = {

csv_column_name json_attribute attribute_type json_attribute_parent json_attribute_parent_type

'euname':       {'Keyword', 'Array', 'KeywordList', 'Object'},
'linked_country': {'Condition', 'Array', 'ConditionList', 'Object'},
'country': {'BriefSummary', 'String', 'DescriptionModule', 'Object'}

}

And have converted csv to list of dictionary
def csv_tolistofDict():
with open('countries.csv', 'r') as read_obj:
# pass the file object to DictReader() to get the DictReader object
dict_reader = DictReader(read_obj)
# get a list of dictionaries from dct_reader
list_of_dict = list(dict_reader)
# print list of dict i.e. rows
print(list_of_dict)
eg - OrderedDict([('euname', 'DOMINIQUE'), ('linked_country', ''), ('country', 'Dominica')])

TODO -
Some how, i need to iterate each key in this list of dict and find a match in the mapper with key to key comparison....... than fetch the json_attribute name and replace its value with matching list of dict value....
I am not sure how i will handle the json datatype and above condition....

[chamkank]

I am not sure how i will handle the json datatype and above condition....

• Support for type casting #11 will add support for JSON arrays, numbers, objects, etc.
• Could you clarify what "above condition" is referring to?