Hi. I am trying to replicate a mysql table using the batch inserts. Here is the code I've written for this:
mysql_cursor = self.mysql.cursor() mysql_cursor.execute("SELECT * FROM" + self.table + "` WHERE " + where)
column_names = [i[0] for i in mysql_cursor.description]
batch = VerticaBatch(
table_name= self.vertica_schema + '.' + self.table,
truncate_table=True,
column_list=column_names,
copy_options={
'DELIMITER': ',',
},
connection = self.vertica,
analyze_constraints= False,
multi_batch= True
)
count = 0
while True:
if count % batch_size == 0:
if count > 0:
print(count, mysql_cursor.rowcount)
error_bool, error_file_obj = batch.get_errors()
if error_bool:
# TODO: Do something with this error
error = error_file_obj.read()
batch.commit()
count += 1
msrow = mysql_cursor.fetchone()
if msrow is None:
break;
batch.insert_list(list(msrow))
error_bool, error_file_obj = batch.get_errors()
if error_bool:
# TODO: Do something with this error
error = error_file_obj.read()
batch.commit()
batch.close_batch()
`
Some times, it logs the warning "Error shutting down task thread!".
When I test this code on an iPython console for the first time, it works well. If I try to make a new class and run this a second time, it hangs after the first batch.commit() call.
Is it possible that threads and pyodbc are buggy? I will try to rewrite pyvertica.batch without threads to see if things get better.
Hi. I am trying to replicate a mysql table using the batch inserts. Here is the code I've written for this:
mysql_cursor = self.mysql.cursor() mysql_cursor.execute("SELECT * FROM" + self.table + "` WHERE " + where)column_names = [i[0] for i in mysql_cursor.description]
`
Some times, it logs the warning "Error shutting down task thread!".
When I test this code on an iPython console for the first time, it works well. If I try to make a new class and run this a second time, it hangs after the first batch.commit() call.
Is it possible that threads and pyodbc are buggy? I will try to rewrite pyvertica.batch without threads to see if things get better.