Batch Size Problem!

[John-8911797110]

Batch Size Problem

First, thanks for your great work!!! 👍👍

I view #2 issue yesterday and understood how to train vecters in TensorGP, but I found OOM problems in training as my vector length >= 10000. Actually, I have hundreds of vector with shape (>=100000, 1). So I wonder how to modify tensorgp/engine.py to satisify train a little Batch.

Appreciate your reply 😊

[AwardOfSky]

Hello @John-8911797110 !

The current version of TensorGP does in fact have some OOM memory problems for large populations on the GPU due to the TensorFlow library memory management. Unfortunately, our testing with dividing the population in smaller batches ran into the same OOM problems.

We're working on a version of TensorGP with PyTorch that, so far, did not exhibit these OOM issues. However, this version is not ready for lauch yet.

In the meantime, I'd advise you to either try to run the populations on a GPU with more VRAM or switch to CPU. If these are not feasible options for you, get back to me with the error message and I might be able to make a work around for your specficic case.

Cheers, and thanks for trying our framework!

[John-8911797110]

Hi @AwardOfSky 😃

I found that OOM problem occured when there is no data scale so that operations increase digital capacity😵. But, I fix this OOM problem myself! 🚀, I modify some code and add sample mechanism in traning process. I can set batch_size=100000 and train my (200000, 100) dataset with 300 and more populations.

Thanks for your advice and expecting your work with PyTorch 👍

[mangomqy]

Batch Size Problem

First, thanks for your great work!!! 👍👍

I view #2 issue yesterday and understood how to train vecters in TensorGP, but I found OOM problems in training as my vector length >= 10000. Actually, I have hundreds of vector with shape (>=100000, 1). So I wonder how to modify tensorgp/engine.py to satisify train a little Batch.

Appreciate your reply 😊

Hi @John-8911797110 ,I have some trouble to train personal data. Could you help me with import personal train_data?

[John-8911797110]

Hello @mangomqy 😄 , you can post your specific problem in this issue, and I'm glad to help you!

[mangomqy]

Thanks @John-8911797110, for example, I want to get a typical symbolic regression y = X0^2 - X1^2 + X1 -1. I generate train_data and test data. How do I train this train_data and anticipate the test_data

`
X_train = rng.uniform(-1, 1, 100).reshape(50, 2)
y_train = X_train[:, 0]**2 - X_train[:, 1]**2 + X_train[:, 1] - 1

X_test = rng.uniform(-1, 1, 100).reshape(50, 2)
y_test = X_test[:, 0]**2 - X_test[:, 1]**2 + X_test[:, 1] - 1`

[John-8911797110]

mangomqy

issue #2 maybe has your answer to train vector data, follow #2 (comment) you can input your X to engine.terminal.set, y (target) to engine.target.

[mangomqy]

mangomqy

issue #2 maybe has your answer to train vector data, follow #2 (comment) you can input your X to engine.terminal.set, y (target) to engine.target.

Thanks @John-8911797110 , I still have questions, if I use the iris data, How do I input them?

5.1,3.5,1.4,0.2,Iris-setosa
4.9,3.0,1.4,0.2,Iris-setosa
4.7,3.2,1.3,0.2,Iris-setosa

data = {'target': [], 'col2': [], 'col3': []}
data['target'] = [1,1,1]
data['col2'] = [5.1,3.5,1.4,0.2]
data['col3'] = [4.9,3.0,1.4,0.2]
data['col4'] = [4.7,3.2,1.3,0.2]

or

data['col2'] = [5.1,4.9,4.7]
data['col3'] = [3.5,3.0,3.2]
data['col4'] = [4,4,3]
data['col5'] = [0.2,0.2,0.2]

[John-8911797110]

@mangomqy
if you get a iris dataframe(iris_df) like this:

     col0  col1  col2  col3  target
0     5.1   3.5   1.4   0.2       0
1     4.9   3.0   1.4   0.2       0
2     4.7   3.2   1.3   0.2       0
3     4.6   3.1   1.5   0.2       0
4     5.0   3.6   1.4   0.2       0

you could add following code:

engine = Engine(target = iris_df["target"].values, 
         target_dims = [len(iris_df), 1]...

ts = engine.get_terminal_set()
ts.remove_from_set("x")
for col in iris_df.columns:
    ts.add_to_set(col, iris_df[col].values, engref=engine)