Hyperparameter tuning for regression models and Graph CNNs

#1

Dear DeepChem community,

I know this issue has been reported and discussed previously, but I don’t know if it has been resolved.

I am trying to tune hyperparameters for my regression and classification models.

While the code reported here:

https://github.com/deepchem/deepchem/blob/master/examples/tutorials/Advanced_Model_Training.ipynb

works well with my classification model for tuning hyperparameters, it fails to work for my regression model even after adjusting some of the parameters accordingly.

Is there are standard code for hyperparameter tuning I could use for regression models, like the one in the link above for classification?

Currently, my code for hyperparameter tuning for regression model looks like this:

params_dict = {
‘n_tasks’: [1],
‘layer_sizes’: [[500], [1000], [1000, 1000]],
‘dropouts’: [0.2, 0.5],
‘learning_rate’: [0.001, 0.0001]
}
optimizer = dc.hyper.GridHyperparamOpt(dc.models.GraphConvModel)

metric = dc.metrics.Metric(dc.metrics.pearson_r2_score)

best_model, best_hyperparams, all_results = optimizer.hyperparam_search(
params_dict, train_dataset, valid_dataset, metric, [normalizer])

And the error I get is:
ValueError: y has more than n_class unique elements.

#2

Are you sure that’s really the code you’re running? I can’t figure out any way that code could produce that error! That exception is thrown by the routine that converts labels to one-hot-encoding, which is used by classification metrics, but not by pearson_r2_score.

Could you post the full stack trace from the exception? That will show exactly what path it’s taking to get there.

#3

Thanks for the reply Peastman.

Here’s the full stack trace from the exception:

ValueError Traceback (most recent call last)
in
9 metric = dc.metrics.Metric(dc.metrics.pearson_r2_score)
10
—> 11 best_model, best_hyperparams, all_results = optimizer.hyperparam_search(
12 params_dict, train, valid, metric, [normalizer])

~\Anaconda3\lib\site-packages\deepchem\hyper\grid_search.py in hyperparam_search(self, params_dict, train_dataset, valid_dataset, metric, output_transformers, nb_epoch, use_max, logdir, **kwargs)
151 # mypy test throws error, so ignoring it in try
152 try:
–> 153 model.fit(train_dataset, nb_epoch=nb_epoch) # type: ignore
154 # Not all models have nb_epoch
155 except TypeError:

~\Anaconda3\lib\site-packages\deepchem\models\keras_model.py in fit(self, dataset, nb_epoch, max_checkpoints_to_keep, checkpoint_interval, deterministic, restore, variables, loss, callbacks, all_losses)
318 The average loss over the most recent checkpoint interval
319 “”"
–> 320 return self.fit_generator(
321 self.default_generator(
322 dataset, epochs=nb_epoch,

~\Anaconda3\lib\site-packages\deepchem\models\keras_model.py in fit_generator(self, generator, max_checkpoints_to_keep, checkpoint_interval, restore, variables, loss, callbacks, all_losses)
395 # Main training loop.
396
–> 397 for batch in generator:
398 self._create_training_ops(batch)
399 if restore:

~\Anaconda3\lib\site-packages\deepchem\models\graph_models.py in default_generator(self, dataset, epochs, mode, deterministic, pad_batches)
982 pad_batches=pad_batches):
983 if y_b is not None and self.mode == ‘classification’:
–> 984 y_b = to_one_hot(y_b.flatten(), self.n_classes).reshape(
985 -1, self.n_tasks, self.n_classes)
986 multiConvMol = ConvMol.agglomerate_mols(X_b)

~\Anaconda3\lib\site-packages\deepchem\metrics\metric.py in to_one_hot(y, n_classes)
398 raise ValueError(“y must be a vector of shape (N,) or (N, 1)”)
399 if len(np.unique(y)) > n_classes:
–> 400 raise ValueError(“y has more than n_class unique elements.”)
401 N = np.shape(y)[0]
402 y_hot = np.zeros((N, n_classes))

ValueError: y has more than n_class unique elements.

#4

Here’s your problem:

983 if y_b is not None and self.mode == ‘classification’:
–> 984 y_b = to_one_hot(y_b.flatten(), self.n_classes).reshape(
985 -1, self.n_tasks, self.n_classes)

You’re fitting a classification model, not a regression model. GraphConvModel has a mode argument that tells it which type of model to create. Add 'mode': ['regression'] to params_dict and it should work.

#5

This worked like a charm. Many thanks :slight_smile: