Please am trying to run a code amd am getting memory error.
Kindl assist oo.
You just need to have 32 GB of RAM on your computer.
More useful answer: it might help to ask yourself if you can decrease somehow the size of this dataset which you’re trying to load.
It seems like your dataset has 52436 columns, which is kinda weird.
Anyway:
We need to know:
- what kind of dataset you’re trying to load
- what data it contains
- are those images or what? over 52k columns seems a bit weird.
- what are you using to load the data before encoding the categorical columns? It seems like you’ve been able to load anything without a problem, until the encoding happens (perhaps the categorical columns have too much possible values?)
The data am using is
The data doesn’t seem to have that much columns.
One possible solution (which doesn’t involve removing anything) I could offer (already described in the link I gave in previous answer, which seems to be neglected anyway): change the encoded columns to an int8 datatype by using dtype argument of OneHotEncoder (that’s what I assume you’re using).
72983 rows * 52436 columns requires around 3.8 billion values.
If they would require only a byte of information, this is 3.8 billion bytes (around 3.56 GB of RAM). But the result would be probably a bit bigger than that, because not all of these columns are categorical encodings.
If the above doesn’t work, then the only option is to remove columns which might be unnecessary for the outcome. You can use feature importance (from second assignment) to decide which columns you might decide without harming the result too much.