I often use "who" to check what variables has been defined in Matlab, but never care about how much memory the variables are actually taking. I recently have to work with a 3GB datafile, and my laptop with 8GB of RAM is not cutting it.So how to check the memory size of variables?
Turns out "whos" shows the variables in memory as well as how much memory each is using. Good to know!
Some other notes on processing large csv files
The data I was analyzing has been growing from less than 500MB to over a GB to now about 3GB. The data comes in the form of one big csv file and so I need to do rewrite my Matlab code to be more memory efficient and process the data by pieces. In the Windows environment, some tools to help with handling big text-based data files are split and good old copy.
Spilt, of unix origin that you can also get for Windows, allows you to split large csv files into pieces. The syntax looks something like this:
split -l 350000 data.csv
where data.csv is the file, 350000 specifies the number of lines per file to be split, and it defaults to generating files xaa, xab, etc.
After processing in Matlab, I had to recombine the files back to one result files to be submitted to someone else for further analysis. I use the basic copy command to append the files together. The syntax for that is
copy file1 + file2 + file3 combinedFile.csv