As the ability to capture information grows, so do the sizes of economics data that economists typically handle. A typical (32-bit) version of Stata can at most allocate about 1500MB of memory to load your data. So if the size of your data gets over 1.5GB, and you are a Stata user, you might see the “Op. sys. refuses to provide memory” error.
Base on some testing I did with my school’s setup, 2 things constraint the amount of memory you can allocate using the “set mem” command.
- 32-bit version of Stata can in theory allocate 2000MB of data, regardless of the amount of RAM on your computer. Realistically, because the operating system uses memory also, you can allocate around 1500MB. [Note that by extension, your operating system needs to be a 64-bit operating system to allocate more than 1500MB of memory.]
- The RAM you have on your computer. If you have 4GB of RAM, even with the 64-bit version of Stata, you cannot allocate more than 4GB. From what I can tell, Stata does not seem to let you allocate more than the amount of physical RAM on your computer.
The solution is to
- get the 64-bit version of Stata and more RAM to relax whichever constraint you are facing. Stata IC, SE, and MP have 64-bit versions. Or,
- learn SAS. SAS is the opposite of Stata in that SAS does everything on disk, while Stata does everything in memory. The tradeoff is that SAS tends to be slower, but can operate on very large datasets. I have used it on 50GB datasets.
References: FAQ: Large datasets under Windows [Stata],
Memory Limits for Windows Releases (Windows) [Microsoft]