Stata Time Series Operators (Lags, Leads, and Indices)

For reference because I always end up having to look this up:

lag: L, L2, … : gen lag2Var = L2.var

lead: F, F2, … : gen lead2Var = F2.var

difference: D, D2, … : gen diffofdiffVar = D2.var

seasonal difference: S, S2, … : gen seadiffVar = S2.var

I have never used the seasonal difference. I assume it depends on the seasonal frequency defined via tsset.

D2 is defined as (y[t]-y[t-1]) - (y[t-1]-y[t-2]), and not y[t]-y[t-2]. For y[t]-y[t-2], if there are no gaps in the time variable, you can use y[_n]-y[_n-2].

Reference: Forecasting in STATA: Tools and Tricks [Economic Forecasting (Econ 390) Spring 2010]

Follow up: Hausman Test on Stata for RE vs. FE

Some follow up comments from a previous post on the Hausman test in the RE/FE context:

RE assumes that Cov(a[i],x[it])= 0. {The model here is: Y[it] = b*x[it] + e[it], where e[it] = a[i] + u[it]).} The coefficient estimates from using RE and FE have the following properties depending on whether that assumption is true:

	b[RE]	b[FE]
Cov(a[i],x[it])=0	efficient, consistent	inefficient, consistent
Cov(a[i],xpit]!=0	inconsistent	consistent

In words, if Cov(a[i],x[it])=0 is true, then since the coefficient estimates are both consistent, they are "likely" to be close and so the difference between b[RE] and b[FE] is small. If Cov(a[i],xpit]!=0, then since b[RE] is inconsistent, you would like that the difference is likely to be large.

The test statistics is (b[RE]-b[FE])' v^(-1) (b[RE]-b[FE]), where v is the estimated asymptotically variance of (b[RE]-b[FE]) with Chi square distribution.

Reference: Hausman test (fe vs. re), use robust SE or not? [Statalist], 
     Stata 12 help for hausman [Stata]

Getting Yahoo Finance Dividend Data in CSV Format and Plotting the Dividend Data in Stata

Here is some code to create a Stata program to get dividend data from Yahoo Finance and plotting it:

program define sdchart
    preserve
    clear
    insheet using "http://ichart.finance.yahoo.com/table.csv?s=`0'&a=00&b=01&c=1900&d=11&e=31&f=2099&g=v&ignore=.csv"
    gen tempDate = date(date, "YMD")
    drop date
    gen date = tempDate
    format date  %td
    drop tempDate
    line dividend date
    drop dividend date
   restore
end

If you put the http URL in your web browser, and replaced `0’ with a stock ticker (like "INTC"), you will be returned with a CSV file with Intel’s dividend. For example, click the following:

http://ichart.finance.yahoo.com/table.csv?s=INTC&a=00&b=01&c=1900&d=11&e=31&f=2099&g=v&ignore=.csv 

The rest of the code just takes the csv file data and plots it in Stata. I like using Stata because

1. I have been using Stata a lot lately for my work, and
2. that it can download web data (like those from Yahoo Finance) directly within the program.

So if you run sdchart INTC, you will get:

I have the code in my profile.do file in the Stata startup directory.

Reference: How to download the history of dividend payments for stocks in the US market [Quantshare]

Stata Code Snippets: Plotting Multiple Series with Different Colors and Customized Legend for Different Observations

Here are some Stata code snippets for plotting multiple series. An example using the auto dataset that comes with Stata:

sysuse auto
twoway (scatter price weight if foreign == 1, mcolor(`f0')) (scatter price weight if foreign == 0, mcolor(`f1')),legend(lab(1 "Foreign") lab(2 "Domestic"))

The resulting graph shows domestic cars in the late 70s tend to be heavier for various price points.

Reference: st: RE: –twoway scatter- different colors for different observations [Statalist: The Stata Listserver], Stata Learning Module Graphics: Combining Twoway Scatterplots [UCLA Academic Technology Services]

Stata Code Snippets: Finding Duplicates in Observations

Here are some code Stata code snippets for checking duplicates in your panel data. (I have to figure out how to do this because my panel dataset have duplicates, and therefore I cannot do a tsset on the data) The command to use is duplicates.

To list any duplicate observations, run:

duplicates list

For the duplication issue I have with tssset, I specific the two relevant panel variables

duplicates list termtype dateIndex

Now, to create a dummy variable isdup to mark the duplicates

duplicates tag termtype dateIndex, gen(isdup)

In my case, I just manually remove the duplicates using the graphical editor:

edit if isdup

Or, to just remove it using the command

duplicates drop termtype dateIndex

An example:

sysuse auto
keep mpg rep78
duplicates list
duplicates list rep78
duplicates tag rep78, gen(isdup)
list if isdup

Stata Code Snippets: Summarize Statistics by Category

I calculate summary statistics by categories in SAS a lot - like finding the annual average of a variable by security type. Here are some code snippets for doing that in Stata.

The command to use is collapse. For example, the following code calculates the average mpg and average weight between foreign and domestic cars from the AUTO dataset.

    sysuse auto
    collapse(mean) mpg (mean)weight, by(foreign)

Unlike SAS, which allows you to output the summary data to a different dataset, Stata alters the dataset.

To make a prettier table with summary data, use the tabdisp command:

tabdisp foreign, cell(mpg weight) format(%9.2f)

If you run the code, you will find that, as expected, foreign automobiles are lighter and have higher mpg in the late 70s according to the data.

Hausman Test on Stata for RE vs. FE

To test whether the fixed effects (FE) or random effects (RE) model is more appropriate for a panel data regression (using xtreg here), do the following:

xtreg y x1 x2 x3, fe
estimates store FE
xtreg y x1 x2 x3, re
estimates store RE
hausman FE RE

If p < 0.05, use the FE model.

This is taken from the discussion here. Note the discussion and the use of xtoverid in some situations.

Another test for FE vs RE using xtgls and lrtest is discussed here and described here.

Edit: I have a follow up post on this here.

Reference: Hausman test (fe vs. re), use robust SE or not? [Statalist], 
     Stata 12 help for hausman [Stata]

Maximum “Set MEM” Size for 32-bit Stata is about 1500MB

As the ability to capture information grows, so do the sizes of economics data that economists typically handle. A typical (32-bit) version of Stata can at most allocate about 1500MB of memory to load your data. So if the size of your data gets over 1.5GB, and you are a Stata user, you might see the “Op. sys. refuses to provide memory” error.

Base on some testing I did with my school’s setup, 2 things constraint the amount of memory you can allocate using the “set mem” command.

1. 32-bit version of Stata can in theory allocate 2000MB of data, regardless of the amount of RAM on your computer. Realistically, because the operating system uses memory also, you can allocate around 1500MB. [Note that by extension, your operating system needs to be a 64-bit operating system to allocate more than 1500MB of memory.]
2. The RAM you have on your computer. If you have 4GB of RAM, even with the 64-bit version of Stata, you cannot allocate more than 4GB.  From what I can tell, Stata does not seem to let you allocate more than the amount of physical RAM on your computer.

The solution is to

1. get the 64-bit version of Stata and more RAM to relax whichever constraint you are facing. Stata IC, SE, and MP have 64-bit versions. Or,
2. learn SAS. SAS is the opposite of Stata in that SAS does everything on disk, while Stata does everything in memory. The tradeoff is that SAS tends to be slower, but can operate on very large datasets. I have used it on 50GB datasets.

References: FAQ: Large datasets under Windows [Stata],
     Memory Limits for Windows Releases (Windows) [Microsoft]