Regression Methods

Methods control the way variables are included into the regression. Quite often you will just want to compute a regression model you have specified, i.e. a dependent variable explained by several independent variables.


or the equivalent syntax

REGRESSSION /Dependent= InfantMortality  
    /ENTER=GDP_PCap UrbanPop illiteracy.

This example uses in fact the ENTER method (default with the menu system).

If you only need to compute regression equations you have specified in advance and want to include all variables, there is no need to read any further.

Including variables into the equation

In the straightforward example above, all variables are introduced in one step (forced) in order of decreasing tolerance. To be included independent variables must pass both tolerance and minimum tolerance tests. is the proportion of the variance of a variable in the equation that is not accounted for by other independent variables in the equation. The minimum tolerance of a variable not yet included in the equation is the smallest tolerance any variable already included in the equation would have, if the variable being considered were included in the analysis. It can happen that some variables are not included, because they do not satisfy the minimum tolerance (defaults to 0.0001).

Sometimes it is useful to control the sequence of inclusion and specify of variables, the first block being considered, before the second etc. and obtain coefficients and statistics for each block.

In the regression dialog when you specify independents, you specify a (model) block. The first model you specify is the first block and Next lets you specify the next block. When blocks are defined the Previous and Next button lets you navigate from one block to the next.

The corresponding syntax is:

  /DEPENDENT <dependent-variable>
  /METHOD=ENTER <Independents-block1>
  /METHOD=ENTER <independents-block2>.
Specifying regression methods

Besides ENTER, several other methods are available to build models, controlling how variables are included into a model; note also that several methods can be combined. The main goal of this methods is to determine the best subset of variables explaining a dependent variable.

Specify a method: Regression menu

from the Regression dialog you can choose one of the methods shown to the left.

Specify a method: Syntax
/METHOD= | STEPWISE varlist [...] [/...]
         | FORWARD varlist 
         | BACKWARD varlist 
         | ENTER varlist
         | REMOVE varlist 
         | TEST(varlist)(varlist)...

Note that the keyword itself /METHOD is optional.

There are basically two types of methods, methods that handle blocks of variables and methods.

For all methods variables must pass the tolerance criterion to be entered in the equation. The default tolerance level is 0.0001. Note that a variable is not entered if it would cause the tolerance of another variable already in the model to drop below that level.

Methods handling blocks of variables

These methods use only the tolerance criterion.

Stepwise methods

Stepwise methods include or remove one independent variable at each step, based (by default) on the probability of F (p-value); alternatively the F value can be used instead. The limits for the criteria controlling variable inclusion or removal can be specified by defining probabilities for F-to-enter/F-to-remove (or values of F-to-enter/F-to-remove).

If you use the dialog, you can change these criteria in the Options dialog (defaults shown on the screenshot).

If you are using syntax, these values can be set using the /CRITERIA keyword.

The following three stepwise methods are available.

Regression  /Dependent= InfantMortality   /stepwise=GDP_PCap UrbanPop illiteracy.

Each step adds a variable to equation until all variables specified are entered.

Regression  /criteria pout(0.1) /Dependent= InfantMortality
          /method=backward GDP_PCap UrbanPop illiteracy.

All variables are entered first, then considerd for elimination based on the probability of F-to-remove, specified here as POUT(0.1)