User Interface

Sources( 
    T           = DefaultSourceTypes; 
    consumers   :: Any = nothing, 
    products    :: Any = nothing, 
    marketsizes :: Any = nothing, 
    draws       :: Any = nothing,
    user        :: Any = nothing
)

Creates a GrumpsSources object with source type entries of type T where the entries are provided in the optional parameters.

Grumps (potentially) uses four data sources: a data source for consumer-level data, one for product-level data, one for market size information, and one for demographic draws. See Spreadsheet formats for data layouts. Only the product-level data are required, but are by themselves insufficient. For instance, for BLP95 one needs information on products, market sizes, and demographics; for the Grumps CLER estimator one needs all four types of data; for a multinomial logit both consumer and product information are needed. Not all data are needed for all markets. For instance, it is ok for some estimators for there to be consumer-level data in some markets but not others.

The T argument is mostly there to allow for future expansion, so the description below applies to the case in which T = DefaultSourceTypes.

By default, the entries can be nothing, a string, a DataFrame, or a SourceFileType. If an entry is nothing, it means that no such data is to be used. If an entry is a string then it is converted to a SourceFileCSV entry with comma delimiter where the string name is the file name. To use other source file types, create a SourceFileType first. A DataFrame can be passed, also. In all cases other than nothing, data will eventually be (converted to) a DataFrame and parsed from that.

The consumers variable specifies where consumer-level data can be found, the products variable is for the product-level data, marketsizes is for market sizes, and draws is for demographic draws; user has not been implemented yet.

Use the Variables() method to specify the way the data sources are formatted and the specification to estimate.

Variables( ; 
  market              :: Symbol = :market,
  product             :: Symbol = :product,
  choice              :: Symbol = :choice,
  interactions        :: Mat{Symbol} = [],
  randomcoefficients  :: Vec{Symbol} = [],
  outsidegood         :: String = "outsidegood",
  share               :: Symbol = :share,
  marketsize          :: Symbol = :N,
  regressors          :: Vec{Symbol} = [],
  instruments         :: Vec{Symbol} = [],
  dummies             :: Vec{Symbol} = [],
  nuisancedummy       :: Symbol = :none,
  microinstruments    :: Mat{Symbol} = [],
  user                :: Mat{Symbol} = []
    )

This method is used to specify regressors, instruments, random coefficients, interactions, variable labels, etcetera, from the sources you have specified in Sources(). It creates an object of type GrumpsVariables.

For instance, the option market specifies the column heading of the column containing the market descriptor (name). The same is true for all other arguments, except outsidegood which describes the spreadsheet entry that indicates the product is an outside good. The same label for the outside good should be used in all spreadsheets and all markets. Outside good entries should only be used in the consumer micro data and then only if there actually are consumers in the micro data choosing the outside good. All descriptors are case and space sensitive.

There is a separation between variables that go into the individual consumer utility and ones that only go into "mean utility". For instance, interactions tells Grumps which interaction terms to use and randomcoefficients which product level regressors are hit with a random coefficient. By contrast, regressors go into the mean utility component and are regressors in the "second stage" (where β is recovered). One can use the special symbol :constant to indicate a constant is to be used; the spreadsheet need not include a column with that heading.

Note that there are three ways that dummy variables can be entered as second stage regressors, which can be useful to incorporate brand, firm, or market effects into δ. The first is via regressors, in which case the onus is on the user to ensure that they have the correct numerical values. The second possibility is via the dummies argument. For each symbol passed via the dummiesi pass one via the *nuisancedummy argument since it saves both computation time and memory. There can only be at most one categorical variable that can be converted to nuisance dummies, but there can be arbitrarily many categories. These dummies and nuisance dummies are automatically assumed to be exogenous and will be included in the instruments, also.

market refers to the variable containing the market indicator in all input datasets. Strings (e.g. "Amarillo, Texas") work best for the market indicators themselves, but it is not a requirement.

product refers to the variable containing the product indicator in the product dataset. Strings (e.g. "Camry") work best for the product indicators themselves, but it is not a requirement.

choice refers to the variable indicating the choice indicator in the consumer level datasets. Strings work best for the choice indicators themselves, but it is not a requirement. These should take the values of the product column in the products data set or of the outsidegood.

interactions refers to the variables indicating consumer and product variable interactions (each row contains consumer variable, product variable)

randomcoefficients refers to the product level variables that have a random coefficient on them

outsidegood refers to the label used for the outside good

share refers to the label used for the product level share; these are shares where the denominator includes the outside good

marketsize refers to the size of the market (number of people)

regressors refers to the label used for the second stage regressors

instruments refers to the label used for the second stage instruments

dummies refers to discrete variables to be converted to second stage dummy regressors and instruments

nuisancedummy refers to at most one variable to be converted to a second stage dummy regressors and instrument whose coefficient value is of no interest

microinstruments refers to micro instruments, which are only relevant for gmm style procedures

user refers to a list of variables to be added to the consumer-product interactions using a user-specified procedure. This is only needed if the Grumps specification itself does not suffice: see Extending Grumps for details.

Variables( 
    microspec           :: String,
    macrospec           :: String,
    dummyspec           :: String = "",
    nuisancedummyspec   :: String = "";
    market              :: String = "market",
    product             :: String = "product",
    outsidegood         :: String = "outsidegood",
    marketsize          :: String = "N",
    microinstruments    :: String = "",
    user                :: String = ""
    )

This method is used to specify regressors, instruments, random coefficients, interactions, variable labels, etcetera, from the sources you have specified in Sources(). It creates an object of type GrumpsVariables. There is another method by the same name that accomplishes much the same thing, but has a different user interface.

The method described here takes string arguments, including two mandatory ones, whereas the other method takes optional arguments only, mostly symbols, vectors of symbols, and matrices of symbols that can be passed in arbitrary order using keywords. The current method parses user input before calling the other method.

The option market specifies the column heading of the column containing the market descriptor (name). The same is true for all other arguments, except outsidegood which describes the spreadsheet entry that indicates the product is an outside good. The same label for the outside good should be used in all spreadsheets and all markets. Outside good entries should only be used in the consumer micro data and then only if there actually are consumers in the micro data choosing the outside good. All descriptors are case sensitive.

There is a separation between variables that go into the individual consumer utility and ones that only go into "mean utility". For instance, to specify what goes into individual consumer utility, one could specify

"choice = loginc * msrp + famsize * logfootprint + famsize * van + urban * truck + rc * suv + rc * truck + rc * van"

as the microspec argument to indicate that consumer choice is in the micro data set in a column headed "choice" and that there are four interaction terms and three random coefficients. The interaction terms have the consumer-level variable as the first factor and the product variable as the second argument. In this example the three random coefficients are on the suv, truck, and van variables and these variable names should correspond to the column headings in the product level data set. One can use constant to indicate a constant is used: there is no need to include a constant in one's data.

The macrospec argument takes the form

"share = constant + logmpg + loghp + logfootprint + msrp | constant, logmpg, loghp, logfootprint, logcurbweight, lagplcon"

where share are product-level market shares, everything between = and | represents regressors, and everything after | represents instruments; both regressors and instruments are for the product level moments portion. One can again use constant to indicate a constant is used, which need not be included in one's data.

For the remaining arguments, see the description of the other method Variables().

OptimizationOptions(; 
θopt = OptimOptionsθ(), 
δopt = OptimOptionsδ(), 
threads = GrumpsThreads(), 
memsave = false, 
maxrepeats = 4, 
probtype = :fast,
id = :Grumps
)

Sets the options used for numerical optimization. θopt is used for the external optimization routine, δopt for the internal one. These are both of type OptimOptions; see the OptimOptionsθ and OptimOptionsδ methods for elaboration. The memsave variable is set to false by default; turning it on will reduce memory consumption significantly, but will also slow down computation. The variable maxrepeats may disappear in the future.

There are two ways of computing choice probabilities: robust and fast, specified by passing :robust or :fast in probtype. Fast choice probabilities are the default for good reason.

Finally, specifying id allows one to add callbacks, e.g. user functions that are called on each inner and outer iteration. See the Extending Grumps portion of the documentation.

OptimOptionsθ(; 
f_tol = 1.0e-8, 
g_tol = 1.0e-4, 
x_tol = 1.0e-5, 
iterations = 25, 
show_trace = true, 
store_trace = true, 
extended_trace = true )

Creates and returns an OptimOptions optimization options variable for the outer optimization algorithm, including the function value tolerance, the gradient tolerance, the solution tolerance, the maximum number of iterations, whether to show the trace, whether to store the trace, and whether to keep the extended trace. See the Optim package for details.

The current version of Grumps will largely ignore the trace-related parameters.

OptimOptionsδ( ; 
f_tol = 1.0e-8, 
g_tol = 1.0e-8, 
x_tol = 1.0e-6, 
iterations = 25, 
show_trace = false, 
store_trace = true, 
extended_trace = false )

Creates and returns an OptimOptions optimization options variable for the inner optimization algorithm, including the function value tolerance, the gradient tolerance, the solution tolerance, the maximum number of iterations, whether to show the trace, whether to store the trace, and whether to keep the extended trace. See the Optim package for details.

The current version of Grumps will largely ignore the trace-related parameters.

GrumpsThreads(; 
    blas = 0, 
    markets = 0, 
    inner = 0 
    )

This sets the number of threads to be used subject to a number of caveats. blas refers to the number of BLAS threads, markets to the number of threads in loops over markets, and inner to the number of threads in inner loops. A value of zero forces the automatic selection of the number of threads.

Of these, inner is not currently used at all, market is only used in memsave mode, and blas is used. However, please note that the number of threads used by Grumps altogether is the number of threads passed in via the command line argument (i.e. via the -t switch), where that number does not include the number of BLAS threads set.

DataOptions(; 
    micromode   = :Hog
    macromode   = :Ant
    balance     = :micro
    σ2          = 1.0
    id          = :Grumps
)

Both this method and the one described below specify how Grumps should store its data and what it should store. This one is simpler but has less flexibility. The first three options are best set to their defaults, unless you know what it is you're doing. The σ2 option is the variance of ξ, i.e. the error variance in the product level moments. The id option is used to extend Grumps with other data constructions.

DataOptions(
    VarξInput   :: VarξInput{T},
    VarξOutput  :: VarξOutput = VarξDefaultOutput,
    micromode   :: Symbol = :Hog,
    macromode   :: Symbol = :Ant,
    balance     :: Symbol = :micro,
    id          :: Symbol = :Grumps   
)

Both this method and the one described above specify how Grumps should store its data and what it should store. This one is both more complex and more flexible. The micromode, macromode, and balance arguments are best kept at their defaults, unless you know what it is you're doing. The id option is used to extend Grumps with other data constructions. VarξInput is the variance matrix to be used in the penalty term weight matrix construction. This should be a J by J matrix where J is the number of products across all markets. Acceptable types for VarξInput include UniformScaling{T} (e.g. 1.0 * I ) and AbstractMatrix{T} (sparse matrix is recommended to conserve space, but a dense matrix is allowed). VarξOutput is used to indicate what assumptions on the variance of ξ must be produced in the solution, which can subsequently be used as an input in a second stage if desirable; options are VarξHomoskedastic, VarξHeteroskedastic, VarξClustering, and VarξUser.

const VarξInput{T} = Union{ UniformScaling{T}, AbstractArray{T} }

Type used to characterize the assumption under which the weight matrix for the product level moments component of the objective function should be computed. This is irrelevant for consistency, conformance, or the convergence rate of the estimator but it can affect asymptotic efficiency.

VarξHomoskedastic()

Creates a variable of type VarξHomoskedastic. This is used to indicate that standard errors should be computed under the assumption of homoskedasticity. This choice does not affect efficiency. It also products an estimate of the matrix V(ξ) as part of the solution object, which can be used as an input into a possible second stage.

VarξHeteroskedastic()

Creates a variable of type VarξHeteroskedastic. This is used to indicate that standard errors should be computed under the assumption of heteroskedasticity. This choice does not affect efficiency. It also products an estimate of the matrix V(ξ) as part of the solution object, which can be used as an input into a possible second stage.

VarξClustering( clusteron :: Symbol )

Creates a variable of type VarξClustering. This is used to indicate that standard errors should be computed under the assumption of clustering. This choice does not affect efficiency. It also products an estimate of the matrix V(ξ) as part of the solution object, which can be used as an input into a possible second stage. The argument is the variable one should cluster on, e.g. VarξClustering( :market ) suggests that Grumps should cluster on the variable contained in the column in the products spreadsheet with column heading market.

VarξUser()

Allows the user to specify its own standard error computation procedure. Look at Grumps.Template to see how this is implemented.

StandardErrorOptions(; θ = true, δ = true, β = true )

Specifies which coefficients to create standard errors for. If you are looking for what type of standard errors to produce, look at DataOptions().

Estimator( s :: String )

Creates and returns a GrumpsEstimator type. Grumps is reasonably good at figuring out what it is that you want, so e.g. Estimator( "maximum likelihood" ) gives you the unpenalized Grumps maximum likelihood estimator.

The estimators currently programmed include:

• the full CLER estimator
• a cheaper alternative to CLER that has the same limit distribution
• MDLE, i.e CLER without product level moments
• mixed logit with share constraints
• mixed logit estimator using micro data only
• GMM estimators of the same model (in progress: not recommended)

Estimator( s :: Symbol )

Creates and returns a GrumpsEstimator type.

This is one method of specifying the estimator used. However, it is unforgiving in that the exact symbol used internally must be passed, so the Estimator( s :: String ) method is usually a better choice.

Possible choices include:

:cler the full CLER estimator

:cheap a cheaper alternative to CLER that has the same limit distribution

:mdle MDLE, i.e CLER without product level moments

:shareconstraint mixed logit with share constraints

:mixedlogit mixed logit estimator using micro data only

:gmm GMM estimator of the same model (in progress: not recommended)

Estimators( elaborate = false )

Prints a list of available estimators. The argument indicates whether a lot of features should be printed or few.

DefaultMicroIntegrator( n :: Int, T :: Type = Float64; options = nothing )

Creates a basic quadrature Integrator using n nodes in each dimension. Type T can be omitted, in which case it is Float64. The options variable is ignored.

DefaultMicroIntegrator( T :: Type; options = nothing )

Creates a basic quadrature Integrator using 11 nodes in each dimension. Type T can be omitted, in which case it is Float64. The options variable is ignored.

DefaultMacroIntegrator( n :: Int, T :: Type; options :: Union{Vec{Symbol}, Nothing} = nothing )

Creates a basic Monte Carlo Integrator using n draws. Type T can be omitted, in which case it is Float64. The optional options argument can be used to indicate two possible changes from the default, namely :randomize can be used to require randomization and :replacement to indicate randomization with replacement. The default for both is false. Note that options is either nothing or a vector of symbols. A further use of options is to specify a column heading containing weight; this symbol should correspond to the desired column heading in the draws spreadsheet.

DefaultMacroIntegrator( T )

Creates a basic Monte Carlo Integrator using 10 000 draws. This is less than recommended, so use the other method to set a number of your choosing. Type T can be omitted, in which case it is Float64. The optional options argument can be used to indicate two possible changes from the default, namely :randomize can be used to require randomization and :replacement to indicate randomization with replacement. Note that options is either nothing or a vector of symbols. The defaults for both is false. A further use of options is to specify a column heading containing weight; this symbol should correspond to the desired column heading in the draws spreadsheet.

MSMMicroIntegrator( n :: Int, T = F64; options = nothing )

Creates a Monte Carlo integrator type for micro integration with GMM with smart moments. The optional type can be omitted. The options variable is ignored.

MSMMicroIntegrator( T = F64; options = nothing )

Creates a Monte Carlo integrator type for micro integration with GMM with smart moments with 10 MC draws (per consumer). The type variable is optional and can be omitted. The options variable is ignored.

Data( 
    e                   :: GrumpsEstimator,
    ss                  :: Sources,
    v                   :: Variables,
    microintegrator     :: MicroIntegrator = DefaultMicroIntegrator(),
    microintegrator     :: MacroIntegrator = DefaultMacroIntegrator(),
    T                   :: Type = F64,
    options             :: DataOptions = GrumpsDataOptions(),
    replicable          :: Bool = false
    )

Takes user inputs and converts them into an object that Grumps can understand. This is synonymous with GrumpsData(...).

Data takes the following arguments, of which the first three are mandatory:

• e: estimator; see Estimator choice
• ss: data sources; see Data entry
• v: variables to be used; see Data entry
• o: optimization options to be used
• microintegrator: micro integrator see Choice of integration method (integrators)
• macrointegrator: macro integrator see Choice of integration method (integrators)
• T: floating point type; not heavily tested
• u: not yet implemented
• options: data options to be used, see Data storage options
• replicable: whether results must be replicable (slows down speed of data creation if set to true)

grumps!( 
    e       :: Estimator,
    d       :: Data{T},
    o       :: OptimizationOptions = OptimizationOptions(),
    θstart  :: StartingVector{T} = nothing,
    seo     :: StandardErrorOptions = StandardErrorOptions()
)

Conducts the optimization. You typically just want to set θstart to nothing, i.e. have a starting vector picked automatically.

getθ( sol :: GrumpsSolution )

Returns a vector of GrumpsEstimate types for θ that can be queried for results. See getcoef, getstde, gettstat, and getname.

getδ( sol :: GrumpsSolution )

Returns a vector of GrumpsEstimate types for δ that can be queried for results. See getcoef, getstde, gettstat, and getname.

getβ( sol :: GrumpsSolution )

Returns a vector of GrumpsEstimate types for β that can be queried for results. See getcoef, getstde, gettstat, and getname.

getcoef( e :: GrumpsEstimate )

Returns the estimated coefficient value.

getstde( e :: GrumpsEstimate )

Returns the standard error.

gettstat( e :: GrumpsEstimate )

Returns the t statistic.

getname( e :: GrumpsEstimate )

Returns the variable name.

getθcoef( sol :: GrumpsSolution )

Returns a vector of θ coefficients

getδcoef( sol :: GrumpsSolution )

Returns a vector of δ coefficients

getβcoef( sol :: GrumpsSolution )

Returns a vector of β coefficients

Save( fn, mt, sol :: GrumpsSolution; keywords... )

Saves the solution stored in sol to a file with filename fn which has mime type mt.

There are several keywords that are described below, some of which will be ignored for some mime types. Allowed mime types are text/plain, text/csv, and text/tex.

Save( fn, sol :: GrumpsSolution; keywords... )

The same as the form of Save with prespecified mime type except that the mime type is now inferred from the file extension. The keywords also have the same meaning, namely...

There are several keywords that are described below, some of which will be ignored for some mime types. Allowed mime types are text/plain, text/csv, and text/tex.

show( io :: IO, e :: GrumpsEstimate{T}, s :: String = ""; keywords...)

Show a GrumpsEstimate object on io; the argument s indicates which parameter family (θ,β,δ) the estimate belongs to. The optional keywords are described in the table below.

show( io :: IO, est :: Vector{ GrumpsEstimate{T} }, s :: String = ""; keywords... )

Shows a vector of estimates on io using the string s (typically one of θ,β,δ). The command takes the following optional keywords.

show( io :: IO, convergence :: GrumpsConvergence{T}; keywords...)

Shows the contents of convergence, where the flags indicated what should be printed and how, as indicated in the following table.

show( io :: IO, sol :: GrumpsSolution{T}; keywords... )

Shows the contents of sol, where the keywords indicate what should be printed and how, as described in the table below.

show( io :: IO, mt :: MIME{Symbol("text/tex")}, sol :: GrumpsSolution; keywords... )

This is the same as Save() except that the contents are spit out on io (which could be stdout or an already opened file).

show( io :: IO, mt :: MIME{Symbol("text/csv")}, sol :: GrumpsSolution; keywords... )

This is the same as Save() except that the contents are spit out on io (which could be stdout or an already opened file).