fyi: 数据挖掘软件大评比

 未分类  fyi: 数据挖掘软件大评比已关闭评论
11月 301999
 
去年年底(2008年11月),德国一家技术咨询公司,mayato,发布了一篇数据挖掘挖掘软件的评估报告,考察了以下12种产品:
 
传统的数据挖掘套件(Classic suites):
SAS Enterprise Miner
5.3
SPSS Clementine 12
 
开源数据挖掘软件(Open Source):
RapidMiner 4.2
KNIME
1.3.5
Weka 3.4.13
 
自动化数据挖掘软件(Self-Acting):
KXEN Analytic Framework
4.04
 
专门化的数据挖掘软件(Specialized):
Viscovery SOMiner
5.0
prudsys Discovery 5.5 / Basket Analyzer 5.2
Bissantz Delta Master
5.3.6
 
BI产品内置的数据挖掘软件(BI Vendors):
SAP NetWear 7.0 Data Mining
Workbench
Oracle 11g Data Mining
Microsoft SQL Server 2005 Analysis
Services
 
最近数据挖掘市场呈现出多元化的态势,除了传统的数据挖掘厂商SAS、SPSS的产品外,又有各种专有用途的数据挖掘软件,加上开源软件和BI产商提供的数据挖掘功能,这个市场看着是红红火火,给各种层次的用户提供了灵活的选择空间。mayato这篇报告的题目就叫做
Data
Mining Software 2009: Successful Analyses at Affordable Prices

(November 2008)。
 
可惜mayato这次的评估不够深入,所用的标准也嫌太过粗糙。在对Enterprise Miner (SAS),
Rapidminer (Rapid-I), Analytic Framework (KXEN), and NetWeaver Data Mining
Workbench (SAP)这四种产品进行了所谓thoroughly的评估后,它的结果是,Analytic Framework
(KXEN)综合排名第一,Enterprise Miner (SAS)紧接其后,然后是SAP NetWeaver Data Mining
Workbench和Rapidminer。
 
这次KXEN排名第一,我们并不感到吃惊(mayato是KXEN的合作伙伴)。不过,KXEN处理数据的速度的确非常值得称道。KXEN号称自动化数据挖掘软件(Self-Acting),客户定制调优的空间比较小,在默认选项下,其他软件在运行速度上就吃亏不少了。最后提一下,这次评估,KXEN在速度上占优,而SAS在性能上最为突出。
 
总的来说,这份评估过于简单,只能作为参考。不过它所展现的(以及遗漏的)数据挖掘软件市场,还是让人比较兴奋的(这次评估,当然有非常多的遗漏,如重要的Teradata
Warehouse Miner、IBM的DB2 Intelligence
Miner、Angoss、Unica等)。

分类模型的性能评估——以SAS Logistic回归为例(3): Lift和Gain

 未分类  分类模型的性能评估——以SAS Logistic回归为例(3): Lift和Gain已关闭评论
11月 301999
 

书接前文。跟ROC类似,Lift(提升)和Gain(增益)也一样能简单地从以前的Confusion Matrix以及Sensitivity、Specificity等信息中推导而来,也有跟一个baseline model的比较,然后也是很容易画出来,很容易解释。以下先修知识,包括所需的数据集:

  1. 分类模型的性能评估——以SAS Logistic回归为例(1): 混淆矩阵
  2. 分类模型的性能评估——以SAS Logistic回归为例(2): ROC和AUC

*更多,见

http://cos.name/2009/02/measure-classification-model-performance-lift-gain/

白话统计(1):平均数、中位数、众数

 统计备忘录  白话统计(1):平均数、中位数、众数已关闭评论
11月 301999
 

********************示例数列为4、4、5、7、10********************

定义1. 各项累加之和除以项数,所得之数值,叫做平均数(6)。

定义2. 众数是出现次数最多的那一项的数值(4)。

定义3. 中位数是这样一个项的数值(或两项之平均数):它(或该两项)的数值大于或等于其余一半项的数值,而小于或等于另外一半项的数值(5)。

注1:平均数受每个改变的牵动,中位数和众数却只受某些改变的牵动。由于这个原因,平均数常常被认为是“敏感的”,是“反映整个分布的”。

注2:当分布含有少数在一端远离的极端项时,平均数的敏感性,对它的代表性反而可能是不利的。

注3:中位数不受少数极端值的影响。

注4:众数显示最大频率,而最大频率是最普遍化的同义语。

一、平均数的两个性质

物理模型:设想一块实验板,其上有4至10的等距刻度。假定5个一磅重的砝码置于板上,砝码的位置为4、4、5、7、10。现在假设有一支点,令该实验板连同其上的砝码在支点上保持平衡,并设该实验板本身无重量。几经失败之后,我们设想找到了平衡点。

 

image

 

 

 

 

结果,支点在各项的平均数之下。任何项的集合的平均数都是那些项的平衡点,而且平均数是唯一的平衡点。

考虑支点左边每个砝码与支点(平均数)的距离。最左端的两个砝码与平均数相距2个单位,第三个相距1个单位,总和是5个单位。同样,支点右边的砝码与支点的距离也是5个单位。正是这个等量平衡了实验板。左右两边砝码与平均数的距离之和总是相同的,也即平均数两边的距离“相抵”。

为了得到上面的距离,从各项减去平均数。有负号的是平均数左边砝码得到的结果,有正号的是右边砝码得到的结果。前面提到,把负号抹去,左边各项的和等于右边各项的和(距离“相抵”)。留住负号,则所有项的和等于0 ,Σ[x-mean(x)]=0,即,与平均数之差的和恒等于0。总结一下:

定理1:在任何分布中,各项对平均数的差之和等于0。反之,若分布中各项对某数的差数之和等于0时,该数即为此分布的平均数。

又考虑对平均数的差数平方:

各项 平均数6做参考点 差数 差数平方
4 6 -2 4
4 6 -2 4
5 6 -1 1
7 6 1 1
10 6 4 16
    总和 26

SAS Logistic回归:一个完整例子

 未分类  SAS Logistic回归:一个完整例子已关闭评论
11月 301999
 

                                                SAS Logistic回归:代码及输出报告详解

这篇将作为五一后一个讲稿的阅读材料之一,先整出来就搁这。如果没有耐心读下去,你可以立即转到以下的参考资料,该篇所有的知识都来自它们:

  1. Cody, R.F. and Smith, J.K. Applied Statistics and the SAS Programming Language,4th ed..NJ:  Prentice-Hall,1997.这书已经出第五版了,北大图书馆只有这第四版。非常容易上手的一本书,前半部分用input和datalines让读者专心做统计,后半部分从导入导出数据开始阐述SAS的通用编程语言。这本书用的是SAS8.这里我们只关注它第九章Multiple-Regression Analysis的最后Logistic Regression部分。我这篇的例子即来于此,有简化;
  2. SAS OnlineDoc V8, 或者SAS OnlineDoc V9, 是要花功夫熟悉它们的结构了。以前我四处下载了数G的电子书,现在才发觉还是它们好使。体例上V8和V9一样,你找到SAS/STAT–>SAS/STAT User’s Guide–>The LOGISTIC Procedure, 就可以跟着学习了,文字都非常简明。

Logistic回归处理因变量是分类型变量如“0、1”的情形。一下就假设你至少对它模模糊糊有些印象,比如说我们用p表示正例(如输出变量为“1”)的概率,那么p/(1-p)就被称作odds ratio,对p做logit变换记做logit(p),它等于log(p/(1-p),我们回归方程的形式就如logit(p)=log(p/(1-p)=a+bx,你可以把它理解成向量形式。

假设我们有一个数据,45个观测值,四个变量,包括:

  1. age(年龄,数值型);
  2. vision(视力状况,分类型,1表示好,0表示有问题);
  3. drive(驾车教育,分类型,1表示参加过驾车教育,0表示没有) 和
  4. 一个分类型输出变量accident(去年是否出过事故,1表示出过事故,0表示没有)。我们的目的就是要考察前三个变量与发生事故的关系。

为了保持程序的可读性,以下我就直接在代码中一行一行敲入数据,在实际工作中,超过20个观测值的数据就要写程序导入了。另外,/*   */之间的语句都是注释,SAS运行时会把它们忽略掉:

—————————————————————————————————————————-

/*

前三行,首先data步,建立一个叫logistic的临时数据集,它在work临时文件夹里。一般推荐用记事本把程序保存起来;input语句表示一下是几个变量;从datalines开始就一行一行输入数据,每个单独的数据之间用空格隔开,每个观测值各占一行。需要注意的是,datalines和下面的数据之间不要有空行,否则SAS会认为出现了一个缺失值。数据输入完毕,以分号结束,这个分号一定要另起一行。

*/

data logistic;
    input accident age vision drive;
datalines;
1 17 1 1
1 44 0 0
1 48 1 0
1 55 0 0
1 75 1 1
0 35 0 1
0 42 1 1
0 57 0 0
0 28 0 1
0 20 0 1
0 38 1 0
0 45 0 1
0 47 1 1
0 52 0 0
0 55 0 1
1 68 1 0
1 18 1 0
1 68 0 0
1 48 1 1
1 17 0 0
1 70 1 1
1 72 1 0
1 35 0 1
1 19 1 0
1 62 1 0
0 39 1 1
0 40 1 1
0 55 0 0
0 68 0 1
0 25 1 0
0 17 0 0
0 45 0 1
0 44 0 1
0 67 0 0
0 55 0 1
1 61 1 0
1 19 1 0
1 69 0 0
1 23 1 1
1 19 0 0
1 72 1 1
1 74 1 0
1 31 0 1
1 16 1 0
1 61 1 0
;

/*

以下两行我们不妨称作过程步甲。过程步以proc开头,加上要实现功能的名字,这里是logistics,接下来是要引用的数据。值得注意的是那个desending选项。SAS的Logistic回归方程log(odds)默认的形式是处理那个变量值比较小的,这里是accident=0,但我们要考察的是发生事故accident=1的情况,加上desending降序排列,它就处理accident=1的log(odds)了。再model引导的就是回归方程的形式,写成“因变量=自变量1    自变量2   自变量3”的样子。最后以run结束语句,与proc对应。

*/

proc logistic data=logistic descending;
     model accident=age vision drive; run;

/*

运行以上程序,就要跑出一大堆结果了。但在处理多元回归时,语句很难得会只像过程步甲一样简洁。以下过程步乙只加入一个变量选择选项forward。SAS在处理自变量选择上采用了5个技术,这里只简单提一下3个常用的技术。

1.forward——前向选择,变量一个个进入回归方程,按照一些卡方标准,最显著的也就是最好的变量最先进入,然后就是次最好的,以此类推。某个变量一旦进入,就不再退出;

2.backward——后向剔除,一开始全部变量都进入回归方程,又按照一些标准,把最不好的变量一一剔除;

3.stepwise——逐步回归,这个跟forward有些类似,不同的是,stepwise在变量进入以后还有一个backward后向剔除的过程,而在forward里,变量一旦进入,就不再退出。

*/ 

proc logistic data=logistic descending;
   model accident=age vision drive  /
                selection=forward;run;

—————————————————————————————————————————-

用上面这个过程步乙替代过程步甲,再运行一遍。这两个过程步的输出结果大同小异,只是过程步乙多了个forward选项。以下用的是过程步乙的输出结果,其中黑体字是输出结果本身,我做的注释语句以红笔描出,包括数字编号。 

                 (1)  The SAS System 10:47 Tuesday, May 4, 2007 1

                             The LOGISTIC Procedure

                               Model Information

Data Set                                         WORK.LOGISTIC
Response Variable                          accident
Number of Response Levels            2
Number of Observations                 45
Model                                              binary logit
Optimization Technique                 Fisher’s scoring

 

                             Response Profile                                     
Ordered  Value     accident          Total Frequency

       1                        1                             25
       2                        0                             20

Probability modeled
is accident=1.

(1) 给出了本模型的基本信息,意思大多自明。需要注意的是Response Profile 中,accident=1排在首位。前面我们说过,SAS的Logistic回归方程log(odds)默认的形式是处理那个变量值比较小的,加上descending选项后,accident=1就排在首位了。

                      (2)   Forward Selection Procedure

Step 0. Intercept entered:

                          Model Convergence Status

Convergence criterion (GCONV=1E-8) satisfied.

                          Residual Chi-Square Test

Chi-Square              DF                Pr > ChiSq

  10.7057                   3                     0.0134

 

Step 1. Effect vision entered:

                              Model Convergence Status

Convergence criterion (GCONV=1E-8) satisfied. 
                               Model Fit Statistics                                               
Criterion     Intercept Only     Intercept and Covariates
AIC                 63.827                   59.244
SC                   65.633                   62.857
-2 Log L           61.827                  55.244

                       Testing Global Null Hypothesis: BETA=0
Test                       Chi-Square    DF    Pr > ChiSq
Likelihood Ratio      6.5830         1         0.0103
Score                       6.4209         1         0.0113
Wald                        6.0756         1         0.0137

                       Residual Chi-Square Test
Chi-Square            DF      Pr > ChiSq
      4.9818              2         0.0828

Step 2. Effect drive entered:
                     Model Convergence Status
                  Convergence criterion (GCONV=1E-8) satisfied.                      

                         Model Fit Statistics
Criterion      Intercept Only     Intercept and Covariates
AIC                  63.827                     56.287
SC                   65.633                     61.707
-2 Log L           61.827                     50.287

 

                 Testing Global Null Hypothesis: BETA=0
Test                        Chi-Square     DF   Pr > ChiSq
Likelihood Ratio         11.5391        2      0.0031
Score                          10.5976        2      0.0050
Wald                            8.5949         2      0.0136
             

                 Residual Chi-Square Test
Chi-Square     DF   Pr > ChiSq
     0.1293        1     0.7191

NOTE: No (additional) effects met the 0.05 significance level for entry into the model.

(2) 给出了自变量进入模型的次序。先是截距项Step 0了,不管它。Step 1 vision第一个进入模型,附带了很多评估它对因变量预测能力的指标。-2 Log L 和Score 用来检测自变量是否显著。-2 Log L中的L就是Likelihood Ratio,它的p值是0.0103Score  的p值是0.0113,都小于0.05,故vision是一个很显著的解释变量。还有,AIC(Akaike Information Criterion)和SC(Schwarz Criterion)两个信息量标准用来比较不同的模型,它们数值越小,模型变现就越好,这个接下来我们看step2 drive变量进入模型后的情况。 我们可以看到模型的表现变好了,因为这是AICSC的值变小了,-2 Log L 和Score对应的p值也更小。

                (3)   Summary of Forward Selection
Step    Effect Entered        DF      Number In      Score Chi-Square        Pr > ChiSq
1            vision                    1           1                      6.4209                        0.0113
2            drive                     1            2                     4.8680                         0.0274

 (3) 总结了我们模型使用的前向选择方法,包括自变量进入模型的次序,以及每个自变量的卡方值和p值。

                    (4) Analysis of Maximum Likelihood Estimates
Parameter   DF   Estimate    Standard Error  Wald Chi-Square   Pr > ChiSq
Intercept     1      0.1110               0.5457           0.0414                 0.8389
vision           1      1.7137               0.7049           5.9113                 0.0150
drive            1     -1.5000               0.7037           4.5440                 0.0330

(4) 给出了模型参数的估计,据此可以写出改回归方程的形式是log(odds of having an accident)=log(p/(1-p))=0.1110+1.7137*vision-1.5000*drive。我们知道,odds=p/(1-p),有p=odds/(1+odds)。假设有个哥们,视力没问题但没有受过驾车教育(vision=0,drive=0),代入方程,有log(odds)=0.1110,再odds=exp(0.110)=1.1174,p=1.1174/2.1174=0.5277,即我们说这人发生事故的概率为0.5277;又另一个,视力有问题同样没受过驾车教育(vision=1,drive=0),同样的步骤得log(odds)=1.8249,odds=exp(1.8249)=6.2022,p=6.2022/7.2022=0.8612,即这人发生事故的概率为0.8612(视力多重要)。

                 (5) Odds Ratio Estimates
Effect       Point Estimate               95% Wald Confidence Limits
vision            5.550
                               1.394           22.093
drive             0.223                               0.056            0.886

 (5) 是对比率Odds Ratio的估计。这里对vision的odds ratio的点估是 5.550 ,这个比率由odds(having an accident|vision=1,drive=0)比odds(having an accident|vision=0,drive=0)得来,根据(4) 的计算,就是6.2022/1.1174=5.550.还有,对vision来说,95% 的置信区间不包括1,说明vision是一个非常显著的解释变量(比率的置信区间不包括1,就跟p值小于0.05一样是一个规则)。

                 (6) Association of Predicted Probabilities and Observed Responses
Percent Concordant             67.2               Somers’ D 0.532
Percent Discordant              14.0               Gamma 0.655
Percent Tied                        18.8                Tau-a 0.269
Pairs                                     500                  c 0.766

(6) 这个东西就有些复杂,大概说些预测概率与观测道德因变量间的关联性,我们看到一致性比率Percent Concordant 为67.2 %,不一致性比率Percent Discordant 为14.0%,说明预测值与观测值在现有水平上有较强的关联性,回归模型有很强的预测能力。

最后注意到,上面我们用过程步乙得出的输出结果没有age这个自变量,用过程步甲得出的输出有,这不是因为年龄在这个预测模型中不重要,而是上面以数值型面目出现的年龄在是否出现事故的两组人中分布不均匀,为了解决这个问题,把年龄分组就是,不多说了。

SAS学习笔记(1):Basic Concepts

 未分类  SAS学习笔记(1):Basic Concepts已关闭评论
11月 301999
 

SAS OnlineTutor®: Basic and Intermediate SAS®的learning path见《SAS学习笔记:开篇》


SAS Programs

A SAS program can consist of a DATA step or a PROC step or any combination of DATA and PROC steps.

DATA steps typically create or modify SAS data sets. They can also be used to produce custom-designed reports.

PROC (procedure) steps are pre-written routines that enable you to analyze and process the data in a SAS data set and to present the data in the form of a report. They sometimes create new SAS data sets that contain the results of the procedure. PROC steps can list, sort, and summarize data.

SAS programs consist of SAS statements. A SAS statement has two important characteristics:

  • It usually begins with a SAS keyword.
  • It always ends with a semicolon.

A DATA step begins with a DATA statement, which begins with the keyword DATA. A PROC step begins with a PROC statement, which begins with the keyword PROC.

SAS statements are free-format. This means that

  • they can begin and end anywhere on a line
  • one statement can continue over several lines
  • several statements can be on a line.

Blanks or special characters separate "words" in a SAS statement.

DATA and PROC statements signal the beginning of a new step. When SAS encounters a subsequent DATA, PROC, or RUN statement (for DATA steps and most procedures) or a QUIT statement (for some procedures), SAS stops reading statements and executes the previous step in the program. In our sample program, each step ends with a RUN statement.

The beginning of a new step (DATA or PROC) implies the end of the previous step. Though the RUN statement is not always required between steps in a SAS program, using it can make the SAS program easier to read and debug, and it makes the SAS log easier to read.


SAS Libraries

Every SAS file is stored in a SAS library, which is a collection of SAS files. A SAS data library is the highest level of organization for information within SAS.

SAS libraries have different implementations depending on your operating environment, but a library generally corresponds to the level of organization that your host operating system uses to access and store files. In some operating environments, a library is a physical collection of files. In others, the files are only logically related.

Windows, UNIX, OpenVMS, OS/2(directory based-systems)

a group of SAS files that are stored in the same directory. Other files can be stored in the directory, but only the files that have SAS file extensions are recognized as part of the SAS data library.

CMS: a group of SAS files that have the same file type.

z/OS (OS/390): a specially formatted host data set in which only SAS files are stored.

Depending on the library name that you use when you create a file, you can store SAS files temporarily or permanently.

Storing files temporarily:  If you don’t specify a library name when you create a file (or if you specify the library name Work), the file is stored in the temporary SAS data library. When you end the session, the temporary library and all of its files are deleted.

Storing files permanently:  To store files permanently in a SAS data library, you specify a library name other than the default library name Work.


Referencing SAS Files

To reference a permanent SAS data set in your SAS programs, you use a two-level name: libref.filename. In the two-level name, libref is the name of the SAS data library that contains the file, and filename is the name of the file itself. A period separates the libref and filename.

To reference temporary SAS files, you can specify the default libref Work, a period, and the filename. For example, the two-level name Work.Test references the SAS data set named Test that is stored in the temporary SAS library Work. Alternatively, you can simply use a one-level name (the filename only) to reference a file in a temporary SAS library. When you specify a one-level name, the default libref Work is assumed.

Rules for SAS Names: SAS data set names

  • can be 1 to 32 characters long
  • must begin with a letter (A-Z, either uppercase or lowercase) or an underscore (_)
  • can continue with any combination of numbers, letters, or underscores.


SAS Data Sets

A SAS data set is a file that consists of two parts: a descriptor portion and a data portion.

The descriptor portion of a SAS data set contains information about the data set, including

  • the name of the data set
  • the date and time that the data set was created
  • the number of observations
  • the number of variables.

The data portion of a SAS data set is a collection of data values that are arranged in a rectangular table.

Rows (called observations) in the data set are collections of data values that usually relate to a single object.

Columns (called variables) in the data set are collections of values that describe a particular characteristic.


Variable Attributes

Name: Each variable has a name that conforms to SAS naming conventions. Variable names follow exactly the same rules as SAS data set names. Like data set names, variable names

  • can be 1 to 32 characters long
  • must begin with a letter (A-Z, either uppercase or lowercase) or an underscore (_)
  • can continue with any combination of numbers, letters, or underscores.

Type: A variable’s type is either character or numeric.

  • Character variables, such as Name (shown below), can contain any values.
  • Numeric variables, such as Policy and Total (shown below), can contain only numeric values (the digits 0 through 9, +, -, ., and E for scientific notation).

A variable’s type determines how missing values for a variable are displayed. In the following data set, Name and Sex are character variables, and Age and Weight are numeric variables.

  • For character variables such as Name, a blank represents a missing value.
  • For numeric variables such as Age, a period represents a missing value.

Length: A variable’s length (the number of bytes used to store it) is related to its type.

  • Character variables can be up to 32K long. In the exa
    mple below, Name has a length of 20 characters and uses 20 bytes of storage.
  • All numeric variables have a default length of 8. Numeric values (no matter how many digits they contain) are stored as floating-point numbers in 8 bytes of storage, unless you specify a different length.

Format: Formats are variable attributes that affect the way data values are written. SAS software offers a variety of character, numeric, and date and time formats. You can also create and store your own formats. To write values out using some particular form, you select the appropriate format.

SAS format

For example, to display the value 1234 as $1234.00 in a report, you can use the DOLLAR8.2 format.

Informat: Whereas formats write values out using some particular form, informats read data values in certain forms into standard SAS values. Informats determine how data values are read into a SAS data set. You must use informats to read numeric values that contain letters or other special characters.

SAS informat

For example, the numeric value $12,345.00 contains two special characters, a dollar sign ($) and a comma (,). You can use an informat to read the value while removing the dollar sign and comma, and then store the resulting value as a standard numeric value.

Label: A variable can have a label, which consists of descriptive text up to 256 characters long.

Technorati Tags: ,

通告

 未分类  通告已关闭评论
11月 301999
 

我的朋友,这段日子我换了个东家,技术路线有所倚重,这个(所谓)技术博客要停止更新了。这期间如果还有些数据挖掘跟应用统计方面的文章,会发布在“统计之都”(cos)上面:

http://cos.name/author/hujiangtang/

有网友说SAS爱好者,“或多或少、有意无意”都有光顾过这个博客。我从文科转入软件工程,以SAS起家,涉足数据挖掘与金融计算,更新不定,主题也不够专注,想想是比较惭愧。喜的是通过这些记录,大致能看到自己的用心所在,同时结交得许多志同道合的朋友,很多场合见面,就没有生疏之感。我现在一家药厂做SAS程序员,多跟临床试验打交道,想先沉淀一段日子再说。

说,技术或有转型,生活还是平稳如初。大伙感兴趣的,不妨先去我的生活博客转转:

http://li-and-jiang.com/blog/

RSS订阅地址是:

http://li-and-jiang.com/blog/feed/

想除了技术,我们还一定有许多聊得开的东西。再会。

SAS Macro的面对对象解释-一个附会

 未分类  SAS Macro的面对对象解释-一个附会已关闭评论
11月 301999
 

SAS编程语言的非OO特点真是让人头大。昨天突然冒出一个想法,可不可以把宏附会成一个“类”,宏参数就类似“属性”,具体的赋值就相当于一个“成员”,这样思路似乎就似乎清楚些。

Technorati Tags: , , ,

SAS和蒙特卡罗模拟(1):开篇

 未分类  SAS和蒙特卡罗模拟(1):开篇已关闭评论
11月 301999
 

SAS for Monte Carlo Simulations (1): Introduction

一、为什么选择SAS做蒙特卡罗模拟?

为什么要用SAS做蒙卡?首先,对我来说,我只会用SAS,而且打算用SAS完成我所有的工作。当然,其他一些通用的理由有(Fan, etc.,2002):

  1. 蒙卡是个计算密集的活,而SAS Base、SAS Macro、SAS/IML强大而灵活的编程能力能满足这一要求;
  2. 做蒙卡时要用到大量的统计/数学技术,而SAS就内置了大量的统计/数学函数(在 SAS/Stat和SAS/ETS);用Fortran或C++当然也是非常好的主意,只是他们缺少内置的统计函数,代码就要冗长复杂很多。

二、什么是蒙卡?一个启发性例子

好,开始,什么是蒙卡?了解它背景知识的最好办法当然是wiki-Monte_Carlo_method。蒙特卡罗是位于摩洛哥的一家赌场,二战时,美国Los Alamos国家实验室把它作为核裂变计算机模拟的代码名称。作为模拟方法,蒙卡以前就叫统计抽样(statistical sampling),我们感兴趣的结果因为输入变量的不确定而不可知,但如果能依概率产生输入变量的样本,我们就可以估计到结果变量的分布。跟蒙卡对应的,还有一种模拟技术叫系统模拟,包括排队、库存等模型,这些模型都跟随时间推移而出现的事件序列有关。下面举个蒙卡的例子,来自Evans, etc.( 2001)的超级简化版。

假设一家企业,利润是其需求量的函数,需求是随机变量。为了简化讨论,假定利润就是需求的两倍。这里输入变量就是不可控的需求,结果变量就是我们感兴趣的利润。假设需求以相同的概率取10、20、30、40、50、60这六种情况。在这样的简化下,我们就可以投一枚均匀的骰子来产生需求的样本,如果点数为1,对应得需求就是10,点数为2,需求就是20,以下类推。这样,我们的模拟过程就是:

  1. 投骰子;
  2. 根据骰子的点数确定需求量;
  3. 根据需求量,求利润。

掷10次骰子,假设我们的模拟结果如下:

重复次数 骰子点数 需求量 利润
1 5 50 100
2 3 30 60
3 3 30 60
4 6 60 120
5 1 10 20
6 3 30 60
7 4 40 40
8 5 50 100
9 2 20 20
10 5 50 50