Commercial meets Open Source Tuning STATISTICA with R Tuning - - PowerPoint PPT Presentation

▶
commercial meets open source tuning statistica with r
SMART_READER_LITE
LIVE PREVIEW

Commercial meets Open Source Tuning STATISTICA with R Tuning - - PowerPoint PPT Presentation

May 17, 2023 197 likes •798 views

Commercial meets Open Source Tuning STATISTICA with R Tuning STATISTICA with R Christian H. Wei STATISTICA and R Christian H. Wei Introduction Introduction Introduction Introduction STATISTICA and R Christian H.

slide-1
SLIDE 1

Commercial meets Open Source – Tuning STATISTICA with R Tuning STATISTICA with R

Christian H. Weiß

slide-2
SLIDE 2

STATISTICA and R – Christian H. Weiß

❋ Introduction Introduction ❋ ❋ Introduction Introduction ❋

slide-3
SLIDE 3

STATISTICA and R – Christian H. Weiß

Extremely powerful environment for statistical computing!

slide-4
SLIDE 4

STATISTICA and R – Christian H. Weiß ▶ Provides packages for different areas (data

mining, econometrics, biostatistics, etc.).

▶ Offers methods from different disciplines (time ▶ Offers methods from different disciplines (time

series analysis, statistical process control, bootstrapping, cluster analysis, etc.).

▶ Reflects state-of-art in statistical sciences. ▶ Freely available!

slide-5
SLIDE 5

STATISTICA and R – Christian H. Weiß

... on the other hand: R is not particularly user-friendly!

▶ No graphical user interface, where whole

repertoire of methods fully integrated.

▶ Methods not available for users, who have not

learnt the R language.

▶ No powerful spreadsheet environment, which

enables intuitive way of data manipulation.

slide-6
SLIDE 6

STATISTICA and R – Christian H. Weiß

⇒ ⇒ ⇒ ⇒ Potential users from applied sciences and industry often do not have the heart to work with R! work with R! Users often prefer the comfort of a commercial package like STATISTICA.

slide-7
SLIDE 7

STATISTICA and R – Christian H. Weiß

slide-8
SLIDE 8

STATISTICA and R – Christian H. Weiß

⇒ ⇒ ⇒ ⇒ Idea: Combine the power of R Combine the power of R with the comfort of STATISTICA!

slide-9
SLIDE 9

STATISTICA and R – Christian H. Weiß

Idea: Use STATISTICA as an easily operated interface with a respectable basic equipment of interface with a respectable basic equipment of statistical procedures. Integrate specialised statistical procedures and sophisticated techniques offered by R into the user interface of STATISTICA.

slide-10
SLIDE 10

STATISTICA and R – Christian H. Weiß

Idea: The user does data analysis in STATISTICA, using the readily available methods, and using macros written in Visual Basic, which access R for advanced computations. ⇒ ⇒ ⇒ ⇒ Use power of R without the need to learn the R language!

slide-11
SLIDE 11

STATISTICA and R – Christian H. Weiß

❋ Procedure Procedure ❋ ❋ Procedure Procedure ❋

slide-12
SLIDE 12

STATISTICA and R – Christian H. Weiß

Required:

▶ Base version of STATISTICA with its

Visual Basic development environment; Visual Basic development environment;

▶ R together with necessary packages; ▶ R DCOM Server of Baier & Neuwirth (2007)

slide-13
SLIDE 13

STATISTICA and R – Christian H. Weiß

After having installed STATISTICA, R, R DCOM Server, ... the remaining steps (programming & application) are done within the user-interface of STATISTICA.

slide-14
SLIDE 14

STATISTICA and R – Christian H. Weiß

How can we write a STATISTICA macro, which How can we write a STATISTICA macro, which is able to access functionalities offered by R?

slide-15
SLIDE 15

STATISTICA and R – Christian H. Weiß

slide-16
SLIDE 16

STATISTICA and R – Christian H. Weiß

Visual Basic environment allows to easily design user easily design user dialogs, ...

slide-17
SLIDE 17

STATISTICA and R – Christian H. Weiß

Compared to a "standard" macro,

  • nly one additional step is necessary:

Include R DCOM ("StatConnector") libraries.

slide-18
SLIDE 18

STATISTICA and R – Christian H. Weiß

slide-19
SLIDE 19

STATISTICA and R – Christian H. Weiß

slide-20
SLIDE 20

STATISTICA and R – Christian H. Weiß

Afterwards, a new type of object is available: StatConnector object. This object allows to communicate with R.

slide-21
SLIDE 21

STATISTICA and R – Christian H. Weiß

StatConnector-objects

  • ffer a number of methods:

Dim rzugriff As StatConnector Set rzugriff = New StatConnector

slide-22
SLIDE 22

STATISTICA and R – Christian H. Weiß

Starting R: rzugriff.Init("R") Receive possible error messages: rzugriff.GetErrorText Shut down connection: rzugriff.Close

slide-23
SLIDE 23

STATISTICA and R – Christian H. Weiß

Most important methods:

  • bj.Evaluate("R command")
  • bj.EvaluateNoReturn("R command")
  • bj.SetSymbol("R variable",value)
  • bj.GetSymbol("R variable")
slide-24
SLIDE 24

STATISTICA and R – Christian H. Weiß

❋ Example 1 Example 1 ❋ ❋ Example 1 Example 1 ❋

slide-25
SLIDE 25

STATISTICA and R – Christian H. Weiß

STATISTICA offers a number of approaches from SQC:

slide-26
SLIDE 26

STATISTICA and R – Christian H. Weiß

In particular, STATISTICA offers a broad variety of control charts, including, e.g., EWMA and CUSUM charts.

slide-27
SLIDE 27

STATISTICA and R – Christian H. Weiß

slide-28
SLIDE 28

Reliable design of EWMA and CUSUM charts is not possible with simple k-σ rule. Instead: Consider ARL performance

  • f charts.
slide-29
SLIDE 29

STATISTICA and R – Christian H. Weiß

However: STATISTICA does not allow to compute ARLs! But R does: spc package of Knoth (2007). ⇒ ⇒ ⇒ ⇒ Tune STATISTICA with R!

slide-30
SLIDE 30

STATISTICA and R – Christian H. Weiß

Macro “ARLwithR.svb”: Dim robj As StatConnector Set robj = New StatConnector Load spc-package:

robj.EvaluateNoReturn("library(spc)")

slide-31
SLIDE 31

STATISTICA and R – Christian H. Weiß

Compute ARL of EWMA chart: robj.Evaluate(" robj.Evaluate(" xewma.arl(l=0.1, c=2.7,mu=0.0, sided=“two”, limits=“vacl”) ")

slide-32
SLIDE 32

STATISTICA and R – Christian H. Weiß

Compute limits of EWMA chart: robj.Evaluate(" robj.Evaluate(" xewma.crit(l=0.1,L 0=370, sided=“two”, limits=“vacl”) ")

slide-33
SLIDE 33

STATISTICA and R – Christian H. Weiß

❋ Example 2 Example 2 ❋ ❋ Example 2 Example 2 ❋

slide-34
SLIDE 34

STATISTICA and R – Christian H. Weiß

slide-35
SLIDE 35

STATISTICA and R – Christian H. Weiß

slide-36
SLIDE 36

STATISTICA and R – Christian H. Weiß

slide-37
SLIDE 37

STATISTICA and R – Christian H. Weiß

STATISTICA offers a large number of methods from time series analysis. E.g., it is able to fit any type of ARIMA model. However, However, STATISTICA not able to fit GARCH models! But R does: tseries package of Trapletti (2007). ⇒ ⇒ ⇒ ⇒ Tune STATISTICA with R!

slide-38
SLIDE 38

STATISTICA and R – Christian H. Weiß

Macro “GARCHwithR.svb”: Dim robj As StatConnector Dim robj As StatConnector Set robj = New StatConnector Load tseries-package:

robj.EvaluateNoReturn("library(tseries)")

slide-39
SLIDE 39

STATISTICA and R – Christian H. Weiß

slide-40
SLIDE 40

STATISTICA and R – Christian H. Weiß

Submit data to R, assign it to R variable called R variable called “data”: robj.SetSymbol ("data", spreadsht.Data)

slide-41
SLIDE 41

STATISTICA and R – Christian H. Weiß

Ask R to fit a GARCH(1,1) model: robj.EvaluateNoReturn(" data.garch<-garch(data,order=c(1,1)) ")

slide-42
SLIDE 42

STATISTICA and R – Christian H. Weiß

Ask R for ... maximized log-likelihood: robj.Evaluate("logLik(daten.garch)") estimated coefficients: robj.Evaluate("coef(daten.garch)") estimated covariance matrix: robj.Evaluate("vcov(daten.garch)") ...

slide-43
SLIDE 43

STATISTICA and R – Christian H. Weiß

... estimated residuals:

robj.Evaluate("residuals(daten.garch)")

Use these results and prepare STATISTICA output:

slide-44
SLIDE 44

STATISTICA and R – Christian H. Weiß

slide-45
SLIDE 45

STATISTICA and R – Christian H. Weiß

Latest Latest ❋ ❋ Latest Latest Developments Developments ❋ ❋

slide-46
SLIDE 46

STATISTICA and R – Christian H. Weiß

Above approach for accessing R can be realized with any version of STATISTICA. Only few days ago, the new release Only few days ago, the new release MR-3 for STATISTICA, version 8

  • ccurred, see www.statsoft.com.

→ several new approaches for interacting with R!

slide-47
SLIDE 47

STATISTICA and R – Christian H. Weiß

Essentially, four main innovations:

▶ Run R scripts straight from STATISTICA. ▶ Call R scripts from STATISTICA macro. ▶ Call R scripts from STATISTICA macro. ▶ New commands for R scripts to simplify data

transfer between R and STATISTICA.

▶ New commands for SVB macros to simplify data

transfer between R and STATISTICA.

slide-48
SLIDE 48

STATISTICA and R – Christian H. Weiß

Run R scripts from STATISTICA: → Simply open file with extension .r or .s. Then run script like usual SVB macro.

slide-49
SLIDE 49

STATISTICA and R – Christian H. Weiß

Output in workbook:

▶ A report (≈ RTF file) with console output. ▶ Graphs generated by plot as separate metafiles.

slide-50
SLIDE 50

STATISTICA and R – Christian H. Weiß

Extend these R scripts with the new commands

  • ffered by STATISTICA:
slide-51
SLIDE 51

STATISTICA and R – Christian H. Weiß

Important new commands for R scripts: ActiveDataSet[FromVar:ToVar] Spreadsheet("path") → Access STATISTICA data file. RouteOutput(R table, name, header) → Transfer R tables to STATISTICA tables, display them separately in a workbook (optional: with name “name”, header “header”).

slide-52
SLIDE 52

STATISTICA and R – Christian H. Weiß

Call R script from SVB macro:

slide-53
SLIDE 53

STATISTICA and R – Christian H. Weiß

Dim oMacro As Macro Set oMacro=Macros.Open("path") Run macro by one of following approaches: Run macro by one of following approaches:

  • Macro.Execute
  • Macro.ExecuteWithArgument(oColl)
  • Macro.executeNoRouteOutput(oColl)
slide-54
SLIDE 54

STATISTICA and R – Christian H. Weiß

  • Macro.Execute

Just execute R macro,

  • utput controlled from R script, e.g., using command

RouteOutput.

slide-55
SLIDE 55

STATISTICA and R – Christian H. Weiß

  • Macro.ExecuteWithArgument(oColl)

Like before, but submit initial values through newly created SVB Collection object: created SVB Collection object: Dim oColl As New Collection

  • Coll.Add(STCAobj, "name")

“name” ist variable’s name in R.

slide-56
SLIDE 56

STATISTICA and R – Christian H. Weiß

  • Macro.executeNoRouteOutput(oColl)

Like before, but no immediate output to workbook. Instead: Instead: Returns an object of newly created type StaDocCollection. Items of this object can be processed in SVB macro.

slide-57
SLIDE 57

STATISTICA and R – Christian H. Weiß

❋ References References ❋ ❋ References References ❋

slide-58
SLIDE 58

STATISTICA and R – Christian H. Weiß

Baier, T., Neuwirth, E.: R/Scilab (D)COM Server V 2.50. March, 2007.

http://cran.r-project.org/contrib/extra/dcom/

Knoth, S.: The spc Package (Statistical Process Control), Version 0.21. October, 2007.

http://cran.r-project.org/src/contrib/Descriptions/spc.html

StatSoft: STATISTICA Data Miner: Integrating R Programs into the Data Miner Environment. StatSoft: STATISTICA Data Miner: Integrating R Programs into the Data Miner Environment. StatSoft Business White Paper, June, 2003. StatSoft: Integration Options and Features to Leverage Specialized R Functionality in STATISTICA and WebSTATISTICA Solutions. StatSoft White Paper, July, 2008. Trapletti, A., Hornik, K.: The tseries Package, Version 0.10-15. May, 2008.

http://cran.r-project.org/src/contrib/Descriptions/tseries.html

Weiß, C.H.: Datenanalyse und Modellierung mit STATISTICA. Oldenbourg Wissenschaftsverlag, München, 2006.

slide-59
SLIDE 59

STATISTICA and R – Christian H. Weiß

Thank You Thank You Thank You Thank You for Your Interest! for Your Interest!