Joint Working Group Meeting: CWG, MWG, ORWG, CBASC August 29-30, - - PowerPoint PPT Presentation

Joint Working Group Meeting: CWG, MWG, ORWG, CBASC

August 29-30, 2012

MARKET TRIALS & PARALLEL TESTING

CASEY CATHEY

2 2

• Overview of Market Trials connectivity test
• Phased Timeline
• BA Waivers – Parallel Ops
• Approach
• Timing Requirements

3

Market Trials/Parallel Testing

• Provide an overview of the Market Trials phases & timeline to

the MWG, RTWG, ORWG, CWG, & CBASC

• Review assumptions made going into the Parallel Operations and

Deployment testing timeframe, currently scheduled for November 2013 to January 2014

4

Overview

Confirm SPP / Participant systems & business processes are ready to support the Integrated Marketplace by:

• Validating the connectivity for each Participant
• Exercising each Marketplace business function through a series
• f controlled scenario tests
• Allowing SPP and Participants the flexibility to further verify

systems and business functionality through collaborative unstructured tests

• Establishing confidence in the Marketplace systems in a

‘production like’ environment prior to market launch

• Allow for cooperative feedback, discussion, and education on the
• perational performance of Integrated Marketplace

5

What is Market Trials?

6

Market Trials Timeline

• TCR Market Trials will consist of two phases.
• Phase I will mimic the first three months of the Integrated

Marketplace (March 2013 through May 2013)

• Phase II will mimic the first Annual TCR process (June 2013

through May 2014). TCR Market Trials will end prior to the TCR Market go-live in October 2013

• MPs will be responsible for creating transactional data for

the TCR System, including all Nominations, Bids and Offers for the Allocation and/or Auction

• Unlike the Mock Auction, purely financial players (Market

Participants who do not own resources or serve load) will be allowed to participate in Market Trials

7

TCR Market Trials Overview

• Connectivity Testing will verify connectivity between SPP and

Participants including digital certificates, XML listeners, and firewall access

• Connectivity Testing will exercise the programmatic interfaces

between SPP and the Market Participant as part of functional

• perations
• Connectivity Testing will also verify Participant access to the

external user interfaces for SPP systems. Some user interfaces may not be available until later in Connectivity Testing

• Successful completion of Connectivity Testing will provide a

measure of both SPP and Participant readiness for Structured Market Trials

8

Connectivity Testing for Marketplace

9

Connectivity Test Preparation Oct 2012 – Dec 2012 Connectivity Test Jan 2013 – Mar 2013

Data Exchange Certification Mar 2013 – May 2013 Structured Test Begin Jun 2013

Connectivity Testing Timeline

• Test Preparation:
• Initiate Local Security Administrator (LSA) and User/Application Certificate Setup
• Exchange notification certificates and Market Participant listener URLs with SPP
• If desired, Market Participants may establish and test firewall access to the SPP

Production Primary site using stubbed Application Programming Interfaces (APIs) during this phase

• Connectivity Test:
• Connectivity Test allows Market Participants to verify certificate authentication

and firewall access to the SPP Production environments, connecting to multiple IP addresses

• Data Exchange Certification:
• Data Exchange Certification allows Market Participants to verify critical data

submission, query, notification operations, and User Interfaces in the SPP Production Primary environment following Connectivity Test

10

Connectivity Testing Timeline

• Market Trials Structured Testing will cover both normal

business processes and exception scenarios

• Market Trials Structured Testing will be performed as a

controlled scenario test, which means that SPP will provide the test data that is to be submitted by the Participants or provide a script for the MP to follow in order to induce a particular scenario outcome

• To ensure that the expected outcomes are achieved, SPP

will execute the operational functions central to each test using the test data provided to the Participants

11

Structured Testing

• SPP plans to publish the Market Trials Structured /

Unstructured Test Approach by December 10, 2012. This will be a single Test Approach covering both test phases

• SPP reviews and informational sessions with Participants

will start in January 2013. These reviews will include a detailed walk-through of each scenario and associated test data

• Market Trials Structured Testing is scheduled to start on

June 3, 2013

12

Structured Testing

• As we validate business functionality through Structured

Testing, SPP will enable Participants to further validate each function throughout the Market Trials Unstructured Testing phase

• Test data will not be provided by SPP, rather Participants

will be able to submit data as desired to exercise and validate various interactions between the Participant and SPP

• This approach should enable a controlled approach to

enabling Participants to interact with broader Marketplace systems leading into Market Trials Parallel Operations

13

Unstructured Testing

• Test data will not be provided by SPP, rather Participants will

submit data as desired to exercise and validate various interactions between the Participant and SPP

• It is expected that the Market Participants will be submitting

production or production like data on a daily basis

• SPP will support Parallel Operations as if in production, and will

be staffed at go-live levels

• Integrated Deployment Testing, a test within Parallel Operations,

will be coordinated across the entire SPP CBA footprint to demonstrate the capability of the Integrated Marketplace to successfully dispatch units and manage congestion

14

Parallel Operations

• The process for cutting over from the SPP EIS Market with 16 Balancing

Authorities to the SPP Marketplace system with a single Consolidated Balancing Authority for a specified period of time

• The intent is to prove that the SPP & Member systems, interfaces, etc.

work as intended

• The goal of each deployment test will be to run the system in real-time

with minimal issues

• Currently there are 5 deployment tests pre-scheduled during Parallel

Operations (number will increase)

• The readiness goal for the program is tentative to have one successful

deployment test per month of Parallel Operations

15

What is a Deployment Test?

• SPP must settle the Deployment Test time periods
• Settle on EIS pricing (LIPs), MW deviations from Schedules, but

deployment is followed by Marketplace Setpoint instructions and URD waivers throughout testing period

• EIS systems will be run in parallel
• Place A/S plan from EIS in Current Operating Plan for Marketplace
• RUC to mimic what resources are planned to have online during

Operating Day for that Deployment test

• Not allowed to settle on Marketplace MCE prices, due to co-
• ptimization, product substitution, etc.
• All 16 BAs are set up to quickly switch from EIS -> Marketplace -> EIS
• Will require line-by-line detail in cut-over plan

16

Assumptions

• New RTO_SS will use exact schedules from old RTO_SS
• Allow MPs to submit schedules, including ramp reservation
• BAs require waivers for CPS/BAAL/DCS,
• SPP working on timeline and approach; prefer a single waiver
• We keep same Reserve Requirement as total 16 BAs
• RSS events
• If one occurs during deployment tests, end-to-end testing done with RSS,

Contingency Reserve Deployment application, etc.

• More detail to come, as extensions and cutting back over take additional

consideration

• Market Flow values to IDC will be same as EIS during test
• Scheduling/Transmission Service during deployment tests assessed

using current methodology

17

Assumptions (cont.)

18

SPP RTBM SPP EIS RTB SPP EMS SPP EMS SPP ICCP SPP ICCP SPP Day Ahead SPP RUC SPP AGC Members ICCP Load & Cap RSS NLS RTOSS NERC IDC Members AGC Local Plant Control SPP TCR SPP EIS Market SPP Marketplace/CBA

X

MFC SETTLE MENT SETTLE MENT MEMBERS Current BA / TO /GO

Dispatch T+5 NSI EIS, NSI Tags Setpoint T+4sec

Real time Real time

X

CRD NEW RTOSS NEW MFC NERC IDC

X

XML XML

X X

NEW RSS

Parallel Operations System Overview

• Upcoming Milestones/Activities
• Continue preparation for the Market Trials - Connectivity Test Sub-Stage
• Publish Template Test Checklist (TBD)
• MPs Populate Test Checklist and Return to SPP (TBD)
• SPP and MPs Finalize Test Checklist (TBD)
• Begin Market Trials Connectivity Test Preparation 22-Oct (TBD)
• Continue drafting Market Trials Structured/Unstructured Test Approach

(TRL.004)

• The MTRG will hold their next meeting in October, 2012 to review Test Scenarios (TBD)
• Finalizing Settlements approach for Parallel Operations. This will go to MWG

in September & MOPC in October for stakeholder approval.

• Working with membership to define cutover approach. This drives Parallel

Operations & Reversion plan.

19

Market Trials Tower

• The Market Trials Review Group (MTRG) is comprised
• f a subset of stakeholders
• Purpose is to review and approve various Market Trials

tasks prior to rolling out to full membership

• Provides insight into top concerns during the creation
• f line-by-line work plans & approaches that support

the Market Trials phases

20

Market Trials Review Group

21

Market Participant Milestones