Outline London CBF and Brussels Photonic Exchange Time and - - PDF document

13/11/14 ¡ 1 ¡

3RD TERENA ARCHITECT WORKSHOP

Sharing of Spectrum and Alien Waves in and around SURFnet

Rob Smets – Architect Transport and Light Systems

Outline

• London CBF and Brussels Photonic Exchange
• Time and Frequency Transfer in SURFnet’s network
• 100G Alien waves for customers
• Multi-domain alien-wave demand planning rules
• Infinera waves on a Ciena light system
• Guy Roberts from GEANT Association

(Some of these slides have been presented at CEF2014 Prague)

13/11/14 ¡ 2 ¡

London CBF / Brussels Photonic Exchange Amsterdam – London CBF

• Joint Collaboration between NORDUnet and SURFnet
• Collaboration: NORDUnet leases the fiber, SURFnet installs and
• perates the light system. We start with two 100G waves.
• CBF between ASD001A – ASD002A – LDN001A
• For SURFnet puts total number of CBFs to four (Geneva, Hamburg, London, Aachen)
• Puts total number of foreign PoPs to six.
• Driven by desire to connect to services in London directly
• Installation took place in September 2013 during ECOC 2013
• Has been in service since December 2013. Expected doubling of

100G services in one year.

13/11/14 ¡ 3 ¡

Amsterdam – London CBF in more detail

London (HEX) Leiston Zandvoort Wherstead Whickford Amsterdam 2 Amsterdam 1

215km 40.6dB/20.3 dB IL RAMAN (RX+TX) = 5.4dB Total IL = 46dB Gain RAMAN: 25.7dB (Ppump=2.88W , Pseed=18mW) 78km 17dB/18.2dB

6500 WL3 Alien

79km 17.9dB/18.2dB 47km 10.3dB/13.5dB 53km 12.8dB/14.8dB

6500 WL3 6500 WL3 Alien

10km + 10dB 2.4dB/13.5dB

Photonic Exchange in Brussels

Hamburg London Amsterdam Brussels Paris Geneva PoP PoP PoP PoP PoP PoP PoP NORDUnet SURFnet

13/11/14 ¡ 4 ¡

Why and how?

Use cases:

• Restoration
• Redundancy
• For both customer and non-customer facing services

Requirements:

• Support >10Gbps
• 50GHz ITU grid / 88 channels (all DWDM channels in the C-band)
• Switch from and to any direction
• Local add/drop
• All waves should be presented in a single fiber

Concept

SURFnet OSI_1

To ASD To GEN To HB To LON Datacenter NORDUnet Datacenter SURFnet WSS WSS WSS WSS

• Ca. 10 Km NDSF + patching: 7dB (max)

Virtual BMD2 & CMD44s (Derived Adj.)

WSS-DIA WSS-DIA

BMD2 SURFnet OSI_2 NORDUnet OSI_1 NORDUnet OSI_2 MLA EDFA MLA MLA MLA MLA MLA Line fiber (0 km) + padding CMD44

Local Add/Drop Local Add/Drop

13/11/14 ¡ 5 ¡

Restoration using OPS switches

6500 6500 TOADM ROADM (50 GHz) TOADM

• Gr. 3
• Gr. 7
• Gr. 3

OTS add/drop OTS ROADM (50GHz) OTS 6500/CPL London ROADM (50GHz) OTS

BRU001A_CPL1P

OTS DIA DIA OTS OTS

To/from Hamburg To/from Geneva Amsterdam 2 Brussels - NORDUnet Brussels - SURFnet

• Ch. 68

OPS OCLD

• Gr. 3

OPS OCLD

• Ch. 28
• Ch. 28
• Ch. 28
• Ch. 68
• Ch. 68
• Ch. 28

WSS WSS OTS OPS

OCLD

ASD-LON link LON-BRU link BRU-ASD link

OTS add/drop add/drop

• Ch. 68

OCLD OPS

Amsterdam 1 6500 OTS add/ drop ROADM (50GHz) add/ drop

Time and Frequency Transfer

13/11/14 ¡ 6 ¡

Time and Frequency Transfer in SURFnet

• Objective: Allow clocks to synchronize their time with an accuracy

better than 500ps

• Two approaches that allow the far end to be compensated for the
• ffset to the middle of a loop:
• Maintain unidirectional traffic and calibrate the system to compensate for the

asymmetry due to different length of each fiber in the fiber pair

• Implement bi-directional transmission and amplification on a single fiber and only

calibrate the time difference occurring in the amplifier and in fiber due to dispersion.

• SURFnet aims to make adjustments to the network that allows

institutions to deploy White Rabbit systems beyond 10km.

Semiconductor Optical Amplifiers

LEDN001A_BIDIRAMP_01 (BDOA100B901) 1470nm In: 1470nm Out: 1490nm In: 1490nm Out: 1470nm 1490nm LEDN001A_BIDIRAMP_02 (BDOA100B902) In: 1490nm Out: 1470nm In: 1470nm Out: 1490nm 1470nm 1490nm

13/11/14 ¡ 7 ¡

Experimental setup over dark fiber

• In 2013 we have started with the following set-up:

VSL-Delft (Dutch Metrology Institute)

LEDN001A 5km

M S M S

0.5km 35.5km 14km 81km Campus TU-Delft / DT001B ASD002A (TC2)

ΔT

NIKHEF-Amsterdam (National Institute for Subatomic Physics)

ASD001A (SARA) G.655 G.655 G.652 G.652 G.652

135km, mixed G.655/G.652 fiber

19.6dB 9.5dB

• 24.0dBm

3.1dB 3.4dB 1.4dB 0.8dB 0.8dB 3.4dB 19dB 1dB 1dB

• 17.3dBm

1.7dBm

• 1.0dBm
• 1.0dBm
• 26.7dBm
• 7.7dBm
• 24.0dBm

Findings:

• BiDi amplifiers are placed asymmetrical in the link.
• Link engineering becomes significantly more complex in the absence of sim-tooling
• Gain of SOAs difficult to control by changing the electrical pump current
• Amplifiers produce about 19dB of gain on 1470nm and 1490nm
• Compatibility with fiber infrastructure is good.
• There are two types of optical budget: too much and too little!
• Initial tests showed a positive budget of 0.1dB on worst link and wavelength combination
• After cleaning a positive budget of 2.0 to 3.5 dB exists
• Total budget equals: 20dB + PTX – SRX = 52 – 57 dB depending on transceiver

combinations.

• Total losses equal: 17.3dB between Delft and Leiden and 26.7dB between Amsterdam

and Leiden sites.

• Both slaves lock
• Round trip delay of 668,981,165ps corresponds to 136km of fiber (272km round trip).
• Frequency offset of -0.3*10-13 Hz/Hz with a spread of 6*10-13 Hz/Hz over several hours.
• 12 digit accuracy of the frequency already present.
• On-going work to calibrate the small difference between 1490nm and 1470nm waves.
• Measurements are ongoing. We expect to achieve time accuracy around 500ps.

13/11/14 ¡ 8 ¡

Press release 100G Alien waves for customers

13/11/14 ¡ 9 ¡

Extend DWDM waves into the customer’s domain requires a New Demarcation Box: Optical Gate

Institution A Institution B SURFnet

Router Router

DWDM

TRX TRX TRX TRX

OG OG

MGMT MGMT MGMT

Institution A Institution B SURFnet

Router Router

DWDM

TRX TRX TRX TRX

OG OG

MGMT

Virtualization of the management function

Challenges

• Optical plane:
• DWDM signals are transported over an infrastructure that is intended for un-

engineered links on campus

• A proper CFP!
• Cross domain management:
• Fault Management
• Configuration Management
• Accounting
• Provisioning
• Security
• Controlled and accessible by both SURFnet and institutions

Optical Gating and Management Information Exchange Functionality may be required to:

• Isolate and protect the DWDM network
• Monitoring
• Measure frequency
• Test photonic path between two gateways
• Exchange of management information and instructions (may be virtualized in a

datacenter)

13/11/14 ¡ 10 ¡

Two Scenario’s

• CPE equipment (router or switch) of both customers is the same:
• Vendor has DWDM blade in portfolio that interoperates with light system
• Vendor has DWDM blade in portfolio that does not interoperate with light system
• Vendor has no DWDM blade in portfolio but does support a CFP slot.
• CPE equipment (router or switch) of both customers is different:
• Both CPEs have a CFP slot
• At least one of the CPEs does not have a CFP slot -> no solution

Can we find such a CFP that:

• Interoperates with SURFnet’s DWDM equipment
• Compatible with Topology of SURFnet7 and SURFnet8
• Is affordable! (<$10,000)

ACACIA 100G Coherent CFP

What if you:

• ptimize power of transmission impairment compensating ASICs;
• start using 28nm/20nm semiconductor process;
• start using Silicon Photonics Integrated Circuits and InP chips;
• pay extreme detail to power consumption and heat management?

Source: Acacia Inc.

• DWDM single lambda solution
• 50GHz C-band grid
• 25ps PMD tolerance & 40 ns/nm CD tolerance
• Compatible with >1000km network solutions
• < 15dB OSNR for up to 2000km
• SD-FEC, HD-FEC
• <24-26W
• <7.5W for DSP
• <7.5W for ADC/DAC/SERDES
• Ca. 5W for PIC
• < 10W for misc. funct.
• 24W for ZR range
• Path to smaller form factors like CFP2
• Supported by hosts

Source: Flickr

13/11/14 ¡ 11 ¡

Multi-domain alien-wave planning Spectrum Sharing on Cross Border Fibers

Three important cross-border fibers

• Amsterdam – London
• Amsterdam – Geneva
• Amsterdam – Hamburg

Photonic Exchange in Brussels! Photonic Exchanges in Hamburg/Geneva ?

13/11/14 ¡ 12 ¡

SURFnet’s CBFs

• In operation for several years using a variety of signals
• 10Gbps with electronic dispersion compensation
• 40Gbps waves
• 40Gbps alien waves
• 100Gbps waves
• 100Gbps alien waves
• GEN & HB CBFs are the CBFS we have the most knowledge on its

performance and capabilities

• Non-linearity (is King!)
• OSNR performance (is King Kong!)
• Chromatic dispersion and PMD (piece of cake!)
• Operational issues (escalation matrix and testing of alarms)
• Can we predict performance impact of an alien wave in order to

validate demands end-to-end traversing multiple domains.

• With reasonable accuracy
• Without complex, time consuming and expensive high-fidelity simulations
• Using planning tooling as provided by vendor of light system

Fiber Non-Linearity & Dispersion

Non-Linearity:

• No DWDM interfaces that use Back-Scatter-Propagation or other

non-linearity compensation.

• Stick to the linear Gaussian noise model
• SPM and XPM present themselves as additive Gaussian noise
• Neighboring channels with different modulations
• Fixed penalty that adds to OSNR budget
• Guard channel

Dispersion:

• Chromatic Dispersion: Linear additive
• PMD: Sum of max. DGDs per domain.

13/11/14 ¡ 13 ¡

OSNR validation using FoM

• Each DWDM interface has a certain OSNR budget.
• EDFA amplification has matured.
• Little difference in EDFA behavior of different vendors.
• Translate multiple-span link to single span, single EDFA link.
• Each traversed domain is represented by a single span.

16dB 17dB 17dB 20dB 23dB 23dB 16dB 13dB 13dB

L=10*log10(40+50+50+100)=23.8dB L=10*log10(200+200)=26dB L=10*log10(40+20+20)=19dB L=10*log10(240+400+80)=28.6dB

25.6dB 25.6dB

L=10*log10(360+360)=28.6dB

Cannot validate this Can validate this

TX TX TX TX RX RX RX RX

Anteneh Beshir, Roeland Nuijts, et.al. “Survivable Impairment-Aware Traffic Grooming”, Networks and Optical Communications (NOC), 2011 16th European Conference on Optical Communications Emmanuel Desurvire, “Erbium-Doped Fiber Amplifiers: Principles and Applications”

Example: ASD-GEN and ASD-HB

• Amplifier nodes
• 22 ASD-GEN
• 8 ASD-HB
• Total length
• 1500km ASD-GEN
• 600km ASD-HB
• 100Gbps PM-QPSK
• All demands pass!
• 1.5dB OSNR margin ASD-GEN
• 6.2dB OSNR margin ASD-HB
• ASD-GEN:
• FoM(ASD-GEN)=1910 with 1.5dB margin: FoM(ASD-GEN,max)=2700
• FoM(GEN-ASD)=2053 with 1.5dB margin: FoM(GEN-ASD,max)=2900
• ASD-HB:
• FoM(ASD-HB)=720 with 6.2dB margin: FoM(ASD-HB)= 3001
• FoM(HB-ASD)=775 with6.2dB margin: FoM(HB-ASD,max)=3230

FoM(avg)=2800 FoM(avg)=3115 Average FoM to calculate with is approx. 2960, Difference between GEN & HB CBF = 0.5dB (ROADM filters 2x)

13/11/14 ¡ 14 ¡

Infinera waves on a Ciena light system